2024-05-08T22:30:46.245 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T22:30:46.245 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130 -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/orig.config.yaml 2024-05-08T22:30:46.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:30:46.390 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698130' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExSQgko9D0zgkl/Hg3SwZs00ssU8GkR0PtLe9LClLRyJdKtJu3A80vDonI9GMYeuA3dcJQkgzIgjl0Nucmj6FI= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDGnZyrPqY7cWTeSkEpUq/sYOwFyw/x4AcFauAS7kVgT3ahtRYDTduCGclckqfJN65xI9vONq9t4z3oSXftcvQ= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAjNLmo6SABaZ6pUrM6MuEwU+fGODvtt7db2YcpWzVxtCG3vZeAlVxxY6soDBzFrp2EeqysrQKe/fBzujKqkQVk= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T22:30:46.390 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T22:30:46.391 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T22:30:46.391 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T22:30:46.392 INFO:teuthology.task.internal:Checking packages... 2024-05-08T22:30:46.450 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T22:30:46.450 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T22:30:46.450 INFO:teuthology.packaging:ref: None 2024-05-08T22:30:46.450 INFO:teuthology.packaging:tag: None 2024-05-08T22:30:46.450 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:30:46.451 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:30:46.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:30:46.590 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T22:30:46.591 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T22:30:46.601 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T22:30:46.601 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T22:30:46.620 INFO:teuthology.task.internal:Saving configuration 2024-05-08T22:30:46.642 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T22:30:46.648 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T22:30:46.674 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.759723', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExSQgko9D0zgkl/Hg3SwZs00ssU8GkR0PtLe9LClLRyJdKtJu3A80vDonI9GMYeuA3dcJQkgzIgjl0Nucmj6FI='} 2024-05-08T22:30:46.700 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.757989', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDGnZyrPqY7cWTeSkEpUq/sYOwFyw/x4AcFauAS7kVgT3ahtRYDTduCGclckqfJN65xI9vONq9t4z3oSXftcvQ='} 2024-05-08T22:30:46.727 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.756579', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAjNLmo6SABaZ6pUrM6MuEwU+fGODvtt7db2YcpWzVxtCG3vZeAlVxxY6soDBzFrp2EeqysrQKe/fBzujKqkQVk='} 2024-05-08T22:30:46.727 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T22:30:46.734 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T22:30:46.734 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T22:30:46.734 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T22:30:46.734 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T22:30:46.876 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77d2b15160>, signals=[15]) 2024-05-08T22:30:46.877 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T22:30:46.883 INFO:teuthology.task.internal:Opening connections... 2024-05-08T22:30:46.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T22:30:46.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:30:46.968 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-05-08T22:30:46.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:30:47.045 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-05-08T22:30:47.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:30:47.120 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T22:30:47.126 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-05-08T22:30:47.148 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-05-08T22:30:47.149 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:ID="rhel" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="fedora" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8.6" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:30:47.210 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:30:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:30:47.212 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-05-08T22:30:47.232 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-05-08T22:30:47.255 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-05-08T22:30:47.255 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-05-08T22:30:47.316 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:30:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:30:47.318 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:30:47.318 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:30:47.318 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:30:47.318 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:30:47.318 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-05-08T22:30:47.341 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-05-08T22:30:47.363 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-05-08T22:30:47.363 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-05-08T22:30:47.423 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:30:47.423 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:30:47.423 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.6" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:30:47.424 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:30:47.425 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:30:47.425 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-05-08T22:30:47.448 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T22:30:47.458 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T22:30:47.465 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T22:30:47.465 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:30:47.467 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:30:47.470 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:30:47.488 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T22:30:47.494 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T22:30:47.494 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:30:47.528 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:30:47.531 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:30:47.569 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T22:30:47.651 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T22:30:47.665 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T22:30:47.665 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-08T22:30:47.666 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-08T22:30:47.666 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:30:47.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:30:47.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-05-08T22:30:47.850 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T22:30:47.850 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T22:30:47.850 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T22:30:47.871 INFO:teuthology.orchestra.run.smithi046.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:30:47.872 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T22:30:47.872 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T22:30:47.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum upgrade -y linux-firmware 2024-05-08T22:30:49.376 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-05-08T22:30:49.376 INFO:teuthology.orchestra.run.smithi046.stdout:Unable to read consumer identity 2024-05-08T22:30:49.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:30:49.682 INFO:teuthology.orchestra.run.smithi046.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:30:49.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:30:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-08T22:30:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 2.9 MB/s | 16 MB 00:05 2024-05-08T22:31:00.838 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 650 kB/s | 24 kB 00:00 2024-05-08T22:31:01.717 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T22:31:01.719 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T22:31:01.719 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T22:31:01.905 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T22:31:01.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T22:31:02.129 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/ 2024-05-08T22:31:02.130 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/ --input-file=- 2024-05-08T22:31:02.132 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-05-08T22:31:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08 22:31:21 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/8/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [425099368/425099368] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-08T22:31:21.409 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-05-08 22:31:21-- 2024-05-08T22:31:21.409 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 19s 2024-05-08T22:31:21.409 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 405M in 19s (21.4 MB/s) 2024-05-08T22:31:21.412 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T22:31:21.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-08T22:31:23.488 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying... ######################################## 2024-05-08T22:31:23.582 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing... ######################################## 2024-05-08T22:31:23.747 INFO:teuthology.orchestra.run.smithi046.stdout:Updating / installing... 2024-05-08T22:32:11.239 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-08T22:32:13.065 INFO:teuthology.orchestra.run.smithi046.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-08T22:32:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-08T22:35:04.473 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-08T22:35:04.474 DEBUG:teuthology.orchestra.run.smithi046:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-08T22:35:04.540 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T22:35:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-08T22:35:04.542 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T22:35:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T22:35:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T22:35:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-08T22:35:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T22:35:04.546 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T22:35:04.547 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T22:35:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T22:35:04.549 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T22:35:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T22:35:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T22:35:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-08T22:35:04.553 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T22:35:04.554 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T22:35:04.555 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T22:35:04.556 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-08T22:35:04.557 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-08T22:35:04.558 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-08T22:35:04.559 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T22:35:04.560 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T22:35:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T22:35:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T22:35:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T22:35:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T22:35:04.565 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T22:35:04.566 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T22:35:04.567 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T22:35:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T22:35:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T22:35:04.570 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T22:35:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T22:35:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T22:35:04.573 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T22:35:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T22:35:04.575 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T22:35:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T22:35:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-08T22:35:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T22:35:04.579 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-08T22:35:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T22:35:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T22:35:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-08T22:35:04.583 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T22:35:04.584 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T22:35:04.585 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T22:35:04.586 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T22:35:04.587 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-08T22:35:04.588 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T22:35:04.589 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T22:35:04.590 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-08T22:35:04.591 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T22:35:04.592 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T22:35:04.593 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T22:35:04.594 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-08T22:35:04.595 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T22:35:04.596 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-08T22:35:04.597 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-08T22:35:04.598 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T22:35:04.599 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T22:35:04.600 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T22:35:04.601 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T22:35:04.602 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-08T22:35:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T22:35:04.604 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T22:35:04.605 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T22:35:04.606 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T22:35:04.607 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-08T22:35:04.608 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T22:35:04.609 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T22:35:04.610 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T22:35:04.611 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-08T22:35:04.612 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T22:35:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T22:35:04.614 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T22:35:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T22:35:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-08T22:35:04.617 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T22:35:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-08T22:35:04.619 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-08T22:35:04.620 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-08T22:35:04.621 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T22:35:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T22:35:04.623 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T22:35:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T22:35:04.625 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T22:35:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T22:35:04.627 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T22:35:04.628 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T22:35:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T22:35:04.630 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T22:35:04.631 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T22:35:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T22:35:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-08T22:35:04.634 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T22:35:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T22:35:04.636 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T22:35:04.637 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T22:35:04.638 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T22:35:04.639 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-08T22:35:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T22:35:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T22:35:04.642 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T22:35:04.643 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T22:35:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T22:35:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-08T22:35:04.646 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-08T22:35:04.647 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-08T22:35:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T22:35:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-08T22:35:04.650 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T22:35:04.651 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T22:35:04.652 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T22:35:04.653 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T22:35:04.654 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T22:35:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-08T22:35:04.656 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T22:35:04.657 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T22:35:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T22:35:04.659 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T22:35:04.660 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T22:35:04.661 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T22:35:04.662 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-08T22:35:04.663 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T22:35:04.664 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T22:35:04.665 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T22:35:04.666 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T22:35:04.667 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T22:35:04.668 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T22:35:04.669 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T22:35:04.670 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T22:35:04.671 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T22:35:04.672 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T22:35:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T22:35:04.674 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-08T22:35:04.675 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T22:35:04.676 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T22:35:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T22:35:04.678 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T22:35:04.679 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-08T22:35:04.680 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T22:35:04.681 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T22:35:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T22:35:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T22:35:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T22:35:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T22:35:04.682 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T22:35:04.686 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.686 DEBUG:teuthology.orchestra.run.smithi046:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:postinstall scriptlet (using /bin/sh): 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:for file in vmlinuz System.map config; do 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: fi 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-08T22:35:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.717 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-05-08T22:35:04.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.02 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 123.el8 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-05-08T22:35:04.801 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Group : System Environment/Base 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 9265450 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Relocations : (not relocatable) 2024-05-08T22:35:04.802 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : Red Hat, Inc. 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : Red Hat, Inc. 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:35:04.803 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T22:35:04.804 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T22:35:04.804 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:04.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T22:35:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-05-08T22:35:07.577 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-05-08T22:35:07.580 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-05-08T22:35:07.600 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.Wn2huqJW5R 2024-05-08T22:35:07.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Wn2huqJW5R 2024-05-08T22:35:07.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.Wn2huqJW5R 2024-05-08T22:35:07.849 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.Wn2huqJW5R is 6KB 2024-05-08T22:35:07.899 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.Wn2huqJW5R 2024-05-08T22:35:07.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T22:35:07.993 INFO:teuthology.orchestra.run.smithi046.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-05-08T22:35:07.993 INFO:teuthology.orchestra.run.smithi046.stdout:2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-g806ec36b2112.conf 2024-05-08T22:35:07.993 INFO:teuthology.orchestra.run.smithi046.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-05-08T22:35:07.993 INFO:teuthology.orchestra.run.smithi046.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-05-08T22:35:07.993 INFO:teuthology.orchestra.run.smithi046.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-05-08T22:35:07.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-g806ec36b2112 2024-05-08T22:35:08.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-05-08T22:35:38.152 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T22:35:38.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:35:38.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:35:56.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T22:36:05.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:36:05.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:36:08.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T22:36:20.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:36:20.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:37:20.642 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T22:37:35.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:37:35.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:37:35.968 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T22:37:36.429 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:37:36.429 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T22:37:36.429 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T22:37:36.470 INFO:teuthology.orchestra.run.smithi046.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T22:37:36.470 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T22:37:36.470 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T22:37:36.470 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-05-08T22:37:37.471 DEBUG:teuthology.parallel:result is None 2024-05-08T22:37:37.472 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T22:37:37.479 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T22:37:37.479 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:37:37.483 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:37:37.486 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:37:37.524 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T22:37:37.532 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T22:37:37.540 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T22:37:37.540 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:37:37.569 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:37:37.571 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:37:37.623 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T22:37:37.631 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T22:37:37.631 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:37:37.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:37.662 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:37:37.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:37.681 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:37:37.685 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:37:37.704 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:37:37.762 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.781 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.783 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.812 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.813 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.845 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:37:37.850 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T22:37:37.856 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T22:37:37.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:37:37.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:37:37.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:37:37.923 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T22:37:37.932 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T22:37:37.932 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:37:37.936 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:37:37.938 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:37:37.972 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.194 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.343 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:37:38.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:37:38.407 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.517 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.614 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:37:38.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:37:38.690 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:37:38.793 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:37:38.923 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:37:38.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:37:38.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-05-08T22:37:38.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-05-08T22:37:38.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-05-08T22:37:39.059 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:37:39.086 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:37:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:37:39.571 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T22:37:39.579 INFO:teuthology.task.internal:Starting timer... 2024-05-08T22:37:39.579 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T22:37:39.596 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T22:37:39.606 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T22:37:39.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-05-08T22:37:39.721 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-05-08T22:37:39.725 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-05-08T22:37:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-05-08T22:37:39.809 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-05-08T22:37:39.876 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-05-08T22:37:39.877 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T22:37:39.877 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-05-08T22:37:39.925 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-05-08T22:37:39.925 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-05-08T22:37:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-05-08T22:37:39.955 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-05-08T22:37:39.983 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-05-08T22:37:39.984 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive'} 2024-05-08T22:37:39.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:37:40.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:40.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:37:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:40.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:37:40.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:37:40.113 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T22:37:40.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:37:40.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:37:40.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:37:40.251 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T22:37:40.262 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T22:37:40.278 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T22:37:40.279 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T22:49:49.716 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2024-05-08T22:49:49.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:49:49.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:49:49.837 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T22:49:49.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T22:49:49.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-08T22:49:49.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:49:50.038 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-05-08T22:49:50.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-08T22:49:50.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-08T22:49:50.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:49:50.195 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-05-08T22:49:50.266 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-08T22:49:50.267 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T22:49:50.282 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T22:49:50.282 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:49:50.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:49:50.287 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:49:50.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:49:50.290 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:49:50.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:49:50.326 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:49:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:49:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:49:50.359 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:49:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:49:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:49:50.413 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-05-08T22:49:50.427 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-05-08T22:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-05-08T22:49:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:49:50.444 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-05-08T22:49:50.459 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:49:50.460 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:49:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:49:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-05-08T22:49:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-05-08T22:49:50.539 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-05-08T22:49:50.542 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:49:50.543 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-08T22:49:50.543 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.543 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.543 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.543 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.554 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2024-05-08T22:49:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:49:50.557 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:49:50.557 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-05-08T22:49:50.557 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.557 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.557 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.558 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.588 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:49:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:bash: ntpq: command not found 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.693 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:49:50.694 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T22:49:50.706 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T22:49:50.707 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T22:49:50.707 DEBUG:teuthology.orchestra.run.smithi046:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:49:50.707 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2024-05-08T22:49:50.708 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:49:50.708 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi086.front.sepia.ceph.com 2024-05-08T22:49:50.708 DEBUG:teuthology.orchestra.run.smithi086:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:49:51.086 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-05-08T22:49:51.143 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-08T22:49:51.228 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-05-08T22:49:52.679 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 kB/s | 4.5 kB 00:00 2024-05-08T22:49:52.972 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-05-08T22:49:53.281 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-05-08T22:49:53.286 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2024-05-08T22:49:53.763 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-08T22:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-05-08T22:49:56.229 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-05-08T22:49:56.229 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout:Resetting modules: 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-05-08T22:49:56.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:49:57.163 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout:Resetting modules: 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-08T22:49:57.165 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:49:57.352 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout:Resetting modules: 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T22:49:57.354 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:09.869 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-08T22:50:10.398 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-08T22:50:11.409 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-05-08T22:50:11.650 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T22:50:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-05-08T22:50:12.087 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 kB/s | 4.5 kB 00:00 2024-05-08T22:50:12.334 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-05-08T22:50:12.863 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-05-08T22:50:14.018 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-05-08T22:50:14.300 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 23 kB/s | 4.1 kB 00:00 2024-05-08T22:50:14.677 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-05-08T22:50:14.760 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-05-08T22:50:16.217 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-08T22:50:16.399 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-08T22:50:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-08T22:50:16.399 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-08T22:50:16.399 INFO:teuthology.orchestra.run.smithi053.stdout:Installing group/module packages: 2024-05-08T22:50:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:50:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:Installing module profiles: 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools/common 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools rhel8 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-05-08T22:50:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:50:16.402 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 26 M 2024-05-08T22:50:16.402 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 87 M 2024-05-08T22:50:16.402 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-08T22:50:16.571 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 147 kB/s | 25 kB 00:00 2024-05-08T22:50:16.587 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 213 kB/s | 39 kB 00:00 2024-05-08T22:50:16.613 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 446 kB/s | 94 kB 00:00 2024-05-08T22:50:16.688 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 350 kB/s | 41 kB 00:00 2024-05-08T22:50:16.797 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.8 MB/s | 1.0 MB 00:00 2024-05-08T22:50:16.831 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.1 MB/s | 739 kB 00:00 2024-05-08T22:50:17.006 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 23 MB/s | 8.8 MB 00:00 2024-05-08T22:50:17.039 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 990 kB/s | 239 kB 00:00 2024-05-08T22:50:17.056 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 3.0 MB/s | 154 kB 00:00 2024-05-08T22:50:17.165 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-08T22:50:17.275 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 12 MB/s | 2.5 MB 00:00 2024-05-08T22:50:17.307 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 349 kB/s | 49 kB 00:00 2024-05-08T22:50:17.440 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T22:50:17.441 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:50:17.441 INFO:teuthology.orchestra.run.smithi053.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-08T22:50:17.524 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-05-08T22:50:17.690 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-05-08T22:50:17.690 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout:Installing group/module packages: 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:50:17.691 INFO:teuthology.orchestra.run.smithi086.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:Installing module profiles: 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools/common 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools rhel8 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout:Install 13 Packages 2024-05-08T22:50:17.692 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:50:17.693 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 26 M 2024-05-08T22:50:17.693 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 87 M 2024-05-08T22:50:17.693 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-05-08T22:50:17.750 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-08T22:50:17.774 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-08T22:50:17.774 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-08T22:50:17.820 INFO:teuthology.orchestra.run.smithi086.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 195 kB/s | 25 kB 00:00 2024-05-08T22:50:17.894 INFO:teuthology.orchestra.run.smithi086.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 468 kB/s | 94 kB 00:00 2024-05-08T22:50:17.929 INFO:teuthology.orchestra.run.smithi086.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 377 kB/s | 41 kB 00:00 2024-05-08T22:50:17.954 INFO:teuthology.orchestra.run.smithi086.stdout:(4/13): criu-libs-3.18-4.module+el8.9.0+20326+3 151 kB/s | 39 kB 00:00 2024-05-08T22:50:18.031 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-08T22:50:18.041 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-08T22:50:18.080 INFO:teuthology.orchestra.run.smithi086.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.5 MB/s | 1.0 MB 00:00 2024-05-08T22:50:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.3 MB/s | 739 kB 00:00 2024-05-08T22:50:18.172 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T22:50:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 26 MB/s | 8.8 MB 00:00 2024-05-08T22:50:18.289 INFO:teuthology.orchestra.run.smithi086.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.1 MB/s | 239 kB 00:00 2024-05-08T22:50:18.357 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:Installing group/module packages: 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:50:18.358 INFO:teuthology.orchestra.run.smithi046.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:Installing module profiles: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools/common 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling module streams: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools rhel8 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T22:50:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-05-08T22:50:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-05-08T22:50:18.360 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 26 M 2024-05-08T22:50:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 87 M 2024-05-08T22:50:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T22:50:18.504 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 174 kB/s | 25 kB 00:00 2024-05-08T22:50:18.529 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 235 kB/s | 39 kB 00:00 2024-05-08T22:50:18.531 INFO:teuthology.orchestra.run.smithi086.stdout:(9/13): skopeo-1.13.3-3.module+el8.9.0+21697+6a 34 MB/s | 8.1 MB 00:00 2024-05-08T22:50:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 486 kB/s | 94 kB 00:00 2024-05-08T22:50:18.556 INFO:teuthology.orchestra.run.smithi086.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 526 kB/s | 154 kB 00:00 2024-05-08T22:50:18.622 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 349 kB/s | 41 kB 00:00 2024-05-08T22:50:18.689 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.4 MB/s | 1.0 MB 00:00 2024-05-08T22:50:18.715 INFO:teuthology.orchestra.run.smithi086.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 14 MB/s | 2.5 MB 00:00 2024-05-08T22:50:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 235 kB/s | 49 kB 00:00 2024-05-08T22:50:18.789 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.3 MB/s | 739 kB 00:00 2024-05-08T22:50:18.831 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.6 MB/s | 239 kB 00:00 2024-05-08T22:50:18.877 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-08T22:50:19.057 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 18 MB/s | 8.8 MB 00:00 2024-05-08T22:50:19.082 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 613 kB/s | 154 kB 00:00 2024-05-08T22:50:19.141 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-08T22:50:19.257 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+21697+6a 424 kB/s | 49 kB 00:00 2024-05-08T22:50:19.333 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 10 MB/s | 2.5 MB 00:00 2024-05-08T22:50:19.449 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:50:19.458 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T22:50:19.458 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:50:19.459 INFO:teuthology.orchestra.run.smithi046.stdout:Total 23 MB/s | 26 MB 00:01 2024-05-08T22:50:19.668 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:50:19.822 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:50:19.936 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T22:50:19.963 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T22:50:19.963 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T22:50:20.011 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:50:20.328 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:50:20.398 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T22:50:20.408 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T22:50:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:50:20.748 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:50:21.279 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T22:50:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:21.865 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:50:21.941 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:50:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:50:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:50:22.372 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:50:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:50:22.813 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:50:23.026 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:50:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:50:23.591 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:50:23.992 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:24.025 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:24.304 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:50:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:50:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:50:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:50:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:50:25.968 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:50:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:50:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:50:26.855 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:50:26.856 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:50:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:50:36.008 INFO:teuthology.orchestra.run.smithi046.stdout:Installed products updated. 2024-05-08T22:50:36.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:36.682 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:50:36.683 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:50:36.684 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T22:50:36.945 DEBUG:teuthology.parallel:result is None 2024-05-08T22:50:37.825 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:50:37.825 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:38.474 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:50:38.475 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-08T22:50:38.679 DEBUG:teuthology.parallel:result is None 2024-05-08T22:50:51.704 INFO:teuthology.orchestra.run.smithi086.stdout:(13/13): netavark-1.7.0-2.module+el8.9.0+21697+ 113 kB/s | 3.7 MB 00:33 2024-05-08T22:50:51.704 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:50:51.705 INFO:teuthology.orchestra.run.smithi086.stdout:Total 770 kB/s | 26 MB 00:34 2024-05-08T22:50:51.991 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-05-08T22:50:52.013 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-05-08T22:50:52.013 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-05-08T22:50:52.241 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-05-08T22:50:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-05-08T22:50:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-05-08T22:50:53.539 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:50:53.766 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:50:53.987 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:50:54.148 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:50:54.367 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:50:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:50:54.878 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:50:55.754 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:56.029 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:56.396 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:50:57.319 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:50:57.550 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:50:57.691 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:58.126 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:50:58.126 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:50:58.127 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:50:58.128 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:50:58.128 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:50:58.128 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:51:12.929 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:51:12.929 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-05-08T22:51:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:51:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:51:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:51:13.502 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-05-08T22:51:13.676 DEBUG:teuthology.parallel:result is None 2024-05-08T22:51:13.676 INFO:teuthology.run_tasks:Running task install... 2024-05-08T22:51:13.685 DEBUG:teuthology.task.install:project ceph 2024-05-08T22:51:13.685 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T22:51:13.686 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T22:51:13.686 INFO:teuthology.task.install:Using flavor: default 2024-05-08T22:51:13.692 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T22:51:13.692 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T22:51:13.692 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:51:13.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:51:13.695 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:51:13.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:51:13.696 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:51:13.697 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:51:13.848 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:51:13.848 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:51:13.917 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:51:13.917 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:51:13.927 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:51:13.927 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:51:14.018 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:51:14.018 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:51:14.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:51:14.023 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:51:14.023 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:51:14.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:51:14.031 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:51:14.032 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:51:14.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:51:14.046 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:51:14.046 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:51:14.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:51:14.070 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:51:14.099 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:51:14.100 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:51:14.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:51:14.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:51:14.192 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:51:14.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:51:14.236 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:51:14.258 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-05-08T22:51:14.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-05-08T22:51:14.268 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-05-08T22:51:14.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-05-08T22:51:14.302 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:51:14.367 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-05-08T22:51:14.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-05-08T22:51:14.633 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:14.705 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:14.850 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:16.078 INFO:teuthology.orchestra.run.smithi086.stdout:63 files removed 2024-05-08T22:51:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:63 files removed 2024-05-08T22:51:16.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:51:16.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:51:16.245 INFO:teuthology.orchestra.run.smithi046.stdout:63 files removed 2024-05-08T22:51:16.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:51:16.467 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:16.562 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:16.783 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-05-08T22:51:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 259 kB/s | 80 kB 00:00 2024-05-08T22:51:18.343 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 271 kB/s | 80 kB 00:00 2024-05-08T22:51:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-08T22:51:18.646 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 190 kB/s | 80 kB 00:00 2024-05-08T22:51:18.652 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 42 kB/s | 11 kB 00:00 2024-05-08T22:51:18.978 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2024-05-08T22:51:19.022 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-05-08T22:51:19.042 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-05-08T22:51:19.195 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-05-08T22:51:19.203 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-05-08T22:51:19.406 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2024-05-08T22:51:19.565 INFO:teuthology.orchestra.run.smithi046.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-05-08T22:51:20.064 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2024-05-08T22:51:20.768 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-05-08T22:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2024-05-08T22:51:24.137 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 407 kB/s | 24 kB 00:00 2024-05-08T22:51:24.980 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 316 kB/s | 24 kB 00:00 2024-05-08T22:51:25.158 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 516 kB/s | 24 kB 00:00 2024-05-08T22:51:25.547 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 61 MB 00:01 2024-05-08T22:51:26.549 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 61 MB 00:01 2024-05-08T22:51:26.859 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 67 MB 00:01 2024-05-08T22:51:37.498 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2024-05-08T22:51:37.983 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 67 MB 00:01 2024-05-08T22:51:38.143 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-08T22:51:40.798 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 67 MB 00:01 2024-05-08T22:51:41.695 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 MB/s | 61 MB 00:01 2024-05-08T22:51:48.650 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T22:51:55.823 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:55.824 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:55.843 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:55.844 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:55.845 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:51:55.847 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:51:55.849 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:51:55.854 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:51:55.855 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:51:56.085 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:51:56.099 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:51:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:51:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:51:56.102 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:51:56.103 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:51:56.104 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:51:56.105 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:51:56.106 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:51:56.107 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:51:56.108 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:51:56.109 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:51:56.110 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql 10 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-05-08T22:51:56.111 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-05-08T22:51:56.112 INFO:teuthology.orchestra.run.smithi086.stdout:Install 130 Packages 2024-05-08T22:51:56.112 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 16 Packages 2024-05-08T22:51:56.112 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:51:56.112 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 215 M 2024-05-08T22:51:56.112 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-05-08T22:51:56.268 INFO:teuthology.orchestra.run.smithi086.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2024-05-08T22:51:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 4.4 MB/s | 860 kB 00:00 2024-05-08T22:51:56.518 INFO:teuthology.orchestra.run.smithi086.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.8 MB/s | 166 kB 00:00 2024-05-08T22:51:56.752 INFO:teuthology.orchestra.run.smithi086.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 9.7 MB/s | 2.3 MB 00:00 2024-05-08T22:51:57.119 INFO:teuthology.orchestra.run.smithi086.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 4.3 MB/s | 1.6 MB 00:00 2024-05-08T22:51:57.194 INFO:teuthology.orchestra.run.smithi086.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 5.3 MB/s | 5.5 MB 00:01 2024-05-08T22:51:57.736 INFO:teuthology.orchestra.run.smithi086.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 7.8 MB/s | 4.8 MB 00:00 2024-05-08T22:51:58.042 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:58.044 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:58.065 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:58.066 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:58.068 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:51:58.069 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:51:58.072 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:51:58.077 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:51:58.079 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:51:58.344 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:51:58.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:51:58.362 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:51:58.363 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:51:58.364 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:51:58.365 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:51:58.366 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:51:58.367 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:51:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:51:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:51:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:51:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql 10 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-08T22:51:58.372 INFO:teuthology.orchestra.run.smithi053.stdout:Install 130 Packages 2024-05-08T22:51:58.373 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 16 Packages 2024-05-08T22:51:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:51:58.373 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 215 M 2024-05-08T22:51:58.373 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-08T22:51:58.790 INFO:teuthology.orchestra.run.smithi053.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2024-05-08T22:51:58.878 INFO:teuthology.orchestra.run.smithi086.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 7.9 MB/s | 22 MB 00:02 2024-05-08T22:51:58.970 INFO:teuthology.orchestra.run.smithi086.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 266 kB/s | 24 kB 00:00 2024-05-08T22:51:59.075 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:59.076 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:51:59.097 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:59.099 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:51:59.101 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:51:59.102 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:51:59.105 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:51:59.107 INFO:teuthology.orchestra.run.smithi053.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 2.7 MB/s | 860 kB 00:00 2024-05-08T22:51:59.110 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:51:59.112 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:51:59.191 INFO:teuthology.orchestra.run.smithi053.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.9 MB/s | 166 kB 00:00 2024-05-08T22:51:59.348 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T22:51:59.364 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-05-08T22:51:59.364 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T22:51:59.364 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:51:59.365 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:51:59.366 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:51:59.367 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:51:59.368 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:51:59.369 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:51:59.370 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:51:59.371 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:51:59.372 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:51:59.373 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:51:59.374 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:51:59.375 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling module streams: 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql 10 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-05-08T22:51:59.376 INFO:teuthology.orchestra.run.smithi046.stdout:Install 130 Packages 2024-05-08T22:51:59.377 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 16 Packages 2024-05-08T22:51:59.377 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:51:59.377 INFO:teuthology.orchestra.run.smithi086.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 8.5 MB/s | 18 MB 00:02 2024-05-08T22:51:59.377 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 215 M 2024-05-08T22:51:59.378 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T22:51:59.504 INFO:teuthology.orchestra.run.smithi086.stdout:(11/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 7.4 MB/s | 13 MB 00:01 2024-05-08T22:51:59.529 INFO:teuthology.orchestra.run.smithi086.stdout:(12/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.6 MB/s | 261 kB 00:00 2024-05-08T22:51:59.583 INFO:teuthology.orchestra.run.smithi053.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 5.8 MB/s | 2.3 MB 00:00 2024-05-08T22:51:59.587 INFO:teuthology.orchestra.run.smithi086.stdout:(13/146): libcephfs-devel-18.2.2-1322.gd21e4f96 371 kB/s | 31 kB 00:00 2024-05-08T22:51:59.688 INFO:teuthology.orchestra.run.smithi086.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 142 kB 00:00 2024-05-08T22:51:59.771 INFO:teuthology.orchestra.run.smithi086.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.6 MB/s | 139 kB 00:00 2024-05-08T22:51:59.796 INFO:teuthology.orchestra.run.smithi086.stdout:(16/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.9 MB/s | 782 kB 00:00 2024-05-08T22:51:59.854 INFO:teuthology.orchestra.run.smithi046.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-05-08T22:51:59.947 INFO:teuthology.orchestra.run.smithi086.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.6 MB/s | 474 kB 00:00 2024-05-08T22:52:00.017 INFO:teuthology.orchestra.run.smithi053.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 3.6 MB/s | 1.6 MB 00:00 2024-05-08T22:52:00.030 INFO:teuthology.orchestra.run.smithi086.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 535 kB/s | 45 kB 00:00 2024-05-08T22:52:00.114 INFO:teuthology.orchestra.run.smithi086.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.5 MB/s | 129 kB 00:00 2024-05-08T22:52:00.239 INFO:teuthology.orchestra.run.smithi086.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.5 MB/s | 197 kB 00:00 2024-05-08T22:52:00.292 INFO:teuthology.orchestra.run.smithi053.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 5.5 MB 00:01 2024-05-08T22:52:00.323 INFO:teuthology.orchestra.run.smithi086.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 4.3 MB/s | 370 kB 00:00 2024-05-08T22:52:00.413 INFO:teuthology.orchestra.run.smithi046.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.5 MB/s | 860 kB 00:00 2024-05-08T22:52:00.505 INFO:teuthology.orchestra.run.smithi046.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.8 MB/s | 166 kB 00:00 2024-05-08T22:52:00.815 INFO:teuthology.orchestra.run.smithi086.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 742 kB/s | 365 kB 00:00 2024-05-08T22:52:00.915 INFO:teuthology.orchestra.run.smithi086.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.0 MB/s | 107 kB 00:00 2024-05-08T22:52:01.015 INFO:teuthology.orchestra.run.smithi086.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 951 kB/s | 95 kB 00:00 2024-05-08T22:52:01.313 INFO:teuthology.orchestra.run.smithi046.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 2.8 MB/s | 2.3 MB 00:00 2024-05-08T22:52:01.906 INFO:teuthology.orchestra.run.smithi046.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 2.6 MB/s | 1.6 MB 00:00 2024-05-08T22:52:02.239 INFO:teuthology.orchestra.run.smithi086.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.8 MB/s | 3.5 MB 00:01 2024-05-08T22:52:02.327 INFO:teuthology.orchestra.run.smithi053.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 4.8 MB 00:02 2024-05-08T22:52:02.358 INFO:teuthology.orchestra.run.smithi086.stdout:(26/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.6 MB/s | 6.5 MB 00:02 2024-05-08T22:52:02.383 INFO:teuthology.orchestra.run.smithi086.stdout:(27/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.2 MB/s | 181 kB 00:00 2024-05-08T22:52:02.458 INFO:teuthology.orchestra.run.smithi086.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 242 kB/s | 24 kB 00:00 2024-05-08T22:52:02.483 INFO:teuthology.orchestra.run.smithi086.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.4 MB/s | 138 kB 00:00 2024-05-08T22:52:03.172 INFO:teuthology.orchestra.run.smithi046.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 1.5 MB/s | 5.5 MB 00:03 2024-05-08T22:52:03.658 INFO:teuthology.orchestra.run.smithi046.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.8 MB/s | 4.8 MB 00:01 2024-05-08T22:52:04.534 INFO:teuthology.orchestra.run.smithi086.stdout:(30/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 8.5 MB/s | 47 MB 00:05 2024-05-08T22:52:04.584 INFO:teuthology.orchestra.run.smithi086.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.7 MB/s | 3.5 MB 00:02 2024-05-08T22:52:04.626 INFO:teuthology.orchestra.run.smithi086.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.6 MB/s | 245 kB 00:00 2024-05-08T22:52:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:(33/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 536 kB/s | 49 kB 00:00 2024-05-08T22:52:04.718 INFO:teuthology.orchestra.run.smithi086.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 176 kB/s | 16 kB 00:00 2024-05-08T22:52:04.819 INFO:teuthology.orchestra.run.smithi086.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.7 MB/s | 170 kB 00:00 2024-05-08T22:52:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:(36/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.5 MB/s | 263 kB 00:00 2024-05-08T22:52:04.911 INFO:teuthology.orchestra.run.smithi086.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 197 kB/s | 18 kB 00:00 2024-05-08T22:52:05.157 INFO:teuthology.orchestra.run.smithi046.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 3.9 MB/s | 22 MB 00:05 2024-05-08T22:52:05.211 INFO:teuthology.orchestra.run.smithi086.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 199 kB/s | 59 kB 00:00 2024-05-08T22:52:05.214 INFO:teuthology.orchestra.run.smithi046.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 419 kB/s | 24 kB 00:00 2024-05-08T22:52:05.245 INFO:teuthology.orchestra.run.smithi086.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 764 kB/s | 306 kB 00:00 2024-05-08T22:52:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 450 kB/s | 75 kB 00:00 2024-05-08T22:52:05.403 INFO:teuthology.orchestra.run.smithi086.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2024-05-08T22:52:05.504 INFO:teuthology.orchestra.run.smithi086.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-05-08T22:52:05.529 INFO:teuthology.orchestra.run.smithi086.stdout:(43/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 289 kB/s | 44 kB 00:00 2024-05-08T22:52:05.596 INFO:teuthology.orchestra.run.smithi086.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2024-05-08T22:52:05.621 INFO:teuthology.orchestra.run.smithi086.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2024-05-08T22:52:05.722 INFO:teuthology.orchestra.run.smithi086.stdout:(46/146): ceph-mgr-diskprediction-local-18.2.2- 2.3 MB/s | 7.4 MB 00:03 2024-05-08T22:52:05.725 INFO:teuthology.orchestra.run.smithi053.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 3.0 MB/s | 22 MB 00:07 2024-05-08T22:52:05.747 INFO:teuthology.orchestra.run.smithi086.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2024-05-08T22:52:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-05-08T22:52:05.839 INFO:teuthology.orchestra.run.smithi086.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2024-05-08T22:52:05.873 INFO:teuthology.orchestra.run.smithi086.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2024-05-08T22:52:05.925 INFO:teuthology.orchestra.run.smithi053.stdout:(9/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x86 3.3 MB/s | 18 MB 00:05 2024-05-08T22:52:05.950 INFO:teuthology.orchestra.run.smithi053.stdout:(10/146): ceph-selinux-18.2.2-1322.gd21e4f96.el 108 kB/s | 24 kB 00:00 2024-05-08T22:52:05.973 INFO:teuthology.orchestra.run.smithi086.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 488 kB/s | 49 kB 00:00 2024-05-08T22:52:06.007 INFO:teuthology.orchestra.run.smithi086.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 7.0 MB/s | 1.2 MB 00:00 2024-05-08T22:52:06.032 INFO:teuthology.orchestra.run.smithi086.stdout:(53/146): python3-google-auth-1.1.1-10.el8.noar 262 kB/s | 81 kB 00:00 2024-05-08T22:52:06.074 INFO:teuthology.orchestra.run.smithi086.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 584 kB/s | 59 kB 00:00 2024-05-08T22:52:06.099 INFO:teuthology.orchestra.run.smithi086.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 654 kB/s | 60 kB 00:00 2024-05-08T22:52:06.101 INFO:teuthology.orchestra.run.smithi053.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.7 MB/s | 261 kB 00:00 2024-05-08T22:52:06.150 INFO:teuthology.orchestra.run.smithi086.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2024-05-08T22:52:06.183 INFO:teuthology.orchestra.run.smithi086.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2024-05-08T22:52:06.184 INFO:teuthology.orchestra.run.smithi053.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 373 kB/s | 31 kB 00:00 2024-05-08T22:52:06.247 INFO:teuthology.orchestra.run.smithi086.stdout:(58/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2024-05-08T22:52:06.284 INFO:teuthology.orchestra.run.smithi086.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 641 kB/s | 64 kB 00:00 2024-05-08T22:52:06.309 INFO:teuthology.orchestra.run.smithi086.stdout:(60/146): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2024-05-08T22:52:06.326 INFO:teuthology.orchestra.run.smithi086.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 238 kB/s | 19 kB 00:00 2024-05-08T22:52:06.351 INFO:teuthology.orchestra.run.smithi086.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 356 kB/s | 24 kB 00:00 2024-05-08T22:52:06.402 INFO:teuthology.orchestra.run.smithi086.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 316 kB/s | 29 kB 00:00 2024-05-08T22:52:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 297 kB/s | 27 kB 00:00 2024-05-08T22:52:06.444 INFO:teuthology.orchestra.run.smithi086.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-05-08T22:52:06.468 INFO:teuthology.orchestra.run.smithi053.stdout:(13/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 3.2 MB/s | 13 MB 00:04 2024-05-08T22:52:06.486 INFO:teuthology.orchestra.run.smithi086.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 918 kB/s | 61 kB 00:00 2024-05-08T22:52:06.501 INFO:teuthology.orchestra.run.smithi053.stdout:(14/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.4 MB/s | 782 kB 00:00 2024-05-08T22:52:06.527 INFO:teuthology.orchestra.run.smithi053.stdout:(15/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.4 MB/s | 142 kB 00:00 2024-05-08T22:52:06.534 INFO:teuthology.orchestra.run.smithi086.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.9 MB/s | 251 kB 00:00 2024-05-08T22:52:06.570 INFO:teuthology.orchestra.run.smithi086.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 680 kB/s | 85 kB 00:00 2024-05-08T22:52:06.585 INFO:teuthology.orchestra.run.smithi053.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 1.6 MB/s | 139 kB 00:00 2024-05-08T22:52:06.595 INFO:teuthology.orchestra.run.smithi086.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-05-08T22:52:06.620 INFO:teuthology.orchestra.run.smithi086.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T22:52:06.685 INFO:teuthology.orchestra.run.smithi053.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.9 MB/s | 474 kB 00:00 2024-05-08T22:52:06.727 INFO:teuthology.orchestra.run.smithi053.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 1.1 MB/s | 45 kB 00:00 2024-05-08T22:52:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2024-05-08T22:52:06.779 INFO:teuthology.orchestra.run.smithi086.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-08T22:52:06.786 INFO:teuthology.orchestra.run.smithi053.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 2.2 MB/s | 129 kB 00:00 2024-05-08T22:52:06.798 INFO:teuthology.orchestra.run.smithi046.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 4.2 MB/s | 13 MB 00:03 2024-05-08T22:52:06.847 INFO:teuthology.orchestra.run.smithi086.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 199 kB/s | 44 kB 00:00 2024-05-08T22:52:06.853 INFO:teuthology.orchestra.run.smithi053.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.9 MB/s | 197 kB 00:00 2024-05-08T22:52:06.907 INFO:teuthology.orchestra.run.smithi046.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.4 MB/s | 261 kB 00:00 2024-05-08T22:52:06.930 INFO:teuthology.orchestra.run.smithi086.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 184 kB/s | 27 kB 00:00 2024-05-08T22:52:06.955 INFO:teuthology.orchestra.run.smithi086.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 506 kB/s | 110 kB 00:00 2024-05-08T22:52:06.974 INFO:teuthology.orchestra.run.smithi046.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 467 kB/s | 31 kB 00:00 2024-05-08T22:52:06.978 INFO:teuthology.orchestra.run.smithi053.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.9 MB/s | 370 kB 00:00 2024-05-08T22:52:07.014 INFO:teuthology.orchestra.run.smithi086.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 759 kB/s | 126 kB 00:00 2024-05-08T22:52:07.047 INFO:teuthology.orchestra.run.smithi086.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2024-05-08T22:52:07.081 INFO:teuthology.orchestra.run.smithi086.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2024-05-08T22:52:07.095 INFO:teuthology.orchestra.run.smithi053.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.1 MB/s | 365 kB 00:00 2024-05-08T22:52:07.170 INFO:teuthology.orchestra.run.smithi053.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.4 MB/s | 107 kB 00:00 2024-05-08T22:52:07.174 INFO:teuthology.orchestra.run.smithi046.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 3.8 MB/s | 782 kB 00:00 2024-05-08T22:52:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-08T22:52:07.241 INFO:teuthology.orchestra.run.smithi046.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.1 MB/s | 142 kB 00:00 2024-05-08T22:52:07.306 INFO:teuthology.orchestra.run.smithi053.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 706 kB/s | 95 kB 00:00 2024-05-08T22:52:07.308 INFO:teuthology.orchestra.run.smithi046.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 2.0 MB/s | 139 kB 00:00 2024-05-08T22:52:07.490 INFO:teuthology.orchestra.run.smithi086.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 95 kB/s | 39 kB 00:00 2024-05-08T22:52:07.491 INFO:teuthology.orchestra.run.smithi046.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.5 MB/s | 474 kB 00:00 2024-05-08T22:52:07.640 INFO:teuthology.orchestra.run.smithi086.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 1.2 MB/s | 709 kB 00:00 2024-05-08T22:52:07.665 INFO:teuthology.orchestra.run.smithi086.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 365 kB/s | 64 kB 00:00 2024-05-08T22:52:07.691 INFO:teuthology.orchestra.run.smithi086.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 520 kB/s | 259 kB 00:00 2024-05-08T22:52:07.732 INFO:teuthology.orchestra.run.smithi086.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2024-05-08T22:52:07.833 INFO:teuthology.orchestra.run.smithi086.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.7 MB/s | 538 kB 00:00 2024-05-08T22:52:07.983 INFO:teuthology.orchestra.run.smithi086.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2024-05-08T22:52:08.042 INFO:teuthology.orchestra.run.smithi086.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-08T22:52:08.058 INFO:teuthology.orchestra.run.smithi086.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2024-05-08T22:52:08.142 INFO:teuthology.orchestra.run.smithi086.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.4 MB/s | 225 kB 00:00 2024-05-08T22:52:08.146 INFO:teuthology.orchestra.run.smithi053.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 4.1 MB/s | 3.5 MB 00:00 2024-05-08T22:52:08.176 INFO:teuthology.orchestra.run.smithi086.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-05-08T22:52:08.238 INFO:teuthology.orchestra.run.smithi053.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.9 MB/s | 181 kB 00:00 2024-05-08T22:52:08.251 INFO:teuthology.orchestra.run.smithi086.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2024-05-08T22:52:08.296 INFO:teuthology.orchestra.run.smithi053.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 417 kB/s | 24 kB 00:00 2024-05-08T22:52:08.318 INFO:teuthology.orchestra.run.smithi086.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 20 MB/s | 5.2 MB 00:00 2024-05-08T22:52:08.352 INFO:teuthology.orchestra.run.smithi086.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 6.9 MB/s | 714 kB 00:00 2024-05-08T22:52:08.380 INFO:teuthology.orchestra.run.smithi053.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.6 MB/s | 138 kB 00:00 2024-05-08T22:52:08.400 INFO:teuthology.orchestra.run.smithi086.stdout:(94/146): libpq-13.11-1.el8.x86_64.rpm 2.4 MB/s | 198 kB 00:00 2024-05-08T22:52:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-05-08T22:52:08.528 INFO:teuthology.orchestra.run.smithi086.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-05-08T22:52:08.564 INFO:teuthology.orchestra.run.smithi053.stdout:(29/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 3.3 MB/s | 6.5 MB 00:01 2024-05-08T22:52:08.570 INFO:teuthology.orchestra.run.smithi086.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 320 kB/s | 54 kB 00:00 2024-05-08T22:52:08.612 INFO:teuthology.orchestra.run.smithi086.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-08T22:52:08.737 INFO:teuthology.orchestra.run.smithi086.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 9.0 MB/s | 1.5 MB 00:00 2024-05-08T22:52:08.770 INFO:teuthology.orchestra.run.smithi086.stdout:(100/146): postgresql-contrib-10.23-4.module+el 5.0 MB/s | 812 kB 00:00 2024-05-08T22:52:08.933 INFO:teuthology.orchestra.run.smithi046.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 4.5 MB/s | 6.5 MB 00:01 2024-05-08T22:52:08.937 INFO:teuthology.orchestra.run.smithi086.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 32 MB/s | 14 MB 00:00 2024-05-08T22:52:09.000 INFO:teuthology.orchestra.run.smithi086.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 78 kB/s | 18 kB 00:00 2024-05-08T22:52:09.046 INFO:teuthology.orchestra.run.smithi086.stdout:(103/146): python3-jwt-1.6.1-2.el8.noarch.rpm 393 kB/s | 43 kB 00:00 2024-05-08T22:52:09.130 INFO:teuthology.orchestra.run.smithi086.stdout:(104/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-08T22:52:09.131 INFO:teuthology.orchestra.run.smithi053.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 4.7 MB/s | 3.5 MB 00:00 2024-05-08T22:52:09.155 INFO:teuthology.orchestra.run.smithi086.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.0 MB/s | 155 kB 00:00 2024-05-08T22:52:09.159 INFO:teuthology.orchestra.run.smithi046.stdout:(18/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 18 MB 00:05 2024-05-08T22:52:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 816 kB/s | 109 kB 00:00 2024-05-08T22:52:09.184 INFO:teuthology.orchestra.run.smithi046.stdout:(19/146): python3-ceph-argparse-18.2.2-1322.gd2 178 kB/s | 45 kB 00:00 2024-05-08T22:52:09.215 INFO:teuthology.orchestra.run.smithi053.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.9 MB/s | 245 kB 00:00 2024-05-08T22:52:09.217 INFO:teuthology.orchestra.run.smithi046.stdout:(20/146): python3-ceph-common-18.2.2-1322.gd21e 2.2 MB/s | 129 kB 00:00 2024-05-08T22:52:09.230 INFO:teuthology.orchestra.run.smithi086.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 425 kB/s | 43 kB 00:00 2024-05-08T22:52:09.251 INFO:teuthology.orchestra.run.smithi046.stdout:(21/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.9 MB/s | 197 kB 00:00 2024-05-08T22:52:09.256 INFO:teuthology.orchestra.run.smithi053.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 1.2 MB/s | 49 kB 00:00 2024-05-08T22:52:09.314 INFO:teuthology.orchestra.run.smithi086.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 117 kB/s | 19 kB 00:00 2024-05-08T22:52:09.335 INFO:teuthology.orchestra.run.smithi046.stdout:(22/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.1 MB/s | 370 kB 00:00 2024-05-08T22:52:09.357 INFO:teuthology.orchestra.run.smithi053.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 162 kB/s | 16 kB 00:00 2024-05-08T22:52:09.360 INFO:teuthology.orchestra.run.smithi046.stdout:(23/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.3 MB/s | 365 kB 00:00 2024-05-08T22:52:09.373 INFO:teuthology.orchestra.run.smithi086.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 228 kB/s | 44 kB 00:00 2024-05-08T22:52:09.394 INFO:teuthology.orchestra.run.smithi046.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.8 MB/s | 107 kB 00:00 2024-05-08T22:52:09.444 INFO:teuthology.orchestra.run.smithi046.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 95 kB 00:00 2024-05-08T22:52:09.448 INFO:teuthology.orchestra.run.smithi053.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.8 MB/s | 263 kB 00:00 2024-05-08T22:52:09.465 INFO:teuthology.orchestra.run.smithi086.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2024-05-08T22:52:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:(111/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-08T22:52:09.503 INFO:teuthology.orchestra.run.smithi046.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 3.1 MB/s | 181 kB 00:00 2024-05-08T22:52:09.515 INFO:teuthology.orchestra.run.smithi086.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2024-05-08T22:52:09.524 INFO:teuthology.orchestra.run.smithi053.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.2 MB/s | 170 kB 00:00 2024-05-08T22:52:09.562 INFO:teuthology.orchestra.run.smithi046.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 415 kB/s | 24 kB 00:00 2024-05-08T22:52:09.574 INFO:teuthology.orchestra.run.smithi053.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 362 kB/s | 18 kB 00:00 2024-05-08T22:52:09.585 INFO:teuthology.orchestra.run.smithi086.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.6 MB/s | 200 kB 00:00 2024-05-08T22:52:09.616 INFO:teuthology.orchestra.run.smithi086.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 383 kB/s | 48 kB 00:00 2024-05-08T22:52:09.637 INFO:teuthology.orchestra.run.smithi046.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.8 MB/s | 138 kB 00:00 2024-05-08T22:52:09.650 INFO:teuthology.orchestra.run.smithi086.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 650 kB/s | 87 kB 00:00 2024-05-08T22:52:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 412 kB/s | 58 kB 00:00 2024-05-08T22:52:09.750 INFO:teuthology.orchestra.run.smithi086.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 288 kB/s | 39 kB 00:00 2024-05-08T22:52:09.775 INFO:teuthology.orchestra.run.smithi086.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 661 kB/s | 83 kB 00:00 2024-05-08T22:52:09.892 INFO:teuthology.orchestra.run.smithi086.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-05-08T22:52:09.909 INFO:teuthology.orchestra.run.smithi086.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T22:52:09.934 INFO:teuthology.orchestra.run.smithi086.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-08T22:52:10.018 INFO:teuthology.orchestra.run.smithi086.stdout:(122/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 577 kB/s | 73 kB 00:00 2024-05-08T22:52:10.060 INFO:teuthology.orchestra.run.smithi086.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 409 kB/s | 62 kB 00:00 2024-05-08T22:52:10.094 INFO:teuthology.orchestra.run.smithi086.stdout:(124/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.5 MB/s | 562 kB 00:00 2024-05-08T22:52:10.119 INFO:teuthology.orchestra.run.smithi086.stdout:(125/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 803 kB/s | 47 kB 00:00 2024-05-08T22:52:10.219 INFO:teuthology.orchestra.run.smithi086.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-05-08T22:52:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 448 kB/s | 306 kB 00:00 2024-05-08T22:52:10.261 INFO:teuthology.orchestra.run.smithi086.stdout:(127/146): python3-cryptography-3.2.1-7.el8_9.x 3.3 MB/s | 559 kB 00:00 2024-05-08T22:52:10.287 INFO:teuthology.orchestra.run.smithi086.stdout:(128/146): NetworkManager-initscripts-updown-1. 862 kB/s | 144 kB 00:00 2024-05-08T22:52:10.337 INFO:teuthology.orchestra.run.smithi086.stdout:(129/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 852 kB/s | 100 kB 00:00 2024-05-08T22:52:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:(29/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 9.2 MB/s | 47 MB 00:05 2024-05-08T22:52:10.341 INFO:teuthology.orchestra.run.smithi053.stdout:(38/146): ceph-mgr-diskprediction-local-18.2.2- 4.2 MB/s | 7.4 MB 00:01 2024-05-08T22:52:10.362 INFO:teuthology.orchestra.run.smithi086.stdout:(130/146): python3-packaging-16.8-9.el8.noarch. 479 kB/s | 48 kB 00:00 2024-05-08T22:52:10.366 INFO:teuthology.orchestra.run.smithi053.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 544 kB/s | 59 kB 00:00 2024-05-08T22:52:10.413 INFO:teuthology.orchestra.run.smithi046.stdout:(30/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.4 MB/s | 3.5 MB 00:01 2024-05-08T22:52:10.487 INFO:teuthology.orchestra.run.smithi086.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2024-05-08T22:52:10.488 INFO:teuthology.orchestra.run.smithi046.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 4.1 MB/s | 3.5 MB 00:00 2024-05-08T22:52:10.513 INFO:teuthology.orchestra.run.smithi046.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.4 MB/s | 245 kB 00:00 2024-05-08T22:52:10.547 INFO:teuthology.orchestra.run.smithi046.stdout:(33/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 843 kB/s | 49 kB 00:00 2024-05-08T22:52:10.558 INFO:teuthology.orchestra.run.smithi053.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2024-05-08T22:52:10.572 INFO:teuthology.orchestra.run.smithi046.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 279 kB/s | 16 kB 00:00 2024-05-08T22:52:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.0 MB/s | 170 kB 00:00 2024-05-08T22:52:10.659 INFO:teuthology.orchestra.run.smithi053.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 437 kB/s | 44 kB 00:00 2024-05-08T22:52:10.679 INFO:teuthology.orchestra.run.smithi086.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-05-08T22:52:10.681 INFO:teuthology.orchestra.run.smithi046.stdout:(36/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.9 MB/s | 263 kB 00:00 2024-05-08T22:52:10.706 INFO:teuthology.orchestra.run.smithi046.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 360 kB/s | 18 kB 00:00 2024-05-08T22:52:10.759 INFO:teuthology.orchestra.run.smithi053.stdout:(42/146): libunwind-1.3.1-3.el8.x86_64.rpm 180 kB/s | 75 kB 00:00 2024-05-08T22:52:10.827 INFO:teuthology.orchestra.run.smithi053.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2024-05-08T22:52:10.861 INFO:teuthology.orchestra.run.smithi053.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2024-05-08T22:52:10.863 INFO:teuthology.orchestra.run.smithi086.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.6 MB/s | 854 kB 00:00 2024-05-08T22:52:10.915 INFO:teuthology.orchestra.run.smithi046.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 282 kB/s | 59 kB 00:00 2024-05-08T22:52:10.927 INFO:teuthology.orchestra.run.smithi053.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-05-08T22:52:10.973 INFO:teuthology.orchestra.run.smithi046.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2024-05-08T22:52:10.980 INFO:teuthology.orchestra.run.smithi086.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.0 MB/s | 711 kB 00:00 2024-05-08T22:52:11.007 INFO:teuthology.orchestra.run.smithi046.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 818 kB/s | 75 kB 00:00 2024-05-08T22:52:11.064 INFO:teuthology.orchestra.run.smithi086.stdout:(135/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.8 MB/s | 3.7 MB 00:00 2024-05-08T22:52:11.104 INFO:teuthology.orchestra.run.smithi053.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-05-08T22:52:11.108 INFO:teuthology.orchestra.run.smithi046.stdout:(41/146): ceph-mgr-diskprediction-local-18.2.2- 9.6 MB/s | 7.4 MB 00:00 2024-05-08T22:52:11.133 INFO:teuthology.orchestra.run.smithi046.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 348 kB/s | 44 kB 00:00 2024-05-08T22:52:11.136 INFO:teuthology.orchestra.run.smithi053.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 627 kB/s | 173 kB 00:00 2024-05-08T22:52:11.139 INFO:teuthology.orchestra.run.smithi086.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-05-08T22:52:11.167 INFO:teuthology.orchestra.run.smithi046.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2024-05-08T22:52:11.197 INFO:teuthology.orchestra.run.smithi086.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.4 MB/s | 333 kB 00:00 2024-05-08T22:52:11.202 INFO:teuthology.orchestra.run.smithi053.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 827 kB/s | 81 kB 00:00 2024-05-08T22:52:11.225 INFO:teuthology.orchestra.run.smithi046.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2024-05-08T22:52:11.228 INFO:teuthology.orchestra.run.smithi053.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2024-05-08T22:52:11.259 INFO:teuthology.orchestra.run.smithi046.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 188 kB/s | 16 kB 00:00 2024-05-08T22:52:11.295 INFO:teuthology.orchestra.run.smithi053.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2024-05-08T22:52:11.305 INFO:teuthology.orchestra.run.smithi086.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-05-08T22:52:11.309 INFO:teuthology.orchestra.run.smithi046.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-05-08T22:52:11.339 INFO:teuthology.orchestra.run.smithi086.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 221 kB 00:00 2024-05-08T22:52:11.342 INFO:teuthology.orchestra.run.smithi046.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2024-05-08T22:52:11.368 INFO:teuthology.orchestra.run.smithi046.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2024-05-08T22:52:11.393 INFO:teuthology.orchestra.run.smithi046.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2024-05-08T22:52:11.412 INFO:teuthology.orchestra.run.smithi053.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2024-05-08T22:52:11.418 INFO:teuthology.orchestra.run.smithi046.stdout:(50/146): python3-beautifulsoup4-4.6.3-2.el8.1. 596 kB/s | 185 kB 00:00 2024-05-08T22:52:11.423 INFO:teuthology.orchestra.run.smithi086.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-08T22:52:11.482 INFO:teuthology.orchestra.run.smithi086.stdout:(141/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 3.6 MB 00:01 2024-05-08T22:52:11.499 INFO:teuthology.orchestra.run.smithi086.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 237 kB 00:00 2024-05-08T22:52:11.512 INFO:teuthology.orchestra.run.smithi053.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 588 kB/s | 59 kB 00:00 2024-05-08T22:52:11.532 INFO:teuthology.orchestra.run.smithi086.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.6 MB/s | 180 kB 00:00 2024-05-08T22:52:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 87 kB/s | 18 kB 00:00 2024-05-08T22:52:11.596 INFO:teuthology.orchestra.run.smithi053.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2024-05-08T22:52:11.613 INFO:teuthology.orchestra.run.smithi053.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-05-08T22:52:11.616 INFO:teuthology.orchestra.run.smithi086.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 741 kB/s | 99 kB 00:00 2024-05-08T22:52:11.675 INFO:teuthology.orchestra.run.smithi086.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-08T22:52:11.694 INFO:teuthology.orchestra.run.smithi046.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 177 kB/s | 49 kB 00:00 2024-05-08T22:52:11.700 INFO:teuthology.orchestra.run.smithi086.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 616 kB/s | 103 kB 00:00 2024-05-08T22:52:11.703 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:52:11.703 INFO:teuthology.orchestra.run.smithi086.stdout:Total 14 MB/s | 215 MB 00:15 2024-05-08T22:52:11.713 INFO:teuthology.orchestra.run.smithi053.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2024-05-08T22:52:11.720 INFO:teuthology.orchestra.run.smithi046.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 393 kB/s | 59 kB 00:00 2024-05-08T22:52:11.738 INFO:teuthology.orchestra.run.smithi053.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2024-05-08T22:52:11.781 INFO:teuthology.orchestra.run.smithi046.stdout:(54/146): python3-kubernetes-11.0.0-6.el8.noarc 3.0 MB/s | 1.2 MB 00:00 2024-05-08T22:52:11.812 INFO:teuthology.orchestra.run.smithi046.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 514 kB/s | 60 kB 00:00 2024-05-08T22:52:11.813 INFO:teuthology.orchestra.run.smithi053.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2024-05-08T22:52:11.837 INFO:teuthology.orchestra.run.smithi046.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2024-05-08T22:52:11.847 INFO:teuthology.orchestra.run.smithi053.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-05-08T22:52:11.863 INFO:teuthology.orchestra.run.smithi046.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 192 kB/s | 16 kB 00:00 2024-05-08T22:52:11.888 INFO:teuthology.orchestra.run.smithi046.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 447 kB/s | 34 kB 00:00 2024-05-08T22:52:11.906 INFO:teuthology.orchestra.run.smithi053.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 700 kB/s | 64 kB 00:00 2024-05-08T22:52:11.913 INFO:teuthology.orchestra.run.smithi046.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2024-05-08T22:52:11.938 INFO:teuthology.orchestra.run.smithi046.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 855 kB/s | 64 kB 00:00 2024-05-08T22:52:11.964 INFO:teuthology.orchestra.run.smithi046.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 247 kB/s | 19 kB 00:00 2024-05-08T22:52:11.989 INFO:teuthology.orchestra.run.smithi046.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2024-05-08T22:52:11.998 INFO:teuthology.orchestra.run.smithi053.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 125 kB/s | 19 kB 00:00 2024-05-08T22:52:12.014 INFO:teuthology.orchestra.run.smithi046.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2024-05-08T22:52:12.040 INFO:teuthology.orchestra.run.smithi046.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2024-05-08T22:52:12.066 INFO:teuthology.orchestra.run.smithi046.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-05-08T22:52:12.090 INFO:teuthology.orchestra.run.smithi046.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-05-08T22:52:12.115 INFO:teuthology.orchestra.run.smithi046.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 814 kB/s | 61 kB 00:00 2024-05-08T22:52:12.140 INFO:teuthology.orchestra.run.smithi053.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 102 kB/s | 24 kB 00:00 2024-05-08T22:52:12.149 INFO:teuthology.orchestra.run.smithi046.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2024-05-08T22:52:12.165 INFO:teuthology.orchestra.run.smithi053.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 175 kB/s | 29 kB 00:00 2024-05-08T22:52:12.174 INFO:teuthology.orchestra.run.smithi046.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 273 kB/s | 23 kB 00:00 2024-05-08T22:52:12.200 INFO:teuthology.orchestra.run.smithi046.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T22:52:12.276 INFO:teuthology.orchestra.run.smithi053.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 203 kB/s | 27 kB 00:00 2024-05-08T22:52:12.300 INFO:teuthology.orchestra.run.smithi046.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 143 kB/s | 18 kB 00:00 2024-05-08T22:52:12.307 INFO:teuthology.orchestra.run.smithi053.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2024-05-08T22:52:12.374 INFO:teuthology.orchestra.run.smithi053.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2024-05-08T22:52:12.375 INFO:teuthology.orchestra.run.smithi046.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 256 kB/s | 43 kB 00:00 2024-05-08T22:52:12.401 INFO:teuthology.orchestra.run.smithi046.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2024-05-08T22:52:12.408 INFO:teuthology.orchestra.run.smithi053.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 611 kB/s | 61 kB 00:00 2024-05-08T22:52:12.475 INFO:teuthology.orchestra.run.smithi053.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 851 kB/s | 85 kB 00:00 2024-05-08T22:52:12.501 INFO:teuthology.orchestra.run.smithi046.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 873 kB/s | 109 kB 00:00 2024-05-08T22:52:12.516 INFO:teuthology.orchestra.run.smithi053.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-05-08T22:52:12.535 INFO:teuthology.orchestra.run.smithi046.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 319 kB/s | 43 kB 00:00 2024-05-08T22:52:12.564 INFO:teuthology.orchestra.run.smithi053.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T22:52:12.568 INFO:teuthology.orchestra.run.smithi046.stdout:(76/146): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2024-05-08T22:52:12.594 INFO:teuthology.orchestra.run.smithi046.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2024-05-08T22:52:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 374 kB/s | 44 kB 00:00 2024-05-08T22:52:12.701 INFO:teuthology.orchestra.run.smithi053.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2024-05-08T22:52:12.729 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-05-08T22:52:12.734 INFO:teuthology.orchestra.run.smithi053.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 118 kB/s | 20 kB 00:00 2024-05-08T22:52:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2024-05-08T22:52:12.787 INFO:teuthology.orchestra.run.smithi046.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2024-05-08T22:52:12.829 INFO:teuthology.orchestra.run.smithi046.stdout:(81/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 976 kB/s | 229 kB 00:00 2024-05-08T22:52:12.862 INFO:teuthology.orchestra.run.smithi046.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2024-05-08T22:52:12.887 INFO:teuthology.orchestra.run.smithi046.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 479 kB/s | 48 kB 00:00 2024-05-08T22:52:12.968 INFO:teuthology.orchestra.run.smithi053.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 165 kB/s | 44 kB 00:00 2024-05-08T22:52:12.993 INFO:teuthology.orchestra.run.smithi053.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 440 kB/s | 110 kB 00:00 2024-05-08T22:52:12.996 INFO:teuthology.orchestra.run.smithi046.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 520 kB/s | 87 kB 00:00 2024-05-08T22:52:13.013 INFO:teuthology.orchestra.run.smithi046.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 308 kB/s | 39 kB 00:00 2024-05-08T22:52:13.030 INFO:teuthology.orchestra.run.smithi046.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 344 kB/s | 58 kB 00:00 2024-05-08T22:52:13.068 INFO:teuthology.orchestra.run.smithi053.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 265 kB/s | 27 kB 00:00 2024-05-08T22:52:13.121 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-05-08T22:52:13.122 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-05-08T22:52:13.155 INFO:teuthology.orchestra.run.smithi046.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-05-08T22:52:13.169 INFO:teuthology.orchestra.run.smithi053.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 721 kB/s | 126 kB 00:00 2024-05-08T22:52:13.180 INFO:teuthology.orchestra.run.smithi046.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 451 kB/s | 83 kB 00:00 2024-05-08T22:52:13.211 INFO:teuthology.orchestra.run.smithi053.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.1 MB/s | 457 kB 00:00 2024-05-08T22:52:13.214 INFO:teuthology.orchestra.run.smithi046.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 938 kB/s | 172 kB 00:00 2024-05-08T22:52:13.290 INFO:teuthology.orchestra.run.smithi046.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 669 kB/s | 73 kB 00:00 2024-05-08T22:52:13.448 INFO:teuthology.orchestra.run.smithi053.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2024-05-08T22:52:13.449 INFO:teuthology.orchestra.run.smithi053.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2024-05-08T22:52:13.449 INFO:teuthology.orchestra.run.smithi053.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 4.8 MB/s | 709 kB 00:00 2024-05-08T22:52:13.449 INFO:teuthology.orchestra.run.smithi046.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-08T22:52:13.449 INFO:teuthology.orchestra.run.smithi046.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 492 kB/s | 62 kB 00:00 2024-05-08T22:52:13.450 INFO:teuthology.orchestra.run.smithi046.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.4 MB/s | 562 kB 00:00 2024-05-08T22:52:13.479 INFO:teuthology.orchestra.run.smithi053.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 258 kB/s | 39 kB 00:00 2024-05-08T22:52:13.524 INFO:teuthology.orchestra.run.smithi046.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-08T22:52:13.558 INFO:teuthology.orchestra.run.smithi046.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 217 kB/s | 47 kB 00:00 2024-05-08T22:52:13.592 INFO:teuthology.orchestra.run.smithi046.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.1 MB/s | 559 kB 00:00 2024-05-08T22:52:13.617 INFO:teuthology.orchestra.run.smithi046.stdout:(97/146): NetworkManager-initscripts-updown-1.4 1.5 MB/s | 144 kB 00:00 2024-05-08T22:52:13.677 INFO:teuthology.orchestra.run.smithi046.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 851 kB/s | 100 kB 00:00 2024-05-08T22:52:13.700 INFO:teuthology.orchestra.run.smithi046.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 442 kB/s | 48 kB 00:00 2024-05-08T22:52:13.726 INFO:teuthology.orchestra.run.smithi046.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 184 kB/s | 20 kB 00:00 2024-05-08T22:52:13.793 INFO:teuthology.orchestra.run.smithi046.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 375 kB/s | 44 kB 00:00 2024-05-08T22:52:13.809 INFO:teuthology.orchestra.run.smithi046.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2024-05-08T22:52:13.852 INFO:teuthology.orchestra.run.smithi046.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 211 kB/s | 27 kB 00:00 2024-05-08T22:52:13.894 INFO:teuthology.orchestra.run.smithi046.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 5.3 MB/s | 457 kB 00:00 2024-05-08T22:52:13.919 INFO:teuthology.orchestra.run.smithi046.stdout:(105/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2024-05-08T22:52:13.945 INFO:teuthology.orchestra.run.smithi053.stdout:(81/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 5.9 MB/s | 47 MB 00:08 2024-05-08T22:52:13.961 INFO:teuthology.orchestra.run.smithi046.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2024-05-08T22:52:13.971 INFO:teuthology.orchestra.run.smithi053.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 130 kB/s | 64 kB 00:00 2024-05-08T22:52:14.003 INFO:teuthology.orchestra.run.smithi046.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2024-05-08T22:52:14.004 INFO:teuthology.orchestra.run.smithi053.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 460 kB/s | 259 kB 00:00 2024-05-08T22:52:14.045 INFO:teuthology.orchestra.run.smithi046.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2024-05-08T22:52:14.093 INFO:teuthology.orchestra.run.smithi046.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 297 kB/s | 39 kB 00:00 2024-05-08T22:52:14.121 INFO:teuthology.orchestra.run.smithi053.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 591 kB/s | 103 kB 00:00 2024-05-08T22:52:14.129 INFO:teuthology.orchestra.run.smithi046.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-05-08T22:52:14.163 INFO:teuthology.orchestra.run.smithi053.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2024-05-08T22:52:14.179 INFO:teuthology.orchestra.run.smithi046.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 478 kB/s | 64 kB 00:00 2024-05-08T22:52:14.205 INFO:teuthology.orchestra.run.smithi046.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 923 kB/s | 103 kB 00:00 2024-05-08T22:52:14.289 INFO:teuthology.orchestra.run.smithi053.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2024-05-08T22:52:14.322 INFO:teuthology.orchestra.run.smithi053.stdout:(87/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-05-08T22:52:14.330 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-05-08T22:52:14.341 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-05-08T22:52:14.380 INFO:teuthology.orchestra.run.smithi046.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-05-08T22:52:14.422 INFO:teuthology.orchestra.run.smithi046.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.2 MB/s | 538 kB 00:00 2024-05-08T22:52:14.422 INFO:teuthology.orchestra.run.smithi053.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-08T22:52:14.456 INFO:teuthology.orchestra.run.smithi053.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-05-08T22:52:14.523 INFO:teuthology.orchestra.run.smithi053.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 963 kB/s | 225 kB 00:00 2024-05-08T22:52:14.530 INFO:teuthology.orchestra.run.smithi046.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-05-08T22:52:14.548 INFO:teuthology.orchestra.run.smithi053.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-05-08T22:52:14.564 INFO:teuthology.orchestra.run.smithi046.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-05-08T22:52:14.589 INFO:teuthology.orchestra.run.smithi046.stdout:(117/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-08T22:52:14.681 INFO:teuthology.orchestra.run.smithi046.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2024-05-08T22:52:14.698 INFO:teuthology.orchestra.run.smithi053.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2024-05-08T22:52:14.707 INFO:teuthology.orchestra.run.smithi046.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-05-08T22:52:14.748 INFO:teuthology.orchestra.run.smithi053.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-08T22:52:14.841 INFO:teuthology.orchestra.run.smithi053.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-08T22:52:14.857 INFO:teuthology.orchestra.run.smithi046.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 18 MB/s | 5.2 MB 00:00 2024-05-08T22:52:14.891 INFO:teuthology.orchestra.run.smithi053.stdout:(95/146): jq-1.6-7.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2024-05-08T22:52:14.907 INFO:teuthology.orchestra.run.smithi046.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-08T22:52:14.916 INFO:teuthology.orchestra.run.smithi053.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 912 kB/s | 198 kB 00:00 2024-05-08T22:52:14.966 INFO:teuthology.orchestra.run.smithi053.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2024-05-08T22:52:14.991 INFO:teuthology.orchestra.run.smithi046.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-05-08T22:52:14.992 INFO:teuthology.orchestra.run.smithi053.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-08T22:52:15.074 INFO:teuthology.orchestra.run.smithi046.stdout:(123/146): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-05-08T22:52:15.100 INFO:teuthology.orchestra.run.smithi046.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.0 MB/s | 202 kB 00:00 2024-05-08T22:52:15.108 INFO:teuthology.orchestra.run.smithi053.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 11 MB/s | 1.5 MB 00:00 2024-05-08T22:52:15.125 INFO:teuthology.orchestra.run.smithi046.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 406 kB/s | 54 kB 00:00 2024-05-08T22:52:15.200 INFO:teuthology.orchestra.run.smithi053.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.8 MB/s | 812 kB 00:00 2024-05-08T22:52:15.267 INFO:teuthology.orchestra.run.smithi046.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 11 MB/s | 1.5 MB 00:00 2024-05-08T22:52:15.342 INFO:teuthology.orchestra.run.smithi046.stdout:(127/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 777 kB/s | 188 kB 00:00 2024-05-08T22:52:15.434 INFO:teuthology.orchestra.run.smithi053.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 25 MB/s | 14 MB 00:00 2024-05-08T22:52:15.518 INFO:teuthology.orchestra.run.smithi053.stdout:(102/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-05-08T22:52:15.543 INFO:teuthology.orchestra.run.smithi053.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 165 kB/s | 18 kB 00:00 2024-05-08T22:52:15.568 INFO:teuthology.orchestra.run.smithi053.stdout:(104/146): python3-packaging-16.8-9.el8.noarch. 131 kB/s | 48 kB 00:00 2024-05-08T22:52:15.610 INFO:teuthology.orchestra.run.smithi053.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 465 kB/s | 43 kB 00:00 2024-05-08T22:52:15.642 INFO:teuthology.orchestra.run.smithi046.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-05-08T22:52:15.684 INFO:teuthology.orchestra.run.smithi046.stdout:(129/146): postgresql-contrib-10.23-4.module+el 1.9 MB/s | 812 kB 00:00 2024-05-08T22:52:15.722 INFO:teuthology.orchestra.run.smithi053.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 866 kB/s | 155 kB 00:00 2024-05-08T22:52:15.752 INFO:teuthology.orchestra.run.smithi053.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 594 kB/s | 109 kB 00:00 2024-05-08T22:52:15.777 INFO:teuthology.orchestra.run.smithi053.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 255 kB/s | 43 kB 00:00 2024-05-08T22:52:15.819 INFO:teuthology.orchestra.run.smithi053.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 193 kB/s | 19 kB 00:00 2024-05-08T22:52:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:(130/146): postgresql-server-10.23-4.module+el8 10 MB/s | 5.1 MB 00:00 2024-05-08T22:52:15.869 INFO:teuthology.orchestra.run.smithi053.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 374 kB/s | 44 kB 00:00 2024-05-08T22:52:15.928 INFO:teuthology.orchestra.run.smithi053.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-05-08T22:52:15.945 INFO:teuthology.orchestra.run.smithi053.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2024-05-08T22:52:15.985 INFO:teuthology.orchestra.run.smithi046.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T22:52:16.054 INFO:teuthology.orchestra.run.smithi053.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-05-08T22:52:16.071 INFO:teuthology.orchestra.run.smithi053.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2024-05-08T22:52:16.097 INFO:teuthology.orchestra.run.smithi053.stdout:(115/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-05-08T22:52:16.122 INFO:teuthology.orchestra.run.smithi046.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.2 MB/s | 303 kB 00:00 2024-05-08T22:52:16.178 INFO:teuthology.orchestra.run.smithi046.stdout:(133/146): librados2-18.2.2-1322.gd21e4f96.el8. 6.9 MB/s | 3.7 MB 00:00 2024-05-08T22:52:16.180 INFO:teuthology.orchestra.run.smithi053.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 531 kB/s | 58 kB 00:00 2024-05-08T22:52:16.214 INFO:teuthology.orchestra.run.smithi053.stdout:(117/146): libsepol-devel-2.9-3.el8.x86_64.rpm 547 kB/s | 87 kB 00:00 2024-05-08T22:52:16.239 INFO:teuthology.orchestra.run.smithi053.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2024-05-08T22:52:16.262 INFO:teuthology.orchestra.run.smithi046.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-08T22:52:16.320 INFO:teuthology.orchestra.run.smithi046.stdout:(135/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 5.6 MB/s | 3.6 MB 00:00 2024-05-08T22:52:16.330 INFO:teuthology.orchestra.run.smithi053.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 551 kB/s | 83 kB 00:00 2024-05-08T22:52:16.356 INFO:teuthology.orchestra.run.smithi053.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2024-05-08T22:52:16.399 INFO:teuthology.orchestra.run.smithi053.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-05-08T22:52:16.413 INFO:teuthology.orchestra.run.smithi046.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.0 MB/s | 711 kB 00:00 2024-05-08T22:52:16.465 INFO:teuthology.orchestra.run.smithi053.stdout:(122/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 669 kB/s | 73 kB 00:00 2024-05-08T22:52:16.471 INFO:teuthology.orchestra.run.smithi046.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.1 MB/s | 1.5 MB 00:00 2024-05-08T22:52:16.490 INFO:teuthology.orchestra.run.smithi053.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 671 kB/s | 62 kB 00:00 2024-05-08T22:52:16.504 INFO:teuthology.orchestra.run.smithi046.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2024-05-08T22:52:16.515 INFO:teuthology.orchestra.run.smithi053.stdout:(124/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-08T22:52:16.546 INFO:teuthology.orchestra.run.smithi046.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.0 MB/s | 1.1 MB 00:00 2024-05-08T22:52:16.603 INFO:teuthology.orchestra.run.smithi053.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.0 MB/s | 562 kB 00:00 2024-05-08T22:52:16.632 INFO:teuthology.orchestra.run.smithi053.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 403 kB/s | 47 kB 00:00 2024-05-08T22:52:16.647 INFO:teuthology.orchestra.run.smithi046.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 221 kB 00:00 2024-05-08T22:52:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-08T22:52:16.705 INFO:teuthology.orchestra.run.smithi046.stdout:(142/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 254 kB 00:00 2024-05-08T22:52:16.707 INFO:teuthology.orchestra.run.smithi053.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 11 MB/s | 2.3 MB 00:00 2024-05-08T22:52:16.733 INFO:teuthology.orchestra.run.smithi053.stdout:(128/146): NetworkManager-initscripts-updown-1. 1.4 MB/s | 144 kB 00:00 2024-05-08T22:52:16.756 INFO:teuthology.orchestra.run.smithi046.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.6 MB/s | 180 kB 00:00 2024-05-08T22:52:16.766 INFO:teuthology.orchestra.run.smithi053.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 3.3 MB/s | 559 kB 00:00 2024-05-08T22:52:16.824 INFO:teuthology.orchestra.run.smithi046.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 696 kB/s | 99 kB 00:00 2024-05-08T22:52:16.892 INFO:teuthology.orchestra.run.smithi053.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 547 kB/s | 100 kB 00:00 2024-05-08T22:52:16.908 INFO:teuthology.orchestra.run.smithi046.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 12 MB/s | 2.5 MB 00:00 2024-05-08T22:52:16.973 INFO:teuthology.orchestra.run.smithi046.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 473 kB/s | 103 kB 00:00 2024-05-08T22:52:16.978 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:52:16.978 INFO:teuthology.orchestra.run.smithi046.stdout:Total 12 MB/s | 215 MB 00:17 2024-05-08T22:52:17.017 INFO:teuthology.orchestra.run.smithi053.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2024-05-08T22:52:17.134 INFO:teuthology.orchestra.run.smithi053.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-05-08T22:52:17.284 INFO:teuthology.orchestra.run.smithi053.stdout:(133/146): librados2-18.2.2-1322.gd21e4f96.el8. 6.7 MB/s | 3.7 MB 00:00 2024-05-08T22:52:17.343 INFO:teuthology.orchestra.run.smithi053.stdout:(134/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 6.2 MB/s | 3.6 MB 00:00 2024-05-08T22:52:17.376 INFO:teuthology.orchestra.run.smithi053.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.4 MB/s | 854 kB 00:00 2024-05-08T22:52:17.410 INFO:teuthology.orchestra.run.smithi053.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.5 MB/s | 711 kB 00:00 2024-05-08T22:52:17.502 INFO:teuthology.orchestra.run.smithi053.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-05-08T22:52:17.535 INFO:teuthology.orchestra.run.smithi053.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.1 MB/s | 333 kB 00:00 2024-05-08T22:52:17.594 INFO:teuthology.orchestra.run.smithi053.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-05-08T22:52:17.628 INFO:teuthology.orchestra.run.smithi053.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-08T22:52:17.703 INFO:teuthology.orchestra.run.smithi053.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 237 kB 00:00 2024-05-08T22:52:17.720 INFO:teuthology.orchestra.run.smithi053.stdout:(142/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 254 kB 00:00 2024-05-08T22:52:17.795 INFO:teuthology.orchestra.run.smithi053.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-08T22:52:17.895 INFO:teuthology.orchestra.run.smithi053.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 516 kB/s | 99 kB 00:00 2024-05-08T22:52:18.069 INFO:teuthology.orchestra.run.smithi053.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 7.1 MB/s | 2.5 MB 00:00 2024-05-08T22:52:18.104 INFO:teuthology.orchestra.run.smithi053.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 333 kB/s | 103 kB 00:00 2024-05-08T22:52:18.108 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:52:18.108 INFO:teuthology.orchestra.run.smithi053.stdout:Total 11 MB/s | 215 MB 00:19 2024-05-08T22:52:19.158 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T22:52:19.253 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-08T22:52:19.635 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T22:52:19.635 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T22:52:19.702 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-08T22:52:19.702 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-08T22:52:20.402 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-05-08T22:52:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:52:20.789 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:21.006 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:21.039 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-08T22:52:21.055 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-08T22:52:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:52:21.292 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T22:52:21.305 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T22:52:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:52:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:52:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:52:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T22:52:22.731 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:23.356 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:23.748 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:24.039 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:24.171 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:24.883 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:52:25.172 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:52:25.786 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:52:25.992 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:52:26.384 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:52:26.676 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:52:26.873 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:26.976 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:27.238 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T22:52:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-08T22:52:27.447 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:52:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:52:27.464 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:52:27.601 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:52:27.730 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:27.734 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:27.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:52:27.986 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:28.122 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:52:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:52:28.318 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:52:28.397 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:52:28.487 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:28.525 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:28.596 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:52:28.607 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:52:28.762 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:28.866 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:52:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:52:29.153 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:52:29.213 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:52:29.242 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T22:52:29.367 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:52:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:52:29.435 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:52:29.437 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T22:52:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T22:52:29.650 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T22:52:29.745 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:29.825 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:52:30.226 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:30.287 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:52:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T22:52:30.437 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:30.509 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T22:52:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T22:52:30.907 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:30.925 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:52:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T22:52:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:52:31.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T22:52:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:31.626 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:52:31.673 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:52:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:52:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:52:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:52:32.024 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:52:32.258 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:32.299 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:52:32.349 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:32.373 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:52:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:52:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:52:32.875 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:52:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:52:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:52:33.235 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:52:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T22:52:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:52:33.593 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T22:52:33.697 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:52:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:52:33.922 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T22:52:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:52:34.095 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:52:34.193 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:34.209 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:52:34.344 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:34.459 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:34.475 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:52:34.627 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T22:52:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:52:34.892 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:52:34.903 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T22:52:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:52:35.275 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T22:52:35.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T22:52:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T22:52:35.788 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T22:52:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:52:35.884 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:52:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:52:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T22:52:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:52:36.518 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T22:52:36.686 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T22:52:36.712 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:52:36.782 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:52:36.925 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:52:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:52:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:52:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T22:52:37.397 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:37.474 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:37.552 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T22:52:37.704 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:52:37.721 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T22:52:38.067 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:52:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T22:52:38.315 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:52:38.356 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T22:52:38.422 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T22:52:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:38.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T22:52:38.811 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T22:52:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T22:52:38.991 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:39.003 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T22:52:39.229 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:52:39.236 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:52:39.346 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:39.371 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:39.402 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:52:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:52:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:52:39.588 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-08T22:52:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:52:39.816 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.7-17.el8.x86_64 50/162 2024-05-08T22:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:52:39.857 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:52:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T22:52:40.059 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 51/162 2024-05-08T22:52:40.233 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 52/162 2024-05-08T22:52:40.440 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 53/162 2024-05-08T22:52:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 54/162 2024-05-08T22:52:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T22:52:40.966 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 55/162 2024-05-08T22:52:41.216 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/162 2024-05-08T22:52:41.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T22:52:41.517 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/162 2024-05-08T22:52:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 58/162 2024-05-08T22:52:41.761 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T22:52:41.964 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/162 2024-05-08T22:52:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:52:42.256 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T22:52:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/162 2024-05-08T22:52:42.673 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 61/162 2024-05-08T22:52:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/162 2024-05-08T22:52:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T22:52:43.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 63/162 2024-05-08T22:52:43.396 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:52:43.682 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/162 2024-05-08T22:52:43.974 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:52:43.991 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/162 2024-05-08T22:52:44.117 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T22:52:44.295 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/162 2024-05-08T22:52:44.341 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T22:52:44.476 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T22:52:44.564 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 68/162 2024-05-08T22:52:44.594 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:52:44.776 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 69/162 2024-05-08T22:52:44.829 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T22:52:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:52:45.006 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 70/162 2024-05-08T22:52:45.059 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:52:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:52:45.228 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T22:52:45.268 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:52:45.304 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T22:52:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:52:45.443 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:52:45.515 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:45.581 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:45.643 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T22:52:45.694 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:52:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:45.868 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:45.962 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T22:52:45.972 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:52:46.128 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T22:52:46.197 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T22:52:46.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:52:46.218 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:46.315 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:52:46.418 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T22:52:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:52:46.508 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:52:46.607 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:46.652 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T22:52:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:52:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T22:52:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:47.020 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:52:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:47.245 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:52:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:52:47.600 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:52:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T22:52:47.860 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:52:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T22:52:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:52:48.091 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:52:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T22:52:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T22:52:48.256 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:52:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T22:52:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T22:52:48.486 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:52:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T22:52:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T22:52:48.732 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:52:48.796 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T22:52:48.883 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T22:52:48.976 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T22:52:49.006 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:52:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T22:52:49.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T22:52:49.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:52:49.260 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T22:52:49.375 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:52:49.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T22:52:49.521 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T22:52:49.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:52:49.651 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T22:52:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T22:52:49.828 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-08T22:52:49.915 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:52:49.926 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T22:52:50.016 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T22:52:50.108 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:52:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T22:52:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T22:52:50.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:52:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T22:52:50.608 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T22:52:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T22:52:50.668 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:52:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T22:52:50.842 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:52:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T22:52:51.102 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:52:51.133 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T22:52:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T22:52:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T22:52:51.481 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T22:52:51.675 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T22:52:51.741 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T22:52:51.923 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T22:52:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T22:52:52.149 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T22:52:52.168 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T22:52:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:52:52.367 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T22:52:52.539 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:52:52.564 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T22:52:52.581 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T22:52:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:52:52.829 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T22:52:52.860 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T22:52:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:52:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-08T22:52:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T22:52:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T22:52:53.319 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:52:53.362 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T22:52:53.436 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T22:52:53.518 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:52:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T22:52:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T22:52:53.943 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T22:52:53.946 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T22:52:54.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T22:52:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T22:52:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T22:52:54.322 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:52:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T22:52:54.412 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T22:52:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T22:52:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T22:52:54.810 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T22:52:54.851 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T22:52:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:52:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T22:52:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:52:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:52:55.287 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:52:55.323 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:52:55.497 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:52:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:52:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:52:55.796 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:52:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:52:56.018 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:52:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:52:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:52:56.227 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:52:56.411 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:52:56.424 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:52:56.628 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:52:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:52:56.836 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:52:56.933 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:52:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:52:57.152 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:52:57.263 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:52:57.295 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:52:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:52:57.565 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:52:57.689 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:52:57.811 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:52:57.908 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:52:58.023 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:52:58.150 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:52:58.284 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:52:58.423 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:52:58.497 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:52:58.685 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:52:58.706 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:52:58.888 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:52:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:52:59.147 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:52:59.371 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:52:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:53:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:00.547 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:53:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:53:00.794 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:53:00.848 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:53:01.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:53:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:53:01.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:53:01.442 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:53:01.514 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:01.635 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:53:01.681 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:53:01.721 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:01.876 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:01.888 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:53:02.456 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:53:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:53:02.702 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:53:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:53:02.888 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:53:02.949 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:53:03.046 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:53:03.592 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:53:07.153 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:08.612 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:09.249 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:09.975 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:53:10.250 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:10.420 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:53:10.557 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /run 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-05-08T22:53:21.091 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-05-08T22:53:21.092 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-05-08T22:53:21.092 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-05-08T22:53:21.092 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-05-08T22:53:21.092 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:53:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:23.102 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:23.331 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:23.549 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:23.765 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:24.238 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:24.465 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:24.482 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:26.377 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:26.550 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:26.721 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:53:27.020 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:53:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:28.284 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:29.498 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:29.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:29.806 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:53:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:31.041 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-05-08T22:53:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-05-08T22:53:31.042 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:53:31.435 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:53:32.089 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:32.123 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:32.331 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:32.373 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:32.488 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /dev 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /run 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /dev 2024-05-08T22:53:32.489 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:53:32.589 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:32.855 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:33.247 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:33.693 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:33.842 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:34.741 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:53:34.947 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:35.226 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:53:35.379 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:35.952 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:53:36.146 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:36.190 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:53:36.702 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:36.895 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:37.009 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:53:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:53:37.503 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:37.607 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:53:38.440 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T22:53:38.646 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:38.712 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:53:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:39.205 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:39.367 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:53:39.390 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:53:39.521 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:53:39.755 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:53:39.767 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T22:53:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:40.045 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T22:53:40.200 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:40.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:53:40.308 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:40.523 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:53:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:53:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:53:40.939 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:53:41.091 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:41.137 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:41.272 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:53:41.325 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:41.496 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:42.656 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:53:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:53:42.830 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:53:43.166 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:43.217 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:43.267 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:43.329 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:53:43.425 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:43.513 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:43.579 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:43.584 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:53:43.616 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:43.753 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:43.783 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:53:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:44.016 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:44.077 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:53:44.494 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:53:44.758 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:53:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:53:45.170 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:53:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:53:45.413 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:45.471 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:53:45.476 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:53:45.695 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:45.769 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:53:45.772 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:53:45.936 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:45.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:53:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:53:46.224 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T22:53:46.363 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:53:46.478 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:53:46.553 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:46.558 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:53:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:53:46.834 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:53:47.007 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:53:47.239 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:53:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:53:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:53:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:53:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:53:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:53:48.049 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:53:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:53:48.580 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:53:48.738 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:53:48.939 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:49.013 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:53:49.014 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:53:49.272 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:53:49.273 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:53:49.432 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:53:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:53:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:53:49.759 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T22:53:49.790 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:53:49.790 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:53:49.983 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:53:50.056 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:53:50.056 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:53:50.347 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:53:50.506 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:53:50.506 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:53:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:53:51.096 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T22:53:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:53:51.475 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:53:51.476 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:53:51.477 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:53:51.490 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:53:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:53:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T22:53:51.493 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T22:53:51.494 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T22:53:51.495 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/162 2024-05-08T22:53:51.496 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 118/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 119/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 120/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 121/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 122/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 123/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 124/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 125/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 126/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 127/162 2024-05-08T22:53:51.497 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 128/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 129/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:53:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:53:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:53:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:53:51.640 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:53:51.704 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:53:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:53:52.062 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:53:52.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:53:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:53:53.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T22:53:53.818 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T22:53:54.270 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T22:53:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T22:53:54.816 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T22:53:54.847 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T22:53:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:55.049 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:53:55.207 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:53:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:55.324 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:55.427 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:53:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:53:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:53:56.345 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:53:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:53:56.716 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:53:57.040 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:53:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:53:57.531 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:53:57.543 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:53:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T22:53:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:53:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:53:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:53:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:53:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:53:58.658 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:53:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:53:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:53:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:53:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:53:59.117 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:59.326 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:53:59.327 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:53:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:53:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:53:59.749 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:53:59.914 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:53:59.935 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:54:00.208 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:54:00.208 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:00.283 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:54:00.467 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:00.467 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:54:00.717 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:54:00.717 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:00.874 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:54:01.054 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:54:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:54:01.249 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:54:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:54:01.366 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:01.602 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:54:01.608 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:54:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:54:01.863 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:54:01.863 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-05-08T22:54:01.917 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:54:02.183 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T22:54:02.312 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T22:54:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-05-08T22:54:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:54:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:54:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:54:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:54:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:54:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:54:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:54:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:54:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:54:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:54:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:54:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:54:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:54:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:54:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:54:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:54:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:54:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:54:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:54:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:54:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:54:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:54:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:54:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:54:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:54:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:54:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:54:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:54:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:54:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:54:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:54:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:54:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:54:02.501 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-05-08T22:54:02.534 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T22:54:02.566 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T22:54:02.699 DEBUG:teuthology.parallel:result is None 2024-05-08T22:54:02.717 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T22:54:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T22:54:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:54:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:54:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:54:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:54:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:54:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:54:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:54:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:54:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:54:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:54:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:54:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:54:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:54:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:54:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T22:54:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T22:54:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T22:54:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T22:54:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T22:54:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T22:54:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T22:54:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T22:54:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T22:54:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T22:54:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T22:54:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T22:54:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T22:54:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T22:54:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T22:54:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T22:54:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T22:54:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T22:54:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T22:54:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T22:54:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T22:54:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T22:54:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T22:54:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T22:54:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T22:54:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T22:54:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T22:54:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T22:54:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T22:54:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T22:54:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T22:54:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-08T22:54:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T22:54:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T22:54:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T22:54:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T22:54:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T22:54:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T22:54:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:54:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:54:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:02.862 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:54:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T22:54:03.041 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:54:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:54:03.523 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:54:03.523 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:54:03.830 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:54:03.831 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:54:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:54:04.083 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:54:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:54:04.619 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:54:04.802 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:54:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:54:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:05.320 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:05.320 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:54:05.620 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:54:05.620 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:05.854 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:05.854 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:54:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:54:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:07.262 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:07.406 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:54:07.406 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:54:08.002 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:54:08.003 INFO:teuthology.orchestra.run.smithi046.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-08T22:54:08.003 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:54:08.719 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:54:08.720 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:54:08.721 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:54:08.722 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:54:08.723 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T22:54:08.724 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T22:54:08.725 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T22:54:08.726 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T22:54:08.727 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T22:54:08.728 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:54:08.730 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:54:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:54:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:54:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:54:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:54:18.656 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.377 INFO:teuthology.orchestra.run.smithi053.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-08T22:54:19.378 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.379 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.380 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.381 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:54:19.382 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:54:19.383 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.384 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:54:19.385 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:54:19.386 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:54:19.387 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:54:19.388 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-08T22:54:19.389 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-08T22:54:21.848 DEBUG:teuthology.parallel:result is None 2024-05-08T22:54:23.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:54:23.469 INFO:teuthology.orchestra.run.smithi046.stdout:Installed products updated. 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T22:54:24.578 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.579 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.580 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.581 INFO:teuthology.orchestra.run.smithi046.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:54:24.582 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:54:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:54:24.584 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:54:24.585 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:54:24.586 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:54:24.587 INFO:teuthology.orchestra.run.smithi046.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:54:24.588 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:54:24.589 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:54:24.589 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:54:24.589 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:54:24.589 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T22:54:25.002 DEBUG:teuthology.parallel:result is None 2024-05-08T22:54:25.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:25.235 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:54:25.287 INFO:teuthology.orchestra.run.smithi046.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.287 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.288 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:54:25.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:25.441 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:54:25.472 INFO:teuthology.orchestra.run.smithi053.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.472 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.472 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:54:25.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:25.620 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:54:25.650 INFO:teuthology.orchestra.run.smithi086.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.650 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:54:25.650 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:54:25.652 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T22:54:25.652 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:54:25.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:54:25.741 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:54:25.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:54:25.795 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:54:25.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:54:25.843 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T22:54:25.844 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:54:25.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:54:25.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:54:26.013 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:54:26.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:54:26.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:54:26.123 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:54:26.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:54:26.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:54:26.227 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T22:54:26.227 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:54:26.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:54:26.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:54:26.397 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:54:26.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:54:26.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:54:26.508 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:54:26.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:54:26.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:54:26.611 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T22:54:26.612 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:54:26.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:54:26.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:54:26.782 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:54:26.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:54:26.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:54:26.893 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:54:26.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:54:26.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:54:26.995 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T22:54:27.098 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T22:54:27.100 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:27.100 INFO:tasks.cephadm:Cluster fsid is ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:54:27.101 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T22:54:27.101 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.86'} 2024-05-08T22:54:27.101 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-05-08T22:54:27.101 INFO:tasks.cephadm:First mgr is x 2024-05-08T22:54:27.101 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T22:54:27.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-05-08T22:54:27.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-05-08T22:54:27.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-05-08T22:54:27.238 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T22:54:27.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:27.464 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T22:54:27.685 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:54:27.687 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T22:54:27.687 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.251 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:54 /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.251 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.632 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:54 /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.633 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:54 /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:28.978 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:29.028 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:29.052 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:54:29.120 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T22:54:29.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:54:29.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:54:29.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:54:30.223 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:54:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:55:02.587 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-05-08T22:55:02.588 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-08T22:55:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T22:55:04.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T22:55:11.035 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-08T22:55:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:55:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:55:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-05-08T22:55:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:55:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-08T22:55:11.036 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-08T22:55:11.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-05-08T22:55:11.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-05-08T22:55:11.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-05-08T22:55:11.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-05-08T22:55:11.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-05-08T22:55:11.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-05-08T22:55:11.348 INFO:tasks.cephadm:Writing seed config... 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T22:55:11.349 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T22:55:11.350 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T22:55:11.351 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T22:55:11.352 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T22:55:11.353 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:55:11.353 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T22:55:11.394 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ebe57712-0d8d-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T22:55:11.395 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T22:55:11.438 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T22:55:11.483 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T22:55:11.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:55:11.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T22:55:11.546 INFO:journalctl@ceph.mgr.x.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T22:55:11.706 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:55:11.707 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', 'ebe57712-0d8d-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-05-08T22:55:11.751 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:55:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T22:55:11.752 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-05-08T22:55:11.798 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:55:11.798 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-05-08T22:55:11.798 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-05-08T22:55:11.812 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T22:55:11.812 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T22:55:11.824 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T22:55:11.824 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:55:11.837 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-08T22:55:11.850 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-08T22:55:11.850 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-08T22:55:11.850 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-05-08T22:55:11.893 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:55:11.893 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T22:55:11.893 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-05-08T22:55:11.893 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-05-08T22:55:11.906 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T22:55:11.906 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T22:55:11.918 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T22:55:11.918 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:55:11.931 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-08T22:55:11.943 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-08T22:55:11.943 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-08T22:55:11.943 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-05-08T22:55:11.944 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:11.944 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 139919767402984 on /run/cephadm/ebe57712-0d8d-11ef-bc97-c7b262605968.lock 2024-05-08T22:55:11.944 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 139919767402984 acquired on /run/cephadm/ebe57712-0d8d-11ef-bc97-c7b262605968.lock 2024-05-08T22:55:11.944 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-05-08T22:55:11.945 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-05-08T22:55:11.945 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-08T22:55:11.953 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-05-08T22:55:11.953 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-05-08T22:55:11.960 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T22:55:11.960 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-08T22:55:11.960 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1774sec hoplimit 64 pref medium 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-05-08T22:55:11.967 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T22:55:11.968 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-08T22:55:11.968 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-08T22:55:11.968 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T22:55:11.969 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T22:55:11.970 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:55:12.324 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T22:55:12.325 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T22:55:13.592 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T22:55:13.593 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T22:55:13.593 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T22:55:14.212 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-05-08T22:55:14.212 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-05-08T22:55:14.795 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDSAjxmFArFHRAApNhX1Yk9gLa/i+r/UKU6Xg== 2024-05-08T22:55:15.402 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDTAjxmOAj8BBAAUgtLZ4EamDY2rDvMyTMqcw== 2024-05-08T22:55:15.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDTAjxmHtg5KRAAJxoMkQYKns6/0CuThFWf/A== 2024-05-08T22:55:15.988 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-05-08T22:55:16.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = pacific 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:55:16.638 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-05-08T22:55:17.304 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-05-08T22:55:17.548 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T22:55:17.748 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T22:55:18.010 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target. 2024-05-08T22:55:18.011 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target. 2024-05-08T22:55:19.400 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a 2024-05-08T22:55:19.400 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T22:55:19.609 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target.wants/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@.service. 2024-05-08T22:55:19.701 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:19 smithi046 systemd[1]: Starting Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:55:20.117 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:55:20.117 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-08T22:55:20.134 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:55:20.134 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:55:20.134 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-08T22:55:20.134 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T22:55:20.135 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-05-08T22:55:20.135 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-05-08T22:55:20.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:20 smithi046 bash[36363]: a369bb626c1688b3f69793ea4480a8da56799213aee65d691dfe6b960264e58d 2024-05-08T22:55:20.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:20 smithi046 systemd[1]: Started Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:55:20.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:20 smithi046 ceph-mon[36401]: mkfs ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:20.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:20 smithi046 ceph-mon[36401]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:55:21.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T22:55:21.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.555553s) 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T22:55:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.600+0000 7f01ba000700 1 Processor -- start 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.600+0000 7f01ba000700 1 -- start start 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.600+0000 7f01ba000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.601+0000 7f01ba000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4079e30 con 0x7f01b4107250 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.601+0000 7f01b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.601+0000 7f01b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38914/0 (socket says 172.21.15.46:38914) 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.601+0000 7f01b3400700 1 -- 172.21.15.46:0/583716489 learned_addr learned my addr 172.21.15.46:0/583716489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.602+0000 7f01b3400700 1 -- 172.21.15.46:0/583716489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4109b80 con 0x7f01b4107250 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.602+0000 7f01b3400700 1 --2- 172.21.15.46:0/583716489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f01a40098a0 tx=0x7f01a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f244fdebd5e4f4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.602+0000 7f01b2000700 1 -- 172.21.15.46:0/583716489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01a4004160 con 0x7f01b4107250 2024-05-08T22:55:21.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.602+0000 7f01b2000700 1 -- 172.21.15.46:0/583716489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a40042c0 con 0x7f01b4107250 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.602+0000 7f01b2000700 1 -- 172.21.15.46:0/583716489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01a40045a0 con 0x7f01b4107250 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 -- 172.21.15.46:0/583716489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 msgr2=0x7f01b4109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 --2- 172.21.15.46:0/583716489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f01a40098a0 tx=0x7f01a4009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 -- 172.21.15.46:0/583716489 shutdown_connections 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 --2- 172.21.15.46:0/583716489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4109640 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 -- 172.21.15.46:0/583716489 >> 172.21.15.46:0/583716489 conn(0x7f01b4100c20 msgr2=0x7f01b4103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 -- 172.21.15.46:0/583716489 shutdown_connections 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.603+0000 7f01ba000700 1 -- 172.21.15.46:0/583716489 wait complete. 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.604+0000 7f01ba000700 1 Processor -- start 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.604+0000 7f01ba000700 1 -- start start 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01ba000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01ba000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b419a110 con 0x7f01b4107250 2024-05-08T22:55:21.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38924/0 (socket says 172.21.15.46:38924) 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01b3400700 1 -- 172.21.15.46:0/3118492683 learned_addr learned my addr 172.21.15.46:0/3118492683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01b3400700 1 -- 172.21.15.46:0/3118492683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4009550 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.605+0000 7f01b3400700 1 --2- 172.21.15.46:0/3118492683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f01b4079270 tx=0x7f01a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01a4011630 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4011790 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01a4011a30 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b419a310 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b419a7b0 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f01a4022070 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.606+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f01a401c450 con 0x7f01b4107250 2024-05-08T22:55:21.038 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.607+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b4193a60 con 0x7f01b4107250 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.610+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f01a401b450 con 0x7f01b4107250 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.687+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f01b410a000 con 0x7f01b4107250 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.687+0000 7f01abe00700 1 -- 172.21.15.46:0/3118492683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f01a401b5f0 con 0x7f01b4107250 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 msgr2=0x7f01b4199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 --2- 172.21.15.46:0/3118492683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f01b4079270 tx=0x7f01a4004750 comp rx=0 tx=0).stop 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 shutdown_connections 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 --2- 172.21.15.46:0/3118492683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b4107250 0x7f01b4199bd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 >> 172.21.15.46:0/3118492683 conn(0x7f01b4100c20 msgr2=0x7f01b4102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.688+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 shutdown_connections 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:20.689+0000 7f01ba000700 1 -- 172.21.15.46:0/3118492683 wait complete. 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-05-08T22:55:21.039 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T22:55:21.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:55:21.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:55:21.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: fsmap 2024-05-08T22:55:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:55:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: mgrmap e1: no daemons active 2024-05-08T22:55:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:21 smithi046 ceph-mon[36401]: from='client.? 172.21.15.46:0/3118492683' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T22:55:22.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:22.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T22:55:22.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:22.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-08T22:55:22.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T22:55:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.455+0000 7f5ba6400700 1 Processor -- start 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.455+0000 7f5ba6400700 1 -- start start 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.456+0000 7f5ba6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.456+0000 7f5ba6400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0107160 con 0x7f5ba0106770 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.456+0000 7f5b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.456+0000 7f5b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38940/0 (socket says 172.21.15.46:38940) 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.456+0000 7f5b9f400700 1 -- 172.21.15.46:0/2490282803 learned_addr learned my addr 172.21.15.46:0/2490282803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.457+0000 7f5b9f400700 1 -- 172.21.15.46:0/2490282803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0107920 con 0x7f5ba0106770 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.457+0000 7f5b9f400700 1 --2- 172.21.15.46:0/2490282803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b900098a0 tx=0x7f5b90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad7487a0c541db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.457+0000 7f5b9ea00700 1 -- 172.21.15.46:0/2490282803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b90004160 con 0x7f5ba0106770 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.457+0000 7f5b9ea00700 1 -- 172.21.15.46:0/2490282803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900042c0 con 0x7f5ba0106770 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.457+0000 7f5b9ea00700 1 -- 172.21.15.46:0/2490282803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b900045a0 con 0x7f5ba0106770 2024-05-08T22:55:22.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 -- 172.21.15.46:0/2490282803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 msgr2=0x7f5ba0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 --2- 172.21.15.46:0/2490282803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b900098a0 tx=0x7f5b90009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 -- 172.21.15.46:0/2490282803 shutdown_connections 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 --2- 172.21.15.46:0/2490282803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba0106b90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 -- 172.21.15.46:0/2490282803 >> 172.21.15.46:0/2490282803 conn(0x7f5ba0101d10 msgr2=0x7f5ba0104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 -- 172.21.15.46:0/2490282803 shutdown_connections 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.458+0000 7f5ba6400700 1 -- 172.21.15.46:0/2490282803 wait complete. 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.459+0000 7f5ba6400700 1 Processor -- start 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.459+0000 7f5ba6400700 1 -- start start 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5ba6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5ba6400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019c2d0 con 0x7f5ba0106770 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38956/0 (socket says 172.21.15.46:38956) 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5b9f400700 1 -- 172.21.15.46:0/3717383369 learned_addr learned my addr 172.21.15.46:0/3717383369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:22.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5b9f400700 1 -- 172.21.15.46:0/3717383369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b90009550 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.460+0000 7f5b9f400700 1 --2- 172.21.15.46:0/3717383369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b90004000 tx=0x7f5b90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b90011630 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90011790 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b90011a30 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba019c4d0 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba019c970 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5b90022070 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.461+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b9001c450 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.462+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba0195cf0 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.465+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5b9001b450 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.539+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5ba0107c90 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.543+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b9001b5f0 con 0x7f5ba0106770 2024-05-08T22:55:22.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.544+0000 7f5b9cc00700 1 -- 172.21.15.46:0/3717383369 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5b900375d0 con 0x7f5ba0106770 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 msgr2=0x7f5ba019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 --2- 172.21.15.46:0/3717383369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b90004000 tx=0x7f5b90004750 comp rx=0 tx=0).stop 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 shutdown_connections 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 --2- 172.21.15.46:0/3717383369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ba0106770 0x7f5ba019bd90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 >> 172.21.15.46:0/3717383369 conn(0x7f5ba0101d10 msgr2=0x7f5ba0192e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 shutdown_connections 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:21.545+0000 7f5ba6400700 1 -- 172.21.15.46:0/3717383369 wait complete. 2024-05-08T22:55:22.030 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-05-08T22:55:22.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:22 smithi046 ceph-mon[36401]: from='client.? 172.21.15.46:0/3717383369' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:55:22.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:22 smithi046 ceph-mon[36401]: from='client.? 172.21.15.46:0/3717383369' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T22:55:22.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.405+0000 7f89a3000700 1 Processor -- start 2024-05-08T22:55:22.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.405+0000 7f89a3000700 1 -- start start 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.405+0000 7f89a3000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.405+0000 7f89a3000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c107170 con 0x7f899c106780 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49782/0 (socket says 172.21.15.46:49782) 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899be00700 1 -- 172.21.15.46:0/1628317550 learned_addr learned my addr 172.21.15.46:0/1628317550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899be00700 1 -- 172.21.15.46:0/1628317550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c107930 con 0x7f899c106780 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899be00700 1 --2- 172.21.15.46:0/1628317550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f898c0098a0 tx=0x7f898c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab986f7e9b4b869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.406+0000 7f899aa00700 1 -- 172.21.15.46:0/1628317550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c004160 con 0x7f899c106780 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f899aa00700 1 -- 172.21.15.46:0/1628317550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f898c0042c0 con 0x7f899c106780 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f899aa00700 1 -- 172.21.15.46:0/1628317550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c011a00 con 0x7f899c106780 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 -- 172.21.15.46:0/1628317550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 msgr2=0x7f899c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 --2- 172.21.15.46:0/1628317550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f898c0098a0 tx=0x7f898c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 -- 172.21.15.46:0/1628317550 shutdown_connections 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 --2- 172.21.15.46:0/1628317550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c106ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 -- 172.21.15.46:0/1628317550 >> 172.21.15.46:0/1628317550 conn(0x7f899c101d20 msgr2=0x7f899c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 -- 172.21.15.46:0/1628317550 shutdown_connections 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.407+0000 7f89a3000700 1 -- 172.21.15.46:0/1628317550 wait complete. 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.408+0000 7f89a3000700 1 Processor -- start 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.408+0000 7f89a3000700 1 -- start start 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f89a3000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f89a3000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c19a980 con 0x7f899c106780 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f899be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f899be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49790/0 (socket says 172.21.15.46:49790) 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f899be00700 1 -- 172.21.15.46:0/912900930 learned_addr learned my addr 172.21.15.46:0/912900930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.409+0000 7f899be00700 1 -- 172.21.15.46:0/912900930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c009550 con 0x7f899c106780 2024-05-08T22:55:22.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f899be00700 1 --2- 172.21.15.46:0/912900930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f898c009130 tx=0x7f898c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c003ab0 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c19ab80 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f898c003c10 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c0247d0 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c19b020 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f898c024d00 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.410+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f898c0193c0 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.411+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c068010 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.414+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f898c01abb0 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.488+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f899c10ab60 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.489+0000 7f8998c00700 1 -- 172.21.15.46:0/912900930 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f898c01ad50 con 0x7f899c106780 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.489+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 msgr2=0x7f899c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.489+0000 7f89a3000700 1 --2- 172.21.15.46:0/912900930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f898c009130 tx=0x7f898c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.490+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 shutdown_connections 2024-05-08T22:55:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.490+0000 7f89a3000700 1 --2- 172.21.15.46:0/912900930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c106780 0x7f899c19a440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.490+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 >> 172.21.15.46:0/912900930 conn(0x7f899c101d20 msgr2=0x7f899c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.490+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 shutdown_connections 2024-05-08T22:55:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:22.490+0000 7f89a3000700 1 -- 172.21.15.46:0/912900930 wait complete. 2024-05-08T22:55:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-05-08T22:55:23.143 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:22 smithi046 systemd[1]: Stopping Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:55:23.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 ceph-mon[36401]: from='client.? 172.21.15.46:0/912900930' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:55:23.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a[36378]: 2024-05-08T22:55:23.240+0000 7f7701400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:55:23.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a[36378]: 2024-05-08T22:55:23.240+0000 7f7701400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T22:55:23.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 bash[36734]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a 2024-05-08T22:55:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 systemd[1]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T22:55:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 systemd[1]: Stopped Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:55:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:23 smithi046 systemd[1]: Starting Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:55:24.414 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 bash[36860]: 4d191837d87e4d85e8ee03a51c74ec534b6f0ca9fabcee1c2ff06dd37723e61d 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 systemd[1]: Started Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: pidfile_write: ignore empty --pid-file 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: load: jerasure load: lrc 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Git sha 0 2024-05-08T22:55:24.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: DB SUMMARY 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: DB Session ID: 680F0664WZ42U07ZIHA9 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99421 ; 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:55:24.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.env: 0x55bfbe3be820 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.info_log: 0x55bfdfd91560 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.statistics: (nil) 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.use_fsync: 0 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:55:24.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.db_log_dir: 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.wal_dir: 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:55:24.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.write_buffer_manager: 0x55bfdf026320 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:55:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.unordered_write: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.row_cache: None 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.wal_filter: None 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.wal_compression: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:55:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:55:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:55:24.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_open_files: -1 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:55:24.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Compression algorithms supported: 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kZSTD supported: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:55:24.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.merge_operator: 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_filter: None 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:55:24.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bfdfd91660) 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-05-08T22:55:24.790 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x55bfdf0a91f0 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-05-08T22:55:24.791 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-05-08T22:55:24.792 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:55:24.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression: NoCompression 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.num_levels: 7 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:55:24.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:55:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:55:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:55:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:55:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.ttl: 2592000 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:55:24.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 55c2e0e3-c648-4845-940e-95cd76035f47 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715208924432102, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715208924435192, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92628, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13781, "raw_average_key_size": 50, "raw_value_size": 85101, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715208924, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "55c2e0e3-c648-4845-940e-95cd76035f47", "db_session_id": "680F0664WZ42U07ZIHA9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715208924435395, "job": 1, "event": "recovery_finished"} 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bfdf146000 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: DB pointer 0x55bfdf134000 2024-05-08T22:55:24.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:55:24.802 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 94.24 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 94.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.803 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 10.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 10.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x55bfdf0a91f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.4e-05 secs_since: 0 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:55:24.804 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???) e1 preinit fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).mds e1 new map 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).mds e1 print_map 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:55:24.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:55:24.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:24 smithi046 ceph-mon[36898]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T22:55:25.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.846+0000 7f1255a00700 1 Processor -- start 2024-05-08T22:55:25.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.846+0000 7f1255a00700 1 -- start start 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.847+0000 7f1255a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.847+0000 7f1255a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501093b0 con 0x7f12501089c0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.848+0000 7f124ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.848+0000 7f124ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49802/0 (socket says 172.21.15.46:49802) 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.848+0000 7f124ea00700 1 -- 172.21.15.46:0/2103344911 learned_addr learned my addr 172.21.15.46:0/2103344911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.848+0000 7f124ea00700 1 -- 172.21.15.46:0/2103344911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250109b70 con 0x7f12501089c0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f124ea00700 1 --2- 172.21.15.46:0/2103344911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12400098a0 tx=0x7f1240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6eed9319a4f219 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f124e000700 1 -- 172.21.15.46:0/2103344911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1240004160 con 0x7f12501089c0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f124e000700 1 -- 172.21.15.46:0/2103344911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12400042c0 con 0x7f12501089c0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f124e000700 1 -- 172.21.15.46:0/2103344911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1240011a00 con 0x7f12501089c0 2024-05-08T22:55:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f1255a00700 1 -- 172.21.15.46:0/2103344911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 msgr2=0x7f1250108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.849+0000 7f1255a00700 1 --2- 172.21.15.46:0/2103344911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12400098a0 tx=0x7f1240009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.850+0000 7f1255a00700 1 -- 172.21.15.46:0/2103344911 shutdown_connections 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.850+0000 7f1255a00700 1 --2- 172.21.15.46:0/2103344911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f1250108de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.850+0000 7f1255a00700 1 -- 172.21.15.46:0/2103344911 >> 172.21.15.46:0/2103344911 conn(0x7f125007be10 msgr2=0x7f12501064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.850+0000 7f1255a00700 1 -- 172.21.15.46:0/2103344911 shutdown_connections 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.850+0000 7f1255a00700 1 -- 172.21.15.46:0/2103344911 wait complete. 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.851+0000 7f1255a00700 1 Processor -- start 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.851+0000 7f1255a00700 1 -- start start 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.851+0000 7f1255a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.851+0000 7f1255a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125019cbe0 con 0x7f12501089c0 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f124ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:25.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f124ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49816/0 (socket says 172.21.15.46:49816) 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f124ea00700 1 -- 172.21.15.46:0/3976667159 learned_addr learned my addr 172.21.15.46:0/3976667159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f124ea00700 1 -- 172.21.15.46:0/3976667159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009550 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f124ea00700 1 --2- 172.21.15.46:0/3976667159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1240009130 tx=0x7f1240011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1240003ab0 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.852+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1240003c10 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.853+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12400247d0 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.853+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125019cde0 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.853+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125019d280 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.853+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1240024d00 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.853+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f124001a430 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.854+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12500625d0 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.857+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f124001abb0 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.931+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f125019fb10 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.934+0000 7f1247400700 1 -- 172.21.15.46:0/3976667159 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f124001ad50 con 0x7f12501089c0 2024-05-08T22:55:25.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 msgr2=0x7f125019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 --2- 172.21.15.46:0/3976667159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1240009130 tx=0x7f1240011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 shutdown_connections 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 --2- 172.21.15.46:0/3976667159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12501089c0 0x7f125019c6a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 >> 172.21.15.46:0/3976667159 conn(0x7f125007be10 msgr2=0x7f1250105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 shutdown_connections 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:24.935+0000 7f1255a00700 1 -- 172.21.15.46:0/3976667159 wait complete. 2024-05-08T22:55:25.294 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T22:55:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:55:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-05-08T22:55:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T22:55:25.322 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T22:55:25.695 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x 2024-05-08T22:55:25.695 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: fsmap 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: mgrmap e1: no daemons active 2024-05-08T22:55:25.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:25 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3976667159' entity='client.admin' 2024-05-08T22:55:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968.target.wants/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@.service. 2024-05-08T22:55:26.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:25 smithi046 systemd[1]: Starting Ceph mgr.x for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:55:26.477 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:55:26.477 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-08T22:55:26.493 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:55:26.493 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:55:26.493 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-08T22:55:26.493 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T22:55:26.512 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:55:26.513 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-05-08T22:55:26.529 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-05-08T22:55:26.723 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:26 smithi046 bash[37135]: b5657c15bd3d6067b21ea2c5e0e1181fe48fded6a5054c6a196f110f1bad5159 2024-05-08T22:55:26.723 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:26 smithi046 systemd[1]: Started Ceph mgr.x for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:55:27.267 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:27 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3740870737' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:55:27.268 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:27 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:27.008+0000 7f8f199de200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:55:27.268 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:27 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:27.146+0000 7f8f199de200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:55:27.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:27.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:55:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:55:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:55:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:55:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:55:27.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:55:27.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:55:20.135373+0000", 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:55:27.350 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.917+0000 7f2723000700 1 Processor -- start 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.917+0000 7f2723000700 1 -- start start 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.918+0000 7f2723000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.918+0000 7f2723000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c107170 con 0x7f271c106780 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.918+0000 7f271be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.918+0000 7f271be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49858/0 (socket says 172.21.15.46:49858) 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.918+0000 7f271be00700 1 -- 172.21.15.46:0/3826927486 learned_addr learned my addr 172.21.15.46:0/3826927486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.919+0000 7f271be00700 1 -- 172.21.15.46:0/3826927486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c107930 con 0x7f271c106780 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.919+0000 7f271be00700 1 --2- 172.21.15.46:0/3826927486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f270c0098a0 tx=0x7f270c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2dfddfcc38daa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.919+0000 7f271aa00700 1 -- 172.21.15.46:0/3826927486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f270c004160 con 0x7f271c106780 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.919+0000 7f271aa00700 1 -- 172.21.15.46:0/3826927486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f270c0042c0 con 0x7f271c106780 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.919+0000 7f271aa00700 1 -- 172.21.15.46:0/3826927486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f270c011a00 con 0x7f271c106780 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 -- 172.21.15.46:0/3826927486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 msgr2=0x7f271c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:27.351 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 --2- 172.21.15.46:0/3826927486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f270c0098a0 tx=0x7f270c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 -- 172.21.15.46:0/3826927486 shutdown_connections 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 --2- 172.21.15.46:0/3826927486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c106ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 -- 172.21.15.46:0/3826927486 >> 172.21.15.46:0/3826927486 conn(0x7f271c101d20 msgr2=0x7f271c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 -- 172.21.15.46:0/3826927486 shutdown_connections 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.920+0000 7f2723000700 1 -- 172.21.15.46:0/3826927486 wait complete. 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.921+0000 7f2723000700 1 Processor -- start 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.921+0000 7f2723000700 1 -- start start 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f2723000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f2723000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c19cbd0 con 0x7f271c106780 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f271be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f271be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49864/0 (socket says 172.21.15.46:49864) 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f271be00700 1 -- 172.21.15.46:0/3740870737 learned_addr learned my addr 172.21.15.46:0/3740870737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.922+0000 7f271be00700 1 -- 172.21.15.46:0/3740870737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c009550 con 0x7f271c106780 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f271be00700 1 --2- 172.21.15.46:0/3740870737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f270c009130 tx=0x7f270c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:27.352 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f270c003ab0 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f270c003c10 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f270c0247d0 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c19cdd0 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c19d270 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f270c024d00 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.923+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f270c0193c0 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.924+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c068010 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:26.927+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f270c01abb0 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.005+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f271c10ab00 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.005+0000 7f2718c00700 1 -- 172.21.15.46:0/3740870737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f270c01ad50 con 0x7f271c106780 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.006+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 msgr2=0x7f271c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.006+0000 7f2723000700 1 --2- 172.21.15.46:0/3740870737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f270c009130 tx=0x7f270c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:27.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.007+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 shutdown_connections 2024-05-08T22:55:27.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.007+0000 7f2723000700 1 --2- 172.21.15.46:0/3740870737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f271c106780 0x7f271c19c690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:27.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.007+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 >> 172.21.15.46:0/3740870737 conn(0x7f271c101d20 msgr2=0x7f271c102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:27.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.007+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 shutdown_connections 2024-05-08T22:55:27.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:27.007+0000 7f2723000700 1 -- 172.21.15.46:0/3740870737 wait complete. 2024-05-08T22:55:27.354 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-05-08T22:55:27.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:27 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:27.267+0000 7f8f199de200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:55:27.527 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:27 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:27.385+0000 7f8f199de200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:55:28.609 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:28.281+0000 7f8f199de200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:55:28.609 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:28.609+0000 7f8f199de200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:55:28.874 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:28.726+0000 7f8f199de200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:55:29.254 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:28.874+0000 7f8f199de200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:55:29.254 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:29.076+0000 7f8f199de200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:55:29.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:29.254+0000 7f8f199de200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:55:29.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:29.372+0000 7f8f199de200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:55:30.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:29.880+0000 7f8f199de200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:55:30.656 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:30.392+0000 7f8f199de200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:55:31.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:30.656+0000 7f8f199de200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:55:31.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:30.784+0000 7f8f199de200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:55:32.239 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:31 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:31.801+0000 7f8f199de200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:55:32.239 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:31 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:31.932+0000 7f8f199de200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:55:32.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:32.137+0000 7f8f199de200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:55:33.274 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:33.023+0000 7f8f199de200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:55:33.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:33.274+0000 7f8f199de200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:55:34.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:33.542+0000 7f8f199de200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:55:34.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:34 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1786644552' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:55:35.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:34 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:34.762+0000 7f8f199de200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:55:35.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:35.162+0000 7f8f199de200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:55:35.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:35.286+0000 7f8f199de200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:55:36.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:35.560+0000 7f8f199de200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: Activating manager daemon x 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: mgrmap e2: x(active, starting, since 0.00495173s) 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: Manager daemon x is now available 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' 2024-05-08T22:55:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' 2024-05-08T22:55:36.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:35 smithi046 ceph-mon[36898]: from='mgr.14100 172.21.15.46:0/1600122675' entity='mgr.x' 2024-05-08T22:55:38.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:37 smithi046 ceph-mon[36898]: mgrmap e3: x(active, since 1.01081s) 2024-05-08T22:55:38.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:55:38.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:55:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:55:38.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:55:38.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:55:20.135373+0000", 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.303+0000 7f6d91800700 1 Processor -- start 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.303+0000 7f6d91800700 1 -- start start 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.303+0000 7f6d91800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.303+0000 7f6d91800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c107170 con 0x7f6d8c106780 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.304+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.304+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48086/0 (socket says 172.21.15.46:48086) 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.304+0000 7f6d8aa00700 1 -- 172.21.15.46:0/61972555 learned_addr learned my addr 172.21.15.46:0/61972555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.304+0000 7f6d8aa00700 1 -- 172.21.15.46:0/61972555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c107930 con 0x7f6d8c106780 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.305+0000 7f6d8aa00700 1 --2- 172.21.15.46:0/61972555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f764bedd90ce546d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.305+0000 7f6d89600700 1 -- 172.21.15.46:0/61972555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c004160 con 0x7f6d8c106780 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.305+0000 7f6d89600700 1 -- 172.21.15.46:0/61972555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d7c0042c0 con 0x7f6d8c106780 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.305+0000 7f6d89600700 1 -- 172.21.15.46:0/61972555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c011a00 con 0x7f6d8c106780 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 -- 172.21.15.46:0/61972555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 msgr2=0x7f6d8c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 --2- 172.21.15.46:0/61972555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 -- 172.21.15.46:0/61972555 shutdown_connections 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 --2- 172.21.15.46:0/61972555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c106ba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 -- 172.21.15.46:0/61972555 >> 172.21.15.46:0/61972555 conn(0x7f6d8c101d20 msgr2=0x7f6d8c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 -- 172.21.15.46:0/61972555 shutdown_connections 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.306+0000 7f6d91800700 1 -- 172.21.15.46:0/61972555 wait complete. 2024-05-08T22:55:38.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.307+0000 7f6d91800700 1 Processor -- start 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.307+0000 7f6d91800700 1 -- start start 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.307+0000 7f6d91800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.307+0000 7f6d91800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19cba0 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48102/0 (socket says 172.21.15.46:48102) 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d8aa00700 1 -- 172.21.15.46:0/1786644552 learned_addr learned my addr 172.21.15.46:0/1786644552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d8aa00700 1 -- 172.21.15.46:0/1786644552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c009550 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d8aa00700 1 --2- 172.21.15.46:0/1786644552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009130 tx=0x7f6d7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.308+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c003ab0 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c19cda0 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d7c003c10 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c0247d0 con 0x7f6d8c106780 2024-05-08T22:55:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c19d240 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6d7c024d50 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.309+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d7c0193c0 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.310+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c068010 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.313+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f6d7c01abb0 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.391+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6d8c10ab60 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.391+0000 7f6d83400700 1 -- 172.21.15.46:0/1786644552 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6d7c035090 con 0x7f6d8c106780 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 msgr2=0x7f6d8c19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 --2- 172.21.15.46:0/1786644552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009130 tx=0x7f6d7c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 shutdown_connections 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 --2- 172.21.15.46:0/1786644552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8c106780 0x7f6d8c19c660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 >> 172.21.15.46:0/1786644552 conn(0x7f6d8c101d20 msgr2=0x7f6d8c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 shutdown_connections 2024-05-08T22:55:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:34.393+0000 7f6d91800700 1 -- 172.21.15.46:0/1786644552 wait complete. 2024-05-08T22:55:38.191 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-05-08T22:55:39.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:38 smithi046 ceph-mon[36898]: mgrmap e4: x(active, since 2s) 2024-05-08T22:55:41.221 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:41.221 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:55:41.222 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T22:55:41.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:40 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/808794783' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:55:41.224 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:55:41.224 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:55:41.224 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:55:41.224 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:55:41.224 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:55:41.225 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:41.226 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:55:41.226 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:55:41.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:55:41.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:41.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:55:20.135373+0000", 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.585+0000 7fca04e00700 1 Processor -- start 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.585+0000 7fca04e00700 1 -- start start 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.585+0000 7fca04e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.585+0000 7fca04e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0007a910 con 0x7fca0007be90 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.585+0000 7fc9fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:41.238 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48178/0 (socket says 172.21.15.46:48178) 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fde00700 1 -- 172.21.15.46:0/1355347900 learned_addr learned my addr 172.21.15.46:0/1355347900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fde00700 1 -- 172.21.15.46:0/1355347900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0007aa50 con 0x7fca0007be90 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fde00700 1 --2- 172.21.15.46:0/1355347900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00098a0 tx=0x7fc9f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecb3d9227459b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fca00700 1 -- 172.21.15.46:0/1355347900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f0004160 con 0x7fca0007be90 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fca00700 1 -- 172.21.15.46:0/1355347900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9f00042c0 con 0x7fca0007be90 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.586+0000 7fc9fca00700 1 -- 172.21.15.46:0/1355347900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f0011a00 con 0x7fca0007be90 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 -- 172.21.15.46:0/1355347900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 msgr2=0x7fca0007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 --2- 172.21.15.46:0/1355347900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00098a0 tx=0x7fc9f0009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 -- 172.21.15.46:0/1355347900 shutdown_connections 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 --2- 172.21.15.46:0/1355347900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca0007a340 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 -- 172.21.15.46:0/1355347900 >> 172.21.15.46:0/1355347900 conn(0x7fca00101d20 msgr2=0x7fca00104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 -- 172.21.15.46:0/1355347900 shutdown_connections 2024-05-08T22:55:41.239 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.587+0000 7fca04e00700 1 -- 172.21.15.46:0/1355347900 wait complete. 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fca04e00700 1 Processor -- start 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fca04e00700 1 -- start start 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fca04e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fca04e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca001a0fb0 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fc9fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fc9fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48184/0 (socket says 172.21.15.46:48184) 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.589+0000 7fc9fde00700 1 -- 172.21.15.46:0/808794783 learned_addr learned my addr 172.21.15.46:0/808794783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fc9fde00700 1 -- 172.21.15.46:0/808794783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0009550 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fc9fde00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011710 tx=0x7fc9f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f00039e0 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9f0003b40 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca001a11b0 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.590+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f0019690 con 0x7fca0007be90 2024-05-08T22:55:41.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.591+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca001a1650 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.592+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fc9f001aa60 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.592+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0019a220 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.592+0000 7fc9f6a00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 0x7fc9e403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.592+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc9f004c770 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.592+0000 7fc9fd400700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 0x7fc9e403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.593+0000 7fc9fd400700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 0x7fc9e403abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006fd0 tx=0x7fc9ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.598+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9f002db40 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.896+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fca0002cfa0 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.896+0000 7fc9f6a00700 1 -- 172.21.15.46:0/808794783 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc9f002db40 con 0x7fca0007be90 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 msgr2=0x7fc9e403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 0x7fc9e403abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006fd0 tx=0x7fc9ec006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 msgr2=0x7fca001a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011710 tx=0x7fc9f0011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:41.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 shutdown_connections 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.898+0000 7fca04e00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fc9e4038700 0x7fc9e403abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.899+0000 7fca04e00700 1 --2- 172.21.15.46:0/808794783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca0007be90 0x7fca001a0a70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.899+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 >> 172.21.15.46:0/808794783 conn(0x7fca00101d20 msgr2=0x7fca00104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.899+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 shutdown_connections 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:40.899+0000 7fca04e00700 1 -- 172.21.15.46:0/808794783 wait complete. 2024-05-08T22:55:41.242 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-05-08T22:55:42.179 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:42.180 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T22:55:42.180 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:42.180 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T22:55:42.180 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T22:55:42.180 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:41 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/699095630' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-08T22:55:42.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.627+0000 7fd23ec00700 1 Processor -- start 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.627+0000 7fd23ec00700 1 -- start start 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.627+0000 7fd23ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.627+0000 7fd23ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23807a8f0 con 0x7fd23807be70 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd23d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd23d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48196/0 (socket says 172.21.15.46:48196) 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd23d800700 1 -- 172.21.15.46:0/1487273722 learned_addr learned my addr 172.21.15.46:0/1487273722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:42.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd23d800700 1 -- 172.21.15.46:0/1487273722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23807aa30 con 0x7fd23807be70 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd23d800700 1 --2- 172.21.15.46:0/1487273722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0098a0 tx=0x7fd22c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123f37a79fc12149 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.628+0000 7fd237e00700 1 -- 172.21.15.46:0/1487273722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd22c004160 con 0x7fd23807be70 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd237e00700 1 -- 172.21.15.46:0/1487273722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd22c0042c0 con 0x7fd23807be70 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd237e00700 1 -- 172.21.15.46:0/1487273722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd22c011a00 con 0x7fd23807be70 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 -- 172.21.15.46:0/1487273722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 msgr2=0x7fd23807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 --2- 172.21.15.46:0/1487273722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0098a0 tx=0x7fd22c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 -- 172.21.15.46:0/1487273722 shutdown_connections 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 --2- 172.21.15.46:0/1487273722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23807a320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 -- 172.21.15.46:0/1487273722 >> 172.21.15.46:0/1487273722 conn(0x7fd238101bc0 msgr2=0x7fd238103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 -- 172.21.15.46:0/1487273722 shutdown_connections 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.629+0000 7fd23ec00700 1 -- 172.21.15.46:0/1487273722 wait complete. 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.630+0000 7fd23ec00700 1 Processor -- start 2024-05-08T22:55:42.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23ec00700 1 -- start start 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23819ca30 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48208/0 (socket says 172.21.15.46:48208) 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23d800700 1 -- 172.21.15.46:0/699095630 learned_addr learned my addr 172.21.15.46:0/699095630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.631+0000 7fd23d800700 1 -- 172.21.15.46:0/699095630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009550 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd23d800700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd22c011a20 tx=0x7fd22c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd22c003af0 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd22c003c50 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23819cc30 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd22c019760 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.632+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23819d0d0 con 0x7fd23807be70 2024-05-08T22:55:42.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.633+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fd22c01ab20 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.633+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23810c950 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.634+0000 7fd236000700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 0x7fd22003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.634+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd22c04c840 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.634+0000 7fd23ce00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 0x7fd22003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.634+0000 7fd23ce00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 0x7fd22003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.639+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd22c02dbd0 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.850+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd23810cb80 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.851+0000 7fd236000700 1 -- 172.21.15.46:0/699095630 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd22c02d7c0 con 0x7fd23807be70 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 msgr2=0x7fd22003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 0x7fd22003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 msgr2=0x7fd23819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd22c011a20 tx=0x7fd22c011b00 comp rx=0 tx=0).stop 2024-05-08T22:55:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 shutdown_connections 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fd220038700 0x7fd22003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 --2- 172.21.15.46:0/699095630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd23807be70 0x7fd23819c4f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.853+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 >> 172.21.15.46:0/699095630 conn(0x7fd238101bc0 msgr2=0x7fd238102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.854+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 shutdown_connections 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:41.854+0000 7fd23ec00700 1 -- 172.21.15.46:0/699095630 wait complete. 2024-05-08T22:55:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-05-08T22:55:43.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:42 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/827289172' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T22:55:43.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setuser ceph since I am not root 2024-05-08T22:55:43.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setgroup ceph since I am not root 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.612+0000 7fe161c00700 1 Processor -- start 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe161c00700 1 -- start start 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe161c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe161c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c107180 con 0x7fe15c106790 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47976/0 (socket says 172.21.15.46:47976) 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.613+0000 7fe15aa00700 1 -- 172.21.15.46:0/1119659628 learned_addr learned my addr 172.21.15.46:0/1119659628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:43.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.614+0000 7fe15aa00700 1 -- 172.21.15.46:0/1119659628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c107940 con 0x7fe15c106790 2024-05-08T22:55:43.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.614+0000 7fe15aa00700 1 --2- 172.21.15.46:0/1119659628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0098a0 tx=0x7fe14c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c74ac52cde95c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:43.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.614+0000 7fe159600700 1 -- 172.21.15.46:0/1119659628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe14c004160 con 0x7fe15c106790 2024-05-08T22:55:43.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.614+0000 7fe159600700 1 -- 172.21.15.46:0/1119659628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe14c0042c0 con 0x7fe15c106790 2024-05-08T22:55:43.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.614+0000 7fe159600700 1 -- 172.21.15.46:0/1119659628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe14c011a00 con 0x7fe15c106790 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 -- 172.21.15.46:0/1119659628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 msgr2=0x7fe15c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 --2- 172.21.15.46:0/1119659628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0098a0 tx=0x7fe14c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 -- 172.21.15.46:0/1119659628 shutdown_connections 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 --2- 172.21.15.46:0/1119659628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c106bb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 -- 172.21.15.46:0/1119659628 >> 172.21.15.46:0/1119659628 conn(0x7fe15c101d50 msgr2=0x7fe15c104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 -- 172.21.15.46:0/1119659628 shutdown_connections 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.615+0000 7fe161c00700 1 -- 172.21.15.46:0/1119659628 wait complete. 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.616+0000 7fe161c00700 1 Processor -- start 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.616+0000 7fe161c00700 1 -- start start 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.616+0000 7fe161c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:43.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.616+0000 7fe161c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19cd00 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47982/0 (socket says 172.21.15.46:47982) 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe15aa00700 1 -- 172.21.15.46:0/827289172 learned_addr learned my addr 172.21.15.46:0/827289172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe15aa00700 1 -- 172.21.15.46:0/827289172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c009550 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe15aa00700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011710 tx=0x7fe14c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.617+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe14c003aa0 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.618+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe14c003c00 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.618+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c19cf00 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.618+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe14c024800 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.618+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c19d3a0 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.619+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fe14c01a430 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.619+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c10ab70 con 0x7fe15c106790 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.619+0000 7fe153400700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 0x7fe14003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:43.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.619+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe14c04c3b0 con 0x7fe15c106790 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.620+0000 7fe15a000700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 0x7fe14003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.620+0000 7fe15a000700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 0x7fe14003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe148006fd0 tx=0x7fe148006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.624+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe14c024e30 con 0x7fe15c106790 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.896+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe15c19fbe0 con 0x7fe15c106790 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.952+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fe14c01abd0 con 0x7fe15c106790 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.954+0000 7fe153400700 1 -- 172.21.15.46:0/827289172 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fe14c04c110 con 0x7fe15c106790 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 msgr2=0x7fe14003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 0x7fe14003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe148006fd0 tx=0x7fe148006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 msgr2=0x7fe15c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011710 tx=0x7fe14c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 shutdown_connections 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7fe140038700 0x7fe14003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 --2- 172.21.15.46:0/827289172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c19c7c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.957+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 >> 172.21.15.46:0/827289172 conn(0x7fe15c101d50 msgr2=0x7fe15c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:43.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.958+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 shutdown_connections 2024-05-08T22:55:43.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:42.958+0000 7fe161c00700 1 -- 172.21.15.46:0/827289172 wait complete. 2024-05-08T22:55:43.834 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:43.584+0000 7f48221fd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:55:43.834 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:43.711+0000 7f48221fd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:55:44.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:43 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/827289172' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T22:55:44.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:43 smithi046 ceph-mon[36898]: mgrmap e5: x(active, since 7s) 2024-05-08T22:55:44.179 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:43.834+0000 7f48221fd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:55:44.179 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:43.949+0000 7f48221fd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:55:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:55:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T22:55:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:55:44.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.712+0000 7f2b84600700 1 Processor -- start 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.713+0000 7f2b84600700 1 -- start start 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.713+0000 7f2b84600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.713+0000 7f2b84600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c107170 con 0x7f2b7c106780 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.713+0000 7f2b81c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b81c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47992/0 (socket says 172.21.15.46:47992) 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b81c00700 1 -- 172.21.15.46:0/3454932704 learned_addr learned my addr 172.21.15.46:0/3454932704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b81c00700 1 -- 172.21.15.46:0/3454932704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c107930 con 0x7f2b7c106780 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b81c00700 1 --2- 172.21.15.46:0/3454932704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009e80 tx=0x7f2b7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bacdf157648dbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b7be00700 1 -- 172.21.15.46:0/3454932704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b7000e070 con 0x7f2b7c106780 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b7be00700 1 -- 172.21.15.46:0/3454932704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b7000b8c0 con 0x7f2b7c106780 2024-05-08T22:55:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.714+0000 7f2b7be00700 1 -- 172.21.15.46:0/3454932704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b700044c0 con 0x7f2b7c106780 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 -- 172.21.15.46:0/3454932704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 msgr2=0x7f2b7c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 --2- 172.21.15.46:0/3454932704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009e80 tx=0x7f2b7000b330 comp rx=0 tx=0).stop 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 -- 172.21.15.46:0/3454932704 shutdown_connections 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 --2- 172.21.15.46:0/3454932704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c106ba0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 -- 172.21.15.46:0/3454932704 >> 172.21.15.46:0/3454932704 conn(0x7f2b7c101d20 msgr2=0x7f2b7c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 -- 172.21.15.46:0/3454932704 shutdown_connections 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.715+0000 7f2b84600700 1 -- 172.21.15.46:0/3454932704 wait complete. 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.717+0000 7f2b84600700 1 Processor -- start 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.717+0000 7f2b84600700 1 -- start start 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b84600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b84600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c19cb80 con 0x7f2b7c106780 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b81c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b81c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47998/0 (socket says 172.21.15.46:47998) 2024-05-08T22:55:44.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b81c00700 1 -- 172.21.15.46:0/1917483582 learned_addr learned my addr 172.21.15.46:0/1917483582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b81c00700 1 -- 172.21.15.46:0/1917483582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009b30 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.718+0000 7f2b81c00700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009fd0 tx=0x7f2b700045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.719+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b7000e040 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.719+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b7000bdf0 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.719+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c19cd80 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.719+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b7001c7a0 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.719+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c19d280 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.720+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f2b70012430 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.720+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c10ab60 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.721+0000 7f2b7a000700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 0x7f2b6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.721+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2b7004c5a0 con 0x7f2b7c106780 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.721+0000 7f2b81200700 1 -- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 msgr2=0x7f2b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.721+0000 7f2b81200700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 0x7f2b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:55:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.726+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2b70024ba0 con 0x7f2b7c106780 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.922+0000 7f2b81200700 1 -- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 msgr2=0x7f2b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:43.922+0000 7f2b81200700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 0x7f2b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.000+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2b7c19fab0 con 0x7f2b7c106780 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.000+0000 7f2b7a000700 1 -- 172.21.15.46:0/1917483582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f2b70020070 con 0x7f2b7c106780 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 msgr2=0x7f2b6403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 0x7f2b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 msgr2=0x7f2b7c19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009fd0 tx=0x7f2b700045f0 comp rx=0 tx=0).stop 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 shutdown_connections 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f2b64038750 0x7f2b6403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 --2- 172.21.15.46:0/1917483582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b7c106780 0x7f2b7c19c640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 >> 172.21.15.46:0/1917483582 conn(0x7f2b7c101d20 msgr2=0x7f2b7c1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 shutdown_connections 2024-05-08T22:55:44.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.003+0000 7f2b84600700 1 -- 172.21.15.46:0/1917483582 wait complete. 2024-05-08T22:55:44.337 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-08T22:55:44.337 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-05-08T22:55:45.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:44 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1917483582' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:55:45.181 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:44.853+0000 7f48221fd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:55:45.450 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.181+0000 7f48221fd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:55:45.451 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.301+0000 7f48221fd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:55:45.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.451+0000 7f48221fd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:55:45.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.654+0000 7f48221fd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:55:46.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.833+0000 7f48221fd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:55:46.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:45.951+0000 7f48221fd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:55:46.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:46.440+0000 7f48221fd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:55:47.231 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:46.966+0000 7f48221fd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:55:47.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:47.231+0000 7f48221fd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:55:47.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:47.360+0000 7f48221fd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:55:48.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:48.383+0000 7f48221fd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:55:48.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:48.516+0000 7f48221fd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:55:49.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:48.722+0000 7f48221fd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:55:49.853 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:49.600+0000 7f48221fd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:55:50.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:49.853+0000 7f48221fd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:55:50.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:50.126+0000 7f48221fd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:55:51.718 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:51.324+0000 7f48221fd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:55:52.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:51.718+0000 7f48221fd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:55:52.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:51.845+0000 7f48221fd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:55:52.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:55:52.113+0000 7f48221fd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:55:52.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: Active manager daemon x restarted 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: Activating manager daemon x 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: mgrmap e6: x(active, starting, since 0.00378856s) 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: Manager daemon x is now available 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:55:52.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:52 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:55:53.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.771+0000 7f5d7d200700 1 Processor -- start 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.771+0000 7f5d7d200700 1 -- start start 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d7d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d7d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7807a8f0 con 0x7f5d7807be70 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48002/0 (socket says 172.21.15.46:48002) 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d75e00700 1 -- 172.21.15.46:0/1419885767 learned_addr learned my addr 172.21.15.46:0/1419885767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:53.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.772+0000 7f5d75e00700 1 -- 172.21.15.46:0/1419885767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7807aa30 con 0x7f5d7807be70 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.773+0000 7f5d75e00700 1 --2- 172.21.15.46:0/1419885767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d680098a0 tx=0x7f5d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad0f1548be76074 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.773+0000 7f5d74a00700 1 -- 172.21.15.46:0/1419885767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68004160 con 0x7f5d7807be70 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.773+0000 7f5d74a00700 1 -- 172.21.15.46:0/1419885767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d680042c0 con 0x7f5d7807be70 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.773+0000 7f5d74a00700 1 -- 172.21.15.46:0/1419885767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68011a00 con 0x7f5d7807be70 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 -- 172.21.15.46:0/1419885767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 msgr2=0x7f5d7807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1419885767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d680098a0 tx=0x7f5d68009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 -- 172.21.15.46:0/1419885767 shutdown_connections 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1419885767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7807a320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 -- 172.21.15.46:0/1419885767 >> 172.21.15.46:0/1419885767 conn(0x7f5d78101d20 msgr2=0x7f5d78104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 -- 172.21.15.46:0/1419885767 shutdown_connections 2024-05-08T22:55:53.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.774+0000 7f5d7d200700 1 -- 172.21.15.46:0/1419885767 wait complete. 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.775+0000 7f5d7d200700 1 Processor -- start 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.775+0000 7f5d7d200700 1 -- start start 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.775+0000 7f5d7d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.775+0000 7f5d7d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819cb70 con 0x7f5d7807be70 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.776+0000 7f5d75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.776+0000 7f5d75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48016/0 (socket says 172.21.15.46:48016) 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.776+0000 7f5d75e00700 1 -- 172.21.15.46:0/1976855221 learned_addr learned my addr 172.21.15.46:0/1976855221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.776+0000 7f5d75e00700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68009550 con 0x7f5d7807be70 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.776+0000 7f5d75e00700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009130 tx=0x7f5d68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:53.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.777+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68003b70 con 0x7f5d7807be70 2024-05-08T22:55:53.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.777+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d68003cd0 con 0x7f5d7807be70 2024-05-08T22:55:53.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.777+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d680197d0 con 0x7f5d7807be70 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.777+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7819cd70 con 0x7f5d7807be70 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.777+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7819d210 con 0x7f5d7807be70 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.778+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5d680199c0 con 0x7f5d7807be70 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.778+0000 7f5d6ea00700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.778+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d5c03b370 con 0x7f5d5c0387a0 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.778+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5d6804ca40 con 0x7f5d7807be70 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.779+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.779+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.980+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:44.980+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:55:53.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:45.381+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:45.381+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:46.182+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:46.182+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:47.783+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:47.783+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:50.987+0000 7f5d75400700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/664114694 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:50.987+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:52.118+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f5d68019cc0 con 0x7f5d7807be70 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:52.118+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 msgr2=0x7f5d5c03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:52.118+0000 7f5d6ea00700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.119+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f5d6804e220 con 0x7f5d7807be70 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.119+0000 7f5d6ea00700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 0x7f5d5c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.119+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d5c03b370 con 0x7f5d5c03bfe0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.119+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 0x7f5d5c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.121+0000 7f5d75400700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 0x7f5d5c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d64003a10 tx=0x7f5d640092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.122+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f5d5c03b370 con 0x7f5d5c03bfe0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.129+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5d7810cad0 con 0x7f5d5c03bfe0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d6ea00700 1 -- 172.21.15.46:0/1976855221 <== mgr.14118 v2:172.21.15.46:6800/2056989721 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5d7810cad0 con 0x7f5d5c03bfe0 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 msgr2=0x7f5d5c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 0x7f5d5c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d64003a10 tx=0x7f5d640092b0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: Found migration_current of "None". Setting to last migration. 2024-05-08T22:55:53.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:55:53.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:55:53.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:53.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:53.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:53 smithi046 ceph-mon[36898]: mgrmap e7: x(active, since 1.00582s) 2024-05-08T22:55:53.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 msgr2=0x7f5d7819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:53.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009130 tx=0x7f5d68011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 shutdown_connections 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d7807be70 0x7f5d7819c630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f5d5c03bfe0 0x7f5d5c03e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 --2- 172.21.15.46:0/1976855221 >> [v2:172.21.15.46:6800/664114694,v1:172.21.15.46:6801/664114694] conn(0x7f5d5c0387a0 0x7f5d5c03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 >> 172.21.15.46:0/1976855221 conn(0x7f5d78101d20 msgr2=0x7f5d78102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 shutdown_connections 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.130+0000 7f5d7d200700 1 -- 172.21.15.46:0/1976855221 wait complete. 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-05-08T22:55:53.512 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-05-08T22:55:54.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.919+0000 7f3313a00700 1 Processor -- start 2024-05-08T22:55:54.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.919+0000 7f3313a00700 1 -- start start 2024-05-08T22:55:54.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.920+0000 7f3313a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:54.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:54 smithi046 ceph-mon[36898]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:55:54.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:54 smithi046 ceph-mon[36898]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.920+0000 7f3313a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c107180 con 0x7f330c106790 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.920+0000 7f3311000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.920+0000 7f3311000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58238/0 (socket says 172.21.15.46:58238) 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.920+0000 7f3311000700 1 -- 172.21.15.46:0/2294714745 learned_addr learned my addr 172.21.15.46:0/2294714745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.921+0000 7f3311000700 1 -- 172.21.15.46:0/2294714745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c107940 con 0x7f330c106790 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.921+0000 7f3311000700 1 --2- 172.21.15.46:0/2294714745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f33000098a0 tx=0x7f3300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab45d6f13f303189 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.921+0000 7f330b400700 1 -- 172.21.15.46:0/2294714745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3300004160 con 0x7f330c106790 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.921+0000 7f330b400700 1 -- 172.21.15.46:0/2294714745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33000042c0 con 0x7f330c106790 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.921+0000 7f330b400700 1 -- 172.21.15.46:0/2294714745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3300011a00 con 0x7f330c106790 2024-05-08T22:55:54.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 -- 172.21.15.46:0/2294714745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 msgr2=0x7f330c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 --2- 172.21.15.46:0/2294714745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f33000098a0 tx=0x7f3300009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 -- 172.21.15.46:0/2294714745 shutdown_connections 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 --2- 172.21.15.46:0/2294714745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c106bb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 -- 172.21.15.46:0/2294714745 >> 172.21.15.46:0/2294714745 conn(0x7f330c101d50 msgr2=0x7f330c104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 -- 172.21.15.46:0/2294714745 shutdown_connections 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.922+0000 7f3313a00700 1 -- 172.21.15.46:0/2294714745 wait complete. 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.923+0000 7f3313a00700 1 Processor -- start 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3313a00700 1 -- start start 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3313a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3313a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c19cd00 con 0x7f330c106790 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3311000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3311000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58254/0 (socket says 172.21.15.46:58254) 2024-05-08T22:55:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.924+0000 7f3311000700 1 -- 172.21.15.46:0/2044793692 learned_addr learned my addr 172.21.15.46:0/2044793692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3311000700 1 -- 172.21.15.46:0/2044793692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300009550 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3311000700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3300011710 tx=0x7f3300011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3300003aa0 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3300003c00 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c19cf00 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3300024800 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.925+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c19d3a0 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.927+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f330001a430 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.927+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c10ab70 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.927+0000 7f3309600700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 0x7f32f403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.927+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f330004c440 con 0x7f330c106790 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.928+0000 7f330be00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 0x7f32f403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.928+0000 7f330be00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 0x7f32f403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32f8006fd0 tx=0x7f32f8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:54.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:53.933+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f330002cb40 con 0x7f330c106790 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.164+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f330c1030b0 con 0x7f32f4038620 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.174+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3300024960 con 0x7f330c106790 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.174+0000 7f3309600700 1 -- 172.21.15.46:0/2044793692 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f330c1030b0 con 0x7f32f4038620 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.177+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 msgr2=0x7f32f403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.177+0000 7f3313a00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 0x7f32f403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32f8006fd0 tx=0x7f32f8006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.177+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 msgr2=0x7f330c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.177+0000 7f3313a00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3300011710 tx=0x7f3300011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 shutdown_connections 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f32f4038620 0x7f32f403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 --2- 172.21.15.46:0/2044793692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330c106790 0x7f330c19c7c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 >> 172.21.15.46:0/2044793692 conn(0x7f330c101d50 msgr2=0x7f330c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 shutdown_connections 2024-05-08T22:55:54.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.178+0000 7f3313a00700 1 -- 172.21.15.46:0/2044793692 wait complete. 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: mgrmap e8: x(active, since 2s) 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: [08/May/2024:22:55:54] ENGINE Bus STARTING 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: [08/May/2024:22:55:54] ENGINE Serving on http://172.21.15.46:8765 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: [08/May/2024:22:55:54] ENGINE Serving on https://172.21.15.46:7150 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: [08/May/2024:22:55:54] ENGINE Bus STARTED 2024-05-08T22:55:55.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:55 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:55:55.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.877+0000 7fb43cc00700 1 Processor -- start 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.877+0000 7fb43cc00700 1 -- start start 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb43cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb43cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43807a820 con 0x7fb43807be30 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb437400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb437400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58268/0 (socket says 172.21.15.46:58268) 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb437400700 1 -- 172.21.15.46:0/1971147360 learned_addr learned my addr 172.21.15.46:0/1971147360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.878+0000 7fb437400700 1 -- 172.21.15.46:0/1971147360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43807a960 con 0x7fb43807be30 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb437400700 1 --2- 172.21.15.46:0/1971147360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4280098a0 tx=0x7fb428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e63fb79b2aa1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb436000700 1 -- 172.21.15.46:0/1971147360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb428004160 con 0x7fb43807be30 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb436000700 1 -- 172.21.15.46:0/1971147360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4280042c0 con 0x7fb43807be30 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb436000700 1 -- 172.21.15.46:0/1971147360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb428011a00 con 0x7fb43807be30 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb43cc00700 1 -- 172.21.15.46:0/1971147360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 msgr2=0x7fb43807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.879+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1971147360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4280098a0 tx=0x7fb428009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.880+0000 7fb43cc00700 1 -- 172.21.15.46:0/1971147360 shutdown_connections 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.880+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1971147360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43807a2e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.880+0000 7fb43cc00700 1 -- 172.21.15.46:0/1971147360 >> 172.21.15.46:0/1971147360 conn(0x7fb438101bc0 msgr2=0x7fb438103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.880+0000 7fb43cc00700 1 -- 172.21.15.46:0/1971147360 shutdown_connections 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.880+0000 7fb43cc00700 1 -- 172.21.15.46:0/1971147360 wait complete. 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb43cc00700 1 Processor -- start 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb43cc00700 1 -- start start 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb43cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb43cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43819ca30 con 0x7fb43807be30 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb437400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb437400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58276/0 (socket says 172.21.15.46:58276) 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.881+0000 7fb437400700 1 -- 172.21.15.46:0/1149931346 learned_addr learned my addr 172.21.15.46:0/1149931346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:55.447 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb437400700 1 -- 172.21.15.46:0/1149931346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb428009550 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb437400700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb428011a20 tx=0x7fb428011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb428003af0 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb428003c50 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb428019760 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43819cc30 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.882+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43819d0d0 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.883+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43810c930 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.883+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb42801ab20 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.884+0000 7fb42fe00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 0x7fb41c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.884+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb42804c9b0 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.884+0000 7fb436a00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 0x7fb41c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.885+0000 7fb436a00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 0x7fb41c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:54.890+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb42801a7b0 con 0x7fb43807be30 2024-05-08T22:55:55.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.104+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb438102e60 con 0x7fb41c038750 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.105+0000 7fb42fe00700 1 -- 172.21.15.46:0/1149931346 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb438102e60 con 0x7fb41c038750 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 msgr2=0x7fb41c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 0x7fb41c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 msgr2=0x7fb43819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb428011a20 tx=0x7fb428011b00 comp rx=0 tx=0).stop 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 shutdown_connections 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb41c038750 0x7fb41c03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 --2- 172.21.15.46:0/1149931346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb43807be30 0x7fb43819c4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.107+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 >> 172.21.15.46:0/1149931346 conn(0x7fb438101bc0 msgr2=0x7fb438102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.108+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 shutdown_connections 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.108+0000 7fb43cc00700 1 -- 172.21.15.46:0/1149931346 wait complete. 2024-05-08T22:55:55.449 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-05-08T22:55:56.502 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:56 smithi046 ceph-mon[36898]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: Generating public/private rsa key pair. 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: Your identification has been saved in /tmp/tmpx63fzuu2/key. 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: Your public key has been saved in /tmp/tmpx63fzuu2/key.pub. 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: The key fingerprint is: 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: SHA256:ky268nKi41BR1NOKEd24JABMMWlFtl7CRm7YGSXFLJA ceph-ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: The key's randomart image is: 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: +---[RSA 3072]----+ 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: |+*OXO* + | 2024-05-08T22:55:56.503 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | EOoBo* o | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: |...X.* + | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | +.+ o o | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | .. S . | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | . . o | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: |. . | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | .. + .. | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: | .oo *o | 2024-05-08T22:55:56.504 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:55:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: +----[SHA256]-----+ 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.850+0000 7f8c66600700 1 Processor -- start 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.850+0000 7f8c66600700 1 -- start start 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.851+0000 7f8c66600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.851+0000 7f8c66600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c601093b0 con 0x7f8c601089c0 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.851+0000 7f8c5f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.851+0000 7f8c5f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58280/0 (socket says 172.21.15.46:58280) 2024-05-08T22:55:56.658 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.851+0000 7f8c5f400700 1 -- 172.21.15.46:0/2154110810 learned_addr learned my addr 172.21.15.46:0/2154110810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.852+0000 7f8c5f400700 1 -- 172.21.15.46:0/2154110810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60109b70 con 0x7f8c601089c0 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.852+0000 7f8c5f400700 1 --2- 172.21.15.46:0/2154110810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8c500098a0 tx=0x7f8c50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1119e6c1b2920ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.852+0000 7f8c5e000700 1 -- 172.21.15.46:0/2154110810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c50004160 con 0x7f8c601089c0 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.852+0000 7f8c5e000700 1 -- 172.21.15.46:0/2154110810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c500042c0 con 0x7f8c601089c0 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.852+0000 7f8c5e000700 1 -- 172.21.15.46:0/2154110810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c50011a00 con 0x7f8c601089c0 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 -- 172.21.15.46:0/2154110810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 msgr2=0x7f8c60108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 --2- 172.21.15.46:0/2154110810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8c500098a0 tx=0x7f8c50009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 -- 172.21.15.46:0/2154110810 shutdown_connections 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 --2- 172.21.15.46:0/2154110810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c60108de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 -- 172.21.15.46:0/2154110810 >> 172.21.15.46:0/2154110810 conn(0x7f8c6007bde0 msgr2=0x7f8c601064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 -- 172.21.15.46:0/2154110810 shutdown_connections 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.853+0000 7f8c66600700 1 -- 172.21.15.46:0/2154110810 wait complete. 2024-05-08T22:55:56.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c66600700 1 Processor -- start 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c66600700 1 -- start start 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c66600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c66600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019cc00 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c5f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c5f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58296/0 (socket says 172.21.15.46:58296) 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.854+0000 7f8c5f400700 1 -- 172.21.15.46:0/740058840 learned_addr learned my addr 172.21.15.46:0/740058840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c5f400700 1 -- 172.21.15.46:0/740058840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50009550 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c5f400700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009fb0 tx=0x7f8c50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c50003a40 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c50003ba0 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c5001a750 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6019ce00 con 0x7f8c601089c0 2024-05-08T22:55:56.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.855+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6019d2a0 con 0x7f8c601089c0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.856+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6010cda0 con 0x7f8c601089c0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.857+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8c50024b60 con 0x7f8c601089c0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.857+0000 7f8c57e00700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 0x7f8c4403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.857+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8c5004c710 con 0x7f8c601089c0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.857+0000 7f8c5ea00700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 0x7f8c4403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.858+0000 7f8c5ea00700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 0x7f8c4403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:55.863+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8c50024430 con 0x7f8c601089c0 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.077+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c6019fb40 con 0x7f8c44038700 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.308+0000 7f8c57e00700 1 -- 172.21.15.46:0/740058840 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8c6019fb40 con 0x7f8c44038700 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 msgr2=0x7f8c4403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 0x7f8c4403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 msgr2=0x7f8c6019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:56.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009fb0 tx=0x7f8c50011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 shutdown_connections 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f8c44038700 0x7f8c4403abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 --2- 172.21.15.46:0/740058840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c601089c0 0x7f8c6019c6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 >> 172.21.15.46:0/740058840 conn(0x7f8c6007bde0 msgr2=0x7f8c60105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 shutdown_connections 2024-05-08T22:55:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:56.311+0000 7f8c66600700 1 -- 172.21.15.46:0/740058840 wait complete. 2024-05-08T22:55:57.611 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:55:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:57 smithi046 ceph-mon[36898]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:55:57.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:57 smithi046 ceph-mon[36898]: Generating ssh key... 2024-05-08T22:55:57.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:57 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:57.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:57 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:55:57.619 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.068+0000 7f0e52400700 1 Processor -- start 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e52400700 1 -- start start 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e52400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e52400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c107170 con 0x7f0e4c106780 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e4b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e4b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58304/0 (socket says 172.21.15.46:58304) 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.069+0000 7f0e4b400700 1 -- 172.21.15.46:0/3064389625 learned_addr learned my addr 172.21.15.46:0/3064389625 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.070+0000 7f0e4b400700 1 -- 172.21.15.46:0/3064389625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c107930 con 0x7f0e4c106780 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.070+0000 7f0e4b400700 1 --2- 172.21.15.46:0/3064389625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098a0 tx=0x7f0e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b80b0082d484e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.070+0000 7f0e4aa00700 1 -- 172.21.15.46:0/3064389625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c004160 con 0x7f0e4c106780 2024-05-08T22:55:57.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.070+0000 7f0e4aa00700 1 -- 172.21.15.46:0/3064389625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e3c0042c0 con 0x7f0e4c106780 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.070+0000 7f0e4aa00700 1 -- 172.21.15.46:0/3064389625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c011a00 con 0x7f0e4c106780 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 -- 172.21.15.46:0/3064389625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 msgr2=0x7f0e4c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 --2- 172.21.15.46:0/3064389625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098a0 tx=0x7f0e3c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 -- 172.21.15.46:0/3064389625 shutdown_connections 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 --2- 172.21.15.46:0/3064389625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c106ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 -- 172.21.15.46:0/3064389625 >> 172.21.15.46:0/3064389625 conn(0x7f0e4c101d20 msgr2=0x7f0e4c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 -- 172.21.15.46:0/3064389625 shutdown_connections 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.071+0000 7f0e52400700 1 -- 172.21.15.46:0/3064389625 wait complete. 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.072+0000 7f0e52400700 1 Processor -- start 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.072+0000 7f0e52400700 1 -- start start 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e52400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:57.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e52400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c07c060 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e4b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e4b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58318/0 (socket says 172.21.15.46:58318) 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e4b400700 1 -- 172.21.15.46:0/753267555 learned_addr learned my addr 172.21.15.46:0/753267555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e4b400700 1 -- 172.21.15.46:0/753267555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009550 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.073+0000 7f0e4b400700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009130 tx=0x7f0e3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.074+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c003ab0 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.074+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e3c003c10 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.074+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c07a2a0 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.074+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c0247d0 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.074+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c07a740 con 0x7f0e4c106780 2024-05-08T22:55:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.075+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0e3c01a430 con 0x7f0e4c106780 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.075+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c068010 con 0x7f0e4c106780 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.075+0000 7f0e48c00700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 0x7f0e3003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.075+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e3c04c7f0 con 0x7f0e4c106780 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.076+0000 7f0e43e00700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 0x7f0e3003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.076+0000 7f0e43e00700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 0x7f0e3003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.080+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0e3c018530 con 0x7f0e4c106780 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.297+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e4c07ad80 con 0x7f0e30038700 2024-05-08T22:55:57.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.298+0000 7f0e48c00700 1 -- 172.21.15.46:0/753267555 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0e4c07ad80 con 0x7f0e30038700 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.300+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 msgr2=0x7f0e3003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.300+0000 7f0e52400700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 0x7f0e3003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).stop 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.300+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 msgr2=0x7f0e4c07bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.300+0000 7f0e52400700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009130 tx=0x7f0e3c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 shutdown_connections 2024-05-08T22:55:57.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f0e30038700 0x7f0e3003abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 --2- 172.21.15.46:0/753267555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e4c106780 0x7f0e4c07bb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 >> 172.21.15.46:0/753267555 conn(0x7f0e4c101d20 msgr2=0x7f0e4c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 shutdown_connections 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:57.301+0000 7f0e52400700 1 -- 172.21.15.46:0/753267555 wait complete. 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T22:55:57.625 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-05-08T22:55:58.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:58 smithi046 ceph-mon[36898]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:55:59.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:55:59 smithi046 ceph-mon[36898]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:00.671 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:00 smithi046 ceph-mon[36898]: Deploying cephadm binary to smithi046 2024-05-08T22:56:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-05-08T22:56:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb4a200700 1 Processor -- start 2024-05-08T22:56:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb4a200700 1 -- start start 2024-05-08T22:56:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb4a200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb4a200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44107180 con 0x7feb44106790 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb43400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb43400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58332/0 (socket says 172.21.15.46:58332) 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.071+0000 7feb43400700 1 -- 172.21.15.46:0/3265308392 learned_addr learned my addr 172.21.15.46:0/3265308392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.072+0000 7feb43400700 1 -- 172.21.15.46:0/3265308392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44107940 con 0x7feb44106790 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.072+0000 7feb43400700 1 --2- 172.21.15.46:0/3265308392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb340098a0 tx=0x7feb34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d218d70df13dbc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.072+0000 7feb42000700 1 -- 172.21.15.46:0/3265308392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb34004160 con 0x7feb44106790 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.072+0000 7feb42000700 1 -- 172.21.15.46:0/3265308392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb340042c0 con 0x7feb44106790 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.072+0000 7feb42000700 1 -- 172.21.15.46:0/3265308392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb34011a00 con 0x7feb44106790 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 -- 172.21.15.46:0/3265308392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 msgr2=0x7feb44106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 --2- 172.21.15.46:0/3265308392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb340098a0 tx=0x7feb34009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 -- 172.21.15.46:0/3265308392 shutdown_connections 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 --2- 172.21.15.46:0/3265308392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb44106bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 -- 172.21.15.46:0/3265308392 >> 172.21.15.46:0/3265308392 conn(0x7feb44101d50 msgr2=0x7feb44104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 -- 172.21.15.46:0/3265308392 shutdown_connections 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.073+0000 7feb4a200700 1 -- 172.21.15.46:0/3265308392 wait complete. 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.074+0000 7feb4a200700 1 Processor -- start 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.074+0000 7feb4a200700 1 -- start start 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.074+0000 7feb4a200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.074+0000 7feb4a200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4407c060 con 0x7feb44106790 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.075+0000 7feb43400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.075+0000 7feb43400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58342/0 (socket says 172.21.15.46:58342) 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.075+0000 7feb43400700 1 -- 172.21.15.46:0/722606547 learned_addr learned my addr 172.21.15.46:0/722606547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.075+0000 7feb43400700 1 -- 172.21.15.46:0/722606547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34009550 con 0x7feb44106790 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.075+0000 7feb43400700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb34011710 tx=0x7feb34011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.076+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb34003aa0 con 0x7feb44106790 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.076+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb34003c00 con 0x7feb44106790 2024-05-08T22:56:01.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.076+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4407a2a0 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.076+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb34024800 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.076+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4407a740 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.077+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440625d0 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.077+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7feb3401a430 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.077+0000 7feb3be00700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 0x7feb2803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.078+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feb3404c860 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.078+0000 7feb42a00700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 0x7feb2803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.078+0000 7feb42a00700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 0x7feb2803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb30006fd0 tx=0x7feb30006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.083+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb3401a6e0 con 0x7feb44106790 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:55:58.300+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) v1 -- 0x7feb4407ada0 con 0x7feb28038700 2024-05-08T22:56:01.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.353+0000 7feb3be00700 1 -- 172.21.15.46:0/722606547 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7feb4407ada0 con 0x7feb28038700 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 msgr2=0x7feb2803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 0x7feb2803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb30006fd0 tx=0x7feb30006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 msgr2=0x7feb4407bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb34011710 tx=0x7feb34011ad0 comp rx=0 tx=0).stop 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 shutdown_connections 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7feb28038700 0x7feb2803abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 --2- 172.21.15.46:0/722606547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106790 0x7feb4407bb20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 >> 172.21.15.46:0/722606547 conn(0x7feb44101d50 msgr2=0x7feb44102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:01.683 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 shutdown_connections 2024-05-08T22:56:01.684 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:01.357+0000 7feb4a200700 1 -- 172.21.15.46:0/722606547 wait complete. 2024-05-08T22:56:01.684 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-05-08T22:56:02.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:02 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:02.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:02 smithi046 ceph-mon[36898]: Added host smithi046 2024-05-08T22:56:02.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:02 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.153+0000 7fd28be00700 1 Processor -- start 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.153+0000 7fd28be00700 1 -- start start 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd28be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd28be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284107180 con 0x7fd284106790 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49978/0 (socket says 172.21.15.46:49978) 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd289400700 1 -- 172.21.15.46:0/2780764282 learned_addr learned my addr 172.21.15.46:0/2780764282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.154+0000 7fd289400700 1 -- 172.21.15.46:0/2780764282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284107940 con 0x7fd284106790 2024-05-08T22:56:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.155+0000 7fd289400700 1 --2- 172.21.15.46:0/2780764282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5474984813d68a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.155+0000 7fd283e00700 1 -- 172.21.15.46:0/2780764282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd278004160 con 0x7fd284106790 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.155+0000 7fd283e00700 1 -- 172.21.15.46:0/2780764282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd2780042c0 con 0x7fd284106790 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.155+0000 7fd283e00700 1 -- 172.21.15.46:0/2780764282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd278011a00 con 0x7fd284106790 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 -- 172.21.15.46:0/2780764282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 msgr2=0x7fd284106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 --2- 172.21.15.46:0/2780764282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 -- 172.21.15.46:0/2780764282 shutdown_connections 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 --2- 172.21.15.46:0/2780764282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd284106bb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 -- 172.21.15.46:0/2780764282 >> 172.21.15.46:0/2780764282 conn(0x7fd284101d50 msgr2=0x7fd284104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 -- 172.21.15.46:0/2780764282 shutdown_connections 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.156+0000 7fd28be00700 1 -- 172.21.15.46:0/2780764282 wait complete. 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.157+0000 7fd28be00700 1 Processor -- start 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.157+0000 7fd28be00700 1 -- start start 2024-05-08T22:56:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.157+0000 7fd28be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.157+0000 7fd28be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28419cca0 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.158+0000 7fd289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.158+0000 7fd289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49982/0 (socket says 172.21.15.46:49982) 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.158+0000 7fd289400700 1 -- 172.21.15.46:0/2061182694 learned_addr learned my addr 172.21.15.46:0/2061182694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.158+0000 7fd289400700 1 -- 172.21.15.46:0/2061182694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278009550 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.158+0000 7fd289400700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd278011710 tx=0x7fd278011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.159+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd278003aa0 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.159+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd278003c00 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.159+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd278024800 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.159+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28419cea0 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.159+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28419d340 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.160+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fd27801a430 con 0x7fd284106790 2024-05-08T22:56:02.819 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.160+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28410ab70 con 0x7fd284106790 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.160+0000 7fd282000700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 0x7fd26c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.160+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd27804c5e0 con 0x7fd284106790 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.161+0000 7fd288a00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 0x7fd26c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.161+0000 7fd288a00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 0x7fd26c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd270006fd0 tx=0x7fd270006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.166+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd27802c9a0 con 0x7fd284106790 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.411+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd284103090 con 0x7fd26c038750 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.417+0000 7fd282000700 1 -- 172.21.15.46:0/2061182694 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd284103090 con 0x7fd26c038750 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 msgr2=0x7fd26c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 0x7fd26c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd270006fd0 tx=0x7fd270006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 msgr2=0x7fd28419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd278011710 tx=0x7fd278011ad0 comp rx=0 tx=0).stop 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 shutdown_connections 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd26c038750 0x7fd26c03ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:02.820 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 --2- 172.21.15.46:0/2061182694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd284106790 0x7fd28419c760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:02.821 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 >> 172.21.15.46:0/2061182694 conn(0x7fd284101d50 msgr2=0x7fd284102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:02.821 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 shutdown_connections 2024-05-08T22:56:02.821 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:02.420+0000 7fd28be00700 1 -- 172.21.15.46:0/2061182694 wait complete. 2024-05-08T22:56:02.821 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-05-08T22:56:03.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:03 smithi046 ceph-mon[36898]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:03.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:03 smithi046 ceph-mon[36898]: Saving service mon spec with placement count:5 2024-05-08T22:56:03.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:03 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:03.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:03 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:03.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:03 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.239+0000 7f3a74c00700 1 Processor -- start 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.239+0000 7f3a74c00700 1 -- start start 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.240+0000 7f3a74c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.240+0000 7f3a74c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7007a8f0 con 0x7f3a7007be70 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.240+0000 7f3a6de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.240+0000 7f3a6de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49984/0 (socket says 172.21.15.46:49984) 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.240+0000 7f3a6de00700 1 -- 172.21.15.46:0/4256173866 learned_addr learned my addr 172.21.15.46:0/4256173866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.241+0000 7f3a6de00700 1 -- 172.21.15.46:0/4256173866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7007aa30 con 0x7f3a7007be70 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.241+0000 7f3a6de00700 1 --2- 172.21.15.46:0/4256173866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098a0 tx=0x7f3a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9558f832b2bc3c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.241+0000 7f3a6ca00700 1 -- 172.21.15.46:0/4256173866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60004160 con 0x7f3a7007be70 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.241+0000 7f3a6ca00700 1 -- 172.21.15.46:0/4256173866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a600042c0 con 0x7f3a7007be70 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.241+0000 7f3a6ca00700 1 -- 172.21.15.46:0/4256173866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a600119b0 con 0x7f3a7007be70 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 -- 172.21.15.46:0/4256173866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 msgr2=0x7f3a7007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 --2- 172.21.15.46:0/4256173866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098a0 tx=0x7f3a60009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 -- 172.21.15.46:0/4256173866 shutdown_connections 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 --2- 172.21.15.46:0/4256173866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7007a320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 -- 172.21.15.46:0/4256173866 >> 172.21.15.46:0/4256173866 conn(0x7f3a70101d80 msgr2=0x7f3a701041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 -- 172.21.15.46:0/4256173866 shutdown_connections 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.242+0000 7f3a74c00700 1 -- 172.21.15.46:0/4256173866 wait complete. 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.243+0000 7f3a74c00700 1 Processor -- start 2024-05-08T22:56:03.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.243+0000 7f3a74c00700 1 -- start start 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a74c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a74c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7019cbf0 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a6de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a6de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49990/0 (socket says 172.21.15.46:49990) 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a6de00700 1 -- 172.21.15.46:0/6292879 learned_addr learned my addr 172.21.15.46:0/6292879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a6de00700 1 -- 172.21.15.46:0/6292879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009550 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.244+0000 7f3a6de00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a60003e60 tx=0x7f3a60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.245+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a600038b0 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.245+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a60003a10 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.245+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7019cdf0 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.245+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60019530 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.245+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7019d290 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.246+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3a6001a430 con 0x7f3a7007be70 2024-05-08T22:56:03.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.246+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7010cb40 con 0x7f3a7007be70 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.247+0000 7f3a66a00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 0x7f3a5403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.247+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a6004c770 con 0x7f3a7007be70 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.247+0000 7f3a6d400700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 0x7f3a5403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.248+0000 7f3a6d400700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 0x7f3a5403abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.251+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3a60023b80 con 0x7f3a7007be70 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.501+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7019fb20 con 0x7f3a54038700 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.506+0000 7f3a66a00700 1 -- 172.21.15.46:0/6292879 <== mgr.14118 v2:172.21.15.46:6800/2056989721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3a7019fb20 con 0x7f3a54038700 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.508+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 msgr2=0x7f3a5403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.508+0000 7f3a74c00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 0x7f3a5403abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.508+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 msgr2=0x7f3a7019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.508+0000 7f3a74c00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a60003e60 tx=0x7f3a60011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 shutdown_connections 2024-05-08T22:56:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f3a54038700 0x7f3a5403abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:03.895 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 --2- 172.21.15.46:0/6292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7007be70 0x7f3a7019c6b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:03.895 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 >> 172.21.15.46:0/6292879 conn(0x7f3a70101d80 msgr2=0x7f3a70102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:03.895 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 shutdown_connections 2024-05-08T22:56:03.895 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:03.509+0000 7f3a74c00700 1 -- 172.21.15.46:0/6292879 wait complete. 2024-05-08T22:56:04.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:04 smithi046 ceph-mon[36898]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:04.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:04 smithi046 ceph-mon[36898]: Saving service mgr spec with placement count:2 2024-05-08T22:56:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:04 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:04 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.446+0000 7fa58b600700 1 Processor -- start 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.446+0000 7fa58b600700 1 -- start start 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.447+0000 7fa58b600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.447+0000 7fa58b600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58407a910 con 0x7fa58407be90 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.447+0000 7fa588c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.447+0000 7fa588c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49998/0 (socket says 172.21.15.46:49998) 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.447+0000 7fa588c00700 1 -- 172.21.15.46:0/1668712135 learned_addr learned my addr 172.21.15.46:0/1668712135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.448+0000 7fa588c00700 1 -- 172.21.15.46:0/1668712135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58407aa50 con 0x7fa58407be90 2024-05-08T22:56:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.448+0000 7fa588c00700 1 --2- 172.21.15.46:0/1668712135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa5780098a0 tx=0x7fa578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c50479f23dd6c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.448+0000 7fa583400700 1 -- 172.21.15.46:0/1668712135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa578004160 con 0x7fa58407be90 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.448+0000 7fa583400700 1 -- 172.21.15.46:0/1668712135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5780042c0 con 0x7fa58407be90 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.448+0000 7fa583400700 1 -- 172.21.15.46:0/1668712135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5780119b0 con 0x7fa58407be90 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 -- 172.21.15.46:0/1668712135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 msgr2=0x7fa58407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 --2- 172.21.15.46:0/1668712135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa5780098a0 tx=0x7fa578009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 -- 172.21.15.46:0/1668712135 shutdown_connections 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 --2- 172.21.15.46:0/1668712135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58407a340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:05.030 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 -- 172.21.15.46:0/1668712135 >> 172.21.15.46:0/1668712135 conn(0x7fa584101d20 msgr2=0x7fa584104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 -- 172.21.15.46:0/1668712135 shutdown_connections 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.449+0000 7fa58b600700 1 -- 172.21.15.46:0/1668712135 wait complete. 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.451+0000 7fa58b600700 1 Processor -- start 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.451+0000 7fa58b600700 1 -- start start 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.451+0000 7fa58b600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.451+0000 7fa58b600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58419cb70 con 0x7fa58407be90 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.452+0000 7fa588c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.452+0000 7fa588c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.452+0000 7fa588c00700 1 -- 172.21.15.46:0/2374640691 learned_addr learned my addr 172.21.15.46:0/2374640691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.452+0000 7fa588c00700 1 -- 172.21.15.46:0/2374640691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578009550 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.452+0000 7fa588c00700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa578011670 tx=0x7fa578011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.453+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa578003b20 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.453+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa578003c80 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.453+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58419cd70 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.453+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa578019770 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.453+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58419d210 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.454+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58410cb60 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.454+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa57801ab60 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.455+0000 7fa581600700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 0x7fa56c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.455+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa57804ca30 con 0x7fa58407be90 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.455+0000 7fa583e00700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 0x7fa56c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:05.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.456+0000 7fa583e00700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 0x7fa56c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.461+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa57801a430 con 0x7fa58407be90 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.678+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa58419faa0 con 0x7fa58407be90 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.681+0000 7fa581600700 1 -- 172.21.15.46:0/2374640691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa57802db40 con 0x7fa58407be90 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 msgr2=0x7fa56c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 0x7fa56c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 msgr2=0x7fa58419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa578011670 tx=0x7fa578011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 shutdown_connections 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fa56c038750 0x7fa56c03ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 --2- 172.21.15.46:0/2374640691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa58407be90 0x7fa58419c630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 >> 172.21.15.46:0/2374640691 conn(0x7fa584101d20 msgr2=0x7fa584102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:05.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 shutdown_connections 2024-05-08T22:56:05.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:04.684+0000 7fa58b600700 1 -- 172.21.15.46:0/2374640691 wait complete. 2024-05-08T22:56:05.933 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:05 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2374640691' entity='client.admin' 2024-05-08T22:56:05.933 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:05 smithi046 ceph-mon[36898]: from='mgr.14118 172.21.15.46:0/2573276721' entity='mgr.x' 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.466+0000 7f7258200700 1 Processor -- start 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.467+0000 7f7258200700 1 -- start start 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.467+0000 7f7258200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.467+0000 7f7258200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250107180 con 0x7f7250106790 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.467+0000 7f7255800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:06.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f7255800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50026/0 (socket says 172.21.15.46:50026) 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f7255800700 1 -- 172.21.15.46:0/2984993338 learned_addr learned my addr 172.21.15.46:0/2984993338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f7255800700 1 -- 172.21.15.46:0/2984993338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250107940 con 0x7f7250106790 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f7255800700 1 --2- 172.21.15.46:0/2984993338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72440098a0 tx=0x7f7244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e793d44199e2c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f724fe00700 1 -- 172.21.15.46:0/2984993338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7244004160 con 0x7f7250106790 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f724fe00700 1 -- 172.21.15.46:0/2984993338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72440042c0 con 0x7f7250106790 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.468+0000 7f724fe00700 1 -- 172.21.15.46:0/2984993338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72440119b0 con 0x7f7250106790 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 -- 172.21.15.46:0/2984993338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 msgr2=0x7f7250106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 --2- 172.21.15.46:0/2984993338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72440098a0 tx=0x7f7244009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:06.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 -- 172.21.15.46:0/2984993338 shutdown_connections 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 --2- 172.21.15.46:0/2984993338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f7250106bb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 -- 172.21.15.46:0/2984993338 >> 172.21.15.46:0/2984993338 conn(0x7f7250101d50 msgr2=0x7f7250104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 -- 172.21.15.46:0/2984993338 shutdown_connections 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.469+0000 7f7258200700 1 -- 172.21.15.46:0/2984993338 wait complete. 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.470+0000 7f7258200700 1 Processor -- start 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.470+0000 7f7258200700 1 -- start start 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7258200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7258200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725019aa80 con 0x7f7250106790 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7255800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7255800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50034/0 (socket says 172.21.15.46:50034) 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7255800700 1 -- 172.21.15.46:0/1856875743 learned_addr learned my addr 172.21.15.46:0/1856875743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.471+0000 7f7255800700 1 -- 172.21.15.46:0/1856875743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009550 con 0x7f7250106790 2024-05-08T22:56:06.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f7255800700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7244011670 tx=0x7f7244011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7244003af0 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7244003c50 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7244024820 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725019ac80 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.472+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725019b120 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.473+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725010ab70 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.473+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f724401ab60 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.474+0000 7f724e000700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 0x7f723803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.474+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f724404c6e0 con 0x7f7250106790 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.474+0000 7f7254e00700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 0x7f723803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.475+0000 7f7254e00700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 0x7f723803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:06.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.479+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7244018760 con 0x7f7250106790 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.725+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f725010adb0 con 0x7f7250106790 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.728+0000 7f724e000700 1 -- 172.21.15.46:0/1856875743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f724400b3e0 con 0x7f7250106790 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 msgr2=0x7f723803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 0x7f723803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 msgr2=0x7f725019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7244011670 tx=0x7f7244011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 shutdown_connections 2024-05-08T22:56:06.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f7238038750 0x7f723803ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:06.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 --2- 172.21.15.46:0/1856875743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7250106790 0x7f725019a540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:06.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 >> 172.21.15.46:0/1856875743 conn(0x7f7250101d50 msgr2=0x7f7250102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:06.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 shutdown_connections 2024-05-08T22:56:06.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:05.732+0000 7f7258200700 1 -- 172.21.15.46:0/1856875743 wait complete. 2024-05-08T22:56:06.086 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-05-08T22:56:06.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:06 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1856875743' entity='client.admin' 2024-05-08T22:56:07.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3446365364' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T22:56:07.982 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setuser ceph since I am not root 2024-05-08T22:56:07.983 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setgroup ceph since I am not root 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.532+0000 7f1ac3400700 1 Processor -- start 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.532+0000 7f1ac3400700 1 -- start start 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.533+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.533+0000 7f1ac3400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc107180 con 0x7f1abc106790 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.533+0000 7f1ac0a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.533+0000 7f1ac0a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50046/0 (socket says 172.21.15.46:50046) 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.533+0000 7f1ac0a00700 1 -- 172.21.15.46:0/1251291700 learned_addr learned my addr 172.21.15.46:0/1251291700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.534+0000 7f1ac0a00700 1 -- 172.21.15.46:0/1251291700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc107940 con 0x7f1abc106790 2024-05-08T22:56:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.534+0000 7f1ac0a00700 1 --2- 172.21.15.46:0/1251291700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00098a0 tx=0x7f1ab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df02279190e29b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.534+0000 7f1abb400700 1 -- 172.21.15.46:0/1251291700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ab0004160 con 0x7f1abc106790 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.534+0000 7f1abb400700 1 -- 172.21.15.46:0/1251291700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab00042c0 con 0x7f1abc106790 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.534+0000 7f1abb400700 1 -- 172.21.15.46:0/1251291700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ab00119b0 con 0x7f1abc106790 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 -- 172.21.15.46:0/1251291700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 msgr2=0x7f1abc106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 --2- 172.21.15.46:0/1251291700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00098a0 tx=0x7f1ab0009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 -- 172.21.15.46:0/1251291700 shutdown_connections 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 --2- 172.21.15.46:0/1251291700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc106bb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 -- 172.21.15.46:0/1251291700 >> 172.21.15.46:0/1251291700 conn(0x7f1abc101d50 msgr2=0x7f1abc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 -- 172.21.15.46:0/1251291700 shutdown_connections 2024-05-08T22:56:08.130 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.535+0000 7f1ac3400700 1 -- 172.21.15.46:0/1251291700 wait complete. 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.536+0000 7f1ac3400700 1 Processor -- start 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.536+0000 7f1ac3400700 1 -- start start 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac3400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19aa80 con 0x7f1abc106790 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac0a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac0a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50062/0 (socket says 172.21.15.46:50062) 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac0a00700 1 -- 172.21.15.46:0/3446365364 learned_addr learned my addr 172.21.15.46:0/3446365364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.537+0000 7f1ac0a00700 1 -- 172.21.15.46:0/3446365364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0009550 con 0x7f1abc106790 2024-05-08T22:56:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ac0a00700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0011670 tx=0x7f1ab0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ab0003af0 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab0003c50 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc19ac80 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ab0024820 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.538+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc19b120 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.539+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f1ab001a430 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.539+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc10ab70 con 0x7f1abc106790 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.540+0000 7f1ab9600700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 0x7f1aa403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.540+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ab004c7a0 con 0x7f1abc106790 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.540+0000 7f1abbe00700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 0x7f1aa403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.541+0000 7f1abbe00700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 0x7f1aa403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8006fd0 tx=0x7f1aa8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.545+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ab002cb00 con 0x7f1abc106790 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:06.829+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1abc10adb0 con 0x7f1abc106790 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.733+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f1ab001a6e0 con 0x7f1abc106790 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.735+0000 7f1ab9600700 1 -- 172.21.15.46:0/3446365364 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f1ab004c130 con 0x7f1abc106790 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 msgr2=0x7f1aa403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 0x7f1aa403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8006fd0 tx=0x7f1aa8006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 msgr2=0x7f1abc19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0011670 tx=0x7f1ab0011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 shutdown_connections 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7f1aa4038700 0x7f1aa403abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 --2- 172.21.15.46:0/3446365364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1abc106790 0x7f1abc19a540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:08.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 >> 172.21.15.46:0/3446365364 conn(0x7f1abc101d50 msgr2=0x7f1abc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:08.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 shutdown_connections 2024-05-08T22:56:08.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:07.739+0000 7f1ac3400700 1 -- 172.21.15.46:0/3446365364 wait complete. 2024-05-08T22:56:08.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:08.410+0000 7f2a377e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:56:08.850 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3446365364' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T22:56:08.850 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:08 smithi046 ceph-mon[36898]: mgrmap e9: x(active, since 15s) 2024-05-08T22:56:08.850 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:08.536+0000 7f2a377e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:56:08.850 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:08.657+0000 7f2a377e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:56:08.851 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:08.772+0000 7f2a377e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:56:09.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:56:09.171 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.535+0000 7fb800c00700 1 Processor -- start 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb800c00700 1 -- start start 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb800c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb800c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc106ff0 con 0x7fb7fc106600 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb7fb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb7fb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50088/0 (socket says 172.21.15.46:50088) 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.536+0000 7fb7fb400700 1 -- 172.21.15.46:0/2039894071 learned_addr learned my addr 172.21.15.46:0/2039894071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:09.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.537+0000 7fb7fb400700 1 -- 172.21.15.46:0/2039894071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc1077b0 con 0x7fb7fc106600 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.537+0000 7fb7fb400700 1 --2- 172.21.15.46:0/2039894071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0098a0 tx=0x7fb7ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82146281084ecd43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.537+0000 7fb7fa000700 1 -- 172.21.15.46:0/2039894071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7ec004160 con 0x7fb7fc106600 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.537+0000 7fb7fa000700 1 -- 172.21.15.46:0/2039894071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7ec0042c0 con 0x7fb7fc106600 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.537+0000 7fb7fa000700 1 -- 172.21.15.46:0/2039894071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7ec0119b0 con 0x7fb7fc106600 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 -- 172.21.15.46:0/2039894071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 msgr2=0x7fb7fc106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 --2- 172.21.15.46:0/2039894071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0098a0 tx=0x7fb7ec009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 -- 172.21.15.46:0/2039894071 shutdown_connections 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 --2- 172.21.15.46:0/2039894071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc106a20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 -- 172.21.15.46:0/2039894071 >> 172.21.15.46:0/2039894071 conn(0x7fb7fc101bc0 msgr2=0x7fb7fc103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 -- 172.21.15.46:0/2039894071 shutdown_connections 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.538+0000 7fb800c00700 1 -- 172.21.15.46:0/2039894071 wait complete. 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.540+0000 7fb800c00700 1 Processor -- start 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.540+0000 7fb800c00700 1 -- start start 2024-05-08T22:56:09.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.540+0000 7fb800c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.540+0000 7fb800c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc07c060 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.541+0000 7fb7fb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.541+0000 7fb7fb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50094/0 (socket says 172.21.15.46:50094) 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.541+0000 7fb7fb400700 1 -- 172.21.15.46:0/537906737 learned_addr learned my addr 172.21.15.46:0/537906737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.541+0000 7fb7fb400700 1 -- 172.21.15.46:0/537906737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009550 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.541+0000 7fb7fb400700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec011670 tx=0x7fb7ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.542+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7ec003af0 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.542+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7ec003c50 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.542+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7ec024820 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.542+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc07a2a0 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.542+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc07a740 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fb7ec024a50 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc10a9e0 con 0x7fb7fc106600 2024-05-08T22:56:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb7f3e00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 0x7fb7e003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb7ec04c9b0 con 0x7fb7fc106600 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb7faa00700 1 -- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 msgr2=0x7fb7e003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.543+0000 7fb7faa00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 0x7fb7e003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.548+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb7ec0189a0 con 0x7fb7fc106600 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.744+0000 7fb7faa00700 1 -- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 msgr2=0x7fb7e003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.744+0000 7fb7faa00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 0x7fb7e003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.825+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb7fc07ae80 con 0x7fb7fc106600 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.826+0000 7fb7f3e00700 1 -- 172.21.15.46:0/537906737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb7ec0187c0 con 0x7fb7fc106600 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 msgr2=0x7fb7e003ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 0x7fb7e003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 msgr2=0x7fb7fc07bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec011670 tx=0x7fb7ec011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 shutdown_connections 2024-05-08T22:56:09.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fb7e00387a0 0x7fb7e003ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 --2- 172.21.15.46:0/537906737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc106600 0x7fb7fc07bb20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 >> 172.21.15.46:0/537906737 conn(0x7fb7fc101bc0 msgr2=0x7fb7fc102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.828+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 shutdown_connections 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:08.829+0000 7fb800c00700 1 -- 172.21.15.46:0/537906737 wait complete. 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-08T22:56:09.176 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-05-08T22:56:10.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/537906737' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:56:10.003 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:09 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:09.673+0000 7f2a377e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:56:10.270 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.002+0000 7f2a377e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:56:10.270 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.121+0000 7f2a377e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:56:10.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.270+0000 7f2a377e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:56:10.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.472+0000 7f2a377e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:56:11.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.648+0000 7f2a377e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:56:11.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:10 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:10.763+0000 7f2a377e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:56:11.526 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:11 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:11.248+0000 7f2a377e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:56:12.018 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:11 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:11.756+0000 7f2a377e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:56:12.018 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:12 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:12.018+0000 7f2a377e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:56:12.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:12 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:12.144+0000 7f2a377e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:56:13.480 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:13 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:13.145+0000 7f2a377e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:56:13.480 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:13 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:13.277+0000 7f2a377e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:56:13.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:13 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:13.480+0000 7f2a377e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:56:14.584 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:14 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:14.332+0000 7f2a377e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:56:14.855 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:14 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:14.584+0000 7f2a377e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:56:15.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:14 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:14.855+0000 7f2a377e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:56:16.447 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:16 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:16.052+0000 7f2a377e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:56:16.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:16 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:16.448+0000 7f2a377e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:56:16.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:16 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:16.573+0000 7f2a377e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:56:17.276 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:16 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:16.841+0000 7f2a377e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: Active manager daemon x restarted 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: Activating manager daemon x 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: mgrmap e10: x(active, starting, since 0.0043126s) 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:56:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:56:17.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:56:17.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: Manager daemon x is now available 2024-05-08T22:56:17.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:56:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:56:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: [08/May/2024:22:56:17] ENGINE Bus STARTING 2024-05-08T22:56:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: [08/May/2024:22:56:17] ENGINE Serving on http://172.21.15.46:8765 2024-05-08T22:56:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: [08/May/2024:22:56:17] ENGINE Serving on https://172.21.15.46:7150 2024-05-08T22:56:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: [08/May/2024:22:56:17] ENGINE Bus STARTED 2024-05-08T22:56:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: mgrmap e11: x(active, since 1.0074s) 2024-05-08T22:56:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:56:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:17 smithi046 ceph-mon[36898]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:56:18.349 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.636+0000 7fd76f400700 1 Processor -- start 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.636+0000 7fd76f400700 1 -- start start 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.637+0000 7fd76f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.637+0000 7fd76f400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768107170 con 0x7fd768106780 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.637+0000 7fd76ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.637+0000 7fd76ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50110/0 (socket says 172.21.15.46:50110) 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.637+0000 7fd76ca00700 1 -- 172.21.15.46:0/3761061565 learned_addr learned my addr 172.21.15.46:0/3761061565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:18.358 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.638+0000 7fd76ca00700 1 -- 172.21.15.46:0/3761061565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768107930 con 0x7fd768106780 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.638+0000 7fd76ca00700 1 --2- 172.21.15.46:0/3761061565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0098a0 tx=0x7fd75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74862c1e575e1c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.638+0000 7fd767400700 1 -- 172.21.15.46:0/3761061565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd75c004160 con 0x7fd768106780 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.638+0000 7fd767400700 1 -- 172.21.15.46:0/3761061565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd75c0042c0 con 0x7fd768106780 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.638+0000 7fd767400700 1 -- 172.21.15.46:0/3761061565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd75c0119b0 con 0x7fd768106780 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 -- 172.21.15.46:0/3761061565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 msgr2=0x7fd768106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 --2- 172.21.15.46:0/3761061565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0098a0 tx=0x7fd75c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 -- 172.21.15.46:0/3761061565 shutdown_connections 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 --2- 172.21.15.46:0/3761061565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd768106ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 -- 172.21.15.46:0/3761061565 >> 172.21.15.46:0/3761061565 conn(0x7fd768101d20 msgr2=0x7fd768104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 -- 172.21.15.46:0/3761061565 shutdown_connections 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.639+0000 7fd76f400700 1 -- 172.21.15.46:0/3761061565 wait complete. 2024-05-08T22:56:18.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.640+0000 7fd76f400700 1 Processor -- start 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.640+0000 7fd76f400700 1 -- start start 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76f400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819cbf0 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50118/0 (socket says 172.21.15.46:50118) 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76ca00700 1 -- 172.21.15.46:0/3376570848 learned_addr learned my addr 172.21.15.46:0/3376570848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.641+0000 7fd76ca00700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c009550 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd76ca00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd75c003e60 tx=0x7fd75c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd75c003a60 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd75c003bc0 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd75c0196c0 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76819cdf0 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.642+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76819d290 con 0x7fd768106780 2024-05-08T22:56:18.360 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.643+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fd75c01a430 con 0x7fd768106780 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.643+0000 7fd765600700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.643+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd75003b300 con 0x7fd750038730 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.643+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd75c04c9b0 con 0x7fd768106780 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.644+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.644+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.845+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:09.845+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:10.246+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:10.246+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:11.047+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:11.047+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:12.648+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.361 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:12.648+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:15.852+0000 7fd767e00700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2056989721 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:15.852+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:16.846+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fd75c01a780 con 0x7fd768106780 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:16.847+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 msgr2=0x7fd75003abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:16.847+0000 7fd765600700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.849+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fd75c029c90 con 0x7fd768106780 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.849+0000 7fd765600700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 0x7fd75003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.849+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd75003b300 con 0x7fd75003bfd0 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.850+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 0x7fd75003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.851+0000 7fd767e00700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 0x7fd75003e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd754003a10 tx=0x7fd7540092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.852+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd75003b300 con 0x7fd75003bfd0 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.859+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd76810ab60 con 0x7fd75003bfd0 2024-05-08T22:56:18.362 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd765600700 1 -- 172.21.15.46:0/3376570848 <== mgr.14150 v2:172.21.15.46:6800/2731582335 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd76810ab60 con 0x7fd75003bfd0 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 msgr2=0x7fd75003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 0x7fd75003e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd754003a10 tx=0x7fd7540092b0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 msgr2=0x7fd76819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd75c003e60 tx=0x7fd75c011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 shutdown_connections 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75003bfd0 0x7fd75003e3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768106780 0x7fd76819c6b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 --2- 172.21.15.46:0/3376570848 >> [v2:172.21.15.46:6800/2056989721,v1:172.21.15.46:6801/2056989721] conn(0x7fd750038730 0x7fd75003abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 >> 172.21.15.46:0/3376570848 conn(0x7fd768101d20 msgr2=0x7fd768102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 shutdown_connections 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:17.860+0000 7fd76f400700 1 -- 172.21.15.46:0/3376570848 wait complete. 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-05-08T22:56:18.363 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.756+0000 7f650ae00700 1 Processor -- start 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.756+0000 7f650ae00700 1 -- start start 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.756+0000 7f650ae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.756+0000 7f650ae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504079e30 con 0x7f6504105040 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.757+0000 7f6503e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.757+0000 7f6503e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55956/0 (socket says 172.21.15.46:55956) 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.757+0000 7f6503e00700 1 -- 172.21.15.46:0/4042715376 learned_addr learned my addr 172.21.15.46:0/4042715376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.757+0000 7f6503e00700 1 -- 172.21.15.46:0/4042715376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65041059a0 con 0x7f6504105040 2024-05-08T22:56:19.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.757+0000 7f6503e00700 1 --2- 172.21.15.46:0/4042715376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64f40098a0 tx=0x7f64f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3fe03c38372f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:19.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:19.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:19.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f6502a00700 1 -- 172.21.15.46:0/4042715376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f4004160 con 0x7f6504105040 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f6502a00700 1 -- 172.21.15.46:0/4042715376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f40042c0 con 0x7f6504105040 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f6502a00700 1 -- 172.21.15.46:0/4042715376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f40119b0 con 0x7f6504105040 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f650ae00700 1 -- 172.21.15.46:0/4042715376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 msgr2=0x7f6504105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f650ae00700 1 --2- 172.21.15.46:0/4042715376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64f40098a0 tx=0x7f64f4009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f650ae00700 1 -- 172.21.15.46:0/4042715376 shutdown_connections 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f650ae00700 1 --2- 172.21.15.46:0/4042715376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f6504105460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.758+0000 7f650ae00700 1 -- 172.21.15.46:0/4042715376 >> 172.21.15.46:0/4042715376 conn(0x7f6504100c20 msgr2=0x7f6504103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.759+0000 7f650ae00700 1 -- 172.21.15.46:0/4042715376 shutdown_connections 2024-05-08T22:56:19.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.759+0000 7f650ae00700 1 -- 172.21.15.46:0/4042715376 wait complete. 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.760+0000 7f650ae00700 1 Processor -- start 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.760+0000 7f650ae00700 1 -- start start 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.760+0000 7f650ae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.760+0000 7f650ae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650419cba0 con 0x7f6504105040 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.761+0000 7f6503e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.761+0000 7f6503e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55958/0 (socket says 172.21.15.46:55958) 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.761+0000 7f6503e00700 1 -- 172.21.15.46:0/1485334916 learned_addr learned my addr 172.21.15.46:0/1485334916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.761+0000 7f6503e00700 1 -- 172.21.15.46:0/1485334916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4009550 con 0x7f6504105040 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.761+0000 7f6503e00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64f4011670 tx=0x7f64f4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.762+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f4003b20 con 0x7f6504105040 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.762+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f4003c80 con 0x7f6504105040 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.762+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f4019770 con 0x7f6504105040 2024-05-08T22:56:19.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.762+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650419cda0 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.762+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650419d240 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.763+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6504196370 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.763+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f64f401a430 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.764+0000 7f6500c00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 0x7f64e803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.764+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64f404c8f0 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.764+0000 7f6503400700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 0x7f64e803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.765+0000 7f6503400700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 0x7f64e803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f64f0006fd0 tx=0x7f64f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.769+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64f402d9a0 con 0x7f6504105040 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:18.989+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f650402cf20 con 0x7f64e8038670 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.305+0000 7f6500c00700 1 -- 172.21.15.46:0/1485334916 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f650402cf20 con 0x7f64e8038670 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 msgr2=0x7f64e803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 0x7f64e803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f64f0006fd0 tx=0x7f64f0006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:19.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 msgr2=0x7f650419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64f4011670 tx=0x7f64f4011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 shutdown_connections 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64e8038670 0x7f64e803ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 --2- 172.21.15.46:0/1485334916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6504105040 0x7f650419c660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 >> 172.21.15.46:0/1485334916 conn(0x7f6504100c20 msgr2=0x7f65041934b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 shutdown_connections 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:19.308+0000 7f650ae00700 1 -- 172.21.15.46:0/1485334916 wait complete. 2024-05-08T22:56:19.644 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-05-08T22:56:20.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:20 smithi046 ceph-mon[36898]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:20.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:20 smithi046 ceph-mon[36898]: mgrmap e12: x(active, since 2s) 2024-05-08T22:56:20.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:20.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.MN1.gGCIgZZ0xgSPPzPceM055rkhmlh1pRSWNnG4fJzfxbvDgp4K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715208980, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T22:56:20.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.063+0000 7f296f200700 1 Processor -- start 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.063+0000 7f296f200700 1 -- start start 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f296f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f296f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2968079e30 con 0x7f2968107250 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f2967e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f2967e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55966/0 (socket says 172.21.15.46:55966) 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f2967e00700 1 -- 172.21.15.46:0/367619171 learned_addr learned my addr 172.21.15.46:0/367619171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.064+0000 7f2967e00700 1 -- 172.21.15.46:0/367619171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968109b80 con 0x7f2968107250 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.065+0000 7f2967e00700 1 --2- 172.21.15.46:0/367619171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29580098a0 tx=0x7f2958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf9119f0a06e716 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.065+0000 7f2966a00700 1 -- 172.21.15.46:0/367619171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2958004160 con 0x7f2968107250 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.065+0000 7f2966a00700 1 -- 172.21.15.46:0/367619171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29580042c0 con 0x7f2968107250 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.065+0000 7f2966a00700 1 -- 172.21.15.46:0/367619171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29580119b0 con 0x7f2968107250 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 -- 172.21.15.46:0/367619171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 msgr2=0x7f2968109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 --2- 172.21.15.46:0/367619171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29580098a0 tx=0x7f2958009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:20.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 -- 172.21.15.46:0/367619171 shutdown_connections 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 --2- 172.21.15.46:0/367619171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968109640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 -- 172.21.15.46:0/367619171 >> 172.21.15.46:0/367619171 conn(0x7f2968100c20 msgr2=0x7f2968103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 -- 172.21.15.46:0/367619171 shutdown_connections 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.066+0000 7f296f200700 1 -- 172.21.15.46:0/367619171 wait complete. 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.067+0000 7f296f200700 1 Processor -- start 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f296f200700 1 -- start start 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f296f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f296f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2968106fc0 con 0x7f2968107250 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f2967e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f2967e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55968/0 (socket says 172.21.15.46:55968) 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.068+0000 7f2967e00700 1 -- 172.21.15.46:0/2943125061 learned_addr learned my addr 172.21.15.46:0/2943125061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f2967e00700 1 -- 172.21.15.46:0/2943125061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009550 con 0x7f2968107250 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f2967e00700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2958011670 tx=0x7f2958011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2958003b20 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2958003c80 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2968105200 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2958019770 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.069+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29681056a0 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.070+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f295801a430 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.071+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2968068010 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.071+0000 7f2964c00700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 0x7f294c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.071+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f295804cb00 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.071+0000 7f2967400700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 0x7f294c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.072+0000 7f2967400700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 0x7f294c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2954006fd0 tx=0x7f2954006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.077+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f295802d9a0 con 0x7f2968107250 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.346+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f29680623f0 con 0x7f294c0387a0 2024-05-08T22:56:20.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.593+0000 7f2964c00700 1 -- 172.21.15.46:0/2943125061 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f29680623f0 con 0x7f294c0387a0 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.595+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 msgr2=0x7f294c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.595+0000 7f296f200700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 0x7f294c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2954006fd0 tx=0x7f2954006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.595+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 msgr2=0x7f2968106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.595+0000 7f296f200700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2958011670 tx=0x7f2958011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 shutdown_connections 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f294c0387a0 0x7f294c03ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 --2- 172.21.15.46:0/2943125061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2968107250 0x7f2968106a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 >> 172.21.15.46:0/2943125061 conn(0x7f2968100c20 msgr2=0x7f2968101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 shutdown_connections 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:20.596+0000 7f296f200700 1 -- 172.21.15.46:0/2943125061 wait complete. 2024-05-08T22:56:20.975 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-05-08T22:56:21.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.343+0000 7fb11e200700 1 Processor -- start 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.343+0000 7fb11e200700 1 -- start start 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.343+0000 7fb11e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.343+0000 7fb11e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb118079e10 con 0x7fb118107240 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.344+0000 7fb117400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.344+0000 7fb117400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55970/0 (socket says 172.21.15.46:55970) 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.344+0000 7fb117400700 1 -- 172.21.15.46:0/657611313 learned_addr learned my addr 172.21.15.46:0/657611313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:21.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.344+0000 7fb117400700 1 -- 172.21.15.46:0/657611313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb118079f50 con 0x7fb118107240 2024-05-08T22:56:21.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.344+0000 7fb117400700 1 --2- 172.21.15.46:0/657611313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098a0 tx=0x7fb108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90e6eb6a0d71509 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:21.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb116000700 1 -- 172.21.15.46:0/657611313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb108004160 con 0x7fb118107240 2024-05-08T22:56:21.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb116000700 1 -- 172.21.15.46:0/657611313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1080042c0 con 0x7fb118107240 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb116000700 1 -- 172.21.15.46:0/657611313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1080119b0 con 0x7fb118107240 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb11e200700 1 -- 172.21.15.46:0/657611313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 msgr2=0x7fb118109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb11e200700 1 --2- 172.21.15.46:0/657611313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098a0 tx=0x7fb108009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb11e200700 1 -- 172.21.15.46:0/657611313 shutdown_connections 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb11e200700 1 --2- 172.21.15.46:0/657611313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb118109630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.345+0000 7fb11e200700 1 -- 172.21.15.46:0/657611313 >> 172.21.15.46:0/657611313 conn(0x7fb118100c50 msgr2=0x7fb118103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.346+0000 7fb11e200700 1 -- 172.21.15.46:0/657611313 shutdown_connections 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.346+0000 7fb11e200700 1 -- 172.21.15.46:0/657611313 wait complete. 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb11e200700 1 Processor -- start 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb11e200700 1 -- start start 2024-05-08T22:56:21.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb11e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb11e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11819cbf0 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb117400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb117400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55978/0 (socket says 172.21.15.46:55978) 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.347+0000 7fb117400700 1 -- 172.21.15.46:0/1084118273 learned_addr learned my addr 172.21.15.46:0/1084118273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb117400700 1 -- 172.21.15.46:0/1084118273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108009550 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb117400700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb108011670 tx=0x7fb108011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb108003b20 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb108003c80 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11819cdf0 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb108019770 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.348+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11819d290 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.349+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fb10801ab60 con 0x7fb118107240 2024-05-08T22:56:21.924 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.349+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11810a8e0 con 0x7fb118107240 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.350+0000 7fb10fe00700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 0x7fb0fc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.350+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb10804cbd0 con 0x7fb118107240 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.350+0000 7fb116a00700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 0x7fb0fc03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.351+0000 7fb116a00700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 0x7fb0fc03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.355+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb10801a430 con 0x7fb118107240 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.567+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb11819fb80 con 0x7fb118107240 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.568+0000 7fb10fe00700 1 -- 172.21.15.46:0/1084118273 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb10802db40 con 0x7fb118107240 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 msgr2=0x7fb0fc03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 0x7fb0fc03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 msgr2=0x7fb11819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb108011670 tx=0x7fb108011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 shutdown_connections 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb0fc038750 0x7fb0fc03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:21.925 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 --2- 172.21.15.46:0/1084118273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb118107240 0x7fb11819c6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:21.926 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.570+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 >> 172.21.15.46:0/1084118273 conn(0x7fb118100c50 msgr2=0x7fb118101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:21.926 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.571+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 shutdown_connections 2024-05-08T22:56:21.926 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:21.571+0000 7fb11e200700 1 -- 172.21.15.46:0/1084118273 wait complete. 2024-05-08T22:56:21.926 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:21 smithi046 ceph-mon[36898]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:21.926 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:21 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:21.926 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:21 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1084118273' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T22:56:21.941 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:56:21.941 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-08T22:56:21.957 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:56:21.957 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T22:56:21.957 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-08T22:56:21.957 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T22:56:21.959 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-05-08T22:56:21.960 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:21.960 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-05-08T22:56:21.960 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-05-08T22:56:21.960 INFO:teuthology.orchestra.run.smithi046.stdout: Password: glvjm7jyzq 2024-05-08T22:56:21.960 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:21.961 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config directory 2024-05-08T22:56:21.961 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-05-08T22:56:22.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.360+0000 7f531bc00700 1 Processor -- start 2024-05-08T22:56:22.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.360+0000 7f531bc00700 1 -- start start 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f531bc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f531bc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314079e10 con 0x7f5314107240 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f5319200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f5319200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57678/0 (socket says 172.21.15.46:57678) 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f5319200700 1 -- 172.21.15.46:0/2997574207 learned_addr learned my addr 172.21.15.46:0/2997574207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.361+0000 7f5319200700 1 -- 172.21.15.46:0/2997574207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314079f50 con 0x7f5314107240 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.362+0000 7f5319200700 1 --2- 172.21.15.46:0/2997574207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53080098a0 tx=0x7f5308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6491e6833b235a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.362+0000 7f5313400700 1 -- 172.21.15.46:0/2997574207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5308004160 con 0x7f5314107240 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.362+0000 7f5313400700 1 -- 172.21.15.46:0/2997574207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53080042c0 con 0x7f5314107240 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.362+0000 7f5313400700 1 -- 172.21.15.46:0/2997574207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53080119b0 con 0x7f5314107240 2024-05-08T22:56:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 -- 172.21.15.46:0/2997574207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 msgr2=0x7f5314109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 --2- 172.21.15.46:0/2997574207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53080098a0 tx=0x7f5308009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 -- 172.21.15.46:0/2997574207 shutdown_connections 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 --2- 172.21.15.46:0/2997574207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f5314109630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 -- 172.21.15.46:0/2997574207 >> 172.21.15.46:0/2997574207 conn(0x7f5314100c50 msgr2=0x7f5314103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 -- 172.21.15.46:0/2997574207 shutdown_connections 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.363+0000 7f531bc00700 1 -- 172.21.15.46:0/2997574207 wait complete. 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.364+0000 7f531bc00700 1 Processor -- start 2024-05-08T22:56:22.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.364+0000 7f531bc00700 1 -- start start 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.364+0000 7f531bc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.364+0000 7f531bc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531419cbf0 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5319200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5319200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57680/0 (socket says 172.21.15.46:57680) 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5319200700 1 -- 172.21.15.46:0/4139710680 learned_addr learned my addr 172.21.15.46:0/4139710680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5319200700 1 -- 172.21.15.46:0/4139710680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308009550 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5319200700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5308011670 tx=0x7f5308011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5308003b20 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5308003c80 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5308019770 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.365+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531419cdf0 con 0x7f5314107240 2024-05-08T22:56:22.954 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.366+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531419d290 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.367+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f530801ab60 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.367+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531410a8e0 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.367+0000 7f5311600700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 0x7f52fc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.367+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f530804cb70 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.367+0000 7f5313e00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 0x7f52fc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.368+0000 7f5313e00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 0x7f52fc03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5300006fd0 tx=0x7f5300006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.372+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f530801a430 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.583+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f531419fb80 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.583+0000 7f5311600700 1 -- 172.21.15.46:0/4139710680 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f530802db40 con 0x7f5314107240 2024-05-08T22:56:22.955 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 msgr2=0x7f52fc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 0x7f52fc03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5300006fd0 tx=0x7f5300006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 msgr2=0x7f531419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5308011670 tx=0x7f5308011a30 comp rx=0 tx=0).stop 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 shutdown_connections 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52fc0387a0 0x7f52fc03ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 --2- 172.21.15.46:0/4139710680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5314107240 0x7f531419c6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 >> 172.21.15.46:0/4139710680 conn(0x7f5314100c50 msgr2=0x7f5314101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 shutdown_connections 2024-05-08T22:56:22.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:22.586+0000 7f531bc00700 1 -- 172.21.15.46:0/4139710680 wait complete. 2024-05-08T22:56:23.958 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.334+0000 7fa228a00700 1 Processor -- start 2024-05-08T22:56:23.959 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:23 smithi046 ceph-mon[36898]: mgrmap e13: x(active, since 6s) 2024-05-08T22:56:23.959 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:23 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3258527127' entity='client.admin' 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.334+0000 7fa228a00700 1 -- start start 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.334+0000 7fa228a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.334+0000 7fa228a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22407a910 con 0x7fa22407be90 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.334+0000 7fa221e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.335+0000 7fa221e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57696/0 (socket says 172.21.15.46:57696) 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.335+0000 7fa221e00700 1 -- 172.21.15.46:0/3983087820 learned_addr learned my addr 172.21.15.46:0/3983087820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.335+0000 7fa221e00700 1 -- 172.21.15.46:0/3983087820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22407aa50 con 0x7fa22407be90 2024-05-08T22:56:23.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.335+0000 7fa221e00700 1 --2- 172.21.15.46:0/3983087820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2140098a0 tx=0x7fa214009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ed6574938a984d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.335+0000 7fa220a00700 1 -- 172.21.15.46:0/3983087820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa214004160 con 0x7fa22407be90 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa220a00700 1 -- 172.21.15.46:0/3983087820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2140042c0 con 0x7fa22407be90 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa220a00700 1 -- 172.21.15.46:0/3983087820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2140119b0 con 0x7fa22407be90 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 -- 172.21.15.46:0/3983087820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 msgr2=0x7fa22407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 --2- 172.21.15.46:0/3983087820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2140098a0 tx=0x7fa214009bb0 comp rx=0 tx=0).stop 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 -- 172.21.15.46:0/3983087820 shutdown_connections 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 --2- 172.21.15.46:0/3983087820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22407a340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 -- 172.21.15.46:0/3983087820 >> 172.21.15.46:0/3983087820 conn(0x7fa224101d50 msgr2=0x7fa224104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 -- 172.21.15.46:0/3983087820 shutdown_connections 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.336+0000 7fa228a00700 1 -- 172.21.15.46:0/3983087820 wait complete. 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.337+0000 7fa228a00700 1 Processor -- start 2024-05-08T22:56:23.966 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa228a00700 1 -- start start 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa228a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa228a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22419cc30 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa221e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa221e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57706/0 (socket says 172.21.15.46:57706) 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.338+0000 7fa221e00700 1 -- 172.21.15.46:0/3258527127 learned_addr learned my addr 172.21.15.46:0/3258527127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa221e00700 1 -- 172.21.15.46:0/3258527127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214009550 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa221e00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa214011980 tx=0x7fa214011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa214003af0 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa214003c50 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22419ce30 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa214019760 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.339+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22419d2d0 con 0x7fa22407be90 2024-05-08T22:56:23.967 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.340+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa21401ab10 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.340+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22410cb70 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.341+0000 7fa21aa00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 0x7fa20803ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.341+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa21404cbf0 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.341+0000 7fa221400700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 0x7fa20803ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.342+0000 7fa221400700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 0x7fa20803ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa210006fd0 tx=0x7fa210006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.346+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa21401a430 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.673+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa22419fbc0 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.676+0000 7fa21aa00700 1 -- 172.21.15.46:0/3258527127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa21402db00 con 0x7fa22407be90 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 msgr2=0x7fa20803ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 0x7fa20803ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa210006fd0 tx=0x7fa210006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 msgr2=0x7fa22419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa214011980 tx=0x7fa214011a60 comp rx=0 tx=0).stop 2024-05-08T22:56:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 shutdown_connections 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa208038890 0x7fa20803ad50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 --2- 172.21.15.46:0/3258527127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa22407be90 0x7fa22419c6f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 >> 172.21.15.46:0/3258527127 conn(0x7fa224101d50 msgr2=0x7fa2241029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 shutdown_connections 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-08T22:56:23.679+0000 7fa228a00700 1 -- 172.21.15.46:0/3258527127 wait complete. 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.969 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:23.970 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-05-08T22:56:24.019 INFO:tasks.cephadm:Fetching config... 2024-05-08T22:56:24.019 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:56:24.019 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T22:56:24.062 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T22:56:24.062 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:56:24.062 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T22:56:24.146 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T22:56:24.146 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:56:24.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T22:56:24.256 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T22:56:24.256 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:56:24.256 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T22:56:24.341 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T22:56:24.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:56:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:24.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:56:24.554 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:24.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:56:24.614 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfFxUI1yGbyzzzrEixXNiGqiqV/Q9fVqM9ClS3ymcvm22o53u1oIDufxgKMzTs9BPTpS/IKc8VKSo95L5+E7oOD3dF4iFttXR4oKYm+Tyfbefa0ydgyg0Pf7GK8JfUpV6YkEYZvFd7HVInuMkfG9ps3PajSkB/0/YLXUXA3+nomenOjW8PSfYtdO1ufX3FPB4n5PiQ4aBG020qxCuNrSr1GYa7XLaWvpFCGtMMfn5unDGlAiJ2paRxkhDoE1u0SbBnZYptrds5r0A4vQcKaPNbHa69Owsbo0xQJWzmgBfvTkrp23v6q9oXakDCFz9Av0i2YUQhwvewp0b8LNRPW/iDSNBi+ioTmSFoXHq5EVpPcVyQUdCPK7A1eJZyt9RwoRC9sb/S215wqmD3RRuu3gV5VNT9wGlq/E4WRBe68oq9vXpilTS8RI1694a8D+uelpeo2BFkqwvgVmWptN36fIK/Rtc2axoY5nkSYFU8/7qayLJu8s02Ui7A6WQWPArGd38= ceph-ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:24.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T22:56:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 -- 172.21.15.46:0/1496104262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 msgr2=0x7f671c1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 --2- 172.21.15.46:0/1496104262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c1037b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6710009910 tx=0x7f6710009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 -- 172.21.15.46:0/1496104262 shutdown_connections 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 --2- 172.21.15.46:0/1496104262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c1037b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 -- 172.21.15.46:0/1496104262 >> 172.21.15.46:0/1496104262 conn(0x7f671c0fadd0 msgr2=0x7f671c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 -- 172.21.15.46:0/1496104262 shutdown_connections 2024-05-08T22:56:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.518+0000 7f6724400700 1 -- 172.21.15.46:0/1496104262 wait complete. 2024-05-08T22:56:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.519+0000 7f6724400700 1 Processor -- start 2024-05-08T22:56:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6724400700 1 -- start start 2024-05-08T22:56:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6724400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6724400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c196270 con 0x7f671c1013c0 2024-05-08T22:56:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6721a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6721a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57736/0 (socket says 172.21.15.46:57736) 2024-05-08T22:56:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.520+0000 7f6721a00700 1 -- 172.21.15.46:0/1415312936 learned_addr learned my addr 172.21.15.46:0/1415312936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.521+0000 7f6721a00700 1 -- 172.21.15.46:0/1415312936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100095f0 con 0x7f671c1013c0 2024-05-08T22:56:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.521+0000 7f6721a00700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6710005b40 tx=0x7f67100057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.521+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f671001d070 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.521+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671000be20 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.521+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c196470 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.522+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6710022d80 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.522+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c196910 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.523+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c18f650 con 0x7f671c1013c0 2024-05-08T22:56:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.523+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f67100225e0 con 0x7f671c1013c0 2024-05-08T22:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.523+0000 7f671a000700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 0x7f670403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.523+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f671004c920 con 0x7f671c1013c0 2024-05-08T22:56:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.524+0000 7f6721000700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 0x7f670403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.524+0000 7f6721000700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 0x7f670403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6708006fd0 tx=0x7f6708006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.529+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f671002a540 con 0x7f671c1013c0 2024-05-08T22:56:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.748+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f671c0625d0 con 0x7f671c1013c0 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.752+0000 7f671a000700 1 -- 172.21.15.46:0/1415312936 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f671c0625d0 con 0x7f671c1013c0 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 msgr2=0x7f670403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 0x7f670403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6708006fd0 tx=0x7f6708006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 msgr2=0x7f671c195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6710005b40 tx=0x7f67100057e0 comp rx=0 tx=0).stop 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 shutdown_connections 2024-05-08T22:56:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f67040387a0 0x7f670403ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:26.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 --2- 172.21.15.46:0/1415312936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f671c1013c0 0x7f671c195d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:26.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 >> 172.21.15.46:0/1415312936 conn(0x7f671c0fadd0 msgr2=0x7f671c0fb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:26.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 shutdown_connections 2024-05-08T22:56:26.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:26.755+0000 7f6724400700 1 -- 172.21.15.46:0/1415312936 wait complete. 2024-05-08T22:56:27.150 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T22:56:27.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T22:56:27.516 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:28.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:27 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1415312936' entity='client.admin' 2024-05-08T22:56:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 -- 172.21.15.46:0/25565716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 msgr2=0x7f59bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 --2- 172.21.15.46:0/25565716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc100d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009910 tx=0x7f59b0009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 -- 172.21.15.46:0/25565716 shutdown_connections 2024-05-08T22:56:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 --2- 172.21.15.46:0/25565716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc100d50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 -- 172.21.15.46:0/25565716 >> 172.21.15.46:0/25565716 conn(0x7f59bc0fbed0 msgr2=0x7f59bc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 -- 172.21.15.46:0/25565716 shutdown_connections 2024-05-08T22:56:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.959+0000 7f59c3800700 1 -- 172.21.15.46:0/25565716 wait complete. 2024-05-08T22:56:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.960+0000 7f59c3800700 1 Processor -- start 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.960+0000 7f59c3800700 1 -- start start 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c3800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c3800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1962f0 con 0x7f59bc100930 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c0e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c0e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57768/0 (socket says 172.21.15.46:57768) 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c0e00700 1 -- 172.21.15.46:0/2785930763 learned_addr learned my addr 172.21.15.46:0/2785930763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c0e00700 1 -- 172.21.15.46:0/2785930763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00095f0 con 0x7f59bc100930 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.961+0000 7f59c0e00700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004f70 tx=0x7f59b0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.962+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59b001d070 con 0x7f59bc100930 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.962+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b000be20 con 0x7f59bc100930 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.962+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59b0022df0 con 0x7f59bc100930 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.962+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc1964f0 con 0x7f59bc100930 2024-05-08T22:56:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.962+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc196990 con 0x7f59bc100930 2024-05-08T22:56:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.963+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc18f650 con 0x7f59bc100930 2024-05-08T22:56:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.963+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f59b000b960 con 0x7f59bc100930 2024-05-08T22:56:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.963+0000 7f59b9600700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 0x7f59a403ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.964+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f59b004cc70 con 0x7f59bc100930 2024-05-08T22:56:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.964+0000 7f59bbe00700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 0x7f59a403ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.964+0000 7f59bbe00700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 0x7f59a403ac40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:28.968+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f59b002bba0 con 0x7f59bc100930 2024-05-08T22:56:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.186+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f59bc0613e0 con 0x7f59a4038780 2024-05-08T22:56:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.190+0000 7f59b9600700 1 -- 172.21.15.46:0/2785930763 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59bc0613e0 con 0x7f59a4038780 2024-05-08T22:56:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 msgr2=0x7f59a403ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 0x7f59a403ac40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 msgr2=0x7f59bc195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:29.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004f70 tx=0x7f59b0004fa0 comp rx=0 tx=0).stop 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 shutdown_connections 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59a4038780 0x7f59a403ac40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 --2- 172.21.15.46:0/2785930763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc100930 0x7f59bc195db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 >> 172.21.15.46:0/2785930763 conn(0x7f59bc0fbed0 msgr2=0x7f59bc0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 shutdown_connections 2024-05-08T22:56:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:29.193+0000 7f59c3800700 1 -- 172.21.15.46:0/2785930763 wait complete. 2024-05-08T22:56:29.584 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-05-08T22:56:29.585 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:56:29.585 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-05-08T22:56:29.607 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:56:29.607 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:29.668 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-05-08T22:56:29.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch host add smithi053 2024-05-08T22:56:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:56:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:30.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:30 smithi046 ceph-mon[36898]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:30.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:30 smithi046 ceph-mon[36898]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T22:56:30.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:30 smithi046 ceph-mon[36898]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.162+0000 7fa845000700 1 -- 172.21.15.46:0/2076591635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 msgr2=0x7fa840073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.162+0000 7fa845000700 1 --2- 172.21.15.46:0/2076591635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa840073720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa830009910 tx=0x7fa830009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.163+0000 7fa845000700 1 -- 172.21.15.46:0/2076591635 shutdown_connections 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.163+0000 7fa845000700 1 --2- 172.21.15.46:0/2076591635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa840073720 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.163+0000 7fa845000700 1 -- 172.21.15.46:0/2076591635 >> 172.21.15.46:0/2076591635 conn(0x7fa8400fc3a0 msgr2=0x7fa8400fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.163+0000 7fa845000700 1 -- 172.21.15.46:0/2076591635 shutdown_connections 2024-05-08T22:56:31.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.163+0000 7fa845000700 1 -- 172.21.15.46:0/2076591635 wait complete. 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.164+0000 7fa845000700 1 Processor -- start 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.164+0000 7fa845000700 1 -- start start 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.164+0000 7fa845000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.164+0000 7fa845000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84019cc80 con 0x7fa840073340 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.165+0000 7fa83de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.165+0000 7fa83de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57782/0 (socket says 172.21.15.46:57782) 2024-05-08T22:56:31.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.165+0000 7fa83de00700 1 -- 172.21.15.46:0/1976705436 learned_addr learned my addr 172.21.15.46:0/1976705436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.165+0000 7fa83de00700 1 -- 172.21.15.46:0/1976705436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300095f0 con 0x7fa840073340 2024-05-08T22:56:31.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.165+0000 7fa83de00700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa830004f40 tx=0x7fa830005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.166+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa83001d070 con 0x7fa840073340 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.166+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84019cee0 con 0x7fa840073340 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.166+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84019d360 con 0x7fa840073340 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.166+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa830004370 con 0x7fa840073340 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.166+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa830003810 con 0x7fa840073340 2024-05-08T22:56:31.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.167+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa830003970 con 0x7fa840073340 2024-05-08T22:56:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.167+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8200052f0 con 0x7fa840073340 2024-05-08T22:56:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.168+0000 7fa836a00700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 0x7fa82403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.168+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa83004c420 con 0x7fa840073340 2024-05-08T22:56:31.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.168+0000 7fa83d400700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 0x7fa82403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:31.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.168+0000 7fa83d400700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 0x7fa82403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006fd0 tx=0x7fa82c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:31.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.174+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa83000f460 con 0x7fa840073340 2024-05-08T22:56:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:31.407+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) v1 -- 0x7fa820000bc0 con 0x7fa8240387f0 2024-05-08T22:56:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:31 smithi046 ceph-mon[36898]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:31 smithi046 ceph-mon[36898]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:56:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:32.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:33.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:32 smithi046 ceph-mon[36898]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.867+0000 7fa836a00700 1 -- 172.21.15.46:0/1976705436 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa820000bc0 con 0x7fa8240387f0 2024-05-08T22:56:33.866 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-05-08T22:56:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 msgr2=0x7fa82403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 0x7fa82403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006fd0 tx=0x7fa82c006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 msgr2=0x7fa84019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa830004f40 tx=0x7fa830005020 comp rx=0 tx=0).stop 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 shutdown_connections 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8240387f0 0x7fa82403acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 --2- 172.21.15.46:0/1976705436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa840073340 0x7fa84019c740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 >> 172.21.15.46:0/1976705436 conn(0x7fa8400fc3a0 msgr2=0x7fa8400fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 shutdown_connections 2024-05-08T22:56:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:33.870+0000 7fa845000700 1 -- 172.21.15.46:0/1976705436 wait complete. 2024-05-08T22:56:33.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:33 smithi046 ceph-mon[36898]: Deploying cephadm binary to smithi053 2024-05-08T22:56:34.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T22:56:34.525 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:34.974 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:34.974 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:34 smithi046 ceph-mon[36898]: Added host smithi053 2024-05-08T22:56:34.974 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:34.975 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:34.975 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:35.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 -- 172.21.15.46:0/4018874877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 msgr2=0x7efff0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:35.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 --2- 172.21.15.46:0/4018874877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effe0009910 tx=0x7effe0009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 -- 172.21.15.46:0/4018874877 shutdown_connections 2024-05-08T22:56:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 --2- 172.21.15.46:0/4018874877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 -- 172.21.15.46:0/4018874877 >> 172.21.15.46:0/4018874877 conn(0x7efff00fc3a0 msgr2=0x7efff00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 -- 172.21.15.46:0/4018874877 shutdown_connections 2024-05-08T22:56:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.627+0000 7efff6a00700 1 -- 172.21.15.46:0/4018874877 wait complete. 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.628+0000 7efff6a00700 1 Processor -- start 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.628+0000 7efff6a00700 1 -- start start 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7efff6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7efff6a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff0198890 con 0x7efff01068b0 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7effefe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7effefe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53362/0 (socket says 172.21.15.46:53362) 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7effefe00700 1 -- 172.21.15.46:0/2068090530 learned_addr learned my addr 172.21.15.46:0/2068090530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.629+0000 7effefe00700 1 -- 172.21.15.46:0/2068090530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00095f0 con 0x7efff01068b0 2024-05-08T22:56:35.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7effefe00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effe0006010 tx=0x7effe0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:35.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe001d070 con 0x7efff01068b0 2024-05-08T22:56:35.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe000bbb0 con 0x7efff01068b0 2024-05-08T22:56:35.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff0198af0 con 0x7efff01068b0 2024-05-08T22:56:35.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe0003810 con 0x7efff01068b0 2024-05-08T22:56:35.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.630+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff0198f70 con 0x7efff01068b0 2024-05-08T22:56:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.631+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7effe000bd20 con 0x7efff01068b0 2024-05-08T22:56:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.631+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff0101ff0 con 0x7efff01068b0 2024-05-08T22:56:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.631+0000 7effecc00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 0x7effd403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:35.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.632+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7effe004c430 con 0x7efff01068b0 2024-05-08T22:56:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.632+0000 7effef400700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 0x7effd403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:35.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.632+0000 7effef400700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 0x7effd403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effdc006fd0 tx=0x7effdc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.637+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7effe002aba0 con 0x7efff01068b0 2024-05-08T22:56:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.858+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efff010abb0 con 0x7effd40387f0 2024-05-08T22:56:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.860+0000 7effecc00700 1 -- 172.21.15.46:0/2068090530 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7efff010abb0 con 0x7effd40387f0 2024-05-08T22:56:35.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:35.859 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-05-08T22:56:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 msgr2=0x7effd403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 0x7effd403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effdc006fd0 tx=0x7effdc006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 msgr2=0x7efff0198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effe0006010 tx=0x7effe0004ff0 comp rx=0 tx=0).stop 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 shutdown_connections 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7effd40387f0 0x7effd403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 --2- 172.21.15.46:0/2068090530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff01068b0 0x7efff0198350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 >> 172.21.15.46:0/2068090530 conn(0x7efff00fc3a0 msgr2=0x7efff0108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.862+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 shutdown_connections 2024-05-08T22:56:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:35.863+0000 7efff6a00700 1 -- 172.21.15.46:0/2068090530 wait complete. 2024-05-08T22:56:36.179 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2024-05-08T22:56:36.179 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:56:36.180 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2024-05-08T22:56:36.200 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:56:36.200 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:36.260 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2024-05-08T22:56:36.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch host add smithi086 2024-05-08T22:56:36.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:36.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:36 smithi046 ceph-mon[36898]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:56:36.615 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 -- 172.21.15.46:0/4145288956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 msgr2=0x7faff4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 --2- 172.21.15.46:0/4145288956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4106c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009910 tx=0x7fafe4009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 -- 172.21.15.46:0/4145288956 shutdown_connections 2024-05-08T22:56:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 --2- 172.21.15.46:0/4145288956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4106c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 -- 172.21.15.46:0/4145288956 >> 172.21.15.46:0/4145288956 conn(0x7faff40fc3a0 msgr2=0x7faff40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 -- 172.21.15.46:0/4145288956 shutdown_connections 2024-05-08T22:56:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.684+0000 7faff8a00700 1 -- 172.21.15.46:0/4145288956 wait complete. 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.685+0000 7faff8a00700 1 Processor -- start 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.685+0000 7faff8a00700 1 -- start start 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.685+0000 7faff8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.685+0000 7faff8a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4198890 con 0x7faff41068b0 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.686+0000 7faff1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.686+0000 7faff1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53392/0 (socket says 172.21.15.46:53392) 2024-05-08T22:56:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.686+0000 7faff1e00700 1 -- 172.21.15.46:0/359642584 learned_addr learned my addr 172.21.15.46:0/359642584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.686+0000 7faff1e00700 1 -- 172.21.15.46:0/359642584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe40095f0 con 0x7faff41068b0 2024-05-08T22:56:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.686+0000 7faff1e00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.687+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafe401d070 con 0x7faff41068b0 2024-05-08T22:56:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.687+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe400bbb0 con 0x7faff41068b0 2024-05-08T22:56:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.687+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff4198af0 con 0x7faff41068b0 2024-05-08T22:56:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.687+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafe4003810 con 0x7faff41068b0 2024-05-08T22:56:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.687+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff4198f70 con 0x7faff41068b0 2024-05-08T22:56:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.688+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff4101ff0 con 0x7faff41068b0 2024-05-08T22:56:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.688+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fafe400bd20 con 0x7faff41068b0 2024-05-08T22:56:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.688+0000 7fafeaa00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 0x7fafd803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.689+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fafe404c410 con 0x7faff41068b0 2024-05-08T22:56:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.689+0000 7faff1400700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 0x7fafd803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:37.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.689+0000 7faff1400700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 0x7fafd803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fafe0006fd0 tx=0x7fafe0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:37.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.693+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fafe4029360 con 0x7faff41068b0 2024-05-08T22:56:37.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:37.920+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}) v1 -- 0x7faff410abb0 con 0x7fafd80387f0 2024-05-08T22:56:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:38 smithi046 ceph-mon[36898]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T22:56:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.230+0000 7fafeaa00700 1 -- 172.21.15.46:0/359642584 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7faff410abb0 con 0x7fafd80387f0 2024-05-08T22:56:40.230 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi086' with addr '172.21.15.86' 2024-05-08T22:56:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.232+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 msgr2=0x7fafd803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.232+0000 7faff8a00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 0x7fafd803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fafe0006fd0 tx=0x7fafe0006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.232+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 msgr2=0x7faff4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.232+0000 7faff8a00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe4004ff0 comp rx=0 tx=0).stop 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 shutdown_connections 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fafd80387f0 0x7fafd803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 --2- 172.21.15.46:0/359642584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff41068b0 0x7faff4198350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 >> 172.21.15.46:0/359642584 conn(0x7faff40fc3a0 msgr2=0x7faff4108430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 shutdown_connections 2024-05-08T22:56:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:40.233+0000 7faff8a00700 1 -- 172.21.15.46:0/359642584 wait complete. 2024-05-08T22:56:40.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: Deploying cephadm binary to smithi086 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:40.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:39 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:40.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T22:56:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:41.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:41.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: Added host smithi086 2024-05-08T22:56:41.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:41.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:41.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:41.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:41 smithi046 ceph-mon[36898]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.041+0000 7f3e79600700 1 -- 172.21.15.46:0/4098449573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 msgr2=0x7f3e74106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.041+0000 7f3e79600700 1 --2- 172.21.15.46:0/4098449573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e74106ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009910 tx=0x7f3e64009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.042+0000 7f3e79600700 1 -- 172.21.15.46:0/4098449573 shutdown_connections 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.042+0000 7f3e79600700 1 --2- 172.21.15.46:0/4098449573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e74106ca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.042+0000 7f3e79600700 1 -- 172.21.15.46:0/4098449573 >> 172.21.15.46:0/4098449573 conn(0x7f3e74075350 msgr2=0x7f3e74075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.042+0000 7f3e79600700 1 -- 172.21.15.46:0/4098449573 shutdown_connections 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.042+0000 7f3e79600700 1 -- 172.21.15.46:0/4098449573 wait complete. 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e79600700 1 Processor -- start 2024-05-08T22:56:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e79600700 1 -- start start 2024-05-08T22:56:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e79600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e79600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e741988f0 con 0x7f3e741068c0 2024-05-08T22:56:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e72a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e72a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45332/0 (socket says 172.21.15.46:45332) 2024-05-08T22:56:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.043+0000 7f3e72a00700 1 -- 172.21.15.46:0/3903103291 learned_addr learned my addr 172.21.15.46:0/3903103291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e72a00700 1 -- 172.21.15.46:0/3903103291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640095f0 con 0x7f3e741068c0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e72a00700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006010 tx=0x7f3e64004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6401d070 con 0x7f3e741068c0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6400bbb0 con 0x7f3e741068c0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74198b50 con 0x7f3e741068c0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.044+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e64003810 con 0x7f3e741068c0 2024-05-08T22:56:42.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.045+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74198fd0 con 0x7f3e741068c0 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.046+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f3e6400bd20 con 0x7f3e741068c0 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.045+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e74102000 con 0x7f3e741068c0 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.046+0000 7f3e6b400700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 0x7f3e5803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.046+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e6404c410 con 0x7f3e741068c0 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.046+0000 7f3e72000700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 0x7f3e5803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.047+0000 7f3e72000700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 0x7f3e5803acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006fd0 tx=0x7f3e60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.052+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3e64029360 con 0x7f3e741068c0 2024-05-08T22:56:42.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.271+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e7410abb0 con 0x7f3e580387f0 2024-05-08T22:56:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.272+0000 7f3e6b400700 1 -- 172.21.15.46:0/3903103291 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f3e7410abb0 con 0x7f3e580387f0 2024-05-08T22:56:42.271 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T22:56:42.271 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2024-05-08T22:56:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 msgr2=0x7f3e5803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 0x7f3e5803acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006fd0 tx=0x7f3e60006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 msgr2=0x7f3e741983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:42.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006010 tx=0x7f3e64004ff0 comp rx=0 tx=0).stop 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 shutdown_connections 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3e580387f0 0x7f3e5803acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 --2- 172.21.15.46:0/3903103291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741068c0 0x7f3e741983b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 >> 172.21.15.46:0/3903103291 conn(0x7f3e74075350 msgr2=0x7f3e740fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 shutdown_connections 2024-05-08T22:56:42.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:42.275+0000 7f3e79600700 1 -- 172.21.15.46:0/3903103291 wait complete. 2024-05-08T22:56:42.676 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T22:56:42.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T22:56:43.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:42 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:56:43.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:43 smithi046 ceph-mon[36898]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:56:43.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:43 smithi046 ceph-mon[36898]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 -- 172.21.15.46:0/3180720825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 msgr2=0x7f5694106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 --2- 172.21.15.46:0/3180720825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694106c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5684009910 tx=0x7f5684009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 -- 172.21.15.46:0/3180720825 shutdown_connections 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 --2- 172.21.15.46:0/3180720825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694106c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 -- 172.21.15.46:0/3180720825 >> 172.21.15.46:0/3180720825 conn(0x7f56940fc3a0 msgr2=0x7f56940fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 -- 172.21.15.46:0/3180720825 shutdown_connections 2024-05-08T22:56:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.071+0000 7f569a000700 1 -- 172.21.15.46:0/3180720825 wait complete. 2024-05-08T22:56:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.072+0000 7f569a000700 1 Processor -- start 2024-05-08T22:56:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.072+0000 7f569a000700 1 -- start start 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.072+0000 7f569a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.072+0000 7f569a000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5694198890 con 0x7f56941068b0 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.073+0000 7f5693400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.073+0000 7f5693400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45352/0 (socket says 172.21.15.46:45352) 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.073+0000 7f5693400700 1 -- 172.21.15.46:0/1116178102 learned_addr learned my addr 172.21.15.46:0/1116178102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.073+0000 7f5693400700 1 -- 172.21.15.46:0/1116178102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840095f0 con 0x7f56941068b0 2024-05-08T22:56:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.073+0000 7f5693400700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5684006010 tx=0x7f5684004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.074+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f568401d070 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.074+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f568400bbb0 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.074+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5694198af0 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.074+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5684003810 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.074+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5694198f70 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.075+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f568400b6f0 con 0x7f56941068b0 2024-05-08T22:56:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.075+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5694101ff0 con 0x7f56941068b0 2024-05-08T22:56:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.075+0000 7f568be00700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 0x7f567803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.075+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f568404c450 con 0x7f56941068b0 2024-05-08T22:56:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.076+0000 7f5692a00700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 0x7f567803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.076+0000 7f5692a00700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 0x7f567803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.080+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5684029540 con 0x7f56941068b0 2024-05-08T22:56:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:44.308+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f56940625d0 con 0x7f56941068b0 2024-05-08T22:56:45.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.044+0000 7f568be00700 1 -- 172.21.15.46:0/1116178102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5684026020 con 0x7f56941068b0 2024-05-08T22:56:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 msgr2=0x7f567803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 0x7f567803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 msgr2=0x7f5694198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5684006010 tx=0x7f5684004ff0 comp rx=0 tx=0).stop 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 shutdown_connections 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f56780387f0 0x7f567803acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 --2- 172.21.15.46:0/1116178102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56941068b0 0x7f5694198350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 >> 172.21.15.46:0/1116178102 conn(0x7f56940fc3a0 msgr2=0x7f5694108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 shutdown_connections 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:56:45.047+0000 7f569a000700 1 -- 172.21.15.46:0/1116178102 wait complete. 2024-05-08T22:56:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:45.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:45.428 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-05-08T22:56:45.428 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-05-08T22:56:45.428 INFO:tasks.cephadm:Adding mon.c on smithi086 2024-05-08T22:56:45.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c' 2024-05-08T22:56:45.677 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:46.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:46 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:56:46.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:46 smithi046 ceph-mon[36898]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 -- 172.21.15.86:0/220509656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 msgr2=0x7fd148105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 --2- 172.21.15.86:0/220509656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148105bd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd144009910 tx=0x7fd144009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 -- 172.21.15.86:0/220509656 shutdown_connections 2024-05-08T22:56:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 --2- 172.21.15.86:0/220509656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148105bd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 -- 172.21.15.86:0/220509656 >> 172.21.15.86:0/220509656 conn(0x7fd1480fb2e0 msgr2=0x7fd1480fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 -- 172.21.15.86:0/220509656 shutdown_connections 2024-05-08T22:56:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.674+0000 7fd14fdae700 1 -- 172.21.15.86:0/220509656 wait complete. 2024-05-08T22:56:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14fdae700 1 Processor -- start 2024-05-08T22:56:46.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14fdae700 1 -- start start 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14fdae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14fdae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1481988b0 con 0x7fd1481057f0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14db4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14db4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.86:49168/0 (socket says 172.21.15.86:49168) 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14db4a700 1 -- 172.21.15.86:0/2702219915 learned_addr learned my addr 172.21.15.86:0/2702219915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.675+0000 7fd14db4a700 1 -- 172.21.15.86:0/2702219915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440095f0 con 0x7fd1481057f0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd14db4a700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd144004f40 tx=0x7fd144005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14401d070 con 0x7fd1481057f0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14400bbb0 con 0x7fd1481057f0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd148198b10 con 0x7fd1481057f0 2024-05-08T22:56:46.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd144003810 con 0x7fd1481057f0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.676+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd148198f90 con 0x7fd1481057f0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.677+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd14400bd20 con 0x7fd1481057f0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.677+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd148100f30 con 0x7fd1481057f0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.677+0000 7fd13effd700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 0x7fd13403ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.677+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd14404c480 con 0x7fd1481057f0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.677+0000 7fd14d349700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 0x7fd13403ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:46.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.678+0000 7fd14d349700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 0x7fd13403ac50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd138006fd0 tx=0x7fd138006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:46.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.683+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd14402ab60 con 0x7fd1481057f0 2024-05-08T22:56:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.882+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fd148108a90 con 0x7fd134038790 2024-05-08T22:56:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.889+0000 7fd13effd700 1 -- 172.21.15.86:0/2702219915 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd148108a90 con 0x7fd134038790 2024-05-08T22:56:46.888 INFO:teuthology.orchestra.run.smithi086.stdout:Scheduled mon update... 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 msgr2=0x7fd13403ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 0x7fd13403ac50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd138006fd0 tx=0x7fd138006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 msgr2=0x7fd148198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd144004f40 tx=0x7fd144005020 comp rx=0 tx=0).stop 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 shutdown_connections 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd134038790 0x7fd13403ac50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 --2- 172.21.15.86:0/2702219915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1481057f0 0x7fd148198370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 >> 172.21.15.86:0/2702219915 conn(0x7fd1480fb2e0 msgr2=0x7fd1481073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 shutdown_connections 2024-05-08T22:56:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:46.891+0000 7fd14fdae700 1 -- 172.21.15.86:0/2702219915 wait complete. 2024-05-08T22:56:47.260 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T22:56:47.263 DEBUG:teuthology.orchestra.run.smithi086:mon.c> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T22:56:47.265 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:56:47.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:56:47.295 INFO:journalctl@ceph.mon.c.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T22:56:47.302 INFO:journalctl@ceph.mon.b.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T22:56:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:48.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:48.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:48.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c;count:3 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 -- 172.21.15.86:0/1643499486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 msgr2=0x7f4058106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 --2- 172.21.15.86:0/1643499486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f4058106ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4054009910 tx=0x7f4054009c20 comp rx=0 tx=0).stop 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 -- 172.21.15.86:0/1643499486 shutdown_connections 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 --2- 172.21.15.86:0/1643499486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f4058106ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 -- 172.21.15.86:0/1643499486 >> 172.21.15.86:0/1643499486 conn(0x7f40580fc410 msgr2=0x7f40580fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 -- 172.21.15.86:0/1643499486 shutdown_connections 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.162+0000 7f40600d1700 1 -- 172.21.15.86:0/1643499486 wait complete. 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.163+0000 7f40600d1700 1 Processor -- start 2024-05-08T22:56:49.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.163+0000 7f40600d1700 1 -- start start 2024-05-08T22:56:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.163+0000 7f40600d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.163+0000 7f40600d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581966e0 con 0x7f4058106900 2024-05-08T22:56:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f405de6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f405de6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.86:49194/0 (socket says 172.21.15.86:49194) 2024-05-08T22:56:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f405de6d700 1 -- 172.21.15.86:0/290320164 learned_addr learned my addr 172.21.15.86:0/290320164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:56:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f405de6d700 1 -- 172.21.15.86:0/290320164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540095f0 con 0x7f4058106900 2024-05-08T22:56:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f405de6d700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4054006010 tx=0x7f4054004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f405401d070 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405400bbb0 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4054003810 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.164+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4058196940 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.165+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4058196dc0 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.165+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4058104820 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.165+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f405400bd20 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.166+0000 7f404effd700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 0x7f404403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.166+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f405404c3b0 con 0x7f4058106900 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.166+0000 7f405d66c700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 0x7f404403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.166+0000 7f405d66c700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 0x7f404403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4048006fd0 tx=0x7f4048006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.171+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4054029360 con 0x7f4058106900 2024-05-08T22:56:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:48 smithi046 ceph-mon[36898]: Deploying daemon mon.c on smithi086 2024-05-08T22:56:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.457+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4058102040 con 0x7f4058106900 2024-05-08T22:56:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.458+0000 7f404effd700 1 -- 172.21.15.86:0/290320164 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f4054026070 con 0x7f4058106900 2024-05-08T22:56:49.458 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:56:49.458 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","modified":"2024-05-08T22:55:16.305587Z","created":"2024-05-08T22:55:16.305587Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T22:56:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 msgr2=0x7f404403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 0x7f404403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4048006fd0 tx=0x7f4048006e40 comp rx=0 tx=0).stop 2024-05-08T22:56:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 msgr2=0x7f40581961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4054006010 tx=0x7f4054004ff0 comp rx=0 tx=0).stop 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 shutdown_connections 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f40440387a0 0x7f404403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 --2- 172.21.15.86:0/290320164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4058106900 0x7f40581961a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 >> 172.21.15.86:0/290320164 conn(0x7f40580fc410 msgr2=0x7f4058108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 shutdown_connections 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:49.461+0000 7f40600d1700 1 -- 172.21.15.86:0/290320164 wait complete. 2024-05-08T22:56:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:dumped monmap epoch 1 2024-05-08T22:56:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 systemd[1]: Starting Ceph mon.c for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:56:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:49 smithi046 ceph-mon[36898]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:49 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/290320164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: pidfile_write: ignore empty --pid-file 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: load: jerasure load: lrc 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Git sha 0 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: DB SUMMARY 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: DB Session ID: 5WB6DSVG6GV15LBS7TJ2 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:56:50.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.env: 0x55e76f4d0820 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.info_log: 0x55e7716ff520 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.statistics: (nil) 2024-05-08T22:56:50.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.use_fsync: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.db_log_dir: 2024-05-08T22:56:50.717 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.wal_dir: 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.write_buffer_manager: 0x55e770994320 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:56:50.718 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.unordered_write: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.row_cache: None 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.wal_filter: None 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:56:50.719 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.wal_compression: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:56:50.720 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_open_files: -1 2024-05-08T22:56:50.721 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Compression algorithms supported: 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kZSTD supported: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:56:50.722 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.merge_operator: 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_filter: None 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e7716ff620) 2024-05-08T22:56:50.723 INFO:journalctl@ceph.mon.c.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: index_type: 0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: data_block_index_type: 0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: index_shortening: 1 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: checksum: 4 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: no_block_cache: 0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_cache: 0x55e770a171f0 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_cache_options: 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: capacity : 536870912 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: num_shard_bits : 4 2024-05-08T22:56:50.724 INFO:journalctl@ceph.mon.c.smithi086.stdout: strict_capacity_limit : 0 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_cache_compressed: (nil) 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: persistent_cache: (nil) 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_size: 4096 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_size_deviation: 10 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_restart_interval: 16 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: index_block_restart_interval: 1 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: metadata_block_size: 4096 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: partition_filters: 0 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: use_delta_encoding: 1 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: filter_policy: bloomfilter 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: whole_key_filtering: 1 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: verify_compression: 0 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: format_version: 5 2024-05-08T22:56:50.725 INFO:journalctl@ceph.mon.c.smithi086.stdout: enable_index_compression: 1 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout: block_align: 0 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout: max_auto_readahead_size: 262144 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout: prepopulate_block_cache: 0 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression: NoCompression 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:56:50.726 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.num_levels: 7 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:56:50.727 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:56:50.728 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:56:50.729 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:56:50.730 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.ttl: 2592000 2024-05-08T22:56:50.731 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T22:56:50.732 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b78741bc-bde0-4788-b146-e3b1efd01c9e 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209010454308, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209010454705, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715209010, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b78741bc-bde0-4788-b146-e3b1efd01c9e", "db_session_id": "5WB6DSVG6GV15LBS7TJ2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209010454772, "job": 1, "event": "recovery_finished"} 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e770ab4000 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: DB pointer 0x55e770aa2000 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout: ** DB Stats ** 2024-05-08T22:56:50.733 INFO:journalctl@ceph.mon.c.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: ** Compaction Stats [default] ** 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:50.734 INFO:journalctl@ceph.mon.c.smithi086.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: ** Compaction Stats [default] ** 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:56:50.735 INFO:journalctl@ceph.mon.c.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: Block cache BinnedLRUCache@0x55e770a171f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: using public_addr v2:172.21.15.86:0/0 -> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: starting mon.c rank -1 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:50.736 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(???) e0 preinit fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 bash[78573]: 99da62d8562690d18ea7838af25f018da08c72a1df7d9bf22cbdbd9097a48e4a 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 systemd[1]: Started Ceph mon.c for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: e1 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: legacy client fscid: -1 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout: No filesystems configured 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T22:56:50.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.738 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:50.739 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c;count:3 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: Deploying daemon mon.c on smithi086 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/290320164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:50.740 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:50 smithi086 ceph-mon[78610]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T22:56:50.854 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:56:50.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:56:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:56:55.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 -- 172.21.15.86:0/2328343659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 msgr2=0x7f447c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2328343659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f447c005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44940ff800 tx=0x7f4488014ec0 comp rx=0 tx=0).stop 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 -- 172.21.15.86:0/2328343659 shutdown_connections 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2328343659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f447c005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 -- 172.21.15.86:0/2328343659 >> 172.21.15.86:0/2328343659 conn(0x7f44940fb400 msgr2=0x7f44940fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.789+0000 7f4498d5c700 1 -- 172.21.15.86:0/2328343659 shutdown_connections 2024-05-08T22:56:55.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.790+0000 7f4498d5c700 1 -- 172.21.15.86:0/2328343659 wait complete. 2024-05-08T22:56:55.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 Processor -- start 2024-05-08T22:56:55.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 -- start start 2024-05-08T22:56:55.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f4494198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494198f60 con 0x7f447c0035c0 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4498d5c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44941990d0 con 0x7f44941989d0 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37042/0 (socket says 172.21.15.86:37042) 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 -- 172.21.15.86:0/2947725413 learned_addr learned my addr 172.21.15.86:0/2947725413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 -- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 msgr2=0x7f449419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 -- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 msgr2=0x7f449419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.791+0000 7f4491d9b700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f449259c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f4494198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:55.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f449259c700 1 -- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 msgr2=0x7f449419cd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f449259c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f449259c700 1 -- 172.21.15.86:0/2947725413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488014910 con 0x7f447c0035c0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f449259c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f4494198490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4488001eb0 tx=0x7f448800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.792+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4488023070 con 0x7f447c0035c0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.793+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4488004b90 con 0x7f447c0035c0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.793+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449419d320 con 0x7f447c0035c0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.793+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4488019040 con 0x7f447c0035c0 2024-05-08T22:56:55.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.793+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449419d870 con 0x7f447c0035c0 2024-05-08T22:56:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.794+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4488012450 con 0x7f447c0035c0 2024-05-08T22:56:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.794+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494101010 con 0x7f447c0035c0 2024-05-08T22:56:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.794+0000 7f44837fe700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 0x7f447403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:56:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.794+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4488000e40 con 0x7f447c0035c0 2024-05-08T22:56:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.794+0000 7f4491d9b700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 0x7f447403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:56:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.795+0000 7f4491d9b700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 0x7f447403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4484009180 tx=0x7f4484008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:56:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:55.800+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4488011440 con 0x7f447c0035c0 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: mon.a calling monitor election 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: mon.c calling monitor election 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: fsmap 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: mgrmap e13: x(active, since 38s) 2024-05-08T22:56:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: overall HEALTH_OK 2024-05-08T22:56:55.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:55.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:55.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:55.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:55.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: mon.a calling monitor election 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: mon.c calling monitor election 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: fsmap 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: mgrmap e13: x(active, since 38s) 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: overall HEALTH_OK 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.073+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f44941997a0 con 0x7f447c0035c0 2024-05-08T22:56:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.074+0000 7f44837fe700 1 -- 172.21.15.86:0/2947725413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f448801d3d0 con 0x7f447c0035c0 2024-05-08T22:56:56.073 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:56:56.073 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","modified":"2024-05-08T22:56:50.487732Z","created":"2024-05-08T22:55:16.305587Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 msgr2=0x7f447403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 0x7f447403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4484009180 tx=0x7f4484008040 comp rx=0 tx=0).stop 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 msgr2=0x7f4494198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f4494198490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4488001eb0 tx=0x7f448800fcd0 comp rx=0 tx=0).stop 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 shutdown_connections 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44740385e0 0x7f447403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c0035c0 0x7f4494198490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 --2- 172.21.15.86:0/2947725413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44941989d0 0x7f449419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 >> 172.21.15.86:0/2947725413 conn(0x7f44940fb400 msgr2=0x7f44940fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 shutdown_connections 2024-05-08T22:56:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:56:56.076+0000 7f4498d5c700 1 -- 172.21.15.86:0/2947725413 wait complete. 2024-05-08T22:56:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:dumped monmap epoch 2 2024-05-08T22:56:56.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:56 smithi086 ceph-mon[78610]: Deploying daemon mon.b on smithi053 2024-05-08T22:56:56.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/2947725413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:56.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:56 smithi046 ceph-mon[36898]: Deploying daemon mon.b on smithi053 2024-05-08T22:56:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/2947725413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:57.490 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:56:57.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:56:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:56:57.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:56:57 smithi046 ceph-mon[36898]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:57.777 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 22:56:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T22:56:57.487+0000 7f2a03800700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T22:56:57.871 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:56:57 smithi086 ceph-mon[78610]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 systemd[1]: Started Ceph mon.b for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:56:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T22:56:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T22:56:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pidfile_write: ignore empty --pid-file 2024-05-08T22:56:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: load: jerasure load: lrc 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Git sha 0 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: DB SUMMARY 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: DB Session ID: FFE3OQKQRLAQKHZ6SEB3 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:56:58.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.env: 0x56469a7fa820 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.info_log: 0x56469c951520 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.statistics: (nil) 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.use_fsync: 0 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:56:58.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.db_log_dir: 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.wal_dir: 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:56:58.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.write_buffer_manager: 0x56469bbe6320 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:56:58.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.unordered_write: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.row_cache: None 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.wal_filter: None 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.wal_compression: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:56:58.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:56:58.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:56:58.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_open_files: -1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Compression algorithms supported: 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kZSTD supported: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:56:58.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.merge_operator: 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_filter: None 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56469c951620) 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:56:58.772 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_type: 0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_index_type: 0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_shortening: 1 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: checksum: 4 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: no_block_cache: 0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache: 0x56469bc691f0 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_options: 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: capacity : 536870912 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_shard_bits : 4 2024-05-08T22:56:58.773 INFO:journalctl@ceph.mon.b.smithi053.stdout: strict_capacity_limit : 0 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_compressed: (nil) 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: persistent_cache: (nil) 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size: 4096 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size_deviation: 10 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_restart_interval: 16 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_block_restart_interval: 1 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: metadata_block_size: 4096 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: partition_filters: 0 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: use_delta_encoding: 1 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: filter_policy: bloomfilter 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: whole_key_filtering: 1 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: verify_compression: 0 2024-05-08T22:56:58.774 INFO:journalctl@ceph.mon.b.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: format_version: 5 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_index_compression: 1 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_align: 0 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: max_auto_readahead_size: 262144 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: prepopulate_block_cache: 0 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression: NoCompression 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.num_levels: 7 2024-05-08T22:56:58.775 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:56:58.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:56:58.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:56:58.778 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:56:58.779 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.ttl: 2592000 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:56:58.780 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T22:56:58.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ff8fad0e-d142-44ca-b285-9a656d491048 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209018325230, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209018325715, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715209018, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ff8fad0e-d142-44ca-b285-9a656d491048", "db_session_id": "FFE3OQKQRLAQKHZ6SEB3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209018325800, "job": 1, "event": "recovery_finished"} 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56469bd06000 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: DB pointer 0x56469bcf4000 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** DB Stats ** 2024-05-08T22:56:58.782 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:58.783 INFO:journalctl@ceph.mon.b.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:56:58.784 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache BinnedLRUCache@0x56469bc691f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: using public_addr v2:172.21.15.53:0/0 -> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: starting mon.b rank -1 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(???) e0 preinit fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T22:56:58.785 INFO:journalctl@ceph.mon.b.smithi053.stdout: e1 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout: legacy client fscid: -1 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout: No filesystems configured 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:58.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1116178102' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.788 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c;count:3 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Deploying daemon mon.c on smithi086 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/290320164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:56:58.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.a calling monitor election 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.c calling monitor election 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: fsmap 2024-05-08T22:56:58.790 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mgrmap e13: x(active, since 38s) 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: overall HEALTH_OK 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: Deploying daemon mon.b on smithi053 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/2947725413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:56:58.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:56:58 smithi053 ceph-mon[77587]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T22:57:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:57:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: mon.a calling monitor election 2024-05-08T22:57:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: mon.c calling monitor election 2024-05-08T22:57:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: mon.b calling monitor election 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: fsmap 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:57:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: mgrmap e13: x(active, since 46s) 2024-05-08T22:57:03.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: overall HEALTH_OK 2024-05-08T22:57:03.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:57:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: mon.a calling monitor election 2024-05-08T22:57:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: mon.c calling monitor election 2024-05-08T22:57:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: mon.b calling monitor election 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: fsmap 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:57:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: mgrmap e13: x(active, since 46s) 2024-05-08T22:57:03.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: overall HEALTH_OK 2024-05-08T22:57:03.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:57:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: mon.a calling monitor election 2024-05-08T22:57:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: mon.c calling monitor election 2024-05-08T22:57:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: mon.b calling monitor election 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:57:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: fsmap 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: mgrmap e13: x(active, since 46s) 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: overall HEALTH_OK 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:03.780 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:04.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:04 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:04 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:04.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:04.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:57:04.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:04.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:04 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.777+0000 7f3964752700 1 -- 172.21.15.86:0/518377174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39480062a0 msgr2=0x7f3948008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.777+0000 7f3964752700 1 --2- 172.21.15.86:0/518377174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39480062a0 0x7f3948008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f395c100980 tx=0x7f395000fe90 comp rx=0 tx=0).stop 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.777+0000 7f3964752700 1 -- 172.21.15.86:0/518377174 shutdown_connections 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.777+0000 7f3964752700 1 --2- 172.21.15.86:0/518377174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39480062a0 0x7f3948008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.777+0000 7f3964752700 1 -- 172.21.15.86:0/518377174 >> 172.21.15.86:0/518377174 conn(0x7f395c076570 msgr2=0x7f395c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- 172.21.15.86:0/518377174 shutdown_connections 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- 172.21.15.86:0/518377174 wait complete. 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 Processor -- start 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- start start 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:04.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 0x7f395c1a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c1a2d80 con 0x7f395c1a0300 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c1a2ef0 con 0x7f395c1977d0 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.778+0000 7f3964752700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c1a30b0 con 0x7f39480062a0 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34426/0 (socket says 172.21.15.86:34426) 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 -- 172.21.15.86:0/289056061 learned_addr learned my addr 172.21.15.86:0/289056061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 0x7f395c1a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 msgr2=0x7f395c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 msgr2=0x7f395c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f39624ee700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:57:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3961ced700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3961ced700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 msgr2=0x7f395c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3961ced700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 msgr2=0x7f395c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3961ced700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3961ced700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 msgr2=0x7f395c197290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 msgr2=0x7f395c197c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 -- 172.21.15.86:0/289056061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395000f8e0 con 0x7f395c1a0300 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.779+0000 7f3962cef700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 0x7f395c1a26f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f395400bd50 tx=0x7f395400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.780+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395400cd10 con 0x7f395c1a0300 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.780+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c1a3390 con 0x7f395c1a0300 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.780+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f395400ce70 con 0x7f395c1a0300 2024-05-08T22:57:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.780+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954012840 con 0x7f395c1a0300 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.780+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c1a3890 con 0x7f395c1a0300 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.781+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c18ef90 con 0x7f395c1a0300 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.781+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f39540129c0 con 0x7f395c1a0300 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.781+0000 7f394f7fe700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 0x7f394003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.781+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f395404ae80 con 0x7f395c1a0300 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.781+0000 7f39624ee700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 0x7f394003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:04.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.782+0000 7f39624ee700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 0x7f394003b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3950005b40 tx=0x7f3950005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:04.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:04.787+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f395401d3b0 con 0x7f395c1a0300 2024-05-08T22:57:05.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.068+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f395c068b90 con 0x7f395c1a0300 2024-05-08T22:57:05.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.068+0000 7f394f7fe700 1 -- 172.21.15.86:0/289056061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1268 (secure 0 0 0) 0x7f3954019030 con 0x7f395c1a0300 2024-05-08T22:57:05.067 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-08T22:57:05.068 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","modified":"2024-05-08T22:56:58.351182Z","created":"2024-05-08T22:55:16.305587Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 msgr2=0x7f394003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 0x7f394003b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3950005b40 tx=0x7f3950005a90 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 msgr2=0x7f395c1a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 0x7f395c1a26f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f395400bd50 tx=0x7f395400be30 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 shutdown_connections 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3940038c70 0x7f394003b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c1a0300 0x7f395c1a26f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39480062a0 0x7f395c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 --2- 172.21.15.86:0/289056061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f395c1977d0 0x7f395c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 >> 172.21.15.86:0/289056061 conn(0x7f395c076570 msgr2=0x7f395c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:05.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 shutdown_connections 2024-05-08T22:57:05.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:05.070+0000 7f3964752700 1 -- 172.21.15.86:0/289056061 wait complete. 2024-05-08T22:57:05.071 INFO:teuthology.orchestra.run.smithi086.stderr:dumped monmap epoch 3 2024-05-08T22:57:05.481 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T22:57:05.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:05 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/289056061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:05 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/289056061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:57:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T22:57:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T22:57:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T22:57:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:05 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/289056061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:57:05.852 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:57:06.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:06.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:06.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:06 smithi046 ceph-mon[36898]: Reconfiguring daemon mon.a on smithi046 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:06 smithi053 ceph-mon[77587]: Reconfiguring daemon mon.a on smithi046 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:57:06.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:06.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:06.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:06.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:06 smithi086 ceph-mon[78610]: Reconfiguring daemon mon.a on smithi046 2024-05-08T22:57:06.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 -- 172.21.15.46:0/144351582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3681068b0 msgr2=0x7fe368106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:06.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 --2- 172.21.15.46:0/144351582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3681068b0 0x7fe368106c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009910 tx=0x7fe35c009c20 comp rx=0 tx=0).stop 2024-05-08T22:57:06.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 -- 172.21.15.46:0/144351582 shutdown_connections 2024-05-08T22:57:06.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 --2- 172.21.15.46:0/144351582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3681068b0 0x7fe368106c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:06.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 -- 172.21.15.46:0/144351582 >> 172.21.15.46:0/144351582 conn(0x7fe3680fc3a0 msgr2=0x7fe3680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:06.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 -- 172.21.15.46:0/144351582 shutdown_connections 2024-05-08T22:57:06.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.978+0000 7fe36fc00700 1 -- 172.21.15.46:0/144351582 wait complete. 2024-05-08T22:57:06.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.980+0000 7fe36fc00700 1 Processor -- start 2024-05-08T22:57:06.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.980+0000 7fe36fc00700 1 -- start start 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3681068b0 0x7fe3681984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3681989f0 0x7fe36819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681990e0 con 0x7fe36819d2e0 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681a5050 con 0x7fe3681989f0 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681a5190 con 0x7fe3681068b0 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44000/0 (socket says 172.21.15.46:44000) 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 -- 172.21.15.46:0/2250559102 learned_addr learned my addr 172.21.15.46:0/2250559102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:57:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36d200700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3681068b0 0x7fe3681984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe367e00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3681989f0 0x7fe36819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 -- 172.21.15.46:0/2250559102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3681068b0 msgr2=0x7fe3681984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3681068b0 0x7fe3681984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 -- 172.21.15.46:0/2250559102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3681989f0 msgr2=0x7fe36819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3681989f0 0x7fe36819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.981+0000 7fe36dc00700 1 -- 172.21.15.46:0/2250559102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0095f0 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe36dc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe36000ba50 tx=0x7fe36000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000c930 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe360014440 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3681a5470 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600075b0 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.982+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681a59c0 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.983+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368101ff0 con 0x7fe36819d2e0 2024-05-08T22:57:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.983+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fe36000ca90 con 0x7fe36819d2e0 2024-05-08T22:57:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.984+0000 7fe365600700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 0x7fe35003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.984+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe36004aa00 con 0x7fe36819d2e0 2024-05-08T22:57:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.984+0000 7fe36d200700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 0x7fe35003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.985+0000 7fe36d200700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 0x7fe35003b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe35c004690 tx=0x7fe35c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:06.989+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3600148b0 con 0x7fe36819d2e0 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.208+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe3680625d0 con 0x7fe36819d2e0 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.208+0000 7fe365600700 1 -- 172.21.15.46:0/2250559102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7fe360012340 con 0x7fe36819d2e0 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T22:57:07.208 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 msgr2=0x7fe35003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 0x7fe35003b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe35c004690 tx=0x7fe35c005fb0 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 msgr2=0x7fe36819d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe36000ba50 tx=0x7fe36000bd60 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 shutdown_connections 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe350038c70 0x7fe35003b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36819d2e0 0x7fe36819d790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3681068b0 0x7fe3681984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 --2- 172.21.15.46:0/2250559102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3681989f0 0x7fe36819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 >> 172.21.15.46:0/2250559102 conn(0x7fe3680fc3a0 msgr2=0x7fe368073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 shutdown_connections 2024-05-08T22:57:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:07.211+0000 7fe36fc00700 1 -- 172.21.15.46:0/2250559102 wait complete. 2024-05-08T22:57:07.596 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T22:57:07.596 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:57:07.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2250559102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:07.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.664 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:57:07.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2250559102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:07.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.770 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:57:07.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:57:07.809 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:57:07.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:57:07.887 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:57:07.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:57:07.918 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:07.918 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2250559102' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:07.919 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:07.924 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:57:07.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:57:07.995 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-05-08T22:57:07.996 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-05-08T22:57:07.996 INFO:tasks.cephadm:Adding mgr.z on smithi086 2024-05-08T22:57:07.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi053=y;smithi086=z' 2024-05-08T22:57:08.252 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:57:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:57:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: Reconfiguring daemon mon.b on smithi053 2024-05-08T22:57:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: Reconfiguring daemon mon.b on smithi053 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:08.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:08.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:57:08.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: Reconfiguring daemon mon.b on smithi053 2024-05-08T22:57:08.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:08.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:57:08.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:57:08.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 -- 172.21.15.86:0/1244180831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 msgr2=0x7fca980ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1244180831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca980ffd60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009910 tx=0x7fca8c009c20 comp rx=0 tx=0).stop 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 -- 172.21.15.86:0/1244180831 shutdown_connections 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1244180831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca980ffd60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 -- 172.21.15.86:0/1244180831 >> 172.21.15.86:0/1244180831 conn(0x7fca980faee0 msgr2=0x7fca980fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 -- 172.21.15.86:0/1244180831 shutdown_connections 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.471+0000 7fca9ee65700 1 -- 172.21.15.86:0/1244180831 wait complete. 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 Processor -- start 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 -- start start 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca98195390 0x7fca9819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 0x7fca9819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9819d0e0 con 0x7fca980ff940 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9819d250 con 0x7fca98195390 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9ee65700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9819d410 con 0x7fca98195810 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9cc01700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9cc01700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.86:40170/0 (socket says 172.21.15.86:40170) 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9cc01700 1 -- 172.21.15.86:0/1815428170 learned_addr learned my addr 172.21.15.86:0/1815428170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca9d402700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 0x7fca9819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.472+0000 7fca97fff700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca98195390 0x7fca9819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 -- 172.21.15.86:0/1815428170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca98195390 msgr2=0x7fca9819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca98195390 0x7fca9819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 -- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 msgr2=0x7fca98194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 -- 172.21.15.86:0/1815428170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0095f0 con 0x7fca98195810 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9cc01700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9d402700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 0x7fca9819c940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fca9000bd50 tx=0x7fca9000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9000cbb0 con 0x7fca98195810 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9000cd10 con 0x7fca98195810 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9819d6f0 con 0x7fca98195810 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90012690 con 0x7fca98195810 2024-05-08T22:57:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.473+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9819dc70 con 0x7fca98195810 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.474+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fca90012830 con 0x7fca98195810 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.474+0000 7fca95ffb700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 0x7fca8803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.474+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca9004ac20 con 0x7fca98195810 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.474+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9818e610 con 0x7fca98195810 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.474+0000 7fca9cc01700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 0x7fca8803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.475+0000 7fca9cc01700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 0x7fca8803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005850 tx=0x7fca8c005200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:09.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.479+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca90011890 con 0x7fca98195810 2024-05-08T22:57:09.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.690+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi086=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fca980613e0 con 0x7fca88038c20 2024-05-08T22:57:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.697+0000 7fca95ffb700 1 -- 172.21.15.86:0/1815428170 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fca980613e0 con 0x7fca88038c20 2024-05-08T22:57:09.697 INFO:teuthology.orchestra.run.smithi086.stdout:Scheduled mgr update... 2024-05-08T22:57:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 msgr2=0x7fca8803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 0x7fca8803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005850 tx=0x7fca8c005200 comp rx=0 tx=0).stop 2024-05-08T22:57:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 msgr2=0x7fca9819c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 0x7fca9819c940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fca9000bd50 tx=0x7fca9000bd80 comp rx=0 tx=0).stop 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 shutdown_connections 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca98195810 0x7fca9819c940 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca88038c20 0x7fca8803b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980ff940 0x7fca98194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 --2- 172.21.15.86:0/1815428170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca98195390 0x7fca9819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 >> 172.21.15.86:0/1815428170 conn(0x7fca980faee0 msgr2=0x7fca980fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 shutdown_connections 2024-05-08T22:57:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:09.699+0000 7fca9ee65700 1 -- 172.21.15.86:0/1815428170 wait complete. 2024-05-08T22:57:09.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:09 smithi046 ceph-mon[36898]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:57:09.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:09 smithi046 ceph-mon[36898]: Reconfiguring daemon mon.c on smithi086 2024-05-08T22:57:09.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:09 smithi046 ceph-mon[36898]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:09.925 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:09 smithi086 ceph-mon[78610]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:57:09.925 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:09 smithi086 ceph-mon[78610]: Reconfiguring daemon mon.c on smithi086 2024-05-08T22:57:09.925 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:09 smithi086 ceph-mon[78610]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:09 smithi053 ceph-mon[77587]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:57:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:09 smithi053 ceph-mon[77587]: Reconfiguring daemon mon.c on smithi086 2024-05-08T22:57:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:09 smithi053 ceph-mon[77587]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:10.121 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T22:57:10.124 DEBUG:teuthology.orchestra.run.smithi086:mgr.z> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T22:57:10.126 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T22:57:10.127 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T22:57:10.127 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:57:10.156 INFO:journalctl@ceph.mgr.z.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T22:57:10.161 INFO:journalctl@ceph.mgr.y.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T22:57:10.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:57:10.173 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 448 Links: 1 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 22:56:27.610291117 +0000 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 22:48:53.747346355 +0000 2024-05-08T22:57:10.259 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 22:48:53.747346355 +0000 2024-05-08T22:57:10.260 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 22:48:53.726346570 +0000 2024-05-08T22:57:10.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:57:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T22:57:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T22:57:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000178033 s, 2.9 MB/s 2024-05-08T22:57:10.367 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:57:10.411 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-05-08T22:57:10.493 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:57:10.493 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:10.493 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 454 Links: 1 2024-05-08T22:57:10.493 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:10.493 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:10.494 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 22:56:27.706290125 +0000 2024-05-08T22:57:10.494 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 22:48:54.254341169 +0000 2024-05-08T22:57:10.494 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 22:48:54.254341169 +0000 2024-05-08T22:57:10.494 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 22:48:54.233341383 +0000 2024-05-08T22:57:10.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:57:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T22:57:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T22:57:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000162707 s, 3.1 MB/s 2024-05-08T22:57:10.598 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:57:10.645 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-05-08T22:57:10.728 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:57:10.728 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:10.728 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 460 Links: 1 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 22:56:27.798289175 +0000 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 22:48:54.776335828 +0000 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 22:48:54.776335828 +0000 2024-05-08T22:57:10.729 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 22:48:54.757336023 +0000 2024-05-08T22:57:10.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:57:10.830 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T22:57:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T22:57:10.831 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000171257 s, 3.0 MB/s 2024-05-08T22:57:10.835 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:57:10.880 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-05-08T22:57:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 466 Links: 1 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 22:56:27.886288266 +0000 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 22:48:55.281330654 +0000 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 22:48:55.281330654 +0000 2024-05-08T22:57:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 22:48:55.261330860 +0000 2024-05-08T22:57:10.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi086=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi086=z;count:3 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:10.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:10.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:10.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:10.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:10.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:10 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi086=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi086=z;count:3 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:11.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:10 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi086=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi086=z;count:3 2024-05-08T22:57:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:11.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:10 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:11.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T22:57:11.064 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T22:57:11.064 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000177958 s, 2.9 MB/s 2024-05-08T22:57:11.067 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:57:11.151 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-08T22:57:11.151 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:57:11.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:57:11.174 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-05-08T22:57:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 210568 Links: 1 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-08 22:56:57.856020003 +0000 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-08 22:47:50.803198281 +0000 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-08 22:47:50.803198281 +0000 2024-05-08T22:57:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-08T22:57:11.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:57:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-08T22:57:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-08T22:57:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000116918 s, 4.4 MB/s 2024-05-08T22:57:11.312 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:57:11.378 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 208884 Links: 1 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:11.444 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-08 22:56:57.856020003 +0000 2024-05-08T22:57:11.445 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-08 22:47:51.226189745 +0000 2024-05-08T22:57:11.445 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-08 22:47:51.226189745 +0000 2024-05-08T22:57:11.445 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-08T22:57:11.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:57:11.463 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:11 smithi086 systemd[1]: Starting Ceph mgr.z for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:57:11.517 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-08T22:57:11.517 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-08T22:57:11.517 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000127401 s, 4.0 MB/s 2024-05-08T22:57:11.518 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:57:11.584 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-05-08T22:57:11.648 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:57:11.648 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:11.648 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 212435 Links: 1 2024-05-08T22:57:11.648 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:11.648 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:11.649 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-08 22:56:57.856020003 +0000 2024-05-08T22:57:11.649 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-08 22:47:51.598182238 +0000 2024-05-08T22:57:11.649 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-08 22:47:51.598182238 +0000 2024-05-08T22:57:11.650 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-08T22:57:11.650 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:57:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-08T22:57:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-08T22:57:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000114515 s, 4.5 MB/s 2024-05-08T22:57:11.723 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:57:11.788 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-05-08T22:57:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 214734 Links: 1 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-08 22:56:57.857019982 +0000 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-08 22:47:51.968174771 +0000 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-08 22:47:51.968174771 +0000 2024-05-08T22:57:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-08T22:57:11.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:57:11.927 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-08T22:57:11.927 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-08T22:57:11.927 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000124523 s, 4.1 MB/s 2024-05-08T22:57:11.928 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:57:11.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:11 smithi086 ceph-mon[78610]: Deploying daemon mgr.z on smithi086 2024-05-08T22:57:11.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:11 smithi086 ceph-mon[78610]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:11.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:11 smithi086 ceph-mgr[80462]: -- 172.21.15.86:0/1315793839 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559786b261a0 con 0x559786afcc00 2024-05-08T22:57:11.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:11 smithi086 bash[80424]: 0a28832d27c4c415b2680936aa7adf47153dd340ec274e7a077b47a1d08a19c1 2024-05-08T22:57:11.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:11 smithi086 systemd[1]: Started Ceph mgr.z for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:57:11.991 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-08T22:57:11.992 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:57:12.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:11 smithi053 ceph-mon[77587]: Deploying daemon mgr.z on smithi086 2024-05-08T22:57:12.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:11 smithi053 ceph-mon[77587]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:12.012 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:57:12.012 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-05-08T22:57:12.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:11 smithi046 ceph-mon[36898]: Deploying daemon mgr.z on smithi086 2024-05-08T22:57:12.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:11 smithi046 ceph-mon[36898]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 204496 Links: 1 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-08 22:57:11.218393731 +0000 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-08 22:47:08.198058151 +0000 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-08 22:47:08.198058151 +0000 2024-05-08T22:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-05-08T22:57:12.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:57:12.139 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:12.109+0000 7f8f4c179200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:57:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-08T22:57:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-08T22:57:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000508323 s, 1.0 MB/s 2024-05-08T22:57:12.144 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:57:12.203 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-05-08T22:57:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:57:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 216130 Links: 1 2024-05-08T22:57:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-08 22:57:11.219393708 +0000 2024-05-08T22:57:12.262 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-08 22:47:08.556050233 +0000 2024-05-08T22:57:12.263 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-08 22:47:08.556050233 +0000 2024-05-08T22:57:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-05-08T22:57:12.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:57:12.328 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-08T22:57:12.328 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-08T22:57:12.328 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000221583 s, 2.3 MB/s 2024-05-08T22:57:12.329 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:57:12.386 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-05-08T22:57:12.387 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:12.210+0000 7f8f4c179200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:57:12.387 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:12.299+0000 7f8f4c179200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:57:12.403 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:57:12.403 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 217222 Links: 1 2024-05-08T22:57:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-08 22:57:11.219393708 +0000 2024-05-08T22:57:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-08 22:47:08.915042292 +0000 2024-05-08T22:57:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-08 22:47:08.915042292 +0000 2024-05-08T22:57:12.404 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-05-08T22:57:12.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:57:12.462 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:12.387+0000 7f8f4c179200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:57:12.470 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-08T22:57:12.470 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-08T22:57:12.471 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000168726 s, 3.0 MB/s 2024-05-08T22:57:12.472 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:57:12.529 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-05-08T22:57:12.587 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:57:12.587 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:57:12.587 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 217322 Links: 1 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-08 22:57:11.219393708 +0000 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-08 22:47:09.261034638 +0000 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-08 22:47:09.261034638 +0000 2024-05-08T22:57:12.588 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-05-08T22:57:12.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:57:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-08T22:57:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-08T22:57:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000233127 s, 2.2 MB/s 2024-05-08T22:57:12.655 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:57:12.713 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-05-08T22:57:12.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:12.797 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:12 smithi086 ceph-mon[78610]: Deploying daemon mgr.y on smithi053 2024-05-08T22:57:12.996 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:12.997 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:12 smithi053 ceph-mon[77587]: Deploying daemon mgr.y on smithi053 2024-05-08T22:57:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:57:13.213 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.040+0000 7f8f4c179200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:57:13.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:13 smithi053 systemd[1]: Starting Ceph mgr.y for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T22:57:13.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:57:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:13.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:13.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:12 smithi046 ceph-mon[36898]: Deploying daemon mgr.y on smithi053 2024-05-08T22:57:13.638 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.289+0000 7f8f4c179200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:57:13.638 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.374+0000 7f8f4c179200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:57:13.638 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.482+0000 7f8f4c179200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:57:13.638 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.638+0000 7f8f4c179200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:57:13.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:13 smithi046 ceph-mon[36898]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:13.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:13 smithi086 ceph-mon[78610]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:13.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:13.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.764+0000 7f8f4c179200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:57:13.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:13.851+0000 7f8f4c179200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:57:14.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:13 smithi053 ceph-mon[77587]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:14.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:14.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:14.011 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:13 smithi053 ceph-mgr[79065]: -- 172.21.15.53:0/1488361594 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557c3a6a81a0 con 0x557c3a67e800 2024-05-08T22:57:14.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:13 smithi053 bash[79027]: 5976b94f93047d48ec0cb8100b0741880d4bf72f6e9dcb3901e5a119e1165b3a 2024-05-08T22:57:14.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:13 smithi053 systemd[1]: Started Ceph mgr.y for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T22:57:14.163 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:14.048+0000 7f07bec26200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:57:14.371 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:14.162+0000 7f07bec26200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:57:14.371 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:14.264+0000 7f07bec26200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:57:14.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:14.370+0000 7f07bec26200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:57:14.675 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:14.242+0000 7f8f4c179200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0323402 s, 324 MB/s 2024-05-08T22:57:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-08T22:57:14.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:14.674+0000 7f8f4c179200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:57:14.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:14.880+0000 7f8f4c179200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:57:14.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:14.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:14.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:15.094 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:15.094 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:15.094 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:15.139 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:15.139 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:15.139 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:15.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-05-08T22:57:15.429 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.139+0000 7f07bec26200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:57:15.463 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:14.974+0000 7f8f4c179200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:57:15.643 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:57:15.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.430+0000 7f07bec26200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:57:15.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.527+0000 7f07bec26200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:57:15.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.651+0000 7f07bec26200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:57:16.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:16.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:15.738+0000 7f8f4c179200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:15.835+0000 7f8f4c179200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:15.983+0000 7f8f4c179200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.832+0000 7f07bec26200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:15.978+0000 7f07bec26200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:16.075+0000 7f07bec26200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:57:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:16.502+0000 7f07bec26200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:16.606+0000 7f8f4c179200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:16.790+0000 7f8f4c179200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: Reconfiguring daemon mgr.x on smithi046 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:16.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 -- 172.21.15.46:0/605796866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 msgr2=0x7fba60101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 --2- 172.21.15.46:0/605796866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60101420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba540099c0 tx=0x7fba54009cd0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 -- 172.21.15.46:0/605796866 shutdown_connections 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 --2- 172.21.15.46:0/605796866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102ba0 0x7fba60109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 --2- 172.21.15.46:0/605796866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60101420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 --2- 172.21.15.46:0/605796866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102200 0x7fba60102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 -- 172.21.15.46:0/605796866 >> 172.21.15.46:0/605796866 conn(0x7fba600fc5a0 msgr2=0x7fba600fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 -- 172.21.15.46:0/605796866 shutdown_connections 2024-05-08T22:57:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.008+0000 7fba67600700 1 -- 172.21.15.46:0/605796866 wait complete. 2024-05-08T22:57:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 Processor -- start 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 -- start start 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 0x7fba600738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60073fb0 con 0x7fba60101000 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60074120 con 0x7fba60102ba0 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.009+0000 7fba67600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a1070 con 0x7fba60102200 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48650/0 (socket says 172.21.15.46:48650) 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba5fe00700 1 -- 172.21.15.46:0/1160063924 learned_addr learned my addr 172.21.15.46:0/1160063924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:57:17.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba65600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 0x7fba600738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 -- 172.21.15.46:0/1160063924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 msgr2=0x7fba60073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 -- 172.21.15.46:0/1160063924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 msgr2=0x7fba600738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 0x7fba600738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:17.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 -- 172.21.15.46:0/1160063924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540095f0 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba65600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 0x7fba600738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba5fe00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.010+0000 7fba64c00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60074ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba540098e0 tx=0x7fba5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.011+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5401d070 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.011+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba54005050 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.011+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54004a50 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.011+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601a1270 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.011+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601a1680 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.012+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba60108480 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.012+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fba540051c0 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.012+0000 7fba5d600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 0x7fba4803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.012+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba5404bf60 con 0x7fba60101000 2024-05-08T22:57:17.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.013+0000 7fba5fe00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 0x7fba4803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:17.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.013+0000 7fba5fe00700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 0x7fba4803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba4c011550 tx=0x7fba4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.018+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fba54029360 con 0x7fba60101000 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: Reconfiguring daemon mgr.x on smithi046 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.168 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:16.950+0000 7f07bec26200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: Reconfiguring daemon mgr.x on smithi046 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:17.169 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:17.242+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fba600613e0 con 0x7fba480389d0 2024-05-08T22:57:17.463 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:16.984+0000 7f8f4c179200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:57:17.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:17 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:17.169+0000 7f07bec26200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:57:17.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:17 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:17.275+0000 7f07bec26200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:57:18.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:17 smithi046 ceph-mon[36898]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:18.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:18.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:18.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:18.119 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:18.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:17 smithi053 ceph-mon[77587]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:18.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:18.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:18.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:18.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:17.905+0000 7f8f4c179200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:17 smithi086 ceph-mon[78610]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:18.187 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:18.431 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:18 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:18.151+0000 7f07bec26200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:57:18.431 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:18 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:18.262+0000 7f07bec26200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:57:18.463 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:18.187+0000 7f8f4c179200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:57:18.463 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:18.275+0000 7f8f4c179200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:57:18.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:18 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:18.431+0000 7f07bec26200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:57:18.848 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 22:57:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T22:57:18.464+0000 7f8f4c179200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:57:18.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:18.857+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7fba54004bb0 con 0x7fba60101000 2024-05-08T22:57:19.152 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:19.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: Standby manager daemon z started 2024-05-08T22:57:19.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:57:19.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:19.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:57:19.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:18 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:19.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:19.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: Standby manager daemon z started 2024-05-08T22:57:19.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:57:19.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:19.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:57:19.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:18 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:19.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:19.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: Standby manager daemon z started 2024-05-08T22:57:19.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:57:19.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:57:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:18 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/844001264' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:19.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:19 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:19.153+0000 7f07bec26200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:57:19.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:19 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:19.360+0000 7f07bec26200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:57:19.857 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:19 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:19.582+0000 7f07bec26200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: mgrmap e14: x(active, since 62s), standbys: z 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1884547055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]': finished 2024-05-08T22:57:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:57:20.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: mgrmap e14: x(active, since 62s), standbys: z 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1884547055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]': finished 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:57:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:20.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: mgrmap e14: x(active, since 62s), standbys: z 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1884547055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]: dispatch 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77bd5d5-1794-4093-becb-18c682bf3da2"}]': finished 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:57:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:20.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:20 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3161719250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:20.970 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:20 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:20.646+0000 7f07bec26200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:57:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:20 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3161719250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:21.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:20 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:20.970+0000 7f07bec26200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:57:21.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:21 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:21.071+0000 7f07bec26200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:57:21.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:20 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3161719250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:21.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 22:57:21 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T22:57:21.288+0000 7f07bec26200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:57:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:21.919+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fba5400dc60 con 0x7fba60101000 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: Standby manager daemon y started 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:57:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:21 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: Standby manager daemon y started 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:57:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:21 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: Standby manager daemon y started 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:57:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:21 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/2718440685' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:57:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:22 smithi086 ceph-mon[78610]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:57:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:57:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:22 smithi086 ceph-mon[78610]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:23.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:22 smithi053 ceph-mon[77587]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:57:23.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:57:23.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:22 smithi053 ceph-mon[77587]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:22 smithi046 ceph-mon[36898]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-05-08T22:57:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:57:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:22 smithi046 ceph-mon[36898]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:25.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:24 smithi086 ceph-mon[78610]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:24 smithi053 ceph-mon[77587]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:25.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:24 smithi046 ceph-mon[36898]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:26.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:26 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:57:26.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:26 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:26.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:26 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:57:26.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:26 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:26.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:26 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:57:26.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:26 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:27.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:27 smithi046 ceph-mon[36898]: Deploying daemon osd.0 on smithi046 2024-05-08T22:57:27.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:27 smithi046 ceph-mon[36898]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:27.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:27 smithi086 ceph-mon[78610]: Deploying daemon osd.0 on smithi046 2024-05-08T22:57:27.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:27 smithi086 ceph-mon[78610]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:27.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:27 smithi053 ceph-mon[77587]: Deploying daemon osd.0 on smithi046 2024-05-08T22:57:27.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:27 smithi053 ceph-mon[77587]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:29.201 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:28 smithi046 ceph-mon[36898]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:29.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:28 smithi086 ceph-mon[78610]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:29.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:28 smithi053 ceph-mon[77587]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:30 smithi086 ceph-mon[78610]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:30 smithi053 ceph-mon[77587]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:31.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:30 smithi046 ceph-mon[36898]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:32.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:32.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:32.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.746+0000 7fba5d600700 1 -- 172.21.15.46:0/1160063924 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fba600613e0 con 0x7fba480389d0 2024-05-08T22:57:33.747 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 msgr2=0x7fba4803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 0x7fba4803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba4c011550 tx=0x7fba4c009260 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 msgr2=0x7fba60074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60074ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba540098e0 tx=0x7fba5400f690 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 shutdown_connections 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba60102ba0 0x7fba600738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba480389d0 0x7fba4803ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba60101000 0x7fba60074ca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 --2- 172.21.15.46:0/1160063924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60102200 0x7fba60073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 >> 172.21.15.46:0/1160063924 conn(0x7fba600fc5a0 msgr2=0x7fba601055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 shutdown_connections 2024-05-08T22:57:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:57:33.748+0000 7fba67600700 1 -- 172.21.15.46:0/1160063924 wait complete. 2024-05-08T22:57:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:34.113 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T22:57:34.117 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-05-08T22:57:34.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:57:34.152 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:34.194 INFO:journalctl@ceph.osd.0.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:34.365 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:57:35.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:34 smithi046 ceph-mon[36898]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:35.027 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:57:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T22:57:35.004+0000 7f8c0bf657c0 -1 osd.0 0 log_to_monitors true 2024-05-08T22:57:35.112 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:34 smithi053 ceph-mon[77587]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:34 smithi086 ceph-mon[78610]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00897998 s, 1.2 GB/s 2024-05-08T22:57:36.067 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-08T22:57:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:35 smithi086 ceph-mon[78610]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:57:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:35 smithi053 ceph-mon[77587]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:57:36.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:35 smithi046 ceph-mon[36898]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:57:36.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-05-08T22:57:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:57:37.174 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:36 smithi046 ceph-mon[36898]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:57:37.175 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:36 smithi046 ceph-mon[36898]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:57:37.175 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:37.175 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:36 smithi046 ceph-mon[36898]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:57:37.175 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:36 smithi046 ceph-mon[36898]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:37.175 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:57:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T22:57:36.979+0000 7f8c00400700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T22:57:37.175 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:57:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T22:57:36.983+0000 7f8bfb200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:57:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:36 smithi086 ceph-mon[78610]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:57:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:36 smithi086 ceph-mon[78610]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:57:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:36 smithi086 ceph-mon[78610]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:57:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:36 smithi086 ceph-mon[78610]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:37.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:36 smithi053 ceph-mon[77587]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:57:37.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:36 smithi053 ceph-mon[77587]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:57:37.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:37.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:36 smithi053 ceph-mon[77587]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:57:37.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:36 smithi053 ceph-mon[77587]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:57:37.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 -- 172.21.15.53:0/799302702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c0fff00 msgr2=0x7f915c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 --2- 172.21.15.53:0/799302702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c0fff00 0x7f915c100320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9158009990 tx=0x7f9158009ca0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 -- 172.21.15.53:0/799302702 shutdown_connections 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 --2- 172.21.15.53:0/799302702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 0x7f915c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 --2- 172.21.15.53:0/799302702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c0fff00 0x7f915c100320 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 --2- 172.21.15.53:0/799302702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c101060 0x7f915c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 -- 172.21.15.53:0/799302702 >> 172.21.15.53:0/799302702 conn(0x7f915c0fb4c0 msgr2=0x7f915c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 -- 172.21.15.53:0/799302702 shutdown_connections 2024-05-08T22:57:37.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.839+0000 7f916478c700 1 -- 172.21.15.53:0/799302702 wait complete. 2024-05-08T22:57:37.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 Processor -- start 2024-05-08T22:57:37.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 -- start start 2024-05-08T22:57:37.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c0fff00 0x7f915c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 0x7f915c19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c19d860 con 0x7f915c0fff00 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c19d9a0 con 0x7f915c101060 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f916478c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c19db60 con 0x7f915c101a20 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f9161d27700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f9161d27700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49932/0 (socket says 172.21.15.53:49932) 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f9161d27700 1 -- 172.21.15.53:0/438127311 learned_addr learned my addr 172.21.15.53:0/438127311 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f9162528700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c0fff00 0x7f915c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.840+0000 7f9162d29700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 0x7f915c19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:37.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 -- 172.21.15.53:0/438127311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 msgr2=0x7f915c19d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 0x7f915c19d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 -- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c0fff00 msgr2=0x7f915c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c0fff00 0x7f915c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 -- 172.21.15.53:0/438127311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580095f0 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f9161d27700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c9b0 tx=0x7f914c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c007b00 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f914c00ce50 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c018660 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c19de40 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.841+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c19e200 con 0x7f915c101060 2024-05-08T22:57:37.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.842+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c107330 con 0x7f915c101060 2024-05-08T22:57:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.844+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f914c00f450 con 0x7f915c101060 2024-05-08T22:57:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.844+0000 7f91537fe700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 0x7f91480a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.844+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f914c0caf90 con 0x7f915c101060 2024-05-08T22:57:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.845+0000 7f9162528700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 0x7f91480a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:37.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.845+0000 7f9162528700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 0x7f91480a3550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f915800b2e0 tx=0x7f915800b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:37.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:37.848+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f914c0962a0 con 0x7f915c101060 2024-05-08T22:57:38.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:38.072+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f915c02ceb0 con 0x7f91480a1090 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi046 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 17273M 2024-05-08T22:57:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:57:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:57:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi046 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 17273M 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi046 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 17273M 2024-05-08T22:57:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:39.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:57:39.534 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] boot 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:39.535 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:38 smithi053 ceph-mon[77587]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] boot 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:39.536 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:38 smithi086 ceph-mon[78610]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: osd.0 [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] boot 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:57:39.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:39.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:39.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:39.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:39.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:38 smithi046 ceph-mon[36898]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:39 smithi053 ceph-mon[77587]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:57:40.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:39 smithi046 ceph-mon[36898]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:57:40.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:39 smithi086 ceph-mon[78610]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/3749408925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]': finished 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:40 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/3653526971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/3749408925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]': finished 2024-05-08T22:57:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:57:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:40 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/3653526971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/3749408925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]: dispatch 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c389d713-72f0-4a78-854e-85a7a352ee10"}]': finished 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:40 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/3653526971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:57:43.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:42 smithi086 ceph-mon[78610]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:43.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:42 smithi053 ceph-mon[77587]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:43.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:42 smithi046 ceph-mon[36898]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:45.208 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:44 smithi053 ceph-mon[77587]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:45.212 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:44 smithi086 ceph-mon[78610]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:45.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:44 smithi046 ceph-mon[36898]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:46.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:57:46.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:57:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:47.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:57:47.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:47.900 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:47 smithi053 ceph-mon[77587]: Deploying daemon osd.1 on smithi053 2024-05-08T22:57:47.900 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:47 smithi053 ceph-mon[77587]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:47.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:47 smithi086 ceph-mon[78610]: Deploying daemon osd.1 on smithi053 2024-05-08T22:57:47.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:47 smithi086 ceph-mon[78610]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:48.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:47 smithi046 ceph-mon[36898]: Deploying daemon osd.1 on smithi053 2024-05-08T22:57:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:47 smithi046 ceph-mon[36898]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:48 smithi086 ceph-mon[78610]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:48 smithi053 ceph-mon[77587]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:48 smithi046 ceph-mon[36898]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:52.110 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.110 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.110 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:52.111 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:51 smithi053 ceph-mon[77587]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:51 smithi086 ceph-mon[78610]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:51 smithi046 ceph-mon[36898]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:53.445 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-05-08T22:57:53.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.445+0000 7f91537fe700 1 -- 172.21.15.53:0/438127311 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f915c02ceb0 con 0x7f91480a1090 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 msgr2=0x7f91480a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 0x7f91480a3550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f915800b2e0 tx=0x7f915800b6f0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 msgr2=0x7f915c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c9b0 tx=0x7f914c00ccc0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 shutdown_connections 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f915c101a20 0x7f915c19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f91480a1090 0x7f91480a3550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f915c0fff00 0x7f915c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 --2- 172.21.15.53:0/438127311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915c101060 0x7f915c1962a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:53.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 >> 172.21.15.53:0/438127311 conn(0x7f915c0fb4c0 msgr2=0x7f915c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:53.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 shutdown_connections 2024-05-08T22:57:53.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:57:53.448+0000 7f916478c700 1 -- 172.21.15.53:0/438127311 wait complete. 2024-05-08T22:57:53.892 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T22:57:53.894 INFO:tasks.cephadm:Deploying osd.2 on smithi086 with /dev/vg_nvme/lv_4... 2024-05-08T22:57:53.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:53.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:53.928 INFO:journalctl@ceph.osd.1.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T22:57:54.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.119 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:57:54.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:54.840 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:57:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T22:57:54.779+0000 7fbfd66ec7c0 -1 osd.1 0 log_to_monitors true 2024-05-08T22:57:54.928 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:54 smithi086 ceph-mon[78610]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:54.928 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:54 smithi086 ceph-mon[78610]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:54 smithi053 ceph-mon[77587]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:54 smithi053 ceph-mon[77587]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:55.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:54 smithi046 ceph-mon[36898]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:55.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:54 smithi046 ceph-mon[36898]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:57:55.452 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:57:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:57:55.453 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-08T22:57:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-08T22:57:55.453 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137131 s, 765 MB/s 2024-05-08T22:57:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-08T22:57:55.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-05-08T22:57:56.089 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:57:56.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:57:56.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:57:56.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:56.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:55 smithi053 ceph-mon[77587]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.194 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:55 smithi086 ceph-mon[78610]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:56.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:57:56.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:57:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: from='osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:57:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:55 smithi046 ceph-mon[36898]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:56.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:57:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T22:57:56.449+0000 7fbfcbd50700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T22:57:56.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:57:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T22:57:56.454+0000 7fbfc3b3d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 -- 172.21.15.86:0/1265371378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 msgr2=0x7eff041083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 --2- 172.21.15.86:0/1265371378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff041083b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efefc00b0a0 tx=0x7efefc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 -- 172.21.15.86:0/1265371378 shutdown_connections 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 --2- 172.21.15.86:0/1265371378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff041083b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 --2- 172.21.15.86:0/1265371378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 0x7eff041019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 --2- 172.21.15.86:0/1265371378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff04100150 0x7eff041005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 -- 172.21.15.86:0/1265371378 >> 172.21.15.86:0/1265371378 conn(0x7eff040fac40 msgr2=0x7eff040fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:57:57.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 -- 172.21.15.86:0/1265371378 shutdown_connections 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 -- 172.21.15.86:0/1265371378 wait complete. 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.081+0000 7eff0b214700 1 Processor -- start 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 -- start start 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff04100150 0x7eff04196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 0x7eff0419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419eb20 con 0x7eff04100150 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419ec60 con 0x7eff04101580 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419edf0 con 0x7eff04100b10 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35240/0 (socket says 172.21.15.86:35240) 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 -- 172.21.15.86:0/3056736616 learned_addr learned my addr 172.21.15.86:0/3056736616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:57:57.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff097b1700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 0x7eff0419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 -- 172.21.15.86:0/3056736616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 msgr2=0x7eff0419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 0x7eff0419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 -- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff04100150 msgr2=0x7eff04196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff04100150 0x7eff04196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 -- 172.21.15.86:0/3056736616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc009d00 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff03fff700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efef000c9e0 tx=0x7efef000cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0007ac0 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efef0007c20 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0419f0d0 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.082+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0018660 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.083+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0419f5d0 con 0x7eff04100b10 2024-05-08T22:57:57.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.084+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0404ec60 con 0x7eff04100b10 2024-05-08T22:57:57.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.084+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7efef000f450 con 0x7eff04100b10 2024-05-08T22:57:57.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.085+0000 7eff01ffb700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 0x7efef40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:57:57.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.085+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7efef00cb000 con 0x7eff04100b10 2024-05-08T22:57:57.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.085+0000 7eff08fb0700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 0x7efef40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:57:57.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.086+0000 7eff08fb0700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 0x7efef40a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efef8005950 tx=0x7efef80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:57:57.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.089+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efef0096060 con 0x7eff04100b10 2024-05-08T22:57:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:57:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:57:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='osd.1 ' entity='osd.1' 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='osd.1 ' entity='osd.1' 2024-05-08T22:57:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:57:57.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='osd.1 ' entity='osd.1' 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:57:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:57:57.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:57:57.292+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7eff040fcd70 con 0x7efef40a1090 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi053 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] boot 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:57:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi053 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] boot 2024-05-08T22:57:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:57:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi053 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: osd.1 [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] boot 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:57:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:57:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:58 smithi086 ceph-mon[78610]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:58 smithi086 ceph-mon[78610]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:57:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:58 smithi053 ceph-mon[77587]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:58 smithi053 ceph-mon[77587]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:57:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:58 smithi046 ceph-mon[36898]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:57:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:58 smithi046 ceph-mon[36898]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/1831536522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]': finished 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:57:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/3321768124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/1831536522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]': finished 2024-05-08T22:58:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:58:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:57:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/3321768124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/1831536522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]: dispatch 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68"}]': finished 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:57:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/3321768124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:00 smithi086 ceph-mon[78610]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:00 smithi053 ceph-mon[77587]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:01.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:00 smithi046 ceph-mon[36898]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:03.130 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:02 smithi086 ceph-mon[78610]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:02 smithi053 ceph-mon[77587]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:03.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:02 smithi046 ceph-mon[36898]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:05.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:04 smithi086 ceph-mon[78610]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:04 smithi053 ceph-mon[77587]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:05.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:04 smithi046 ceph-mon[36898]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:58:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:05 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:05 smithi086 ceph-mon[78610]: Deploying daemon osd.2 on smithi086 2024-05-08T22:58:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:58:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:05 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:05 smithi053 ceph-mon[77587]: Deploying daemon osd.2 on smithi086 2024-05-08T22:58:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:58:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:05 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:05 smithi046 ceph-mon[36898]: Deploying daemon osd.2 on smithi086 2024-05-08T22:58:07.139 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:06 smithi086 ceph-mon[78610]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:07.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:06 smithi053 ceph-mon[77587]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:07.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:06 smithi046 ceph-mon[36898]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:08 smithi086 ceph-mon[78610]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:09.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:08 smithi053 ceph-mon[77587]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:09.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:08 smithi046 ceph-mon[36898]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:10.171 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.171 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.171 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:11.302 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 2 on host 'smithi086' 2024-05-08T22:58:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.302+0000 7eff01ffb700 1 -- 172.21.15.86:0/3056736616 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff040fcd70 con 0x7efef40a1090 2024-05-08T22:58:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 msgr2=0x7efef40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 0x7efef40a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efef8005950 tx=0x7efef80058e0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 msgr2=0x7eff04197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efef000c9e0 tx=0x7efef000cda0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 shutdown_connections 2024-05-08T22:58:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff04101580 0x7eff0419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efef40a1090 0x7efef40a3550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff04100150 0x7eff04196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 --2- 172.21.15.86:0/3056736616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff04100b10 0x7eff04197520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 >> 172.21.15.86:0/3056736616 conn(0x7eff040fac40 msgr2=0x7eff040fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 shutdown_connections 2024-05-08T22:58:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:11.305+0000 7eff0b214700 1 -- 172.21.15.86:0/3056736616 wait complete. 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:11.671 DEBUG:teuthology.orchestra.run.smithi086:osd.2> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T22:58:11.673 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-05-08T22:58:11.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:58:11.704 INFO:journalctl@ceph.osd.2.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:11.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:58:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:12 smithi086 ceph-mon[78610]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:13.214 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T22:58:12.984+0000 7fddee5077c0 -1 osd.2 0 log_to_monitors true 2024-05-08T22:58:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:12 smithi053 ceph-mon[77587]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:13.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:12 smithi046 ceph-mon[36898]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:13.757 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:58:13.758 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:58:13.758 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-08T22:58:13.758 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-08T22:58:13.758 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0527295 s, 199 MB/s 2024-05-08T22:58:13.758 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-08T22:58:14.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:14.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:14.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi086 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 17114M 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:14 smithi086 ceph-mon[78610]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi086 2024-05-08T22:58:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 17114M 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:14 smithi053 ceph-mon[77587]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi086 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 17114M 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: from='osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:58:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:14 smithi046 ceph-mon[36898]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 -- 172.21.15.46:0/2557186270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0102ba0 msgr2=0x7ff6e0109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 --2- 172.21.15.46:0/2557186270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0102ba0 0x7ff6e0109450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800b0d0 tx=0x7ff6d800b3e0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 -- 172.21.15.46:0/2557186270 shutdown_connections 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 --2- 172.21.15.46:0/2557186270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0102ba0 0x7ff6e0109450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 --2- 172.21.15.46:0/2557186270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0101000 0x7ff6e0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 --2- 172.21.15.46:0/2557186270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 -- 172.21.15.46:0/2557186270 >> 172.21.15.46:0/2557186270 conn(0x7ff6e00fc5a0 msgr2=0x7ff6e00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 -- 172.21.15.46:0/2557186270 shutdown_connections 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.608+0000 7ff6e8000700 1 -- 172.21.15.46:0/2557186270 wait complete. 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.609+0000 7ff6e8000700 1 Processor -- start 2024-05-08T22:58:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.609+0000 7ff6e8000700 1 -- start start 2024-05-08T22:58:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0102ba0 0x7ff6e01a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01a0d90 con 0x7ff6e0101000 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01a0ed0 con 0x7ff6e0102200 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e8000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01a1060 con 0x7ff6e0102ba0 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e5600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e5600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55160/0 (socket says 172.21.15.46:55160) 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e5600700 1 -- 172.21.15.46:0/4080244725 learned_addr learned my addr 172.21.15.46:0/4080244725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e6000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0102ba0 0x7ff6e01a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 -- 172.21.15.46:0/4080244725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0102ba0 msgr2=0x7ff6e01a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0102ba0 0x7ff6e01a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 -- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 msgr2=0x7ff6e0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e4c00700 1 -- 172.21.15.46:0/4080244725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009520 con 0x7ff6e0102200 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.610+0000 7ff6e5600700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6e4c00700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0199790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e9f0 tx=0x7ff6cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00c9c0 con 0x7ff6e0102200 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc010430 con 0x7ff6e0102200 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d8009d00 con 0x7ff6e0102200 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc018680 con 0x7ff6e0102200 2024-05-08T22:58:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.611+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a16b0 con 0x7ff6e0102200 2024-05-08T22:58:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.612+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e004ec60 con 0x7ff6e0102200 2024-05-08T22:58:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.614+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7ff6cc010860 con 0x7ff6e0102200 2024-05-08T22:58:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.615+0000 7ff6de000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 0x7ff6c80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.615+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7ff6cc014070 con 0x7ff6e0102200 2024-05-08T22:58:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.615+0000 7ff6e5600700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 0x7ff6c80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.616+0000 7ff6e5600700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 0x7ff6c80a35d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009fd0 tx=0x7ff6d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.620+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6cc052d60 con 0x7ff6e0102200 2024-05-08T22:58:15.713 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T22:58:15.451+0000 7fdde3b6b700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T22:58:15.713 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T22:58:15.457+0000 7fdddb958700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:58:15.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:15.843+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6e00748f0 con 0x7ff6c80a1110 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='osd.2 ' entity='osd.2' 2024-05-08T22:58:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:15 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='osd.2 ' entity='osd.2' 2024-05-08T22:58:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:15 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='osd.2 ' entity='osd.2' 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:15 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] boot 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:17 smithi053 ceph-mon[77587]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:58:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:58:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] boot 2024-05-08T22:58:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:58:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:17 smithi046 ceph-mon[36898]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: osd.2 [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] boot 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:58:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:17 smithi086 ceph-mon[78610]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]: dispatch 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]': finished 2024-05-08T22:58:18.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:58:18.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:18.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:58:18.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]: dispatch 2024-05-08T22:58:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]': finished 2024-05-08T22:58:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:58:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]: dispatch 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1647299851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d63af76a-815c-4170-ab60-6ed7dbe596ff"}]': finished 2024-05-08T22:58:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:58:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:19.276 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:18 smithi046 sudo[61308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:19.276 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:18 smithi046 sudo[61308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:19.277 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:18 smithi046 sudo[61308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:19.277 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:18 smithi046 sudo[61308]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:19.486+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff6cc096500 con 0x7ff6e0102200 2024-05-08T22:58:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:19 smithi053 ceph-mon[77587]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:19 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1203901379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:19 smithi053 ceph-mon[77587]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:58:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:19.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:19 smithi046 ceph-mon[36898]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:19 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1203901379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:19 smithi046 ceph-mon[36898]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:58:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:19.777 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:19 smithi046 sudo[61361]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:19.777 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:19 smithi046 sudo[61361]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:19.777 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:19 smithi046 sudo[61361]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:19.777 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 22:58:19 smithi046 sudo[61361]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:19 smithi086 ceph-mon[78610]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:19 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1203901379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:19 smithi086 ceph-mon[78610]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:58:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:20.262 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:19 smithi053 sudo[88104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:20.262 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:19 smithi053 sudo[88104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:20.262 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:19 smithi053 sudo[88104]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:20.262 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:20 smithi053 sudo[88104]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:20.736 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:20 smithi086 ceph-mon[78610]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:58:20.737 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:20.737 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89784]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:20.737 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89784]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:20.737 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89784]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:20.737 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89784]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:20.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:20 smithi053 ceph-mon[77587]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:58:20.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:20.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:20 smithi053 sudo[88107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:20.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:20 smithi053 sudo[88107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:20.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:20 smithi053 sudo[88107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:20.761 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 22:58:20 smithi053 sudo[88107]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:20.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:20 smithi046 ceph-mon[36898]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T22:58:20.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:21.213 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:21.213 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:21.213 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:21.213 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 22:58:20 smithi086 sudo[89787]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:22.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:21 smithi046 sudo[62487]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:21 smithi046 sudo[62487]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:21 smithi046 sudo[62487]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:21 smithi046 sudo[62487]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 sudo[62501]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 sudo[62501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 sudo[62501]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 sudo[62501]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 ceph-mon[36898]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:22.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 ceph-mon[78610]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89790]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:22.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:22 smithi053 ceph-mon[77587]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:22.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:22 smithi086 sudo[89793]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88110]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:58:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88110]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88110]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:23.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:23 smithi046 ceph-mon[36898]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:23.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:23 smithi086 ceph-mon[78610]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:58:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:58:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88113]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 sudo[88113]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:58:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:58:24.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:58:24.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:23 smithi053 ceph-mon[77587]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:24.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:24 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:24.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:24 smithi046 ceph-mon[36898]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:24.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:24 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:24.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:24 smithi086 ceph-mon[78610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:25.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:24 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:58:25.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:24 smithi053 ceph-mon[77587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:58:25.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:25 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:58:25.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:25 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:25.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:25 smithi046 ceph-mon[36898]: Deploying daemon osd.3 on smithi046 2024-05-08T22:58:25.742 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:25 smithi046 ceph-mon[36898]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:25 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:58:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:25 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:25 smithi086 ceph-mon[78610]: Deploying daemon osd.3 on smithi046 2024-05-08T22:58:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:25 smithi086 ceph-mon[78610]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:25 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:58:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:25 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:25 smithi053 ceph-mon[77587]: Deploying daemon osd.3 on smithi046 2024-05-08T22:58:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:25 smithi053 ceph-mon[77587]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:26 smithi086 ceph-mon[78610]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:27.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:26 smithi053 ceph-mon[77587]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:27.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:26 smithi046 ceph-mon[36898]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:29.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:28 smithi086 ceph-mon[78610]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-08T22:58:29.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:28 smithi053 ceph-mon[77587]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-08T22:58:29.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:28 smithi046 ceph-mon[36898]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-08T22:58:30.951 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:30 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:30.951 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:30 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:30.951 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:30 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:30 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:30 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:30 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:30 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:30 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:30 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:32.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:31 smithi046 ceph-mon[36898]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-08T22:58:32.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:31 smithi086 ceph-mon[78610]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-08T22:58:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:31 smithi053 ceph-mon[77587]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-08T22:58:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.595+0000 7ff6de000700 1 -- 172.21.15.46:0/4080244725 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff6e00748f0 con 0x7ff6c80a1110 2024-05-08T22:58:32.594 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-05-08T22:58:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 msgr2=0x7ff6c80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 0x7ff6c80a35d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009fd0 tx=0x7ff6d4009260 comp rx=0 tx=0).stop 2024-05-08T22:58:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 msgr2=0x7ff6e0199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:32.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0199790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e9f0 tx=0x7ff6cc00ed00 comp rx=0 tx=0).stop 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 shutdown_connections 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e0102ba0 0x7ff6e01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff6c80a1110 0x7ff6c80a35d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e0101000 0x7ff6e0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 --2- 172.21.15.46:0/4080244725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e0102200 0x7ff6e0199790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.597+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 >> 172.21.15.46:0/4080244725 conn(0x7ff6e00fc5a0 msgr2=0x7ff6e00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.598+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 shutdown_connections 2024-05-08T22:58:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:58:32.598+0000 7ff6e8000700 1 -- 172.21.15.46:0/4080244725 wait complete. 2024-05-08T22:58:32.961 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T22:58:32.965 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-05-08T22:58:32.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:58:33.045 INFO:journalctl@ceph.osd.3.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:33.221 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:33 smithi046 ceph-mon[36898]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:34.027 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 22:58:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T22:58:33.883+0000 7fe60df077c0 -1 osd.3 0 log_to_monitors true 2024-05-08T22:58:34.170 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:33 smithi053 ceph-mon[77587]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:33 smithi086 ceph-mon[78610]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:34.746 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:58:34.746 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:58:34.746 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-08T22:58:34.747 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-08T22:58:34.747 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00917027 s, 1.1 GB/s 2024-05-08T22:58:34.747 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-08T22:58:35.062 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:34 smithi053 ceph-mon[77587]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:58:35.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-05-08T22:58:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:34 smithi086 ceph-mon[78610]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:58:35.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:34 smithi046 ceph-mon[36898]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:58:35.390 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:58:36.100 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:35 smithi046 ceph-mon[36898]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:58:36.100 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:35 smithi046 ceph-mon[36898]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:58:36.100 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:36.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:35 smithi046 ceph-mon[36898]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:58:36.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:35 smithi046 ceph-mon[36898]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:36.101 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 22:58:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T22:58:35.853+0000 7fe602400700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T22:58:36.101 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 22:58:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T22:58:35.859+0000 7fe5fd200700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:58:36.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:35 smithi053 ceph-mon[77587]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:58:36.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:35 smithi053 ceph-mon[77587]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:58:36.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:36.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:35 smithi053 ceph-mon[77587]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:58:36.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:35 smithi053 ceph-mon[77587]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:35 smithi086 ceph-mon[78610]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:58:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:35 smithi086 ceph-mon[78610]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:58:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:35 smithi086 ceph-mon[78610]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:58:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:35 smithi086 ceph-mon[78610]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.405+0000 7fdec8869700 1 -- 172.21.15.53:0/3778164620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec40fff00 msgr2=0x7fdec4100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.405+0000 7fdec8869700 1 --2- 172.21.15.53:0/3778164620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec40fff00 0x7fdec4100320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009990 tx=0x7fdeb4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 -- 172.21.15.53:0/3778164620 shutdown_connections 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 --2- 172.21.15.53:0/3778164620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec4101a20 0x7fdec4108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 --2- 172.21.15.53:0/3778164620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec40fff00 0x7fdec4100320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 --2- 172.21.15.53:0/3778164620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 -- 172.21.15.53:0/3778164620 >> 172.21.15.53:0/3778164620 conn(0x7fdec40fb4a0 msgr2=0x7fdec40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:36.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 -- 172.21.15.53:0/3778164620 shutdown_connections 2024-05-08T22:58:36.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 -- 172.21.15.53:0/3778164620 wait complete. 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.406+0000 7fdec8869700 1 Processor -- start 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 -- start start 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 0x7fdec4103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41031b0 con 0x7fdec4101060 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4103320 con 0x7fdec4101a20 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec8869700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41a0fa0 con 0x7fdec40fff00 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:36.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec2d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36308/0 (socket says 172.21.15.53:36308) 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 0x7fdec4103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec2d9d700 1 -- 172.21.15.53:0/4152613648 learned_addr learned my addr 172.21.15.53:0/4152613648 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdeb9d9b700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec259c700 1 -- 172.21.15.53:0/4152613648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 msgr2=0x7fdec4102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.407+0000 7fdec259c700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec259c700 1 -- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 msgr2=0x7fdec41024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec259c700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec259c700 1 -- 172.21.15.53:0/4152613648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40095f0 con 0x7fdec40fff00 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdeb9d9b700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec2d9d700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec259c700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 0x7fdec4103e30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb401d070 con 0x7fdec40fff00 2024-05-08T22:58:36.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb4021410 con 0x7fdec40fff00 2024-05-08T22:58:36.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec41a11d0 con 0x7fdec40fff00 2024-05-08T22:58:36.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017650 con 0x7fdec40fff00 2024-05-08T22:58:36.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.408+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41a15e0 con 0x7fdec40fff00 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.409+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec404ec60 con 0x7fdec40fff00 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.410+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdeb4021580 con 0x7fdec40fff00 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.411+0000 7fdebbfff700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 0x7fdeb00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.411+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fdeb40cd030 con 0x7fdec40fff00 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.411+0000 7fdeb9d9b700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 0x7fdeb00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:36.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.412+0000 7fdeb9d9b700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 0x7fdeb00a35e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009cb0 tx=0x7fdeac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:36.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.415+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdeb4097b90 con 0x7fdec40fff00 2024-05-08T22:58:36.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:36.640+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdec40fd6f0 con 0x7fdeb00a1120 2024-05-08T22:58:37.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:58:37.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:58:37.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:37.097 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:58:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:58:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi046 2024-05-08T22:58:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 8636M 2024-05-08T22:58:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] boot 2024-05-08T22:58:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:58:38.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:37 smithi086 ceph-mon[78610]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:58:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi046 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 8636M 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] boot 2024-05-08T22:58:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:58:38.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:37 smithi053 ceph-mon[77587]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi046 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 8636M 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: osd.3 [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] boot 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:58:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:37 smithi046 ceph-mon[36898]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/3409836620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]': finished 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:58:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/3409836620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]': finished 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:58:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/3409836620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]: dispatch 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca7a742-9c35-448a-9287-63b6fe77a632"}]': finished 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:58:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:39 smithi086 ceph-mon[78610]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:39 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/2964805048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:39 smithi086 ceph-mon[78610]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:58:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:39 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:39 smithi053 ceph-mon[77587]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:39 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/2964805048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:39 smithi053 ceph-mon[77587]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:58:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:39 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:40.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:39 smithi046 ceph-mon[36898]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:39 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/2964805048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:39 smithi046 ceph-mon[36898]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:58:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:39 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:41 smithi053 ceph-mon[77587]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:58:41.885 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:41.885 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:41 smithi053 ceph-mon[77587]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:41.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:41 smithi046 ceph-mon[36898]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:58:41.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:41.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:41 smithi046 ceph-mon[36898]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:41.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:41 smithi086 ceph-mon[78610]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:58:41.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:41.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:41 smithi086 ceph-mon[78610]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:43.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:42 smithi053 ceph-mon[77587]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:58:43.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:42 smithi086 ceph-mon[78610]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:58:43.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:42 smithi046 ceph-mon[36898]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:58:45.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:44 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:58:45.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:44 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:45.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:44 smithi053 ceph-mon[77587]: Deploying daemon osd.4 on smithi053 2024-05-08T22:58:45.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:44 smithi053 ceph-mon[77587]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-08T22:58:45.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:44 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:58:45.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:44 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:45.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:44 smithi086 ceph-mon[78610]: Deploying daemon osd.4 on smithi053 2024-05-08T22:58:45.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:44 smithi086 ceph-mon[78610]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-08T22:58:45.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:44 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:58:45.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:44 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:45.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:44 smithi046 ceph-mon[36898]: Deploying daemon osd.4 on smithi053 2024-05-08T22:58:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:44 smithi046 ceph-mon[36898]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-08T22:58:47.167 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:46 smithi053 ceph-mon[77587]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:58:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:46 smithi086 ceph-mon[78610]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:58:47.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:46 smithi046 ceph-mon[36898]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:58:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:48 smithi086 ceph-mon[78610]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T22:58:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:48 smithi053 ceph-mon[77587]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T22:58:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:48 smithi046 ceph-mon[36898]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T22:58:50.510 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:50.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:50.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:50.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:51.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.748+0000 7fdebbfff700 1 -- 172.21.15.53:0/4152613648 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdec40fd6f0 con 0x7fdeb00a1120 2024-05-08T22:58:51.749 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.750+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 msgr2=0x7fdeb00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.750+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 0x7fdeb00a35e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009cb0 tx=0x7fdeac009260 comp rx=0 tx=0).stop 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.750+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 msgr2=0x7fdec4103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 0x7fdec4103e30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4009580 comp rx=0 tx=0).stop 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 shutdown_connections 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4101a20 0x7fdec4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:51.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec40fff00 0x7fdec4103e30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:51.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdeb00a1120 0x7fdeb00a35e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:51.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 --2- 172.21.15.53:0/4152613648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4101060 0x7fdec41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:51.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 >> 172.21.15.53:0/4152613648 conn(0x7fdec40fb4a0 msgr2=0x7fdec40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:51.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 shutdown_connections 2024-05-08T22:58:51.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:58:51.751+0000 7fdec8869700 1 -- 172.21.15.53:0/4152613648 wait complete. 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:51 smithi086 ceph-mon[78610]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T22:58:51.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:51.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:51.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:51.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:51 smithi053 ceph-mon[77587]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T22:58:52.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:51 smithi046 ceph-mon[36898]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-08T22:58:52.278 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T22:58:52.283 INFO:tasks.cephadm:Deploying osd.5 on smithi086 with /dev/vg_nvme/lv_3... 2024-05-08T22:58:52.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:58:52.322 INFO:journalctl@ceph.osd.4.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T22:58:52.502 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:58:53.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:53.484 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:58:53.512 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 22:58:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T22:58:53.190+0000 7fd9366257c0 -1 osd.4 0 log_to_monitors true 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113034 s, 928 MB/s 2024-05-08T22:58:53.937 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-08T22:58:53.976 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:53 smithi086 ceph-mon[78610]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:58:53.976 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:53 smithi086 ceph-mon[78610]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:53.977 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:53 smithi086 ceph-mon[78610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:53 smithi053 ceph-mon[77587]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:58:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:53 smithi053 ceph-mon[77587]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:53 smithi053 ceph-mon[77587]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:54.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:53 smithi046 ceph-mon[36898]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T22:58:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:53 smithi046 ceph-mon[36898]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:53 smithi046 ceph-mon[36898]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:58:54.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-05-08T22:58:54.594 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:58:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:54 smithi053 ceph-mon[77587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:58:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:54 smithi053 ceph-mon[77587]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:58:54.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:54.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:54 smithi053 ceph-mon[77587]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:54.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:54 smithi053 ceph-mon[77587]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:55.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:54 smithi046 ceph-mon[36898]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:58:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:54 smithi046 ceph-mon[36898]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:58:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:54 smithi046 ceph-mon[36898]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:54 smithi046 ceph-mon[36898]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:55.077 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:54 smithi086 ceph-mon[78610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:58:55.078 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:54 smithi086 ceph-mon[78610]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:58:55.078 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:55.078 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:54 smithi086 ceph-mon[78610]: from='osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:55.078 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:54 smithi086 ceph-mon[78610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:58:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/442062993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341022a0 msgr2=0x7f7c34102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/442062993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341022a0 0x7f7c34102700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009990 tx=0x7f7c30009ca0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/442062993 shutdown_connections 2024-05-08T22:58:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/442062993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c34102c40 0x7f7c341094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/442062993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c340ff2c0 0x7f7c340ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/442062993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341022a0 0x7f7c34102700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/442062993 >> 172.21.15.86:0/442062993 conn(0x7f7c340fac40 msgr2=0x7f7c340fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/442062993 shutdown_connections 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.657+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/442062993 wait complete. 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 Processor -- start 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 -- start start 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c341022a0 0x7f7c341974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c34102c40 0x7f7c3419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419eac0 con 0x7f7c34102c40 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419ec00 con 0x7f7c341022a0 2024-05-08T22:58:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3c2a6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419ed90 con 0x7f7c340ff2c0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47952/0 (socket says 172.21.15.86:47952) 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 -- 172.21.15.86:0/2609391762 learned_addr learned my addr 172.21.15.86:0/2609391762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c39841700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c341022a0 0x7f7c341974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 -- 172.21.15.86:0/2609391762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c341022a0 msgr2=0x7f7c341974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c341022a0 0x7f7c341974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 -- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c34102c40 msgr2=0x7f7c3419e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c34102c40 0x7f7c3419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 -- 172.21.15.86:0/2609391762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300095f0 con 0x7f7c340ff2c0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c3a042700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d9b0 tx=0x7f7c2400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.658+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24009930 con 0x7f7c340ff2c0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.659+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c24010460 con 0x7f7c340ff2c0 2024-05-08T22:58:55.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.659+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400f620 con 0x7f7c340ff2c0 2024-05-08T22:58:55.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.659+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3419f070 con 0x7f7c340ff2c0 2024-05-08T22:58:55.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.659+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3419f520 con 0x7f7c340ff2c0 2024-05-08T22:58:55.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.659+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c34108580 con 0x7f7c340ff2c0 2024-05-08T22:58:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.661+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7c24010860 con 0x7f7c340ff2c0 2024-05-08T22:58:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.662+0000 7f7c2b7fe700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 0x7f7c200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:58:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.662+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f7c240cb210 con 0x7f7c340ff2c0 2024-05-08T22:58:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.662+0000 7f7c39841700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 0x7f7c200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:58:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.663+0000 7f7c39841700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 0x7f7c200a35e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c30003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:58:55.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.666+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c24095b30 con 0x7f7c340ff2c0 2024-05-08T22:58:55.723 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 22:58:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T22:58:55.269+0000 7fd92bc89700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T22:58:55.723 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 22:58:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T22:58:55.276+0000 7fd923a76700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:58:55.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:58:55.869+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c3402ce10 con 0x7f7c200a1120 2024-05-08T22:58:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:58:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:58:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi053 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:55 smithi053 ceph-mon[77587]: from='osd.4 ' entity='osd.4' 2024-05-08T22:58:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:58:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:58:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi053 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:55 smithi046 ceph-mon[36898]: from='osd.4 ' entity='osd.4' 2024-05-08T22:58:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:58:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:58:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi053 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:58:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:55 smithi086 ceph-mon[78610]: from='osd.4 ' entity='osd.4' 2024-05-08T22:58:56.910 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] boot 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:58:56.911 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:58:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:58:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] boot 2024-05-08T22:58:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:58:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:58:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:58:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: osd.4 [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] boot 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:58:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:58:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:58:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/3998055023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]': finished 2024-05-08T22:58:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:58:58.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:58:58.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:58:58.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/3998055023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]': finished 2024-05-08T22:58:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:58:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:58:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:58:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/3998055023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]: dispatch 2024-05-08T22:58:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ef3604-5955-42ac-b4f7-5d211ef7d553"}]': finished 2024-05-08T22:58:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:58:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:58:59.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/156103969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:59.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/156103969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:58:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/156103969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:58:59 smithi053 ceph-mon[77587]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:00.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:58:59 smithi046 ceph-mon[36898]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:58:59 smithi086 ceph-mon[78610]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:00 smithi086 ceph-mon[78610]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:00 smithi053 ceph-mon[77587]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:01.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:00 smithi046 ceph-mon[36898]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:02 smithi086 ceph-mon[78610]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:02 smithi053 ceph-mon[77587]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:02 smithi046 ceph-mon[36898]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:04.256 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:59:04.256 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:04.256 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:03 smithi086 ceph-mon[78610]: Deploying daemon osd.5 on smithi086 2024-05-08T22:59:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:59:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:03 smithi053 ceph-mon[77587]: Deploying daemon osd.5 on smithi086 2024-05-08T22:59:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:59:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:03 smithi046 ceph-mon[36898]: Deploying daemon osd.5 on smithi086 2024-05-08T22:59:05.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:04 smithi086 ceph-mon[78610]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:04 smithi053 ceph-mon[77587]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:05.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:04 smithi046 ceph-mon[36898]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:07.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:06 smithi086 ceph-mon[78610]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:07.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:06 smithi053 ceph-mon[77587]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:07.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:06 smithi046 ceph-mon[36898]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:08 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:08.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:08.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:08.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:08 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:09.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:08 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.629+0000 7f7c2b7fe700 1 -- 172.21.15.86:0/2609391762 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7c3402ce10 con 0x7f7c200a1120 2024-05-08T22:59:09.629 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 5 on host 'smithi086' 2024-05-08T22:59:09.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 msgr2=0x7f7c200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 0x7f7c200a35e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c30003790 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 msgr2=0x7f7c34196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d9b0 tx=0x7f7c2400dcc0 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 shutdown_connections 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7c200a1120 0x7f7c200a35e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c34102c40 0x7f7c3419e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c340ff2c0 0x7f7c34196f80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 --2- 172.21.15.86:0/2609391762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c341022a0 0x7f7c341974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 >> 172.21.15.86:0/2609391762 conn(0x7f7c340fac40 msgr2=0x7f7c341056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 shutdown_connections 2024-05-08T22:59:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:09.632+0000 7f7c3c2a6700 1 -- 172.21.15.86:0/2609391762 wait complete. 2024-05-08T22:59:09.868 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.868 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.868 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:09.869 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:09.869 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.869 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:09.869 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.869 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:09.870 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:10.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:10.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:10.125 DEBUG:teuthology.orchestra.run.smithi086:osd.5> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T22:59:10.128 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-05-08T22:59:10.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:59:10.157 INFO:journalctl@ceph.osd.5.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T22:59:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:59:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:10 smithi053 ceph-mon[77587]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:10 smithi046 ceph-mon[36898]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:10 smithi086 ceph-mon[78610]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:11.843 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 22:59:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T22:59:11.498+0000 7f19f16567c0 -1 osd.5 0 log_to_monitors true 2024-05-08T22:59:12.146 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:11 smithi086 ceph-mon[78610]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.146 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:11 smithi086 ceph-mon[78610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0393625 s, 266 MB/s 2024-05-08T22:59:12.212 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-08T22:59:12.253 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:11 smithi046 ceph-mon[36898]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.253 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:11 smithi046 ceph-mon[36898]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:11 smithi053 ceph-mon[77587]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:11 smithi053 ceph-mon[77587]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:59:12.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-05-08T22:59:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:59:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:59:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:59:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi086 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 8557M 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:12 smithi053 ceph-mon[77587]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:13.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:59:13.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi086 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 8557M 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:12 smithi046 ceph-mon[36898]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:13.463 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 22:59:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T22:59:12.984+0000 7f19e6cba700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T22:59:13.463 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 22:59:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T22:59:12.991+0000 7f19e2aaf700 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:59:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:59:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:59:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi086 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 8557M 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:12 smithi086 ceph-mon[78610]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:13 smithi053 ceph-mon[77587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:59:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:13 smithi053 ceph-mon[77587]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:59:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:13 smithi046 ceph-mon[36898]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:59:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:13 smithi046 ceph-mon[36898]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:59:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 -- 172.21.15.46:0/2688607018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 msgr2=0x7f7ce0103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/2688607018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0103e80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009990 tx=0x7f7cd0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 -- 172.21.15.46:0/2688607018 shutdown_connections 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/2688607018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce0106cf0 0x7f7ce0109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/2688607018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0103e80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/2688607018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce01043c0 0x7f7ce01067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 -- 172.21.15.46:0/2688607018 >> 172.21.15.46:0/2688607018 conn(0x7f7ce00fb4a0 msgr2=0x7f7ce00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:14.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 -- 172.21.15.46:0/2688607018 shutdown_connections 2024-05-08T22:59:14.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.344+0000 7f7ce6e00700 1 -- 172.21.15.46:0/2688607018 wait complete. 2024-05-08T22:59:14.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 Processor -- start 2024-05-08T22:59:14.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 -- start start 2024-05-08T22:59:14.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce01043c0 0x7f7ce01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce0106cf0 0x7f7ce019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce019eab0 con 0x7f7ce0101a90 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce019ebf0 con 0x7f7ce01043c0 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.345+0000 7f7ce6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce019ed80 con 0x7f7ce0106cf0 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45172/0 (socket says 172.21.15.46:45172) 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 -- 172.21.15.46:0/1102180930 learned_addr learned my addr 172.21.15.46:0/1102180930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7ce4e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce0106cf0 0x7f7ce019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdf400700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce01043c0 0x7f7ce01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 -- 172.21.15.46:0/1102180930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce0106cf0 msgr2=0x7f7ce019e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce0106cf0 0x7f7ce019e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 -- 172.21.15.46:0/1102180930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce01043c0 msgr2=0x7f7ce01974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce01043c0 0x7f7ce01974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 -- 172.21.15.46:0/1102180930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00095f0 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.346+0000 7f7cdfe00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0005910 tx=0x7f7cd0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.347+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd001d070 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.347+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd0022470 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.347+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce019f000 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.347+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017650 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.347+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce019f410 con 0x7f7ce0101a90 2024-05-08T22:59:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.348+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00fd020 con 0x7f7ce0101a90 2024-05-08T22:59:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.350+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7cd000fad0 con 0x7f7ce0101a90 2024-05-08T22:59:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.351+0000 7f7cdcc00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 0x7f7cc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.351+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f7cd00ccda0 con 0x7f7ce0101a90 2024-05-08T22:59:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.351+0000 7f7cdf400700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 0x7f7cc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.352+0000 7f7cdf400700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 0x7f7cc40a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009c60 tx=0x7f7ccc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.356+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7cd00973c0 con 0x7f7ce0101a90 2024-05-08T22:59:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:13 smithi086 ceph-mon[78610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T22:59:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:13 smithi086 ceph-mon[78610]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:59:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:14.584+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce002cdf0 con 0x7f7cc40a1120 2024-05-08T22:59:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:59:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:59:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] boot 2024-05-08T22:59:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T22:59:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:14 smithi053 ceph-mon[77587]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:59:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:59:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] boot 2024-05-08T22:59:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:14 smithi046 ceph-mon[36898]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: osd.5 [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] boot 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:14 smithi086 ceph-mon[78610]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:59:16.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:16 smithi053 ceph-mon[77587]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:59:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:16 smithi046 ceph-mon[36898]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:59:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:16 smithi086 ceph-mon[78610]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:59:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:59:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]: dispatch 2024-05-08T22:59:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]': finished 2024-05-08T22:59:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:59:17.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:59:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:17.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]: dispatch 2024-05-08T22:59:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]': finished 2024-05-08T22:59:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:59:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T22:59:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]: dispatch 2024-05-08T22:59:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/772557845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9977b050-bdba-4adc-8d0a-8ee2935643bf"}]': finished 2024-05-08T22:59:17.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:59:17.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:18 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/436958495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:18.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:18 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/436958495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:18 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/436958495' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:19 smithi046 ceph-mon[36898]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-08T22:59:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:19 smithi086 ceph-mon[78610]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-08T22:59:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:19 smithi053 ceph-mon[77587]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-08T22:59:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:20 smithi053 ceph-mon[77587]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:59:21.567 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:20 smithi086 ceph-mon[78610]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:59:21.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:20 smithi046 ceph-mon[36898]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:59:23.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:22 smithi053 ceph-mon[77587]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:59:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:22 smithi046 ceph-mon[36898]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:59:23.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:22 smithi086 ceph-mon[78610]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T22:59:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:59:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:23 smithi053 ceph-mon[77587]: Deploying daemon osd.6 on smithi046 2024-05-08T22:59:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:59:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:23 smithi046 ceph-mon[36898]: Deploying daemon osd.6 on smithi046 2024-05-08T22:59:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:59:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:23 smithi086 ceph-mon[78610]: Deploying daemon osd.6 on smithi046 2024-05-08T22:59:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:24 smithi053 ceph-mon[77587]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T22:59:25.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:24 smithi046 ceph-mon[36898]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T22:59:25.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:24 smithi086 ceph-mon[78610]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T22:59:27.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:26 smithi046 ceph-mon[36898]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:59:27.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:26 smithi053 ceph-mon[77587]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:59:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:26 smithi086 ceph-mon[78610]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:59:29.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:29.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:29.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:29.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:29 smithi046 ceph-mon[36898]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-08T22:59:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:29 smithi086 ceph-mon[78610]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-08T22:59:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:29 smithi053 ceph-mon[77587]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-08T22:59:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:30 smithi053 ceph-mon[77587]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:31.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:30 smithi046 ceph-mon[36898]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:31.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:30 smithi086 ceph-mon[78610]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.738+0000 7f7cdcc00700 1 -- 172.21.15.46:0/1102180930 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7ce002cdf0 con 0x7f7cc40a1120 2024-05-08T22:59:31.738 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.740+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 msgr2=0x7f7cc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.740+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 0x7f7cc40a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009c60 tx=0x7f7ccc009260 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.740+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 msgr2=0x7f7ce0196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.740+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0005910 tx=0x7f7cd0009580 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 shutdown_connections 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce0106cf0 0x7f7ce019e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7cc40a1120 0x7f7cc40a35e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0101a90 0x7f7ce0196f70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 --2- 172.21.15.46:0/1102180930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce01043c0 0x7f7ce01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 >> 172.21.15.46:0/1102180930 conn(0x7f7ce00fb4a0 msgr2=0x7f7ce0103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 shutdown_connections 2024-05-08T22:59:31.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T22:59:31.741+0000 7f7ce6e00700 1 -- 172.21.15.46:0/1102180930 wait complete. 2024-05-08T22:59:32.133 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T22:59:32.137 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-05-08T22:59:32.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:32.245 INFO:journalctl@ceph.osd.6.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:32.406 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:32.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:33.239 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:32 smithi046 ceph-mon[36898]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:33.239 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 22:59:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T22:59:32.977+0000 7f17837d87c0 -1 osd.6 0 log_to_monitors true 2024-05-08T22:59:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:32 smithi053 ceph-mon[77587]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:33.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:32 smithi086 ceph-mon[78610]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:59:33.879 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:59:33.879 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-08T22:59:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-08T22:59:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00912565 s, 1.1 GB/s 2024-05-08T22:59:33.880 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-08T22:59:34.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:33 smithi053 ceph-mon[77587]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:59:34.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:33 smithi046 ceph-mon[36898]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:59:34.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-05-08T22:59:34.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:33 smithi086 ceph-mon[78610]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:59:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:59:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:34 smithi053 ceph-mon[77587]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:59:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:34 smithi053 ceph-mon[77587]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:59:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:34 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:34 smithi053 ceph-mon[77587]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:59:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:34 smithi053 ceph-mon[77587]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:35.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:34 smithi046 ceph-mon[36898]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:59:35.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:34 smithi046 ceph-mon[36898]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:59:35.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:35.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:34 smithi046 ceph-mon[36898]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:59:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:34 smithi046 ceph-mon[36898]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:35 smithi086 ceph-mon[78610]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:59:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:35 smithi086 ceph-mon[78610]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:59:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:35 smithi086 ceph-mon[78610]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T22:59:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:35 smithi086 ceph-mon[78610]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 -- 172.21.15.53:0/2058210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7181043d0 msgr2=0x7fc7181067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/2058210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7181043d0 0x7fc7181067c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009990 tx=0x7fc70c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 -- 172.21.15.53:0/2058210474 shutdown_connections 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/2058210474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc718106d00 0x7fc718109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/2058210474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718101aa0 0x7fc718103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/2058210474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7181043d0 0x7fc7181067c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 -- 172.21.15.53:0/2058210474 >> 172.21.15.53:0/2058210474 conn(0x7fc7180fb490 msgr2=0x7fc7180fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 -- 172.21.15.53:0/2058210474 shutdown_connections 2024-05-08T22:59:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.582+0000 7fc71d9b0700 1 -- 172.21.15.53:0/2058210474 wait complete. 2024-05-08T22:59:35.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 Processor -- start 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 -- start start 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7181043d0 0x7fc718197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718106d00 0x7fc71819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819ea90 con 0x7fc718106d00 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819ebd0 con 0x7fc718101aa0 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc71d9b0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71819ed60 con 0x7fc7181043d0 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc716ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc716ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48014/0 (socket says 172.21.15.53:48014) 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.583+0000 7fc716ffd700 1 -- 172.21.15.53:0/86551140 learned_addr learned my addr 172.21.15.53:0/86551140 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-08T22:59:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc7167fc700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7181043d0 0x7fc718197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc7177fe700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718106d00 0x7fc71819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 -- 172.21.15.53:0/86551140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7181043d0 msgr2=0x7fc718197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7181043d0 0x7fc718197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 -- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718106d00 msgr2=0x7fc71819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718106d00 0x7fc71819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 -- 172.21.15.53:0/86551140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0095f0 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc716ffd700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e9a0 tx=0x7fc70800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708009790 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc708010430 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708018680 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71819efe0 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.584+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71819f3a0 con 0x7fc718101aa0 2024-05-08T22:59:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.585+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7180fd010 con 0x7fc718101aa0 2024-05-08T22:59:35.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.587+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc7080105a0 con 0x7fc718101aa0 2024-05-08T22:59:35.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.588+0000 7fc71c9ae700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 0x7fc7000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:35.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.588+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4685+0+0 (secure 0 0 0) 0x7fc708014070 con 0x7fc718101aa0 2024-05-08T22:59:35.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.588+0000 7fc7167fc700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 0x7fc7000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.589+0000 7fc7167fc700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 0x7fc7000a35e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b2e0 tx=0x7fc70c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:35.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.591+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc7080966d0 con 0x7fc718101aa0 2024-05-08T22:59:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:35.814+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc71802cf50 con 0x7fc7000a1120 2024-05-08T22:59:35.999 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 22:59:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T22:59:35.576+0000 7f1777e00700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T22:59:35.999 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 22:59:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T22:59:35.598+0000 7f176dc00700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:59:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:59:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:59:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:59:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi046 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 5757M 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:36 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:36.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi046 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 5757M 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:36 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:59:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:59:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T22:59:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:59:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi046 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 5757M 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662]' entity='osd.6' 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:36.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:36 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:37 smithi053 ceph-mon[77587]: osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] boot 2024-05-08T22:59:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:37 smithi053 ceph-mon[77587]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T22:59:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:37 smithi053 ceph-mon[77587]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:37.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:37 smithi046 ceph-mon[36898]: osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] boot 2024-05-08T22:59:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:37 smithi046 ceph-mon[36898]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T22:59:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:37 smithi046 ceph-mon[36898]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:37 smithi086 ceph-mon[78610]: osd.6 [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] boot 2024-05-08T22:59:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:37 smithi086 ceph-mon[78610]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T22:59:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:59:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:37 smithi086 ceph-mon[78610]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/898149726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]': finished 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:38.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/366684633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:59:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/898149726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]': finished 2024-05-08T22:59:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T22:59:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/366684633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:39.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/898149726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]: dispatch 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1305daf7-451a-42b5-bc27-04165d5de6eb"}]': finished 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/366684633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:39.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:39 smithi086 ceph-mon[78610]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:40.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:39 smithi053 ceph-mon[77587]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:40.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:39 smithi046 ceph-mon[36898]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:40 smithi053 ceph-mon[77587]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:40 smithi046 ceph-mon[36898]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:40 smithi086 ceph-mon[78610]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:43.227 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:42 smithi053 ceph-mon[77587]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:43.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:42 smithi046 ceph-mon[36898]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:43.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:42 smithi086 ceph-mon[78610]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:44.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:44 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:59:44.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:44 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:44.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:44 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:59:44.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:44 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:44 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:59:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:44 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:45.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:45 smithi053 ceph-mon[77587]: Deploying daemon osd.7 on smithi053 2024-05-08T22:59:45.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:45 smithi053 ceph-mon[77587]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:45.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:45 smithi046 ceph-mon[36898]: Deploying daemon osd.7 on smithi053 2024-05-08T22:59:45.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:45 smithi046 ceph-mon[36898]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:45 smithi086 ceph-mon[78610]: Deploying daemon osd.7 on smithi053 2024-05-08T22:59:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:45 smithi086 ceph-mon[78610]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:47.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:46 smithi053 ceph-mon[77587]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:47.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:46 smithi046 ceph-mon[36898]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:47.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:46 smithi086 ceph-mon[78610]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:49 smithi053 ceph-mon[77587]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:49 smithi086 ceph-mon[78610]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:50.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:49 smithi046 ceph-mon[36898]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:51.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.126+0000 7fc71c9ae700 1 -- 172.21.15.53:0/86551140 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc71802cf50 con 0x7fc7000a1120 2024-05-08T22:59:51.126 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-05-08T22:59:51.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.129+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 msgr2=0x7fc7000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:51.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.129+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 0x7fc7000a35e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b2e0 tx=0x7fc70c005fb0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 msgr2=0x7fc718196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e9a0 tx=0x7fc70800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 shutdown_connections 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc7000a1120 0x7fc7000a35e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718106d00 0x7fc71819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc718101aa0 0x7fc718196f50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 --2- 172.21.15.53:0/86551140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7181043d0 0x7fc718197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 >> 172.21.15.53:0/86551140 conn(0x7fc7180fb490 msgr2=0x7fc718103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 shutdown_connections 2024-05-08T22:59:51.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T22:59:51.130+0000 7fc71d9b0700 1 -- 172.21.15.53:0/86551140 wait complete. 2024-05-08T22:59:51.515 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T22:59:51.517 INFO:tasks.cephadm:Deploying osd.8 on smithi086 with /dev/vg_nvme/lv_2... 2024-05-08T22:59:51.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:59:51.553 INFO:journalctl@ceph.osd.7.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:51.762 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:59:51.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:51.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:51.853 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:59:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:52 smithi053 ceph-mon[77587]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:52 smithi053 ceph-mon[77587]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:52.762 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 22:59:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T22:59:52.545+0000 7f12e25d97c0 -1 osd.7 0 log_to_monitors true 2024-05-08T22:59:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:52 smithi086 ceph-mon[78610]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:52 smithi086 ceph-mon[78610]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:53.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:52 smithi046 ceph-mon[36898]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:53.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:52 smithi046 ceph-mon[36898]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011226 s, 934 MB/s 2024-05-08T22:59:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-08T22:59:53.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-05-08T22:59:53.742 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:53.856 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:53 smithi053 ceph-mon[77587]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:53.861 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:53 smithi086 ceph-mon[78610]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:54.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:59:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:59:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: from='osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T22:59:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:53 smithi046 ceph-mon[36898]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:59:54.511 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 22:59:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T22:59:54.171+0000 7f12d7c3d700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T22:59:54.511 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 22:59:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T22:59:54.184+0000 7f12cfa2a700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:59:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 -- 172.21.15.86:0/4173223234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 msgr2=0x7fb13c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 --2- 172.21.15.86:0/4173223234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c101420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009ca0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 -- 172.21.15.86:0/4173223234 shutdown_connections 2024-05-08T22:59:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 --2- 172.21.15.86:0/4173223234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102ba0 0x7fb13c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 --2- 172.21.15.86:0/4173223234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c101420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 --2- 172.21.15.86:0/4173223234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102200 0x7fb13c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 -- 172.21.15.86:0/4173223234 >> 172.21.15.86:0/4173223234 conn(0x7fb13c0fc5c0 msgr2=0x7fb13c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 -- 172.21.15.86:0/4173223234 shutdown_connections 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.791+0000 7fb1423c5700 1 -- 172.21.15.86:0/4173223234 wait complete. 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 Processor -- start 2024-05-08T22:59:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 -- start start 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102200 0x7fb13c197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 0x7fb13c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c19ea40 con 0x7fb13c102ba0 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c19eb80 con 0x7fb13c102200 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb1423c5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c19ed10 con 0x7fb13c101000 2024-05-08T22:59:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57496/0 (socket says 172.21.15.86:57496) 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 -- 172.21.15.86:0/539243193 learned_addr learned my addr 172.21.15.86:0/539243193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13b7fe700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102200 0x7fb13c197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb140962700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 0x7fb13c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 -- 172.21.15.86:0/539243193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102200 msgr2=0x7fb13c197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102200 0x7fb13c197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 -- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 msgr2=0x7fb13c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 0x7fb13c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:59:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 -- 172.21.15.86:0/539243193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300095f0 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb140962700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 0x7fb13c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.792+0000 7fb13bfff700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb130003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.793+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13001d070 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.793+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13000f460 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.793+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c19ef90 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.793+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130017650 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.793+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c19f2f0 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.794+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c1084e0 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.795+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb13000f5d0 con 0x7fb13c101000 2024-05-08T22:59:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.795+0000 7fb1397fa700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 0x7fb1240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:59:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.795+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5430+0+0 (secure 0 0 0) 0x7fb1300cd330 con 0x7fb13c101000 2024-05-08T22:59:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.796+0000 7fb13b7fe700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 0x7fb1240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:59:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.796+0000 7fb13b7fe700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 0x7fb1240a35e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005fd0 tx=0x7fb12c00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:59:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:54.799+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb130097690 con 0x7fb13c101000 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='osd.7 ' entity='osd.7' 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] boot 2024-05-08T22:59:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T22:59:54.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T22:59:55.000+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb13c02ceb0 con 0x7fb1240a1120 2024-05-08T22:59:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:59:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:59:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='osd.7 ' entity='osd.7' 2024-05-08T22:59:55.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] boot 2024-05-08T22:59:55.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T22:59:55.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T22:59:55.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='osd.7 ' entity='osd.7' 2024-05-08T22:59:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: osd.7 [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] boot 2024-05-08T22:59:55.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T22:59:55.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi053 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:55.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:55.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:55.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:55 smithi086 ceph-mon[78610]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:59:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T22:59:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T22:59:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi053 2024-05-08T22:59:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-08T22:59:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:55 smithi053 ceph-mon[77587]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:59:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi053 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:59:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:55 smithi046 ceph-mon[36898]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:59:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:56 smithi086 ceph-mon[78610]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:56 smithi086 ceph-mon[78610]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:59:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:56 smithi053 ceph-mon[77587]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:56 smithi053 ceph-mon[77587]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:59:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:56 smithi046 ceph-mon[36898]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:59:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:56 smithi046 ceph-mon[36898]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:59:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/2469427914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: pgmap v149: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]': finished 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:59:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/965154910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:58.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/2469427914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:58.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:58.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: pgmap v149: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:58.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]': finished 2024-05-08T22:59:58.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:59:58.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:59:58.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/965154910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/2469427914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]: dispatch 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: pgmap v149: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79a15f0-16af-47ba-af13-ceb2e1df5f43"}]': finished 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:59:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:59:58.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/965154910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:59:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:58 smithi086 ceph-mon[78610]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:59:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:58 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:59:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:58 smithi053 ceph-mon[77587]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:59:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:58 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:59:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:58 smithi046 ceph-mon[36898]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:59:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:58 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 22:59:59 smithi086 ceph-mon[78610]: pgmap v152: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 22:59:59 smithi053 ceph-mon[77587]: pgmap v152: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 22:59:59 smithi046 ceph-mon[36898]: pgmap v152: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:00 smithi086 ceph-mon[78610]: overall HEALTH_OK 2024-05-08T23:00:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:00 smithi053 ceph-mon[77587]: overall HEALTH_OK 2024-05-08T23:00:01.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:00 smithi046 ceph-mon[36898]: overall HEALTH_OK 2024-05-08T23:00:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:01 smithi086 ceph-mon[78610]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:01 smithi053 ceph-mon[77587]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:01 smithi046 ceph-mon[36898]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:03 smithi086 ceph-mon[78610]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:00:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:03 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:03 smithi053 ceph-mon[77587]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:00:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:03 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:03 smithi046 ceph-mon[36898]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:00:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:03 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:05.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:04 smithi086 ceph-mon[78610]: Deploying daemon osd.8 on smithi086 2024-05-08T23:00:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:04 smithi053 ceph-mon[77587]: Deploying daemon osd.8 on smithi086 2024-05-08T23:00:05.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:04 smithi046 ceph-mon[36898]: Deploying daemon osd.8 on smithi086 2024-05-08T23:00:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:05 smithi086 ceph-mon[78610]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:05 smithi053 ceph-mon[77587]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:05 smithi046 ceph-mon[36898]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:07 smithi086 ceph-mon[78610]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:07 smithi053 ceph-mon[77587]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:07 smithi046 ceph-mon[36898]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.259+0000 7fb1397fa700 1 -- 172.21.15.86:0/539243193 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb13c02ceb0 con 0x7fb1240a1120 2024-05-08T23:00:09.259 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 8 on host 'smithi086' 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 msgr2=0x7fb1240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 0x7fb1240a35e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005fd0 tx=0x7fb12c00a560 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 msgr2=0x7fb13c196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb130003c20 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 shutdown_connections 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb1240a1120 0x7fb1240a35e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c102ba0 0x7fb13c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c101000 0x7fb13c196f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 --2- 172.21.15.86:0/539243193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb13c102200 0x7fb13c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 >> 172.21.15.86:0/539243193 conn(0x7fb13c0fc5c0 msgr2=0x7fb13c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 shutdown_connections 2024-05-08T23:00:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:09.261+0000 7fb1423c5700 1 -- 172.21.15.86:0/539243193 wait complete. 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:09.758 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:09.940 DEBUG:teuthology.orchestra.run.smithi086:osd.8> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T23:00:09.943 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-05-08T23:00:09.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:00:09.975 INFO:journalctl@ceph.osd.8.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:00:10.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:10.316 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:00:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:10 smithi086 ceph-mon[78610]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:10 smithi086 ceph-mon[78610]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:10 smithi086 ceph-mon[78610]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:11.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:00:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:00:10.877+0000 7fcbfed677c0 -1 osd.8 0 log_to_monitors true 2024-05-08T23:00:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:10 smithi053 ceph-mon[77587]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:10 smithi053 ceph-mon[77587]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:10 smithi053 ceph-mon[77587]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:10 smithi046 ceph-mon[36898]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:10 smithi046 ceph-mon[36898]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:00:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:10 smithi046 ceph-mon[36898]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:00:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:00:12.119 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-08T23:00:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-08T23:00:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.029424 s, 356 MB/s 2024-05-08T23:00:12.120 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 2024-05-08T23:00:12.120 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-08T23:00:12.213 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:00:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:00:11.954+0000 7fcbf43cb700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T23:00:12.213 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:00:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:00:11.965+0000 7fcbf01c0700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:00:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:00:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi086 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 5704M 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi086 2024-05-08T23:00:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 5704M 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi086 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 5704M 2024-05-08T23:00:12.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:12.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:12.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:12.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-05-08T23:00:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:00:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:12 smithi086 ceph-mon[78610]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:00:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:12 smithi086 ceph-mon[78610]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:00:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:12 smithi086 ceph-mon[78610]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:12 smithi053 ceph-mon[77587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:00:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:12 smithi053 ceph-mon[77587]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:00:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:12 smithi053 ceph-mon[77587]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:12 smithi046 ceph-mon[36898]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:00:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:12 smithi046 ceph-mon[36898]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T23:00:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:12 smithi046 ceph-mon[36898]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:00:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T23:00:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T23:00:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] boot 2024-05-08T23:00:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:00:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T23:00:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T23:00:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] boot 2024-05-08T23:00:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:00:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T23:00:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T23:00:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: osd.8 [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] boot 2024-05-08T23:00:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:00:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 -- 172.21.15.46:0/890941392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181041d0 msgr2=0x7fa5181065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 --2- 172.21.15.46:0/890941392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181041d0 0x7fa5181065c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa504009990 tx=0x7fa504009ca0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 -- 172.21.15.46:0/890941392 shutdown_connections 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 --2- 172.21.15.46:0/890941392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa518106b00 0x7fa518108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 --2- 172.21.15.46:0/890941392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5181018a0 0x7fa518103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 --2- 172.21.15.46:0/890941392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181041d0 0x7fa5181065c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 -- 172.21.15.46:0/890941392 >> 172.21.15.46:0/890941392 conn(0x7fa5180fb2b0 msgr2=0x7fa5180fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 -- 172.21.15.46:0/890941392 shutdown_connections 2024-05-08T23:00:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.278+0000 7fa517e00700 1 -- 172.21.15.46:0/890941392 wait complete. 2024-05-08T23:00:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.279+0000 7fa517e00700 1 Processor -- start 2024-05-08T23:00:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 -- start start 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 0x7fa518199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa518106b00 0x7fa5181a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a0a80 con 0x7fa5181018a0 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a0bc0 con 0x7fa518106b00 2024-05-08T23:00:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a0d50 con 0x7fa5181041d0 2024-05-08T23:00:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa516a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa516a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50926/0 (socket says 172.21.15.46:50926) 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa516a00700 1 -- 172.21.15.46:0/6438538 learned_addr learned my addr 172.21.15.46:0/6438538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa517400700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa518106b00 0x7fa5181a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.280+0000 7fa516000700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 0x7fa518199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 -- 172.21.15.46:0/6438538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 msgr2=0x7fa518199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 0x7fa518199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 -- 172.21.15.46:0/6438538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa518106b00 msgr2=0x7fa5181a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa518106b00 0x7fa5181a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 -- 172.21.15.46:0/6438538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040095f0 con 0x7fa5181018a0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516000700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 0x7fa518199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa516a00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9d0 tx=0x7fa50800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508009790 con 0x7fa5181018a0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50800cbd0 con 0x7fa5181018a0 2024-05-08T23:00:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508018680 con 0x7fa5181018a0 2024-05-08T23:00:14.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.281+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5181a0fd0 con 0x7fa5181018a0 2024-05-08T23:00:14.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.282+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a1390 con 0x7fa5181018a0 2024-05-08T23:00:14.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.283+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5181086d0 con 0x7fa5181018a0 2024-05-08T23:00:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.285+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa50800cd40 con 0x7fa5181018a0 2024-05-08T23:00:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.287+0000 7fa51ca00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 0x7fa4fc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.287+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fa508014070 con 0x7fa5181018a0 2024-05-08T23:00:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.287+0000 7fa516000700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 0x7fa4fc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.288+0000 7fa516000700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 0x7fa4fc0a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b2e0 tx=0x7fa504005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:14.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.289+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa5080965d0 con 0x7fa5181018a0 2024-05-08T23:00:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:14.513+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa51802cf20 con 0x7fa4fc0a1120 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:15.232 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:14 smithi046 ceph-mon[36898]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T23:00:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:14 smithi053 ceph-mon[77587]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:14 smithi086 ceph-mon[78610]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:15 smithi053 ceph-mon[77587]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:00:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:15 smithi046 ceph-mon[36898]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:00:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:15 smithi086 ceph-mon[78610]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T23:00:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:16 smithi053 ceph-mon[77587]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:17.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:16 smithi046 ceph-mon[36898]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:17.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:16 smithi086 ceph-mon[78610]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:00:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]: dispatch 2024-05-08T23:00:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]': finished 2024-05-08T23:00:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:00:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:18.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3677963434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:18.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:00:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]: dispatch 2024-05-08T23:00:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]': finished 2024-05-08T23:00:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:00:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3677963434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T23:00:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]: dispatch 2024-05-08T23:00:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2650290676' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "885ac456-a429-4688-b3f7-c66c1d785c1c"}]': finished 2024-05-08T23:00:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:00:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:18.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3677963434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:19.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:18 smithi053 ceph-mon[77587]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:19.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:18 smithi046 ceph-mon[36898]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:18 smithi086 ceph-mon[78610]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:20 smithi053 ceph-mon[77587]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:00:21.262 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:20 smithi046 ceph-mon[36898]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:00:21.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:20 smithi086 ceph-mon[78610]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:00:23.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:22 smithi053 ceph-mon[77587]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T23:00:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:22 smithi046 ceph-mon[36898]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T23:00:23.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:22 smithi086 ceph-mon[78610]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-08T23:00:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:00:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:23 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:23 smithi053 ceph-mon[77587]: Deploying daemon osd.9 on smithi046 2024-05-08T23:00:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:00:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:23 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:23 smithi046 ceph-mon[36898]: Deploying daemon osd.9 on smithi046 2024-05-08T23:00:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:00:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:23 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:23 smithi086 ceph-mon[78610]: Deploying daemon osd.9 on smithi046 2024-05-08T23:00:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:24 smithi053 ceph-mon[77587]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:00:25.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:24 smithi046 ceph-mon[36898]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:00:25.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:24 smithi086 ceph-mon[78610]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:00:27.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:26 smithi053 ceph-mon[77587]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:00:27.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:26 smithi046 ceph-mon[36898]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:00:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:26 smithi086 ceph-mon[78610]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:00:30.169 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.170 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.170 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:30.170 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:29 smithi046 ceph-mon[36898]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:29 smithi086 ceph-mon[78610]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:29 smithi053 ceph-mon[77587]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.714+0000 7fa51ca00700 1 -- 172.21.15.46:0/6438538 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa51802cf20 con 0x7fa4fc0a1120 2024-05-08T23:00:31.714 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-05-08T23:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 msgr2=0x7fa4fc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 0x7fa4fc0a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b2e0 tx=0x7fa504005fb0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 msgr2=0x7fa518198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9d0 tx=0x7fa50800ece0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 shutdown_connections 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa518106b00 0x7fa5181a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa4fc0a1120 0x7fa4fc0a35e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5181018a0 0x7fa518198f40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 --2- 172.21.15.46:0/6438538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5181041d0 0x7fa518199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.716+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 >> 172.21.15.46:0/6438538 conn(0x7fa5180fb2b0 msgr2=0x7fa518101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.717+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 shutdown_connections 2024-05-08T23:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:00:31.717+0000 7fa517e00700 1 -- 172.21.15.46:0/6438538 wait complete. 2024-05-08T23:00:32.017 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T23:00:32.021 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-05-08T23:00:32.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:00:32.110 INFO:journalctl@ceph.osd.9.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:32.274 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:00:32.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:33.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:32 smithi053 ceph-mon[77587]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:32 smithi053 ceph-mon[77587]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.183 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:32 smithi046 ceph-mon[36898]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.184 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:32 smithi046 ceph-mon[36898]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.184 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:00:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:00:32.900+0000 7f9dbabf67c0 -1 osd.9 0 log_to_monitors true 2024-05-08T23:00:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:32 smithi086 ceph-mon[78610]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:32 smithi086 ceph-mon[78610]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00916139 s, 1.1 GB/s 2024-05-08T23:00:33.730 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:33 smithi053 ceph-mon[77587]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:33 smithi086 ceph-mon[78610]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:00:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:00:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: from='osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:33 smithi046 ceph-mon[36898]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-08T23:00:34.278 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:00:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:00:33.958+0000 7f9daf200700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T23:00:34.278 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:00:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:00:33.975+0000 7f9daa000700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:00:34.336 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:00:35.172 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:34 smithi053 ceph-mon[77587]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T23:00:35.172 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:34 smithi053 ceph-mon[77587]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:00:35.172 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:34 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.172 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:34 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.172 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:34 smithi053 ceph-mon[77587]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:34 smithi086 ceph-mon[78610]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T23:00:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:34 smithi086 ceph-mon[78610]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:00:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:34 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:34 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:34 smithi086 ceph-mon[78610]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:35.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:34 smithi046 ceph-mon[36898]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-08T23:00:35.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:34 smithi046 ceph-mon[36898]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T23:00:35.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:34 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:35.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:34 smithi046 ceph-mon[36898]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 -- 172.21.15.53:0/1828673569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8102b80 msgr2=0x7fcfe8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 --2- 172.21.15.53:0/1828673569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8102b80 0x7fcfe8109430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000b0d0 tx=0x7fcfe000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 -- 172.21.15.53:0/1828673569 shutdown_connections 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 --2- 172.21.15.53:0/1828673569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8102b80 0x7fcfe8109430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 --2- 172.21.15.53:0/1828673569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 --2- 172.21.15.53:0/1828673569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe81021e0 0x7fcfe8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 -- 172.21.15.53:0/1828673569 >> 172.21.15.53:0/1828673569 conn(0x7fcfe80fc580 msgr2=0x7fcfe80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 -- 172.21.15.53:0/1828673569 shutdown_connections 2024-05-08T23:00:35.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.382+0000 7fcfed44b700 1 -- 172.21.15.53:0/1828673569 wait complete. 2024-05-08T23:00:35.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 Processor -- start 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 -- start start 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81021e0 0x7fcfe8195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe8102b80 0x7fcfe819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe819c8f0 con 0x7fcfe81021e0 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe819ca30 con 0x7fcfe8100fe0 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfed44b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe819cbf0 con 0x7fcfe8102b80 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfe6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:35.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfe6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33680/0 (socket says 172.21.15.53:33680) 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfe6ffd700 1 -- 172.21.15.53:0/2157252704 learned_addr learned my addr 172.21.15.53:0/2157252704 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfe67fc700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81021e0 0x7fcfe8195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.383+0000 7fcfe77fe700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe8102b80 0x7fcfe819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 -- 172.21.15.53:0/2157252704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe8102b80 msgr2=0x7fcfe819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe8102b80 0x7fcfe819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 -- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81021e0 msgr2=0x7fcfe8195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81021e0 0x7fcfe8195380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 -- 172.21.15.53:0/2157252704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0009d00 con 0x7fcfe8100fe0 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfe6ffd700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009c70 tx=0x7fcfd8009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd801d070 con 0x7fcfe8100fe0 2024-05-08T23:00:35.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfd8015040 con 0x7fcfe8100fe0 2024-05-08T23:00:35.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe819ced0 con 0x7fcfe8100fe0 2024-05-08T23:00:35.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80039f0 con 0x7fcfe8100fe0 2024-05-08T23:00:35.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.384+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe819d2f0 con 0x7fcfe8100fe0 2024-05-08T23:00:35.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.385+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe8108470 con 0x7fcfe8100fe0 2024-05-08T23:00:35.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.386+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcfd8007980 con 0x7fcfe8100fe0 2024-05-08T23:00:35.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.387+0000 7fcfcffff700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 0x7fcfd00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:35.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.388+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fcfd80cfa20 con 0x7fcfe8100fe0 2024-05-08T23:00:35.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.388+0000 7fcfe67fc700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 0x7fcfd00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:35.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.388+0000 7fcfe67fc700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 0x7fcfd00a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009c60 tx=0x7fcfdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:35.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.391+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcfd80999a0 con 0x7fcfe8100fe0 2024-05-08T23:00:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:35.615+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfe802ceb0 con 0x7fcfd00a11a0 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] boot 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:00:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi046 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 4318M 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:36.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T23:00:36.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T23:00:36.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] boot 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi046 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 4318M 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:36.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:36.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:36.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:36.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: osd.9 [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] boot 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi046 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 4318M 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:37.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:37 smithi086 ceph-mon[78610]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T23:00:37.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:37 smithi086 ceph-mon[78610]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:37.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:37 smithi053 ceph-mon[77587]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T23:00:37.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:37 smithi053 ceph-mon[77587]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:37.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:37 smithi046 ceph-mon[36898]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T23:00:37.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:37 smithi046 ceph-mon[36898]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:38.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/1504783981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]': finished 2024-05-08T23:00:38.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:38 smithi086 ceph-mon[78610]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:00:38.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:38.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/1504783981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]': finished 2024-05-08T23:00:38.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:38 smithi053 ceph-mon[77587]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:00:38.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:38.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/1504783981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]: dispatch 2024-05-08T23:00:38.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66e06a38-ad3e-4b9d-af1f-8f62447ec12a"}]': finished 2024-05-08T23:00:38.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:38 smithi046 ceph-mon[36898]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:00:38.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:39.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:39 smithi086 ceph-mon[78610]: from='client.? 172.21.15.53:0/3838498806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:39.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:39 smithi086 ceph-mon[78610]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:39.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:39 smithi053 ceph-mon[77587]: from='client.? 172.21.15.53:0/3838498806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:39.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:39 smithi053 ceph-mon[77587]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:39.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:39 smithi046 ceph-mon[36898]: from='client.? 172.21.15.53:0/3838498806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:39.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:39 smithi046 ceph-mon[36898]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:40 smithi053 ceph-mon[77587]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:40 smithi046 ceph-mon[36898]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:40 smithi086 ceph-mon[78610]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:43.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:42 smithi053 ceph-mon[77587]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:43.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:42 smithi046 ceph-mon[36898]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:43.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:42 smithi086 ceph-mon[78610]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:44.256 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:00:44.256 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:44.256 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:43 smithi053 ceph-mon[77587]: Deploying daemon osd.10 on smithi053 2024-05-08T23:00:44.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:00:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:43 smithi046 ceph-mon[36898]: Deploying daemon osd.10 on smithi053 2024-05-08T23:00:44.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:00:44.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:44.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:43 smithi086 ceph-mon[78610]: Deploying daemon osd.10 on smithi053 2024-05-08T23:00:45.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:44 smithi053 ceph-mon[77587]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:44 smithi046 ceph-mon[36898]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:45.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:44 smithi086 ceph-mon[78610]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:47.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:46 smithi053 ceph-mon[77587]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:47.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:46 smithi046 ceph-mon[36898]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:47.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:46 smithi086 ceph-mon[78610]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:48.854 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:48.854 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:48.854 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:49.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.023+0000 7fcfcffff700 1 -- 172.21.15.53:0/2157252704 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fcfe802ceb0 con 0x7fcfd00a11a0 2024-05-08T23:00:50.023 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 msgr2=0x7fcfd00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 0x7fcfd00a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009c60 tx=0x7fcfdc009260 comp rx=0 tx=0).stop 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 msgr2=0x7fcfe8194e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009c70 tx=0x7fcfd8009f80 comp rx=0 tx=0).stop 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 shutdown_connections 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe8102b80 0x7fcfe819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe8100fe0 0x7fcfe8194e40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcfd00a11a0 0x7fcfd00a3660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:50.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 --2- 172.21.15.53:0/2157252704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81021e0 0x7fcfe8195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:50.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 >> 172.21.15.53:0/2157252704 conn(0x7fcfe80fc580 msgr2=0x7fcfe81055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:50.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 shutdown_connections 2024-05-08T23:00:50.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-08T23:00:50.026+0000 7fcfed44b700 1 -- 172.21.15.53:0/2157252704 wait complete. 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:50.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:50.456 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T23:00:50.459 INFO:tasks.cephadm:Deploying osd.11 on smithi086 with /dev/vg_nvme/lv_1... 2024-05-08T23:00:50.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:00:50.494 INFO:journalctl@ceph.osd.10.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T23:00:50.694 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:00:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:50 smithi053 ceph-mon[77587]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:51.154 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.154 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:51.155 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.155 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:50 smithi086 ceph-mon[78610]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:00:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:50 smithi046 ceph-mon[36898]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:52.011 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:00:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:00:51.741+0000 7ff9810d17c0 -1 osd.10 0 log_to_monitors true 2024-05-08T23:00:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:51 smithi053 ceph-mon[77587]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:51 smithi053 ceph-mon[77587]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.071 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:00:52.072 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:00:52.072 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-08T23:00:52.072 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-08T23:00:52.072 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132592 s, 791 MB/s 2024-05-08T23:00:52.072 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-08T23:00:52.254 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:51 smithi086 ceph-mon[78610]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.255 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:51 smithi086 ceph-mon[78610]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:51 smithi046 ceph-mon[36898]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:51 smithi046 ceph-mon[36898]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:00:52.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-05-08T23:00:52.687 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:00:53.212 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:00:53.212 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.261 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:00:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:00:52.997+0000 7ff976735700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T23:00:53.261 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:00:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:00:53.013+0000 7ff97252a700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:00:53.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:53.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:53.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:00:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:00:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:00:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:00:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1068391611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 msgr2=0x7f3c24103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1068391611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24103f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009990 tx=0x7f3c14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1068391611 shutdown_connections 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1068391611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24106d90 0x7f3c24109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1068391611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24103f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1068391611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24104460 0x7f3c24106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1068391611 >> 172.21.15.86:0/1068391611 conn(0x7f3c240fb4c0 msgr2=0x7f3c240fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1068391611 shutdown_connections 2024-05-08T23:00:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1068391611 wait complete. 2024-05-08T23:00:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 Processor -- start 2024-05-08T23:00:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.750+0000 7f3c28a0c700 1 -- start start 2024-05-08T23:00:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24104460 0x7f3c24197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24106d90 0x7f3c2419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2419ea20 con 0x7f3c24104460 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2419eb60 con 0x7f3c24106d90 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2419ecf0 con 0x7f3c24101b30 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53542/0 (socket says 172.21.15.86:53542) 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 -- 172.21.15.86:0/1368497738 learned_addr learned my addr 172.21.15.86:0/1368497738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c22d9d700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24106d90 0x7f3c2419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 -- 172.21.15.86:0/1368497738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24106d90 msgr2=0x7f3c2419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24106d90 0x7f3c2419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 -- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24104460 msgr2=0x7f3c24197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24104460 0x7f3c24197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 -- 172.21.15.86:0/1368497738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140095f0 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c2259c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b5c0 tx=0x7f3c1400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1401d070 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c14022470 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14017650 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2419ef70 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.751+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2419f2d0 con 0x7f3c24101b30 2024-05-08T23:00:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.752+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c240fd040 con 0x7f3c24101b30 2024-05-08T23:00:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.753+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3c140225e0 con 0x7f3c24101b30 2024-05-08T23:00:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.754+0000 7f3c137fe700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 0x7f3c0c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:00:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.754+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3c140cd670 con 0x7f3c24101b30 2024-05-08T23:00:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.754+0000 7f3c21d9b700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 0x7f3c0c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:00:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.755+0000 7f3c21d9b700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 0x7f3c0c0a35e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005950 tx=0x7f3c180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:00:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.758+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3c14097720 con 0x7f3c24101b30 2024-05-08T23:00:53.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:00:53.959+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c2402cf00 con 0x7f3c0c0a1120 2024-05-08T23:00:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi053 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:53 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:54.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi053 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:53 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:54.360 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi053 2024-05-08T23:00:54.360 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:00:54.361 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:53 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:00:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T23:00:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T23:00:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] boot 2024-05-08T23:00:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:00:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T23:00:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:54 smithi053 ceph-mon[77587]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:55.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T23:00:55.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] boot 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T23:00:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:54 smithi046 ceph-mon[36898]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T23:00:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T23:00:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:00:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: osd.10 [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] boot 2024-05-08T23:00:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:00:55.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:00:55.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T23:00:55.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:54 smithi086 ceph-mon[78610]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/970952277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]': finished 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.86:0/3434845619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/970952277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]': finished 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.86:0/3434845619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/970952277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]: dispatch 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b97781b-2ef0-485c-b42b-47099ce3ff54"}]': finished 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:57.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.86:0/3434845619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:00:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:57 smithi086 ceph-mon[78610]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:00:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:57 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:57 smithi086 ceph-mon[78610]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:57 smithi053 ceph-mon[77587]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:00:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:57 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:57 smithi053 ceph-mon[77587]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:57 smithi046 ceph-mon[36898]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:00:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:57 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:00:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:57 smithi046 ceph-mon[36898]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:00:58 smithi053 ceph-mon[77587]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:00:58 smithi046 ceph-mon[36898]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:00:59.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:00:58 smithi086 ceph-mon[78610]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:00 smithi053 ceph-mon[77587]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:01:01.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:00 smithi046 ceph-mon[36898]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:01:01.307 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:00 smithi086 ceph-mon[78610]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:01:02.244 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:01 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:01:02.245 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:01 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:02.245 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:01 smithi086 ceph-mon[78610]: Deploying daemon osd.11 on smithi086 2024-05-08T23:01:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:01 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:01:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:01 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:01 smithi053 ceph-mon[77587]: Deploying daemon osd.11 on smithi086 2024-05-08T23:01:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:01 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:01:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:01 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:01 smithi046 ceph-mon[36898]: Deploying daemon osd.11 on smithi086 2024-05-08T23:01:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:02 smithi053 ceph-mon[77587]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T23:01:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:02 smithi046 ceph-mon[36898]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T23:01:03.316 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:02 smithi086 ceph-mon[78610]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-08T23:01:05.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:04 smithi086 ceph-mon[78610]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:01:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:04 smithi053 ceph-mon[77587]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:01:05.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:04 smithi046 ceph-mon[36898]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:01:06.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:06.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:06.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:06 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:07.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:06 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:07.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:06 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T23:01:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:07.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:07.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:07 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:07 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T23:01:08.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:07 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:08.166 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 11 on host 'smithi086' 2024-05-08T23:01:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.166+0000 7f3c137fe700 1 -- 172.21.15.86:0/1368497738 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f3c2402cf00 con 0x7f3c0c0a1120 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 msgr2=0x7f3c0c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 0x7f3c0c0a35e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005950 tx=0x7f3c180058e0 comp rx=0 tx=0).stop 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 msgr2=0x7f3c24196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b5c0 tx=0x7f3c1400f690 comp rx=0 tx=0).stop 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 shutdown_connections 2024-05-08T23:01:08.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c24106d90 0x7f3c2419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c24101b30 0x7f3c24196ee0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3c0c0a1120 0x7f3c0c0a35e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 --2- 172.21.15.86:0/1368497738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24104460 0x7f3c24197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 >> 172.21.15.86:0/1368497738 conn(0x7f3c240fb4c0 msgr2=0x7f3c241025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 shutdown_connections 2024-05-08T23:01:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:08.168+0000 7f3c28a0c700 1 -- 172.21.15.86:0/1368497738 wait complete. 2024-05-08T23:01:08.568 DEBUG:teuthology.orchestra.run.smithi086:osd.11> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T23:01:08.570 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T23:01:08.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:01:08.602 INFO:journalctl@ceph.osd.11.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:01:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:01:09.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:09 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:09.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:09 smithi046 ceph-mon[36898]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:01:09.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:09 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:09.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:09 smithi086 ceph-mon[78610]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:01:09.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:09.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:09 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:09.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:09 smithi053 ceph-mon[77587]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T23:01:09.963 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:01:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:01:09.611+0000 7f66af6cf7c0 -1 osd.11 0 log_to_monitors true 2024-05-08T23:01:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 -- 172.21.15.46:0/779313127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 msgr2=0x7fbae41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 --2- 172.21.15.46:0/779313127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae41067b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009990 tx=0x7fbad0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 -- 172.21.15.46:0/779313127 shutdown_connections 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 --2- 172.21.15.46:0/779313127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 --2- 172.21.15.46:0/779313127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae4103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 --2- 172.21.15.46:0/779313127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae41067b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 -- 172.21.15.46:0/779313127 >> 172.21.15.46:0/779313127 conn(0x7fbae40fb4a0 msgr2=0x7fbae40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 -- 172.21.15.46:0/779313127 shutdown_connections 2024-05-08T23:01:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.261+0000 7fbaebc00700 1 -- 172.21.15.46:0/779313127 wait complete. 2024-05-08T23:01:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.262+0000 7fbaebc00700 1 Processor -- start 2024-05-08T23:01:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.262+0000 7fbaebc00700 1 -- start start 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae41959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4192560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4192d00 con 0x7fbae41043c0 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4192e70 con 0x7fbae4101a90 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbaebc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4193030 con 0x7fbae4106cf0 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44988/0 (socket says 172.21.15.46:44988) 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4192560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae3e00700 1 -- 172.21.15.46:0/1898422620 learned_addr learned my addr 172.21.15.46:0/1898422620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9200700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae41959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:10.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 -- 172.21.15.46:0/1898422620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 msgr2=0x7fbae41959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae41959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 -- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 msgr2=0x7fbae4195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9c00700 1 -- 172.21.15.46:0/1898422620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00095f0 con 0x7fbae4106cf0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae3e00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.263+0000 7fbae9200700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae41959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbae9c00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4192560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00ef90 tx=0x7fbadc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc018070 con 0x7fbae4106cf0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41a5630 con 0x7fbae4106cf0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbadc00f040 con 0x7fbae4106cf0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc00cc30 con 0x7fbae4106cf0 2024-05-08T23:01:10.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.264+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41a5ab0 con 0x7fbae4106cf0 2024-05-08T23:01:10.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.265+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae41088f0 con 0x7fbae4106cf0 2024-05-08T23:01:10.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.267+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbadc003a00 con 0x7fbae4106cf0 2024-05-08T23:01:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.268+0000 7fbae1600700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 0x7fbacc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.268+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fbadc0cbf20 con 0x7fbae4106cf0 2024-05-08T23:01:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.268+0000 7fbae9200700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 0x7fbacc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:10.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.269+0000 7fbae9200700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 0x7fbacc0a35e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbad8011490 tx=0x7fbad8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.272+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbadc095c70 con 0x7fbae4106cf0 2024-05-08T23:01:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:10 smithi046 ceph-mon[36898]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:10 smithi046 ceph-mon[36898]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.432 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:10 smithi086 ceph-mon[78610]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.432 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:10 smithi086 ceph-mon[78610]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.486+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbae410c860 con 0x7fbae4106cf0 2024-05-08T23:01:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.486+0000 7fbae1600700 1 -- 172.21.15.46:0/1898422620 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) v1 ==== 74+0+133 (secure 0 0 0) 0x7fbadc099290 con 0x7fbae4106cf0 2024-05-08T23:01:10.486 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 msgr2=0x7fbacc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 0x7fbacc0a35e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbad8011490 tx=0x7fbad8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 msgr2=0x7fbae4192560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4192560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00ef90 tx=0x7fbadc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 shutdown_connections 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae4106cf0 0x7fbae4192560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae4101a90 0x7fbae41959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbacc0a1120 0x7fbacc0a35e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 --2- 172.21.15.46:0/1898422620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae41043c0 0x7fbae4195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 >> 172.21.15.46:0/1898422620 conn(0x7fbae40fb4a0 msgr2=0x7fbae41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 shutdown_connections 2024-05-08T23:01:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:10.489+0000 7fbaebc00700 1 -- 172.21.15.46:0/1898422620 wait complete. 2024-05-08T23:01:10.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:10 smithi053 ceph-mon[77587]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:10 smithi053 ceph-mon[77587]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:01:10.875 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715209253,"num_in_osds":12,"osd_in_since":1715209255,"num_remapped_pgs":0} 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1898422620' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:11.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: Detected new or changed devices on smithi086 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 4278M 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:11 smithi086 ceph-mon[78610]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:11.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:01:11.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:01:11.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:11.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1898422620' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: Detected new or changed devices on smithi086 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 4278M 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:11.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:11 smithi053 ceph-mon[77587]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1898422620' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: Detected new or changed devices on smithi086 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:01:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 4278M 2024-05-08T23:01:11.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:11.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:11.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:11.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:11 smithi046 ceph-mon[36898]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:11.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:01:11.963 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:01:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:01:11.603+0000 7f66a4d33700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T23:01:11.963 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:01:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:01:11.622+0000 7f669cb20700 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:01:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:01:12.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:12 smithi046 ceph-mon[36898]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:01:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:12 smithi046 ceph-mon[36898]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:01:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:12 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:12 smithi046 ceph-mon[36898]: from='osd.11 ' entity='osd.11' 2024-05-08T23:01:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:12 smithi086 ceph-mon[78610]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:01:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:12 smithi086 ceph-mon[78610]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:01:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:12 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:12 smithi086 ceph-mon[78610]: from='osd.11 ' entity='osd.11' 2024-05-08T23:01:12.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:12 smithi053 ceph-mon[77587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-08T23:01:12.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:12 smithi053 ceph-mon[77587]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T23:01:12.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:12 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:12.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:12 smithi053 ceph-mon[77587]: from='osd.11 ' entity='osd.11' 2024-05-08T23:01:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 -- 172.21.15.46:0/3157682104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c106cf0 msgr2=0x7fdc0c109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 --2- 172.21.15.46:0/3157682104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c109160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400b0a0 tx=0x7fdc0400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 -- 172.21.15.46:0/3157682104 shutdown_connections 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 --2- 172.21.15.46:0/3157682104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c109160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 --2- 172.21.15.46:0/3157682104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c101a90 0x7fdc0c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 --2- 172.21.15.46:0/3157682104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 -- 172.21.15.46:0/3157682104 >> 172.21.15.46:0/3157682104 conn(0x7fdc0c0fb4a0 msgr2=0x7fdc0c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 -- 172.21.15.46:0/3157682104 shutdown_connections 2024-05-08T23:01:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.384+0000 7fdc12a00700 1 -- 172.21.15.46:0/3157682104 wait complete. 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.385+0000 7fdc12a00700 1 Processor -- start 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.385+0000 7fdc12a00700 1 -- start start 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c101a90 0x7fdc0c197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19eb70 con 0x7fdc0c1043c0 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19ecb0 con 0x7fdc0c106cf0 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc12a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19ee40 con 0x7fdc0c101a90 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36210/0 (socket says 172.21.15.46:36210) 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 -- 172.21.15.46:0/2257878147 learned_addr learned my addr 172.21.15.46:0/2257878147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:13.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0be00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c101a90 0x7fdc0c197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc10a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 -- 172.21.15.46:0/2257878147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c101a90 msgr2=0x7fdc0c197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c101a90 0x7fdc0c197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 -- 172.21.15.46:0/2257878147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c106cf0 msgr2=0x7fdc0c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.386+0000 7fdc0b400700 1 -- 172.21.15.46:0/2257878147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009d00 con 0x7fdc0c1043c0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc0b400700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e9d0 tx=0x7fdbf800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8009790 con 0x7fdc0c1043c0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf800ee70 con 0x7fdc0c1043c0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c19f120 con 0x7fdc0c1043c0 2024-05-08T23:01:13.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8010ea0 con 0x7fdc0c1043c0 2024-05-08T23:01:13.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.387+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c19f620 con 0x7fdc0c1043c0 2024-05-08T23:01:13.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.388+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c0fd020 con 0x7fdc0c1043c0 2024-05-08T23:01:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.390+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdbf800cbd0 con 0x7fdc0c1043c0 2024-05-08T23:01:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.393+0000 7fdc08c00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 0x7fdbf00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.393+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbf8014070 con 0x7fdc0c1043c0 2024-05-08T23:01:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.393+0000 7fdc0be00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 0x7fdbf00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.393+0000 7fdc0be00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 0x7fdbf00a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009c60 tx=0x7fdbfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:13.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.395+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdbf80966a0 con 0x7fdc0c1043c0 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: purged_snaps scrub starts 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: purged_snaps scrub ok 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] boot 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:13 smithi086 ceph-mon[78610]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: purged_snaps scrub starts 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: purged_snaps scrub ok 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] boot 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:01:13.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:13 smithi053 ceph-mon[77587]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:13.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: purged_snaps scrub starts 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: purged_snaps scrub ok 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: osd.11 [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] boot 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:01:13.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:13 smithi046 ceph-mon[36898]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:01:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.611+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fdc0c04ec60 con 0x7fdc0c1043c0 2024-05-08T23:01:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.611+0000 7fdc08c00700 1 -- 172.21.15.46:0/2257878147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7fdbf8099cc0 con 0x7fdc0c1043c0 2024-05-08T23:01:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 msgr2=0x7fdbf00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 0x7fdbf00a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009c60 tx=0x7fdbfc009260 comp rx=0 tx=0).stop 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 msgr2=0x7fdc0c197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e9d0 tx=0x7fdbf800ece0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 shutdown_connections 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c106cf0 0x7fdc0c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc0c101a90 0x7fdc0c197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdbf00a1120 0x7fdbf00a35e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 --2- 172.21.15.46:0/2257878147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c1043c0 0x7fdc0c197570 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 >> 172.21.15.46:0/2257878147 conn(0x7fdc0c0fb4a0 msgr2=0x7fdc0c102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 shutdown_connections 2024-05-08T23:01:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:13.614+0000 7fdc12a00700 1 -- 172.21.15.46:0/2257878147 wait complete. 2024-05-08T23:01:13.975 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1715209272,"num_in_osds":12,"osd_in_since":1715209255,"num_remapped_pgs":0} 2024-05-08T23:01:13.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:01:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:01:14.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:14 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2257878147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:14.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:14 smithi046 ceph-mon[36898]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:01:14.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:14 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2257878147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:14.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:14 smithi086 ceph-mon[78610]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:01:15.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:14 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2257878147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:01:15.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:14 smithi053 ceph-mon[77587]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 -- 172.21.15.46:0/3770646965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c107020 msgr2=0x7f8d8c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 --2- 172.21.15.46:0/3770646965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c107020 0x7f8d8c107400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009990 tx=0x7f8d80009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 -- 172.21.15.46:0/3770646965 shutdown_connections 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 --2- 172.21.15.46:0/3770646965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c0692f0 0x7f8d8c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 --2- 172.21.15.46:0/3770646965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c107020 0x7f8d8c107400 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 --2- 172.21.15.46:0/3770646965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c068950 0x7f8d8c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 -- 172.21.15.46:0/3770646965 >> 172.21.15.46:0/3770646965 conn(0x7f8d8c075f80 msgr2=0x7f8d8c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.476+0000 7f8d94200700 1 -- 172.21.15.46:0/3770646965 shutdown_connections 2024-05-08T23:01:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.477+0000 7f8d94200700 1 -- 172.21.15.46:0/3770646965 wait complete. 2024-05-08T23:01:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 Processor -- start 2024-05-08T23:01:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 -- start start 2024-05-08T23:01:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0692f0 0x7f8d8c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c107020 0x7f8d8c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c19c2b0 con 0x7f8d8c068950 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c19c420 con 0x7f8d8c107020 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d94200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c19c5e0 con 0x7f8d8c0692f0 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d91800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d91800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36224/0 (socket says 172.21.15.46:36224) 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d91800700 1 -- 172.21.15.46:0/3351987122 learned_addr learned my addr 172.21.15.46:0/3351987122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.478+0000 7f8d90e00700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0692f0 0x7f8d8c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d92200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c107020 0x7f8d8c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 -- 172.21.15.46:0/3351987122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0692f0 msgr2=0x7f8d8c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0692f0 0x7f8d8c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 -- 172.21.15.46:0/3351987122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c107020 msgr2=0x7f8d8c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c107020 0x7f8d8c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 -- 172.21.15.46:0/3351987122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800095f0 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d91800700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009fd0 tx=0x7f8d8000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001d070 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d80005050 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80004a90 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.479+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c19c860 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.480+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c1a4120 con 0x7f8d8c068950 2024-05-08T23:01:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.481+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c109540 con 0x7f8d8c068950 2024-05-08T23:01:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.484+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8d800051c0 con 0x7f8d8c068950 2024-05-08T23:01:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.485+0000 7f8d8a000700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 0x7f8d740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.486+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d800cd560 con 0x7f8d8c068950 2024-05-08T23:01:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.486+0000 7f8d90e00700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 0x7f8d740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.486+0000 7f8d90e00700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 0x7f8d740a3660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c198910 tx=0x7f8d7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.488+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d80097180 con 0x7f8d8c068950 2024-05-08T23:01:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.707+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d8c1986c0 con 0x7f8d8c068950 2024-05-08T23:01:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.708+0000 7f8d8a000700 1 -- 172.21.15.46:0/3351987122 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f8d8009a7a0 con 0x7f8d8c068950 2024-05-08T23:01:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:15.708 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:13.608146+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 msgr2=0x7f8d740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 0x7f8d740a3660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c198910 tx=0x7f8d7800b410 comp rx=0 tx=0).stop 2024-05-08T23:01:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 msgr2=0x7f8d8c1973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009fd0 tx=0x7f8d8000f670 comp rx=0 tx=0).stop 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 shutdown_connections 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d8c107020 0x7f8d8c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8d740a11a0 0x7f8d740a3660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d8c068950 0x7f8d8c1973b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 --2- 172.21.15.46:0/3351987122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0692f0 0x7f8d8c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.710+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 >> 172.21.15.46:0/3351987122 conn(0x7f8d8c075f80 msgr2=0x7f8d8c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.711+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 shutdown_connections 2024-05-08T23:01:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:15.711+0000 7f8d94200700 1 -- 172.21.15.46:0/3351987122 wait complete. 2024-05-08T23:01:15.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:15 smithi046 ceph-mon[36898]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:15.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:15 smithi086 ceph-mon[78610]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:16.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:15 smithi053 ceph-mon[77587]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:16.074 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:01:16.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T23:01:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:01:16.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:16 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3351987122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:16.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:16 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3351987122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:17.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:16 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3351987122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 -- 172.21.15.46:0/1273802899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4107030 msgr2=0x7fe6f4107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1273802899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4107030 0x7fe6f4107410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009990 tx=0x7fe6e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 -- 172.21.15.46:0/1273802899 shutdown_connections 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1273802899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4101990 0x7fe6f410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1273802899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4107030 0x7fe6f4107410 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1273802899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 0x7fe6f4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.592+0000 7fe6fc400700 1 -- 172.21.15.46:0/1273802899 >> 172.21.15.46:0/1273802899 conn(0x7fe6f4078190 msgr2=0x7fe6f40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.593+0000 7fe6fc400700 1 -- 172.21.15.46:0/1273802899 shutdown_connections 2024-05-08T23:01:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.593+0000 7fe6fc400700 1 -- 172.21.15.46:0/1273802899 wait complete. 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.593+0000 7fe6fc400700 1 Processor -- start 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 -- start start 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 0x7fe6f4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4107030 0x7fe6f419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f419e510 con 0x7fe6f4100ff0 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f419e680 con 0x7fe6f4101990 2024-05-08T23:01:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f419e840 con 0x7fe6f4107030 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6f9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 0x7fe6f4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6f9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:45688/0 (socket says 172.21.15.46:45688) 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6f9000700 1 -- 172.21.15.46:0/1337077259 learned_addr learned my addr 172.21.15.46:0/1337077259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.594+0000 7fe6fa400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4107030 0x7fe6f419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 -- 172.21.15.46:0/1337077259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4107030 msgr2=0x7fe6f419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4107030 0x7fe6f419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 -- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 msgr2=0x7fe6f4199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 0x7fe6f4199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 -- 172.21.15.46:0/1337077259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0009520 con 0x7fe6f4101990 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f9000700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000eb30 tx=0x7fe6e000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0009880 con 0x7fe6f4101990 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e00044e0 con 0x7fe6f4101990 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0003d80 con 0x7fe6f4101990 2024-05-08T23:01:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e80095f0 con 0x7fe6f4101990 2024-05-08T23:01:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.595+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f41a61b0 con 0x7fe6f4101990 2024-05-08T23:01:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.596+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f4104370 con 0x7fe6f4101990 2024-05-08T23:01:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.598+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe6e0003f00 con 0x7fe6f4101990 2024-05-08T23:01:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.600+0000 7fe6f2000700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 0x7fe6dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.600+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe6e0014070 con 0x7fe6f4101990 2024-05-08T23:01:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.600+0000 7fe6f9a00700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 0x7fe6dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.600+0000 7fe6f9a00700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 0x7fe6dc0a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800b2e0 tx=0x7fe6e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:17.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.602+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6e0096700 con 0x7fe6f4101990 2024-05-08T23:01:17.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.814+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6f419a920 con 0x7fe6f4101990 2024-05-08T23:01:17.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.815+0000 7fe6f2000700 1 -- 172.21.15.46:0/1337077259 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6e000fe60 con 0x7fe6f4101990 2024-05-08T23:01:17.814 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:01:17.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 msgr2=0x7fe6dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:17.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 0x7fe6dc0a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800b2e0 tx=0x7fe6e8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 msgr2=0x7fe6f4199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000eb30 tx=0x7fe6e000ee40 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 shutdown_connections 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6f4107030 0x7fe6f419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6dc0a1120 0x7fe6dc0a35e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6f4100ff0 0x7fe6f4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 --2- 172.21.15.46:0/1337077259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6f4101990 0x7fe6f4199b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.817+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 >> 172.21.15.46:0/1337077259 conn(0x7fe6f4078190 msgr2=0x7fe6f410c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.818+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 shutdown_connections 2024-05-08T23:01:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:17.818+0000 7fe6fc400700 1 -- 172.21.15.46:0/1337077259 wait complete. 2024-05-08T23:01:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:17 smithi086 ceph-mon[78610]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:17.997 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:17 smithi046 ceph-mon[36898]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:18.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:17 smithi053 ceph-mon[77587]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-05-08T23:01:18.191 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.c on smithi086 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.f on smithi086 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.i on smithi086 2024-05-08T23:01:18.192 INFO:tasks.cephadm:Adding mds.l on smithi086 2024-05-08T23:01:18.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l' 2024-05-08T23:01:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:01:18.959 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:18 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1337077259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:19.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:18 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1337077259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:19.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:18 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1337077259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 -- 172.21.15.86:0/1938786400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130108f00 msgr2=0x7fd13010b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1938786400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130108f00 0x7fd13010b370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b0a0 tx=0x7fd12800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 -- 172.21.15.86:0/1938786400 shutdown_connections 2024-05-08T23:01:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1938786400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130108f00 0x7fd13010b370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1938786400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130074f50 0x7fd1300733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1938786400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 0x7fd130073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 -- 172.21.15.86:0/1938786400 >> 172.21.15.86:0/1938786400 conn(0x7fd1300fc5a0 msgr2=0x7fd1300fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 -- 172.21.15.86:0/1938786400 shutdown_connections 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.525+0000 7fd1358fb700 1 -- 172.21.15.86:0/1938786400 wait complete. 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 Processor -- start 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 -- start start 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 0x7fd130199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130108f00 0x7fd1301a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:19.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301a0db0 con 0x7fd130108f00 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301a0ef0 con 0x7fd130073980 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd1358fb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301a1080 con 0x7fd130074f50 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd12e7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd12e7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51746/0 (socket says 172.21.15.86:51746) 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.526+0000 7fd12e7fc700 1 -- 172.21.15.86:0/1945011665 learned_addr learned my addr 172.21.15.86:0/1945011665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12effd700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 0x7fd130199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12f7fe700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130108f00 0x7fd1301a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 -- 172.21.15.86:0/1945011665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 msgr2=0x7fd130199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 0x7fd130199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 -- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130108f00 msgr2=0x7fd1301a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130108f00 0x7fd1301a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 -- 172.21.15.86:0/1945011665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd128009d00 con 0x7fd130074f50 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd12e7fc700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd12000d980 tx=0x7fd12000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120009b40 con 0x7fd130074f50 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1301a1360 con 0x7fd130074f50 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd12000b6e0 con 0x7fd130074f50 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12000f620 con 0x7fd130074f50 2024-05-08T23:01:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.527+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301a18b0 con 0x7fd130074f50 2024-05-08T23:01:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.528+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd130107d10 con 0x7fd130074f50 2024-05-08T23:01:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.529+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd120010460 con 0x7fd130074f50 2024-05-08T23:01:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.530+0000 7fd1348f9700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 0x7fd1180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.530+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1200cc770 con 0x7fd130074f50 2024-05-08T23:01:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.530+0000 7fd12effd700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 0x7fd1180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.531+0000 7fd12effd700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 0x7fd1180a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd124009a10 tx=0x7fd124009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.534+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1200963a0 con 0x7fd130074f50 2024-05-08T23:01:19.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.736+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1300613e0 con 0x7fd1180a1120 2024-05-08T23:01:19.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.745+0000 7fd1348f9700 1 -- 172.21.15.86:0/1945011665 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd1300613e0 con 0x7fd1180a1120 2024-05-08T23:01:19.745 INFO:teuthology.orchestra.run.smithi086.stdout:Scheduled mds.all update... 2024-05-08T23:01:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.747+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 msgr2=0x7fd1180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.747+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 0x7fd1180a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd124009a10 tx=0x7fd124009190 comp rx=0 tx=0).stop 2024-05-08T23:01:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.747+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 msgr2=0x7fd1301997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.747+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd12000d980 tx=0x7fd12000dc90 comp rx=0 tx=0).stop 2024-05-08T23:01:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 shutdown_connections 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd1180a1120 0x7fd1180a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd130108f00 0x7fd1301a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd130073980 0x7fd130199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 --2- 172.21.15.86:0/1945011665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd130074f50 0x7fd1301997b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 >> 172.21.15.86:0/1945011665 conn(0x7fd1300fc5a0 msgr2=0x7fd130104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 shutdown_connections 2024-05-08T23:01:19.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-08T23:01:19.748+0000 7fd1358fb700 1 -- 172.21.15.86:0/1945011665 wait complete. 2024-05-08T23:01:19.926 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:19 smithi086 ceph-mon[78610]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:19 smithi053 ceph-mon[77587]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:19 smithi046 ceph-mon[36898]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:20.140 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T23:01:20.144 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T23:01:20.189 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T23:01:20.220 INFO:journalctl@ceph.mds.a.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T23:01:20.223 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T23:01:20.260 INFO:journalctl@ceph.mds.d.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T23:01:20.263 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T23:01:20.267 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T23:01:20.296 INFO:journalctl@ceph.mds.g.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T23:01:20.306 INFO:journalctl@ceph.mds.b.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T23:01:20.307 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T23:01:20.331 INFO:journalctl@ceph.mds.j.smithi046.stdout:-- Logs begin at Wed 2024-05-08 22:36:42 UTC. -- 2024-05-08T23:01:20.339 INFO:journalctl@ceph.mds.e.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T23:01:20.341 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T23:01:20.372 INFO:journalctl@ceph.mds.h.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T23:01:20.373 DEBUG:teuthology.orchestra.run.smithi086:mds.c> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T23:01:20.375 DEBUG:teuthology.orchestra.run.smithi086:mds.f> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T23:01:20.404 INFO:journalctl@ceph.mds.k.smithi053.stdout:-- Logs begin at Wed 2024-05-08 22:29:23 UTC. -- 2024-05-08T23:01:20.405 INFO:journalctl@ceph.mds.c.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:01:20.406 DEBUG:teuthology.orchestra.run.smithi086:mds.i> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T23:01:20.433 INFO:journalctl@ceph.mds.f.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:01:20.434 DEBUG:teuthology.orchestra.run.smithi086:mds.l> sudo journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T23:01:20.459 INFO:journalctl@ceph.mds.i.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:01:20.460 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T23:01:20.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:01:20.488 INFO:journalctl@ceph.mds.l.smithi086.stdout:-- Logs begin at Wed 2024-05-08 22:29:42 UTC. -- 2024-05-08T23:01:20.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 -- 172.21.15.46:0/3968617754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 msgr2=0x7fe430102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 --2- 172.21.15.46:0/3968617754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430102450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0098f0 tx=0x7fe41c009c00 comp rx=0 tx=0).stop 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 -- 172.21.15.46:0/3968617754 shutdown_connections 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 --2- 172.21.15.46:0/3968617754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430102990 0x7fe4301092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 --2- 172.21.15.46:0/3968617754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430100e80 0x7fe4301012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 --2- 172.21.15.46:0/3968617754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430102450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 -- 172.21.15.46:0/3968617754 >> 172.21.15.46:0/3968617754 conn(0x7fe4300fc410 msgr2=0x7fe4300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 -- 172.21.15.46:0/3968617754 shutdown_connections 2024-05-08T23:01:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.672+0000 7fe435000700 1 -- 172.21.15.46:0/3968617754 wait complete. 2024-05-08T23:01:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.673+0000 7fe435000700 1 Processor -- start 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 -- start start 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 0x7fe430074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430102990 0x7fe4300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430074070 con 0x7fe430101fd0 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300741e0 con 0x7fe430102990 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe435000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301a0ee0 con 0x7fe430100e80 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 0x7fe430074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36260/0 (socket says 172.21.15.46:36260) 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 0x7fe430074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49776/0 (socket says 172.21.15.46:49776) 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 -- 172.21.15.46:0/3318218875 learned_addr learned my addr 172.21.15.46:0/3318218875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42e800700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430102990 0x7fe4300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 -- 172.21.15.46:0/3318218875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 msgr2=0x7fe430074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 0x7fe430074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 -- 172.21.15.46:0/3318218875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430102990 msgr2=0x7fe4300738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430102990 0x7fe4300738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.674+0000 7fe42d400700 1 -- 172.21.15.46:0/3318218875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200095f0 con 0x7fe430101fd0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe42d400700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005e90 tx=0x7fe41c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c01c070 con 0x7fe430101fd0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe41c00f7f0 con 0x7fe430101fd0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41c009520 con 0x7fe430101fd0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c003a80 con 0x7fe430101fd0 2024-05-08T23:01:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.675+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4301a13a0 con 0x7fe430101fd0 2024-05-08T23:01:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.676+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe430108310 con 0x7fe430101fd0 2024-05-08T23:01:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.678+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe41c020030 con 0x7fe430101fd0 2024-05-08T23:01:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.680+0000 7fe426a00700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 0x7fe4140a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.680+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe41c0cd7d0 con 0x7fe430101fd0 2024-05-08T23:01:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.681+0000 7fe42de00700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 0x7fe4140a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.681+0000 7fe42de00700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 0x7fe4140a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe420005fd0 tx=0x7fe420009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.683+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe41c0973f0 con 0x7fe430101fd0 2024-05-08T23:01:20.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l;count:12 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:20.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:20 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:20.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.908+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe43010d070 con 0x7fe430101fd0 2024-05-08T23:01:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.909+0000 7fe426a00700 1 -- 172.21.15.46:0/3318218875 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fe41c09aa10 con 0x7fe430101fd0 2024-05-08T23:01:20.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:20.908 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:13.608146+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:20.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.911+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 msgr2=0x7fe4140a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.911+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 0x7fe4140a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe420005fd0 tx=0x7fe420009310 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.911+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 msgr2=0x7fe430073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.911+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005e90 tx=0x7fe41c00b730 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 shutdown_connections 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe430102990 0x7fe4300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe430100e80 0x7fe430074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe4140a11a0 0x7fe4140a3660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 --2- 172.21.15.46:0/3318218875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe430101fd0 0x7fe430073390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 >> 172.21.15.46:0/3318218875 conn(0x7fe4300fc410 msgr2=0x7fe430105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 shutdown_connections 2024-05-08T23:01:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:20.912+0000 7fe435000700 1 -- 172.21.15.46:0/3318218875 wait complete. 2024-05-08T23:01:20.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:01:20.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:01:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:01:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l;count:12 2024-05-08T23:01:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:20 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:21.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:01:21.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l;count:12 2024-05-08T23:01:21.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:21.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:21.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:21.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:21.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:21.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:21.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:21.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:20 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 -- 172.21.15.46:0/3537503419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 msgr2=0x7f7bb8073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 --2- 172.21.15.46:0/3537503419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8073430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009990 tx=0x7f7bac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 -- 172.21.15.46:0/3537503419 shutdown_connections 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 --2- 172.21.15.46:0/3537503419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8108d40 0x7f7bb810b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 --2- 172.21.15.46:0/3537503419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8073430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 --2- 172.21.15.46:0/3537503419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8073a00 0x7f7bb8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 -- 172.21.15.46:0/3537503419 >> 172.21.15.46:0/3537503419 conn(0x7f7bb80fc3b0 msgr2=0x7f7bb80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.134+0000 7f7bc0000700 1 -- 172.21.15.46:0/3537503419 shutdown_connections 2024-05-08T23:01:21.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.135+0000 7f7bc0000700 1 -- 172.21.15.46:0/3537503419 wait complete. 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.136+0000 7f7bc0000700 1 Processor -- start 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.136+0000 7f7bc0000700 1 -- start start 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.136+0000 7f7bc0000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bc0000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bc0000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8108d40 0x7f7bb81a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bc0000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81a0b70 con 0x7f7bb8073a00 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bc0000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81a0cb0 con 0x7f7bb8108d40 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bc0000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81a0e40 con 0x7f7bb8074fe0 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36282/0 (socket says 172.21.15.46:36282) 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 -- 172.21.15.46:0/1266674915 learned_addr learned my addr 172.21.15.46:0/1266674915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbcc00700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbe000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8108d40 0x7f7bb81a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 -- 172.21.15.46:0/1266674915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 msgr2=0x7f7bb8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 -- 172.21.15.46:0/1266674915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8108d40 msgr2=0x7f7bb81a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8108d40 0x7f7bb81a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbd600700 1 -- 172.21.15.46:0/1266674915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0095f0 con 0x7f7bb8073a00 2024-05-08T23:01:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.137+0000 7f7bbcc00700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bbd600700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac01d070 con 0x7f7bb8073a00 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bac005030 con 0x7f7bb8073a00 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb81a10c0 con 0x7f7bb8073a00 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac004b40 con 0x7f7bb8073a00 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.138+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb81a1540 con 0x7f7bb8073a00 2024-05-08T23:01:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.139+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb8107b00 con 0x7f7bb8073a00 2024-05-08T23:01:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.141+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7bac004d40 con 0x7f7bb8073a00 2024-05-08T23:01:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.142+0000 7f7bb6000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 0x7f7ba00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.142+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7bac0cd500 con 0x7f7bb8073a00 2024-05-08T23:01:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.142+0000 7f7bbcc00700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 0x7f7ba00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.143+0000 7f7bbcc00700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 0x7f7ba00a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8074d00 tx=0x7f7ba4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.146+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7bac097120 con 0x7f7bb8073a00 2024-05-08T23:01:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.363+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bb810eb70 con 0x7f7bb8073a00 2024-05-08T23:01:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.363+0000 7f7bb6000700 1 -- 172.21.15.46:0/1266674915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bac09a740 con 0x7f7bb8073a00 2024-05-08T23:01:21.362 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.365+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 msgr2=0x7f7ba00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.365+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 0x7f7ba00a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8074d00 tx=0x7f7ba4007400 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 msgr2=0x7f7bb8199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac004730 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 shutdown_connections 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bb8108d40 0x7f7bb81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7ba00a11a0 0x7f7ba00a3660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bb8073a00 0x7f7bb8199040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 --2- 172.21.15.46:0/1266674915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb8074fe0 0x7f7bb8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 >> 172.21.15.46:0/1266674915 conn(0x7f7bb80fc3b0 msgr2=0x7f7bb8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 shutdown_connections 2024-05-08T23:01:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.366+0000 7f7bc0000700 1 -- 172.21.15.46:0/1266674915 wait complete. 2024-05-08T23:01:21.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 -- 172.21.15.46:0/339902377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488100e80 msgr2=0x7f54881012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 --2- 172.21.15.46:0/339902377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488100e80 0x7f54881012a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f547c009990 tx=0x7f547c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 -- 172.21.15.46:0/339902377 shutdown_connections 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 --2- 172.21.15.46:0/339902377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488102990 0x7f54881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 --2- 172.21.15.46:0/339902377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488100e80 0x7f54881012a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 --2- 172.21.15.46:0/339902377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488101fd0 0x7f5488102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.581+0000 7f548fe00700 1 -- 172.21.15.46:0/339902377 >> 172.21.15.46:0/339902377 conn(0x7f54880fc410 msgr2=0x7f54880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.582+0000 7f548fe00700 1 -- 172.21.15.46:0/339902377 shutdown_connections 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.582+0000 7f548fe00700 1 -- 172.21.15.46:0/339902377 wait complete. 2024-05-08T23:01:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 Processor -- start 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 -- start start 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488100e80 0x7f5488074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 0x7f54880738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488074070 con 0x7f5488101fd0 2024-05-08T23:01:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880741e0 con 0x7f5488100e80 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881a0e60 con 0x7f5488102990 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 0x7f54880738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36312/0 (socket says 172.21.15.46:36312) 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548ca00700 1 -- 172.21.15.46:0/1094074510 learned_addr learned my addr 172.21.15.46:0/1094074510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.583+0000 7f548d400700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488100e80 0x7f5488074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 -- 172.21.15.46:0/1094074510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488100e80 msgr2=0x7f5488074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488100e80 0x7f5488074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 -- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 msgr2=0x7f5488073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 -- 172.21.15.46:0/1094074510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009c40 con 0x7f5488102990 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548ca00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548de00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 0x7f54880738d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f548000efb0 tx=0x7f548000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480003a00 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548000f040 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c0095f0 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480004540 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.584+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54881a1360 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.585+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54881082c0 con 0x7f5488102990 2024-05-08T23:01:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.587+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5480003bf0 con 0x7f5488102990 2024-05-08T23:01:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.589+0000 7f5486000700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 0x7f54700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.589+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54800ccfe0 con 0x7f5488102990 2024-05-08T23:01:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.589+0000 7f548d400700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 0x7f54700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.589+0000 7f548d400700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 0x7f54700a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b2e0 tx=0x7f547c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.591+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5480096b70 con 0x7f5488102990 2024-05-08T23:01:21.713 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:01:21 smithi086 systemd[1]: Starting Ceph mds.l for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.859+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5488068160 con 0x7f5488102990 2024-05-08T23:01:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.859+0000 7f5486000700 1 -- 172.21.15.46:0/1094074510 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f548009a190 con 0x7f5488102990 2024-05-08T23:01:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:21.858 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 msgr2=0x7f54700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 0x7f54700a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b2e0 tx=0x7f547c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 msgr2=0x7f54880738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 0x7f54880738d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f548000efb0 tx=0x7f548000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 shutdown_connections 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488102990 0x7f54880738d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5488100e80 0x7f5488074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f54700a1120 0x7f54700a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 --2- 172.21.15.46:0/1094074510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5488101fd0 0x7f5488073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 >> 172.21.15.46:0/1094074510 conn(0x7f54880fc410 msgr2=0x7f5488105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 shutdown_connections 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:21.862+0000 7f548fe00700 1 -- 172.21.15.46:0/1094074510 wait complete. 2024-05-08T23:01:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-05-08T23:01:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:21 smithi053 ceph-mon[77587]: Deploying daemon mds.l on smithi086 2024-05-08T23:01:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:21 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3318218875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:21 smithi053 ceph-mon[77587]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:22.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:21 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1266674915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:22.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:21 smithi046 ceph-mon[36898]: Deploying daemon mds.l on smithi086 2024-05-08T23:01:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:21 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3318218875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:21 smithi046 ceph-mon[36898]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:21 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1266674915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:21 smithi086 ceph-mon[78610]: Deploying daemon mds.l on smithi086 2024-05-08T23:01:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:21 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3318218875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:21 smithi086 ceph-mon[78610]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:21 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1266674915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:22.214 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:01:21 smithi086 ceph-mds[115094]: -- 172.21.15.86:0/3655653132 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624f907a1a0 con 0x5624f9051000 2024-05-08T23:01:22.214 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:01:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: starting mds.l at 2024-05-08T23:01:22.214 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:01:21 smithi086 bash[115056]: 559cf1242a9f9a4091da76e558a022b447d72b98704b8dfe9b76bbb3aa1312cb 2024-05-08T23:01:22.214 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:01:21 smithi086 systemd[1]: Started Ceph mds.l for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:23.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1094074510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6832/272583724,v1:172.21.15.86:6833/272583724] up:boot 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: fsmap 1 up:standby 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:23.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:22 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:23.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1094074510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:23.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6832/272583724,v1:172.21.15.86:6833/272583724] up:boot 2024-05-08T23:01:23.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: fsmap 1 up:standby 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:22 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1094074510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6832/272583724,v1:172.21.15.86:6833/272583724] up:boot 2024-05-08T23:01:23.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: fsmap 1 up:standby 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:23.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:22 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:23.367 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:01:23 smithi053 systemd[1]: Starting Ceph mds.k for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:23.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:23 smithi053 ceph-mon[77587]: Deploying daemon mds.k on smithi053 2024-05-08T23:01:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:23 smithi053 ceph-mon[77587]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:24.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:01:23 smithi053 ceph-mds[113306]: -- 172.21.15.53:0/631934254 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555aee5441a0 con 0x555aee51ac00 2024-05-08T23:01:24.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:01:23 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: starting mds.k at 2024-05-08T23:01:24.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:01:23 smithi053 bash[113267]: 0c7cc412a6df9fe6c7650592126d012a354994674d8ed4eb9215e1585ebd1913 2024-05-08T23:01:24.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:01:23 smithi053 systemd[1]: Started Ceph mds.k for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:24.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:23 smithi046 ceph-mon[36898]: Deploying daemon mds.k on smithi053 2024-05-08T23:01:24.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:23 smithi046 ceph-mon[36898]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 -- 172.21.15.46:0/425638744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c073a00 msgr2=0x7f172c073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 --2- 172.21.15.46:0/425638744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c073a00 0x7f172c073e80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1718009990 tx=0x7f1718009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 -- 172.21.15.46:0/425638744 shutdown_connections 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 --2- 172.21.15.46:0/425638744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c108d40 0x7f172c10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 --2- 172.21.15.46:0/425638744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c074fe0 0x7f172c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 --2- 172.21.15.46:0/425638744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c073a00 0x7f172c073e80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 -- 172.21.15.46:0/425638744 >> 172.21.15.46:0/425638744 conn(0x7f172c0fc410 msgr2=0x7f172c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 -- 172.21.15.46:0/425638744 shutdown_connections 2024-05-08T23:01:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.103+0000 7f1731e00700 1 -- 172.21.15.46:0/425638744 wait complete. 2024-05-08T23:01:24.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.104+0000 7f1731e00700 1 Processor -- start 2024-05-08T23:01:24.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.104+0000 7f1731e00700 1 -- start start 2024-05-08T23:01:24.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c073a00 0x7f172c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 0x7f172c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a0b80 con 0x7f172c074fe0 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a0cc0 con 0x7f172c108d40 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f1731e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a0e50 con 0x7f172c073a00 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41236/0 (socket says 172.21.15.46:41236) 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 -- 172.21.15.46:0/3790689471 learned_addr learned my addr 172.21.15.46:0/3790689471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172b400700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 0x7f172c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172aa00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c073a00 0x7f172c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 -- 172.21.15.46:0/3790689471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c073a00 msgr2=0x7f172c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c073a00 0x7f172c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 -- 172.21.15.46:0/3790689471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 msgr2=0x7f172c1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 0x7f172c1a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172a000700 1 -- 172.21.15.46:0/3790689471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180095f0 con 0x7f172c074fe0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.105+0000 7f172b400700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 0x7f172c1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f172a000700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1718000c00 tx=0x7f1718009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171801d070 con 0x7f172c074fe0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1718021410 con 0x7f172c074fe0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718017650 con 0x7f172c074fe0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c1a10d0 con 0x7f172c074fe0 2024-05-08T23:01:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.106+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c1a14a0 con 0x7f172c074fe0 2024-05-08T23:01:24.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.107+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c107b00 con 0x7f172c074fe0 2024-05-08T23:01:24.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.109+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1718021580 con 0x7f172c074fe0 2024-05-08T23:01:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.110+0000 7f1730a00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 0x7f17100a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.111+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17180cd6e0 con 0x7f172c074fe0 2024-05-08T23:01:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.111+0000 7f172aa00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 0x7f17100a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.111+0000 7f172aa00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 0x7f17100a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f171c009c60 tx=0x7f171c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.114+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1718097300 con 0x7f172c074fe0 2024-05-08T23:01:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:23 smithi086 ceph-mon[78610]: Deploying daemon mds.k on smithi053 2024-05-08T23:01:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:23 smithi086 ceph-mon[78610]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.391+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172c067d10 con 0x7f172c074fe0 2024-05-08T23:01:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.392+0000 7f1730a00700 1 -- 172.21.15.46:0/3790689471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1254 (secure 0 0 0) 0x7f1718005740 con 0x7f172c074fe0 2024-05-08T23:01:24.391 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:24.391 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:01:24.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 msgr2=0x7f17100a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:24.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 0x7f17100a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f171c009c60 tx=0x7f171c009260 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 msgr2=0x7f172c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1718000c00 tx=0x7f1718009580 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 shutdown_connections 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172c108d40 0x7f172c1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c073a00 0x7f172c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17100a11a0 0x7f17100a3660 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 --2- 172.21.15.46:0/3790689471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172c074fe0 0x7f172c1995e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.394+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 >> 172.21.15.46:0/3790689471 conn(0x7f172c0fc410 msgr2=0x7f172c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.395+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 shutdown_connections 2024-05-08T23:01:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:24.395+0000 7f1731e00700 1 -- 172.21.15.46:0/3790689471 wait complete. 2024-05-08T23:01:24.395 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-05-08T23:01:25.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: Deploying daemon mds.j on smithi046 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3790689471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6832/3863967478,v1:172.21.15.53:6833/3863967478] up:boot 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: fsmap 2 up:standby 2024-05-08T23:01:25.106 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:24 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:01:25.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: Deploying daemon mds.j on smithi046 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3790689471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6832/3863967478,v1:172.21.15.53:6833/3863967478] up:boot 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: fsmap 2 up:standby 2024-05-08T23:01:25.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:24 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: Deploying daemon mds.j on smithi046 2024-05-08T23:01:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3790689471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6832/3863967478,v1:172.21.15.53:6833/3863967478] up:boot 2024-05-08T23:01:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: fsmap 2 up:standby 2024-05-08T23:01:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:24 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:01:25.777 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:01:25 smithi046 systemd[1]: Starting Ceph mds.j for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:26.136 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:25 smithi046 ceph-mon[36898]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:26.143 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:01:26 smithi046 bash[87483]: 47cfb5ecfb8bbfd13c9c7bad5cfeff8c632f941df935dc0bbd2c75bee74058be 2024-05-08T23:01:26.143 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:01:26 smithi046 systemd[1]: Started Ceph mds.j for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:25 smithi086 ceph-mon[78610]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:25 smithi053 ceph-mon[77587]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:26.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:26.478 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:01:26 smithi046 ceph-mds[87521]: -- 172.21.15.46:0/449689639 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f328a41a0 con 0x557f3287c800 2024-05-08T23:01:26.478 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:01:26 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-j[87498]: starting mds.j at 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 -- 172.21.15.46:0/954143226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448106b60 msgr2=0x7f2448108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 --2- 172.21.15.46:0/954143226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448106b60 0x7f2448108fd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f244000b0a0 tx=0x7f244000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 -- 172.21.15.46:0/954143226 shutdown_connections 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 --2- 172.21.15.46:0/954143226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448106b60 0x7f2448108fd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 --2- 172.21.15.46:0/954143226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448101900 0x7f2448103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 --2- 172.21.15.46:0/954143226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 -- 172.21.15.46:0/954143226 >> 172.21.15.46:0/954143226 conn(0x7f24480fb2f0 msgr2=0x7f24480fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 -- 172.21.15.46:0/954143226 shutdown_connections 2024-05-08T23:01:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.629+0000 7f244ec00700 1 -- 172.21.15.46:0/954143226 wait complete. 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.630+0000 7f244ec00700 1 Processor -- start 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.630+0000 7f244ec00700 1 -- start start 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.630+0000 7f244ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.630+0000 7f244ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.630+0000 7f244ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 0x7f244819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f244ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244819ea60 con 0x7f2448101900 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f244ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244819eba0 con 0x7f2448104230 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f244ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244819ed30 con 0x7f2448106b60 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41274/0 (socket says 172.21.15.46:41274) 2024-05-08T23:01:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 -- 172.21.15.46:0/1172570220 learned_addr learned my addr 172.21.15.46:0/1172570220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447400700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f244cc00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 0x7f244819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 -- 172.21.15.46:0/1172570220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 msgr2=0x7f244819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 0x7f244819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 -- 172.21.15.46:0/1172570220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 msgr2=0x7f2448197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 -- 172.21.15.46:0/1172570220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440009d00 con 0x7f2448101900 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f244cc00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 0x7f244819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447400700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.631+0000 7f2447e00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f243800e9c0 tx=0x7f243800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.632+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380097b0 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.632+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2438004cf0 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.632+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244819efb0 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.632+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438005600 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.632+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244819f430 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.633+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24480fce70 con 0x7f2448101900 2024-05-08T23:01:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.635+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f243800cc60 con 0x7f2448101900 2024-05-08T23:01:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.636+0000 7f2444c00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 0x7f242c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.636+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2438014070 con 0x7f2448101900 2024-05-08T23:01:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.637+0000 7f2447400700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 0x7f242c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.637+0000 7f2447400700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 0x7f242c0a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2434011490 tx=0x7f2434009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:26.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.641+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f24380962a0 con 0x7f2448101900 2024-05-08T23:01:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.933+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f244810c720 con 0x7f2448101900 2024-05-08T23:01:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.934+0000 7f2444c00700 1 -- 172.21.15.46:0/1172570220 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2768 (secure 0 0 0) 0x7f24380998c0 con 0x7f2448101900 2024-05-08T23:01:26.933 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:26.933 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:01:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 msgr2=0x7f242c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 0x7f242c0a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2434011490 tx=0x7f2434009260 comp rx=0 tx=0).stop 2024-05-08T23:01:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 msgr2=0x7f2448196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f243800e9c0 tx=0x7f243800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 shutdown_connections 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2448106b60 0x7f244819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f242c0a11a0 0x7f242c0a3660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2448101900 0x7f2448196f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 --2- 172.21.15.46:0/1172570220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2448104230 0x7f2448197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 >> 172.21.15.46:0/1172570220 conn(0x7f24480fb2f0 msgr2=0x7f2448102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 shutdown_connections 2024-05-08T23:01:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:26.937+0000 7f244ec00700 1 -- 172.21.15.46:0/1172570220 wait complete. 2024-05-08T23:01:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-05-08T23:01:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: Deploying daemon mds.d on smithi046 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6834/3787979221,v1:172.21.15.46:6835/3787979221] up:boot 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: fsmap 3 up:standby 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:27.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:27 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1172570220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:27.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: Deploying daemon mds.d on smithi046 2024-05-08T23:01:27.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6834/3787979221,v1:172.21.15.46:6835/3787979221] up:boot 2024-05-08T23:01:27.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: fsmap 3 up:standby 2024-05-08T23:01:27.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:01:27.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:27.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:27 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1172570220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: Deploying daemon mds.d on smithi046 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6834/3787979221,v1:172.21.15.46:6835/3787979221] up:boot 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: fsmap 3 up:standby 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:01:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:27.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:27 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1172570220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:28.095 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:01:28 smithi046 systemd[1]: Starting Ceph mds.d for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:28.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:29.019 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:01:28 smithi046 bash[87873]: c34a970844a8281a55ac57dc9b851a1ba055a7e880c730d101b7354c2ce1846e 2024-05-08T23:01:29.019 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:01:28 smithi046 systemd[1]: Started Ceph mds.d for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:29.019 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:01:28 smithi046 ceph-mds[87912]: -- 172.21.15.46:0/2629308877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620a77101a0 con 0x5620a76e9000 2024-05-08T23:01:29.019 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:01:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: starting mds.d at 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.163+0000 7fe716200700 1 -- 172.21.15.46:0/2467650955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710101ea0 msgr2=0x7fe710102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.163+0000 7fe716200700 1 --2- 172.21.15.46:0/2467650955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710101ea0 0x7fe710102300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0099c0 tx=0x7fe6fc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 -- 172.21.15.46:0/2467650955 shutdown_connections 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 --2- 172.21.15.46:0/2467650955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710102840 0x7fe7101090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 --2- 172.21.15.46:0/2467650955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710100cb0 0x7fe7101010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 --2- 172.21.15.46:0/2467650955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710101ea0 0x7fe710102300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 -- 172.21.15.46:0/2467650955 >> 172.21.15.46:0/2467650955 conn(0x7fe7100fc220 msgr2=0x7fe7100fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:29.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 -- 172.21.15.46:0/2467650955 shutdown_connections 2024-05-08T23:01:29.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.164+0000 7fe716200700 1 -- 172.21.15.46:0/2467650955 wait complete. 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 Processor -- start 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 -- start start 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 0x7fe710073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710102840 0x7fe7100738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710073fd0 con 0x7fe710102840 2024-05-08T23:01:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710074140 con 0x7fe710100cb0 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe716200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101a0cf0 con 0x7fe710101ea0 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe714e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe714e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35304/0 (socket says 172.21.15.46:35304) 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe714e00700 1 -- 172.21.15.46:0/2972169998 learned_addr learned my addr 172.21.15.46:0/2972169998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.165+0000 7fe715800700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710102840 0x7fe7100738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe70fe00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 0x7fe710073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 -- 172.21.15.46:0/2972169998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 msgr2=0x7fe710073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 0x7fe710073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 -- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710102840 msgr2=0x7fe7100738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710102840 0x7fe7100738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 -- 172.21.15.46:0/2972169998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0095f0 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe70fe00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 0x7fe710073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe714e00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe70400e950 tx=0x7fe70400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704009770 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7101a0f50 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70400eea0 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.166+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704003cf0 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.167+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7101a1320 con 0x7fe710100cb0 2024-05-08T23:01:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.168+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71004ed40 con 0x7fe710100cb0 2024-05-08T23:01:29.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.169+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe70400ccd0 con 0x7fe710100cb0 2024-05-08T23:01:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.171+0000 7fe70d600700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 0x7fe6f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.171+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe704014070 con 0x7fe710100cb0 2024-05-08T23:01:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.171+0000 7fe70fe00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 0x7fe6f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.171+0000 7fe70fe00700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 0x7fe6f80a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00b580 tx=0x7fe6fc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.175+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe71004ed40 con 0x7fe710100cb0 2024-05-08T23:01:29.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.438+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe71004ecd0 con 0x7fe710100cb0 2024-05-08T23:01:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.439+0000 7fe70d600700 1 -- 172.21.15.46:0/2972169998 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7fe704096300 con 0x7fe710100cb0 2024-05-08T23:01:29.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 msgr2=0x7fe6f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 0x7fe6f80a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00b580 tx=0x7fe6fc01a040 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 msgr2=0x7fe710074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe70400e950 tx=0x7fe70400ec60 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 shutdown_connections 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe6f80a1120 0x7fe6f80a35e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe710102840 0x7fe7100738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe710100cb0 0x7fe710074cb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 --2- 172.21.15.46:0/2972169998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe710101ea0 0x7fe710073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.441+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 >> 172.21.15.46:0/2972169998 conn(0x7fe7100fc220 msgr2=0x7fe7101051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.442+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 shutdown_connections 2024-05-08T23:01:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:29.442+0000 7fe716200700 1 -- 172.21.15.46:0/2972169998 wait complete. 2024-05-08T23:01:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-05-08T23:01:29.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: Deploying daemon mds.e on smithi053 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6836/1812878749,v1:172.21.15.46:6837/1812878749] up:boot 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: fsmap 4 up:standby 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:29 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2972169998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: Deploying daemon mds.e on smithi053 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6836/1812878749,v1:172.21.15.46:6837/1812878749] up:boot 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: fsmap 4 up:standby 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:29.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:29 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2972169998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: Deploying daemon mds.e on smithi053 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6836/1812878749,v1:172.21.15.46:6837/1812878749] up:boot 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: fsmap 4 up:standby 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:29.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:29 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2972169998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:30.088 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:01:30 smithi053 systemd[1]: Starting Ceph mds.e for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:31.011 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:01:30 smithi053 ceph-mds[113650]: -- 172.21.15.53:0/1359787612 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5589a447a1a0 con 0x5589a4450800 2024-05-08T23:01:31.011 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:01:30 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: starting mds.e at 2024-05-08T23:01:31.011 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:01:30 smithi053 bash[113612]: 4780d7bb5d5cab5602cb144e2f8b1766600fb71cbc9439cac31d1c9c30332876 2024-05-08T23:01:31.011 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:01:30 smithi053 systemd[1]: Started Ceph mds.e for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:31.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 -- 172.21.15.46:0/3018503543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c101fd0 msgr2=0x7f5e6c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 --2- 172.21.15.46:0/3018503543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c102450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009990 tx=0x7f5e58009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 -- 172.21.15.46:0/3018503543 shutdown_connections 2024-05-08T23:01:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 --2- 172.21.15.46:0/3018503543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c102990 0x7f5e6c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 --2- 172.21.15.46:0/3018503543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c100e80 0x7f5e6c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 --2- 172.21.15.46:0/3018503543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c102450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 -- 172.21.15.46:0/3018503543 >> 172.21.15.46:0/3018503543 conn(0x7f5e6c0fc410 msgr2=0x7f5e6c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:31.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 -- 172.21.15.46:0/3018503543 shutdown_connections 2024-05-08T23:01:31.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.672+0000 7f5e71200700 1 -- 172.21.15.46:0/3018503543 wait complete. 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 Processor -- start 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 -- start start 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c102990 0x7f5e6c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c074070 con 0x7f5e6c102990 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c0741e0 con 0x7f5e6c101fd0 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.673+0000 7f5e71200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1a0e30 con 0x7f5e6c100e80 2024-05-08T23:01:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e6a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c102990 0x7f5e6c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48926/0 (socket says 172.21.15.46:48926) 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 -- 172.21.15.46:0/3911178697 learned_addr learned my addr 172.21.15.46:0/3911178697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69400700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 -- 172.21.15.46:0/3911178697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c101fd0 msgr2=0x7f5e6c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 -- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c102990 msgr2=0x7f5e6c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c102990 0x7f5e6c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 -- 172.21.15.46:0/3911178697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580095f0 con 0x7f5e6c100e80 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.674+0000 7f5e69e00700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9a0 tx=0x7f5e5c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.675+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c009790 con 0x7f5e6c100e80 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.675+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5c010430 con 0x7f5e6c100e80 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.675+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c018680 con 0x7f5e6c100e80 2024-05-08T23:01:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.675+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c1a1060 con 0x7f5e6c100e80 2024-05-08T23:01:31.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.675+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c1a1510 con 0x7f5e6c100e80 2024-05-08T23:01:31.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.676+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c1082a0 con 0x7f5e6c100e80 2024-05-08T23:01:31.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.678+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5e5c0105a0 con 0x7f5e6c100e80 2024-05-08T23:01:31.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.679+0000 7f5e62a00700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 0x7f5e500a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:31.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.679+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e5c014070 con 0x7f5e6c100e80 2024-05-08T23:01:31.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.679+0000 7f5e69400700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 0x7f5e500a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:31.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.680+0000 7f5e69400700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 0x7f5e500a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b2e0 tx=0x7f5e58005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.681+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5e5c096420 con 0x7f5e6c100e80 2024-05-08T23:01:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: Deploying daemon mds.f on smithi086 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6834/1845130080,v1:172.21.15.53:6835/1845130080] up:boot 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: fsmap 5 up:standby 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:01:31.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:31 smithi086 ceph-mon[78610]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:31.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: Deploying daemon mds.f on smithi086 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6834/1845130080,v1:172.21.15.53:6835/1845130080] up:boot 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: fsmap 5 up:standby 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:01:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:31 smithi046 ceph-mon[36898]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.951+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e6c067d10 con 0x7f5e6c100e80 2024-05-08T23:01:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.952+0000 7f5e62a00700 1 -- 172.21.15.46:0/3911178697 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f5e5c099a40 con 0x7f5e6c100e80 2024-05-08T23:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-08T23:01:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 msgr2=0x7f5e500a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 0x7f5e500a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b2e0 tx=0x7f5e58005fd0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 msgr2=0x7f5e6c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9a0 tx=0x7f5e5c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 shutdown_connections 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5e500a11a0 0x7f5e500a3660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c102990 0x7f5e6c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e6c100e80 0x7f5e6c074d00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 --2- 172.21.15.46:0/3911178697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6c101fd0 0x7f5e6c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 >> 172.21.15.46:0/3911178697 conn(0x7f5e6c0fc410 msgr2=0x7f5e6c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 shutdown_connections 2024-05-08T23:01:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:31.955+0000 7f5e71200700 1 -- 172.21.15.46:0/3911178697 wait complete. 2024-05-08T23:01:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-05-08T23:01:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: Deploying daemon mds.f on smithi086 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6834/1845130080,v1:172.21.15.53:6835/1845130080] up:boot 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: fsmap 5 up:standby 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:01:32.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:31 smithi053 ceph-mon[77587]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:32.089 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:01:31 smithi086 systemd[1]: Starting Ceph mds.f for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:32.463 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mds[115435]: -- 172.21.15.86:0/676956608 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8b332a1a0 con 0x55a8b3300800 2024-05-08T23:01:32.463 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:01:32 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: starting mds.f at 2024-05-08T23:01:32.463 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:01:32 smithi086 bash[115397]: 537202b25f782454ca2ddd097b3bae64a5320148dc7a5664970a234fb709f1a2 2024-05-08T23:01:32.463 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:01:32 smithi086 systemd[1]: Started Ceph mds.f for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3911178697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:33.034 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:33.035 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:32 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3911178697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:32 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3911178697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:32 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:33.713 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:01:33 smithi086 systemd[1]: Starting Ceph mds.c for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:33.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.189+0000 7f81be800700 1 -- 172.21.15.46:0/2235745764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8100e80 msgr2=0x7f81b81012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.189+0000 7f81be800700 1 --2- 172.21.15.46:0/2235745764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8100e80 0x7f81b81012a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81a80098f0 tx=0x7f81a8009c00 comp rx=0 tx=0).stop 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 -- 172.21.15.46:0/2235745764 shutdown_connections 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 --2- 172.21.15.46:0/2235745764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8102990 0x7f81b81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 --2- 172.21.15.46:0/2235745764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8100e80 0x7f81b81012a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 --2- 172.21.15.46:0/2235745764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8101fd0 0x7f81b8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 -- 172.21.15.46:0/2235745764 >> 172.21.15.46:0/2235745764 conn(0x7f81b80fc410 msgr2=0x7f81b80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 -- 172.21.15.46:0/2235745764 shutdown_connections 2024-05-08T23:01:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.190+0000 7f81be800700 1 -- 172.21.15.46:0/2235745764 wait complete. 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 Processor -- start 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 -- start start 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 0x7f81b80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8074070 con 0x7f81b8100e80 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80741e0 con 0x7f81b8101fd0 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81be800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b81a0e20 con 0x7f81b8102990 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55690/0 (socket says 172.21.15.46:55690) 2024-05-08T23:01:34.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 0x7f81b80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59766/0 (socket says 172.21.15.46:59766) 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.191+0000 7f81b6a00700 1 -- 172.21.15.46:0/1682549872 learned_addr learned my addr 172.21.15.46:0/1682549872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 -- 172.21.15.46:0/1682549872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 msgr2=0x7f81b80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 0x7f81b80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 -- 172.21.15.46:0/1682549872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 msgr2=0x7f81b8073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 -- 172.21.15.46:0/1682549872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009520 con 0x7f81b8100e80 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7e00700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 0x7f81b80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b6a00700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81b7400700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009230 tx=0x7f81a800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a801c070 con 0x7f81b8100e80 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b81a1050 con 0x7f81b8100e80 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a800f7d0 con 0x7f81b8100e80 2024-05-08T23:01:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80038e0 con 0x7f81b8100e80 2024-05-08T23:01:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.192+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b81a1470 con 0x7f81b8100e80 2024-05-08T23:01:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.194+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b8108310 con 0x7f81b8100e80 2024-05-08T23:01:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.195+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f81a8020030 con 0x7f81b8100e80 2024-05-08T23:01:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.197+0000 7f81afe00700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 0x7f819c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.197+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81a80cd2f0 con 0x7f81b8100e80 2024-05-08T23:01:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.197+0000 7f81b6a00700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 0x7f819c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.197+0000 7f81b6a00700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 0x7f819c0a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005950 tx=0x7f81a4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.201+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f81a8096f10 con 0x7f81b8100e80 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mon[78610]: Deploying daemon mds.c on smithi086 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:boot 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mon[78610]: fsmap 6 up:standby 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mon[78610]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:34.213 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-mds[115774]: -- 172.21.15.86:0/1487379478 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1ead981a0 con 0x55f1ead6f000 2024-05-08T23:01:34.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:01:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: starting mds.c at 2024-05-08T23:01:34.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:01:34 smithi086 bash[115736]: cae6151f52c9c8bd49e2378f79d0e6b7642591c791f7082bce9d6de94835304b 2024-05-08T23:01:34.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:01:34 smithi086 systemd[1]: Started Ceph mds.c for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:33 smithi053 ceph-mon[77587]: Deploying daemon mds.c on smithi086 2024-05-08T23:01:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:33 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:boot 2024-05-08T23:01:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:33 smithi053 ceph-mon[77587]: fsmap 6 up:standby 2024-05-08T23:01:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:33 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:01:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:33 smithi053 ceph-mon[77587]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:33 smithi046 ceph-mon[36898]: Deploying daemon mds.c on smithi086 2024-05-08T23:01:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:33 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:boot 2024-05-08T23:01:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:33 smithi046 ceph-mon[36898]: fsmap 6 up:standby 2024-05-08T23:01:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:33 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:01:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:33 smithi046 ceph-mon[36898]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.467+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81b8067d10 con 0x7f81b8100e80 2024-05-08T23:01:34.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.467+0000 7f81afe00700 1 -- 172.21.15.46:0/1682549872 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5039 (secure 0 0 0) 0x7f81a809a530 con 0x7f81b8100e80 2024-05-08T23:01:34.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:34.466 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 msgr2=0x7f819c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 0x7f819c0a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005950 tx=0x7f81a4009310 comp rx=0 tx=0).stop 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 msgr2=0x7f81b8074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009230 tx=0x7f81a800b790 comp rx=0 tx=0).stop 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 shutdown_connections 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b8102990 0x7f81b80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f819c0a1120 0x7f819c0a35e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b8100e80 0x7f81b8074d00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 --2- 172.21.15.46:0/1682549872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b8101fd0 0x7f81b8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 >> 172.21.15.46:0/1682549872 conn(0x7f81b80fc410 msgr2=0x7f81b8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 shutdown_connections 2024-05-08T23:01:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:34.470+0000 7f81be800700 1 -- 172.21.15.46:0/1682549872 wait complete. 2024-05-08T23:01:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: Deploying daemon mds.h on smithi053 2024-05-08T23:01:35.189 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1682549872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:35.190 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:boot 2024-05-08T23:01:35.190 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: fsmap 7 up:standby 2024-05-08T23:01:35.190 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:01:35.190 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mon[77587]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:35.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: Deploying daemon mds.h on smithi053 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1682549872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:boot 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: fsmap 7 up:standby 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:01:35.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:35 smithi086 ceph-mon[78610]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: Deploying daemon mds.h on smithi053 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1682549872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:boot 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: fsmap 7 up:standby 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:01:35.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:35 smithi046 ceph-mon[36898]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:35.845 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:01:35 smithi053 systemd[1]: Starting Ceph mds.h for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:36.261 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:01:35 smithi053 bash[113954]: c3ba9674224764ac6a48e017a79fd1e304e8b6b153248a5e03cebcd51cb8a758 2024-05-08T23:01:36.261 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:01:35 smithi053 systemd[1]: Started Ceph mds.h for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:36.262 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:01:35 smithi053 ceph-mds[113992]: -- 172.21.15.53:0/3899642052 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b61448c1a0 con 0x55b614463000 2024-05-08T23:01:36.262 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:01:35 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: starting mds.h at 2024-05-08T23:01:36.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 -- 172.21.15.46:0/2326839206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388101fd0 msgr2=0x7f9388102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 --2- 172.21.15.46:0/2326839206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388101fd0 0x7f9388102450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9374009990 tx=0x7f9374009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 -- 172.21.15.46:0/2326839206 shutdown_connections 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 --2- 172.21.15.46:0/2326839206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388102990 0x7f93881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 --2- 172.21.15.46:0/2326839206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388100e80 0x7f93881012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 --2- 172.21.15.46:0/2326839206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388101fd0 0x7f9388102450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 -- 172.21.15.46:0/2326839206 >> 172.21.15.46:0/2326839206 conn(0x7f93880fc410 msgr2=0x7f93880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 -- 172.21.15.46:0/2326839206 shutdown_connections 2024-05-08T23:01:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.712+0000 7f9390000700 1 -- 172.21.15.46:0/2326839206 wait complete. 2024-05-08T23:01:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.713+0000 7f9390000700 1 Processor -- start 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 -- start start 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 0x7f9388196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388101fd0 0x7f9388197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819e8f0 con 0x7f9388102990 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819ea60 con 0x7f9388101fd0 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f9390000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819ec20 con 0x7f9388100e80 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f938e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f938e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59778/0 (socket says 172.21.15.46:59778) 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f938e000700 1 -- 172.21.15.46:0/3557152131 learned_addr learned my addr 172.21.15.46:0/3557152131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f938cc00700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388101fd0 0x7f9388197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.714+0000 7f938d600700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 0x7f9388196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 -- 172.21.15.46:0/3557152131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388101fd0 msgr2=0x7f9388197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388101fd0 0x7f9388197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 -- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 msgr2=0x7f938819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 -- 172.21.15.46:0/3557152131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740095f0 con 0x7f9388100e80 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938e000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f938d600700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 0x7f9388196e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f937c00e9f0 tx=0x7f937c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00c9c0 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938819ef00 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f937c010430 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c018680 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.715+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938819f330 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.716+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93881082a0 con 0x7f9388100e80 2024-05-08T23:01:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.718+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f937c010860 con 0x7f9388100e80 2024-05-08T23:01:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.719+0000 7f9386000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 0x7f93700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.719+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f937c014070 con 0x7f9388100e80 2024-05-08T23:01:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.720+0000 7f938cc00700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 0x7f93700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.720+0000 7f938cc00700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 0x7f93700a35e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f937400b2e0 tx=0x7f9374005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.722+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f937c0960f0 con 0x7f9388100e80 2024-05-08T23:01:36.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.991+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9388067d10 con 0x7f9388100e80 2024-05-08T23:01:36.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.991+0000 7f9386000700 1 -- 172.21.15.46:0/3557152131 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6550 (secure 0 0 0) 0x7f937c099710 con 0x7f9388100e80 2024-05-08T23:01:36.990 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:36.991 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T23:01:36.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 msgr2=0x7f93700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:36.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 0x7f93700a35e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f937400b2e0 tx=0x7f9374005fb0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 msgr2=0x7f9388196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 0x7f9388196e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f937c00e9f0 tx=0x7f937c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 shutdown_connections 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f93700a1120 0x7f93700a35e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9388102990 0x7f938819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9388100e80 0x7f9388196e00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 --2- 172.21.15.46:0/3557152131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388101fd0 0x7f9388197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 >> 172.21.15.46:0/3557152131 conn(0x7f93880fc410 msgr2=0x7f93881053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 shutdown_connections 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:36.994+0000 7f9390000700 1 -- 172.21.15.46:0/3557152131 wait complete. 2024-05-08T23:01:36.994 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: Deploying daemon mds.i on smithi086 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6836/170717769,v1:172.21.15.53:6837/170717769] up:boot 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: fsmap 8 up:standby 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:37.068 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3557152131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:37.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: Deploying daemon mds.i on smithi086 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6836/170717769,v1:172.21.15.53:6837/170717769] up:boot 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: fsmap 8 up:standby 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:37 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3557152131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: Deploying daemon mds.i on smithi086 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6836/170717769,v1:172.21.15.53:6837/170717769] up:boot 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: fsmap 8 up:standby 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:37.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:37 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3557152131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:37.609 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:01:37 smithi086 systemd[1]: Starting Ceph mds.i for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:37.963 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:01:37 smithi086 bash[116076]: 690363dbe5dad64b817b3c34a9922d135b1ba26f7e71cfd17f24d7b095733b65 2024-05-08T23:01:37.963 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:01:37 smithi086 systemd[1]: Started Ceph mds.i for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:37.963 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:01:37 smithi086 ceph-mds[116114]: -- 172.21.15.86:0/3463182178 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8c7c361a0 con 0x55f8c7c0cc00 2024-05-08T23:01:37.964 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:01:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: starting mds.i at 2024-05-08T23:01:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: Deploying daemon mds.g on smithi046 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:boot 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: fsmap 9 up:standby 2024-05-08T23:01:39.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:38 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:01:39.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:39.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: Deploying daemon mds.g on smithi046 2024-05-08T23:01:39.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:boot 2024-05-08T23:01:39.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: fsmap 9 up:standby 2024-05-08T23:01:39.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:38 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:39.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: Deploying daemon mds.g on smithi046 2024-05-08T23:01:39.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:boot 2024-05-08T23:01:39.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: fsmap 9 up:standby 2024-05-08T23:01:39.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:38 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 -- 172.21.15.46:0/849916809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8108d40 msgr2=0x7f4af810b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 --2- 172.21.15.46:0/849916809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8108d40 0x7f4af810b1d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4af000b0a0 tx=0x7f4af000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 -- 172.21.15.46:0/849916809 shutdown_connections 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 --2- 172.21.15.46:0/849916809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8108d40 0x7f4af810b1d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 --2- 172.21.15.46:0/849916809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8074fe0 0x7f4af8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 --2- 172.21.15.46:0/849916809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 0x7f4af8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 -- 172.21.15.46:0/849916809 >> 172.21.15.46:0/849916809 conn(0x7f4af80fc410 msgr2=0x7f4af80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 -- 172.21.15.46:0/849916809 shutdown_connections 2024-05-08T23:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.236+0000 7f4afe200700 1 -- 172.21.15.46:0/849916809 wait complete. 2024-05-08T23:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.238+0000 7f4afe200700 1 Processor -- start 2024-05-08T23:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.238+0000 7f4afe200700 1 -- start start 2024-05-08T23:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 0x7f4af8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a0c70 con 0x7f4af8108d40 2024-05-08T23:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a0db0 con 0x7f4af8074fe0 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4afe200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a0f40 con 0x7f4af8073a00 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 0x7f4af8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59804/0 (socket says 172.21.15.46:59804) 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55728/0 (socket says 172.21.15.46:55728) 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.239+0000 7f4af7e00700 1 -- 172.21.15.46:0/1653106256 learned_addr learned my addr 172.21.15.46:0/1653106256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 -- 172.21.15.46:0/1653106256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 msgr2=0x7f4af8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 0x7f4af8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 -- 172.21.15.46:0/1653106256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 msgr2=0x7f4af8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 -- 172.21.15.46:0/1653106256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009520 con 0x7f4af8108d40 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af6a00700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4af7e00700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4af0007a60 tx=0x7f4af0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000e040 con 0x7f4af8108d40 2024-05-08T23:01:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af0009d00 con 0x7f4af8108d40 2024-05-08T23:01:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af0004c50 con 0x7f4af8108d40 2024-05-08T23:01:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.240+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af001b6c0 con 0x7f4af8108d40 2024-05-08T23:01:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.241+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af81a1400 con 0x7f4af8108d40 2024-05-08T23:01:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.242+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af8107b00 con 0x7f4af8108d40 2024-05-08T23:01:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.244+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4af001d470 con 0x7f4af8108d40 2024-05-08T23:01:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.245+0000 7f4aefe00700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 0x7f4adc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.245+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4af00cd790 con 0x7f4af8108d40 2024-05-08T23:01:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.245+0000 7f4af7400700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 0x7f4adc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.246+0000 7f4af7400700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 0x7f4adc0a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009df0 tx=0x7f4ae8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.254+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4af00973b0 con 0x7f4af8108d40 2024-05-08T23:01:39.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.528+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4af8067480 con 0x7f4af8108d40 2024-05-08T23:01:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.529+0000 7f4aefe00700 1 -- 172.21.15.46:0/1653106256 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7309 (secure 0 0 0) 0x7f4af009a9d0 con 0x7f4af8108d40 2024-05-08T23:01:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:39.529 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 msgr2=0x7f4adc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 0x7f4adc0a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009df0 tx=0x7f4ae8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 msgr2=0x7f4af81a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4af0007a60 tx=0x7f4af0007930 comp rx=0 tx=0).stop 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 shutdown_connections 2024-05-08T23:01:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4adc0a1120 0x7f4adc0a35e0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009df0 tx=0x7f4ae8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8108d40 0x7f4af81a06a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4af8073a00 0x7f4af8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 --2- 172.21.15.46:0/1653106256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af8074fe0 0x7f4af8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.533+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 >> 172.21.15.46:0/1653106256 conn(0x7f4af80fc410 msgr2=0x7f4af8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.534+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 shutdown_connections 2024-05-08T23:01:39.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:39.535+0000 7f4afe200700 1 -- 172.21.15.46:0/1653106256 wait complete. 2024-05-08T23:01:39.537 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-05-08T23:01:40.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:39 smithi053 ceph-mon[77587]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:40.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:39 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1653106256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:40.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:39 smithi046 ceph-mon[36898]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:40.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:39 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1653106256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:40.027 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:01:39 smithi046 systemd[1]: Starting Ceph mds.g for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:39 smithi086 ceph-mon[78610]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:39 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1653106256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:40.776 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:01:40 smithi046 bash[88441]: fb8e37e015127d865c2d97064b94d511bea0ee1b6d38eecb01676d5f7554c998 2024-05-08T23:01:40.777 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:01:40 smithi046 systemd[1]: Started Ceph mds.g for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:40.777 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:01:40 smithi046 ceph-mds[88479]: -- 172.21.15.46:0/2867328262 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c263861a0 con 0x559c2635e800 2024-05-08T23:01:40.777 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:01:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: starting mds.g at 2024-05-08T23:01:41.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:41.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: Deploying daemon mds.a on smithi046 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:boot 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: fsmap 10 up:standby 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:01:41.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:41 smithi046 ceph-mon[36898]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: Deploying daemon mds.a on smithi046 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:boot 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: fsmap 10 up:standby 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:01:41.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:41 smithi086 ceph-mon[78610]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: Deploying daemon mds.a on smithi046 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:boot 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: fsmap 10 up:standby 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:01:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:41 smithi053 ceph-mon[77587]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 -- 172.21.15.46:0/2626142040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0100e80 msgr2=0x7fb8b01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 --2- 172.21.15.46:0/2626142040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0100e80 0x7fb8b01012a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009990 tx=0x7fb8a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 -- 172.21.15.46:0/2626142040 shutdown_connections 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 --2- 172.21.15.46:0/2626142040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0102990 0x7fb8b01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 --2- 172.21.15.46:0/2626142040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0100e80 0x7fb8b01012a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 --2- 172.21.15.46:0/2626142040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 0x7fb8b0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 -- 172.21.15.46:0/2626142040 >> 172.21.15.46:0/2626142040 conn(0x7fb8b00fc410 msgr2=0x7fb8b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:41.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 -- 172.21.15.46:0/2626142040 shutdown_connections 2024-05-08T23:01:41.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.799+0000 7fb8b7800700 1 -- 172.21.15.46:0/2626142040 wait complete. 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 Processor -- start 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 -- start start 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 0x7fb8b0197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 0x7fb8b019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b019e950 con 0x7fb8b0102990 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b019ea90 con 0x7fb8b0100e80 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.800+0000 7fb8b7800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b019ec20 con 0x7fb8b0101fd0 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b5800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 0x7fb8b019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55752/0 (socket says 172.21.15.46:55752) 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 -- 172.21.15.46:0/1997250010 learned_addr learned my addr 172.21.15.46:0/1997250010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8afe00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 0x7fb8b0197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 -- 172.21.15.46:0/1997250010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 msgr2=0x7fb8b0197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 0x7fb8b0197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 -- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 msgr2=0x7fb8b019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 0x7fb8b019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 -- 172.21.15.46:0/1997250010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40095f0 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b5800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 0x7fb8b019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.801+0000 7fb8b4e00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4005910 tx=0x7fb8a4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.802+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a401d070 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.802+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a4021410 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.802+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b019eea0 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.802+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017650 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.802+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b019f210 con 0x7fb8b0100e80 2024-05-08T23:01:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.803+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b0067d80 con 0x7fb8b0100e80 2024-05-08T23:01:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.805+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb8a4021580 con 0x7fb8b0100e80 2024-05-08T23:01:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.806+0000 7fb8ad600700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 0x7fb8980a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.806+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8a40cd6b0 con 0x7fb8b0100e80 2024-05-08T23:01:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.806+0000 7fb8afe00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 0x7fb8980a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.807+0000 7fb8afe00700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 0x7fb8980a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009c60 tx=0x7fb89c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:41.809+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8a4097350 con 0x7fb8b0100e80 2024-05-08T23:01:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.101+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8b0067d10 con 0x7fb8b0100e80 2024-05-08T23:01:42.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.102+0000 7fb8ad600700 1 -- 172.21.15.46:0/1997250010 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8067 (secure 0 0 0) 0x7fb8a409a970 con 0x7fb8b0100e80 2024-05-08T23:01:42.101 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:42.101 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-05-08T23:01:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 msgr2=0x7fb8980a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 0x7fb8980a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009c60 tx=0x7fb89c009260 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 msgr2=0x7fb8b0196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4005910 tx=0x7fb8a4005740 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 shutdown_connections 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8980a11a0 0x7fb8980a3660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8b0102990 0x7fb8b019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b0100e80 0x7fb8b0196e20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 --2- 172.21.15.46:0/1997250010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8b0101fd0 0x7fb8b0197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 >> 172.21.15.46:0/1997250010 conn(0x7fb8b00fc410 msgr2=0x7fb8b0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 shutdown_connections 2024-05-08T23:01:42.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:42.105+0000 7fb8b7800700 1 -- 172.21.15.46:0/1997250010 wait complete. 2024-05-08T23:01:42.106 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-05-08T23:01:42.527 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:01:42 smithi046 systemd[1]: Starting Ceph mds.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:43.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:42 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1997250010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:43.027 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:01:42 smithi046 bash[88828]: 365257641f44159cd08e53ada692a62be5cf585249c92923b23bb1317ad2417b 2024-05-08T23:01:43.027 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:01:42 smithi046 systemd[1]: Started Ceph mds.a for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:43.027 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:01:42 smithi046 ceph-mds[88866]: -- 172.21.15.46:0/538900679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561293ac1a0 con 0x556129384c00 2024-05-08T23:01:43.027 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:01:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-a[88843]: starting mds.a at 2024-05-08T23:01:43.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:42 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1997250010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:43.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:42 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1997250010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:44.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: Deploying daemon mds.b on smithi053 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6840/1402919380,v1:172.21.15.46:6841/1402919380] up:boot 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: fsmap 11 up:standby 2024-05-08T23:01:44.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:43 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:01:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: Deploying daemon mds.b on smithi053 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6840/1402919380,v1:172.21.15.46:6841/1402919380] up:boot 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: fsmap 11 up:standby 2024-05-08T23:01:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:43 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:01:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:01:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: Deploying daemon mds.b on smithi053 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6840/1402919380,v1:172.21.15.46:6841/1402919380] up:boot 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: fsmap 11 up:standby 2024-05-08T23:01:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:43 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 -- 172.21.15.46:0/4133974683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c074fe0 msgr2=0x7f961c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 --2- 172.21.15.46:0/4133974683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c074fe0 0x7f961c073430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9610009990 tx=0x7f9610009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 -- 172.21.15.46:0/4133974683 shutdown_connections 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 --2- 172.21.15.46:0/4133974683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 --2- 172.21.15.46:0/4133974683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c074fe0 0x7f961c073430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 --2- 172.21.15.46:0/4133974683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c073a00 0x7f961c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 -- 172.21.15.46:0/4133974683 >> 172.21.15.46:0/4133974683 conn(0x7f961c0fc410 msgr2=0x7f961c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 -- 172.21.15.46:0/4133974683 shutdown_connections 2024-05-08T23:01:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.342+0000 7f9623600700 1 -- 172.21.15.46:0/4133974683 wait complete. 2024-05-08T23:01:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 Processor -- start 2024-05-08T23:01:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 -- start start 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c074fe0 0x7f961c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1a0b70 con 0x7f961c073a00 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1a0cb0 con 0x7f961c108d40 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9623600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1a0e40 con 0x7f961c074fe0 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.343+0000 7f9620c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49522/0 (socket says 172.21.15.46:49522) 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9621600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 -- 172.21.15.46:0/59075963 learned_addr learned my addr 172.21.15.46:0/59075963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f961be00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c074fe0 0x7f961c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 -- 172.21.15.46:0/59075963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c074fe0 msgr2=0x7f961c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c074fe0 0x7f961c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 -- 172.21.15.46:0/59075963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 msgr2=0x7f961c1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 -- 172.21.15.46:0/59075963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96100095f0 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9621600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9620c00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f961000b680 tx=0x7f9610009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961001d070 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c1a10c0 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.344+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9610022470 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.345+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610017650 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.345+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1a14e0 con 0x7f961c073a00 2024-05-08T23:01:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.346+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c107b00 con 0x7f961c073a00 2024-05-08T23:01:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.347+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f96100225e0 con 0x7f961c073a00 2024-05-08T23:01:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.349+0000 7f9619600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 0x7f96040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.349+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96100cd790 con 0x7f961c073a00 2024-05-08T23:01:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.349+0000 7f961be00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 0x7f96040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.350+0000 7f961be00700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 0x7f96040a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f961c074d00 tx=0x7f9608009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:44.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.353+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f96100973b0 con 0x7f961c073a00 2024-05-08T23:01:44.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.619+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f961c067d10 con 0x7f961c073a00 2024-05-08T23:01:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.620+0000 7f9619600700 1 -- 172.21.15.46:0/59075963 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8825 (secure 0 0 0) 0x7f961009a9d0 con 0x7f961c073a00 2024-05-08T23:01:44.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:44.619 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.622+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 msgr2=0x7f96040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.622+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 0x7f96040a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f961c074d00 tx=0x7f9608009260 comp rx=0 tx=0).stop 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.622+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 msgr2=0x7f961c199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.622+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f961000b680 tx=0x7f9610009580 comp rx=0 tx=0).stop 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 shutdown_connections 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f961c108d40 0x7f961c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f96040a1120 0x7f96040a35e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f961c073a00 0x7f961c199090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 --2- 172.21.15.46:0/59075963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f961c074fe0 0x7f961c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 >> 172.21.15.46:0/59075963 conn(0x7f961c0fc410 msgr2=0x7f961c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 shutdown_connections 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:44.623+0000 7f9623600700 1 -- 172.21.15.46:0/59075963 wait complete. 2024-05-08T23:01:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 12 2024-05-08T23:01:44.762 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:44 smithi053 systemd[1]: Starting Ceph mds.b for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:01:45.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:44 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/59075963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:45.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:44 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/59075963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:45.261 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:45 smithi053 bash[114299]: 24090139d1d84de951f93f878435c610eae13ed4bb6de92e5258a8350daeeeda 2024-05-08T23:01:45.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mds[114337]: -- 172.21.15.53:0/66960397 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7b45561a0 con 0x55e7b452cc00 2024-05-08T23:01:45.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:45 smithi053 systemd[1]: Started Ceph mds.b for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:01:45.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: starting mds.b at 2024-05-08T23:01:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:44 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/59075963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:45 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:45 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:45 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:46.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 -- 172.21.15.46:0/2818269406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 msgr2=0x7fdccc103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 --2- 172.21.15.46:0/2818269406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc103cf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009990 tx=0x7fdcbc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 -- 172.21.15.46:0/2818269406 shutdown_connections 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 --2- 172.21.15.46:0/2818269406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 0x7fdccc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 --2- 172.21.15.46:0/2818269406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc103cf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 --2- 172.21.15.46:0/2818269406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 0x7fdccc106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 -- 172.21.15.46:0/2818269406 >> 172.21.15.46:0/2818269406 conn(0x7fdccc0fb310 msgr2=0x7fdccc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 -- 172.21.15.46:0/2818269406 shutdown_connections 2024-05-08T23:01:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.859+0000 7fdcd1200700 1 -- 172.21.15.46:0/2818269406 wait complete. 2024-05-08T23:01:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.860+0000 7fdcd1200700 1 Processor -- start 2024-05-08T23:01:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 -- start start 2024-05-08T23:01:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 0x7fdccc199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 0x7fdccc1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1a0af0 con 0x7fdccc101900 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1a0c30 con 0x7fdccc106b60 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcd1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1a0dc0 con 0x7fdccc104230 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcc9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcc9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49538/0 (socket says 172.21.15.46:49538) 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcc9e00700 1 -- 172.21.15.46:0/1043185376 learned_addr learned my addr 172.21.15.46:0/1043185376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcca800700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 0x7fdccc1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.861+0000 7fdcc9400700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 0x7fdccc199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 -- 172.21.15.46:0/1043185376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 msgr2=0x7fdccc199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 0x7fdccc199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 -- 172.21.15.46:0/1043185376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 msgr2=0x7fdccc1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 0x7fdccc1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 -- 172.21.15.46:0/1043185376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0095f0 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc9e00700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005910 tx=0x7fdcbc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc01d070 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc1a1040 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcbc021410 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc1a14c0 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.862+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc005720 con 0x7fdccc101900 2024-05-08T23:01:46.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.863+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc067d10 con 0x7fdccc101900 2024-05-08T23:01:46.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.865+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdcbc021580 con 0x7fdccc101900 2024-05-08T23:01:46.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.867+0000 7fdcc2a00700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 0x7fdcb00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:46.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.867+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcbc0cd780 con 0x7fdccc101900 2024-05-08T23:01:46.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.867+0000 7fdcc9400700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 0x7fdcb00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:46.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.868+0000 7fdcc9400700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 0x7fdcb00a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c60 tx=0x7fdcb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:46.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:46.871+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdcbc0974c0 con 0x7fdccc101900 2024-05-08T23:01:47.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:boot 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: fsmap 12 up:standby 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:46 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.162+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdccc0ffe70 con 0x7fdccc101900 2024-05-08T23:01:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.163+0000 7fdcc2a00700 1 -- 172.21.15.46:0/1043185376 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9583 (secure 0 0 0) 0x7fdcbc09aae0 con 0x7fdccc101900 2024-05-08T23:01:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:47.163 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 msgr2=0x7fdcb00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 0x7fdcb00a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c60 tx=0x7fdcb8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 msgr2=0x7fdccc198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005910 tx=0x7fdcbc009580 comp rx=0 tx=0).stop 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 shutdown_connections 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdccc106b60 0x7fdccc1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdcb00a1120 0x7fdcb00a35e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdccc101900 0x7fdccc198fc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 --2- 172.21.15.46:0/1043185376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdccc104230 0x7fdccc199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 >> 172.21.15.46:0/1043185376 conn(0x7fdccc0fb310 msgr2=0x7fdccc106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 shutdown_connections 2024-05-08T23:01:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.166+0000 7fdcd1200700 1 -- 172.21.15.46:0/1043185376 wait complete. 2024-05-08T23:01:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-05-08T23:01:47.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:01:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:boot 2024-05-08T23:01:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: fsmap 12 up:standby 2024-05-08T23:01:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:01:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:46 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:boot 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: fsmap 12 up:standby 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:46 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 -- 172.21.15.46:0/2905815947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 msgr2=0x7f1590108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 --2- 172.21.15.46:0/2905815947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f1590108ff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f158800b0a0 tx=0x7f158800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 -- 172.21.15.46:0/2905815947 shutdown_connections 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 --2- 172.21.15.46:0/2905815947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f1590108ff0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 --2- 172.21.15.46:0/2905815947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901019a0 0x7f1590103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 --2- 172.21.15.46:0/2905815947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901042d0 0x7f15901066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 -- 172.21.15.46:0/2905815947 >> 172.21.15.46:0/2905815947 conn(0x7f15900fb310 msgr2=0x7f15900fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 -- 172.21.15.46:0/2905815947 shutdown_connections 2024-05-08T23:01:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.393+0000 7f1595000700 1 -- 172.21.15.46:0/2905815947 wait complete. 2024-05-08T23:01:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.394+0000 7f1595000700 1 Processor -- start 2024-05-08T23:01:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.394+0000 7f1595000700 1 -- start start 2024-05-08T23:01:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f15901a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a0c20 con 0x7f1590106c00 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a0d60 con 0x7f15901042d0 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f1595000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901a0ef0 con 0x7f15901019a0 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55168/0 (socket says 172.21.15.46:55168) 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38700/0 (socket says 172.21.15.46:38700) 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158d400700 1 -- 172.21.15.46:0/3447966750 learned_addr learned my addr 172.21.15.46:0/3447966750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.395+0000 7f158e800700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f15901a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 -- 172.21.15.46:0/3447966750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 msgr2=0x7f15901990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 -- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 msgr2=0x7f15901a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f15901a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 -- 172.21.15.46:0/3447966750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1580009520 con 0x7f15901042d0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158de00700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158e800700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f15901a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f158d400700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f157c00e9a0 tx=0x7f157c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c009790 con 0x7f15901042d0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1588009d00 con 0x7f15901042d0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f157c010430 con 0x7f15901042d0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c018680 con 0x7f15901042d0 2024-05-08T23:01:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.396+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a15a0 con 0x7f15901042d0 2024-05-08T23:01:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.397+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590108780 con 0x7f15901042d0 2024-05-08T23:01:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.400+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f157c0105a0 con 0x7f15901042d0 2024-05-08T23:01:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.401+0000 7f1586a00700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 0x7f15740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.402+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f157c014070 con 0x7f15901042d0 2024-05-08T23:01:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.402+0000 7f158de00700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 0x7f15740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.402+0000 7f158de00700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 0x7f15740a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1580009fd0 tx=0x7f1580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.404+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f157c096430 con 0x7f15901042d0 2024-05-08T23:01:47.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.631+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1590067480 con 0x7f15901042d0 2024-05-08T23:01:47.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.633+0000 7f1586a00700 1 -- 172.21.15.46:0/3447966750 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f157c099a50 con 0x7f15901042d0 2024-05-08T23:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:13.608146+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 msgr2=0x7f15740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 0x7f15740a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1580009fd0 tx=0x7f1580009260 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 msgr2=0x7f1590199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f157c00e9a0 tx=0x7f157c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 shutdown_connections 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f15740a11a0 0x7f15740a3660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1590106c00 0x7f15901a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15901019a0 0x7f15901990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 --2- 172.21.15.46:0/3447966750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15901042d0 0x7f1590199630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 >> 172.21.15.46:0/3447966750 conn(0x7f15900fb310 msgr2=0x7f1590101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 shutdown_connections 2024-05-08T23:01:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.636+0000 7f1595000700 1 -- 172.21.15.46:0/3447966750 wait complete. 2024-05-08T23:01:47.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:01:47.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:01:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 -- 172.21.15.46:0/663669025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac41018e0 msgr2=0x7f0ac4103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 --2- 172.21.15.46:0/663669025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac41018e0 0x7f0ac4103cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009990 tx=0x7f0ab4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 -- 172.21.15.46:0/663669025 shutdown_connections 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 --2- 172.21.15.46:0/663669025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 0x7f0ac4108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 --2- 172.21.15.46:0/663669025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac41018e0 0x7f0ac4103cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 --2- 172.21.15.46:0/663669025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac4104210 0x7f0ac4106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 -- 172.21.15.46:0/663669025 >> 172.21.15.46:0/663669025 conn(0x7f0ac40fb290 msgr2=0x7f0ac40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 -- 172.21.15.46:0/663669025 shutdown_connections 2024-05-08T23:01:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.875+0000 7f0aca800700 1 -- 172.21.15.46:0/663669025 wait complete. 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.876+0000 7f0aca800700 1 Processor -- start 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.876+0000 7f0aca800700 1 -- start start 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.876+0000 7f0aca800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.876+0000 7f0aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac4104210 0x7f0ac4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0aca800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 0x7f0ac41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0aca800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41a0b40 con 0x7f0ac4104210 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0aca800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41a0c80 con 0x7f0ac4106b40 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0aca800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41a0e10 con 0x7f0ac41018e0 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38720/0 (socket says 172.21.15.46:38720) 2024-05-08T23:01:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 -- 172.21.15.46:0/3624870689 learned_addr learned my addr 172.21.15.46:0/3624870689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac2a00700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac4104210 0x7f0ac4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3e00700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 0x7f0ac41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 -- 172.21.15.46:0/3624870689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 msgr2=0x7f0ac41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 0x7f0ac41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 -- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac4104210 msgr2=0x7f0ac4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac4104210 0x7f0ac4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 -- 172.21.15.46:0/3624870689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0009520 con 0x7f0ac41018e0 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.877+0000 7f0ac3400700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b680 tx=0x7f0ab4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.878+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401d070 con 0x7f0ac41018e0 2024-05-08T23:01:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.878+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ab4022470 con 0x7f0ac41018e0 2024-05-08T23:01:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.878+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40095f0 con 0x7f0ac41018e0 2024-05-08T23:01:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.878+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4017650 con 0x7f0ac41018e0 2024-05-08T23:01:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.878+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41a1380 con 0x7f0ac41018e0 2024-05-08T23:01:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.879+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4067d10 con 0x7f0ac41018e0 2024-05-08T23:01:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.882+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0ab40225e0 con 0x7f0ac41018e0 2024-05-08T23:01:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.884+0000 7f0abbe00700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 0x7f0aa80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.884+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ab40cd860 con 0x7f0ac41018e0 2024-05-08T23:01:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.884+0000 7f0ac2a00700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 0x7f0aa80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.884+0000 7f0ac2a00700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 0x7f0aa80a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009df0 tx=0x7f0ab0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:47.885+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ab4097480 con 0x7f0ac41018e0 2024-05-08T23:01:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.117+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ac410c820 con 0x7f0ac41018e0 2024-05-08T23:01:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.118+0000 7f0abbe00700 1 -- 172.21.15.46:0/3624870689 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ab409aaa0 con 0x7f0ac41018e0 2024-05-08T23:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.120+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 msgr2=0x7f0aa80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.120+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 0x7f0aa80a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009df0 tx=0x7f0ab0009260 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.120+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 msgr2=0x7f0ac4199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.120+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b680 tx=0x7f0ab4009580 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 shutdown_connections 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4106b40 0x7f0ac41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac41018e0 0x7f0ac4199010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0aa80a11a0 0x7f0aa80a3660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 --2- 172.21.15.46:0/3624870689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac4104210 0x7f0ac4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 >> 172.21.15.46:0/3624870689 conn(0x7f0ac40fb290 msgr2=0x7f0ac40ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:48.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 shutdown_connections 2024-05-08T23:01:48.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.121+0000 7f0aca800700 1 -- 172.21.15.46:0/3624870689 wait complete. 2024-05-08T23:01:48.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:48.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.203 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1043185376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.203 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.203 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:47 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3447966750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1043185376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:47 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3447966750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:48.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1043185376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:47 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3447966750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.350+0000 7f58f0e00700 1 -- 172.21.15.46:0/2728546204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec100e80 msgr2=0x7f58ec1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.350+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2728546204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec100e80 0x7f58ec1012a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009990 tx=0x7f58dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 -- 172.21.15.46:0/2728546204 shutdown_connections 2024-05-08T23:01:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2728546204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec102990 0x7f58ec1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2728546204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec100e80 0x7f58ec1012a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2728546204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec101fd0 0x7f58ec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 -- 172.21.15.46:0/2728546204 >> 172.21.15.46:0/2728546204 conn(0x7f58ec0fc410 msgr2=0x7f58ec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 -- 172.21.15.46:0/2728546204 shutdown_connections 2024-05-08T23:01:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.351+0000 7f58f0e00700 1 -- 172.21.15.46:0/2728546204 wait complete. 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 Processor -- start 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 -- start start 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec101fd0 0x7f58ec073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec102990 0x7f58ec0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec074070 con 0x7f58ec101fd0 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0741e0 con 0x7f58ec100e80 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58f0e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1a0e20 con 0x7f58ec102990 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58e9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58e9400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec101fd0 0x7f58ec073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58e9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55218/0 (socket says 172.21.15.46:55218) 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.352+0000 7f58e9e00700 1 -- 172.21.15.46:0/2857525273 learned_addr learned my addr 172.21.15.46:0/2857525273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58ea800700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec102990 0x7f58ec0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 -- 172.21.15.46:0/2857525273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec102990 msgr2=0x7f58ec0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec102990 0x7f58ec0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 -- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec101fd0 msgr2=0x7f58ec073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec101fd0 0x7f58ec073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 -- 172.21.15.46:0/2857525273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0095f0 con 0x7f58ec100e80 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e9e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58dc005910 tx=0x7f58dc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc01d070 con 0x7f58ec100e80 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1a1050 con 0x7f58ec100e80 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58dc021410 con 0x7f58ec100e80 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.353+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc005720 con 0x7f58ec100e80 2024-05-08T23:01:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.354+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1a1470 con 0x7f58ec100e80 2024-05-08T23:01:48.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.356+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec108310 con 0x7f58ec100e80 2024-05-08T23:01:48.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.357+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f58dc021580 con 0x7f58ec100e80 2024-05-08T23:01:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.358+0000 7f58e2a00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 0x7f58d00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.358+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58dc0cd790 con 0x7f58ec100e80 2024-05-08T23:01:48.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.359+0000 7f58e9400700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 0x7f58d00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.360+0000 7f58e9400700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 0x7f58d00a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009c60 tx=0x7f58d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:48.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.363+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f58dc0973b0 con 0x7f58ec100e80 2024-05-08T23:01:48.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.601+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f58ec02cda0 con 0x7f58ec100e80 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.602+0000 7f58e2a00700 1 -- 172.21.15.46:0/2857525273 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+65 (secure 0 0 0) 0x7f58dc09a9d0 con 0x7f58ec100e80 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-05-08T23:01:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 msgr2=0x7f58d00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 0x7f58d00a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009c60 tx=0x7f58d8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 msgr2=0x7f58ec074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58dc005910 tx=0x7f58dc009580 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 shutdown_connections 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ec102990 0x7f58ec0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec100e80 0x7f58ec074d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f58d00a1120 0x7f58d00a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.604+0000 7f58f0e00700 1 --2- 172.21.15.46:0/2857525273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec101fd0 0x7f58ec073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.605+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 >> 172.21.15.46:0/2857525273 conn(0x7f58ec0fc410 msgr2=0x7f58ec105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.605+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 shutdown_connections 2024-05-08T23:01:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.605+0000 7f58f0e00700 1 -- 172.21.15.46:0/2857525273 wait complete. 2024-05-08T23:01:48.624 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:01:48.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 -- 172.21.15.46:0/3700602367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4073a00 msgr2=0x7fa5b4073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 --2- 172.21.15.46:0/3700602367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4073a00 0x7fa5b4073e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009990 tx=0x7fa5a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 -- 172.21.15.46:0/3700602367 shutdown_connections 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 --2- 172.21.15.46:0/3700602367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4108d40 0x7fa5b410b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 --2- 172.21.15.46:0/3700602367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 --2- 172.21.15.46:0/3700602367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4073a00 0x7fa5b4073e80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 -- 172.21.15.46:0/3700602367 >> 172.21.15.46:0/3700602367 conn(0x7fa5b40fc3f0 msgr2=0x7fa5b40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 -- 172.21.15.46:0/3700602367 shutdown_connections 2024-05-08T23:01:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.846+0000 7fa5b9200700 1 -- 172.21.15.46:0/3700602367 wait complete. 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 Processor -- start 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 -- start start 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4108d40 0x7fa5b41a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41a0b70 con 0x7fa5b4074fe0 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41a0cb0 con 0x7fa5b4073a00 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.847+0000 7fa5b9200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41a0e40 con 0x7fa5b4108d40 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55230/0 (socket says 172.21.15.46:55230) 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49604/0 (socket says 172.21.15.46:49604) 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1e00700 1 -- 172.21.15.46:0/1119008748 learned_addr learned my addr 172.21.15.46:0/1119008748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b2800700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4108d40 0x7fa5b41a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 -- 172.21.15.46:0/1119008748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4108d40 msgr2=0x7fa5b41a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4108d40 0x7fa5b41a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 -- 172.21.15.46:0/1119008748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 msgr2=0x7fa5b4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 -- 172.21.15.46:0/1119008748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00095f0 con 0x7fa5b4074fe0 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1e00700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.848+0000 7fa5b1400700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b680 tx=0x7fa5a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.849+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a001d070 con 0x7fa5b4074fe0 2024-05-08T23:01:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.849+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5a0021410 con 0x7fa5b4074fe0 2024-05-08T23:01:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.849+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b41a10c0 con 0x7fa5b4074fe0 2024-05-08T23:01:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.849+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0017650 con 0x7fa5b4074fe0 2024-05-08T23:01:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.849+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b41a1540 con 0x7fa5b4074fe0 2024-05-08T23:01:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.851+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b4107b00 con 0x7fa5b4074fe0 2024-05-08T23:01:48.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.852+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa5a000fb10 con 0x7fa5b4074fe0 2024-05-08T23:01:48.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.853+0000 7fa5aaa00700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 0x7fa5980a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:48.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.853+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5a00cd5e0 con 0x7fa5b4074fe0 2024-05-08T23:01:48.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.854+0000 7fa5b1e00700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 0x7fa5980a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:48.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.854+0000 7fa5b1e00700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 0x7fa5980a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4011550 tx=0x7fa5a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:48.857+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa5a0097200 con 0x7fa5b4074fe0 2024-05-08T23:01:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.085+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa5b41074e0 con 0x7fa5b4074fe0 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3624870689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:48 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2857525273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:01:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3624870689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:48 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2857525273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:01:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3624870689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:01:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:49.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:48 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2857525273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:01:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.931+0000 7fa5aaa00700 1 -- 172.21.15.46:0/1119008748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5a0021e60 con 0x7fa5b4074fe0 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 msgr2=0x7fa5980a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 0x7fa5980a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4011550 tx=0x7fa5a4009260 comp rx=0 tx=0).stop 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 msgr2=0x7fa5b4199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b680 tx=0x7fa5a0004730 comp rx=0 tx=0).stop 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 shutdown_connections 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5b4108d40 0x7fa5b41a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b4073a00 0x7fa5b4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa5980a11a0 0x7fa5980a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 --2- 172.21.15.46:0/1119008748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b4074fe0 0x7fa5b4199580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 >> 172.21.15.46:0/1119008748 conn(0x7fa5b40fc3f0 msgr2=0x7fa5b4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.936+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 shutdown_connections 2024-05-08T23:01:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:49.937+0000 7fa5b9200700 1 -- 172.21.15.46:0/1119008748 wait complete. 2024-05-08T23:01:49.938 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-05-08T23:01:49.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T23:01:50.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 -- 172.21.15.46:0/1996717308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc100e80 msgr2=0x7f05bc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 --2- 172.21.15.46:0/1996717308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc100e80 0x7f05bc1012a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 -- 172.21.15.46:0/1996717308 shutdown_connections 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 --2- 172.21.15.46:0/1996717308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc102990 0x7f05bc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 --2- 172.21.15.46:0/1996717308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc100e80 0x7f05bc1012a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 --2- 172.21.15.46:0/1996717308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc101fd0 0x7f05bc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 -- 172.21.15.46:0/1996717308 >> 172.21.15.46:0/1996717308 conn(0x7f05bc0fc3b0 msgr2=0x7f05bc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 -- 172.21.15.46:0/1996717308 shutdown_connections 2024-05-08T23:01:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.163+0000 7f05c3400700 1 -- 172.21.15.46:0/1996717308 wait complete. 2024-05-08T23:01:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.164+0000 7f05c3400700 1 Processor -- start 2024-05-08T23:01:50.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.164+0000 7f05c3400700 1 -- start start 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc101fd0 0x7f05bc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 0x7f05bc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc074070 con 0x7f05bc102990 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0741e0 con 0x7f05bc101fd0 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc1a0ea0 con 0x7f05bc100e80 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 0x7f05bc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc101fd0 0x7f05bc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 0x7f05bc0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49632/0 (socket says 172.21.15.46:49632) 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c1400700 1 -- 172.21.15.46:0/3338267938 learned_addr learned my addr 172.21.15.46:0/3338267938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38770/0 (socket says 172.21.15.46:38770) 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c0a00700 1 -- 172.21.15.46:0/3338267938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc101fd0 msgr2=0x7f05bc073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.165+0000 7f05c0a00700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc101fd0 0x7f05bc073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c0a00700 1 -- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 msgr2=0x7f05bc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c0a00700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 0x7f05bc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c0a00700 1 -- 172.21.15.46:0/3338267938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4009c40 con 0x7f05bc100e80 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c0a00700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000c00 tx=0x7f05b0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b001d070 con 0x7f05bc100e80 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b00095f0 con 0x7f05bc100e80 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b0021410 con 0x7f05bc100e80 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017650 con 0x7f05bc100e80 2024-05-08T23:01:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.166+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc1a1310 con 0x7f05bc100e80 2024-05-08T23:01:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.167+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc108290 con 0x7f05bc100e80 2024-05-08T23:01:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.169+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f05b0021580 con 0x7f05bc100e80 2024-05-08T23:01:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.171+0000 7f05b9600700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 0x7f05a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.171+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f05b00cdfc0 con 0x7f05bc100e80 2024-05-08T23:01:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.171+0000 7f05bbe00700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 0x7f05a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.171+0000 7f05bbe00700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 0x7f05a40a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009c60 tx=0x7f05a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.173+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05b0097a70 con 0x7f05bc100e80 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:49 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:01:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:49 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:01:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:01:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:01:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:49 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:01:50.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.390+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f05bc067d10 con 0x7f05bc100e80 2024-05-08T23:01:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.942+0000 7f05b9600700 1 -- 172.21.15.46:0/3338267938 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) v1 ==== 171+0+0 (secure 0 0 0) 0x7f05b0005740 con 0x7f05bc100e80 2024-05-08T23:01:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.948+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 msgr2=0x7f05a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.948+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 0x7f05a40a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009c60 tx=0x7f05a8009260 comp rx=0 tx=0).stop 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.948+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 msgr2=0x7f05bc074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.948+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000c00 tx=0x7f05b0009580 comp rx=0 tx=0).stop 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 shutdown_connections 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05a40a1120 0x7f05a40a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05bc102990 0x7f05bc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc100e80 0x7f05bc074d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 --2- 172.21.15.46:0/3338267938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05bc101fd0 0x7f05bc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 >> 172.21.15.46:0/3338267938 conn(0x7f05bc0fc3b0 msgr2=0x7f05bc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 shutdown_connections 2024-05-08T23:01:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:50.949+0000 7f05c3400700 1 -- 172.21.15.46:0/3338267938 wait complete. 2024-05-08T23:01:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-05-08T23:01:50.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.179+0000 7f17f0400700 1 -- 172.21.15.46:0/4160996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 msgr2=0x7f17e8073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.179+0000 7f17f0400700 1 --2- 172.21.15.46:0/4160996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8073e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009990 tx=0x7f17d4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 -- 172.21.15.46:0/4160996828 shutdown_connections 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 --2- 172.21.15.46:0/4160996828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 0x7f17e810b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 --2- 172.21.15.46:0/4160996828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 0x7f17e8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 --2- 172.21.15.46:0/4160996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8073e80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 -- 172.21.15.46:0/4160996828 >> 172.21.15.46:0/4160996828 conn(0x7f17e80fc3d0 msgr2=0x7f17e80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 -- 172.21.15.46:0/4160996828 shutdown_connections 2024-05-08T23:01:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.180+0000 7f17f0400700 1 -- 172.21.15.46:0/4160996828 wait complete. 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 Processor -- start 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 -- start start 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 0x7f17e8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 0x7f17e81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e81a0b10 con 0x7f17e8108d40 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e81a0c50 con 0x7f17e8074fe0 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17f0400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e81a0de0 con 0x7f17e8073a00 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17eda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17ed000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 0x7f17e8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17eda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38788/0 (socket says 172.21.15.46:38788) 2024-05-08T23:01:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17eda00700 1 -- 172.21.15.46:0/1731274238 learned_addr learned my addr 172.21.15.46:0/1731274238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.181+0000 7f17ee400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 0x7f17e81a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 -- 172.21.15.46:0/1731274238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 msgr2=0x7f17e8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 0x7f17e8199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 -- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 msgr2=0x7f17e81a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 0x7f17e81a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 -- 172.21.15.46:0/1731274238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d40095f0 con 0x7f17e8073a00 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17eda00700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00e9c0 tx=0x7f17dc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0097b0 con 0x7f17e8073a00 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17dc00ee60 con 0x7f17e8073a00 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e81a10c0 con 0x7f17e8073a00 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.182+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0054f0 con 0x7f17e8073a00 2024-05-08T23:01:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.183+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e81a15d0 con 0x7f17e8073a00 2024-05-08T23:01:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.184+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e8107b00 con 0x7f17e8073a00 2024-05-08T23:01:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.186+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f17dc004750 con 0x7f17e8073a00 2024-05-08T23:01:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.187+0000 7f17e6000700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 0x7f17d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.187+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f17dc014070 con 0x7f17e8073a00 2024-05-08T23:01:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.187+0000 7f17ed000700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 0x7f17d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.188+0000 7f17ed000700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 0x7f17d00a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b680 tx=0x7f17d4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.189+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f17dc0969a0 con 0x7f17e8073a00 2024-05-08T23:01:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:50 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:01:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:50 smithi086 ceph-mon[78610]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:01:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:50 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3338267938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:50 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:50 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:01:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:50 smithi053 ceph-mon[77587]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:01:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:50 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3338267938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:50 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:50 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1119008748' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:01:51.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:50 smithi046 ceph-mon[36898]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:01:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:50 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3338267938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:50 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:01:51.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.457+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f17e810ead0 con 0x7f17e8073a00 2024-05-08T23:01:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.467+0000 7f17e6000700 1 -- 172.21.15.46:0/1731274238 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f17dc099fc0 con 0x7f17e8073a00 2024-05-08T23:01:51.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.471+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 msgr2=0x7f17d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.471+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 0x7f17d00a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b680 tx=0x7f17d4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.471+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 msgr2=0x7f17e8199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.471+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00e9c0 tx=0x7f17dc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 shutdown_connections 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17d00a11a0 0x7f17d00a3660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e8108d40 0x7f17e81a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e8073a00 0x7f17e8199030 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 --2- 172.21.15.46:0/1731274238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8074fe0 0x7f17e8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.472+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 >> 172.21.15.46:0/1731274238 conn(0x7f17e80fc3d0 msgr2=0x7f17e8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.473+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 shutdown_connections 2024-05-08T23:01:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.473+0000 7f17f0400700 1 -- 172.21.15.46:0/1731274238 wait complete. 2024-05-08T23:01:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:01:51.507 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T23:01:51.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T23:01:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 -- 172.21.15.46:0/2036850066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c073a00 msgr2=0x7fb52c073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 --2- 172.21.15.46:0/2036850066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c073a00 0x7fb52c073e80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb518009990 tx=0x7fb518009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 -- 172.21.15.46:0/2036850066 shutdown_connections 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 --2- 172.21.15.46:0/2036850066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c108d40 0x7fb52c10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 --2- 172.21.15.46:0/2036850066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 0x7fb52c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 --2- 172.21.15.46:0/2036850066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c073a00 0x7fb52c073e80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 -- 172.21.15.46:0/2036850066 >> 172.21.15.46:0/2036850066 conn(0x7fb52c0fc3b0 msgr2=0x7fb52c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 -- 172.21.15.46:0/2036850066 shutdown_connections 2024-05-08T23:01:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.710+0000 7fb530c00700 1 -- 172.21.15.46:0/2036850066 wait complete. 2024-05-08T23:01:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.711+0000 7fb530c00700 1 Processor -- start 2024-05-08T23:01:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.711+0000 7fb530c00700 1 -- start start 2024-05-08T23:01:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 0x7fb52c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 0x7fb52c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 0x7fb52c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1a0b90 con 0x7fb52c108d40 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1a0cd0 con 0x7fb52c074fe0 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb530c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1a0e60 con 0x7fb52c073a00 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb529e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 0x7fb52c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 0x7fb52c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb529e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 0x7fb52c199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38804/0 (socket says 172.21.15.46:38804) 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb529e00700 1 -- 172.21.15.46:0/1727500678 learned_addr learned my addr 172.21.15.46:0/1727500678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb529400700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 0x7fb52c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 -- 172.21.15.46:0/1727500678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 msgr2=0x7fb52c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 0x7fb52c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 -- 172.21.15.46:0/1727500678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 msgr2=0x7fb52c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 0x7fb52c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.712+0000 7fb52a800700 1 -- 172.21.15.46:0/1727500678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180095f0 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb52a800700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 0x7fb52c1a05c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb52400efc0 tx=0x7fb52400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524018070 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52400f040 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c1a10e0 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400cd40 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.713+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1a15f0 con 0x7fb52c108d40 2024-05-08T23:01:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.714+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c107b00 con 0x7fb52c108d40 2024-05-08T23:01:51.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.716+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb524003a00 con 0x7fb52c108d40 2024-05-08T23:01:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.717+0000 7fb522a00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 0x7fb5100a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.717+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb5240cccc0 con 0x7fb52c108d40 2024-05-08T23:01:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.718+0000 7fb529e00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 0x7fb5100a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.718+0000 7fb529e00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 0x7fb5100a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007d10 tx=0x7fb51c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.720+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb524096590 con 0x7fb52c108d40 2024-05-08T23:01:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:51.953+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb52c067d10 con 0x7fb52c108d40 2024-05-08T23:01:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: pgmap v232: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:01:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:01:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:01:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1731274238' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:01:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:51 smithi086 ceph-mon[78610]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:01:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: pgmap v232: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:01:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:01:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1731274238' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:01:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:51 smithi053 ceph-mon[77587]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: pgmap v232: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1731274238' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:01:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:51 smithi046 ceph-mon[36898]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:01:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.471+0000 7fb522a00700 1 -- 172.21.15.46:0/1727500678 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v78) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb524094e60 con 0x7fb52c108d40 2024-05-08T23:01:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 msgr2=0x7fb5100a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 0x7fb5100a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007d10 tx=0x7fb51c007480 comp rx=0 tx=0).stop 2024-05-08T23:01:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 msgr2=0x7fb52c1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 0x7fb52c1a05c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb52400efc0 tx=0x7fb52400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 shutdown_connections 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb5100a11a0 0x7fb5100a3660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52c108d40 0x7fb52c1a05c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52c073a00 0x7fb52c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 --2- 172.21.15.46:0/1727500678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074fe0 0x7fb52c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 >> 172.21.15.46:0/1727500678 conn(0x7fb52c0fc3b0 msgr2=0x7fb52c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 shutdown_connections 2024-05-08T23:01:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.474+0000 7fb530c00700 1 -- 172.21.15.46:0/1727500678 wait complete. 2024-05-08T23:01:52.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:01:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 -- 172.21.15.46:0/2724545131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98100e80 msgr2=0x7f6a981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/2724545131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98100e80 0x7f6a981012a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009990 tx=0x7f6a88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 -- 172.21.15.46:0/2724545131 shutdown_connections 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/2724545131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98102990 0x7f6a981092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/2724545131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98100e80 0x7f6a981012a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/2724545131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98101fd0 0x7f6a98102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 -- 172.21.15.46:0/2724545131 >> 172.21.15.46:0/2724545131 conn(0x7f6a980fc3d0 msgr2=0x7f6a980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 -- 172.21.15.46:0/2724545131 shutdown_connections 2024-05-08T23:01:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.710+0000 7f6a9ec00700 1 -- 172.21.15.46:0/2724545131 wait complete. 2024-05-08T23:01:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.711+0000 7f6a9ec00700 1 Processor -- start 2024-05-08T23:01:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.711+0000 7f6a9ec00700 1 -- start start 2024-05-08T23:01:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.711+0000 7f6a9ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98101fd0 0x7f6a98073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98102990 0x7f6a980738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98074070 con 0x7f6a98102990 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980741e0 con 0x7f6a98101fd0 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981a0e80 con 0x7f6a98100e80 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55124/0 (socket says 172.21.15.46:55124) 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 -- 172.21.15.46:0/3116091691 learned_addr learned my addr 172.21.15.46:0/3116091691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a9cc00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98102990 0x7f6a980738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97400700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98101fd0 0x7f6a98073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 -- 172.21.15.46:0/3116091691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98101fd0 msgr2=0x7f6a98073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98101fd0 0x7f6a98073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 -- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98102990 msgr2=0x7f6a980738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98102990 0x7f6a980738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.712+0000 7f6a97e00700 1 -- 172.21.15.46:0/3116091691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880095f0 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a97e00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8801d070 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a981a10b0 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a88021410 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88017650 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.713+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a981a14d0 con 0x7f6a98100e80 2024-05-08T23:01:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.714+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a98108310 con 0x7f6a98100e80 2024-05-08T23:01:52.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.716+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6a88021580 con 0x7f6a98100e80 2024-05-08T23:01:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.717+0000 7f6a94c00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 0x7f6a7c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.717+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f6a880ce0a0 con 0x7f6a98100e80 2024-05-08T23:01:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.717+0000 7f6a97400700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 0x7f6a7c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.718+0000 7f6a97400700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 0x7f6a7c0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009bf0 tx=0x7f6a84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.719+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a88097880 con 0x7f6a98100e80 2024-05-08T23:01:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:52.941+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6a9810c800 con 0x7f6a98100e80 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:active 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:01:53.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:52 smithi086 ceph-mon[78610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:53.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:01:53.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:active 2024-05-08T23:01:53.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:01:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:01:53.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:52 smithi053 ceph-mon[77587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:01:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] up:active 2024-05-08T23:01:53.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1727500678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:01:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:01:53.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:52 smithi046 ceph-mon[36898]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:54 smithi053 ceph-mon[77587]: pgmap v236: 97 pgs: 3 creating+activating, 7 creating+peering, 39 active+clean, 48 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:01:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3116091691' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:54 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:54 smithi053 ceph-mon[77587]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:01:54.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:54 smithi053 ceph-mon[77587]: Cluster is now healthy 2024-05-08T23:01:54.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:54 smithi046 ceph-mon[36898]: pgmap v236: 97 pgs: 3 creating+activating, 7 creating+peering, 39 active+clean, 48 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:01:54.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3116091691' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:54 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:54 smithi046 ceph-mon[36898]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:01:54.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:54 smithi046 ceph-mon[36898]: Cluster is now healthy 2024-05-08T23:01:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.947+0000 7f6a94c00700 1 -- 172.21.15.46:0/3116091691 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v80) v1 ==== 228+0+0 (secure 0 0 0) 0x7f6a8809aea0 con 0x7f6a98100e80 2024-05-08T23:01:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.953+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 msgr2=0x7f6a7c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.953+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 0x7f6a7c0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009bf0 tx=0x7f6a84009260 comp rx=0 tx=0).stop 2024-05-08T23:01:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.953+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 msgr2=0x7f6a98074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.953+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005740 comp rx=0 tx=0).stop 2024-05-08T23:01:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.953+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 shutdown_connections 2024-05-08T23:01:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6a7c0a1120 0x7f6a7c0a35e0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009bf0 tx=0x7f6a84009260 comp rx=0 tx=0).stop 2024-05-08T23:01:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a98102990 0x7f6a980738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a98100e80 0x7f6a98074d00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 --2- 172.21.15.46:0/3116091691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a98101fd0 0x7f6a98073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 >> 172.21.15.46:0/3116091691 conn(0x7f6a980fc3d0 msgr2=0x7f6a98105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 shutdown_connections 2024-05-08T23:01:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:54.954+0000 7f6a9ec00700 1 -- 172.21.15.46:0/3116091691 wait complete. 2024-05-08T23:01:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-05-08T23:01:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:54 smithi086 ceph-mon[78610]: pgmap v236: 97 pgs: 3 creating+activating, 7 creating+peering, 39 active+clean, 48 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:01:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3116091691' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:54 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:01:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:54 smithi086 ceph-mon[78610]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:01:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:54 smithi086 ceph-mon[78610]: Cluster is now healthy 2024-05-08T23:01:54.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T23:01:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 -- 172.21.15.46:0/2547662179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 msgr2=0x7f99841012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 --2- 172.21.15.46:0/2547662179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f99841012a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9978009990 tx=0x7f9978009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 -- 172.21.15.46:0/2547662179 shutdown_connections 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 --2- 172.21.15.46:0/2547662179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 0x7f99841092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 --2- 172.21.15.46:0/2547662179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f99841012a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 --2- 172.21.15.46:0/2547662179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 0x7f9984102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 -- 172.21.15.46:0/2547662179 >> 172.21.15.46:0/2547662179 conn(0x7f99840fc3d0 msgr2=0x7f99840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 -- 172.21.15.46:0/2547662179 shutdown_connections 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.181+0000 7f998ba00700 1 -- 172.21.15.46:0/2547662179 wait complete. 2024-05-08T23:01:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 Processor -- start 2024-05-08T23:01:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 -- start start 2024-05-08T23:01:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 0x7f9984199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 0x7f99841a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841a0b00 con 0x7f9984102990 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841a0c40 con 0x7f9984101fd0 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f998ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841a0dd0 con 0x7f9984100e80 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.183+0000 7f9989000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9983e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 0x7f9984199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55154/0 (socket says 172.21.15.46:55154) 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 -- 172.21.15.46:0/731388808 learned_addr learned my addr 172.21.15.46:0/731388808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989a00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 0x7f99841a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 -- 172.21.15.46:0/731388808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 msgr2=0x7f9984199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 0x7f9984199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 -- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 msgr2=0x7f99841a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 0x7f99841a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 -- 172.21.15.46:0/731388808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009c40 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9989000700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f997800b680 tx=0x7f9978009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997801d070 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9978022470 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.184+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99780095f0 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.185+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978017650 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.185+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99841a1390 con 0x7f9984100e80 2024-05-08T23:01:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.185+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9984067d10 con 0x7f9984100e80 2024-05-08T23:01:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.188+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9978022870 con 0x7f9984100e80 2024-05-08T23:01:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.189+0000 7f9981600700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 0x7f996c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.189+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f99780cdfe0 con 0x7f9984100e80 2024-05-08T23:01:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.189+0000 7f9983e00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 0x7f996c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.190+0000 7f9983e00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 0x7f996c0a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9970009c60 tx=0x7f9970009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:55.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.191+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f99780975d0 con 0x7f9984100e80 2024-05-08T23:01:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.406+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f998410c8e0 con 0x7f9984100e80 2024-05-08T23:01:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:01:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: pgmap v238: 97 pgs: 3 creating+activating, 7 creating+peering, 82 active+clean, 5 unknown; 578 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-05-08T23:01:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:01:55.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:01:55.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/731388808' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: pgmap v238: 97 pgs: 3 creating+activating, 7 creating+peering, 82 active+clean, 5 unknown; 578 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/731388808' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:55 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.950+0000 7f9981600700 1 -- 172.21.15.46:0/731388808 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f997809abf0 con 0x7f9984100e80 2024-05-08T23:01:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 msgr2=0x7f996c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 0x7f996c0a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9970009c60 tx=0x7f9970009260 comp rx=0 tx=0).stop 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 msgr2=0x7f9984198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f997800b680 tx=0x7f9978009580 comp rx=0 tx=0).stop 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 shutdown_connections 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f996c0a11a0 0x7f996c0a3660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9984102990 0x7f99841a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9984100e80 0x7f9984198fd0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f997800b680 tx=0x7f9978009580 comp rx=0 tx=0).stop 2024-05-08T23:01:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 --2- 172.21.15.46:0/731388808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9984101fd0 0x7f9984199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.956+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 >> 172.21.15.46:0/731388808 conn(0x7f99840fc3d0 msgr2=0x7f99840731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.957+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 shutdown_connections 2024-05-08T23:01:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:55.957+0000 7f998ba00700 1 -- 172.21.15.46:0/731388808 wait complete. 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: pgmap v238: 97 pgs: 3 creating+activating, 7 creating+peering, 82 active+clean, 5 unknown; 578 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/731388808' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:55 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:01:55.964 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T23:01:55.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T23:01:56.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 -- 172.21.15.46:0/75050927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 msgr2=0x7f001c103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 --2- 172.21.15.46:0/75050927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c103cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f000c0098f0 tx=0x7f000c009c00 comp rx=0 tx=0).stop 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 -- 172.21.15.46:0/75050927 shutdown_connections 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 --2- 172.21.15.46:0/75050927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 0x7f001c108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 --2- 172.21.15.46:0/75050927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c103cd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 --2- 172.21.15.46:0/75050927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 0x7f001c106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 -- 172.21.15.46:0/75050927 >> 172.21.15.46:0/75050927 conn(0x7f001c0fb290 msgr2=0x7f001c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 -- 172.21.15.46:0/75050927 shutdown_connections 2024-05-08T23:01:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.190+0000 7f0022600700 1 -- 172.21.15.46:0/75050927 wait complete. 2024-05-08T23:01:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.191+0000 7f0022600700 1 Processor -- start 2024-05-08T23:01:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 -- start start 2024-05-08T23:01:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 0x7f001c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 0x7f001c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a0bb0 con 0x7f001c106b40 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a0cf0 con 0x7f001c1018e0 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f0022600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a0e80 con 0x7f001c104210 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f001b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f001b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:45056/0 (socket says 172.21.15.46:45056) 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f001b400700 1 -- 172.21.15.46:0/2884583765 learned_addr learned my addr 172.21.15.46:0/2884583765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f001aa00700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 0x7f001c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.192+0000 7f001be00700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 0x7f001c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 -- 172.21.15.46:0/2884583765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 msgr2=0x7f001c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 0x7f001c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 -- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 msgr2=0x7f001c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 0x7f001c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 -- 172.21.15.46:0/2884583765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009520 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f001b400700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f000c009810 tx=0x7f000c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c01c070 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c00f7d0 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c003970 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1a1100 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.193+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a14d0 con 0x7f001c1018e0 2024-05-08T23:01:56.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.194+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c067d10 con 0x7f001c1018e0 2024-05-08T23:01:56.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.196+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f000c020030 con 0x7f001c1018e0 2024-05-08T23:01:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.198+0000 7f0013e00700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 0x7f00000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.198+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f000c0cdc70 con 0x7f001c1018e0 2024-05-08T23:01:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.198+0000 7f001aa00700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 0x7f00000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.198+0000 7f001aa00700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 0x7f00000a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0008005fd0 tx=0x7f0008009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.200+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f000c097260 con 0x7f001c1018e0 2024-05-08T23:01:56.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:01:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:01:55.919+0000 7fb1a990b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:01:56.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.471+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f001c10c6b0 con 0x7f001c1018e0 2024-05-08T23:01:56.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.480+0000 7f0013e00700 1 -- 172.21.15.46:0/2884583765 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f000c09a880 con 0x7f001c1018e0 2024-05-08T23:01:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 msgr2=0x7f00000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 0x7f00000a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0008005fd0 tx=0x7f0008009310 comp rx=0 tx=0).stop 2024-05-08T23:01:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 msgr2=0x7f001c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f000c009810 tx=0x7f000c00b790 comp rx=0 tx=0).stop 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 shutdown_connections 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00000a1120 0x7f00000a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c106b40 0x7f001c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1018e0 0x7f001c199080 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 --2- 172.21.15.46:0/2884583765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c104210 0x7f001c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 >> 172.21.15.46:0/2884583765 conn(0x7f001c0fb290 msgr2=0x7f001c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:56.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 shutdown_connections 2024-05-08T23:01:56.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.482+0000 7f0022600700 1 -- 172.21.15.46:0/2884583765 wait complete. 2024-05-08T23:01:56.484 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-05-08T23:01:56.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 -- 172.21.15.46:0/2998888829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4101fd0 msgr2=0x7fb2d4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 --2- 172.21.15.46:0/2998888829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4102450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009990 tx=0x7fb2c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 -- 172.21.15.46:0/2998888829 shutdown_connections 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 --2- 172.21.15.46:0/2998888829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4102990 0x7fb2d41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 --2- 172.21.15.46:0/2998888829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4100e80 0x7fb2d41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 --2- 172.21.15.46:0/2998888829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4102450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 -- 172.21.15.46:0/2998888829 >> 172.21.15.46:0/2998888829 conn(0x7fb2d40fc410 msgr2=0x7fb2d40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:56.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 -- 172.21.15.46:0/2998888829 shutdown_connections 2024-05-08T23:01:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.699+0000 7fb2da800700 1 -- 172.21.15.46:0/2998888829 wait complete. 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 Processor -- start 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 -- start start 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4102990 0x7fb2d41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a0b40 con 0x7fb2d4100e80 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a0c80 con 0x7fb2d4102990 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.700+0000 7fb2da800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41a0e10 con 0x7fb2d4101fd0 2024-05-08T23:01:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56284/0 (socket says 172.21.15.46:56284) 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 -- 172.21.15.46:0/708667407 learned_addr learned my addr 172.21.15.46:0/708667407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3e00700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4102990 0x7fb2d41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d2a00700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 -- 172.21.15.46:0/708667407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4101fd0 msgr2=0x7fb2d4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 -- 172.21.15.46:0/708667407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4102990 msgr2=0x7fb2d41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4102990 0x7fb2d41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 -- 172.21.15.46:0/708667407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00095f0 con 0x7fb2d4100e80 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2d3400700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e9a0 tx=0x7fb2c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.701+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4009790 con 0x7fb2d4100e80 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.702+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c4010430 con 0x7fb2d4100e80 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.702+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d41a10f0 con 0x7fb2d4100e80 2024-05-08T23:01:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.702+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4018680 con 0x7fb2d4100e80 2024-05-08T23:01:56.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.702+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a1650 con 0x7fb2d4100e80 2024-05-08T23:01:56.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.703+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d4067d10 con 0x7fb2d4100e80 2024-05-08T23:01:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.705+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb2c40105a0 con 0x7fb2d4100e80 2024-05-08T23:01:56.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.706+0000 7fb2cbe00700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 0x7fb2b80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:56.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.706+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2c4014070 con 0x7fb2d4100e80 2024-05-08T23:01:56.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.706+0000 7fb2d2a00700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 0x7fb2b80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:56.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.707+0000 7fb2d2a00700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 0x7fb2b80a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000b2e0 tx=0x7fb2c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.709+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb2c4096430 con 0x7fb2d4100e80 2024-05-08T23:01:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.977+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d410c980 con 0x7fb2d4100e80 2024-05-08T23:01:56.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.978+0000 7fb2cbe00700 1 -- 172.21.15.46:0/708667407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10792 (secure 0 0 0) 0x7fb2c4026020 con 0x7fb2d4100e80 2024-05-08T23:01:56.978 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:56.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:01:56.474677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 msgr2=0x7fb2b80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 0x7fb2b80a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000b2e0 tx=0x7fb2c0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 msgr2=0x7fb2d4199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e9a0 tx=0x7fb2c400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 shutdown_connections 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d4102990 0x7fb2d41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb2b80a11a0 0x7fb2b80a3660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d4100e80 0x7fb2d4199010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 --2- 172.21.15.46:0/708667407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d4101fd0 0x7fb2d4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 >> 172.21.15.46:0/708667407 conn(0x7fb2d40fc410 msgr2=0x7fb2d40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 shutdown_connections 2024-05-08T23:01:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:56.981+0000 7fb2da800700 1 -- 172.21.15.46:0/708667407 wait complete. 2024-05-08T23:01:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-05-08T23:01:57.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 -- 172.21.15.46:0/517460748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc101900 msgr2=0x7fd9fc103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 --2- 172.21.15.46:0/517460748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc101900 0x7fd9fc103cf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009990 tx=0x7fd9f0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 -- 172.21.15.46:0/517460748 shutdown_connections 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 --2- 172.21.15.46:0/517460748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 0x7fd9fc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 --2- 172.21.15.46:0/517460748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc101900 0x7fd9fc103cf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 --2- 172.21.15.46:0/517460748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc104230 0x7fd9fc106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 -- 172.21.15.46:0/517460748 >> 172.21.15.46:0/517460748 conn(0x7fd9fc0fb310 msgr2=0x7fd9fc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 -- 172.21.15.46:0/517460748 shutdown_connections 2024-05-08T23:01:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.201+0000 7fda03c00700 1 -- 172.21.15.46:0/517460748 wait complete. 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.202+0000 7fda03c00700 1 Processor -- start 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.202+0000 7fda03c00700 1 -- start start 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc101900 0x7fd9fc198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 0x7fd9fc1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1a0ac0 con 0x7fd9fc104230 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1a0c00 con 0x7fd9fc101900 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda03c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1a0d90 con 0x7fd9fc106b60 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56306/0 (socket says 172.21.15.46:56306) 2024-05-08T23:01:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 -- 172.21.15.46:0/3516178896 learned_addr learned my addr 172.21.15.46:0/3516178896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda01200700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc101900 0x7fd9fc198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fda01c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 0x7fd9fc1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 -- 172.21.15.46:0/3516178896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 msgr2=0x7fd9fc1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 0x7fd9fc1a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 -- 172.21.15.46:0/3516178896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc101900 msgr2=0x7fd9fc198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc101900 0x7fd9fc198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.203+0000 7fd9fbe00700 1 -- 172.21.15.46:0/3516178896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f00095f0 con 0x7fd9fc104230 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fd9fbe00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800d950 tx=0x7fd9e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800f890 con 0x7fd9fc104230 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e8010460 con 0x7fd9fc104230 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800e610 con 0x7fd9fc104230 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc1a1070 con 0x7fd9fc104230 2024-05-08T23:01:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.204+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc1a1580 con 0x7fd9fc104230 2024-05-08T23:01:57.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.205+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc108730 con 0x7fd9fc104230 2024-05-08T23:01:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.207+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd9e80108d0 con 0x7fd9fc104230 2024-05-08T23:01:57.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.209+0000 7fd9f9600700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 0x7fd9e40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.209+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9e80cc8b0 con 0x7fd9fc104230 2024-05-08T23:01:57.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.210+0000 7fda01200700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 0x7fd9e40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.210+0000 7fda01200700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 0x7fd9e40a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000b5c0 tx=0x7fd9f0003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2884583765' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:01:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:56 smithi086 ceph-mon[78610]: pgmap v242: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 8 op/s 2024-05-08T23:01:57.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.214+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd9e8095df0 con 0x7fd9fc104230 2024-05-08T23:01:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:01:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:01:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2884583765' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:01:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:01:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:56 smithi053 ceph-mon[77587]: pgmap v242: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 8 op/s 2024-05-08T23:01:57.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:01:57.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:01:57.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2884583765' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:01:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:01:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:01:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:01:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:56 smithi046 ceph-mon[36898]: pgmap v242: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 8 op/s 2024-05-08T23:01:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.434+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9fc067d10 con 0x7fd9fc104230 2024-05-08T23:01:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.436+0000 7fd9f9600700 1 -- 172.21.15.46:0/3516178896 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7fd9e8016090 con 0x7fd9fc104230 2024-05-08T23:01:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:57.436 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 msgr2=0x7fd9e40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 0x7fd9e40a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000b5c0 tx=0x7fd9f0003790 comp rx=0 tx=0).stop 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 msgr2=0x7fd9fc1994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800d950 tx=0x7fd9e800dc60 comp rx=0 tx=0).stop 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 shutdown_connections 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc106b60 0x7fd9fc1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9fc101900 0x7fd9fc198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd9e40a11a0 0x7fd9e40a3660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 --2- 172.21.15.46:0/3516178896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9fc104230 0x7fd9fc1994d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 >> 172.21.15.46:0/3516178896 conn(0x7fd9fc0fb310 msgr2=0x7fd9fc1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 shutdown_connections 2024-05-08T23:01:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.438+0000 7fda03c00700 1 -- 172.21.15.46:0/3516178896 wait complete. 2024-05-08T23:01:57.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:01:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 -- 172.21.15.46:0/543060101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84106b60 msgr2=0x7f6e84108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 --2- 172.21.15.46:0/543060101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84106b60 0x7f6e84108fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00b0a0 tx=0x7f6e7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 -- 172.21.15.46:0/543060101 shutdown_connections 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 --2- 172.21.15.46:0/543060101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84106b60 0x7f6e84108fd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 --2- 172.21.15.46:0/543060101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 --2- 172.21.15.46:0/543060101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84104230 0x7f6e84106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 -- 172.21.15.46:0/543060101 >> 172.21.15.46:0/543060101 conn(0x7f6e840fb310 msgr2=0x7f6e840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 -- 172.21.15.46:0/543060101 shutdown_connections 2024-05-08T23:01:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.656+0000 7f6e8c400700 1 -- 172.21.15.46:0/543060101 wait complete. 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.657+0000 7f6e8c400700 1 Processor -- start 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.657+0000 7f6e8c400700 1 -- start start 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 0x7f6e840ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84100590 con 0x7f6e84104230 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84100700 con 0x7f6e84101900 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841a0f40 con 0x7f6e84106b60 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56322/0 (socket says 172.21.15.46:56322) 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 -- 172.21.15.46:0/115406544 learned_addr learned my addr 172.21.15.46:0/115406544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89a00700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8a400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 0x7f6e840ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 -- 172.21.15.46:0/115406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 msgr2=0x7f6e840ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 0x7f6e840ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 -- 172.21.15.46:0/115406544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 msgr2=0x7f6e84101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89000700 1 -- 172.21.15.46:0/115406544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009d00 con 0x7f6e84104230 2024-05-08T23:01:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e8a400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 0x7f6e840ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.658+0000 7f6e89a00700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e89000700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000e9c0 tx=0x7f6e7000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700097b0 con 0x7f6e84104230 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e70004830 con 0x7f6e84104230 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e841a11d0 con 0x7f6e84104230 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70005620 con 0x7f6e84104230 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.659+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841a16e0 con 0x7f6e84104230 2024-05-08T23:01:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.660+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e840fce90 con 0x7f6e84104230 2024-05-08T23:01:57.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.662+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6e700049a0 con 0x7f6e84104230 2024-05-08T23:01:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.663+0000 7f6e82000700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 0x7f6e6c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.663+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6e70014070 con 0x7f6e84104230 2024-05-08T23:01:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.663+0000 7f6e89a00700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 0x7f6e6c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.664+0000 7f6e89a00700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 0x7f6e6c0a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009cb0 tx=0x7f6e78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:57.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.667+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e70096430 con 0x7f6e84104230 2024-05-08T23:01:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.889+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e84067d10 con 0x7f6e84104230 2024-05-08T23:01:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.890+0000 7f6e82000700 1 -- 172.21.15.46:0/115406544 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7f6e70099a50 con 0x7f6e84104230 2024-05-08T23:01:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:57.891 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 msgr2=0x7f6e6c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 0x7f6e6c0a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009cb0 tx=0x7f6e78009260 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 msgr2=0x7f6e840ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000e9c0 tx=0x7f6e7000ed80 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 shutdown_connections 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e84106b60 0x7f6e840ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e84101900 0x7f6e84101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6e6c0a11a0 0x7f6e6c0a3660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 --2- 172.21.15.46:0/115406544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e84104230 0x7f6e840ff8b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 >> 172.21.15.46:0/115406544 conn(0x7f6e840fb310 msgr2=0x7f6e84102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 shutdown_connections 2024-05-08T23:01:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:57.893+0000 7f6e8c400700 1 -- 172.21.15.46:0/115406544 wait complete. 2024-05-08T23:01:57.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 -- 172.21.15.46:0/1043938694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078101fd0 msgr2=0x7f5078102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 --2- 172.21.15.46:0/1043938694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078101fd0 0x7f5078102450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5064009990 tx=0x7f5064009ca0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 -- 172.21.15.46:0/1043938694 shutdown_connections 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 --2- 172.21.15.46:0/1043938694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078102990 0x7f50781092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 --2- 172.21.15.46:0/1043938694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 0x7f50781012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 --2- 172.21.15.46:0/1043938694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078101fd0 0x7f5078102450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.126+0000 7f507ce00700 1 -- 172.21.15.46:0/1043938694 >> 172.21.15.46:0/1043938694 conn(0x7f50780fc410 msgr2=0x7f50780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.127+0000 7f507ce00700 1 -- 172.21.15.46:0/1043938694 shutdown_connections 2024-05-08T23:01:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.127+0000 7f507ce00700 1 -- 172.21.15.46:0/1043938694 wait complete. 2024-05-08T23:01:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.127+0000 7f507ce00700 1 Processor -- start 2024-05-08T23:01:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 -- start start 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 0x7f5078196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 0x7f507819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507819e8e0 con 0x7f5078101fd0 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507819ea50 con 0x7f5078100e80 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f507ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507819ec10 con 0x7f5078102990 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f5075400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f5075400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56336/0 (socket says 172.21.15.46:56336) 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f5075400700 1 -- 172.21.15.46:0/3655823561 learned_addr learned my addr 172.21.15.46:0/3655823561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f5076800700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 0x7f507819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.128+0000 7f5075e00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 0x7f5078196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 -- 172.21.15.46:0/3655823561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 msgr2=0x7f5078196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 0x7f5078196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 -- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 msgr2=0x7f5078197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 -- 172.21.15.46:0/3655823561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009c40 con 0x7f5078102990 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5075400700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f5076800700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 0x7f507819e310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f507000efb0 tx=0x7f507000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070003a00 con 0x7f5078102990 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f507000f040 con 0x7f5078102990 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640095f0 con 0x7f5078102990 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070004540 con 0x7f5078102990 2024-05-08T23:01:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.129+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507819f1c0 con 0x7f5078102990 2024-05-08T23:01:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.130+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50781082f0 con 0x7f5078102990 2024-05-08T23:01:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.132+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5070003bf0 con 0x7f5078102990 2024-05-08T23:01:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.134+0000 7f506ea00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 0x7f505c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.134+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f50700cc6c0 con 0x7f5078102990 2024-05-08T23:01:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.134+0000 7f5075e00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 0x7f505c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.134+0000 7f5075e00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 0x7f505c0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5078198270 tx=0x7f5068009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.136+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5070095cf0 con 0x7f5078102990 2024-05-08T23:01:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/708667407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3516178896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/115406544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/708667407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3516178896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/115406544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/708667407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3516178896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/115406544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:58.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.408+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5078067d10 con 0x7f5078102990 2024-05-08T23:01:58.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.409+0000 7f506ea00700 1 -- 172.21.15.46:0/3655823561 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10792 (secure 0 0 0) 0x7f5070099310 con 0x7f5078102990 2024-05-08T23:01:58.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:58.409 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24490,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24571,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:01:56.474677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T23:01:58.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 msgr2=0x7f505c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 0x7f505c0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5078198270 tx=0x7f5068009260 comp rx=0 tx=0).stop 2024-05-08T23:01:58.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 msgr2=0x7f507819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 0x7f507819e310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f507000efb0 tx=0x7f507000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 shutdown_connections 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5078102990 0x7f507819e310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5078100e80 0x7f5078196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f505c0a1120 0x7f505c0a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 --2- 172.21.15.46:0/3655823561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5078101fd0 0x7f5078197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 >> 172.21.15.46:0/3655823561 conn(0x7f50780fc410 msgr2=0x7f5078105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 shutdown_connections 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.411+0000 7f507ce00700 1 -- 172.21.15.46:0/3655823561 wait complete. 2024-05-08T23:01:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-05-08T23:01:58.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:01:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 -- 172.21.15.46:0/3228428019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c100cb0 msgr2=0x7f035c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 --2- 172.21.15.46:0/3228428019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c100cb0 0x7f035c1010d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03500099c0 tx=0x7f0350009cd0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 -- 172.21.15.46:0/3228428019 shutdown_connections 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 --2- 172.21.15.46:0/3228428019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c102840 0x7f035c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 --2- 172.21.15.46:0/3228428019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c100cb0 0x7f035c1010d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 --2- 172.21.15.46:0/3228428019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 -- 172.21.15.46:0/3228428019 >> 172.21.15.46:0/3228428019 conn(0x7f035c0fc220 msgr2=0x7f035c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 -- 172.21.15.46:0/3228428019 shutdown_connections 2024-05-08T23:01:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.628+0000 7f0362200700 1 -- 172.21.15.46:0/3228428019 wait complete. 2024-05-08T23:01:58.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.629+0000 7f0362200700 1 Processor -- start 2024-05-08T23:01:58.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.629+0000 7f0362200700 1 -- start start 2024-05-08T23:01:58.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 0x7f035c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c102840 0x7f035c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c074070 con 0x7f035c102840 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0741e0 con 0x7f035c100cb0 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0362200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1a0ce0 con 0x7f035c101ea0 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55260/0 (socket says 172.21.15.46:55260) 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0360e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 0x7f035c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 -- 172.21.15.46:0/934424425 learned_addr learned my addr 172.21.15.46:0/934424425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f0361800700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c102840 0x7f035c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 -- 172.21.15.46:0/934424425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 msgr2=0x7f035c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 0x7f035c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 -- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c102840 msgr2=0x7f035c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c102840 0x7f035c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.630+0000 7f035be00700 1 -- 172.21.15.46:0/934424425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500095f0 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0360e00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 0x7f035c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f035be00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f034800e9a0 tx=0x7f034800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348009790 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0348010430 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c1a0f10 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348018680 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.631+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c1a1330 con 0x7f035c101ea0 2024-05-08T23:01:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.632+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c04ed40 con 0x7f035c101ea0 2024-05-08T23:01:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.634+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f03480105a0 con 0x7f035c101ea0 2024-05-08T23:01:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.635+0000 7f0359600700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 0x7f03440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.636+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0348014070 con 0x7f035c101ea0 2024-05-08T23:01:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.636+0000 7f0360e00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 0x7f03440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.636+0000 7f0360e00700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 0x7f03440a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f035001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.638+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f03480962c0 con 0x7f035c101ea0 2024-05-08T23:01:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.858+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f035c108130 con 0x7f035c101ea0 2024-05-08T23:01:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.860+0000 7f0359600700 1 -- 172.21.15.46:0/934424425 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7f0348026090 con 0x7f035c101ea0 2024-05-08T23:01:58.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:01:58.860 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 msgr2=0x7f03440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 0x7f03440a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f035001a040 comp rx=0 tx=0).stop 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 msgr2=0x7f035c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f034800e9a0 tx=0x7f034800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 shutdown_connections 2024-05-08T23:01:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f03440a1120 0x7f03440a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c102840 0x7f035c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035c100cb0 0x7f035c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 --2- 172.21.15.46:0/934424425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035c101ea0 0x7f035c073390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 >> 172.21.15.46:0/934424425 conn(0x7f035c0fc220 msgr2=0x7f035c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 shutdown_connections 2024-05-08T23:01:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:58.863+0000 7f0362200700 1 -- 172.21.15.46:0/934424425 wait complete. 2024-05-08T23:01:58.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:01:51.460837+0000', 'modified': '2024-05-08T23:01:56.474677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24587}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/3374696949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3374696949}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3374696949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-08T23:01:58.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:01:58.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 -- 172.21.15.46:0/1111871199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48102990 msgr2=0x7fba481092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 --2- 172.21.15.46:0/1111871199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48102990 0x7fba481092b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b0a0 tx=0x7fba4000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 -- 172.21.15.46:0/1111871199 shutdown_connections 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 --2- 172.21.15.46:0/1111871199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48102990 0x7fba481092b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 --2- 172.21.15.46:0/1111871199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48100e80 0x7fba481012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 --2- 172.21.15.46:0/1111871199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 0x7fba48102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 -- 172.21.15.46:0/1111871199 >> 172.21.15.46:0/1111871199 conn(0x7fba480fc3b0 msgr2=0x7fba480fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 -- 172.21.15.46:0/1111871199 shutdown_connections 2024-05-08T23:01:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.086+0000 7fba4fe00700 1 -- 172.21.15.46:0/1111871199 wait complete. 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 Processor -- start 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 -- start start 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 0x7fba48199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48102990 0x7fba481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481a0c60 con 0x7fba48100e80 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481a0da0 con 0x7fba48102990 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.087+0000 7fba4fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481a0f30 con 0x7fba48101fd0 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56382/0 (socket says 172.21.15.46:56382) 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 -- 172.21.15.46:0/7987967 learned_addr learned my addr 172.21.15.46:0/7987967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4ca00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 0x7fba48199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4de00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48102990 0x7fba481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 -- 172.21.15.46:0/7987967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 msgr2=0x7fba48199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 0x7fba48199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 -- 172.21.15.46:0/7987967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48102990 msgr2=0x7fba481a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48102990 0x7fba481a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 -- 172.21.15.46:0/7987967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40009d00 con 0x7fba48100e80 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba4d400700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9a0 tx=0x7fba3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.088+0000 7fba46000700 1 -- 172.21.15.46:0/7987967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c009790 con 0x7fba48100e80 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.089+0000 7fba46000700 1 -- 172.21.15.46:0/7987967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3c010430 con 0x7fba48100e80 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.089+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba481a1210 con 0x7fba48100e80 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.089+0000 7fba46000700 1 -- 172.21.15.46:0/7987967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c018680 con 0x7fba48100e80 2024-05-08T23:01:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.089+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481a1720 con 0x7fba48100e80 2024-05-08T23:01:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.092+0000 7fba46000700 1 -- 172.21.15.46:0/7987967 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fba3c0105a0 con 0x7fba48100e80 2024-05-08T23:01:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.093+0000 7fba46000700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 0x7fba300abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.093+0000 7fba63819b80 1 Processor -- start 2024-05-08T23:01:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.093+0000 7fba46000700 1 -- 172.21.15.46:0/7987967 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba3c014070 con 0x7fba48100e80 2024-05-08T23:01:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.093+0000 7fba63819b80 1 -- start start 2024-05-08T23:01:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.093+0000 7fba4ca00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 0x7fba300abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x557900801910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579008022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557900758100 con 0x557900802810 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557900755ac0 con 0x5579008014f0 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba63819b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579007513b0 con 0x557900801e50 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56388/0 (socket says 172.21.15.46:56388) 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4de00700 1 -- 172.21.15.46:0/2408382025 learned_addr learned my addr 172.21.15.46:0/2408382025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4ca00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 0x7fba300abdf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba34009cb0 tx=0x7fba34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4ca00700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579008022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.094+0000 7fba4d400700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x557900801910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 -- 172.21.15.46:0/2408382025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 msgr2=0x557900801910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x557900801910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 -- 172.21.15.46:0/2408382025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 msgr2=0x557900802c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 -- 172.21.15.46:0/2408382025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579005d2360 con 0x557900801e50 2024-05-08T23:01:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4de00700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba4ca00700 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579008022d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba3400f3f0 tx=0x7fba34014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba3cd456f659d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba3a600700 1 -- 172.21.15.46:0/2408382025 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400fd90 con 0x557900801e50 2024-05-08T23:01:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba3a600700 1 -- 172.21.15.46:0/2408382025 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba34021070 con 0x557900801e50 2024-05-08T23:01:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.095+0000 7fba3a600700 1 -- 172.21.15.46:0/2408382025 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34026470 con 0x557900801e50 2024-05-08T23:01:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 -- 172.21.15.46:0/2408382025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 msgr2=0x5579008022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579008022d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba3400f3f0 tx=0x7fba34014000 comp rx=0 tx=0).stop 2024-05-08T23:01:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 -- 172.21.15.46:0/2408382025 shutdown_connections 2024-05-08T23:01:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900802c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x557900801910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 --2- 172.21.15.46:0/2408382025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579008022d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 -- 172.21.15.46:0/2408382025 >> 172.21.15.46:0/2408382025 conn(0x55790075f1c0 msgr2=0x5579007fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 -- 172.21.15.46:0/2408382025 shutdown_connections 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.096+0000 7fba63819b80 1 -- 172.21.15.46:0/2408382025 wait complete. 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 -- start start 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 Processor -- start 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x55790080ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x55790080b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579006ed510 con 0x557900801e50 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579006ed1b0 con 0x5579008014f0 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579005d4650 con 0x557900802810 2024-05-08T23:01:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.097+0000 7fba63819b80 10 client.0 _ll_register_callbacks cb 0x5579006f6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x55790080ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x55790080b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55302/0 (socket says 172.21.15.46:55302) 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4de00700 1 -- 172.21.15.46:0/982776408 learned_addr learned my addr 172.21.15.46:0/982776408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 -- 172.21.15.46:0/982776408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 msgr2=0x557900812360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 -- 172.21.15.46:0/982776408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 msgr2=0x55790080b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x55790080b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 -- 172.21.15.46:0/982776408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba48109be0 con 0x5579008014f0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4de00700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba4d400700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x55790080ade0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ca5a0 tx=0x7fba3c09a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f45b44e556c0493 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba2ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.098+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c09dd20 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba3c0a2370 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579006aa220 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba3c0b8170 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 10 client.24695 fetch_fsmap learned FSMap version 17 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7fba34017050 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3c063080 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fba3c095e50 con 0x5579008014f0 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 10 client.24695 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 10 client.24695 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 20 client.24695 populate_metadata read hostname 'smithi046' 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900815f50 0x55790081b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 4 client.24695 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:01:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.099+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579007d07c0 con 0x557900815f50 2024-05-08T23:01:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.100+0000 7fba4ca00700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900815f50 0x55790081b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.100+0000 7fba4ca00700 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900815f50 0x55790081b230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0948cbdaf1da91 server_cookie=542ca3a52ca619a2 in_seq=0 out_seq=0 2024-05-08T23:01:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.100+0000 7fba2ea00700 10 client.24695 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba2ea00700 1 -- 172.21.15.46:0/982776408 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579007d07c0 con 0x557900815f50 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba2ea00700 10 client.24695 handle_command_reply: tid=0 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 client.24695 shutdown 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 msgr2=0x55790080ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x55790080ade0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ca5a0 tx=0x7fba3c09a430 comp rx=0 tx=0).stop 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 shutdown_connections 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x557900812360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x55790080ade0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900815f50 0x55790081b230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 --2- 172.21.15.46:0/982776408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x55790080b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.101+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 >> 172.21.15.46:0/982776408 conn(0x55790075f1c0 msgr2=0x5579007fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.102+0000 7fba2ea00700 0 client.24695 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.102+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 shutdown_connections 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.102+0000 7fba63819b80 1 -- 172.21.15.46:0/982776408 wait complete. 2024-05-08T23:01:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.102+0000 7fba63819b80 20 client.24695 trim_cache size 0 max 16384 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.120+0000 7fba63819b80 1 Processor -- start 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 -- start start 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579008018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x557900812a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579006f05a0 con 0x557900801e50 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579007f8d70 con 0x5579008014f0 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba63819b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579008140f0 con 0x557900802810 2024-05-08T23:01:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba4ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x557900812a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba4de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba4d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579008018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba4de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55308/0 (socket says 172.21.15.46:55308) 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.121+0000 7fba4de00700 1 -- 172.21.15.46:0/332095943 learned_addr learned my addr 172.21.15.46:0/332095943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 -- 172.21.15.46:0/332095943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 msgr2=0x55790080a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 -- 172.21.15.46:0/332095943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 msgr2=0x557900812a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x557900812a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 -- 172.21.15.46:0/332095943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55790057eb10 con 0x5579008014f0 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4ca00700 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x557900812a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4de00700 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:01:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba4d400700 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579008018d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba3c099bf0 tx=0x7fba3c0984a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78450e2b9a381eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba3b000700 1 -- 172.21.15.46:0/332095943 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c099090 con 0x5579008014f0 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba3b000700 1 -- 172.21.15.46:0/332095943 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579006aa220 con 0x5579008014f0 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.122+0000 7fba3b000700 1 -- 172.21.15.46:0/332095943 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0b8300 con 0x5579008014f0 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- 172.21.15.46:0/332095943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 msgr2=0x5579008018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579008018d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba3c099bf0 tx=0x7fba3c0984a0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- 172.21.15.46:0/332095943 shutdown_connections 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900802810 0x55790080a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579008018d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 --2- 172.21.15.46:0/332095943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900801e50 0x557900812a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- 172.21.15.46:0/332095943 >> 172.21.15.46:0/332095943 conn(0x55790075f1c0 msgr2=0x557900802bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- 172.21.15.46:0/332095943 shutdown_connections 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- 172.21.15.46:0/332095943 wait complete. 2024-05-08T23:01:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 -- start start 2024-05-08T23:01:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.123+0000 7fba63819b80 1 Processor -- start 2024-05-08T23:01:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579007facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579007fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579007fba20 con 0x557900802810 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579007fbb90 con 0x5579008014f0 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579008118d0 con 0x557900801e50 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba63819b80 10 client.0 _ll_register_callbacks cb 0x5579006f6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56422/0 (socket says 172.21.15.46:56422) 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 -- 172.21.15.46:0/2819633318 learned_addr learned my addr 172.21.15.46:0/2819633318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4ca00700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579007fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4d400700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579007facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 -- 172.21.15.46:0/2819633318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 msgr2=0x5579007fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579007fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 -- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 msgr2=0x5579007facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:01:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579007facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.124+0000 7fba4de00700 1 -- 172.21.15.46:0/2819633318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579005d2360 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba4de00700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b070 tx=0x7fba4000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375397edc4e970d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba2ea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40023070 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba34021070 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba40003c10 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba4000e040 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba63819b80 10 client.14709 fetch_fsmap learned FSMap version 17 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.125+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7fba40007d50 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba34021070 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fba4000b0d0 con 0x557900802810 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:01:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 20 client.14709 populate_metadata read hostname 'smithi046' 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900870c80 0x55790086d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 4 client.14709 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557900749e70 con 0x557900870c80 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.126+0000 7fba4d400700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900870c80 0x55790086d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba4d400700 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900870c80 0x55790086d610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb6fcbd2e3a0b172 server_cookie=b3c630c3e2412a40 in_seq=0 out_seq=0 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba2ea00700 10 client.14709 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba2ea00700 1 -- 172.21.15.46:0/2819633318 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x557900749e70 con 0x557900870c80 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba2ea00700 10 client.14709 handle_command_reply: tid=0 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 client.14709 shutdown 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 msgr2=0x557900811310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b070 tx=0x7fba4000bdc0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 shutdown_connections 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557900870c80 0x55790086d610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557900802810 0x557900811310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579008014f0 0x5579007facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 --2- 172.21.15.46:0/2819633318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557900801e50 0x5579007fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 >> 172.21.15.46:0/2819633318 conn(0x55790075f1c0 msgr2=0x557900802bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba2ea00700 0 client.14709 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 shutdown_connections 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 1 -- 172.21.15.46:0/2819633318 wait complete. 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.127+0000 7fba63819b80 20 client.14709 trim_cache size 0 max 16384 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:01:59.133 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24587, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 7.6637966139999998, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 82, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 14.114754434 2024-05-08T23:01:59.134 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 msgr2=0x7fba300abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 0x7fba300abdf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba34009cb0 tx=0x7fba34009260 comp rx=0 tx=0).stop 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 msgr2=0x7fba48199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9a0 tx=0x7fba3c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 shutdown_connections 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba48102990 0x7fba481a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fba300a9930 0x7fba300abdf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba48100e80 0x7fba48199130 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 --2- 172.21.15.46:0/7987967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba48101fd0 0x7fba48199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:01:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 >> 172.21.15.46:0/7987967 conn(0x7fba480fc3b0 msgr2=0x7fba480731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:01:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 shutdown_connections 2024-05-08T23:01:59.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:01:59.128+0000 7fba4fe00700 1 -- 172.21.15.46:0/7987967 wait complete. 2024-05-08T23:01:59.144 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3655823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.145 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/934424425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.145 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:01:58 smithi046 ceph-mon[36898]: pgmap v243: 129 pgs: 8 creating+peering, 119 active+clean, 2 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 7 op/s 2024-05-08T23:01:59.147 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:01:59.147 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:01:59.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T23:01:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3655823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/934424425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:01:58 smithi086 ceph-mon[78610]: pgmap v243: 129 pgs: 8 creating+peering, 119 active+clean, 2 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 7 op/s 2024-05-08T23:01:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3655823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/934424425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:01:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:01:58 smithi053 ceph-mon[77587]: pgmap v243: 129 pgs: 8 creating+peering, 119 active+clean, 2 unknown; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 7 op/s 2024-05-08T23:01:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 -- 172.21.15.46:0/1859600188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 msgr2=0x7ff79c10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1859600188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c10de70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b0a0 tx=0x7ff79400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 -- 172.21.15.46:0/1859600188 shutdown_connections 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1859600188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c10de70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1859600188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c107030 0x7ff79c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1859600188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c100ff0 0x7ff79c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 -- 172.21.15.46:0/1859600188 >> 172.21.15.46:0/1859600188 conn(0x7ff79c078190 msgr2=0x7ff79c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 -- 172.21.15.46:0/1859600188 shutdown_connections 2024-05-08T23:02:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.850+0000 7ff7a3400700 1 -- 172.21.15.46:0/1859600188 wait complete. 2024-05-08T23:02:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 Processor -- start 2024-05-08T23:02:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 -- start start 2024-05-08T23:02:00.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c107030 0x7ff79c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19c420 con 0x7ff79c101990 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19c590 con 0x7ff79c100ff0 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.851+0000 7ff7a3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19c750 con 0x7ff79c107030 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff7a0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff7a0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:45218/0 (socket says 172.21.15.46:45218) 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff7a0a00700 1 -- 172.21.15.46:0/1447516582 learned_addr learned my addr 172.21.15.46:0/1447516582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff7a1400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c107030 0x7ff79c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 -- 172.21.15.46:0/1447516582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c107030 msgr2=0x7ff79c19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c107030 0x7ff79c19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 -- 172.21.15.46:0/1447516582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 msgr2=0x7ff79c197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 -- 172.21.15.46:0/1447516582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794009d00 con 0x7ff79c101990 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff7a0a00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.852+0000 7ff79be00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c197a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9f0 tx=0x7ff78800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.853+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800c9c0 con 0x7ff79c101990 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.853+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff788010430 con 0x7ff79c101990 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.853+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788018680 con 0x7ff79c101990 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.853+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c19ca30 con 0x7ff79c101990 2024-05-08T23:02:00.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.853+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c1a4290 con 0x7ff79c101990 2024-05-08T23:02:00.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.854+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c109550 con 0x7ff79c101990 2024-05-08T23:02:00.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.856+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff788010860 con 0x7ff79c101990 2024-05-08T23:02:00.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.857+0000 7ff799600700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 0x7ff7840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:00.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.857+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff788014070 con 0x7ff79c101990 2024-05-08T23:02:00.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.857+0000 7ff7a0a00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 0x7ff7840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:00.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.858+0000 7ff7a0a00700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 0x7ff7840a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff790005950 tx=0x7ff79000b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:00.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:00.862+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff788052d60 con 0x7ff79c101990 2024-05-08T23:02:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.122+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff79c1026e0 con 0x7ff79c101990 2024-05-08T23:02:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.123+0000 7ff799600700 1 -- 172.21.15.46:0/1447516582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff788096140 con 0x7ff79c101990 2024-05-08T23:02:01.123 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 msgr2=0x7ff7840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 0x7ff7840a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff790005950 tx=0x7ff79000b500 comp rx=0 tx=0).stop 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 msgr2=0x7ff79c197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c197a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9f0 tx=0x7ff78800ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 shutdown_connections 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff79c107030 0x7ff79c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff79c100ff0 0x7ff79c197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7840a11a0 0x7ff7840a3660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 --2- 172.21.15.46:0/1447516582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff79c101990 0x7ff79c197a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 >> 172.21.15.46:0/1447516582 conn(0x7ff79c078190 msgr2=0x7ff79c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 shutdown_connections 2024-05-08T23:02:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:01.125+0000 7ff7a3400700 1 -- 172.21.15.46:0/1447516582 wait complete. 2024-05-08T23:02:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:00 smithi053 ceph-mon[77587]: pgmap v244: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:02:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:00 smithi046 ceph-mon[36898]: pgmap v244: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:02:01.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:00 smithi086 ceph-mon[78610]: pgmap v244: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:02:01.519 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:02:01.519 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:02:01.519 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:02:01.520 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T23:02:01.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:01.520 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-05-08T23:02:01.564 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41697sec preferred_lft 41697sec 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:02:01.565 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-05-08T23:02:01.566 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:02:01.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:01.566 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:02:01.566 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:02:01.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:02:01.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-05-08T23:02:01.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-05-08T23:02:01.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:02:01.567 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:02:02.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.023 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:02:02.091 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-05-08T23:02:02.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.095 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-05-08T23:02:02.158 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-05-08T23:02:02.158 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:02:02.158 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:02:02.158 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:02:02.159 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:02:02.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.159 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:02:02.160 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:02:02.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:02:02.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:02:02.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:02:02.160 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:02:02.258 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:01 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1447516582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:02:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:01 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1447516582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:02:02.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:01 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1447516582' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:02:02.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.855 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-08T23:02:02.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.898 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-05-08T23:02:02.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:02.980 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:02:02.980 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:02:02.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T23:02:02.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T23:02:02.980 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:02:03.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:03 smithi046 ceph-mon[36898]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-08T23:02:03.164 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T23:02:03.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:03.165 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:02:03.165 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:02:03.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T23:02:03.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:02:03.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T23:02:03.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T23:02:03.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T23:02:03.166 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:02:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:03 smithi053 ceph-mon[77587]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-08T23:02:03.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:03.442 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:02:03.442 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:02:03.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-05-08T23:02:03.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:02:03.442 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:02:03.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:03 smithi086 ceph-mon[78610]: pgmap v245: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 1 op/s 2024-05-08T23:02:03.611 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T23:02:03.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:02:03.611 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:03.653 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T23:02:03.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:02:03.654 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:03.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-05-08T23:02:03.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:02:03.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:02:03.907 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:02:03.909 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:02:03.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:03.910 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-05-08T23:02:03.952 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:02:03.953 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T23:02:03.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-05-08T23:02:04.067 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:02:04.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:02:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:02:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:02:04.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:04.139 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-05-08T23:02:04.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-05-08T23:02:04.205+0000 7fe770b075c0 -1 init, newargv = 0x55fd62575a70 newargc=15 2024-05-08T23:02:04.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[91280]: starting ceph client 2024-05-08T23:02:04.215 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[91280]: starting fuse 2024-05-08T23:02:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:04 smithi053 ceph-mon[77587]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-05-08T23:02:05.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:04 smithi046 ceph-mon[36898]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-05-08T23:02:05.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-05-08T23:02:05.357 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:02:05.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:02:05.424 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:02:05.427 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:02:05.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:05.427 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-05-08T23:02:05.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:04 smithi086 ceph-mon[78610]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 114 B/s wr, 0 op/s 2024-05-08T23:02:05.469 INFO:teuthology.orchestra.run.smithi046.stdout:123 2024-05-08T23:02:05.469 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [123] 2024-05-08T23:02:05.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:02:05.469 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-05-08T23:02:05.469 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-05-08T23:02:05.469 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-05-08T23:02:05.469 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:02:05.470 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-05-08T23:02:05.471 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-05-08T23:02:05.651 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.91280.asok 2024-05-08T23:02:05.828 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.91280.asok 2024-05-08T23:02:05.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:05.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.91280.asok status 2024-05-08T23:02:06.055 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "91280", 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 14721, 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-05-08T23:02:06.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 14721 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 532107338 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 532107338 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:02:06.057 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.14721 192.168.0.1:0/532107338", 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/532107338", 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 17, 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 82, 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-05-08T23:02:06.058 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:02:06.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:06.070 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-05-08T23:02:06.070 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:06.112 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-05-08T23:02:06.112 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:06.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:06.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:06.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:06.220 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T23:02:06.343 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-05-08T23:02:06.343 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T23:02:06.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:06.345 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:02:06.345 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:02:06.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.624+0000 7f29d7800700 1 -- 172.21.15.46:0/636019062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 msgr2=0x7f29d01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.624+0000 7f29d7800700 1 --2- 172.21.15.46:0/636019062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01092b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b0a0 tx=0x7f29c800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 -- 172.21.15.46:0/636019062 shutdown_connections 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 --2- 172.21.15.46:0/636019062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01092b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 --2- 172.21.15.46:0/636019062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 --2- 172.21.15.46:0/636019062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 0x7f29d0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 -- 172.21.15.46:0/636019062 >> 172.21.15.46:0/636019062 conn(0x7f29d00fc410 msgr2=0x7f29d00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 -- 172.21.15.46:0/636019062 shutdown_connections 2024-05-08T23:02:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.625+0000 7f29d7800700 1 -- 172.21.15.46:0/636019062 wait complete. 2024-05-08T23:02:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.626+0000 7f29d7800700 1 Processor -- start 2024-05-08T23:02:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.626+0000 7f29d7800700 1 -- start start 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d0199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 0x7f29d0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a0c20 con 0x7f29d0102990 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a0d60 con 0x7f29d0100e80 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d7800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a0ef0 con 0x7f29d0101fd0 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d5800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d0199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d5800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52722/0 (socket says 172.21.15.46:52722) 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29d5800700 1 -- 172.21.15.46:0/3621126239 learned_addr learned my addr 172.21.15.46:0/3621126239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.627+0000 7f29cfe00700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 0x7f29d0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 -- 172.21.15.46:0/3621126239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 msgr2=0x7f29d0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 0x7f29d0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 -- 172.21.15.46:0/3621126239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 msgr2=0x7f29d0199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d0199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 -- 172.21.15.46:0/3621126239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4009520 con 0x7f29d0102990 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d4e00700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d0199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d5800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b780 tx=0x7f29c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c800e040 con 0x7f29d0102990 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c8009480 con 0x7f29d0102990 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c8009d00 con 0x7f29d0102990 2024-05-08T23:02:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c80052a0 con 0x7f29d0102990 2024-05-08T23:02:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.628+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01a1400 con 0x7f29d0102990 2024-05-08T23:02:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.629+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d0067d10 con 0x7f29d0102990 2024-05-08T23:02:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.631+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f29c80054c0 con 0x7f29d0102990 2024-05-08T23:02:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.634+0000 7f29cd600700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 0x7f29b80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.634+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f29c80cdda0 con 0x7f29d0102990 2024-05-08T23:02:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.634+0000 7f29d4e00700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 0x7f29b80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.635+0000 7f29d4e00700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 0x7f29b80a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fd0 tx=0x7f29c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.637+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f29c8097370 con 0x7f29d0102990 2024-05-08T23:02:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.857+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29d010c910 con 0x7f29d0102990 2024-05-08T23:02:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.859+0000 7f29cd600700 1 -- 172.21.15.46:0/3621126239 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7f29c809a990 con 0x7f29d0102990 2024-05-08T23:02:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:06.860 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 msgr2=0x7f29b80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 0x7f29b80a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fd0 tx=0x7f29c4009260 comp rx=0 tx=0).stop 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 msgr2=0x7f29d01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b780 tx=0x7f29c800bc80 comp rx=0 tx=0).stop 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 shutdown_connections 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f29b80a11a0 0x7f29b80a3660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d0102990 0x7f29d01a0650 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29d0100e80 0x7f29d0199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 --2- 172.21.15.46:0/3621126239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0101fd0 0x7f29d0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 >> 172.21.15.46:0/3621126239 conn(0x7f29d00fc410 msgr2=0x7f29d00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 shutdown_connections 2024-05-08T23:02:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:06.862+0000 7f29d7800700 1 -- 172.21.15.46:0/3621126239 wait complete. 2024-05-08T23:02:06.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:02:06.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:02:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 -- 172.21.15.46:0/3095237925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac104220 msgr2=0x7f94ac106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 --2- 172.21.15.46:0/3095237925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac104220 0x7f94ac106610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f94980098f0 tx=0x7f9498009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 -- 172.21.15.46:0/3095237925 shutdown_connections 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 --2- 172.21.15.46:0/3095237925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac106b50 0x7f94ac108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 --2- 172.21.15.46:0/3095237925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac1018f0 0x7f94ac103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 --2- 172.21.15.46:0/3095237925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac104220 0x7f94ac106610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 -- 172.21.15.46:0/3095237925 >> 172.21.15.46:0/3095237925 conn(0x7f94ac0fb300 msgr2=0x7f94ac0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 -- 172.21.15.46:0/3095237925 shutdown_connections 2024-05-08T23:02:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.085+0000 7f94b2e00700 1 -- 172.21.15.46:0/3095237925 wait complete. 2024-05-08T23:02:07.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.086+0000 7f94b2e00700 1 Processor -- start 2024-05-08T23:02:07.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.086+0000 7f94b2e00700 1 -- start start 2024-05-08T23:02:07.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac104220 0x7f94ac0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac106b50 0x7f94ac0ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac100580 con 0x7f94ac1018f0 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1006f0 con 0x7f94ac104220 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1a0eb0 con 0x7f94ac106b50 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52752/0 (socket says 172.21.15.46:52752) 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 -- 172.21.15.46:0/2793175799 learned_addr learned my addr 172.21.15.46:0/2793175799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94b0e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac106b50 0x7f94ac0ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94ab400700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac104220 0x7f94ac0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 -- 172.21.15.46:0/2793175799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac106b50 msgr2=0x7f94ac0ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac106b50 0x7f94ac0ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 -- 172.21.15.46:0/2793175799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac104220 msgr2=0x7f94ac0ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac104220 0x7f94ac0ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.087+0000 7f94abe00700 1 -- 172.21.15.46:0/2793175799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4009d00 con 0x7f94ac1018f0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94abe00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9a0 tx=0x7f949c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c009790 con 0x7f94ac1018f0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949c00ce70 con 0x7f94ac1018f0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c018420 con 0x7f94ac1018f0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9498009520 con 0x7f94ac1018f0 2024-05-08T23:02:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.088+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac1a1470 con 0x7f94ac1018f0 2024-05-08T23:02:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.089+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0fce80 con 0x7f94ac1018f0 2024-05-08T23:02:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.091+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f949c0106f0 con 0x7f94ac1018f0 2024-05-08T23:02:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.092+0000 7f94a8c00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 0x7f94900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.092+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f949c014070 con 0x7f94ac1018f0 2024-05-08T23:02:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.093+0000 7f94ab400700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 0x7f94900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.093+0000 7f94ab400700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 0x7f94900a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f949800fa40 tx=0x7f9498019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.096+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f949c052d60 con 0x7f94ac1018f0 2024-05-08T23:02:07.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:06 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3621126239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:07.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:06 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3621126239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:07.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:06 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3621126239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.314+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94ac02cda0 con 0x7f94ac1018f0 2024-05-08T23:02:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.315+0000 7f94a8c00700 1 -- 172.21.15.46:0/2793175799 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f949c0961e0 con 0x7f94ac1018f0 2024-05-08T23:02:07.315 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:02:07.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 msgr2=0x7f94900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 0x7f94900a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f949800fa40 tx=0x7f9498019040 comp rx=0 tx=0).stop 2024-05-08T23:02:07.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 msgr2=0x7f94ac101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9a0 tx=0x7f949c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 shutdown_connections 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94ac106b50 0x7f94ac0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f94900a1120 0x7f94900a35e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac1018f0 0x7f94ac101210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 --2- 172.21.15.46:0/2793175799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac104220 0x7f94ac0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.317+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 >> 172.21.15.46:0/2793175799 conn(0x7f94ac0fb300 msgr2=0x7f94ac102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.318+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 shutdown_connections 2024-05-08T23:02:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.318+0000 7f94b2e00700 1 -- 172.21.15.46:0/2793175799 wait complete. 2024-05-08T23:02:07.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 -- 172.21.15.46:0/1444059704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690100e80 msgr2=0x7f76901012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 --2- 172.21.15.46:0/1444059704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690100e80 0x7f76901012a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7680009990 tx=0x7f7680009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 -- 172.21.15.46:0/1444059704 shutdown_connections 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 --2- 172.21.15.46:0/1444059704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690102990 0x7f76901092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 --2- 172.21.15.46:0/1444059704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690100e80 0x7f76901012a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 --2- 172.21.15.46:0/1444059704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690101fd0 0x7f7690102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 -- 172.21.15.46:0/1444059704 >> 172.21.15.46:0/1444059704 conn(0x7f76900fc3b0 msgr2=0x7f76900fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 -- 172.21.15.46:0/1444059704 shutdown_connections 2024-05-08T23:02:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.542+0000 7f7695000700 1 -- 172.21.15.46:0/1444059704 wait complete. 2024-05-08T23:02:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.543+0000 7f7695000700 1 Processor -- start 2024-05-08T23:02:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.543+0000 7f7695000700 1 -- start start 2024-05-08T23:02:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 0x7f7690199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690102990 0x7f76901a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a0b70 con 0x7f7690101fd0 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a0cb0 con 0x7f7690100e80 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f7695000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76901a0e40 con 0x7f7690102990 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f768d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f768d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52762/0 (socket says 172.21.15.46:52762) 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f768d400700 1 -- 172.21.15.46:0/1284465320 learned_addr learned my addr 172.21.15.46:0/1284465320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f768de00700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 0x7f7690199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.544+0000 7f768e800700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690102990 0x7f76901a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 -- 172.21.15.46:0/1284465320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690102990 msgr2=0x7f76901a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690102990 0x7f76901a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 -- 172.21.15.46:0/1284465320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 msgr2=0x7f7690199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 0x7f7690199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 -- 172.21.15.46:0/1284465320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c009520 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768de00700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 0x7f7690199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f768d400700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f767c00eb30 tx=0x7f767c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c009880 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c0044e0 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c003e90 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76800095f0 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.545+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76901a14a0 con 0x7f7690101fd0 2024-05-08T23:02:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.546+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7690067d10 con 0x7f7690101fd0 2024-05-08T23:02:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.548+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f767c004000 con 0x7f7690101fd0 2024-05-08T23:02:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.550+0000 7f7686a00700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 0x7f76740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.550+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f767c014070 con 0x7f7690101fd0 2024-05-08T23:02:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.550+0000 7f768de00700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 0x7f76740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.550+0000 7f768de00700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 0x7f76740a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f768000b2e0 tx=0x7f7680005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.552+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f767c0965c0 con 0x7f7690101fd0 2024-05-08T23:02:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.771+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f769010c920 con 0x7f7690101fd0 2024-05-08T23:02:07.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.772+0000 7f7686a00700 1 -- 172.21.15.46:0/1284465320 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f767c099be0 con 0x7f7690101fd0 2024-05-08T23:02:07.772 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 msgr2=0x7f76740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 0x7f76740a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f768000b2e0 tx=0x7f7680005fb0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 msgr2=0x7f7690199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f767c00eb30 tx=0x7f767c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 shutdown_connections 2024-05-08T23:02:07.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7690102990 0x7f76901a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100e80 0x7f7690199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f76740a11a0 0x7f76740a3660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 --2- 172.21.15.46:0/1284465320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7690101fd0 0x7f7690199580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 >> 172.21.15.46:0/1284465320 conn(0x7f76900fc3b0 msgr2=0x7f7690107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 shutdown_connections 2024-05-08T23:02:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.774+0000 7f7695000700 1 -- 172.21.15.46:0/1284465320 wait complete. 2024-05-08T23:02:07.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:02:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 -- 172.21.15.46:0/1627266170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158101fd0 msgr2=0x7fe158102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 --2- 172.21.15.46:0/1627266170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158101fd0 0x7fe158102450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe144009990 tx=0x7fe144009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 -- 172.21.15.46:0/1627266170 shutdown_connections 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 --2- 172.21.15.46:0/1627266170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 0x7fe1581092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 --2- 172.21.15.46:0/1627266170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158100e80 0x7fe1581012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 --2- 172.21.15.46:0/1627266170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158101fd0 0x7fe158102450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 -- 172.21.15.46:0/1627266170 >> 172.21.15.46:0/1627266170 conn(0x7fe1580fc3b0 msgr2=0x7fe1580fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 -- 172.21.15.46:0/1627266170 shutdown_connections 2024-05-08T23:02:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.998+0000 7fe15e200700 1 -- 172.21.15.46:0/1627266170 wait complete. 2024-05-08T23:02:08.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.999+0000 7fe15e200700 1 Processor -- start 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:07.999+0000 7fe15e200700 1 -- start start 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158101fd0 0x7fe1581995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 0x7fe1581a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581a0b70 con 0x7fe158102990 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581a0cb0 con 0x7fe158100e80 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe15e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581a0e40 con 0x7fe158101fd0 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58456/0 (socket says 172.21.15.46:58456) 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 -- 172.21.15.46:0/2413502093 learned_addr learned my addr 172.21.15.46:0/2413502093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157e00700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 0x7fe1581a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe156a00700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158101fd0 0x7fe1581995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 -- 172.21.15.46:0/2413502093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158101fd0 msgr2=0x7fe1581995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158101fd0 0x7fe1581995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 -- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 msgr2=0x7fe1581a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 0x7fe1581a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.000+0000 7fe157400700 1 -- 172.21.15.46:0/2413502093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440095f0 con 0x7fe158100e80 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe157400700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e9c0 tx=0x7fe14800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480097b0 con 0x7fe158100e80 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14800ee60 con 0x7fe158100e80 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480054f0 con 0x7fe158100e80 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1581a1120 con 0x7fe158100e80 2024-05-08T23:02:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.001+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1581a15e0 con 0x7fe158100e80 2024-05-08T23:02:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.002+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe158067d10 con 0x7fe158100e80 2024-05-08T23:02:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.004+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe148004750 con 0x7fe158100e80 2024-05-08T23:02:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.005+0000 7fe14fe00700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 0x7fe13c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.005+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe148014070 con 0x7fe158100e80 2024-05-08T23:02:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.005+0000 7fe156a00700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 0x7fe13c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.006+0000 7fe156a00700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 0x7fe13c0a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe14400b2e0 tx=0x7fe144005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.010+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe148096220 con 0x7fe158100e80 2024-05-08T23:02:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:07 smithi086 ceph-mon[78610]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:07 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2793175799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:07 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1284465320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.226+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1581a1960 con 0x7fe158100e80 2024-05-08T23:02:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.227+0000 7fe14fe00700 1 -- 172.21.15.46:0/2413502093 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe148099840 con 0x7fe158100e80 2024-05-08T23:02:08.227 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:02:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 msgr2=0x7fe13c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 0x7fe13c0a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe14400b2e0 tx=0x7fe144005fd0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 msgr2=0x7fe158199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e9c0 tx=0x7fe14800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 shutdown_connections 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe13c0a11a0 0x7fe13c0a3660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe158102990 0x7fe1581a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe158100e80 0x7fe158199090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 --2- 172.21.15.46:0/2413502093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe158101fd0 0x7fe1581995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 >> 172.21.15.46:0/2413502093 conn(0x7fe1580fc3b0 msgr2=0x7fe1580731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 shutdown_connections 2024-05-08T23:02:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.230+0000 7fe15e200700 1 -- 172.21.15.46:0/2413502093 wait complete. 2024-05-08T23:02:08.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:07 smithi046 ceph-mon[36898]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:08.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2793175799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1284465320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:02:08.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:07 smithi053 ceph-mon[77587]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:08.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:07 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2793175799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:07 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1284465320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 -- 172.21.15.46:0/260182447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 msgr2=0x7f3b44103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 --2- 172.21.15.46:0/260182447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44103cd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009990 tx=0x7f3b34009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 -- 172.21.15.46:0/260182447 shutdown_connections 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 --2- 172.21.15.46:0/260182447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44106b40 0x7f3b44108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 --2- 172.21.15.46:0/260182447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44103cd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 --2- 172.21.15.46:0/260182447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44104210 0x7f3b44106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 -- 172.21.15.46:0/260182447 >> 172.21.15.46:0/260182447 conn(0x7f3b440fb290 msgr2=0x7f3b440fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 -- 172.21.15.46:0/260182447 shutdown_connections 2024-05-08T23:02:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.451+0000 7f3b49e00700 1 -- 172.21.15.46:0/260182447 wait complete. 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.452+0000 7f3b49e00700 1 Processor -- start 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.452+0000 7f3b49e00700 1 -- start start 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44104210 0x7f3b441994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 0x7f3b441a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a0ad0 con 0x7f3b441018e0 2024-05-08T23:02:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a0c10 con 0x7f3b44106b40 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b49e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441a0da0 con 0x7f3b44104210 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52796/0 (socket says 172.21.15.46:52796) 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 -- 172.21.15.46:0/2542011863 learned_addr learned my addr 172.21.15.46:0/2542011863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42000700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44104210 0x7f3b441994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b43400700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 0x7f3b441a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 -- 172.21.15.46:0/2542011863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44104210 msgr2=0x7f3b441994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44104210 0x7f3b441994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 -- 172.21.15.46:0/2542011863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 msgr2=0x7f3b441a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 0x7f3b441a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b42a00700 1 -- 172.21.15.46:0/2542011863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340095f0 con 0x7f3b441018e0 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.453+0000 7f3b43400700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 0x7f3b441a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b42a00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098e0 tx=0x7f3b34005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3401d070 con 0x7f3b441018e0 2024-05-08T23:02:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b34022470 con 0x7f3b441018e0 2024-05-08T23:02:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34017650 con 0x7f3b441018e0 2024-05-08T23:02:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b441a1020 con 0x7f3b441018e0 2024-05-08T23:02:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.454+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b441a1440 con 0x7f3b441018e0 2024-05-08T23:02:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.455+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b44108710 con 0x7f3b441018e0 2024-05-08T23:02:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.457+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3b34022850 con 0x7f3b441018e0 2024-05-08T23:02:08.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.458+0000 7f3b48a00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 0x7f3b280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.458+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b340cdfd0 con 0x7f3b441018e0 2024-05-08T23:02:08.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.459+0000 7f3b42000700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 0x7f3b280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.459+0000 7f3b42000700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 0x7f3b280a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009c60 tx=0x7f3b30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.462+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b340976c0 con 0x7f3b441018e0 2024-05-08T23:02:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.684+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3b4402ce60 con 0x7f3b441018e0 2024-05-08T23:02:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.684+0000 7f3b48a00700 1 -- 172.21.15.46:0/2542011863 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v82) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3b34027020 con 0x7f3b441018e0 2024-05-08T23:02:08.684 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 msgr2=0x7f3b280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 0x7f3b280a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009c60 tx=0x7f3b30009260 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 msgr2=0x7f3b44198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098e0 tx=0x7f3b34005720 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 shutdown_connections 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.686+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44106b40 0x7f3b441a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f3b280a11a0 0x7f3b280a3660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b441018e0 0x7f3b44198fa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 --2- 172.21.15.46:0/2542011863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44104210 0x7f3b441994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 >> 172.21.15.46:0/2542011863 conn(0x7f3b440fb290 msgr2=0x7f3b44101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 shutdown_connections 2024-05-08T23:02:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.687+0000 7f3b49e00700 1 -- 172.21.15.46:0/2542011863 wait complete. 2024-05-08T23:02:08.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:02:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 -- 172.21.15.46:0/4962286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 msgr2=0x7ffa4c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 --2- 172.21.15.46:0/4962286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c073430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa400098f0 tx=0x7ffa40009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 -- 172.21.15.46:0/4962286 shutdown_connections 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 --2- 172.21.15.46:0/4962286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c108d40 0x7ffa4c10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 --2- 172.21.15.46:0/4962286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c073430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 --2- 172.21.15.46:0/4962286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c073a00 0x7ffa4c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 -- 172.21.15.46:0/4962286 >> 172.21.15.46:0/4962286 conn(0x7ffa4c0fc3b0 msgr2=0x7ffa4c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 -- 172.21.15.46:0/4962286 shutdown_connections 2024-05-08T23:02:08.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.904+0000 7ffa53400700 1 -- 172.21.15.46:0/4962286 wait complete. 2024-05-08T23:02:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.905+0000 7ffa53400700 1 Processor -- start 2024-05-08T23:02:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.905+0000 7ffa53400700 1 -- start start 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c073a00 0x7ffa4c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1a0b90 con 0x7ffa4c108d40 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1a0cd0 con 0x7ffa4c074fe0 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa53400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1a0e60 con 0x7ffa4c073a00 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52808/0 (socket says 172.21.15.46:52808) 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 -- 172.21.15.46:0/4201011852 learned_addr learned my addr 172.21.15.46:0/4201011852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa50a00700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c073a00 0x7ffa4c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa4be00700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 -- 172.21.15.46:0/4201011852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c073a00 msgr2=0x7ffa4c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c073a00 0x7ffa4c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 -- 172.21.15.46:0/4201011852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 msgr2=0x7ffa4c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.906+0000 7ffa51400700 1 -- 172.21.15.46:0/4201011852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009520 con 0x7ffa4c108d40 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa51400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ef30 tx=0x7ffa4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44018070 con 0x7ffa4c108d40 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4400f040 con 0x7ffa4c108d40 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400ccf0 con 0x7ffa4c108d40 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c1a10e0 con 0x7ffa4c108d40 2024-05-08T23:02:08.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.907+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1a15a0 con 0x7ffa4c108d40 2024-05-08T23:02:08.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.908+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c107b00 con 0x7ffa4c108d40 2024-05-08T23:02:08.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.910+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffa44003a00 con 0x7ffa4c108d40 2024-05-08T23:02:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.911+0000 7ffa49600700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 0x7ffa340a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.911+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffa440cc5e0 con 0x7ffa4c108d40 2024-05-08T23:02:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.912+0000 7ffa50a00700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 0x7ffa340a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.912+0000 7ffa50a00700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 0x7ffa340a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000fa40 tx=0x7ffa40019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:08.916+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa44095bb0 con 0x7ffa4c108d40 2024-05-08T23:02:09.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2413502093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2542011863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.152+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa4c067d10 con 0x7ffa4c108d40 2024-05-08T23:02:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.152+0000 7ffa49600700 1 -- 172.21.15.46:0/4201011852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) v1 ==== 81+0+588 (secure 0 0 0) 0x7ffa440991d0 con 0x7ffa4c108d40 2024-05-08T23:02:09.152 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2015473581 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/387189775 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/652393065 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1980881468 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3841409271 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4047035481 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:09.153 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 msgr2=0x7ffa340a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 0x7ffa340a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000fa40 tx=0x7ffa40019040 comp rx=0 tx=0).stop 2024-05-08T23:02:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 msgr2=0x7ffa4c1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ef30 tx=0x7ffa4400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 shutdown_connections 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa340a11a0 0x7ffa340a3660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c108d40 0x7ffa4c1a05c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c073a00 0x7ffa4c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 --2- 172.21.15.46:0/4201011852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa4c074fe0 0x7ffa4c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 >> 172.21.15.46:0/4201011852 conn(0x7ffa4c0fc3b0 msgr2=0x7ffa4c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 shutdown_connections 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.155+0000 7ffa53400700 1 -- 172.21.15.46:0/4201011852 wait complete. 2024-05-08T23:02:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-05-08T23:02:09.174 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:02:09.174 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:02:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:08 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2413502093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:08 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2542011863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:02:09.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:08 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2413502093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:08 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2542011863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:09.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 -- 172.21.15.46:0/3319810151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690101fd0 msgr2=0x7f6690102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 --2- 172.21.15.46:0/3319810151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690101fd0 0x7f6690102450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f667c009990 tx=0x7f667c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 -- 172.21.15.46:0/3319810151 shutdown_connections 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 --2- 172.21.15.46:0/3319810151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 0x7f66901092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 --2- 172.21.15.46:0/3319810151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690100e80 0x7f66901012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 --2- 172.21.15.46:0/3319810151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690101fd0 0x7f6690102450 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 -- 172.21.15.46:0/3319810151 >> 172.21.15.46:0/3319810151 conn(0x7f66900fc410 msgr2=0x7f66900fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 -- 172.21.15.46:0/3319810151 shutdown_connections 2024-05-08T23:02:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.454+0000 7f6697200700 1 -- 172.21.15.46:0/3319810151 wait complete. 2024-05-08T23:02:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.455+0000 7f6697200700 1 Processor -- start 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.455+0000 7f6697200700 1 -- start start 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690101fd0 0x7f6690073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 0x7f66900738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690074070 con 0x7f6690100e80 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900741e0 con 0x7f6690102990 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6697200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66901a0e60 con 0x7f6690101fd0 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f668fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f668fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52834/0 (socket says 172.21.15.46:52834) 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f668f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690101fd0 0x7f6690073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f668fe00700 1 -- 172.21.15.46:0/2191951039 learned_addr learned my addr 172.21.15.46:0/2191951039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f6695200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 0x7f66900738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.456+0000 7f668fe00700 1 -- 172.21.15.46:0/2191951039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690101fd0 msgr2=0x7f6690073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668fe00700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690101fd0 0x7f6690073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668fe00700 1 -- 172.21.15.46:0/2191951039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 msgr2=0x7f66900738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668fe00700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 0x7f66900738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668fe00700 1 -- 172.21.15.46:0/2191951039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009520 con 0x7f6690100e80 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668fe00700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f668000eb30 tx=0x7f668000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680009880 con 0x7f6690100e80 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668000ce30 con 0x7f6690100e80 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c0095f0 con 0x7f6690100e80 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680003cc0 con 0x7f6690100e80 2024-05-08T23:02:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.457+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901a1470 con 0x7f6690100e80 2024-05-08T23:02:09.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.458+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690108240 con 0x7f6690100e80 2024-05-08T23:02:09.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.460+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f668000c970 con 0x7f6690100e80 2024-05-08T23:02:09.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.461+0000 7f668cc00700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 0x7f66740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.462+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6680014070 con 0x7f6690100e80 2024-05-08T23:02:09.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.462+0000 7f668f400700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 0x7f66740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.462+0000 7f668f400700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 0x7f66740a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b2e0 tx=0x7f667c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:09.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.465+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6680096530 con 0x7f6690100e80 2024-05-08T23:02:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.685+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6690067d10 con 0x7f6690100e80 2024-05-08T23:02:09.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.687+0000 7f668cc00700 1 -- 172.21.15.46:0/2191951039 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7f6680099b50 con 0x7f6690100e80 2024-05-08T23:02:09.687 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:09.687 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 msgr2=0x7f66740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 0x7f66740a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b2e0 tx=0x7f667c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 msgr2=0x7f6690074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f668000eb30 tx=0x7f668000ee40 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 shutdown_connections 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6690102990 0x7f66900738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f66740a1120 0x7f66740a35e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6690100e80 0x7f6690074d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.689+0000 7f6697200700 1 --2- 172.21.15.46:0/2191951039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6690101fd0 0x7f6690073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.690+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 >> 172.21.15.46:0/2191951039 conn(0x7f66900fc410 msgr2=0x7f6690105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.690+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 shutdown_connections 2024-05-08T23:02:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.690+0000 7f6697200700 1 -- 172.21.15.46:0/2191951039 wait complete. 2024-05-08T23:02:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:02:09.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 -- 172.21.15.46:0/2579057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 msgr2=0x7f73380733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 --2- 172.21.15.46:0/2579057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f73380733d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7328009990 tx=0x7f7328009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 -- 172.21.15.46:0/2579057186 shutdown_connections 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 --2- 172.21.15.46:0/2579057186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f733810b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 --2- 172.21.15.46:0/2579057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f73380733d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 --2- 172.21.15.46:0/2579057186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 -- 172.21.15.46:0/2579057186 >> 172.21.15.46:0/2579057186 conn(0x7f73380fc3d0 msgr2=0x7f73380fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 -- 172.21.15.46:0/2579057186 shutdown_connections 2024-05-08T23:02:09.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.911+0000 7f733ea00700 1 -- 172.21.15.46:0/2579057186 wait complete. 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 Processor -- start 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 -- start start 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f7338199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f73381a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381a0b20 con 0x7f7338074f80 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381a0c60 con 0x7f7338108d50 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.912+0000 7f733ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381a0df0 con 0x7f73380739a0 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f7338199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f733ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f73381a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46572/0 (socket says 172.21.15.46:46572) 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 -- 172.21.15.46:0/3024668940 learned_addr learned my addr 172.21.15.46:0/3024668940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 -- 172.21.15.46:0/3024668940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 msgr2=0x7f73381a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f73381a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 -- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 msgr2=0x7f7338199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f7338199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 -- 172.21.15.46:0/3024668940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280095f0 con 0x7f73380739a0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f733ca00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f73381a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337400700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f7338199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.913+0000 7f7337e00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7328006010 tx=0x7f7328004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.914+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732801d070 con 0x7f73380739a0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.914+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7328005030 con 0x7f73380739a0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.914+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328004b40 con 0x7f73380739a0 2024-05-08T23:02:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.914+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73381a1070 con 0x7f73380739a0 2024-05-08T23:02:09.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.914+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381a1490 con 0x7f73380739a0 2024-05-08T23:02:09.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.915+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7338107b60 con 0x7f73380739a0 2024-05-08T23:02:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.917+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7328004d00 con 0x7f73380739a0 2024-05-08T23:02:09.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.918+0000 7f7334c00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 0x7f731c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:09.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.919+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f73280cdf50 con 0x7f73380739a0 2024-05-08T23:02:09.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.919+0000 7f7337400700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 0x7f731c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:09.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.919+0000 7f7337400700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 0x7f731c0a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7324011490 tx=0x7f7324009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:09.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:09.921+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7328097640 con 0x7f73380739a0 2024-05-08T23:02:10.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:09 smithi046 ceph-mon[36898]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:10.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4201011852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2191951039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.144+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f733810e8d0 con 0x7f73380739a0 2024-05-08T23:02:10.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.145+0000 7f7334c00700 1 -- 172.21.15.46:0/3024668940 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7328026020 con 0x7f73380739a0 2024-05-08T23:02:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.147+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 msgr2=0x7f731c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.147+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 0x7f731c0a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7324011490 tx=0x7f7324009260 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.147+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 msgr2=0x7f7338198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.147+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7328006010 tx=0x7f7328004730 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.147+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 shutdown_connections 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338108d50 0x7f73381a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73380739a0 0x7f7338198ff0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f731c0a11a0 0x7f731c0a3660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 --2- 172.21.15.46:0/3024668940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338074f80 0x7f7338199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 >> 172.21.15.46:0/3024668940 conn(0x7f73380fc3d0 msgr2=0x7f7338104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 shutdown_connections 2024-05-08T23:02:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.148+0000 7f733ea00700 1 -- 172.21.15.46:0/3024668940 wait complete. 2024-05-08T23:02:10.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:02:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:09 smithi086 ceph-mon[78610]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:09 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4201011852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:09 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2191951039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:09 smithi053 ceph-mon[77587]: pgmap v248: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:02:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:09 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4201011852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:09 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2191951039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:10.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 -- 172.21.15.46:0/3248901194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324102990 msgr2=0x7fe3241092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 --2- 172.21.15.46:0/3248901194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324102990 0x7fe3241092b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00b0a0 tx=0x7fe31c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 -- 172.21.15.46:0/3248901194 shutdown_connections 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 --2- 172.21.15.46:0/3248901194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324102990 0x7fe3241092b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 --2- 172.21.15.46:0/3248901194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324100e80 0x7fe3241012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 --2- 172.21.15.46:0/3248901194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324101fd0 0x7fe324102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 -- 172.21.15.46:0/3248901194 >> 172.21.15.46:0/3248901194 conn(0x7fe3240fc3b0 msgr2=0x7fe3240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 -- 172.21.15.46:0/3248901194 shutdown_connections 2024-05-08T23:02:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.366+0000 7fe329000700 1 -- 172.21.15.46:0/3248901194 wait complete. 2024-05-08T23:02:10.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.367+0000 7fe329000700 1 Processor -- start 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.367+0000 7fe329000700 1 -- start start 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 0x7fe324074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324102990 0x7fe3240738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe324074070 con 0x7fe324102990 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240741e0 con 0x7fe324101fd0 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe329000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3241a0f60 con 0x7fe324100e80 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58526/0 (socket says 172.21.15.46:58526) 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321400700 1 -- 172.21.15.46:0/2540389192 learned_addr learned my addr 172.21.15.46:0/2540389192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321e00700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 0x7fe324074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe322800700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324102990 0x7fe3240738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321e00700 1 -- 172.21.15.46:0/2540389192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 msgr2=0x7fe324073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321e00700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321e00700 1 -- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324102990 msgr2=0x7fe3240738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.368+0000 7fe321e00700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324102990 0x7fe3240738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe321e00700 1 -- 172.21.15.46:0/2540389192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe314009520 con 0x7fe324100e80 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe321400700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe321e00700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 0x7fe324074d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9f0 tx=0x7fe31400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400cb70 con 0x7fe324100e80 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe31400ee90 con 0x7fe324100e80 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c009d00 con 0x7fe324100e80 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314003cf0 con 0x7fe324100e80 2024-05-08T23:02:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.369+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3241a1510 con 0x7fe324100e80 2024-05-08T23:02:10.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.370+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324108220 con 0x7fe324100e80 2024-05-08T23:02:10.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.372+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe31400ccd0 con 0x7fe324100e80 2024-05-08T23:02:10.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.373+0000 7fe31aa00700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 0x7fe3080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.373+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe314014070 con 0x7fe324100e80 2024-05-08T23:02:10.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.374+0000 7fe321400700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 0x7fe3080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.374+0000 7fe321400700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 0x7fe3080a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe310011590 tx=0x7fe310009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.376+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe314096560 con 0x7fe324100e80 2024-05-08T23:02:10.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.591+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe32410c940 con 0x7fe324100e80 2024-05-08T23:02:10.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.592+0000 7fe31aa00700 1 -- 172.21.15.46:0/2540389192 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe314099b80 con 0x7fe324100e80 2024-05-08T23:02:10.592 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.594+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 msgr2=0x7fe3080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.594+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 0x7fe3080a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe310011590 tx=0x7fe310009260 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.594+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 msgr2=0x7fe324074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.594+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 0x7fe324074d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9f0 tx=0x7fe31400ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 shutdown_connections 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe3080a1120 0x7fe3080a35e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe324102990 0x7fe3240738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe324100e80 0x7fe324074d00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 --2- 172.21.15.46:0/2540389192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe324101fd0 0x7fe324073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 >> 172.21.15.46:0/2540389192 conn(0x7fe3240fc3b0 msgr2=0x7fe324105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:10.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 shutdown_connections 2024-05-08T23:02:10.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.595+0000 7fe329000700 1 -- 172.21.15.46:0/2540389192 wait complete. 2024-05-08T23:02:10.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 -- 172.21.15.46:0/3610186656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec1018e0 msgr2=0x7fc8ec103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 --2- 172.21.15.46:0/3610186656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec103cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009990 tx=0x7fc8dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 -- 172.21.15.46:0/3610186656 shutdown_connections 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 --2- 172.21.15.46:0/3610186656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec106b40 0x7fc8ec108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 --2- 172.21.15.46:0/3610186656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec103cd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 --2- 172.21.15.46:0/3610186656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 -- 172.21.15.46:0/3610186656 >> 172.21.15.46:0/3610186656 conn(0x7fc8ec0fb290 msgr2=0x7fc8ec0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 -- 172.21.15.46:0/3610186656 shutdown_connections 2024-05-08T23:02:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.814+0000 7fc8f1600700 1 -- 172.21.15.46:0/3610186656 wait complete. 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.815+0000 7fc8f1600700 1 Processor -- start 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 -- start start 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec106b40 0x7fc8ec1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a0b40 con 0x7fc8ec106b40 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a0c80 con 0x7fc8ec1018e0 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8f1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a0e10 con 0x7fc8ec104210 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8eaa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46600/0 (socket says 172.21.15.46:46600) 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8ea000700 1 -- 172.21.15.46:0/791709141 learned_addr learned my addr 172.21.15.46:0/791709141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.816+0000 7fc8eb400700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec106b40 0x7fc8ec1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 -- 172.21.15.46:0/791709141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 msgr2=0x7fc8ec199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 -- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec106b40 msgr2=0x7fc8ec1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec106b40 0x7fc8ec1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 -- 172.21.15.46:0/791709141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0095f0 con 0x7fc8ec1018e0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8ea000700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8eaa00700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec199010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006010 tx=0x7fc8dc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc01d070 con 0x7fc8ec1018e0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc005510 con 0x7fc8ec1018e0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec1a1090 con 0x7fc8ec1018e0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc004b00 con 0x7fc8ec1018e0 2024-05-08T23:02:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.817+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec1a14b0 con 0x7fc8ec1018e0 2024-05-08T23:02:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.818+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec108710 con 0x7fc8ec1018e0 2024-05-08T23:02:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.820+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc8dc00fb90 con 0x7fc8ec1018e0 2024-05-08T23:02:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.822+0000 7fc8e3400700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 0x7fc8d00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.822+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc8dc0cdda0 con 0x7fc8ec1018e0 2024-05-08T23:02:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.822+0000 7fc8ea000700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 0x7fc8d00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.822+0000 7fc8ea000700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 0x7fc8d00a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80104e0 tx=0x7fc8d8007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:10.824+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8dc097370 con 0x7fc8ec1018e0 2024-05-08T23:02:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.044+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8ec10c690 con 0x7fc8ec1018e0 2024-05-08T23:02:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.045+0000 7fc8e3400700 1 -- 172.21.15.46:0/791709141 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8dc09a990 con 0x7fc8ec1018e0 2024-05-08T23:02:11.045 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 msgr2=0x7fc8d00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 0x7fc8d00a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80104e0 tx=0x7fc8d8007480 comp rx=0 tx=0).stop 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 msgr2=0x7fc8ec199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec199010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006010 tx=0x7fc8dc00f690 comp rx=0 tx=0).stop 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 shutdown_connections 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8d00a1120 0x7fc8d00a35e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8ec106b40 0x7fc8ec1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ec1018e0 0x7fc8ec199010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 --2- 172.21.15.46:0/791709141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ec104210 0x7fc8ec199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 >> 172.21.15.46:0/791709141 conn(0x7fc8ec0fb290 msgr2=0x7fc8ec101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 shutdown_connections 2024-05-08T23:02:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.048+0000 7fc8f1600700 1 -- 172.21.15.46:0/791709141 wait complete. 2024-05-08T23:02:11.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:02:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:10 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3024668940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:10 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2540389192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:10 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3024668940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:10 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2540389192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 -- 172.21.15.46:0/1137638955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 msgr2=0x7fe1101012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 --2- 172.21.15.46:0/1137638955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe1101012a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe104009990 tx=0x7fe104009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 -- 172.21.15.46:0/1137638955 shutdown_connections 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 --2- 172.21.15.46:0/1137638955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 0x7fe1101092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 --2- 172.21.15.46:0/1137638955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe1101012a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 --2- 172.21.15.46:0/1137638955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 0x7fe110102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 -- 172.21.15.46:0/1137638955 >> 172.21.15.46:0/1137638955 conn(0x7fe1100fc3b0 msgr2=0x7fe1100fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 -- 172.21.15.46:0/1137638955 shutdown_connections 2024-05-08T23:02:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.268+0000 7fe117c00700 1 -- 172.21.15.46:0/1137638955 wait complete. 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 Processor -- start 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 -- start start 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 0x7fe110197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 0x7fe11019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019e960 con 0x7fe110100e80 2024-05-08T23:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019eaa0 con 0x7fe110102990 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.269+0000 7fe117c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019ec30 con 0x7fe110101fd0 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52894/0 (socket says 172.21.15.46:52894) 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 -- 172.21.15.46:0/3485763821 learned_addr learned my addr 172.21.15.46:0/3485763821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 0x7fe11019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe10fe00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 0x7fe110197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 -- 172.21.15.46:0/3485763821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 msgr2=0x7fe110197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 0x7fe110197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 -- 172.21.15.46:0/3485763821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 msgr2=0x7fe11019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 0x7fe11019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 -- 172.21.15.46:0/3485763821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040095f0 con 0x7fe110100e80 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.270+0000 7fe115200700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe104005910 tx=0x7fe104005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.271+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10401d070 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.271+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe104021410 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.271+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104017650 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.271+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11019eeb0 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.271+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11019f220 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.272+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe110108290 con 0x7fe110100e80 2024-05-08T23:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.274+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe104021580 con 0x7fe110100e80 2024-05-08T23:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.275+0000 7fe10d600700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 0x7fe0f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.275+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1040cde40 con 0x7fe110100e80 2024-05-08T23:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.275+0000 7fe10fe00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 0x7fe0f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.276+0000 7fe10fe00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 0x7fe0f80a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009c60 tx=0x7fe0fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:11.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:10 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3024668940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:10 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2540389192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.279+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe104097410 con 0x7fe110100e80 2024-05-08T23:02:11.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.500+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe11010c9a0 con 0x7fe110100e80 2024-05-08T23:02:11.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.501+0000 7fe10d600700 1 -- 172.21.15.46:0/3485763821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v82) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe10409aa30 con 0x7fe110100e80 2024-05-08T23:02:11.501 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:02:11.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.503+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 msgr2=0x7fe0f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.503+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 0x7fe0f80a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009c60 tx=0x7fe0fc009260 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.503+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 msgr2=0x7fe110196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.503+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe104005910 tx=0x7fe104005740 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.503+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 shutdown_connections 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe110102990 0x7fe11019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe0f80a11a0 0x7fe0f80a3660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe110100e80 0x7fe110196e30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 --2- 172.21.15.46:0/3485763821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe110101fd0 0x7fe110197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 >> 172.21.15.46:0/3485763821 conn(0x7fe1100fc3b0 msgr2=0x7fe1101053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 shutdown_connections 2024-05-08T23:02:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.504+0000 7fe117c00700 1 -- 172.21.15.46:0/3485763821 wait complete. 2024-05-08T23:02:11.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:02:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3161302692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4100e80 msgr2=0x7f9dd41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3161302692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4100e80 0x7f9dd41012a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009990 tx=0x7f9dc8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3161302692 shutdown_connections 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3161302692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4102990 0x7f9dd41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3161302692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4100e80 0x7f9dd41012a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3161302692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4101fd0 0x7f9dd4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3161302692 >> 172.21.15.46:0/3161302692 conn(0x7f9dd40fc3b0 msgr2=0x7f9dd40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3161302692 shutdown_connections 2024-05-08T23:02:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.722+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3161302692 wait complete. 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.723+0000 7f9ddbe00700 1 Processor -- start 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.723+0000 7f9ddbe00700 1 -- start start 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4100e80 0x7f9dd4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4102990 0x7f9dd41a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41a0ae0 con 0x7f9dd4101fd0 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41a0c20 con 0x7f9dd4100e80 2024-05-08T23:02:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9ddbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41a0db0 con 0x7f9dd4102990 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 -- 172.21.15.46:0/3459712034 learned_addr learned my addr 172.21.15.46:0/3459712034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd9400700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4100e80 0x7f9dd4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd9e00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4102990 0x7f9dd41a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 -- 172.21.15.46:0/3459712034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4102990 msgr2=0x7f9dd41a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4102990 0x7f9dd41a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 -- 172.21.15.46:0/3459712034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4100e80 msgr2=0x7f9dd4198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4100e80 0x7f9dd4198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.724+0000 7f9dd8a00700 1 -- 172.21.15.46:0/3459712034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80095f0 con 0x7f9dd4101fd0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9dd8a00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9a0 tx=0x7f9dc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0009790 con 0x7f9dd4101fd0 2024-05-08T23:02:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc0010430 con 0x7f9dd4101fd0 2024-05-08T23:02:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0018680 con 0x7f9dd4101fd0 2024-05-08T23:02:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd41a1090 con 0x7f9dd4101fd0 2024-05-08T23:02:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.725+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd41a1550 con 0x7f9dd4101fd0 2024-05-08T23:02:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.726+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd4067d10 con 0x7f9dd4101fd0 2024-05-08T23:02:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.728+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9dc00105a0 con 0x7f9dd4101fd0 2024-05-08T23:02:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.729+0000 7f9dd2000700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 0x7f9dbc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.729+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9dc0014070 con 0x7f9dd4101fd0 2024-05-08T23:02:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.730+0000 7f9dd9400700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 0x7f9dbc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.730+0000 7f9dd9400700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 0x7f9dbc0a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800b2e0 tx=0x7f9dc8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.733+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9dc0052d60 con 0x7f9dd4101fd0 2024-05-08T23:02:11.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.973+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9dd410c670 con 0x7f9dd4101fd0 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.973+0000 7f9dd2000700 1 -- 172.21.15.46:0/3459712034 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) v1 ==== 81+0+588 (secure 0 0 0) 0x7f9dc0096230 con 0x7f9dd4101fd0 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2015473581 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/387189775 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/652393065 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1980881468 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3841409271 2024-05-09T22:55:52.116773+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4047035481 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:11.974 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 msgr2=0x7f9dbc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 0x7f9dbc0a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800b2e0 tx=0x7f9dc8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 msgr2=0x7f9dd41994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9a0 tx=0x7f9dc000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 shutdown_connections 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd4102990 0x7f9dd41a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd4100e80 0x7f9dd4198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9dbc0a1120 0x7f9dbc0a35e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 --2- 172.21.15.46:0/3459712034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dd4101fd0 0x7f9dd41994f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 >> 172.21.15.46:0/3459712034 conn(0x7f9dd40fc3b0 msgr2=0x7f9dd40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 shutdown_connections 2024-05-08T23:02:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:11.976+0000 7f9ddbe00700 1 -- 172.21.15.46:0/3459712034 wait complete. 2024-05-08T23:02:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-05-08T23:02:11.994 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-05-08T23:02:11.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:11.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:02:12.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[91280]: fuse finished with error 0 and tester_r 0 2024-05-08T23:02:12.062 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:02:12.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:02:12.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:02:12.163 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T23:02:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:11 smithi086 ceph-mon[78610]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/791709141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3485763821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:02:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:11 smithi053 ceph-mon[77587]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/791709141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3485763821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:11 smithi046 ceph-mon[36898]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:12.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/791709141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3485763821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 -- 172.21.15.46:0/1320044377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc1042f0 msgr2=0x7fb9dc1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 --2- 172.21.15.46:0/1320044377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1066e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 -- 172.21.15.46:0/1320044377 shutdown_connections 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 --2- 172.21.15.46:0/1320044377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc106c20 0x7fb9dc109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 --2- 172.21.15.46:0/1320044377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 0x7fb9dc103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 --2- 172.21.15.46:0/1320044377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1066e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.420+0000 7fb9e1400700 1 -- 172.21.15.46:0/1320044377 >> 172.21.15.46:0/1320044377 conn(0x7fb9dc0fb310 msgr2=0x7fb9dc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.421+0000 7fb9e1400700 1 -- 172.21.15.46:0/1320044377 shutdown_connections 2024-05-08T23:02:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.421+0000 7fb9e1400700 1 -- 172.21.15.46:0/1320044377 wait complete. 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 Processor -- start 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 -- start start 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 0x7fb9dc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc106c20 0x7fb9dc19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19e8e0 con 0x7fb9dc1042f0 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19ea20 con 0x7fb9dc106c20 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9e1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19ebb0 con 0x7fb9dc1019c0 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54664/0 (socket says 172.21.15.46:54664) 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9d9e00700 1 -- 172.21.15.46:0/4234746969 learned_addr learned my addr 172.21.15.46:0/4234746969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9da800700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 0x7fb9dc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.422+0000 7fb9db200700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc106c20 0x7fb9dc19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 -- 172.21.15.46:0/4234746969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 msgr2=0x7fb9dc196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 0x7fb9dc196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 -- 172.21.15.46:0/4234746969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc106c20 msgr2=0x7fb9dc19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc106c20 0x7fb9dc19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 -- 172.21.15.46:0/4234746969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0095f0 con 0x7fb9dc1042f0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d9e00700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009810 tx=0x7fb9c800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c801c070 con 0x7fb9dc1042f0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c8009520 con 0x7fb9dc1042f0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9c800f7d0 con 0x7fb9dc1042f0 2024-05-08T23:02:12.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.423+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8003970 con 0x7fb9dc1042f0 2024-05-08T23:02:12.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.424+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc19f120 con 0x7fb9dc1042f0 2024-05-08T23:02:12.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.425+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc0fce90 con 0x7fb9dc1042f0 2024-05-08T23:02:12.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.427+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb9c8020030 con 0x7fb9dc1042f0 2024-05-08T23:02:12.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.428+0000 7fb9d3400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 0x7fb9c00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.428+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb9c80cddc0 con 0x7fb9dc1042f0 2024-05-08T23:02:12.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.428+0000 7fb9da800700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 0x7fb9c00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.429+0000 7fb9da800700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 0x7fb9c00a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc005fd0 tx=0x7fb9cc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:12.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.430+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb9c8097390 con 0x7fb9dc1042f0 2024-05-08T23:02:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.648+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9dc10c660 con 0x7fb9dc1042f0 2024-05-08T23:02:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.650+0000 7fb9d3400700 1 -- 172.21.15.46:0/4234746969 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7fb9c809a9b0 con 0x7fb9dc1042f0 2024-05-08T23:02:12.650 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 msgr2=0x7fb9c00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 0x7fb9c00a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc005fd0 tx=0x7fb9cc009310 comp rx=0 tx=0).stop 2024-05-08T23:02:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 msgr2=0x7fb9dc1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009810 tx=0x7fb9c800b790 comp rx=0 tx=0).stop 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 shutdown_connections 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc106c20 0x7fb9dc19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9dc1019c0 0x7fb9dc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb9c00a11a0 0x7fb9c00a3660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 --2- 172.21.15.46:0/4234746969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9dc1042f0 0x7fb9dc1972f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 >> 172.21.15.46:0/4234746969 conn(0x7fb9dc0fb310 msgr2=0x7fb9dc102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 shutdown_connections 2024-05-08T23:02:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.653+0000 7fb9e1400700 1 -- 172.21.15.46:0/4234746969 wait complete. 2024-05-08T23:02:12.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:02:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 -- 172.21.15.46:0/3810791384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8100e80 msgr2=0x7efdc81012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3810791384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8100e80 0x7efdc81012a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efdb80098f0 tx=0x7efdb8009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 -- 172.21.15.46:0/3810791384 shutdown_connections 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3810791384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8102990 0x7efdc81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3810791384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8100e80 0x7efdc81012a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3810791384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8101fd0 0x7efdc8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 -- 172.21.15.46:0/3810791384 >> 172.21.15.46:0/3810791384 conn(0x7efdc80fc410 msgr2=0x7efdc80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 -- 172.21.15.46:0/3810791384 shutdown_connections 2024-05-08T23:02:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.872+0000 7efdcdc00700 1 -- 172.21.15.46:0/3810791384 wait complete. 2024-05-08T23:02:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.873+0000 7efdcdc00700 1 Processor -- start 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 -- start start 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8101fd0 0x7efdc8197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8102990 0x7efdc819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819e940 con 0x7efdc8100e80 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819ea80 con 0x7efdc8102990 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdcdc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819ec10 con 0x7efdc8101fd0 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdc6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdc6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54680/0 (socket says 172.21.15.46:54680) 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdc6a00700 1 -- 172.21.15.46:0/3467262836 learned_addr learned my addr 172.21.15.46:0/3467262836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdc6000700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8101fd0 0x7efdc8197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.874+0000 7efdc7400700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8102990 0x7efdc819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 -- 172.21.15.46:0/3467262836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8101fd0 msgr2=0x7efdc8197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8101fd0 0x7efdc8197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 -- 172.21.15.46:0/3467262836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8102990 msgr2=0x7efdc819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8102990 0x7efdc819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 -- 172.21.15.46:0/3467262836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0009d00 con 0x7efdc8100e80 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdc6a00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009810 tx=0x7efdb800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb801c070 con 0x7efdc8100e80 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb800f840 con 0x7efdc8100e80 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb8009520 con 0x7efdc8100e80 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8003a90 con 0x7efdc8100e80 2024-05-08T23:02:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.875+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc819f180 con 0x7efdc8100e80 2024-05-08T23:02:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.876+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc81082f0 con 0x7efdc8100e80 2024-05-08T23:02:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.879+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efdb8020030 con 0x7efdc8100e80 2024-05-08T23:02:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.880+0000 7efdbf400700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 0x7efdac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.880+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efdb80cdfe0 con 0x7efdc8100e80 2024-05-08T23:02:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.880+0000 7efdc6000700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 0x7efdac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:12.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.881+0000 7efdc6000700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 0x7efdac0a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007c60 tx=0x7efdb40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:12.882+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efdb8097610 con 0x7efdc8100e80 2024-05-08T23:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.110+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdc810c670 con 0x7efdc8100e80 2024-05-08T23:02:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.112+0000 7efdbf400700 1 -- 172.21.15.46:0/3467262836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+23862 (secure 0 0 0) 0x7efdb809ac30 con 0x7efdc8100e80 2024-05-08T23:02:13.112 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:13.112 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:01:56.474426+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 msgr2=0x7efdac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 0x7efdac0a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007c60 tx=0x7efdb40073d0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 msgr2=0x7efdc8196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009810 tx=0x7efdb800b790 comp rx=0 tx=0).stop 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 shutdown_connections 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc8102990 0x7efdc819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efdac0a1120 0x7efdac0a35e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdc8100e80 0x7efdc8196e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 --2- 172.21.15.46:0/3467262836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8101fd0 0x7efdc8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 >> 172.21.15.46:0/3467262836 conn(0x7efdc80fc410 msgr2=0x7efdc8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 shutdown_connections 2024-05-08T23:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.114+0000 7efdcdc00700 1 -- 172.21.15.46:0/3467262836 wait complete. 2024-05-08T23:02:13.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:02:13.191 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3459712034' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:13.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4234746969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3459712034' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4234746969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3459712034' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:02:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4234746969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 -- 172.21.15.46:0/36601704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 msgr2=0x7f17d41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 --2- 172.21.15.46:0/36601704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41092b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00b0a0 tx=0x7f17cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 -- 172.21.15.46:0/36601704 shutdown_connections 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 --2- 172.21.15.46:0/36601704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41092b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 --2- 172.21.15.46:0/36601704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4100e80 0x7f17d41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 --2- 172.21.15.46:0/36601704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4101fd0 0x7f17d4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 -- 172.21.15.46:0/36601704 >> 172.21.15.46:0/36601704 conn(0x7f17d40fc3b0 msgr2=0x7f17d40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 -- 172.21.15.46:0/36601704 shutdown_connections 2024-05-08T23:02:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.336+0000 7f17da000700 1 -- 172.21.15.46:0/36601704 wait complete. 2024-05-08T23:02:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 Processor -- start 2024-05-08T23:02:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 -- start start 2024-05-08T23:02:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 0x7f17d41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41a0c60 con 0x7f17d4102990 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41a0da0 con 0x7f17d4101fd0 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.338+0000 7f17da000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41a0f30 con 0x7f17d4100e80 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:54858/0 (socket says 172.21.15.46:54858) 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 -- 172.21.15.46:0/3314338944 learned_addr learned my addr 172.21.15.46:0/3314338944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3e00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d2a00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 0x7f17d41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 -- 172.21.15.46:0/3314338944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 msgr2=0x7f17d41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 0x7f17d41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 -- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 msgr2=0x7f17d41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 -- 172.21.15.46:0/3314338944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc009d00 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d2a00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 0x7f17d41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.339+0000 7f17d3400700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17c400e9a0 tx=0x7f17c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.340+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4009790 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.340+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c400cbd0 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.340+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d41a1210 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.340+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4018680 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.340+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d41a1720 con 0x7f17d4100e80 2024-05-08T23:02:13.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.341+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d4067d10 con 0x7f17d4100e80 2024-05-08T23:02:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.343+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f17c400cd40 con 0x7f17d4100e80 2024-05-08T23:02:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.344+0000 7f17cbe00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 0x7f17b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.344+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17c4014070 con 0x7f17d4100e80 2024-05-08T23:02:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.344+0000 7f17d2a00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 0x7f17b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.345+0000 7f17d2a00700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 0x7f17b80a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009cb0 tx=0x7f17c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.347+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f17c40963b0 con 0x7f17d4100e80 2024-05-08T23:02:13.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.611+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17d410c980 con 0x7f17d4100e80 2024-05-08T23:02:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.948+0000 7f17cbe00700 1 -- 172.21.15.46:0/3314338944 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f17c40999d0 con 0x7f17d4100e80 2024-05-08T23:02:13.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 msgr2=0x7f17b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 0x7f17b80a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009cb0 tx=0x7f17c0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 msgr2=0x7f17d4199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17c400e9a0 tx=0x7f17c400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 shutdown_connections 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f17b80a1120 0x7f17b80a35e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d4102990 0x7f17d41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17d4100e80 0x7f17d4199180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 --2- 172.21.15.46:0/3314338944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17d4101fd0 0x7f17d41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 >> 172.21.15.46:0/3314338944 conn(0x7f17d40fc3b0 msgr2=0x7f17d40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 shutdown_connections 2024-05-08T23:02:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:13.951+0000 7f17da000700 1 -- 172.21.15.46:0/3314338944 wait complete. 2024-05-08T23:02:13.971 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T23:02:13.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-08T23:02:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 -- 172.21.15.46:0/3284482979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c8104c30 msgr2=0x7f68c810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 --2- 172.21.15.46:0/3284482979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c8104c30 0x7f68c810b4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68c000b0a0 tx=0x7f68c000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 -- 172.21.15.46:0/3284482979 shutdown_connections 2024-05-08T23:02:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 --2- 172.21.15.46:0/3284482979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c8104c30 0x7f68c810b4e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 --2- 172.21.15.46:0/3284482979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c81030a0 0x7f68c81034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 --2- 172.21.15.46:0/3284482979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 -- 172.21.15.46:0/3284482979 >> 172.21.15.46:0/3284482979 conn(0x7f68c80fe610 msgr2=0x7f68c8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:14.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 -- 172.21.15.46:0/3284482979 shutdown_connections 2024-05-08T23:02:14.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.173+0000 7f68d0000700 1 -- 172.21.15.46:0/3284482979 wait complete. 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.174+0000 7f68d0000700 1 Processor -- start 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.174+0000 7f68d0000700 1 -- start start 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.174+0000 7f68d0000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 0x7f68c8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.174+0000 7f68d0000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68d0000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 0x7f68c81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68d0000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81a0c50 con 0x7f68c8104c30 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68d0000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81a0d90 con 0x7f68c8104290 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68d0000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81a0f20 con 0x7f68c81030a0 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ccc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 0x7f68c8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ccc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49712/0 (socket says 172.21.15.46:49712) 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ccc00700 1 -- 172.21.15.46:0/2700069258 learned_addr learned my addr 172.21.15.46:0/2700069258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ce000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 0x7f68c81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 -- 172.21.15.46:0/2700069258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 msgr2=0x7f68c8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 -- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 msgr2=0x7f68c81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 0x7f68c81a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68cd600700 1 -- 172.21.15.46:0/2700069258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009d00 con 0x7f68c81030a0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ce000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 0x7f68c81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.175+0000 7f68ccc00700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68cd600700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 0x7f68c8199120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e9c0 tx=0x7f68bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc004220 con 0x7f68c81030a0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68bc004380 con 0x7f68c81030a0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc003be0 con 0x7f68c81030a0 2024-05-08T23:02:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c81a1200 con 0x7f68c81030a0 2024-05-08T23:02:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.176+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81a16c0 con 0x7f68c81030a0 2024-05-08T23:02:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.177+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c810a450 con 0x7f68c81030a0 2024-05-08T23:02:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.179+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f68bc003d40 con 0x7f68c81030a0 2024-05-08T23:02:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.180+0000 7f68c6000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 0x7f68b00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.180+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f68bc014070 con 0x7f68c81030a0 2024-05-08T23:02:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.180+0000 7f68ccc00700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 0x7f68b00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.181+0000 7f68ccc00700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 0x7f68b00a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68b4011590 tx=0x7f68b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.184+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68bc096320 con 0x7f68c81030a0 2024-05-08T23:02:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:13 smithi086 ceph-mon[78610]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3467262836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3314338944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:13 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:13 smithi053 ceph-mon[77587]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3467262836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3314338944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:13 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:13 smithi046 ceph-mon[36898]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T23:02:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3467262836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3314338944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:13 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:02:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.449+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f68c802cda0 con 0x7f68c81030a0 2024-05-08T23:02:14.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.631+0000 7f68c6000700 1 -- 172.21.15.46:0/2700069258 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f68bc099940 con 0x7f68c81030a0 2024-05-08T23:02:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.634+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 msgr2=0x7f68b00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 0x7f68b00a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68b4011590 tx=0x7f68b4009260 comp rx=0 tx=0).stop 2024-05-08T23:02:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 msgr2=0x7f68c8199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 0x7f68c8199120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e9c0 tx=0x7f68bc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 shutdown_connections 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f68b00a1120 0x7f68b00a35e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8104c30 0x7f68c81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c81030a0 0x7f68c8199120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 --2- 172.21.15.46:0/2700069258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68c8104290 0x7f68c8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 >> 172.21.15.46:0/2700069258 conn(0x7f68c80fe610 msgr2=0x7f68c8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.635+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 shutdown_connections 2024-05-08T23:02:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.636+0000 7f68d0000700 1 -- 172.21.15.46:0/2700069258 wait complete. 2024-05-08T23:02:14.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 -- 172.21.15.46:0/2293398515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c104110 msgr2=0x7f103c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 --2- 172.21.15.46:0/2293398515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c104110 0x7f103c106500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1028009990 tx=0x7f1028009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 -- 172.21.15.46:0/2293398515 shutdown_connections 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 --2- 172.21.15.46:0/2293398515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c106a40 0x7f103c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 --2- 172.21.15.46:0/2293398515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 --2- 172.21.15.46:0/2293398515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c104110 0x7f103c106500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 -- 172.21.15.46:0/2293398515 >> 172.21.15.46:0/2293398515 conn(0x7f103c0fb130 msgr2=0x7f103c0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 -- 172.21.15.46:0/2293398515 shutdown_connections 2024-05-08T23:02:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.860+0000 7f103b200700 1 -- 172.21.15.46:0/2293398515 wait complete. 2024-05-08T23:02:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.861+0000 7f103b200700 1 Processor -- start 2024-05-08T23:02:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 -- start start 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 0x7f103c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 0x7f103c1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a0860 con 0x7f103c106a40 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a09d0 con 0x7f103c1017e0 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a0b90 con 0x7f103c104110 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f1039e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f1039400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 0x7f103c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f103a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 0x7f103c1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f1039400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 0x7f103c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:54888/0 (socket says 172.21.15.46:54888) 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f1039e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49730/0 (socket says 172.21.15.46:49730) 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.862+0000 7f1039400700 1 -- 172.21.15.46:0/3299667048 learned_addr learned my addr 172.21.15.46:0/3299667048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 -- 172.21.15.46:0/3299667048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 msgr2=0x7f103c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 0x7f103c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 -- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 msgr2=0x7f103c1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 0x7f103c1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 -- 172.21.15.46:0/3299667048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280095f0 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f103a800700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 0x7f103c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1039e00700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c00c9c0 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102c010430 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c1a0e70 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.863+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c018680 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.864+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c1a1290 con 0x7f103c1017e0 2024-05-08T23:02:14.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.864+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c1085c0 con 0x7f103c1017e0 2024-05-08T23:02:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.867+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f102c010860 con 0x7f103c1017e0 2024-05-08T23:02:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.868+0000 7f1032a00700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 0x7f10200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.868+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f102c014070 con 0x7f103c1017e0 2024-05-08T23:02:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.868+0000 7f1039400700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 0x7f10200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.869+0000 7f1039400700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 0x7f10200a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f102800b2e0 tx=0x7f102800b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:14.870+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f102c0962e0 con 0x7f103c1017e0 2024-05-08T23:02:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.140+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f103c04ec60 con 0x7f103c1017e0 2024-05-08T23:02:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:02:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:02:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2700069258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:02:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:02:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T23:02:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-08T23:02:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-08T23:02:15.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:14 smithi086 ceph-mon[78610]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2700069258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:02:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:02:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T23:02:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-08T23:02:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-08T23:02:15.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:14 smithi053 ceph-mon[77587]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:02:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:02:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:02:15.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2700069258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-08T23:02:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:14 smithi046 ceph-mon[36898]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:02:15.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.636+0000 7f1032a00700 1 -- 172.21.15.46:0/3299667048 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f102c099900 con 0x7f103c1017e0 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.639+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 msgr2=0x7f10200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.639+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 0x7f10200a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f102800b2e0 tx=0x7f102800b750 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.639+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 msgr2=0x7f103c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.639+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 shutdown_connections 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f10200a11c0 0x7f10200a3680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f103c106a40 0x7f103c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c1017e0 0x7f103c198d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 --2- 172.21.15.46:0/3299667048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c104110 0x7f103c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 >> 172.21.15.46:0/3299667048 conn(0x7f103c0fb130 msgr2=0x7f103c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 shutdown_connections 2024-05-08T23:02:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.640+0000 7f103b200700 1 -- 172.21.15.46:0/3299667048 wait complete. 2024-05-08T23:02:15.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 -- 172.21.15.46:0/3280362675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0101fd0 msgr2=0x7f51d0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 --2- 172.21.15.46:0/3280362675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0101fd0 0x7f51d0102450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009990 tx=0x7f51bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 -- 172.21.15.46:0/3280362675 shutdown_connections 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 --2- 172.21.15.46:0/3280362675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 --2- 172.21.15.46:0/3280362675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0100e80 0x7f51d01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 --2- 172.21.15.46:0/3280362675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0101fd0 0x7f51d0102450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 -- 172.21.15.46:0/3280362675 >> 172.21.15.46:0/3280362675 conn(0x7f51d00fc3d0 msgr2=0x7f51d00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 -- 172.21.15.46:0/3280362675 shutdown_connections 2024-05-08T23:02:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.867+0000 7f51d7600700 1 -- 172.21.15.46:0/3280362675 wait complete. 2024-05-08T23:02:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.868+0000 7f51d7600700 1 Processor -- start 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 -- start start 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0100e80 0x7f51d0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0101fd0 0x7f51d01995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01a0b70 con 0x7f51d0102990 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01a0cb0 con 0x7f51d0100e80 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d7600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01a0e40 con 0x7f51d0101fd0 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d5600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d5600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54750/0 (socket says 172.21.15.46:54750) 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d5600700 1 -- 172.21.15.46:0/4064482612 learned_addr learned my addr 172.21.15.46:0/4064482612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51d4c00700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0100e80 0x7f51d0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.869+0000 7f51cfe00700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0101fd0 0x7f51d01995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 -- 172.21.15.46:0/4064482612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0101fd0 msgr2=0x7f51d01995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0101fd0 0x7f51d01995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 -- 172.21.15.46:0/4064482612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0100e80 msgr2=0x7f51d0199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0100e80 0x7f51d0199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 -- 172.21.15.46:0/4064482612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009c40 con 0x7f51d0102990 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d5600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c370 tx=0x7f51c800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8017070 con 0x7f51d0102990 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c800f040 con 0x7f51d0102990 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc0095f0 con 0x7f51d0102990 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800e050 con 0x7f51d0102990 2024-05-08T23:02:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.870+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d01a1440 con 0x7f51d0102990 2024-05-08T23:02:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.871+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d0067d10 con 0x7f51d0102990 2024-05-08T23:02:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.873+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f51c800cb60 con 0x7f51d0102990 2024-05-08T23:02:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.875+0000 7f51cd600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 0x7f51b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.875+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f51c80cc4b0 con 0x7f51d0102990 2024-05-08T23:02:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.875+0000 7f51d4c00700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 0x7f51b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.876+0000 7f51d4c00700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 0x7f51b80a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009c60 tx=0x7f51c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:15.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:15.879+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51c80959f0 con 0x7f51d0102990 2024-05-08T23:02:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.148+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f51d010c6f0 con 0x7f51d0102990 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3299667048' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:active 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-08T23:02:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:02:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:15 smithi086 ceph-mon[78610]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:02:16.261 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:02:15.920+0000 7fb1a990b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T23:02:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3299667048' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:active 2024-05-08T23:02:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:02:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:02:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-08T23:02:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:02:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:15 smithi053 ceph-mon[77587]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:02:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3299667048' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] up:active 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-05-08T23:02:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:15 smithi046 ceph-mon[36898]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T23:02:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.643+0000 7f51cd600700 1 -- 172.21.15.46:0/4064482612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f51c8099010 con 0x7f51d0102990 2024-05-08T23:02:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 msgr2=0x7f51b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 0x7f51b80a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009c60 tx=0x7f51c4009260 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 msgr2=0x7f51d01a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c370 tx=0x7f51c800c680 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 shutdown_connections 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f51b80a1120 0x7f51b80a35e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51d0102990 0x7f51d01a05a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d0100e80 0x7f51d0199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 --2- 172.21.15.46:0/4064482612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51d0101fd0 0x7f51d01995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.646+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 >> 172.21.15.46:0/4064482612 conn(0x7f51d00fc3d0 msgr2=0x7f51d00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.647+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 shutdown_connections 2024-05-08T23:02:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.647+0000 7f51d7600700 1 -- 172.21.15.46:0/4064482612 wait complete. 2024-05-08T23:02:16.673 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-08T23:02:16.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T23:02:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.879+0000 7fcaad000700 1 -- 172.21.15.46:0/239679377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 msgr2=0x7fcaa8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.879+0000 7fcaad000700 1 --2- 172.21.15.46:0/239679377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa8109120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009fc0 tx=0x7fcaa000b330 comp rx=0 tx=0).stop 2024-05-08T23:02:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 -- 172.21.15.46:0/239679377 shutdown_connections 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 --2- 172.21.15.46:0/239679377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa8109120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 --2- 172.21.15.46:0/239679377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 --2- 172.21.15.46:0/239679377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8101ed0 0x7fcaa8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 -- 172.21.15.46:0/239679377 >> 172.21.15.46:0/239679377 conn(0x7fcaa80fc210 msgr2=0x7fcaa80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 -- 172.21.15.46:0/239679377 shutdown_connections 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.880+0000 7fcaad000700 1 -- 172.21.15.46:0/239679377 wait complete. 2024-05-08T23:02:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 Processor -- start 2024-05-08T23:02:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 -- start start 2024-05-08T23:02:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8101ed0 0x7fcaa8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819e7c0 con 0x7fcaa8102870 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819e930 con 0x7fcaa8101ed0 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaad000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819eaf0 con 0x7fcaa8100ce0 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaa7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.881+0000 7fcaa7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:54918/0 (socket says 172.21.15.46:54918) 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 -- 172.21.15.46:0/2182256940 learned_addr learned my addr 172.21.15.46:0/2182256940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa6a00700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8101ed0 0x7fcaa8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 -- 172.21.15.46:0/2182256940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8101ed0 msgr2=0x7fcaa8197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8101ed0 0x7fcaa8197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 -- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 msgr2=0x7fcaa819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 -- 172.21.15.46:0/2182256940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009c40 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7e00700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fcaa7400700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9f0 tx=0x7fca9800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.882+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98004220 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.883+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa819edd0 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.883+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca98004380 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.883+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98003be0 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.883+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa819f250 con 0x7fcaa8100ce0 2024-05-08T23:02:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.884+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa8107fc0 con 0x7fcaa8100ce0 2024-05-08T23:02:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.886+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fca98003d40 con 0x7fcaa8100ce0 2024-05-08T23:02:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.887+0000 7fca9fe00700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 0x7fca8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.887+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fca98014070 con 0x7fcaa8100ce0 2024-05-08T23:02:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.887+0000 7fcaa6a00700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 0x7fca8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.888+0000 7fcaa6a00700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 0x7fca8c0a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca94009c60 tx=0x7fca94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:16.889+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca980961e0 con 0x7fcaa8100ce0 2024-05-08T23:02:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.145+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa802ceb0 con 0x7fca8c0a11c0 2024-05-08T23:02:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.174+0000 7fca9fe00700 1 -- 172.21.15.46:0/2182256940 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcaa802ceb0 con 0x7fca8c0a11c0 2024-05-08T23:02:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 msgr2=0x7fca8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 0x7fca8c0a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca94009c60 tx=0x7fca94009260 comp rx=0 tx=0).stop 2024-05-08T23:02:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 msgr2=0x7fcaa8196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9f0 tx=0x7fca9800ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 shutdown_connections 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fca8c0a11c0 0x7fca8c0a3680 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaa8102870 0x7fcaa819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa8100ce0 0x7fcaa8196cd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 --2- 172.21.15.46:0/2182256940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa8101ed0 0x7fcaa8197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 >> 172.21.15.46:0/2182256940 conn(0x7fcaa80fc210 msgr2=0x7fcaa8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 shutdown_connections 2024-05-08T23:02:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.177+0000 7fcaad000700 1 -- 172.21.15.46:0/2182256940 wait complete. 2024-05-08T23:02:17.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:active 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:02:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-08T23:02:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:16 smithi086 ceph-mon[78610]: pgmap v252: 129 pgs: 129 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T23:02:17.253 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:02:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:02:17.152+0000 7f298d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:02:17.253 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:02:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:02:17.152+0000 7f298d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:02:17.253 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:02:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:02:17.152+0000 7f298d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:02:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:02:17.152+0000 7f298d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:02:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:02:17.152+0000 7f298d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:active 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T23:02:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-08T23:02:17.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:16 smithi046 ceph-mon[36898]: pgmap v252: 129 pgs: 129 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T23:02:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:02:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] up:active 2024-05-08T23:02:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4064482612' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:02:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-08T23:02:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-08T23:02:17.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-08T23:02:17.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:16 smithi053 ceph-mon[77587]: pgmap v252: 129 pgs: 129 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T23:02:17.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 -- 172.21.15.46:0/4056736796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 msgr2=0x7fa88c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 --2- 172.21.15.46:0/4056736796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c073430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009990 tx=0x7fa87c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 -- 172.21.15.46:0/4056736796 shutdown_connections 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 --2- 172.21.15.46:0/4056736796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 --2- 172.21.15.46:0/4056736796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c073430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 --2- 172.21.15.46:0/4056736796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 -- 172.21.15.46:0/4056736796 >> 172.21.15.46:0/4056736796 conn(0x7fa88c0fc3b0 msgr2=0x7fa88c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 -- 172.21.15.46:0/4056736796 shutdown_connections 2024-05-08T23:02:17.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.400+0000 7fa890a00700 1 -- 172.21.15.46:0/4056736796 wait complete. 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 Processor -- start 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 -- start start 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c1a0af0 con 0x7fa88c074fe0 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c1a0c30 con 0x7fa88c108d40 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.401+0000 7fa890a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c1a0dc0 con 0x7fa88c073a00 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:54962/0 (socket says 172.21.15.46:54962) 2024-05-08T23:02:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54812/0 (socket says 172.21.15.46:54812) 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889e00700 1 -- 172.21.15.46:0/1626040697 learned_addr learned my addr 172.21.15.46:0/1626040697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa88a800700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 -- 172.21.15.46:0/1626040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 msgr2=0x7fa88c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 -- 172.21.15.46:0/1626040697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 msgr2=0x7fa88c1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 -- 172.21.15.46:0/1626040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0095f0 con 0x7fa88c074fe0 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa88a800700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889e00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.402+0000 7fa889400700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9c0 tx=0x7fa87800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.403+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cbd0 con 0x7fa88c074fe0 2024-05-08T23:02:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.403+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c1a10a0 con 0x7fa88c074fe0 2024-05-08T23:02:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.403+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87800eeb0 con 0x7fa88c074fe0 2024-05-08T23:02:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.403+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878003c70 con 0x7fa88c074fe0 2024-05-08T23:02:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.403+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c1a15b0 con 0x7fa88c074fe0 2024-05-08T23:02:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.404+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c107b00 con 0x7fa88c074fe0 2024-05-08T23:02:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.406+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa878003dd0 con 0x7fa88c074fe0 2024-05-08T23:02:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.408+0000 7fa882a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 0x7fa8700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.408+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa878014070 con 0x7fa88c074fe0 2024-05-08T23:02:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.408+0000 7fa889e00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 0x7fa8700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.408+0000 7fa889e00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 0x7fa8700a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00b2e0 tx=0x7fa87c00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.410+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa8780964a0 con 0x7fa88c074fe0 2024-05-08T23:02:17.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.675+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fa88c02ce10 con 0x7fa8700a1120 2024-05-08T23:02:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.699+0000 7fa882a00700 1 -- 172.21.15.46:0/1626040697 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa88c02ce10 con 0x7fa8700a1120 2024-05-08T23:02:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 msgr2=0x7fa8700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 0x7fa8700a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00b2e0 tx=0x7fa87c00b750 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 msgr2=0x7fa88c199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9c0 tx=0x7fa87800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 shutdown_connections 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa88c108d40 0x7fa88c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa88c073a00 0x7fa88c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8700a1120 0x7fa8700a35e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 --2- 172.21.15.46:0/1626040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa88c074fe0 0x7fa88c199550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 >> 172.21.15.46:0/1626040697 conn(0x7fa88c0fc3b0 msgr2=0x7fa88c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.701+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 shutdown_connections 2024-05-08T23:02:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.702+0000 7fa890a00700 1 -- 172.21.15.46:0/1626040697 wait complete. 2024-05-08T23:02:17.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T23:02:17.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 -- 172.21.15.46:0/1811822206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec100e80 msgr2=0x7f8bec1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/1811822206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec100e80 0x7f8bec1012a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009990 tx=0x7f8bdc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 -- 172.21.15.46:0/1811822206 shutdown_connections 2024-05-08T23:02:17.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/1811822206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec102990 0x7f8bec1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/1811822206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec100e80 0x7f8bec1012a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/1811822206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec101fd0 0x7f8bec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 -- 172.21.15.46:0/1811822206 >> 172.21.15.46:0/1811822206 conn(0x7f8bec0fc3b0 msgr2=0x7f8bec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 -- 172.21.15.46:0/1811822206 shutdown_connections 2024-05-08T23:02:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.926+0000 7f8bf2e00700 1 -- 172.21.15.46:0/1811822206 wait complete. 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 Processor -- start 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 -- start start 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec101fd0 0x7f8bec199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec102990 0x7f8bec1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0b30 con 0x7f8bec102990 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0c70 con 0x7f8bec100e80 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.927+0000 7f8bf2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1a0e00 con 0x7f8bec101fd0 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49818/0 (socket says 172.21.15.46:49818) 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 -- 172.21.15.46:0/2557897699 learned_addr learned my addr 172.21.15.46:0/2557897699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bf0e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec102990 0x7f8bec1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8beb400700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec101fd0 0x7f8bec199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 -- 172.21.15.46:0/2557897699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec101fd0 msgr2=0x7f8bec199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec101fd0 0x7f8bec199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 -- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec102990 msgr2=0x7f8bec1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec102990 0x7f8bec1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 -- 172.21.15.46:0/2557897699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0095f0 con 0x7f8bec100e80 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.928+0000 7f8bebe00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005910 tx=0x7f8bdc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.929+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc01d070 con 0x7f8bec100e80 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.929+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec1a1080 con 0x7f8bec100e80 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.929+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc021410 con 0x7f8bec100e80 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.929+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc017650 con 0x7f8bec100e80 2024-05-08T23:02:17.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.929+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec1a1500 con 0x7f8bec100e80 2024-05-08T23:02:17.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.930+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec067d10 con 0x7f8bec100e80 2024-05-08T23:02:17.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.932+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8bdc021580 con 0x7f8bec100e80 2024-05-08T23:02:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.933+0000 7f8be8c00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 0x7f8bd00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.933+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8bdc0ce660 con 0x7f8bec100e80 2024-05-08T23:02:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.933+0000 7f8beb400700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 0x7f8bd00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.934+0000 7f8beb400700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 0x7f8bd00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009c60 tx=0x7f8bd8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:17.936+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bdc097b10 con 0x7f8bec100e80 2024-05-08T23:02:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.190+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bec0748f0 con 0x7f8bd00a1120 2024-05-08T23:02:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.195+0000 7f8be8c00700 1 -- 172.21.15.46:0/2557897699 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f8bec0748f0 con 0x7f8bd00a1120 2024-05-08T23:02:18.195 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/1b539da7-bc32-47fd-a928-4d19d083059b 2024-05-08T23:02:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 msgr2=0x7f8bd00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 0x7f8bd00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009c60 tx=0x7f8bd8009260 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 msgr2=0x7f8bec199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005910 tx=0x7f8bdc005740 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 shutdown_connections 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8bd00a1120 0x7f8bd00a35e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bec102990 0x7f8bec1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bec100e80 0x7f8bec199000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 --2- 172.21.15.46:0/2557897699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bec101fd0 0x7f8bec199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 >> 172.21.15.46:0/2557897699 conn(0x7f8bec0fc3b0 msgr2=0x7f8bec0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.197+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 shutdown_connections 2024-05-08T23:02:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.198+0000 7f8bf2e00700 1 -- 172.21.15.46:0/2557897699 wait complete. 2024-05-08T23:02:18.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T23:02:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 -- 172.21.15.46:0/2125566229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc101fd0 msgr2=0x7f09fc102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 --2- 172.21.15.46:0/2125566229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc101fd0 0x7f09fc102450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009990 tx=0x7f09e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 -- 172.21.15.46:0/2125566229 shutdown_connections 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 --2- 172.21.15.46:0/2125566229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc102990 0x7f09fc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 --2- 172.21.15.46:0/2125566229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc100e80 0x7f09fc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 --2- 172.21.15.46:0/2125566229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc101fd0 0x7f09fc102450 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 -- 172.21.15.46:0/2125566229 >> 172.21.15.46:0/2125566229 conn(0x7f09fc0fc3b0 msgr2=0x7f09fc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 -- 172.21.15.46:0/2125566229 shutdown_connections 2024-05-08T23:02:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.420+0000 7f0a03000700 1 -- 172.21.15.46:0/2125566229 wait complete. 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 Processor -- start 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 -- start start 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc101fd0 0x7f09fc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc102990 0x7f09fc1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0b50 con 0x7f09fc100e80 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0c90 con 0x7f09fc101fd0 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.421+0000 7f0a03000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0e20 con 0x7f09fc102990 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54846/0 (socket says 172.21.15.46:54846) 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 -- 172.21.15.46:0/3815637884 learned_addr learned my addr 172.21.15.46:0/3815637884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fb400700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc101fd0 0x7f09fc199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f0a01000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc102990 0x7f09fc1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 -- 172.21.15.46:0/3815637884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc102990 msgr2=0x7f09fc1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc102990 0x7f09fc1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 -- 172.21.15.46:0/3815637884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc101fd0 msgr2=0x7f09fc199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc101fd0 0x7f09fc199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 -- 172.21.15.46:0/3815637884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4009c40 con 0x7f09fc100e80 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.422+0000 7f09fbe00700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e9a0 tx=0x7f09ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.423+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec009790 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.423+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec00ce80 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.423+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec018680 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.423+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80095f0 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.423+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc1a1430 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.424+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc067d10 con 0x7f09fc100e80 2024-05-08T23:02:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.426+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f09ec010430 con 0x7f09fc100e80 2024-05-08T23:02:18.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.427+0000 7f09f8c00700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 0x7f09e00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.427+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f09ec014070 con 0x7f09fc100e80 2024-05-08T23:02:18.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.427+0000 7f09fb400700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 0x7f09e00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.428+0000 7f09fb400700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 0x7f09e00a3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b2e0 tx=0x7f09e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.431+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09ec052d60 con 0x7f09fc100e80 2024-05-08T23:02:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.697+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f09fc108fa0 con 0x7f09e00a11a0 2024-05-08T23:02:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.723+0000 7f09f8c00700 1 -- 172.21.15.46:0/3815637884 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f09fc108fa0 con 0x7f09e00a11a0 2024-05-08T23:02:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 msgr2=0x7f09e00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 0x7f09e00a3660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b2e0 tx=0x7f09e8005fb0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 msgr2=0x7f09fc199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e9a0 tx=0x7f09ec00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 shutdown_connections 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc102990 0x7f09fc1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09e00a11a0 0x7f09e00a3660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc100e80 0x7f09fc199020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 --2- 172.21.15.46:0/3815637884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09fc101fd0 0x7f09fc199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 >> 172.21.15.46:0/3815637884 conn(0x7f09fc0fc3b0 msgr2=0x7f09fc107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 shutdown_connections 2024-05-08T23:02:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.726+0000 7f0a03000700 1 -- 172.21.15.46:0/3815637884 wait complete. 2024-05-08T23:02:18.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T23:02:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 -- 172.21.15.46:0/667520737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398074fe0 msgr2=0x7f5398073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 --2- 172.21.15.46:0/667520737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398074fe0 0x7f5398073430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5388009990 tx=0x7f5388009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 -- 172.21.15.46:0/667520737 shutdown_connections 2024-05-08T23:02:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 --2- 172.21.15.46:0/667520737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398108d40 0x7f539810b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 --2- 172.21.15.46:0/667520737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398074fe0 0x7f5398073430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 --2- 172.21.15.46:0/667520737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 0x7f5398073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 -- 172.21.15.46:0/667520737 >> 172.21.15.46:0/667520737 conn(0x7f53980fc3b0 msgr2=0x7f53980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 -- 172.21.15.46:0/667520737 shutdown_connections 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.948+0000 7f539de00700 1 -- 172.21.15.46:0/667520737 wait complete. 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 Processor -- start 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 -- start start 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 0x7f5398199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53981a0b40 con 0x7f5398108d40 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53981a0c80 con 0x7f5398074fe0 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.949+0000 7f539de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53981a0e10 con 0x7f5398073a00 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5396000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5396000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49864/0 (socket says 172.21.15.46:49864) 2024-05-08T23:02:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54860/0 (socket says 172.21.15.46:54860) 2024-05-08T23:02:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5396000700 1 -- 172.21.15.46:0/3811191691 learned_addr learned my addr 172.21.15.46:0/3811191691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5396a00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 0x7f5398199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 -- 172.21.15.46:0/3811191691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 msgr2=0x7f5398199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 0x7f5398199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 -- 172.21.15.46:0/3811191691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 msgr2=0x7f53981995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 -- 172.21.15.46:0/3811191691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880095f0 con 0x7f5398108d40 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5396000700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.950+0000 7f5397400700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5390007f10 tx=0x7f539000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.951+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390009ea0 con 0x7f5398108d40 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.951+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539000f040 con 0x7f5398108d40 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.951+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53981a10f0 con 0x7f5398108d40 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.951+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390004350 con 0x7f5398108d40 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.951+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53981a1650 con 0x7f5398108d40 2024-05-08T23:02:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.952+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5398107b00 con 0x7f5398108d40 2024-05-08T23:02:18.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.954+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f539000db20 con 0x7f5398108d40 2024-05-08T23:02:18.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.955+0000 7f539ca00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 0x7f537c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:18.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.955+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53900cc450 con 0x7f5398108d40 2024-05-08T23:02:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.955+0000 7f5396a00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 0x7f537c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.956+0000 7f5396a00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 0x7f537c0a3660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f538800b5c0 tx=0x7f5388003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:18.959+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5390095870 con 0x7f5398108d40 2024-05-08T23:02:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:18.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T23:02:18.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:active 2024-05-08T23:02:18.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T23:02:18.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:02:18.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: Cluster is now healthy 2024-05-08T23:02:18.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T23:02:18.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T23:02:18.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:18 smithi086 ceph-mon[78610]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-08T23:02:19.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-08T23:02:19.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:active 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:02:19.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: Cluster is now healthy 2024-05-08T23:02:19.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T23:02:19.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T23:02:19.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:18 smithi053 ceph-mon[77587]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T23:02:19.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] up:active 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: Cluster is now healthy 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T23:02:19.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:18 smithi046 ceph-mon[36898]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.214+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f539802ce10 con 0x7f537c0a11a0 2024-05-08T23:02:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.218+0000 7f539ca00700 1 -- 172.21.15.46:0/3811191691 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f539802ce10 con 0x7f537c0a11a0 2024-05-08T23:02:19.218 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/d6af9f64-4946-4177-ac36-39bf5e3d4b15 2024-05-08T23:02:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 msgr2=0x7f537c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 0x7f537c0a3660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f538800b5c0 tx=0x7f5388003790 comp rx=0 tx=0).stop 2024-05-08T23:02:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 msgr2=0x7f53981a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5390007f10 tx=0x7f539000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 shutdown_connections 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f537c0a11a0 0x7f537c0a3660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5398108d40 0x7f53981a0570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5398073a00 0x7f5398199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 --2- 172.21.15.46:0/3811191691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5398074fe0 0x7f53981995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 >> 172.21.15.46:0/3811191691 conn(0x7f53980fc3b0 msgr2=0x7f5398104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 shutdown_connections 2024-05-08T23:02:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.221+0000 7f539de00700 1 -- 172.21.15.46:0/3811191691 wait complete. 2024-05-08T23:02:19.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:02:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 -- 172.21.15.46:0/3544777233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc100e80 msgr2=0x7fa0bc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/3544777233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc100e80 0x7fa0bc1012a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009990 tx=0x7fa0ac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 -- 172.21.15.46:0/3544777233 shutdown_connections 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/3544777233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc102990 0x7fa0bc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/3544777233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc100e80 0x7fa0bc1012a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/3544777233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 -- 172.21.15.46:0/3544777233 >> 172.21.15.46:0/3544777233 conn(0x7fa0bc0fc410 msgr2=0x7fa0bc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 -- 172.21.15.46:0/3544777233 shutdown_connections 2024-05-08T23:02:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.439+0000 7fa0c1c00700 1 -- 172.21.15.46:0/3544777233 wait complete. 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.440+0000 7fa0c1c00700 1 Processor -- start 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.440+0000 7fa0c1c00700 1 -- start start 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 0x7fa0bc198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc102990 0x7fa0bc1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc1a0b10 con 0x7fa0bc101fd0 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc1a0c50 con 0x7fa0bc100e80 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0c1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc1a0de0 con 0x7fa0bc102990 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0ba000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 0x7fa0bc198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0ba000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54884/0 (socket says 172.21.15.46:54884) 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc102990 0x7fa0bc1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0ba000700 1 -- 172.21.15.46:0/393675637 learned_addr learned my addr 172.21.15.46:0/393675637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 -- 172.21.15.46:0/393675637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc102990 msgr2=0x7fa0bc1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc102990 0x7fa0bc1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 -- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 msgr2=0x7fa0bc199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0baa00700 1 -- 172.21.15.46:0/393675637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4009c40 con 0x7fa0bc100e80 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.441+0000 7fa0ba000700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0baa00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 0x7fa0bc198fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000c00 tx=0x7fa0ac009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac01d070 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0095f0 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc1a13a0 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac022470 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.442+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac017650 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.443+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc067d10 con 0x7fa0bc100e80 2024-05-08T23:02:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.445+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa0ac022870 con 0x7fa0bc100e80 2024-05-08T23:02:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.447+0000 7fa0b3400700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 0x7fa0a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.447+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0ac0cdfe0 con 0x7fa0bc100e80 2024-05-08T23:02:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.447+0000 7fa0ba000700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 0x7fa0a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.447+0000 7fa0ba000700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 0x7fa0a00a35e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011590 tx=0x7fa0a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.449+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0ac0974b0 con 0x7fa0bc100e80 2024-05-08T23:02:19.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.675+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa0ac097880 con 0x7fa0bc100e80 2024-05-08T23:02:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.725+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0bc10c640 con 0x7fa0bc100e80 2024-05-08T23:02:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.726+0000 7fa0b3400700 1 -- 172.21.15.46:0/393675637 <== mon.2 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fa0ac09af40 con 0x7fa0bc100e80 2024-05-08T23:02:19.726 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:19.726 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 msgr2=0x7fa0a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 0x7fa0a00a35e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011590 tx=0x7fa0a8009260 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 msgr2=0x7fa0bc198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 0x7fa0bc198fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000c00 tx=0x7fa0ac009580 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 shutdown_connections 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc102990 0x7fa0bc1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0bc100e80 0x7fa0bc198fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa0a00a1120 0x7fa0a00a35e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 --2- 172.21.15.46:0/393675637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0bc101fd0 0x7fa0bc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.728+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 >> 172.21.15.46:0/393675637 conn(0x7fa0bc0fc410 msgr2=0x7fa0bc0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.729+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 shutdown_connections 2024-05-08T23:02:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.729+0000 7fa0c1c00700 1 -- 172.21.15.46:0/393675637 wait complete. 2024-05-08T23:02:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:02:19.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 -- 172.21.15.46:0/2675173314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 msgr2=0x7f09701034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 --2- 172.21.15.46:0/2675173314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f09701034a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09600099c0 tx=0x7f0960009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 -- 172.21.15.46:0/2675173314 shutdown_connections 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 --2- 172.21.15.46:0/2675173314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104c10 0x7f097010b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 --2- 172.21.15.46:0/2675173314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f09701034a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 --2- 172.21.15.46:0/2675173314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104270 0x7f09701046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 -- 172.21.15.46:0/2675173314 >> 172.21.15.46:0/2675173314 conn(0x7f09700fe610 msgr2=0x7f0970100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 -- 172.21.15.46:0/2675173314 shutdown_connections 2024-05-08T23:02:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.948+0000 7f0976c00700 1 -- 172.21.15.46:0/2675173314 wait complete. 2024-05-08T23:02:19.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 Processor -- start 2024-05-08T23:02:19.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 -- start start 2024-05-08T23:02:19.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104270 0x7f0970199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104c10 0x7f09701a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701a0af0 con 0x7f0970103080 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701a0c30 con 0x7f0970104c10 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.949+0000 7f0976c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701a0dc0 con 0x7f0970104270 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54904/0 (socket says 172.21.15.46:54904) 2024-05-08T23:02:19.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 -- 172.21.15.46:0/3308985086 learned_addr learned my addr 172.21.15.46:0/3308985086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096f400700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104270 0x7f0970199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f0974c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104c10 0x7f09701a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 -- 172.21.15.46:0/3308985086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104270 msgr2=0x7f0970199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104270 0x7f0970199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 -- 172.21.15.46:0/3308985086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104c10 msgr2=0x7f09701a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104c10 0x7f09701a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 -- 172.21.15.46:0/3308985086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600095f0 con 0x7f0970103080 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.950+0000 7f096fe00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0960003730 tx=0x7f096000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.951+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096001d070 con 0x7f0970103080 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.951+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0960005050 con 0x7f0970103080 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.951+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960004970 con 0x7f0970103080 2024-05-08T23:02:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.951+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09701a1040 con 0x7f0970103080 2024-05-08T23:02:19.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.951+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09701a14a0 con 0x7f0970103080 2024-05-08T23:02:19.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.952+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097010a520 con 0x7f0970103080 2024-05-08T23:02:19.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.954+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0960005470 con 0x7f0970103080 2024-05-08T23:02:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.955+0000 7f096cc00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 0x7f09540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.955+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f09600cd8e0 con 0x7f0970103080 2024-05-08T23:02:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.955+0000 7f096f400700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 0x7f09540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.956+0000 7f096f400700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 0x7f09540a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f095c009c60 tx=0x7f095c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:19.959+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0960096c90 con 0x7f0970103080 2024-05-08T23:02:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:02:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-08T23:02:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:19.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: pgmap v255: 129 pgs: 4 peering, 125 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-05-08T23:02:19.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:active 2024-05-08T23:02:19.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:19 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:02:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: pgmap v255: 129 pgs: 4 peering, 125 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:active 2024-05-08T23:02:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:19 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:02:20.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: pgmap v255: 129 pgs: 4 peering, 125 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] up:active 2024-05-08T23:02:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:19 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:02:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.183+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f097010e8c0 con 0x7f0970103080 2024-05-08T23:02:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.185+0000 7f096cc00700 1 -- 172.21.15.46:0/3308985086 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7f096009a2b0 con 0x7f0970103080 2024-05-08T23:02:20.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:20.185 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.187+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 msgr2=0x7f09540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.187+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 0x7f09540a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f095c009c60 tx=0x7f095c009260 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.187+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 msgr2=0x7f0970198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.187+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0960003730 tx=0x7f096000f690 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 shutdown_connections 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0970104c10 0x7f09701a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f09540a1230 0x7f09540a36f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0970103080 0x7f0970198fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 --2- 172.21.15.46:0/3308985086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0970104270 0x7f0970199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 >> 172.21.15.46:0/3308985086 conn(0x7f09700fe610 msgr2=0x7f0970107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 shutdown_connections 2024-05-08T23:02:20.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.188+0000 7f0976c00700 1 -- 172.21.15.46:0/3308985086 wait complete. 2024-05-08T23:02:20.207 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:02:20.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:02:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 -- 172.21.15.46:0/3852217605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 msgr2=0x7fe5e4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/3852217605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4102450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009990 tx=0x7fe5d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 -- 172.21.15.46:0/3852217605 shutdown_connections 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/3852217605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/3852217605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/3852217605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4102450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.406+0000 7fe5ebc00700 1 -- 172.21.15.46:0/3852217605 >> 172.21.15.46:0/3852217605 conn(0x7fe5e40fc410 msgr2=0x7fe5e40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.407+0000 7fe5ebc00700 1 -- 172.21.15.46:0/3852217605 shutdown_connections 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.407+0000 7fe5ebc00700 1 -- 172.21.15.46:0/3852217605 wait complete. 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.407+0000 7fe5ebc00700 1 Processor -- start 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 -- start start 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e4199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41a0b40 con 0x7fe5e4101fd0 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41a0c80 con 0x7fe5e4100e80 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5ebc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41a0e10 con 0x7fe5e4102990 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5e3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5e3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54926/0 (socket says 172.21.15.46:54926) 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5e3e00700 1 -- 172.21.15.46:0/552599973 learned_addr learned my addr 172.21.15.46:0/552599973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5e9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e4199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.408+0000 7fe5e9c00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 -- 172.21.15.46:0/552599973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 msgr2=0x7fe5e41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 -- 172.21.15.46:0/552599973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 msgr2=0x7fe5e4199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e4199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 -- 172.21.15.46:0/552599973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00095f0 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e9c00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e9200700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e4199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e3e00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000b620 tx=0x7fe5d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d001d070 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5d0022470 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017650 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e41a1090 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.409+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41a14b0 con 0x7fe5e4101fd0 2024-05-08T23:02:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.410+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4067d10 con 0x7fe5e4101fd0 2024-05-08T23:02:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.412+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe5d00225e0 con 0x7fe5e4101fd0 2024-05-08T23:02:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.413+0000 7fe5e1600700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 0x7fe5cc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.414+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5d00ce1a0 con 0x7fe5e4101fd0 2024-05-08T23:02:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.414+0000 7fe5e9200700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 0x7fe5cc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.414+0000 7fe5e9200700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 0x7fe5cc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011490 tx=0x7fe5d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.418+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe5d00974f0 con 0x7fe5e4101fd0 2024-05-08T23:02:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.631+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5e40748f0 con 0x7fe5cc0a1230 2024-05-08T23:02:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.646+0000 7fe5e1600700 1 -- 172.21.15.46:0/552599973 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422271 (secure 0 0 0) 0x7fe5e40748f0 con 0x7fe5cc0a1230 2024-05-08T23:02:20.653 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 msgr2=0x7fe5cc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 0x7fe5cc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011490 tx=0x7fe5d8009260 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 msgr2=0x7fe5e4199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000b620 tx=0x7fe5d0004730 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 shutdown_connections 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102990 0x7fe5e41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e4100e80 0x7fe5e4199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe5cc0a1230 0x7fe5cc0a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 --2- 172.21.15.46:0/552599973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e4101fd0 0x7fe5e4199550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 >> 172.21.15.46:0/552599973 conn(0x7fe5e40fc410 msgr2=0x7fe5e40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 shutdown_connections 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.652+0000 7fe5ebc00700 1 -- 172.21.15.46:0/552599973 wait complete. 2024-05-08T23:02:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:02:20.671 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-05-08T23:02:18.923976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598050,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":296,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371864,"kb_used_data":10780,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569960,"statfs":{"total":1151940427776,"available":1151559639040,"internally_reserved":0,"allocated":11038720,"data_stored":7122346,"data_compressed":253759,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3229,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001928"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132805+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:01:57.132805+0000","last_peered":"2024-05-08T23:01:57.132805+0000","last_clean":"2024-05-08T23:01:57.132805+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:01:57.132805+0000","last_undegraded":"2024-05-08T23:01:57.132805+0000","last_fullsized":"2024-05-08T23:01:57.132805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133301+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:01:57.133301+0000","last_peered":"2024-05-08T23:01:57.133301+0000","last_clean":"2024-05-08T23:01:57.133301+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:01:57.133301+0000","last_undegraded":"2024-05-08T23:01:57.133301+0000","last_fullsized":"2024-05-08T23:01:57.133301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132530+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:01:57.132530+0000","last_peered":"2024-05-08T23:01:57.132530+0000","last_clean":"2024-05-08T23:01:57.132530+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:01:57.132530+0000","last_undegraded":"2024-05-08T23:01:57.132530+0000","last_fullsized":"2024-05-08T23:01:57.132530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950890+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:01:55.950890+0000","last_peered":"2024-05-08T23:01:55.950890+0000","last_clean":"2024-05-08T23:01:55.950890+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:01:55.950890+0000","last_undegraded":"2024-05-08T23:01:55.950890+0000","last_fullsized":"2024-05-08T23:01:55.950890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954464+0000","last_change":"2024-05-08T23:01:51.493439+0000","last_active":"2024-05-08T23:01:55.954464+0000","last_peered":"2024-05-08T23:01:55.954464+0000","last_clean":"2024-05-08T23:01:55.954464+0000","last_became_active":"2024-05-08T23:01:51.493055+0000","last_became_peered":"2024-05-08T23:01:51.493055+0000","last_unstale":"2024-05-08T23:01:55.954464+0000","last_undegraded":"2024-05-08T23:01:55.954464+0000","last_fullsized":"2024-05-08T23:01:55.954464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:25.226427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.260854+0000","last_change":"2024-05-08T23:01:51.496941+0000","last_active":"2024-05-08T23:01:56.260854+0000","last_peered":"2024-05-08T23:01:56.260854+0000","last_clean":"2024-05-08T23:01:56.260854+0000","last_became_active":"2024-05-08T23:01:51.496579+0000","last_became_peered":"2024-05-08T23:01:51.496579+0000","last_unstale":"2024-05-08T23:01:56.260854+0000","last_undegraded":"2024-05-08T23:01:56.260854+0000","last_fullsized":"2024-05-08T23:01:56.260854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:57.435809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.958586+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:01:55.958586+0000","last_peered":"2024-05-08T23:01:55.958586+0000","last_clean":"2024-05-08T23:01:55.958586+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:01:55.958586+0000","last_undegraded":"2024-05-08T23:01:55.958586+0000","last_fullsized":"2024-05-08T23:01:55.958586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478678+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:01:56.478678+0000","last_peered":"2024-05-08T23:01:56.478678+0000","last_clean":"2024-05-08T23:01:56.478678+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:01:56.478678+0000","last_undegraded":"2024-05-08T23:01:56.478678+0000","last_fullsized":"2024-05-08T23:01:56.478678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622558+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:01:56.622558+0000","last_peered":"2024-05-08T23:01:56.622558+0000","last_clean":"2024-05-08T23:01:56.622558+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:01:56.622558+0000","last_undegraded":"2024-05-08T23:01:56.622558+0000","last_fullsized":"2024-05-08T23:01:56.622558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.262423+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:01:56.262423+0000","last_peered":"2024-05-08T23:01:56.262423+0000","last_clean":"2024-05-08T23:01:56.262423+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:01:56.262423+0000","last_undegraded":"2024-05-08T23:01:56.262423+0000","last_fullsized":"2024-05-08T23:01:56.262423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959348+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:01:55.959348+0000","last_peered":"2024-05-08T23:01:55.959348+0000","last_clean":"2024-05-08T23:01:55.959348+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:01:55.959348+0000","last_undegraded":"2024-05-08T23:01:55.959348+0000","last_fullsized":"2024-05-08T23:01:55.959348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950835+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:01:55.950835+0000","last_peered":"2024-05-08T23:01:55.950835+0000","last_clean":"2024-05-08T23:01:55.950835+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:01:55.950835+0000","last_undegraded":"2024-05-08T23:01:55.950835+0000","last_fullsized":"2024-05-08T23:01:55.950835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963347+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:01:55.963347+0000","last_peered":"2024-05-08T23:01:55.963347+0000","last_clean":"2024-05-08T23:01:55.963347+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:01:55.963347+0000","last_undegraded":"2024-05-08T23:01:55.963347+0000","last_fullsized":"2024-05-08T23:01:55.963347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131527+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:01:57.131527+0000","last_peered":"2024-05-08T23:01:57.131527+0000","last_clean":"2024-05-08T23:01:57.131527+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:01:57.131527+0000","last_undegraded":"2024-05-08T23:01:57.131527+0000","last_fullsized":"2024-05-08T23:01:57.131527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131664+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:01:57.131664+0000","last_peered":"2024-05-08T23:01:57.131664+0000","last_clean":"2024-05-08T23:01:57.131664+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:01:57.131664+0000","last_undegraded":"2024-05-08T23:01:57.131664+0000","last_fullsized":"2024-05-08T23:01:57.131664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622204+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:01:56.622204+0000","last_peered":"2024-05-08T23:01:56.622204+0000","last_clean":"2024-05-08T23:01:56.622204+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:01:56.622204+0000","last_undegraded":"2024-05-08T23:01:56.622204+0000","last_fullsized":"2024-05-08T23:01:56.622204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132822+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:01:57.132822+0000","last_peered":"2024-05-08T23:01:57.132822+0000","last_clean":"2024-05-08T23:01:57.132822+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:01:57.132822+0000","last_undegraded":"2024-05-08T23:01:57.132822+0000","last_fullsized":"2024-05-08T23:01:57.132822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.969661+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:01:55.969661+0000","last_peered":"2024-05-08T23:01:55.969661+0000","last_clean":"2024-05-08T23:01:55.969661+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:01:55.969661+0000","last_undegraded":"2024-05-08T23:01:55.969661+0000","last_fullsized":"2024-05-08T23:01:55.969661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.956179+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:01:55.956179+0000","last_peered":"2024-05-08T23:01:55.956179+0000","last_clean":"2024-05-08T23:01:55.956179+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:01:55.956179+0000","last_undegraded":"2024-05-08T23:01:55.956179+0000","last_fullsized":"2024-05-08T23:01:55.956179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005052+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:01:57.005052+0000","last_peered":"2024-05-08T23:01:57.005052+0000","last_clean":"2024-05-08T23:01:57.005052+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:01:57.005052+0000","last_undegraded":"2024-05-08T23:01:57.005052+0000","last_fullsized":"2024-05-08T23:01:57.005052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651019+0000","last_change":"2024-05-08T23:01:50.956587+0000","last_active":"2024-05-08T23:02:15.651019+0000","last_peered":"2024-05-08T23:02:15.651019+0000","last_clean":"2024-05-08T23:02:15.651019+0000","last_became_active":"2024-05-08T23:01:50.956248+0000","last_became_peered":"2024-05-08T23:01:50.956248+0000","last_unstale":"2024-05-08T23:02:15.651019+0000","last_undegraded":"2024-05-08T23:02:15.651019+0000","last_fullsized":"2024-05-08T23:02:15.651019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:54.517099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479226+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:01:56.479226+0000","last_peered":"2024-05-08T23:01:56.479226+0000","last_clean":"2024-05-08T23:01:56.479226+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:01:56.479226+0000","last_undegraded":"2024-05-08T23:01:56.479226+0000","last_fullsized":"2024-05-08T23:01:56.479226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744289+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:01:56.744289+0000","last_peered":"2024-05-08T23:01:56.744289+0000","last_clean":"2024-05-08T23:01:56.744289+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:01:56.744289+0000","last_undegraded":"2024-05-08T23:01:56.744289+0000","last_fullsized":"2024-05-08T23:01:56.744289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.649246+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:15.649246+0000","last_peered":"2024-05-08T23:02:15.649246+0000","last_clean":"2024-05-08T23:02:15.649246+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:15.649246+0000","last_undegraded":"2024-05-08T23:02:15.649246+0000","last_fullsized":"2024-05-08T23:02:15.649246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133554+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:01:57.133554+0000","last_peered":"2024-05-08T23:01:57.133554+0000","last_clean":"2024-05-08T23:01:57.133554+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:01:57.133554+0000","last_undegraded":"2024-05-08T23:01:57.133554+0000","last_fullsized":"2024-05-08T23:01:57.133554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744332+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:01:56.744332+0000","last_peered":"2024-05-08T23:01:56.744332+0000","last_clean":"2024-05-08T23:01:56.744332+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:01:56.744332+0000","last_undegraded":"2024-05-08T23:01:56.744332+0000","last_fullsized":"2024-05-08T23:01:56.744332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"77'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955814+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:01:55.955814+0000","last_peered":"2024-05-08T23:01:55.955814+0000","last_clean":"2024-05-08T23:01:55.955814+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:01:55.955814+0000","last_undegraded":"2024-05-08T23:01:55.955814+0000","last_fullsized":"2024-05-08T23:01:55.955814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622264+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:01:56.622264+0000","last_peered":"2024-05-08T23:01:56.622264+0000","last_clean":"2024-05-08T23:01:56.622264+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:01:56.622264+0000","last_undegraded":"2024-05-08T23:01:56.622264+0000","last_fullsized":"2024-05-08T23:01:56.622264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744320+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:01:56.744320+0000","last_peered":"2024-05-08T23:01:56.744320+0000","last_clean":"2024-05-08T23:01:56.744320+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:01:56.744320+0000","last_undegraded":"2024-05-08T23:01:56.744320+0000","last_fullsized":"2024-05-08T23:01:56.744320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.957157+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:01:55.957157+0000","last_peered":"2024-05-08T23:01:55.957157+0000","last_clean":"2024-05-08T23:01:55.957157+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:01:55.957157+0000","last_undegraded":"2024-05-08T23:01:55.957157+0000","last_fullsized":"2024-05-08T23:01:55.957157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950263+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:01:55.950263+0000","last_peered":"2024-05-08T23:01:55.950263+0000","last_clean":"2024-05-08T23:01:55.950263+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:01:55.950263+0000","last_undegraded":"2024-05-08T23:01:55.950263+0000","last_fullsized":"2024-05-08T23:01:55.950263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005132+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:01:57.005132+0000","last_peered":"2024-05-08T23:01:57.005132+0000","last_clean":"2024-05-08T23:01:57.005132+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:01:57.005132+0000","last_undegraded":"2024-05-08T23:01:57.005132+0000","last_fullsized":"2024-05-08T23:01:57.005132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643489+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:14.643489+0000","last_peered":"2024-05-08T23:02:14.643489+0000","last_clean":"2024-05-08T23:02:14.643489+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:14.643489+0000","last_undegraded":"2024-05-08T23:02:14.643489+0000","last_fullsized":"2024-05-08T23:02:14.643489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.958457+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:01:55.958457+0000","last_peered":"2024-05-08T23:01:55.958457+0000","last_clean":"2024-05-08T23:01:55.958457+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:01:55.958457+0000","last_undegraded":"2024-05-08T23:01:55.958457+0000","last_fullsized":"2024-05-08T23:01:55.958457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005099+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:01:57.005099+0000","last_peered":"2024-05-08T23:01:57.005099+0000","last_clean":"2024-05-08T23:01:57.005099+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:01:57.005099+0000","last_undegraded":"2024-05-08T23:01:57.005099+0000","last_fullsized":"2024-05-08T23:01:57.005099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"77'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.260838+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:01:56.260838+0000","last_peered":"2024-05-08T23:01:56.260838+0000","last_clean":"2024-05-08T23:01:56.260838+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:01:56.260838+0000","last_undegraded":"2024-05-08T23:01:56.260838+0000","last_fullsized":"2024-05-08T23:01:56.260838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.949992+0000","last_change":"2024-05-08T23:01:51.495473+0000","last_active":"2024-05-08T23:01:55.949992+0000","last_peered":"2024-05-08T23:01:55.949992+0000","last_clean":"2024-05-08T23:01:55.949992+0000","last_became_active":"2024-05-08T23:01:51.495165+0000","last_became_peered":"2024-05-08T23:01:51.495165+0000","last_unstale":"2024-05-08T23:01:55.949992+0000","last_undegraded":"2024-05-08T23:01:55.949992+0000","last_fullsized":"2024-05-08T23:01:55.949992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:58.599864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178543+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:01:58.178543+0000","last_peered":"2024-05-08T23:01:58.178543+0000","last_clean":"2024-05-08T23:01:58.178543+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:01:58.178543+0000","last_undegraded":"2024-05-08T23:01:58.178543+0000","last_fullsized":"2024-05-08T23:01:58.178543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650498+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:15.650498+0000","last_peered":"2024-05-08T23:02:15.650498+0000","last_clean":"2024-05-08T23:02:15.650498+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:15.650498+0000","last_undegraded":"2024-05-08T23:02:15.650498+0000","last_fullsized":"2024-05-08T23:02:15.650498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478560+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:01:56.478560+0000","last_peered":"2024-05-08T23:01:56.478560+0000","last_clean":"2024-05-08T23:01:56.478560+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:01:56.478560+0000","last_undegraded":"2024-05-08T23:01:56.478560+0000","last_fullsized":"2024-05-08T23:01:56.478560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988198+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:01:57.988198+0000","last_peered":"2024-05-08T23:01:57.988198+0000","last_clean":"2024-05-08T23:01:57.988198+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:01:57.988198+0000","last_undegraded":"2024-05-08T23:01:57.988198+0000","last_fullsized":"2024-05-08T23:01:57.988198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131549+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:01:57.131549+0000","last_peered":"2024-05-08T23:01:57.131549+0000","last_clean":"2024-05-08T23:01:57.131549+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:01:57.131549+0000","last_undegraded":"2024-05-08T23:01:57.131549+0000","last_fullsized":"2024-05-08T23:01:57.131549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962314+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:01:55.962314+0000","last_peered":"2024-05-08T23:01:55.962314+0000","last_clean":"2024-05-08T23:01:55.962314+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:01:55.962314+0000","last_undegraded":"2024-05-08T23:01:55.962314+0000","last_fullsized":"2024-05-08T23:01:55.962314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865359+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:01:57.865359+0000","last_peered":"2024-05-08T23:01:57.865359+0000","last_clean":"2024-05-08T23:01:57.865359+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:01:57.865359+0000","last_undegraded":"2024-05-08T23:01:57.865359+0000","last_fullsized":"2024-05-08T23:01:57.865359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.262324+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:01:56.262324+0000","last_peered":"2024-05-08T23:01:56.262324+0000","last_clean":"2024-05-08T23:01:56.262324+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:01:56.262324+0000","last_undegraded":"2024-05-08T23:01:56.262324+0000","last_fullsized":"2024-05-08T23:01:56.262324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622634+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:01:56.622634+0000","last_peered":"2024-05-08T23:01:56.622634+0000","last_clean":"2024-05-08T23:01:56.622634+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:01:56.622634+0000","last_undegraded":"2024-05-08T23:01:56.622634+0000","last_fullsized":"2024-05-08T23:01:56.622634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.978044+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:01:55.978044+0000","last_peered":"2024-05-08T23:01:55.978044+0000","last_clean":"2024-05-08T23:01:55.978044+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:01:55.978044+0000","last_undegraded":"2024-05-08T23:01:55.978044+0000","last_fullsized":"2024-05-08T23:01:55.978044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959199+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:01:55.959199+0000","last_peered":"2024-05-08T23:01:55.959199+0000","last_clean":"2024-05-08T23:01:55.959199+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:01:55.959199+0000","last_undegraded":"2024-05-08T23:01:55.959199+0000","last_fullsized":"2024-05-08T23:01:55.959199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133339+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:01:57.133339+0000","last_peered":"2024-05-08T23:01:57.133339+0000","last_clean":"2024-05-08T23:01:57.133339+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:01:57.133339+0000","last_undegraded":"2024-05-08T23:01:57.133339+0000","last_fullsized":"2024-05-08T23:01:57.133339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744357+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:01:56.744357+0000","last_peered":"2024-05-08T23:01:56.744357+0000","last_clean":"2024-05-08T23:01:56.744357+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:01:56.744357+0000","last_undegraded":"2024-05-08T23:01:56.744357+0000","last_fullsized":"2024-05-08T23:01:56.744357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132007+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:01:57.132007+0000","last_peered":"2024-05-08T23:01:57.132007+0000","last_clean":"2024-05-08T23:01:57.132007+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:01:57.132007+0000","last_undegraded":"2024-05-08T23:01:57.132007+0000","last_fullsized":"2024-05-08T23:01:57.132007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622409+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:01:56.622409+0000","last_peered":"2024-05-08T23:01:56.622409+0000","last_clean":"2024-05-08T23:01:56.622409+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:01:56.622409+0000","last_undegraded":"2024-05-08T23:01:56.622409+0000","last_fullsized":"2024-05-08T23:01:56.622409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977451+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:01:55.977451+0000","last_peered":"2024-05-08T23:01:55.977451+0000","last_clean":"2024-05-08T23:01:55.977451+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:01:55.977451+0000","last_undegraded":"2024-05-08T23:01:55.977451+0000","last_fullsized":"2024-05-08T23:01:55.977451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"82'10","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:12.062413+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:12.062413+0000","last_peered":"2024-05-08T23:02:12.062413+0000","last_clean":"2024-05-08T23:02:12.062413+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:12.062413+0000","last_undegraded":"2024-05-08T23:02:12.062413+0000","last_fullsized":"2024-05-08T23:02:12.062413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955389+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:01:55.955389+0000","last_peered":"2024-05-08T23:01:55.955389+0000","last_clean":"2024-05-08T23:01:55.955389+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:01:55.955389+0000","last_undegraded":"2024-05-08T23:01:55.955389+0000","last_fullsized":"2024-05-08T23:01:55.955389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.964135+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:01:55.964135+0000","last_peered":"2024-05-08T23:01:55.964135+0000","last_clean":"2024-05-08T23:01:55.964135+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:01:55.964135+0000","last_undegraded":"2024-05-08T23:01:55.964135+0000","last_fullsized":"2024-05-08T23:01:55.964135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650769+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:15.650769+0000","last_peered":"2024-05-08T23:02:15.650769+0000","last_clean":"2024-05-08T23:02:15.650769+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:15.650769+0000","last_undegraded":"2024-05-08T23:02:15.650769+0000","last_fullsized":"2024-05-08T23:02:15.650769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963754+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:01:55.963754+0000","last_peered":"2024-05-08T23:01:55.963754+0000","last_clean":"2024-05-08T23:01:55.963754+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:01:55.963754+0000","last_undegraded":"2024-05-08T23:01:55.963754+0000","last_fullsized":"2024-05-08T23:01:55.963754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178512+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:01:58.178512+0000","last_peered":"2024-05-08T23:01:58.178512+0000","last_clean":"2024-05-08T23:01:58.178512+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:01:58.178512+0000","last_undegraded":"2024-05-08T23:01:58.178512+0000","last_fullsized":"2024-05-08T23:01:58.178512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643267+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:14.643267+0000","last_peered":"2024-05-08T23:02:14.643267+0000","last_clean":"2024-05-08T23:02:14.643267+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:14.643267+0000","last_undegraded":"2024-05-08T23:02:14.643267+0000","last_fullsized":"2024-05-08T23:02:14.643267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959401+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:01:55.959401+0000","last_peered":"2024-05-08T23:01:55.959401+0000","last_clean":"2024-05-08T23:01:55.959401+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:01:55.959401+0000","last_undegraded":"2024-05-08T23:01:55.959401+0000","last_fullsized":"2024-05-08T23:01:55.959401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977932+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:01:55.977932+0000","last_peered":"2024-05-08T23:01:55.977932+0000","last_clean":"2024-05-08T23:01:55.977932+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:01:55.977932+0000","last_undegraded":"2024-05-08T23:01:55.977932+0000","last_fullsized":"2024-05-08T23:01:55.977932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.655417+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:16.655417+0000","last_peered":"2024-05-08T23:02:16.655417+0000","last_clean":"2024-05-08T23:02:16.655417+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:16.655417+0000","last_undegraded":"2024-05-08T23:02:16.655417+0000","last_fullsized":"2024-05-08T23:02:16.655417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.958092+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:01:55.958092+0000","last_peered":"2024-05-08T23:01:55.958092+0000","last_clean":"2024-05-08T23:01:55.958092+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:01:55.958092+0000","last_undegraded":"2024-05-08T23:01:55.958092+0000","last_fullsized":"2024-05-08T23:01:55.958092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015956+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:01:57.015956+0000","last_peered":"2024-05-08T23:01:57.015956+0000","last_clean":"2024-05-08T23:01:57.015956+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:01:57.015956+0000","last_undegraded":"2024-05-08T23:01:57.015956+0000","last_fullsized":"2024-05-08T23:01:57.015956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"77'8","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.953306+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:01:55.953306+0000","last_peered":"2024-05-08T23:01:55.953306+0000","last_clean":"2024-05-08T23:01:55.953306+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:01:55.953306+0000","last_undegraded":"2024-05-08T23:01:55.953306+0000","last_fullsized":"2024-05-08T23:01:55.953306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.949921+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:01:55.949921+0000","last_peered":"2024-05-08T23:01:55.949921+0000","last_clean":"2024-05-08T23:01:55.949921+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:01:55.949921+0000","last_undegraded":"2024-05-08T23:01:55.949921+0000","last_fullsized":"2024-05-08T23:01:55.949921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963596+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:01:55.963596+0000","last_peered":"2024-05-08T23:01:55.963596+0000","last_clean":"2024-05-08T23:01:55.963596+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:01:55.963596+0000","last_undegraded":"2024-05-08T23:01:55.963596+0000","last_fullsized":"2024-05-08T23:01:55.963596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:01:51.470821+0000","last_change":"2024-05-08T23:02:18.181005+0000","last_active":"2024-05-08T23:02:18.181005+0000","last_peered":"2024-05-08T23:01:51.470821+0000","last_clean":"2024-05-08T23:01:51.470821+0000","last_became_active":"2024-05-08T23:01:50.955995+0000","last_became_peered":"2024-05-08T23:01:50.955995+0000","last_unstale":"2024-05-08T23:01:51.470821+0000","last_undegraded":"2024-05-08T23:01:51.470821+0000","last_fullsized":"2024-05-08T23:01:51.470821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.658743+0000","last_change":"2024-05-08T23:02:17.658905+0000","last_active":"2024-05-08T23:02:17.658743+0000","last_peered":"2024-05-08T23:01:57.132603+0000","last_clean":"2024-05-08T23:01:57.132603+0000","last_became_active":"2024-05-08T23:01:51.489719+0000","last_became_peered":"2024-05-08T23:01:51.489719+0000","last_unstale":"2024-05-08T23:02:17.658743+0000","last_undegraded":"2024-05-08T23:02:17.658743+0000","last_fullsized":"2024-05-08T23:02:17.658743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:15.412899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.481346+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:01:56.481346+0000","last_peered":"2024-05-08T23:01:56.481346+0000","last_clean":"2024-05-08T23:01:56.481346+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:01:56.481346+0000","last_undegraded":"2024-05-08T23:01:56.481346+0000","last_fullsized":"2024-05-08T23:01:56.481346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.967914+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:01:55.967914+0000","last_peered":"2024-05-08T23:01:55.967914+0000","last_clean":"2024-05-08T23:01:55.967914+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:01:55.967914+0000","last_undegraded":"2024-05-08T23:01:55.967914+0000","last_fullsized":"2024-05-08T23:01:55.967914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.261460+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:01:56.261460+0000","last_peered":"2024-05-08T23:01:56.261460+0000","last_clean":"2024-05-08T23:01:56.261460+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:01:56.261460+0000","last_undegraded":"2024-05-08T23:01:56.261460+0000","last_fullsized":"2024-05-08T23:01:56.261460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015922+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:01:57.015922+0000","last_peered":"2024-05-08T23:01:57.015922+0000","last_clean":"2024-05-08T23:01:57.015922+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:01:57.015922+0000","last_undegraded":"2024-05-08T23:01:57.015922+0000","last_fullsized":"2024-05-08T23:01:57.015922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.958851+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:01:55.958851+0000","last_peered":"2024-05-08T23:01:55.958851+0000","last_clean":"2024-05-08T23:01:55.958851+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:01:55.958851+0000","last_undegraded":"2024-05-08T23:01:55.958851+0000","last_fullsized":"2024-05-08T23:01:55.958851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.261074+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:01:56.261074+0000","last_peered":"2024-05-08T23:01:56.261074+0000","last_clean":"2024-05-08T23:01:56.261074+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:01:56.261074+0000","last_undegraded":"2024-05-08T23:01:56.261074+0000","last_fullsized":"2024-05-08T23:01:56.261074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015977+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:01:57.015977+0000","last_peered":"2024-05-08T23:01:57.015977+0000","last_clean":"2024-05-08T23:01:57.015977+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:01:57.015977+0000","last_undegraded":"2024-05-08T23:01:57.015977+0000","last_fullsized":"2024-05-08T23:01:57.015977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.951139+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:01:55.951139+0000","last_peered":"2024-05-08T23:01:55.951139+0000","last_clean":"2024-05-08T23:01:55.951139+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:01:55.951139+0000","last_undegraded":"2024-05-08T23:01:55.951139+0000","last_fullsized":"2024-05-08T23:01:55.951139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651613+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:15.651613+0000","last_peered":"2024-05-08T23:02:15.651613+0000","last_clean":"2024-05-08T23:02:15.651613+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:15.651613+0000","last_undegraded":"2024-05-08T23:02:15.651613+0000","last_fullsized":"2024-05-08T23:02:15.651613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479482+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:01:56.479482+0000","last_peered":"2024-05-08T23:01:56.479482+0000","last_clean":"2024-05-08T23:01:56.479482+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:01:56.479482+0000","last_undegraded":"2024-05-08T23:01:56.479482+0000","last_fullsized":"2024-05-08T23:01:56.479482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478647+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:01:56.478647+0000","last_peered":"2024-05-08T23:01:56.478647+0000","last_clean":"2024-05-08T23:01:56.478647+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:01:56.478647+0000","last_undegraded":"2024-05-08T23:01:56.478647+0000","last_fullsized":"2024-05-08T23:01:56.478647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.654869+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:16.654869+0000","last_peered":"2024-05-08T23:02:16.654869+0000","last_clean":"2024-05-08T23:02:16.654869+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:16.654869+0000","last_undegraded":"2024-05-08T23:02:16.654869+0000","last_fullsized":"2024-05-08T23:02:16.654869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":336,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479099+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:01:56.479099+0000","last_peered":"2024-05-08T23:01:56.479099+0000","last_clean":"2024-05-08T23:01:56.479099+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:01:56.479099+0000","last_undegraded":"2024-05-08T23:01:56.479099+0000","last_fullsized":"2024-05-08T23:01:56.479099+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.262778+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:01:56.262778+0000","last_peered":"2024-05-08T23:01:56.262778+0000","last_clean":"2024-05-08T23:01:56.262778+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:01:56.262778+0000","last_undegraded":"2024-05-08T23:01:56.262778+0000","last_fullsized":"2024-05-08T23:01:56.262778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962023+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:01:55.962023+0000","last_peered":"2024-05-08T23:01:55.962023+0000","last_clean":"2024-05-08T23:01:55.962023+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:01:55.962023+0000","last_undegraded":"2024-05-08T23:01:55.962023+0000","last_fullsized":"2024-05-08T23:01:55.962023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:14.643705+0000","last_change":"2024-05-08T23:02:18.180861+0000","last_active":"2024-05-08T23:02:18.180861+0000","last_peered":"2024-05-08T23:02:14.643705+0000","last_clean":"2024-05-08T23:02:14.643705+0000","last_became_active":"2024-05-08T23:01:50.954756+0000","last_became_peered":"2024-05-08T23:01:50.954756+0000","last_unstale":"2024-05-08T23:02:14.643705+0000","last_undegraded":"2024-05-08T23:02:14.643705+0000","last_fullsized":"2024-05-08T23:02:14.643705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178486+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:01:58.178486+0000","last_peered":"2024-05-08T23:01:58.178486+0000","last_clean":"2024-05-08T23:01:58.178486+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:01:58.178486+0000","last_undegraded":"2024-05-08T23:01:58.178486+0000","last_fullsized":"2024-05-08T23:01:58.178486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955496+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:01:55.955496+0000","last_peered":"2024-05-08T23:01:55.955496+0000","last_clean":"2024-05-08T23:01:55.955496+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:01:55.955496+0000","last_undegraded":"2024-05-08T23:01:55.955496+0000","last_fullsized":"2024-05-08T23:01:55.955496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962747+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:01:55.962747+0000","last_peered":"2024-05-08T23:01:55.962747+0000","last_clean":"2024-05-08T23:01:55.962747+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:01:55.962747+0000","last_undegraded":"2024-05-08T23:01:55.962747+0000","last_fullsized":"2024-05-08T23:01:55.962747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865314+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:01:57.865314+0000","last_peered":"2024-05-08T23:01:57.865314+0000","last_clean":"2024-05-08T23:01:57.865314+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:01:57.865314+0000","last_undegraded":"2024-05-08T23:01:57.865314+0000","last_fullsized":"2024-05-08T23:01:57.865314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.648953+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:15.648953+0000","last_peered":"2024-05-08T23:02:15.648953+0000","last_clean":"2024-05-08T23:02:15.648953+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:15.648953+0000","last_undegraded":"2024-05-08T23:02:15.648953+0000","last_fullsized":"2024-05-08T23:02:15.648953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005033+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:01:57.005033+0000","last_peered":"2024-05-08T23:01:57.005033+0000","last_clean":"2024-05-08T23:01:57.005033+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:01:57.005033+0000","last_undegraded":"2024-05-08T23:01:57.005033+0000","last_fullsized":"2024-05-08T23:01:57.005033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.642858+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:14.642858+0000","last_peered":"2024-05-08T23:02:14.642858+0000","last_clean":"2024-05-08T23:02:14.642858+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:14.642858+0000","last_undegraded":"2024-05-08T23:02:14.642858+0000","last_fullsized":"2024-05-08T23:02:14.642858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133386+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:01:57.133386+0000","last_peered":"2024-05-08T23:01:57.133386+0000","last_clean":"2024-05-08T23:01:57.133386+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:01:57.133386+0000","last_undegraded":"2024-05-08T23:01:57.133386+0000","last_fullsized":"2024-05-08T23:01:57.133386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744412+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:01:56.744412+0000","last_peered":"2024-05-08T23:01:56.744412+0000","last_clean":"2024-05-08T23:01:56.744412+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:01:56.744412+0000","last_undegraded":"2024-05-08T23:01:56.744412+0000","last_fullsized":"2024-05-08T23:01:56.744412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643210+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:14.643210+0000","last_peered":"2024-05-08T23:02:14.643210+0000","last_clean":"2024-05-08T23:02:14.643210+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:14.643210+0000","last_undegraded":"2024-05-08T23:02:14.643210+0000","last_fullsized":"2024-05-08T23:02:14.643210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.953080+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:01:55.953080+0000","last_peered":"2024-05-08T23:01:55.953080+0000","last_clean":"2024-05-08T23:01:55.953080+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:01:55.953080+0000","last_undegraded":"2024-05-08T23:01:55.953080+0000","last_fullsized":"2024-05-08T23:01:55.953080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.975817+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:01:55.975817+0000","last_peered":"2024-05-08T23:01:55.975817+0000","last_clean":"2024-05-08T23:01:55.975817+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:01:55.975817+0000","last_undegraded":"2024-05-08T23:01:55.975817+0000","last_fullsized":"2024-05-08T23:01:55.975817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.672991+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.672991+0000","last_peered":"2024-05-08T23:02:17.672991+0000","last_clean":"2024-05-08T23:02:17.672991+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.672991+0000","last_undegraded":"2024-05-08T23:02:17.672991+0000","last_fullsized":"2024-05-08T23:02:17.672991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132676+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:01:57.132676+0000","last_peered":"2024-05-08T23:01:57.132676+0000","last_clean":"2024-05-08T23:01:57.132676+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:01:57.132676+0000","last_undegraded":"2024-05-08T23:01:57.132676+0000","last_fullsized":"2024-05-08T23:01:57.132676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744371+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:01:56.744371+0000","last_peered":"2024-05-08T23:01:56.744371+0000","last_clean":"2024-05-08T23:01:56.744371+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:01:56.744371+0000","last_undegraded":"2024-05-08T23:01:56.744371+0000","last_fullsized":"2024-05-08T23:01:56.744371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.121362+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:01:57.121362+0000","last_peered":"2024-05-08T23:01:57.121362+0000","last_clean":"2024-05-08T23:01:57.121362+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:01:57.121362+0000","last_undegraded":"2024-05-08T23:01:57.121362+0000","last_fullsized":"2024-05-08T23:01:57.121362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744399+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:01:56.744399+0000","last_peered":"2024-05-08T23:01:56.744399+0000","last_clean":"2024-05-08T23:01:56.744399+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:01:56.744399+0000","last_undegraded":"2024-05-08T23:01:56.744399+0000","last_fullsized":"2024-05-08T23:01:56.744399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"77'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954579+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:01:55.954579+0000","last_peered":"2024-05-08T23:01:55.954579+0000","last_clean":"2024-05-08T23:01:55.954579+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:01:55.954579+0000","last_undegraded":"2024-05-08T23:01:55.954579+0000","last_fullsized":"2024-05-08T23:01:55.954579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959236+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:01:55.959236+0000","last_peered":"2024-05-08T23:01:55.959236+0000","last_clean":"2024-05-08T23:01:55.959236+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:01:55.959236+0000","last_undegraded":"2024-05-08T23:01:55.959236+0000","last_fullsized":"2024-05-08T23:01:55.959236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.878492+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:01:57.878492+0000","last_peered":"2024-05-08T23:01:57.878492+0000","last_clean":"2024-05-08T23:01:57.878492+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:01:57.878492+0000","last_undegraded":"2024-05-08T23:01:57.878492+0000","last_fullsized":"2024-05-08T23:01:57.878492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.658345+0000","last_change":"2024-05-08T23:01:50.962884+0000","last_active":"2024-05-08T23:02:16.658345+0000","last_peered":"2024-05-08T23:02:16.658345+0000","last_clean":"2024-05-08T23:02:16.658345+0000","last_became_active":"2024-05-08T23:01:50.962543+0000","last_became_peered":"2024-05-08T23:01:50.962543+0000","last_unstale":"2024-05-08T23:02:16.658345+0000","last_undegraded":"2024-05-08T23:02:16.658345+0000","last_fullsized":"2024-05-08T23:02:16.658345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:37.332235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959419+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:01:55.959419+0000","last_peered":"2024-05-08T23:01:55.959419+0000","last_clean":"2024-05-08T23:01:55.959419+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:01:55.959419+0000","last_undegraded":"2024-05-08T23:01:55.959419+0000","last_fullsized":"2024-05-08T23:01:55.959419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988167+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:01:57.988167+0000","last_peered":"2024-05-08T23:01:57.988167+0000","last_clean":"2024-05-08T23:01:57.988167+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:01:57.988167+0000","last_undegraded":"2024-05-08T23:01:57.988167+0000","last_fullsized":"2024-05-08T23:01:57.988167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950383+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:01:55.950383+0000","last_peered":"2024-05-08T23:01:55.950383+0000","last_clean":"2024-05-08T23:01:55.950383+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:01:55.950383+0000","last_undegraded":"2024-05-08T23:01:55.950383+0000","last_fullsized":"2024-05-08T23:01:55.950383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959016+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:01:55.959016+0000","last_peered":"2024-05-08T23:01:55.959016+0000","last_clean":"2024-05-08T23:01:55.959016+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:01:55.959016+0000","last_undegraded":"2024-05-08T23:01:55.959016+0000","last_fullsized":"2024-05-08T23:01:55.959016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963647+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:01:55.963647+0000","last_peered":"2024-05-08T23:01:55.963647+0000","last_clean":"2024-05-08T23:01:55.963647+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:01:55.963647+0000","last_undegraded":"2024-05-08T23:01:55.963647+0000","last_fullsized":"2024-05-08T23:01:55.963647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.134019+0000","last_change":"2024-05-08T23:01:50.960393+0000","last_active":"2024-05-08T23:01:57.134019+0000","last_peered":"2024-05-08T23:01:57.134019+0000","last_clean":"2024-05-08T23:01:57.134019+0000","last_became_active":"2024-05-08T23:01:50.960061+0000","last_became_peered":"2024-05-08T23:01:50.960061+0000","last_unstale":"2024-05-08T23:01:57.134019+0000","last_undegraded":"2024-05-08T23:01:57.134019+0000","last_fullsized":"2024-05-08T23:01:57.134019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:54.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.959015+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:01:55.959015+0000","last_peered":"2024-05-08T23:01:55.959015+0000","last_clean":"2024-05-08T23:01:55.959015+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:01:55.959015+0000","last_undegraded":"2024-05-08T23:01:55.959015+0000","last_fullsized":"2024-05-08T23:01:55.959015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132548+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:01:57.132548+0000","last_peered":"2024-05-08T23:01:57.132548+0000","last_clean":"2024-05-08T23:01:57.132548+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:01:57.132548+0000","last_undegraded":"2024-05-08T23:01:57.132548+0000","last_fullsized":"2024-05-08T23:01:57.132548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955938+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:01:55.955938+0000","last_peered":"2024-05-08T23:01:55.955938+0000","last_clean":"2024-05-08T23:01:55.955938+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:01:55.955938+0000","last_undegraded":"2024-05-08T23:01:55.955938+0000","last_fullsized":"2024-05-08T23:01:55.955938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954500+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:01:55.954500+0000","last_peered":"2024-05-08T23:01:55.954500+0000","last_clean":"2024-05-08T23:01:55.954500+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:01:55.954500+0000","last_undegraded":"2024-05-08T23:01:55.954500+0000","last_fullsized":"2024-05-08T23:01:55.954500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132006+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:01:57.132006+0000","last_peered":"2024-05-08T23:01:57.132006+0000","last_clean":"2024-05-08T23:01:57.132006+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:01:57.132006+0000","last_undegraded":"2024-05-08T23:01:57.132006+0000","last_fullsized":"2024-05-08T23:01:57.132006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962832+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:01:55.962832+0000","last_peered":"2024-05-08T23:01:55.962832+0000","last_clean":"2024-05-08T23:01:55.962832+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:01:55.962832+0000","last_undegraded":"2024-05-08T23:01:55.962832+0000","last_fullsized":"2024-05-08T23:01:55.962832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132832+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:01:57.132832+0000","last_peered":"2024-05-08T23:01:57.132832+0000","last_clean":"2024-05-08T23:01:57.132832+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:01:57.132832+0000","last_undegraded":"2024-05-08T23:01:57.132832+0000","last_fullsized":"2024-05-08T23:01:57.132832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954356+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:01:55.954356+0000","last_peered":"2024-05-08T23:01:55.954356+0000","last_clean":"2024-05-08T23:01:55.954356+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:01:55.954356+0000","last_undegraded":"2024-05-08T23:01:55.954356+0000","last_fullsized":"2024-05-08T23:01:55.954356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.651608+0000","last_change":"2024-05-08T23:02:17.651768+0000","last_active":"2024-05-08T23:02:17.651608+0000","last_peered":"2024-05-08T23:01:56.479359+0000","last_clean":"2024-05-08T23:01:56.479359+0000","last_became_active":"2024-05-08T23:01:51.493814+0000","last_became_peered":"2024-05-08T23:01:51.493814+0000","last_unstale":"2024-05-08T23:02:17.651608+0000","last_undegraded":"2024-05-08T23:02:17.651608+0000","last_fullsized":"2024-05-08T23:02:17.651608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:28.292897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950871+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:01:55.950871+0000","last_peered":"2024-05-08T23:01:55.950871+0000","last_clean":"2024-05-08T23:01:55.950871+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:01:55.950871+0000","last_undegraded":"2024-05-08T23:01:55.950871+0000","last_fullsized":"2024-05-08T23:01:55.950871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478747+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:01:56.478747+0000","last_peered":"2024-05-08T23:01:56.478747+0000","last_clean":"2024-05-08T23:01:56.478747+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:01:56.478747+0000","last_undegraded":"2024-05-08T23:01:56.478747+0000","last_fullsized":"2024-05-08T23:01:56.478747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622317+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:01:56.622317+0000","last_peered":"2024-05-08T23:01:56.622317+0000","last_clean":"2024-05-08T23:01:56.622317+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:01:56.622317+0000","last_undegraded":"2024-05-08T23:01:56.622317+0000","last_fullsized":"2024-05-08T23:01:56.622317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963505+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:01:55.963505+0000","last_peered":"2024-05-08T23:01:55.963505+0000","last_clean":"2024-05-08T23:01:55.963505+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:01:55.963505+0000","last_undegraded":"2024-05-08T23:01:55.963505+0000","last_fullsized":"2024-05-08T23:01:55.963505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132462+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:01:57.132462+0000","last_peered":"2024-05-08T23:01:57.132462+0000","last_clean":"2024-05-08T23:01:57.132462+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:01:57.132462+0000","last_undegraded":"2024-05-08T23:01:57.132462+0000","last_fullsized":"2024-05-08T23:01:57.132462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7682,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":42329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612623,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":913408,"data_stored":452932,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841553,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1036636,"data_compressed":47572,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.629},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.84099999999999997}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.88300000000000001}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.71099999999999997}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82099999999999995}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.95699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.024}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.05},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.0640000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":9,"up_from":61,"seq":261993005078,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":443062,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.879},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":8,"up_from":54,"seq":231928234010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":442882,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.443}]},{"osd":1,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66700000000000004}]},{"osd":2,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73099999999999998}]},{"osd":4,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.62},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71499999999999997}]},{"osd":7,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.747},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.64900000000000002}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.77700000000000002}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":7,"up_from":47,"seq":201863462942,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":942080,"data_stored":1033228,"data_compressed":47572,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.499}]},{"osd":2,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.499,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.626}]},{"osd":3,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71899999999999997}]},{"osd":4,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54100000000000004}]},{"osd":5,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.55200000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68300000000000005}]},{"osd":6,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.66400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.05},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.69899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69899999999999995}]},{"osd":8,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.78300000000000003}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":892928,"data_stored":442368,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47599999999999998},"min":{"1min":0.441,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.60499999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78100000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.83499999999999996}]},{"osd":4,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]},{"osd":5,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.53700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.90400000000000003}]},{"osd":8,"last update":"Wed May 8 23:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":452949,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed May 8 23:01:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.626,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94999999999999996,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":1.03,"15min":1.03},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79400000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72599999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":444296,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.46899999999999997,"5min":0.25,"15min":0.25},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.38200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed May 8 23:01:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.52800000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.751},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.83899999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":446972,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 23:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.44,"15min":0.435},"min":{"1min":0.32700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.503}]},{"osd":6,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.70999999999999996,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3529999999999998,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.73999999999999999,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":7.3259999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed May 8 23:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":451909,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.44,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.443,"15min":0.41499999999999998},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.54700000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.627,"15min":0.59199999999999997},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":442565,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.442}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.63100000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":35,"seq":150323855398,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":1032547,"data_compressed":47572,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.76700000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.72899999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.371},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82599999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.45800000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed May 8 23:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.502}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51600000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.45000000000000001}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78100000000000003}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:02:20.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 -- 172.21.15.46:0/676909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00102990 msgr2=0x7f2c001092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 --2- 172.21.15.46:0/676909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00102990 0x7f2c001092b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009f90 tx=0x7f2bf800b330 comp rx=0 tx=0).stop 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 -- 172.21.15.46:0/676909240 shutdown_connections 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 --2- 172.21.15.46:0/676909240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00102990 0x7f2c001092b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 --2- 172.21.15.46:0/676909240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00100e80 0x7f2c001012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 --2- 172.21.15.46:0/676909240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00101fd0 0x7f2c00102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 -- 172.21.15.46:0/676909240 >> 172.21.15.46:0/676909240 conn(0x7f2c000fc410 msgr2=0x7f2c000fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:20.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.910+0000 7f2c05400700 1 -- 172.21.15.46:0/676909240 shutdown_connections 2024-05-08T23:02:20.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.911+0000 7f2c05400700 1 -- 172.21.15.46:0/676909240 wait complete. 2024-05-08T23:02:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.911+0000 7f2c05400700 1 Processor -- start 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 -- start start 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00100e80 0x7f2c00199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00102990 0x7f2c001a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c001a0c60 con 0x7f2c00101fd0 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c001a0da0 con 0x7f2c00100e80 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2c05400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c001a0f30 con 0x7f2c00102990 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2bfde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2bfde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54946/0 (socket says 172.21.15.46:54946) 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2bfde00700 1 -- 172.21.15.46:0/3980067042 learned_addr learned my addr 172.21.15.46:0/3980067042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2bff200700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00102990 0x7f2c001a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.912+0000 7f2bfe800700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00100e80 0x7f2c00199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 -- 172.21.15.46:0/3980067042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00102990 msgr2=0x7f2c001a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00102990 0x7f2c001a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 -- 172.21.15.46:0/3980067042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00100e80 msgr2=0x7f2c00199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00100e80 0x7f2c00199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 -- 172.21.15.46:0/3980067042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0095f0 con 0x7f2c00101fd0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bfde00700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00ec10 tx=0x7f2bec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00cc90 con 0x7f2c00101fd0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec0044e0 con 0x7f2c00101fd0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8009c40 con 0x7f2c00101fd0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec010430 con 0x7f2c00101fd0 2024-05-08T23:02:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.913+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c001a15e0 con 0x7f2c00101fd0 2024-05-08T23:02:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.914+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00067d10 con 0x7f2c00101fd0 2024-05-08T23:02:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.916+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2bec003fb0 con 0x7f2c00101fd0 2024-05-08T23:02:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.918+0000 7f2bf7400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 0x7f2be40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.919+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2bec014070 con 0x7f2c00101fd0 2024-05-08T23:02:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.919+0000 7f2bfe800700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 0x7f2be40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.919+0000 7f2bfe800700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 0x7f2be40a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009c60 tx=0x7f2bf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:20.920 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-mon[77587]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.920 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-mon[77587]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T23:02:20.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/393675637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:02:20.921 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3308985086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:20.923+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2bec096560 con 0x7f2c00101fd0 2024-05-08T23:02:20.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:20 smithi086 ceph-mon[78610]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:20 smithi086 ceph-mon[78610]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T23:02:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:20 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/393675637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:02:20.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:20 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3308985086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:21.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:20 smithi046 ceph-mon[36898]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:02:21.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:20 smithi046 ceph-mon[36898]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T23:02:21.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:20 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/393675637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:02:21.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:20 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3308985086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.137+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c000748f0 con 0x7f2be40a1230 2024-05-08T23:02:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.150+0000 7f2bf7400700 1 -- 172.21.15.46:0/3980067042 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422971 (secure 0 0 0) 0x7f2c000748f0 con 0x7f2be40a1230 2024-05-08T23:02:21.155 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 msgr2=0x7f2be40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 0x7f2be40a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009c60 tx=0x7f2bf0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 msgr2=0x7f2c00199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00ec10 tx=0x7f2bec00ef20 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 shutdown_connections 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c00102990 0x7f2c001a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00100e80 0x7f2c00199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2be40a1230 0x7f2be40a36f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 --2- 172.21.15.46:0/3980067042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c00101fd0 0x7f2c00199670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 >> 172.21.15.46:0/3980067042 conn(0x7f2c000fc410 msgr2=0x7f2c000731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 shutdown_connections 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.155+0000 7f2c05400700 1 -- 172.21.15.46:0/3980067042 wait complete. 2024-05-08T23:02:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:02:21.175 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T23:02:20.924315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638431,"num_objects":87,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":144,"num_read_kb":126,"num_write":375,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372120,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569704,"statfs":{"total":1151940427776,"available":1151559376896,"internally_reserved":0,"allocated":11288576,"data_stored":7255274,"data_compressed":279264,"data_compressed_allocated":376832,"data_compressed_original":2252800,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43610,"num_objects":63,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":147,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132805+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:01:57.132805+0000","last_peered":"2024-05-08T23:01:57.132805+0000","last_clean":"2024-05-08T23:01:57.132805+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:01:57.132805+0000","last_undegraded":"2024-05-08T23:01:57.132805+0000","last_fullsized":"2024-05-08T23:01:57.132805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133301+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:01:57.133301+0000","last_peered":"2024-05-08T23:01:57.133301+0000","last_clean":"2024-05-08T23:01:57.133301+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:01:57.133301+0000","last_undegraded":"2024-05-08T23:01:57.133301+0000","last_fullsized":"2024-05-08T23:01:57.133301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132530+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:01:57.132530+0000","last_peered":"2024-05-08T23:01:57.132530+0000","last_clean":"2024-05-08T23:01:57.132530+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:01:57.132530+0000","last_undegraded":"2024-05-08T23:01:57.132530+0000","last_fullsized":"2024-05-08T23:01:57.132530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950890+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:01:55.950890+0000","last_peered":"2024-05-08T23:01:55.950890+0000","last_clean":"2024-05-08T23:01:55.950890+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:01:55.950890+0000","last_undegraded":"2024-05-08T23:01:55.950890+0000","last_fullsized":"2024-05-08T23:01:55.950890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478678+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:01:56.478678+0000","last_peered":"2024-05-08T23:01:56.478678+0000","last_clean":"2024-05-08T23:01:56.478678+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:01:56.478678+0000","last_undegraded":"2024-05-08T23:01:56.478678+0000","last_fullsized":"2024-05-08T23:01:56.478678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622558+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:01:56.622558+0000","last_peered":"2024-05-08T23:01:56.622558+0000","last_clean":"2024-05-08T23:01:56.622558+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:01:56.622558+0000","last_undegraded":"2024-05-08T23:01:56.622558+0000","last_fullsized":"2024-05-08T23:01:56.622558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950835+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:01:55.950835+0000","last_peered":"2024-05-08T23:01:55.950835+0000","last_clean":"2024-05-08T23:01:55.950835+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:01:55.950835+0000","last_undegraded":"2024-05-08T23:01:55.950835+0000","last_fullsized":"2024-05-08T23:01:55.950835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963347+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:01:55.963347+0000","last_peered":"2024-05-08T23:01:55.963347+0000","last_clean":"2024-05-08T23:01:55.963347+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:01:55.963347+0000","last_undegraded":"2024-05-08T23:01:55.963347+0000","last_fullsized":"2024-05-08T23:01:55.963347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131527+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:01:57.131527+0000","last_peered":"2024-05-08T23:01:57.131527+0000","last_clean":"2024-05-08T23:01:57.131527+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:01:57.131527+0000","last_undegraded":"2024-05-08T23:01:57.131527+0000","last_fullsized":"2024-05-08T23:01:57.131527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131664+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:01:57.131664+0000","last_peered":"2024-05-08T23:01:57.131664+0000","last_clean":"2024-05-08T23:01:57.131664+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:01:57.131664+0000","last_undegraded":"2024-05-08T23:01:57.131664+0000","last_fullsized":"2024-05-08T23:01:57.131664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622204+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:01:56.622204+0000","last_peered":"2024-05-08T23:01:56.622204+0000","last_clean":"2024-05-08T23:01:56.622204+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:01:56.622204+0000","last_undegraded":"2024-05-08T23:01:56.622204+0000","last_fullsized":"2024-05-08T23:01:56.622204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132822+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:01:57.132822+0000","last_peered":"2024-05-08T23:01:57.132822+0000","last_clean":"2024-05-08T23:01:57.132822+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:01:57.132822+0000","last_undegraded":"2024-05-08T23:01:57.132822+0000","last_fullsized":"2024-05-08T23:01:57.132822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.969661+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:01:55.969661+0000","last_peered":"2024-05-08T23:01:55.969661+0000","last_clean":"2024-05-08T23:01:55.969661+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:01:55.969661+0000","last_undegraded":"2024-05-08T23:01:55.969661+0000","last_fullsized":"2024-05-08T23:01:55.969661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.956179+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:01:55.956179+0000","last_peered":"2024-05-08T23:01:55.956179+0000","last_clean":"2024-05-08T23:01:55.956179+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:01:55.956179+0000","last_undegraded":"2024-05-08T23:01:55.956179+0000","last_fullsized":"2024-05-08T23:01:55.956179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005052+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:01:57.005052+0000","last_peered":"2024-05-08T23:01:57.005052+0000","last_clean":"2024-05-08T23:01:57.005052+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:01:57.005052+0000","last_undegraded":"2024-05-08T23:01:57.005052+0000","last_fullsized":"2024-05-08T23:01:57.005052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651019+0000","last_change":"2024-05-08T23:01:50.956587+0000","last_active":"2024-05-08T23:02:15.651019+0000","last_peered":"2024-05-08T23:02:15.651019+0000","last_clean":"2024-05-08T23:02:15.651019+0000","last_became_active":"2024-05-08T23:01:50.956248+0000","last_became_peered":"2024-05-08T23:01:50.956248+0000","last_unstale":"2024-05-08T23:02:15.651019+0000","last_undegraded":"2024-05-08T23:02:15.651019+0000","last_fullsized":"2024-05-08T23:02:15.651019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:54.517099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479226+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:01:56.479226+0000","last_peered":"2024-05-08T23:01:56.479226+0000","last_clean":"2024-05-08T23:01:56.479226+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:01:56.479226+0000","last_undegraded":"2024-05-08T23:01:56.479226+0000","last_fullsized":"2024-05-08T23:01:56.479226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744289+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:01:56.744289+0000","last_peered":"2024-05-08T23:01:56.744289+0000","last_clean":"2024-05-08T23:01:56.744289+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:01:56.744289+0000","last_undegraded":"2024-05-08T23:01:56.744289+0000","last_fullsized":"2024-05-08T23:01:56.744289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.649246+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:15.649246+0000","last_peered":"2024-05-08T23:02:15.649246+0000","last_clean":"2024-05-08T23:02:15.649246+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:15.649246+0000","last_undegraded":"2024-05-08T23:02:15.649246+0000","last_fullsized":"2024-05-08T23:02:15.649246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133554+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:01:57.133554+0000","last_peered":"2024-05-08T23:01:57.133554+0000","last_clean":"2024-05-08T23:01:57.133554+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:01:57.133554+0000","last_undegraded":"2024-05-08T23:01:57.133554+0000","last_fullsized":"2024-05-08T23:01:57.133554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744332+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:01:56.744332+0000","last_peered":"2024-05-08T23:01:56.744332+0000","last_clean":"2024-05-08T23:01:56.744332+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:01:56.744332+0000","last_undegraded":"2024-05-08T23:01:56.744332+0000","last_fullsized":"2024-05-08T23:01:56.744332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'11","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675690+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:17.675690+0000","last_peered":"2024-05-08T23:02:17.675690+0000","last_clean":"2024-05-08T23:02:17.675690+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:17.675690+0000","last_undegraded":"2024-05-08T23:02:17.675690+0000","last_fullsized":"2024-05-08T23:02:17.675690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622264+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:01:56.622264+0000","last_peered":"2024-05-08T23:01:56.622264+0000","last_clean":"2024-05-08T23:01:56.622264+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:01:56.622264+0000","last_undegraded":"2024-05-08T23:01:56.622264+0000","last_fullsized":"2024-05-08T23:01:56.622264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744320+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:01:56.744320+0000","last_peered":"2024-05-08T23:01:56.744320+0000","last_clean":"2024-05-08T23:01:56.744320+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:01:56.744320+0000","last_undegraded":"2024-05-08T23:01:56.744320+0000","last_fullsized":"2024-05-08T23:01:56.744320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675262+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:17.675262+0000","last_peered":"2024-05-08T23:02:17.675262+0000","last_clean":"2024-05-08T23:02:17.675262+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:17.675262+0000","last_undegraded":"2024-05-08T23:02:17.675262+0000","last_fullsized":"2024-05-08T23:02:17.675262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005132+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:01:57.005132+0000","last_peered":"2024-05-08T23:01:57.005132+0000","last_clean":"2024-05-08T23:01:57.005132+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:01:57.005132+0000","last_undegraded":"2024-05-08T23:01:57.005132+0000","last_fullsized":"2024-05-08T23:01:57.005132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.640210+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:19.640210+0000","last_peered":"2024-05-08T23:02:19.640210+0000","last_clean":"2024-05-08T23:02:19.640210+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:19.640210+0000","last_undegraded":"2024-05-08T23:02:19.640210+0000","last_fullsized":"2024-05-08T23:02:19.640210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005099+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:01:57.005099+0000","last_peered":"2024-05-08T23:01:57.005099+0000","last_clean":"2024-05-08T23:01:57.005099+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:01:57.005099+0000","last_undegraded":"2024-05-08T23:01:57.005099+0000","last_fullsized":"2024-05-08T23:01:57.005099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178543+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:01:58.178543+0000","last_peered":"2024-05-08T23:01:58.178543+0000","last_clean":"2024-05-08T23:01:58.178543+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:01:58.178543+0000","last_undegraded":"2024-05-08T23:01:58.178543+0000","last_fullsized":"2024-05-08T23:01:58.178543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650498+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:15.650498+0000","last_peered":"2024-05-08T23:02:15.650498+0000","last_clean":"2024-05-08T23:02:15.650498+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:15.650498+0000","last_undegraded":"2024-05-08T23:02:15.650498+0000","last_fullsized":"2024-05-08T23:02:15.650498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478560+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:01:56.478560+0000","last_peered":"2024-05-08T23:01:56.478560+0000","last_clean":"2024-05-08T23:01:56.478560+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:01:56.478560+0000","last_undegraded":"2024-05-08T23:01:56.478560+0000","last_fullsized":"2024-05-08T23:01:56.478560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988198+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:01:57.988198+0000","last_peered":"2024-05-08T23:01:57.988198+0000","last_clean":"2024-05-08T23:01:57.988198+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:01:57.988198+0000","last_undegraded":"2024-05-08T23:01:57.988198+0000","last_fullsized":"2024-05-08T23:01:57.988198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962314+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:01:55.962314+0000","last_peered":"2024-05-08T23:01:55.962314+0000","last_clean":"2024-05-08T23:01:55.962314+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:01:55.962314+0000","last_undegraded":"2024-05-08T23:01:55.962314+0000","last_fullsized":"2024-05-08T23:01:55.962314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865359+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:01:57.865359+0000","last_peered":"2024-05-08T23:01:57.865359+0000","last_clean":"2024-05-08T23:01:57.865359+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:01:57.865359+0000","last_undegraded":"2024-05-08T23:01:57.865359+0000","last_fullsized":"2024-05-08T23:01:57.865359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657736+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:17.657736+0000","last_peered":"2024-05-08T23:02:17.657736+0000","last_clean":"2024-05-08T23:02:17.657736+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:17.657736+0000","last_undegraded":"2024-05-08T23:02:17.657736+0000","last_fullsized":"2024-05-08T23:02:17.657736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622634+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:01:56.622634+0000","last_peered":"2024-05-08T23:01:56.622634+0000","last_clean":"2024-05-08T23:01:56.622634+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:01:56.622634+0000","last_undegraded":"2024-05-08T23:01:56.622634+0000","last_fullsized":"2024-05-08T23:01:56.622634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.978044+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:01:55.978044+0000","last_peered":"2024-05-08T23:01:55.978044+0000","last_clean":"2024-05-08T23:01:55.978044+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:01:55.978044+0000","last_undegraded":"2024-05-08T23:01:55.978044+0000","last_fullsized":"2024-05-08T23:01:55.978044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133339+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:01:57.133339+0000","last_peered":"2024-05-08T23:01:57.133339+0000","last_clean":"2024-05-08T23:01:57.133339+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:01:57.133339+0000","last_undegraded":"2024-05-08T23:01:57.133339+0000","last_fullsized":"2024-05-08T23:01:57.133339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744357+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:01:56.744357+0000","last_peered":"2024-05-08T23:01:56.744357+0000","last_clean":"2024-05-08T23:01:56.744357+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:01:56.744357+0000","last_undegraded":"2024-05-08T23:01:56.744357+0000","last_fullsized":"2024-05-08T23:01:56.744357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622409+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:01:56.622409+0000","last_peered":"2024-05-08T23:01:56.622409+0000","last_clean":"2024-05-08T23:01:56.622409+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:01:56.622409+0000","last_undegraded":"2024-05-08T23:01:56.622409+0000","last_fullsized":"2024-05-08T23:01:56.622409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977451+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:01:55.977451+0000","last_peered":"2024-05-08T23:01:55.977451+0000","last_clean":"2024-05-08T23:01:55.977451+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:01:55.977451+0000","last_undegraded":"2024-05-08T23:01:55.977451+0000","last_fullsized":"2024-05-08T23:01:55.977451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'25","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.717151+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:18.717151+0000","last_peered":"2024-05-08T23:02:18.717151+0000","last_clean":"2024-05-08T23:02:18.717151+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:18.717151+0000","last_undegraded":"2024-05-08T23:02:18.717151+0000","last_fullsized":"2024-05-08T23:02:18.717151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41862,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650769+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:15.650769+0000","last_peered":"2024-05-08T23:02:15.650769+0000","last_clean":"2024-05-08T23:02:15.650769+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:15.650769+0000","last_undegraded":"2024-05-08T23:02:15.650769+0000","last_fullsized":"2024-05-08T23:02:15.650769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963754+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:01:55.963754+0000","last_peered":"2024-05-08T23:01:55.963754+0000","last_clean":"2024-05-08T23:01:55.963754+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:01:55.963754+0000","last_undegraded":"2024-05-08T23:01:55.963754+0000","last_fullsized":"2024-05-08T23:01:55.963754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178512+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:01:58.178512+0000","last_peered":"2024-05-08T23:01:58.178512+0000","last_clean":"2024-05-08T23:01:58.178512+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:01:58.178512+0000","last_undegraded":"2024-05-08T23:01:58.178512+0000","last_fullsized":"2024-05-08T23:01:58.178512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977932+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:01:55.977932+0000","last_peered":"2024-05-08T23:01:55.977932+0000","last_clean":"2024-05-08T23:01:55.977932+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:01:55.977932+0000","last_undegraded":"2024-05-08T23:01:55.977932+0000","last_fullsized":"2024-05-08T23:01:55.977932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.655417+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:16.655417+0000","last_peered":"2024-05-08T23:02:16.655417+0000","last_clean":"2024-05-08T23:02:16.655417+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:16.655417+0000","last_undegraded":"2024-05-08T23:02:16.655417+0000","last_fullsized":"2024-05-08T23:02:16.655417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015956+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:01:57.015956+0000","last_peered":"2024-05-08T23:01:57.015956+0000","last_clean":"2024-05-08T23:01:57.015956+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:01:57.015956+0000","last_undegraded":"2024-05-08T23:01:57.015956+0000","last_fullsized":"2024-05-08T23:01:57.015956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673531+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:17.673531+0000","last_peered":"2024-05-08T23:02:17.673531+0000","last_clean":"2024-05-08T23:02:17.673531+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:17.673531+0000","last_undegraded":"2024-05-08T23:02:17.673531+0000","last_fullsized":"2024-05-08T23:02:17.673531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:01:51.470821+0000","last_change":"2024-05-08T23:02:18.181005+0000","last_active":"2024-05-08T23:02:18.181005+0000","last_peered":"2024-05-08T23:01:51.470821+0000","last_clean":"2024-05-08T23:01:51.470821+0000","last_became_active":"2024-05-08T23:01:50.955995+0000","last_became_peered":"2024-05-08T23:01:50.955995+0000","last_unstale":"2024-05-08T23:01:51.470821+0000","last_undegraded":"2024-05-08T23:01:51.470821+0000","last_fullsized":"2024-05-08T23:01:51.470821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.658743+0000","last_change":"2024-05-08T23:02:17.658905+0000","last_active":"2024-05-08T23:02:17.658743+0000","last_peered":"2024-05-08T23:01:57.132603+0000","last_clean":"2024-05-08T23:01:57.132603+0000","last_became_active":"2024-05-08T23:01:51.489719+0000","last_became_peered":"2024-05-08T23:01:51.489719+0000","last_unstale":"2024-05-08T23:02:17.658743+0000","last_undegraded":"2024-05-08T23:02:17.658743+0000","last_fullsized":"2024-05-08T23:02:17.658743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:15.412899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.967914+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:01:55.967914+0000","last_peered":"2024-05-08T23:01:55.967914+0000","last_clean":"2024-05-08T23:01:55.967914+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:01:55.967914+0000","last_undegraded":"2024-05-08T23:01:55.967914+0000","last_fullsized":"2024-05-08T23:01:55.967914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015922+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:01:57.015922+0000","last_peered":"2024-05-08T23:01:57.015922+0000","last_clean":"2024-05-08T23:01:57.015922+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:01:57.015922+0000","last_undegraded":"2024-05-08T23:01:57.015922+0000","last_fullsized":"2024-05-08T23:01:57.015922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.676766+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:17.676766+0000","last_peered":"2024-05-08T23:02:17.676766+0000","last_clean":"2024-05-08T23:02:17.676766+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:17.676766+0000","last_undegraded":"2024-05-08T23:02:17.676766+0000","last_fullsized":"2024-05-08T23:02:17.676766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015977+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:01:57.015977+0000","last_peered":"2024-05-08T23:01:57.015977+0000","last_clean":"2024-05-08T23:01:57.015977+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:01:57.015977+0000","last_undegraded":"2024-05-08T23:01:57.015977+0000","last_fullsized":"2024-05-08T23:01:57.015977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651613+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:15.651613+0000","last_peered":"2024-05-08T23:02:15.651613+0000","last_clean":"2024-05-08T23:02:15.651613+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:15.651613+0000","last_undegraded":"2024-05-08T23:02:15.651613+0000","last_fullsized":"2024-05-08T23:02:15.651613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478647+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:01:56.478647+0000","last_peered":"2024-05-08T23:01:56.478647+0000","last_clean":"2024-05-08T23:01:56.478647+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:01:56.478647+0000","last_undegraded":"2024-05-08T23:01:56.478647+0000","last_fullsized":"2024-05-08T23:01:56.478647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.654869+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:16.654869+0000","last_peered":"2024-05-08T23:02:16.654869+0000","last_clean":"2024-05-08T23:02:16.654869+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:16.654869+0000","last_undegraded":"2024-05-08T23:02:16.654869+0000","last_fullsized":"2024-05-08T23:02:16.654869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962023+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:01:55.962023+0000","last_peered":"2024-05-08T23:01:55.962023+0000","last_clean":"2024-05-08T23:01:55.962023+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:01:55.962023+0000","last_undegraded":"2024-05-08T23:01:55.962023+0000","last_fullsized":"2024-05-08T23:01:55.962023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:14.643705+0000","last_change":"2024-05-08T23:02:18.180861+0000","last_active":"2024-05-08T23:02:18.180861+0000","last_peered":"2024-05-08T23:02:14.643705+0000","last_clean":"2024-05-08T23:02:14.643705+0000","last_became_active":"2024-05-08T23:01:50.954756+0000","last_became_peered":"2024-05-08T23:01:50.954756+0000","last_unstale":"2024-05-08T23:02:14.643705+0000","last_undegraded":"2024-05-08T23:02:14.643705+0000","last_fullsized":"2024-05-08T23:02:14.643705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178486+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:01:58.178486+0000","last_peered":"2024-05-08T23:01:58.178486+0000","last_clean":"2024-05-08T23:01:58.178486+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:01:58.178486+0000","last_undegraded":"2024-05-08T23:01:58.178486+0000","last_fullsized":"2024-05-08T23:01:58.178486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674233+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:17.674233+0000","last_peered":"2024-05-08T23:02:17.674233+0000","last_clean":"2024-05-08T23:02:17.674233+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:17.674233+0000","last_undegraded":"2024-05-08T23:02:17.674233+0000","last_fullsized":"2024-05-08T23:02:17.674233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962747+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:01:55.962747+0000","last_peered":"2024-05-08T23:01:55.962747+0000","last_clean":"2024-05-08T23:01:55.962747+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:01:55.962747+0000","last_undegraded":"2024-05-08T23:01:55.962747+0000","last_fullsized":"2024-05-08T23:01:55.962747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865314+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:01:57.865314+0000","last_peered":"2024-05-08T23:01:57.865314+0000","last_clean":"2024-05-08T23:01:57.865314+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:01:57.865314+0000","last_undegraded":"2024-05-08T23:01:57.865314+0000","last_fullsized":"2024-05-08T23:01:57.865314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.648953+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:15.648953+0000","last_peered":"2024-05-08T23:02:15.648953+0000","last_clean":"2024-05-08T23:02:15.648953+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:15.648953+0000","last_undegraded":"2024-05-08T23:02:15.648953+0000","last_fullsized":"2024-05-08T23:02:15.648953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005033+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:01:57.005033+0000","last_peered":"2024-05-08T23:01:57.005033+0000","last_clean":"2024-05-08T23:01:57.005033+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:01:57.005033+0000","last_undegraded":"2024-05-08T23:01:57.005033+0000","last_fullsized":"2024-05-08T23:01:57.005033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.642858+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:14.642858+0000","last_peered":"2024-05-08T23:02:14.642858+0000","last_clean":"2024-05-08T23:02:14.642858+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:14.642858+0000","last_undegraded":"2024-05-08T23:02:14.642858+0000","last_fullsized":"2024-05-08T23:02:14.642858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133386+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:01:57.133386+0000","last_peered":"2024-05-08T23:01:57.133386+0000","last_clean":"2024-05-08T23:01:57.133386+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:01:57.133386+0000","last_undegraded":"2024-05-08T23:01:57.133386+0000","last_fullsized":"2024-05-08T23:01:57.133386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744412+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:01:56.744412+0000","last_peered":"2024-05-08T23:01:56.744412+0000","last_clean":"2024-05-08T23:01:56.744412+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:01:56.744412+0000","last_undegraded":"2024-05-08T23:01:56.744412+0000","last_fullsized":"2024-05-08T23:01:56.744412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643210+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:14.643210+0000","last_peered":"2024-05-08T23:02:14.643210+0000","last_clean":"2024-05-08T23:02:14.643210+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:14.643210+0000","last_undegraded":"2024-05-08T23:02:14.643210+0000","last_fullsized":"2024-05-08T23:02:14.643210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.975817+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:01:55.975817+0000","last_peered":"2024-05-08T23:01:55.975817+0000","last_clean":"2024-05-08T23:01:55.975817+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:01:55.975817+0000","last_undegraded":"2024-05-08T23:01:55.975817+0000","last_fullsized":"2024-05-08T23:01:55.975817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.672991+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.672991+0000","last_peered":"2024-05-08T23:02:17.672991+0000","last_clean":"2024-05-08T23:02:17.672991+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.672991+0000","last_undegraded":"2024-05-08T23:02:17.672991+0000","last_fullsized":"2024-05-08T23:02:17.672991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132676+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:01:57.132676+0000","last_peered":"2024-05-08T23:01:57.132676+0000","last_clean":"2024-05-08T23:01:57.132676+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:01:57.132676+0000","last_undegraded":"2024-05-08T23:01:57.132676+0000","last_fullsized":"2024-05-08T23:01:57.132676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744371+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:01:56.744371+0000","last_peered":"2024-05-08T23:01:56.744371+0000","last_clean":"2024-05-08T23:01:56.744371+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:01:56.744371+0000","last_undegraded":"2024-05-08T23:01:56.744371+0000","last_fullsized":"2024-05-08T23:01:56.744371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.121362+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:01:57.121362+0000","last_peered":"2024-05-08T23:01:57.121362+0000","last_clean":"2024-05-08T23:01:57.121362+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:01:57.121362+0000","last_undegraded":"2024-05-08T23:01:57.121362+0000","last_fullsized":"2024-05-08T23:01:57.121362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744399+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:01:56.744399+0000","last_peered":"2024-05-08T23:01:56.744399+0000","last_clean":"2024-05-08T23:01:56.744399+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:01:56.744399+0000","last_undegraded":"2024-05-08T23:01:56.744399+0000","last_fullsized":"2024-05-08T23:01:56.744399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653937+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:17.653937+0000","last_peered":"2024-05-08T23:02:17.653937+0000","last_clean":"2024-05-08T23:02:17.653937+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:17.653937+0000","last_undegraded":"2024-05-08T23:02:17.653937+0000","last_fullsized":"2024-05-08T23:02:17.653937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.878492+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:01:57.878492+0000","last_peered":"2024-05-08T23:01:57.878492+0000","last_clean":"2024-05-08T23:01:57.878492+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:01:57.878492+0000","last_undegraded":"2024-05-08T23:01:57.878492+0000","last_fullsized":"2024-05-08T23:01:57.878492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.658345+0000","last_change":"2024-05-08T23:01:50.962884+0000","last_active":"2024-05-08T23:02:16.658345+0000","last_peered":"2024-05-08T23:02:16.658345+0000","last_clean":"2024-05-08T23:02:16.658345+0000","last_became_active":"2024-05-08T23:01:50.962543+0000","last_became_peered":"2024-05-08T23:01:50.962543+0000","last_unstale":"2024-05-08T23:02:16.658345+0000","last_undegraded":"2024-05-08T23:02:16.658345+0000","last_fullsized":"2024-05-08T23:02:16.658345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:37.332235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988167+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:01:57.988167+0000","last_peered":"2024-05-08T23:01:57.988167+0000","last_clean":"2024-05-08T23:01:57.988167+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:01:57.988167+0000","last_undegraded":"2024-05-08T23:01:57.988167+0000","last_fullsized":"2024-05-08T23:01:57.988167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.134019+0000","last_change":"2024-05-08T23:01:50.960393+0000","last_active":"2024-05-08T23:01:57.134019+0000","last_peered":"2024-05-08T23:01:57.134019+0000","last_clean":"2024-05-08T23:01:57.134019+0000","last_became_active":"2024-05-08T23:01:50.960061+0000","last_became_peered":"2024-05-08T23:01:50.960061+0000","last_unstale":"2024-05-08T23:01:57.134019+0000","last_undegraded":"2024-05-08T23:01:57.134019+0000","last_fullsized":"2024-05-08T23:01:57.134019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:54.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132548+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:01:57.132548+0000","last_peered":"2024-05-08T23:01:57.132548+0000","last_clean":"2024-05-08T23:01:57.132548+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:01:57.132548+0000","last_undegraded":"2024-05-08T23:01:57.132548+0000","last_fullsized":"2024-05-08T23:01:57.132548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955938+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:01:55.955938+0000","last_peered":"2024-05-08T23:01:55.955938+0000","last_clean":"2024-05-08T23:01:55.955938+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:01:55.955938+0000","last_undegraded":"2024-05-08T23:01:55.955938+0000","last_fullsized":"2024-05-08T23:01:55.955938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954500+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:01:55.954500+0000","last_peered":"2024-05-08T23:01:55.954500+0000","last_clean":"2024-05-08T23:01:55.954500+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:01:55.954500+0000","last_undegraded":"2024-05-08T23:01:55.954500+0000","last_fullsized":"2024-05-08T23:01:55.954500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132006+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:01:57.132006+0000","last_peered":"2024-05-08T23:01:57.132006+0000","last_clean":"2024-05-08T23:01:57.132006+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:01:57.132006+0000","last_undegraded":"2024-05-08T23:01:57.132006+0000","last_fullsized":"2024-05-08T23:01:57.132006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962832+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:01:55.962832+0000","last_peered":"2024-05-08T23:01:55.962832+0000","last_clean":"2024-05-08T23:01:55.962832+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:01:55.962832+0000","last_undegraded":"2024-05-08T23:01:55.962832+0000","last_fullsized":"2024-05-08T23:01:55.962832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132832+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:01:57.132832+0000","last_peered":"2024-05-08T23:01:57.132832+0000","last_clean":"2024-05-08T23:01:57.132832+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:01:57.132832+0000","last_undegraded":"2024-05-08T23:01:57.132832+0000","last_fullsized":"2024-05-08T23:01:57.132832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954356+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:01:55.954356+0000","last_peered":"2024-05-08T23:01:55.954356+0000","last_clean":"2024-05-08T23:01:55.954356+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:01:55.954356+0000","last_undegraded":"2024-05-08T23:01:55.954356+0000","last_fullsized":"2024-05-08T23:01:55.954356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.651608+0000","last_change":"2024-05-08T23:02:17.651768+0000","last_active":"2024-05-08T23:02:17.651608+0000","last_peered":"2024-05-08T23:01:56.479359+0000","last_clean":"2024-05-08T23:01:56.479359+0000","last_became_active":"2024-05-08T23:01:51.493814+0000","last_became_peered":"2024-05-08T23:01:51.493814+0000","last_unstale":"2024-05-08T23:02:17.651608+0000","last_undegraded":"2024-05-08T23:02:17.651608+0000","last_fullsized":"2024-05-08T23:02:17.651608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:28.292897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478747+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:01:56.478747+0000","last_peered":"2024-05-08T23:01:56.478747+0000","last_clean":"2024-05-08T23:01:56.478747+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:01:56.478747+0000","last_undegraded":"2024-05-08T23:01:56.478747+0000","last_fullsized":"2024-05-08T23:01:56.478747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622317+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:01:56.622317+0000","last_peered":"2024-05-08T23:01:56.622317+0000","last_clean":"2024-05-08T23:01:56.622317+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:01:56.622317+0000","last_undegraded":"2024-05-08T23:01:56.622317+0000","last_fullsized":"2024-05-08T23:01:56.622317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963505+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:01:55.963505+0000","last_peered":"2024-05-08T23:01:55.963505+0000","last_clean":"2024-05-08T23:01:55.963505+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:01:55.963505+0000","last_undegraded":"2024-05-08T23:01:55.963505+0000","last_fullsized":"2024-05-08T23:01:55.963505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132462+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:01:57.132462+0000","last_peered":"2024-05-08T23:01:57.132462+0000","last_clean":"2024-05-08T23:01:57.132462+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:01:57.132462+0000","last_undegraded":"2024-05-08T23:01:57.132462+0000","last_fullsized":"2024-05-08T23:01:57.132462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47946,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":193,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":86637,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612623,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":913408,"data_stored":452932,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841554,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095383,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.85099999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":1.2829999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.3680000000000001}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":2.3719999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":2.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.79},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.4099999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.496}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.026}]}]},{"osd":9,"up_from":61,"seq":261993005078,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":443062,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.879},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":8,"up_from":54,"seq":231928234011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69899999999999995}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72899999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.86799999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.755},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":7,"up_from":47,"seq":201863462943,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":995328,"data_stored":1058837,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.499,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.63},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.55200000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.66400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.05},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.69899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.84099999999999997}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":892928,"data_stored":442368,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47599999999999998},"min":{"1min":0.441,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.60499999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78100000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.83499999999999996}]},{"osd":4,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]},{"osd":5,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.53700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.90400000000000003}]},{"osd":8,"last update":"Wed May 8 23:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":452949,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed May 8 23:01:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.626,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94999999999999996,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":1.03,"15min":1.03},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79400000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72599999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":444296,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.46899999999999997,"5min":0.25,"15min":0.25},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.38200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed May 8 23:01:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.52800000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.751},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.83899999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":446972,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 23:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.44,"15min":0.435},"min":{"1min":0.32700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.503}]},{"osd":6,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.70999999999999996,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3529999999999998,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.73999999999999999,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":7.3259999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed May 8 23:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":451909,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.44,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.443,"15min":0.41499999999999998},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.54700000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.627,"15min":0.59199999999999997},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":442565,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.442}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.63100000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":35,"seq":150323855399,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48999999999999999}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47699999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43943,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:02:21.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:02:21.261 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:02:20.921+0000 7fb1a990b700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-08T23:02:21.261 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:02:20 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:02:20.921+0000 7fb1a990b700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 -- 172.21.15.46:0/2347889448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f659410d450 msgr2=0x7f659410f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 --2- 172.21.15.46:0/2347889448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f659410d450 0x7f659410f840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f658c00b0a0 tx=0x7f658c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 -- 172.21.15.46:0/2347889448 shutdown_connections 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 --2- 172.21.15.46:0/2347889448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f659410d450 0x7f659410f840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 --2- 172.21.15.46:0/2347889448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 0x7f6594073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 --2- 172.21.15.46:0/2347889448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6594073a00 0x7f6594073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.412+0000 7f659c600700 1 -- 172.21.15.46:0/2347889448 >> 172.21.15.46:0/2347889448 conn(0x7f65940fc410 msgr2=0x7f65940fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.413+0000 7f659c600700 1 -- 172.21.15.46:0/2347889448 shutdown_connections 2024-05-08T23:02:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.413+0000 7f659c600700 1 -- 172.21.15.46:0/2347889448 wait complete. 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 Processor -- start 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 -- start start 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 0x7f659419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f659410d450 0x7f65941a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941a5050 con 0x7f6594073a00 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941a5190 con 0x7f659410d450 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941a5320 con 0x7f6594074fe0 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f6599c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f6599c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54960/0 (socket says 172.21.15.46:54960) 2024-05-08T23:02:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f6599c00700 1 -- 172.21.15.46:0/2841438846 learned_addr learned my addr 172.21.15.46:0/2841438846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f6599200700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 0x7f659419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.414+0000 7f659a600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f659410d450 0x7f65941a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 -- 172.21.15.46:0/2841438846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 msgr2=0x7f659419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 0x7f659419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 -- 172.21.15.46:0/2841438846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f659410d450 msgr2=0x7f65941a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f659410d450 0x7f65941a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 -- 172.21.15.46:0/2841438846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009d00 con 0x7f6594073a00 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6599c00700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f658800e9c0 tx=0x7f658800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880097b0 con 0x7f6594073a00 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658800ee60 con 0x7f6594073a00 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65941a5600 con 0x7f6594073a00 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880054f0 con 0x7f6594073a00 2024-05-08T23:02:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.415+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941a5ac0 con 0x7f6594073a00 2024-05-08T23:02:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.416+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65941036b0 con 0x7f6594073a00 2024-05-08T23:02:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.418+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6588004750 con 0x7f6594073a00 2024-05-08T23:02:21.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.419+0000 7f6592000700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 0x7f657c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.420+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6588014070 con 0x7f6594073a00 2024-05-08T23:02:21.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.420+0000 7f6599200700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 0x7f657c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.420+0000 7f6599200700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 0x7f657c0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6594074d00 tx=0x7f6580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:21.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.424+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f65880962f0 con 0x7f6594073a00 2024-05-08T23:02:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.639+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f659402ce10 con 0x7f657c0a11b0 2024-05-08T23:02:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.652+0000 7f6592000700 1 -- 172.21.15.46:0/2841438846 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422971 (secure 0 0 0) 0x7f659402ce10 con 0x7f657c0a11b0 2024-05-08T23:02:21.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 msgr2=0x7f657c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 0x7f657c0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6594074d00 tx=0x7f6580009260 comp rx=0 tx=0).stop 2024-05-08T23:02:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 msgr2=0x7f659419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f658800e9c0 tx=0x7f658800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 shutdown_connections 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f659410d450 0x7f65941a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f657c0a11b0 0x7f657c0a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6594073a00 0x7f659419d520 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 --2- 172.21.15.46:0/2841438846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6594074fe0 0x7f659419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 >> 172.21.15.46:0/2841438846 conn(0x7f65940fc410 msgr2=0x7f65941007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 shutdown_connections 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.657+0000 7f659c600700 1 -- 172.21.15.46:0/2841438846 wait complete. 2024-05-08T23:02:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:02:21.676 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T23:02:20.924315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638431,"num_objects":87,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":144,"num_read_kb":126,"num_write":375,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372120,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569704,"statfs":{"total":1151940427776,"available":1151559376896,"internally_reserved":0,"allocated":11288576,"data_stored":7255274,"data_compressed":279264,"data_compressed_allocated":376832,"data_compressed_original":2252800,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43610,"num_objects":63,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":147,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132805+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:01:57.132805+0000","last_peered":"2024-05-08T23:01:57.132805+0000","last_clean":"2024-05-08T23:01:57.132805+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:01:57.132805+0000","last_undegraded":"2024-05-08T23:01:57.132805+0000","last_fullsized":"2024-05-08T23:01:57.132805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133301+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:01:57.133301+0000","last_peered":"2024-05-08T23:01:57.133301+0000","last_clean":"2024-05-08T23:01:57.133301+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:01:57.133301+0000","last_undegraded":"2024-05-08T23:01:57.133301+0000","last_fullsized":"2024-05-08T23:01:57.133301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132530+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:01:57.132530+0000","last_peered":"2024-05-08T23:01:57.132530+0000","last_clean":"2024-05-08T23:01:57.132530+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:01:57.132530+0000","last_undegraded":"2024-05-08T23:01:57.132530+0000","last_fullsized":"2024-05-08T23:01:57.132530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950890+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:01:55.950890+0000","last_peered":"2024-05-08T23:01:55.950890+0000","last_clean":"2024-05-08T23:01:55.950890+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:01:55.950890+0000","last_undegraded":"2024-05-08T23:01:55.950890+0000","last_fullsized":"2024-05-08T23:01:55.950890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478678+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:01:56.478678+0000","last_peered":"2024-05-08T23:01:56.478678+0000","last_clean":"2024-05-08T23:01:56.478678+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:01:56.478678+0000","last_undegraded":"2024-05-08T23:01:56.478678+0000","last_fullsized":"2024-05-08T23:01:56.478678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622558+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:01:56.622558+0000","last_peered":"2024-05-08T23:01:56.622558+0000","last_clean":"2024-05-08T23:01:56.622558+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:01:56.622558+0000","last_undegraded":"2024-05-08T23:01:56.622558+0000","last_fullsized":"2024-05-08T23:01:56.622558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950835+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:01:55.950835+0000","last_peered":"2024-05-08T23:01:55.950835+0000","last_clean":"2024-05-08T23:01:55.950835+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:01:55.950835+0000","last_undegraded":"2024-05-08T23:01:55.950835+0000","last_fullsized":"2024-05-08T23:01:55.950835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963347+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:01:55.963347+0000","last_peered":"2024-05-08T23:01:55.963347+0000","last_clean":"2024-05-08T23:01:55.963347+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:01:55.963347+0000","last_undegraded":"2024-05-08T23:01:55.963347+0000","last_fullsized":"2024-05-08T23:01:55.963347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131527+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:01:57.131527+0000","last_peered":"2024-05-08T23:01:57.131527+0000","last_clean":"2024-05-08T23:01:57.131527+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:01:57.131527+0000","last_undegraded":"2024-05-08T23:01:57.131527+0000","last_fullsized":"2024-05-08T23:01:57.131527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131664+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:01:57.131664+0000","last_peered":"2024-05-08T23:01:57.131664+0000","last_clean":"2024-05-08T23:01:57.131664+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:01:57.131664+0000","last_undegraded":"2024-05-08T23:01:57.131664+0000","last_fullsized":"2024-05-08T23:01:57.131664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622204+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:01:56.622204+0000","last_peered":"2024-05-08T23:01:56.622204+0000","last_clean":"2024-05-08T23:01:56.622204+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:01:56.622204+0000","last_undegraded":"2024-05-08T23:01:56.622204+0000","last_fullsized":"2024-05-08T23:01:56.622204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132822+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:01:57.132822+0000","last_peered":"2024-05-08T23:01:57.132822+0000","last_clean":"2024-05-08T23:01:57.132822+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:01:57.132822+0000","last_undegraded":"2024-05-08T23:01:57.132822+0000","last_fullsized":"2024-05-08T23:01:57.132822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.969661+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:01:55.969661+0000","last_peered":"2024-05-08T23:01:55.969661+0000","last_clean":"2024-05-08T23:01:55.969661+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:01:55.969661+0000","last_undegraded":"2024-05-08T23:01:55.969661+0000","last_fullsized":"2024-05-08T23:01:55.969661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.956179+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:01:55.956179+0000","last_peered":"2024-05-08T23:01:55.956179+0000","last_clean":"2024-05-08T23:01:55.956179+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:01:55.956179+0000","last_undegraded":"2024-05-08T23:01:55.956179+0000","last_fullsized":"2024-05-08T23:01:55.956179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005052+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:01:57.005052+0000","last_peered":"2024-05-08T23:01:57.005052+0000","last_clean":"2024-05-08T23:01:57.005052+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:01:57.005052+0000","last_undegraded":"2024-05-08T23:01:57.005052+0000","last_fullsized":"2024-05-08T23:01:57.005052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651019+0000","last_change":"2024-05-08T23:01:50.956587+0000","last_active":"2024-05-08T23:02:15.651019+0000","last_peered":"2024-05-08T23:02:15.651019+0000","last_clean":"2024-05-08T23:02:15.651019+0000","last_became_active":"2024-05-08T23:01:50.956248+0000","last_became_peered":"2024-05-08T23:01:50.956248+0000","last_unstale":"2024-05-08T23:02:15.651019+0000","last_undegraded":"2024-05-08T23:02:15.651019+0000","last_fullsized":"2024-05-08T23:02:15.651019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:54.517099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479226+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:01:56.479226+0000","last_peered":"2024-05-08T23:01:56.479226+0000","last_clean":"2024-05-08T23:01:56.479226+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:01:56.479226+0000","last_undegraded":"2024-05-08T23:01:56.479226+0000","last_fullsized":"2024-05-08T23:01:56.479226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744289+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:01:56.744289+0000","last_peered":"2024-05-08T23:01:56.744289+0000","last_clean":"2024-05-08T23:01:56.744289+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:01:56.744289+0000","last_undegraded":"2024-05-08T23:01:56.744289+0000","last_fullsized":"2024-05-08T23:01:56.744289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.649246+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:15.649246+0000","last_peered":"2024-05-08T23:02:15.649246+0000","last_clean":"2024-05-08T23:02:15.649246+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:15.649246+0000","last_undegraded":"2024-05-08T23:02:15.649246+0000","last_fullsized":"2024-05-08T23:02:15.649246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133554+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:01:57.133554+0000","last_peered":"2024-05-08T23:01:57.133554+0000","last_clean":"2024-05-08T23:01:57.133554+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:01:57.133554+0000","last_undegraded":"2024-05-08T23:01:57.133554+0000","last_fullsized":"2024-05-08T23:01:57.133554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744332+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:01:56.744332+0000","last_peered":"2024-05-08T23:01:56.744332+0000","last_clean":"2024-05-08T23:01:56.744332+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:01:56.744332+0000","last_undegraded":"2024-05-08T23:01:56.744332+0000","last_fullsized":"2024-05-08T23:01:56.744332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'11","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675690+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:17.675690+0000","last_peered":"2024-05-08T23:02:17.675690+0000","last_clean":"2024-05-08T23:02:17.675690+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:17.675690+0000","last_undegraded":"2024-05-08T23:02:17.675690+0000","last_fullsized":"2024-05-08T23:02:17.675690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622264+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:01:56.622264+0000","last_peered":"2024-05-08T23:01:56.622264+0000","last_clean":"2024-05-08T23:01:56.622264+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:01:56.622264+0000","last_undegraded":"2024-05-08T23:01:56.622264+0000","last_fullsized":"2024-05-08T23:01:56.622264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744320+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:01:56.744320+0000","last_peered":"2024-05-08T23:01:56.744320+0000","last_clean":"2024-05-08T23:01:56.744320+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:01:56.744320+0000","last_undegraded":"2024-05-08T23:01:56.744320+0000","last_fullsized":"2024-05-08T23:01:56.744320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675262+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:17.675262+0000","last_peered":"2024-05-08T23:02:17.675262+0000","last_clean":"2024-05-08T23:02:17.675262+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:17.675262+0000","last_undegraded":"2024-05-08T23:02:17.675262+0000","last_fullsized":"2024-05-08T23:02:17.675262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005132+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:01:57.005132+0000","last_peered":"2024-05-08T23:01:57.005132+0000","last_clean":"2024-05-08T23:01:57.005132+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:01:57.005132+0000","last_undegraded":"2024-05-08T23:01:57.005132+0000","last_fullsized":"2024-05-08T23:01:57.005132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.640210+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:19.640210+0000","last_peered":"2024-05-08T23:02:19.640210+0000","last_clean":"2024-05-08T23:02:19.640210+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:19.640210+0000","last_undegraded":"2024-05-08T23:02:19.640210+0000","last_fullsized":"2024-05-08T23:02:19.640210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005099+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:01:57.005099+0000","last_peered":"2024-05-08T23:01:57.005099+0000","last_clean":"2024-05-08T23:01:57.005099+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:01:57.005099+0000","last_undegraded":"2024-05-08T23:01:57.005099+0000","last_fullsized":"2024-05-08T23:01:57.005099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178543+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:01:58.178543+0000","last_peered":"2024-05-08T23:01:58.178543+0000","last_clean":"2024-05-08T23:01:58.178543+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:01:58.178543+0000","last_undegraded":"2024-05-08T23:01:58.178543+0000","last_fullsized":"2024-05-08T23:01:58.178543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650498+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:15.650498+0000","last_peered":"2024-05-08T23:02:15.650498+0000","last_clean":"2024-05-08T23:02:15.650498+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:15.650498+0000","last_undegraded":"2024-05-08T23:02:15.650498+0000","last_fullsized":"2024-05-08T23:02:15.650498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478560+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:01:56.478560+0000","last_peered":"2024-05-08T23:01:56.478560+0000","last_clean":"2024-05-08T23:01:56.478560+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:01:56.478560+0000","last_undegraded":"2024-05-08T23:01:56.478560+0000","last_fullsized":"2024-05-08T23:01:56.478560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988198+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:01:57.988198+0000","last_peered":"2024-05-08T23:01:57.988198+0000","last_clean":"2024-05-08T23:01:57.988198+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:01:57.988198+0000","last_undegraded":"2024-05-08T23:01:57.988198+0000","last_fullsized":"2024-05-08T23:01:57.988198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962314+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:01:55.962314+0000","last_peered":"2024-05-08T23:01:55.962314+0000","last_clean":"2024-05-08T23:01:55.962314+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:01:55.962314+0000","last_undegraded":"2024-05-08T23:01:55.962314+0000","last_fullsized":"2024-05-08T23:01:55.962314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865359+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:01:57.865359+0000","last_peered":"2024-05-08T23:01:57.865359+0000","last_clean":"2024-05-08T23:01:57.865359+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:01:57.865359+0000","last_undegraded":"2024-05-08T23:01:57.865359+0000","last_fullsized":"2024-05-08T23:01:57.865359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657736+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:17.657736+0000","last_peered":"2024-05-08T23:02:17.657736+0000","last_clean":"2024-05-08T23:02:17.657736+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:17.657736+0000","last_undegraded":"2024-05-08T23:02:17.657736+0000","last_fullsized":"2024-05-08T23:02:17.657736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622634+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:01:56.622634+0000","last_peered":"2024-05-08T23:01:56.622634+0000","last_clean":"2024-05-08T23:01:56.622634+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:01:56.622634+0000","last_undegraded":"2024-05-08T23:01:56.622634+0000","last_fullsized":"2024-05-08T23:01:56.622634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.978044+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:01:55.978044+0000","last_peered":"2024-05-08T23:01:55.978044+0000","last_clean":"2024-05-08T23:01:55.978044+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:01:55.978044+0000","last_undegraded":"2024-05-08T23:01:55.978044+0000","last_fullsized":"2024-05-08T23:01:55.978044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133339+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:01:57.133339+0000","last_peered":"2024-05-08T23:01:57.133339+0000","last_clean":"2024-05-08T23:01:57.133339+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:01:57.133339+0000","last_undegraded":"2024-05-08T23:01:57.133339+0000","last_fullsized":"2024-05-08T23:01:57.133339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744357+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:01:56.744357+0000","last_peered":"2024-05-08T23:01:56.744357+0000","last_clean":"2024-05-08T23:01:56.744357+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:01:56.744357+0000","last_undegraded":"2024-05-08T23:01:56.744357+0000","last_fullsized":"2024-05-08T23:01:56.744357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622409+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:01:56.622409+0000","last_peered":"2024-05-08T23:01:56.622409+0000","last_clean":"2024-05-08T23:01:56.622409+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:01:56.622409+0000","last_undegraded":"2024-05-08T23:01:56.622409+0000","last_fullsized":"2024-05-08T23:01:56.622409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977451+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:01:55.977451+0000","last_peered":"2024-05-08T23:01:55.977451+0000","last_clean":"2024-05-08T23:01:55.977451+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:01:55.977451+0000","last_undegraded":"2024-05-08T23:01:55.977451+0000","last_fullsized":"2024-05-08T23:01:55.977451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'25","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.717151+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:18.717151+0000","last_peered":"2024-05-08T23:02:18.717151+0000","last_clean":"2024-05-08T23:02:18.717151+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:18.717151+0000","last_undegraded":"2024-05-08T23:02:18.717151+0000","last_fullsized":"2024-05-08T23:02:18.717151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41862,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650769+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:15.650769+0000","last_peered":"2024-05-08T23:02:15.650769+0000","last_clean":"2024-05-08T23:02:15.650769+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:15.650769+0000","last_undegraded":"2024-05-08T23:02:15.650769+0000","last_fullsized":"2024-05-08T23:02:15.650769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963754+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:01:55.963754+0000","last_peered":"2024-05-08T23:01:55.963754+0000","last_clean":"2024-05-08T23:01:55.963754+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:01:55.963754+0000","last_undegraded":"2024-05-08T23:01:55.963754+0000","last_fullsized":"2024-05-08T23:01:55.963754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178512+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:01:58.178512+0000","last_peered":"2024-05-08T23:01:58.178512+0000","last_clean":"2024-05-08T23:01:58.178512+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:01:58.178512+0000","last_undegraded":"2024-05-08T23:01:58.178512+0000","last_fullsized":"2024-05-08T23:01:58.178512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977932+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:01:55.977932+0000","last_peered":"2024-05-08T23:01:55.977932+0000","last_clean":"2024-05-08T23:01:55.977932+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:01:55.977932+0000","last_undegraded":"2024-05-08T23:01:55.977932+0000","last_fullsized":"2024-05-08T23:01:55.977932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.655417+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:16.655417+0000","last_peered":"2024-05-08T23:02:16.655417+0000","last_clean":"2024-05-08T23:02:16.655417+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:16.655417+0000","last_undegraded":"2024-05-08T23:02:16.655417+0000","last_fullsized":"2024-05-08T23:02:16.655417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015956+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:01:57.015956+0000","last_peered":"2024-05-08T23:01:57.015956+0000","last_clean":"2024-05-08T23:01:57.015956+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:01:57.015956+0000","last_undegraded":"2024-05-08T23:01:57.015956+0000","last_fullsized":"2024-05-08T23:01:57.015956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673531+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:17.673531+0000","last_peered":"2024-05-08T23:02:17.673531+0000","last_clean":"2024-05-08T23:02:17.673531+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:17.673531+0000","last_undegraded":"2024-05-08T23:02:17.673531+0000","last_fullsized":"2024-05-08T23:02:17.673531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:01:51.470821+0000","last_change":"2024-05-08T23:02:18.181005+0000","last_active":"2024-05-08T23:02:18.181005+0000","last_peered":"2024-05-08T23:01:51.470821+0000","last_clean":"2024-05-08T23:01:51.470821+0000","last_became_active":"2024-05-08T23:01:50.955995+0000","last_became_peered":"2024-05-08T23:01:50.955995+0000","last_unstale":"2024-05-08T23:01:51.470821+0000","last_undegraded":"2024-05-08T23:01:51.470821+0000","last_fullsized":"2024-05-08T23:01:51.470821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.658743+0000","last_change":"2024-05-08T23:02:17.658905+0000","last_active":"2024-05-08T23:02:17.658743+0000","last_peered":"2024-05-08T23:01:57.132603+0000","last_clean":"2024-05-08T23:01:57.132603+0000","last_became_active":"2024-05-08T23:01:51.489719+0000","last_became_peered":"2024-05-08T23:01:51.489719+0000","last_unstale":"2024-05-08T23:02:17.658743+0000","last_undegraded":"2024-05-08T23:02:17.658743+0000","last_fullsized":"2024-05-08T23:02:17.658743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:15.412899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.967914+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:01:55.967914+0000","last_peered":"2024-05-08T23:01:55.967914+0000","last_clean":"2024-05-08T23:01:55.967914+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:01:55.967914+0000","last_undegraded":"2024-05-08T23:01:55.967914+0000","last_fullsized":"2024-05-08T23:01:55.967914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015922+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:01:57.015922+0000","last_peered":"2024-05-08T23:01:57.015922+0000","last_clean":"2024-05-08T23:01:57.015922+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:01:57.015922+0000","last_undegraded":"2024-05-08T23:01:57.015922+0000","last_fullsized":"2024-05-08T23:01:57.015922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.676766+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:17.676766+0000","last_peered":"2024-05-08T23:02:17.676766+0000","last_clean":"2024-05-08T23:02:17.676766+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:17.676766+0000","last_undegraded":"2024-05-08T23:02:17.676766+0000","last_fullsized":"2024-05-08T23:02:17.676766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015977+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:01:57.015977+0000","last_peered":"2024-05-08T23:01:57.015977+0000","last_clean":"2024-05-08T23:01:57.015977+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:01:57.015977+0000","last_undegraded":"2024-05-08T23:01:57.015977+0000","last_fullsized":"2024-05-08T23:01:57.015977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651613+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:15.651613+0000","last_peered":"2024-05-08T23:02:15.651613+0000","last_clean":"2024-05-08T23:02:15.651613+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:15.651613+0000","last_undegraded":"2024-05-08T23:02:15.651613+0000","last_fullsized":"2024-05-08T23:02:15.651613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478647+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:01:56.478647+0000","last_peered":"2024-05-08T23:01:56.478647+0000","last_clean":"2024-05-08T23:01:56.478647+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:01:56.478647+0000","last_undegraded":"2024-05-08T23:01:56.478647+0000","last_fullsized":"2024-05-08T23:01:56.478647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.654869+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:16.654869+0000","last_peered":"2024-05-08T23:02:16.654869+0000","last_clean":"2024-05-08T23:02:16.654869+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:16.654869+0000","last_undegraded":"2024-05-08T23:02:16.654869+0000","last_fullsized":"2024-05-08T23:02:16.654869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962023+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:01:55.962023+0000","last_peered":"2024-05-08T23:01:55.962023+0000","last_clean":"2024-05-08T23:01:55.962023+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:01:55.962023+0000","last_undegraded":"2024-05-08T23:01:55.962023+0000","last_fullsized":"2024-05-08T23:01:55.962023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:14.643705+0000","last_change":"2024-05-08T23:02:18.180861+0000","last_active":"2024-05-08T23:02:18.180861+0000","last_peered":"2024-05-08T23:02:14.643705+0000","last_clean":"2024-05-08T23:02:14.643705+0000","last_became_active":"2024-05-08T23:01:50.954756+0000","last_became_peered":"2024-05-08T23:01:50.954756+0000","last_unstale":"2024-05-08T23:02:14.643705+0000","last_undegraded":"2024-05-08T23:02:14.643705+0000","last_fullsized":"2024-05-08T23:02:14.643705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178486+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:01:58.178486+0000","last_peered":"2024-05-08T23:01:58.178486+0000","last_clean":"2024-05-08T23:01:58.178486+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:01:58.178486+0000","last_undegraded":"2024-05-08T23:01:58.178486+0000","last_fullsized":"2024-05-08T23:01:58.178486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674233+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:17.674233+0000","last_peered":"2024-05-08T23:02:17.674233+0000","last_clean":"2024-05-08T23:02:17.674233+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:17.674233+0000","last_undegraded":"2024-05-08T23:02:17.674233+0000","last_fullsized":"2024-05-08T23:02:17.674233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962747+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:01:55.962747+0000","last_peered":"2024-05-08T23:01:55.962747+0000","last_clean":"2024-05-08T23:01:55.962747+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:01:55.962747+0000","last_undegraded":"2024-05-08T23:01:55.962747+0000","last_fullsized":"2024-05-08T23:01:55.962747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865314+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:01:57.865314+0000","last_peered":"2024-05-08T23:01:57.865314+0000","last_clean":"2024-05-08T23:01:57.865314+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:01:57.865314+0000","last_undegraded":"2024-05-08T23:01:57.865314+0000","last_fullsized":"2024-05-08T23:01:57.865314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.648953+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:15.648953+0000","last_peered":"2024-05-08T23:02:15.648953+0000","last_clean":"2024-05-08T23:02:15.648953+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:15.648953+0000","last_undegraded":"2024-05-08T23:02:15.648953+0000","last_fullsized":"2024-05-08T23:02:15.648953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005033+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:01:57.005033+0000","last_peered":"2024-05-08T23:01:57.005033+0000","last_clean":"2024-05-08T23:01:57.005033+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:01:57.005033+0000","last_undegraded":"2024-05-08T23:01:57.005033+0000","last_fullsized":"2024-05-08T23:01:57.005033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.642858+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:14.642858+0000","last_peered":"2024-05-08T23:02:14.642858+0000","last_clean":"2024-05-08T23:02:14.642858+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:14.642858+0000","last_undegraded":"2024-05-08T23:02:14.642858+0000","last_fullsized":"2024-05-08T23:02:14.642858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133386+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:01:57.133386+0000","last_peered":"2024-05-08T23:01:57.133386+0000","last_clean":"2024-05-08T23:01:57.133386+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:01:57.133386+0000","last_undegraded":"2024-05-08T23:01:57.133386+0000","last_fullsized":"2024-05-08T23:01:57.133386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744412+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:01:56.744412+0000","last_peered":"2024-05-08T23:01:56.744412+0000","last_clean":"2024-05-08T23:01:56.744412+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:01:56.744412+0000","last_undegraded":"2024-05-08T23:01:56.744412+0000","last_fullsized":"2024-05-08T23:01:56.744412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643210+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:14.643210+0000","last_peered":"2024-05-08T23:02:14.643210+0000","last_clean":"2024-05-08T23:02:14.643210+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:14.643210+0000","last_undegraded":"2024-05-08T23:02:14.643210+0000","last_fullsized":"2024-05-08T23:02:14.643210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.975817+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:01:55.975817+0000","last_peered":"2024-05-08T23:01:55.975817+0000","last_clean":"2024-05-08T23:01:55.975817+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:01:55.975817+0000","last_undegraded":"2024-05-08T23:01:55.975817+0000","last_fullsized":"2024-05-08T23:01:55.975817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.672991+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.672991+0000","last_peered":"2024-05-08T23:02:17.672991+0000","last_clean":"2024-05-08T23:02:17.672991+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.672991+0000","last_undegraded":"2024-05-08T23:02:17.672991+0000","last_fullsized":"2024-05-08T23:02:17.672991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132676+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:01:57.132676+0000","last_peered":"2024-05-08T23:01:57.132676+0000","last_clean":"2024-05-08T23:01:57.132676+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:01:57.132676+0000","last_undegraded":"2024-05-08T23:01:57.132676+0000","last_fullsized":"2024-05-08T23:01:57.132676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744371+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:01:56.744371+0000","last_peered":"2024-05-08T23:01:56.744371+0000","last_clean":"2024-05-08T23:01:56.744371+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:01:56.744371+0000","last_undegraded":"2024-05-08T23:01:56.744371+0000","last_fullsized":"2024-05-08T23:01:56.744371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.121362+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:01:57.121362+0000","last_peered":"2024-05-08T23:01:57.121362+0000","last_clean":"2024-05-08T23:01:57.121362+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:01:57.121362+0000","last_undegraded":"2024-05-08T23:01:57.121362+0000","last_fullsized":"2024-05-08T23:01:57.121362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744399+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:01:56.744399+0000","last_peered":"2024-05-08T23:01:56.744399+0000","last_clean":"2024-05-08T23:01:56.744399+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:01:56.744399+0000","last_undegraded":"2024-05-08T23:01:56.744399+0000","last_fullsized":"2024-05-08T23:01:56.744399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653937+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:17.653937+0000","last_peered":"2024-05-08T23:02:17.653937+0000","last_clean":"2024-05-08T23:02:17.653937+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:17.653937+0000","last_undegraded":"2024-05-08T23:02:17.653937+0000","last_fullsized":"2024-05-08T23:02:17.653937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.878492+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:01:57.878492+0000","last_peered":"2024-05-08T23:01:57.878492+0000","last_clean":"2024-05-08T23:01:57.878492+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:01:57.878492+0000","last_undegraded":"2024-05-08T23:01:57.878492+0000","last_fullsized":"2024-05-08T23:01:57.878492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.658345+0000","last_change":"2024-05-08T23:01:50.962884+0000","last_active":"2024-05-08T23:02:16.658345+0000","last_peered":"2024-05-08T23:02:16.658345+0000","last_clean":"2024-05-08T23:02:16.658345+0000","last_became_active":"2024-05-08T23:01:50.962543+0000","last_became_peered":"2024-05-08T23:01:50.962543+0000","last_unstale":"2024-05-08T23:02:16.658345+0000","last_undegraded":"2024-05-08T23:02:16.658345+0000","last_fullsized":"2024-05-08T23:02:16.658345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:37.332235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988167+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:01:57.988167+0000","last_peered":"2024-05-08T23:01:57.988167+0000","last_clean":"2024-05-08T23:01:57.988167+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:01:57.988167+0000","last_undegraded":"2024-05-08T23:01:57.988167+0000","last_fullsized":"2024-05-08T23:01:57.988167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.134019+0000","last_change":"2024-05-08T23:01:50.960393+0000","last_active":"2024-05-08T23:01:57.134019+0000","last_peered":"2024-05-08T23:01:57.134019+0000","last_clean":"2024-05-08T23:01:57.134019+0000","last_became_active":"2024-05-08T23:01:50.960061+0000","last_became_peered":"2024-05-08T23:01:50.960061+0000","last_unstale":"2024-05-08T23:01:57.134019+0000","last_undegraded":"2024-05-08T23:01:57.134019+0000","last_fullsized":"2024-05-08T23:01:57.134019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:54.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132548+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:01:57.132548+0000","last_peered":"2024-05-08T23:01:57.132548+0000","last_clean":"2024-05-08T23:01:57.132548+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:01:57.132548+0000","last_undegraded":"2024-05-08T23:01:57.132548+0000","last_fullsized":"2024-05-08T23:01:57.132548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955938+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:01:55.955938+0000","last_peered":"2024-05-08T23:01:55.955938+0000","last_clean":"2024-05-08T23:01:55.955938+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:01:55.955938+0000","last_undegraded":"2024-05-08T23:01:55.955938+0000","last_fullsized":"2024-05-08T23:01:55.955938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954500+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:01:55.954500+0000","last_peered":"2024-05-08T23:01:55.954500+0000","last_clean":"2024-05-08T23:01:55.954500+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:01:55.954500+0000","last_undegraded":"2024-05-08T23:01:55.954500+0000","last_fullsized":"2024-05-08T23:01:55.954500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132006+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:01:57.132006+0000","last_peered":"2024-05-08T23:01:57.132006+0000","last_clean":"2024-05-08T23:01:57.132006+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:01:57.132006+0000","last_undegraded":"2024-05-08T23:01:57.132006+0000","last_fullsized":"2024-05-08T23:01:57.132006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962832+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:01:55.962832+0000","last_peered":"2024-05-08T23:01:55.962832+0000","last_clean":"2024-05-08T23:01:55.962832+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:01:55.962832+0000","last_undegraded":"2024-05-08T23:01:55.962832+0000","last_fullsized":"2024-05-08T23:01:55.962832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132832+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:01:57.132832+0000","last_peered":"2024-05-08T23:01:57.132832+0000","last_clean":"2024-05-08T23:01:57.132832+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:01:57.132832+0000","last_undegraded":"2024-05-08T23:01:57.132832+0000","last_fullsized":"2024-05-08T23:01:57.132832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954356+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:01:55.954356+0000","last_peered":"2024-05-08T23:01:55.954356+0000","last_clean":"2024-05-08T23:01:55.954356+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:01:55.954356+0000","last_undegraded":"2024-05-08T23:01:55.954356+0000","last_fullsized":"2024-05-08T23:01:55.954356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.651608+0000","last_change":"2024-05-08T23:02:17.651768+0000","last_active":"2024-05-08T23:02:17.651608+0000","last_peered":"2024-05-08T23:01:56.479359+0000","last_clean":"2024-05-08T23:01:56.479359+0000","last_became_active":"2024-05-08T23:01:51.493814+0000","last_became_peered":"2024-05-08T23:01:51.493814+0000","last_unstale":"2024-05-08T23:02:17.651608+0000","last_undegraded":"2024-05-08T23:02:17.651608+0000","last_fullsized":"2024-05-08T23:02:17.651608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:28.292897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478747+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:01:56.478747+0000","last_peered":"2024-05-08T23:01:56.478747+0000","last_clean":"2024-05-08T23:01:56.478747+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:01:56.478747+0000","last_undegraded":"2024-05-08T23:01:56.478747+0000","last_fullsized":"2024-05-08T23:01:56.478747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622317+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:01:56.622317+0000","last_peered":"2024-05-08T23:01:56.622317+0000","last_clean":"2024-05-08T23:01:56.622317+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:01:56.622317+0000","last_undegraded":"2024-05-08T23:01:56.622317+0000","last_fullsized":"2024-05-08T23:01:56.622317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963505+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:01:55.963505+0000","last_peered":"2024-05-08T23:01:55.963505+0000","last_clean":"2024-05-08T23:01:55.963505+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:01:55.963505+0000","last_undegraded":"2024-05-08T23:01:55.963505+0000","last_fullsized":"2024-05-08T23:01:55.963505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132462+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:01:57.132462+0000","last_peered":"2024-05-08T23:01:57.132462+0000","last_clean":"2024-05-08T23:01:57.132462+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:01:57.132462+0000","last_undegraded":"2024-05-08T23:01:57.132462+0000","last_fullsized":"2024-05-08T23:01:57.132462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47946,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":193,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":86637,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612623,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":913408,"data_stored":452932,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841554,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095383,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.85099999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":1.2829999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.3680000000000001}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":2.3719999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":2.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.79},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.4099999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.496}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.026}]}]},{"osd":9,"up_from":61,"seq":261993005078,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":443062,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.879},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":8,"up_from":54,"seq":231928234011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69899999999999995}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72899999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.86799999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.755},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":7,"up_from":47,"seq":201863462943,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":995328,"data_stored":1058837,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.499,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.63},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.55200000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.66400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.05},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.69899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.84099999999999997}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":892928,"data_stored":442368,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47599999999999998},"min":{"1min":0.441,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.60499999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78100000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.83499999999999996}]},{"osd":4,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]},{"osd":5,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.53700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.90400000000000003}]},{"osd":8,"last update":"Wed May 8 23:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":452949,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed May 8 23:01:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.626,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94999999999999996,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":1.03,"15min":1.03},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79400000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72599999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":444296,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.46899999999999997,"5min":0.25,"15min":0.25},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.38200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed May 8 23:01:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.52800000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.751},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.83899999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":446972,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 23:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.44,"15min":0.435},"min":{"1min":0.32700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.503}]},{"osd":6,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.70999999999999996,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3529999999999998,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.73999999999999999,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":7.3259999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed May 8 23:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":451909,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.44,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.443,"15min":0.41499999999999998},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.54700000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.627,"15min":0.59199999999999997},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":442565,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.442}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.63100000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":35,"seq":150323855399,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48999999999999999}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47699999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43943,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "77'1", 'reported_seq': 2, 'reported_epoch': 83, 'state': 'peering', 'last_fresh': '2024-05-08T23:01:51.470821+0000', 'last_change': '2024-05-08T23:02:18.181005+0000', 'last_active': '2024-05-08T23:02:18.181005+0000', 'last_peered': '2024-05-08T23:01:51.470821+0000', 'last_clean': '2024-05-08T23:01:51.470821+0000', 'last_became_active': '2024-05-08T23:01:50.955995+0000', 'last_became_peered': '2024-05-08T23:01:50.955995+0000', 'last_unstale': '2024-05-08T23:01:51.470821+0000', 'last_undegraded': '2024-05-08T23:01:51.470821+0000', 'last_fullsized': '2024-05-08T23:01:51.470821+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 76, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'last_clean_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 5, 4], 'acting': [11, 5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 17, 'reported_epoch': 83, 'state': 'peering', 'last_fresh': '2024-05-08T23:02:17.658743+0000', 'last_change': '2024-05-08T23:02:17.658905+0000', 'last_active': '2024-05-08T23:02:17.658743+0000', 'last_peered': '2024-05-08T23:01:57.132603+0000', 'last_clean': '2024-05-08T23:01:57.132603+0000', 'last_became_active': '2024-05-08T23:01:51.489719+0000', 'last_became_peered': '2024-05-08T23:01:51.489719+0000', 'last_unstale': '2024-05-08T23:02:17.658743+0000', 'last_undegraded': '2024-05-08T23:02:17.658743+0000', 'last_fullsized': '2024-05-08T23:02:17.658743+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 77, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'last_clean_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:23:15.412899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 9, 4], 'acting': [11, 9, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4, 5, 9], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "82'3", 'reported_seq': 2, 'reported_epoch': 83, 'state': 'peering', 'last_fresh': '2024-05-08T23:02:14.643705+0000', 'last_change': '2024-05-08T23:02:18.180861+0000', 'last_active': '2024-05-08T23:02:18.180861+0000', 'last_peered': '2024-05-08T23:02:14.643705+0000', 'last_clean': '2024-05-08T23:02:14.643705+0000', 'last_became_active': '2024-05-08T23:01:50.954756+0000', 'last_became_peered': '2024-05-08T23:01:50.954756+0000', 'last_unstale': '2024-05-08T23:02:14.643705+0000', 'last_undegraded': '2024-05-08T23:02:14.643705+0000', 'last_fullsized': '2024-05-08T23:02:14.643705+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 75, 'last_epoch_clean': 76, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'last_clean_scrub_stamp': '2024-05-08T23:01:49.924676+0000', 'objects_scrubbed': 0, 'log_size': 3, 'log_dups_size': 0, 'ondisk_log_size': 3, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 1, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 5, 8], 'acting': [11, 5, 8], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-05-08T23:02:21.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 17, 'reported_epoch': 83, 'state': 'peering', 'last_fresh': '2024-05-08T23:02:17.651608+0000', 'last_change': '2024-05-08T23:02:17.651768+0000', 'last_active': '2024-05-08T23:02:17.651608+0000', 'last_peered': '2024-05-08T23:01:56.479359+0000', 'last_clean': '2024-05-08T23:01:56.479359+0000', 'last_became_active': '2024-05-08T23:01:51.493814+0000', 'last_became_peered': '2024-05-08T23:01:51.493814+0000', 'last_unstale': '2024-05-08T23:02:17.651608+0000', 'last_undegraded': '2024-05-08T23:02:17.651608+0000', 'last_fullsized': '2024-05-08T23:02:17.651608+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 77, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'last_clean_scrub_stamp': '2024-05-08T23:01:50.930938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:57:28.292897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6, 2], 'acting': [3, 6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2, 4, 6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:02:21.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 -- 172.21.15.46:0/2944768825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244101ea0 msgr2=0x7f6244102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 --2- 172.21.15.46:0/2944768825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244101ea0 0x7f6244102300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62300099c0 tx=0x7f6230009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 -- 172.21.15.46:0/2944768825 shutdown_connections 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 --2- 172.21.15.46:0/2944768825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244102840 0x7f62441090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 --2- 172.21.15.46:0/2944768825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244100cb0 0x7f62441010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 --2- 172.21.15.46:0/2944768825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244101ea0 0x7f6244102300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 -- 172.21.15.46:0/2944768825 >> 172.21.15.46:0/2944768825 conn(0x7f62440fc220 msgr2=0x7f62440fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 -- 172.21.15.46:0/2944768825 shutdown_connections 2024-05-08T23:02:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.914+0000 7f624aa00700 1 -- 172.21.15.46:0/2944768825 wait complete. 2024-05-08T23:02:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.915+0000 7f624aa00700 1 Processor -- start 2024-05-08T23:02:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.915+0000 7f624aa00700 1 -- start start 2024-05-08T23:02:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 0x7f6244198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244101ea0 0x7f62441993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 0x7f62441a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a0990 con 0x7f6244102840 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a0ad0 con 0x7f6244101ea0 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a0c60 con 0x7f6244100cb0 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 0x7f62441a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 0x7f62441a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46468/0 (socket says 172.21.15.46:46468) 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f624a000700 1 -- 172.21.15.46:0/1105475548 learned_addr learned my addr 172.21.15.46:0/1105475548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f6249600700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 0x7f6244198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.916+0000 7f6248c00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244101ea0 0x7f62441993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 -- 172.21.15.46:0/1105475548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244101ea0 msgr2=0x7f62441993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244101ea0 0x7f62441993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 -- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 msgr2=0x7f62441a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:21.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 0x7f62441a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 -- 172.21.15.46:0/1105475548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009c40 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6249600700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 0x7f6244198eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f623800d720 tx=0x7f623800da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238004a30 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62380054e0 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62300095f0 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800fc30 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.917+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441a1200 con 0x7f6244100cb0 2024-05-08T23:02:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.918+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624404ecd0 con 0x7f6244100cb0 2024-05-08T23:02:21.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.920+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f623800fd90 con 0x7f6244100cb0 2024-05-08T23:02:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.921+0000 7f6242000700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 0x7f622c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.921+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f62380cccc0 con 0x7f6244100cb0 2024-05-08T23:02:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.922+0000 7f6248c00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 0x7f622c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.922+0000 7f6248c00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 0x7f622c0a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f623000b580 tx=0x7f623001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:21.924+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6238095f80 con 0x7f6244100cb0 2024-05-08T23:02:21.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:21 smithi086 ceph-mon[78610]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:21.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:21 smithi086 ceph-mon[78610]: pgmap v256: 129 pgs: 4 peering, 125 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 10 KiB/s wr, 19 op/s 2024-05-08T23:02:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:21 smithi053 ceph-mon[77587]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:22.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:21 smithi053 ceph-mon[77587]: pgmap v256: 129 pgs: 4 peering, 125 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 10 KiB/s wr, 19 op/s 2024-05-08T23:02:22.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:21 smithi046 ceph-mon[36898]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:22.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:21 smithi046 ceph-mon[36898]: pgmap v256: 129 pgs: 4 peering, 125 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 10 KiB/s wr, 19 op/s 2024-05-08T23:02:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.236+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f624410c510 con 0x7f6244100cb0 2024-05-08T23:02:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.237+0000 7f6242000700 1 -- 172.21.15.46:0/1105475548 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3461 (secure 0 0 0) 0x7f623801fda0 con 0x7f6244100cb0 2024-05-08T23:02:22.237 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:22.237 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":318,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1715209272,"num_in_osds":12,"osd_in_since":1715209255,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":125},{"state_name":"peering","count":4}],"num_pgs":129,"num_pools":4,"num_objects":87,"data_bytes":638431,"bytes_used":381050880,"bytes_avail":1151559376896,"bytes_total":1151940427776,"inactive_pgs_ratio":0.031007751822471619,"read_bytes_sec":895,"write_bytes_sec":10749,"read_op_per_sec":0,"write_op_per_sec":18},"fsmap":{"epoch":26,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"k","status":"up:standby-replay","gid":24476},{"filesystem_id":1,"rank":3,"name":"l","status":"up:standby-replay","gid":24490},{"filesystem_id":1,"rank":2,"name":"d","status":"up:standby-replay","gid":24500},{"filesystem_id":1,"rank":1,"name":"e","status":"up:standby-replay","gid":24512},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24539},{"filesystem_id":1,"rank":4,"name":"f","status":"up:active","gid":24541},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24547},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24571},{"filesystem_id":1,"rank":1,"name":"g","status":"up:active","gid":24583},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24587}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.46:8443/"}},"servicemap":{"epoch":18,"modified":"2024-05-08T23:02:14.923217+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:02:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 msgr2=0x7f622c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 0x7f622c0a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f623000b580 tx=0x7f623001a040 comp rx=0 tx=0).stop 2024-05-08T23:02:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 msgr2=0x7f6244198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 0x7f6244198eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f623800d720 tx=0x7f623800da30 comp rx=0 tx=0).stop 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 shutdown_connections 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f622c0a11b0 0x7f622c0a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6244102840 0x7f62441a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6244100cb0 0x7f6244198eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 --2- 172.21.15.46:0/1105475548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6244101ea0 0x7f62441993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 >> 172.21.15.46:0/1105475548 conn(0x7f62440fc220 msgr2=0x7f62440731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 shutdown_connections 2024-05-08T23:02:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.239+0000 7f624aa00700 1 -- 172.21.15.46:0/1105475548 wait complete. 2024-05-08T23:02:22.259 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T23:02:22.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 -- 172.21.15.46:0/515644605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104230 msgr2=0x7f7410106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 --2- 172.21.15.46:0/515644605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104230 0x7f7410106620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009990 tx=0x7f73fc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 -- 172.21.15.46:0/515644605 shutdown_connections 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 --2- 172.21.15.46:0/515644605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410106b60 0x7f7410108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 --2- 172.21.15.46:0/515644605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 --2- 172.21.15.46:0/515644605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104230 0x7f7410106620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.457+0000 7f7415a00700 1 -- 172.21.15.46:0/515644605 >> 172.21.15.46:0/515644605 conn(0x7f74100fb310 msgr2=0x7f74100fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.458+0000 7f7415a00700 1 -- 172.21.15.46:0/515644605 shutdown_connections 2024-05-08T23:02:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.458+0000 7f7415a00700 1 -- 172.21.15.46:0/515644605 wait complete. 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 Processor -- start 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 -- start start 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410106b60 0x7f74101a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a0b20 con 0x7f7410106b60 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a0c60 con 0x7f7410101900 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f7415a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74101a0df0 con 0x7f7410104230 2024-05-08T23:02:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41484/0 (socket says 172.21.15.46:41484) 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48986/0 (socket says 172.21.15.46:48986) 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740e000700 1 -- 172.21.15.46:0/1346052947 learned_addr learned my addr 172.21.15.46:0/1346052947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.459+0000 7f740f400700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410106b60 0x7f74101a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 -- 172.21.15.46:0/1346052947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 msgr2=0x7f7410199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 -- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410106b60 msgr2=0x7f74101a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410106b60 0x7f74101a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 -- 172.21.15.46:0/1346052947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0095f0 con 0x7f7410101900 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740e000700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f740ea00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9f0 tx=0x7f740000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000c9c0 con 0x7f7410101900 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74101a10d0 con 0x7f7410101900 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7400010430 con 0x7f7410101900 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400018680 con 0x7f7410101900 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.460+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74101a15e0 con 0x7f7410101900 2024-05-08T23:02:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.461+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7410108730 con 0x7f7410101900 2024-05-08T23:02:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.463+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7400010860 con 0x7f7410101900 2024-05-08T23:02:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.465+0000 7f7407400700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 0x7f73f40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.465+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7400014070 con 0x7f7410101900 2024-05-08T23:02:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.465+0000 7f740e000700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 0x7f73f40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.465+0000 7f740e000700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 0x7f73f40a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b2e0 tx=0x7f73fc00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.468+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7400096190 con 0x7f7410101900 2024-05-08T23:02:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.681+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f741002ce10 con 0x7f73f40a11b0 2024-05-08T23:02:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.694+0000 7f7407400700 1 -- 172.21.15.46:0/1346052947 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422971 (secure 0 0 0) 0x7f741002ce10 con 0x7f73f40a11b0 2024-05-08T23:02:22.700 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 msgr2=0x7f73f40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 0x7f73f40a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b2e0 tx=0x7f73fc00b750 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 msgr2=0x7f7410198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9f0 tx=0x7f740000ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 shutdown_connections 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f73f40a11b0 0x7f73f40a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410106b60 0x7f74101a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7410101900 0x7f7410198ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 --2- 172.21.15.46:0/1346052947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7410104230 0x7f7410199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 >> 172.21.15.46:0/1346052947 conn(0x7f74100fb310 msgr2=0x7f7410101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 shutdown_connections 2024-05-08T23:02:22.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:22.699+0000 7f7415a00700 1 -- 172.21.15.46:0/1346052947 wait complete. 2024-05-08T23:02:22.705 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:02:22.719 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T23:02:20.924315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638431,"num_objects":87,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":144,"num_read_kb":126,"num_write":375,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372120,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569704,"statfs":{"total":1151940427776,"available":1151559376896,"internally_reserved":0,"allocated":11288576,"data_stored":7255274,"data_compressed":279264,"data_compressed_allocated":376832,"data_compressed_original":2252800,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43610,"num_objects":63,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":147,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132805+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:01:57.132805+0000","last_peered":"2024-05-08T23:01:57.132805+0000","last_clean":"2024-05-08T23:01:57.132805+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:01:57.132805+0000","last_undegraded":"2024-05-08T23:01:57.132805+0000","last_fullsized":"2024-05-08T23:01:57.132805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133301+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:01:57.133301+0000","last_peered":"2024-05-08T23:01:57.133301+0000","last_clean":"2024-05-08T23:01:57.133301+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:01:57.133301+0000","last_undegraded":"2024-05-08T23:01:57.133301+0000","last_fullsized":"2024-05-08T23:01:57.133301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132530+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:01:57.132530+0000","last_peered":"2024-05-08T23:01:57.132530+0000","last_clean":"2024-05-08T23:01:57.132530+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:01:57.132530+0000","last_undegraded":"2024-05-08T23:01:57.132530+0000","last_fullsized":"2024-05-08T23:01:57.132530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950890+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:01:55.950890+0000","last_peered":"2024-05-08T23:01:55.950890+0000","last_clean":"2024-05-08T23:01:55.950890+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:01:55.950890+0000","last_undegraded":"2024-05-08T23:01:55.950890+0000","last_fullsized":"2024-05-08T23:01:55.950890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478678+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:01:56.478678+0000","last_peered":"2024-05-08T23:01:56.478678+0000","last_clean":"2024-05-08T23:01:56.478678+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:01:56.478678+0000","last_undegraded":"2024-05-08T23:01:56.478678+0000","last_fullsized":"2024-05-08T23:01:56.478678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622558+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:01:56.622558+0000","last_peered":"2024-05-08T23:01:56.622558+0000","last_clean":"2024-05-08T23:01:56.622558+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:01:56.622558+0000","last_undegraded":"2024-05-08T23:01:56.622558+0000","last_fullsized":"2024-05-08T23:01:56.622558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.950835+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:01:55.950835+0000","last_peered":"2024-05-08T23:01:55.950835+0000","last_clean":"2024-05-08T23:01:55.950835+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:01:55.950835+0000","last_undegraded":"2024-05-08T23:01:55.950835+0000","last_fullsized":"2024-05-08T23:01:55.950835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963347+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:01:55.963347+0000","last_peered":"2024-05-08T23:01:55.963347+0000","last_clean":"2024-05-08T23:01:55.963347+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:01:55.963347+0000","last_undegraded":"2024-05-08T23:01:55.963347+0000","last_fullsized":"2024-05-08T23:01:55.963347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131527+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:01:57.131527+0000","last_peered":"2024-05-08T23:01:57.131527+0000","last_clean":"2024-05-08T23:01:57.131527+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:01:57.131527+0000","last_undegraded":"2024-05-08T23:01:57.131527+0000","last_fullsized":"2024-05-08T23:01:57.131527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.131664+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:01:57.131664+0000","last_peered":"2024-05-08T23:01:57.131664+0000","last_clean":"2024-05-08T23:01:57.131664+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:01:57.131664+0000","last_undegraded":"2024-05-08T23:01:57.131664+0000","last_fullsized":"2024-05-08T23:01:57.131664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622204+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:01:56.622204+0000","last_peered":"2024-05-08T23:01:56.622204+0000","last_clean":"2024-05-08T23:01:56.622204+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:01:56.622204+0000","last_undegraded":"2024-05-08T23:01:56.622204+0000","last_fullsized":"2024-05-08T23:01:56.622204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132822+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:01:57.132822+0000","last_peered":"2024-05-08T23:01:57.132822+0000","last_clean":"2024-05-08T23:01:57.132822+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:01:57.132822+0000","last_undegraded":"2024-05-08T23:01:57.132822+0000","last_fullsized":"2024-05-08T23:01:57.132822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.969661+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:01:55.969661+0000","last_peered":"2024-05-08T23:01:55.969661+0000","last_clean":"2024-05-08T23:01:55.969661+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:01:55.969661+0000","last_undegraded":"2024-05-08T23:01:55.969661+0000","last_fullsized":"2024-05-08T23:01:55.969661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.956179+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:01:55.956179+0000","last_peered":"2024-05-08T23:01:55.956179+0000","last_clean":"2024-05-08T23:01:55.956179+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:01:55.956179+0000","last_undegraded":"2024-05-08T23:01:55.956179+0000","last_fullsized":"2024-05-08T23:01:55.956179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005052+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:01:57.005052+0000","last_peered":"2024-05-08T23:01:57.005052+0000","last_clean":"2024-05-08T23:01:57.005052+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:01:57.005052+0000","last_undegraded":"2024-05-08T23:01:57.005052+0000","last_fullsized":"2024-05-08T23:01:57.005052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651019+0000","last_change":"2024-05-08T23:01:50.956587+0000","last_active":"2024-05-08T23:02:15.651019+0000","last_peered":"2024-05-08T23:02:15.651019+0000","last_clean":"2024-05-08T23:02:15.651019+0000","last_became_active":"2024-05-08T23:01:50.956248+0000","last_became_peered":"2024-05-08T23:01:50.956248+0000","last_unstale":"2024-05-08T23:02:15.651019+0000","last_undegraded":"2024-05-08T23:02:15.651019+0000","last_fullsized":"2024-05-08T23:02:15.651019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:54.517099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.479226+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:01:56.479226+0000","last_peered":"2024-05-08T23:01:56.479226+0000","last_clean":"2024-05-08T23:01:56.479226+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:01:56.479226+0000","last_undegraded":"2024-05-08T23:01:56.479226+0000","last_fullsized":"2024-05-08T23:01:56.479226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744289+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:01:56.744289+0000","last_peered":"2024-05-08T23:01:56.744289+0000","last_clean":"2024-05-08T23:01:56.744289+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:01:56.744289+0000","last_undegraded":"2024-05-08T23:01:56.744289+0000","last_fullsized":"2024-05-08T23:01:56.744289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.649246+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:15.649246+0000","last_peered":"2024-05-08T23:02:15.649246+0000","last_clean":"2024-05-08T23:02:15.649246+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:15.649246+0000","last_undegraded":"2024-05-08T23:02:15.649246+0000","last_fullsized":"2024-05-08T23:02:15.649246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133554+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:01:57.133554+0000","last_peered":"2024-05-08T23:01:57.133554+0000","last_clean":"2024-05-08T23:01:57.133554+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:01:57.133554+0000","last_undegraded":"2024-05-08T23:01:57.133554+0000","last_fullsized":"2024-05-08T23:01:57.133554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744332+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:01:56.744332+0000","last_peered":"2024-05-08T23:01:56.744332+0000","last_clean":"2024-05-08T23:01:56.744332+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:01:56.744332+0000","last_undegraded":"2024-05-08T23:01:56.744332+0000","last_fullsized":"2024-05-08T23:01:56.744332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'11","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675690+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:17.675690+0000","last_peered":"2024-05-08T23:02:17.675690+0000","last_clean":"2024-05-08T23:02:17.675690+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:17.675690+0000","last_undegraded":"2024-05-08T23:02:17.675690+0000","last_fullsized":"2024-05-08T23:02:17.675690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622264+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:01:56.622264+0000","last_peered":"2024-05-08T23:01:56.622264+0000","last_clean":"2024-05-08T23:01:56.622264+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:01:56.622264+0000","last_undegraded":"2024-05-08T23:01:56.622264+0000","last_fullsized":"2024-05-08T23:01:56.622264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744320+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:01:56.744320+0000","last_peered":"2024-05-08T23:01:56.744320+0000","last_clean":"2024-05-08T23:01:56.744320+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:01:56.744320+0000","last_undegraded":"2024-05-08T23:01:56.744320+0000","last_fullsized":"2024-05-08T23:01:56.744320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675262+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:17.675262+0000","last_peered":"2024-05-08T23:02:17.675262+0000","last_clean":"2024-05-08T23:02:17.675262+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:17.675262+0000","last_undegraded":"2024-05-08T23:02:17.675262+0000","last_fullsized":"2024-05-08T23:02:17.675262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005132+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:01:57.005132+0000","last_peered":"2024-05-08T23:01:57.005132+0000","last_clean":"2024-05-08T23:01:57.005132+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:01:57.005132+0000","last_undegraded":"2024-05-08T23:01:57.005132+0000","last_fullsized":"2024-05-08T23:01:57.005132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.640210+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:19.640210+0000","last_peered":"2024-05-08T23:02:19.640210+0000","last_clean":"2024-05-08T23:02:19.640210+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:19.640210+0000","last_undegraded":"2024-05-08T23:02:19.640210+0000","last_fullsized":"2024-05-08T23:02:19.640210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005099+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:01:57.005099+0000","last_peered":"2024-05-08T23:01:57.005099+0000","last_clean":"2024-05-08T23:01:57.005099+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:01:57.005099+0000","last_undegraded":"2024-05-08T23:01:57.005099+0000","last_fullsized":"2024-05-08T23:01:57.005099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178543+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:01:58.178543+0000","last_peered":"2024-05-08T23:01:58.178543+0000","last_clean":"2024-05-08T23:01:58.178543+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:01:58.178543+0000","last_undegraded":"2024-05-08T23:01:58.178543+0000","last_fullsized":"2024-05-08T23:01:58.178543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650498+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:15.650498+0000","last_peered":"2024-05-08T23:02:15.650498+0000","last_clean":"2024-05-08T23:02:15.650498+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:15.650498+0000","last_undegraded":"2024-05-08T23:02:15.650498+0000","last_fullsized":"2024-05-08T23:02:15.650498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478560+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:01:56.478560+0000","last_peered":"2024-05-08T23:01:56.478560+0000","last_clean":"2024-05-08T23:01:56.478560+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:01:56.478560+0000","last_undegraded":"2024-05-08T23:01:56.478560+0000","last_fullsized":"2024-05-08T23:01:56.478560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988198+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:01:57.988198+0000","last_peered":"2024-05-08T23:01:57.988198+0000","last_clean":"2024-05-08T23:01:57.988198+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:01:57.988198+0000","last_undegraded":"2024-05-08T23:01:57.988198+0000","last_fullsized":"2024-05-08T23:01:57.988198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962314+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:01:55.962314+0000","last_peered":"2024-05-08T23:01:55.962314+0000","last_clean":"2024-05-08T23:01:55.962314+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:01:55.962314+0000","last_undegraded":"2024-05-08T23:01:55.962314+0000","last_fullsized":"2024-05-08T23:01:55.962314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865359+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:01:57.865359+0000","last_peered":"2024-05-08T23:01:57.865359+0000","last_clean":"2024-05-08T23:01:57.865359+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:01:57.865359+0000","last_undegraded":"2024-05-08T23:01:57.865359+0000","last_fullsized":"2024-05-08T23:01:57.865359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657736+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:17.657736+0000","last_peered":"2024-05-08T23:02:17.657736+0000","last_clean":"2024-05-08T23:02:17.657736+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:17.657736+0000","last_undegraded":"2024-05-08T23:02:17.657736+0000","last_fullsized":"2024-05-08T23:02:17.657736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622634+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:01:56.622634+0000","last_peered":"2024-05-08T23:01:56.622634+0000","last_clean":"2024-05-08T23:01:56.622634+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:01:56.622634+0000","last_undegraded":"2024-05-08T23:01:56.622634+0000","last_fullsized":"2024-05-08T23:01:56.622634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.978044+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:01:55.978044+0000","last_peered":"2024-05-08T23:01:55.978044+0000","last_clean":"2024-05-08T23:01:55.978044+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:01:55.978044+0000","last_undegraded":"2024-05-08T23:01:55.978044+0000","last_fullsized":"2024-05-08T23:01:55.978044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133339+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:01:57.133339+0000","last_peered":"2024-05-08T23:01:57.133339+0000","last_clean":"2024-05-08T23:01:57.133339+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:01:57.133339+0000","last_undegraded":"2024-05-08T23:01:57.133339+0000","last_fullsized":"2024-05-08T23:01:57.133339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744357+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:01:56.744357+0000","last_peered":"2024-05-08T23:01:56.744357+0000","last_clean":"2024-05-08T23:01:56.744357+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:01:56.744357+0000","last_undegraded":"2024-05-08T23:01:56.744357+0000","last_fullsized":"2024-05-08T23:01:56.744357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622409+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:01:56.622409+0000","last_peered":"2024-05-08T23:01:56.622409+0000","last_clean":"2024-05-08T23:01:56.622409+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:01:56.622409+0000","last_undegraded":"2024-05-08T23:01:56.622409+0000","last_fullsized":"2024-05-08T23:01:56.622409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977451+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:01:55.977451+0000","last_peered":"2024-05-08T23:01:55.977451+0000","last_clean":"2024-05-08T23:01:55.977451+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:01:55.977451+0000","last_undegraded":"2024-05-08T23:01:55.977451+0000","last_fullsized":"2024-05-08T23:01:55.977451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'25","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.717151+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:18.717151+0000","last_peered":"2024-05-08T23:02:18.717151+0000","last_clean":"2024-05-08T23:02:18.717151+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:18.717151+0000","last_undegraded":"2024-05-08T23:02:18.717151+0000","last_fullsized":"2024-05-08T23:02:18.717151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41862,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.650769+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:15.650769+0000","last_peered":"2024-05-08T23:02:15.650769+0000","last_clean":"2024-05-08T23:02:15.650769+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:15.650769+0000","last_undegraded":"2024-05-08T23:02:15.650769+0000","last_fullsized":"2024-05-08T23:02:15.650769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963754+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:01:55.963754+0000","last_peered":"2024-05-08T23:01:55.963754+0000","last_clean":"2024-05-08T23:01:55.963754+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:01:55.963754+0000","last_undegraded":"2024-05-08T23:01:55.963754+0000","last_fullsized":"2024-05-08T23:01:55.963754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178512+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:01:58.178512+0000","last_peered":"2024-05-08T23:01:58.178512+0000","last_clean":"2024-05-08T23:01:58.178512+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:01:58.178512+0000","last_undegraded":"2024-05-08T23:01:58.178512+0000","last_fullsized":"2024-05-08T23:01:58.178512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.977932+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:01:55.977932+0000","last_peered":"2024-05-08T23:01:55.977932+0000","last_clean":"2024-05-08T23:01:55.977932+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:01:55.977932+0000","last_undegraded":"2024-05-08T23:01:55.977932+0000","last_fullsized":"2024-05-08T23:01:55.977932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.655417+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:16.655417+0000","last_peered":"2024-05-08T23:02:16.655417+0000","last_clean":"2024-05-08T23:02:16.655417+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:16.655417+0000","last_undegraded":"2024-05-08T23:02:16.655417+0000","last_fullsized":"2024-05-08T23:02:16.655417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015956+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:01:57.015956+0000","last_peered":"2024-05-08T23:01:57.015956+0000","last_clean":"2024-05-08T23:01:57.015956+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:01:57.015956+0000","last_undegraded":"2024-05-08T23:01:57.015956+0000","last_fullsized":"2024-05-08T23:01:57.015956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673531+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:17.673531+0000","last_peered":"2024-05-08T23:02:17.673531+0000","last_clean":"2024-05-08T23:02:17.673531+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:17.673531+0000","last_undegraded":"2024-05-08T23:02:17.673531+0000","last_fullsized":"2024-05-08T23:02:17.673531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:01:51.470821+0000","last_change":"2024-05-08T23:02:18.181005+0000","last_active":"2024-05-08T23:02:18.181005+0000","last_peered":"2024-05-08T23:01:51.470821+0000","last_clean":"2024-05-08T23:01:51.470821+0000","last_became_active":"2024-05-08T23:01:50.955995+0000","last_became_peered":"2024-05-08T23:01:50.955995+0000","last_unstale":"2024-05-08T23:01:51.470821+0000","last_undegraded":"2024-05-08T23:01:51.470821+0000","last_fullsized":"2024-05-08T23:01:51.470821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.658743+0000","last_change":"2024-05-08T23:02:17.658905+0000","last_active":"2024-05-08T23:02:17.658743+0000","last_peered":"2024-05-08T23:01:57.132603+0000","last_clean":"2024-05-08T23:01:57.132603+0000","last_became_active":"2024-05-08T23:01:51.489719+0000","last_became_peered":"2024-05-08T23:01:51.489719+0000","last_unstale":"2024-05-08T23:02:17.658743+0000","last_undegraded":"2024-05-08T23:02:17.658743+0000","last_fullsized":"2024-05-08T23:02:17.658743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:15.412899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,9],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.967914+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:01:55.967914+0000","last_peered":"2024-05-08T23:01:55.967914+0000","last_clean":"2024-05-08T23:01:55.967914+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:01:55.967914+0000","last_undegraded":"2024-05-08T23:01:55.967914+0000","last_fullsized":"2024-05-08T23:01:55.967914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015922+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:01:57.015922+0000","last_peered":"2024-05-08T23:01:57.015922+0000","last_clean":"2024-05-08T23:01:57.015922+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:01:57.015922+0000","last_undegraded":"2024-05-08T23:01:57.015922+0000","last_fullsized":"2024-05-08T23:01:57.015922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.676766+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:17.676766+0000","last_peered":"2024-05-08T23:02:17.676766+0000","last_clean":"2024-05-08T23:02:17.676766+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:17.676766+0000","last_undegraded":"2024-05-08T23:02:17.676766+0000","last_fullsized":"2024-05-08T23:02:17.676766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.015977+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:01:57.015977+0000","last_peered":"2024-05-08T23:01:57.015977+0000","last_clean":"2024-05-08T23:01:57.015977+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:01:57.015977+0000","last_undegraded":"2024-05-08T23:01:57.015977+0000","last_fullsized":"2024-05-08T23:01:57.015977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.651613+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:15.651613+0000","last_peered":"2024-05-08T23:02:15.651613+0000","last_clean":"2024-05-08T23:02:15.651613+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:15.651613+0000","last_undegraded":"2024-05-08T23:02:15.651613+0000","last_fullsized":"2024-05-08T23:02:15.651613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478647+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:01:56.478647+0000","last_peered":"2024-05-08T23:01:56.478647+0000","last_clean":"2024-05-08T23:01:56.478647+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:01:56.478647+0000","last_undegraded":"2024-05-08T23:01:56.478647+0000","last_fullsized":"2024-05-08T23:01:56.478647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.654869+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:16.654869+0000","last_peered":"2024-05-08T23:02:16.654869+0000","last_clean":"2024-05-08T23:02:16.654869+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:16.654869+0000","last_undegraded":"2024-05-08T23:02:16.654869+0000","last_fullsized":"2024-05-08T23:02:16.654869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962023+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:01:55.962023+0000","last_peered":"2024-05-08T23:01:55.962023+0000","last_clean":"2024-05-08T23:01:55.962023+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:01:55.962023+0000","last_undegraded":"2024-05-08T23:01:55.962023+0000","last_fullsized":"2024-05-08T23:01:55.962023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:14.643705+0000","last_change":"2024-05-08T23:02:18.180861+0000","last_active":"2024-05-08T23:02:18.180861+0000","last_peered":"2024-05-08T23:02:14.643705+0000","last_clean":"2024-05-08T23:02:14.643705+0000","last_became_active":"2024-05-08T23:01:50.954756+0000","last_became_peered":"2024-05-08T23:01:50.954756+0000","last_unstale":"2024-05-08T23:02:14.643705+0000","last_undegraded":"2024-05-08T23:02:14.643705+0000","last_fullsized":"2024-05-08T23:02:14.643705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:58.178486+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:01:58.178486+0000","last_peered":"2024-05-08T23:01:58.178486+0000","last_clean":"2024-05-08T23:01:58.178486+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:01:58.178486+0000","last_undegraded":"2024-05-08T23:01:58.178486+0000","last_fullsized":"2024-05-08T23:01:58.178486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674233+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:17.674233+0000","last_peered":"2024-05-08T23:02:17.674233+0000","last_clean":"2024-05-08T23:02:17.674233+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:17.674233+0000","last_undegraded":"2024-05-08T23:02:17.674233+0000","last_fullsized":"2024-05-08T23:02:17.674233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962747+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:01:55.962747+0000","last_peered":"2024-05-08T23:01:55.962747+0000","last_clean":"2024-05-08T23:01:55.962747+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:01:55.962747+0000","last_undegraded":"2024-05-08T23:01:55.962747+0000","last_fullsized":"2024-05-08T23:01:55.962747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.865314+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:01:57.865314+0000","last_peered":"2024-05-08T23:01:57.865314+0000","last_clean":"2024-05-08T23:01:57.865314+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:01:57.865314+0000","last_undegraded":"2024-05-08T23:01:57.865314+0000","last_fullsized":"2024-05-08T23:01:57.865314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:15.648953+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:15.648953+0000","last_peered":"2024-05-08T23:02:15.648953+0000","last_clean":"2024-05-08T23:02:15.648953+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:15.648953+0000","last_undegraded":"2024-05-08T23:02:15.648953+0000","last_fullsized":"2024-05-08T23:02:15.648953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.005033+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:01:57.005033+0000","last_peered":"2024-05-08T23:01:57.005033+0000","last_clean":"2024-05-08T23:01:57.005033+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:01:57.005033+0000","last_undegraded":"2024-05-08T23:01:57.005033+0000","last_fullsized":"2024-05-08T23:01:57.005033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.642858+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:14.642858+0000","last_peered":"2024-05-08T23:02:14.642858+0000","last_clean":"2024-05-08T23:02:14.642858+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:14.642858+0000","last_undegraded":"2024-05-08T23:02:14.642858+0000","last_fullsized":"2024-05-08T23:02:14.642858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.133386+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:01:57.133386+0000","last_peered":"2024-05-08T23:01:57.133386+0000","last_clean":"2024-05-08T23:01:57.133386+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:01:57.133386+0000","last_undegraded":"2024-05-08T23:01:57.133386+0000","last_fullsized":"2024-05-08T23:01:57.133386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744412+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:01:56.744412+0000","last_peered":"2024-05-08T23:01:56.744412+0000","last_clean":"2024-05-08T23:01:56.744412+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:01:56.744412+0000","last_undegraded":"2024-05-08T23:01:56.744412+0000","last_fullsized":"2024-05-08T23:01:56.744412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:14.643210+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:14.643210+0000","last_peered":"2024-05-08T23:02:14.643210+0000","last_clean":"2024-05-08T23:02:14.643210+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:14.643210+0000","last_undegraded":"2024-05-08T23:02:14.643210+0000","last_fullsized":"2024-05-08T23:02:14.643210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.975817+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:01:55.975817+0000","last_peered":"2024-05-08T23:01:55.975817+0000","last_clean":"2024-05-08T23:01:55.975817+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:01:55.975817+0000","last_undegraded":"2024-05-08T23:01:55.975817+0000","last_fullsized":"2024-05-08T23:01:55.975817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.672991+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.672991+0000","last_peered":"2024-05-08T23:02:17.672991+0000","last_clean":"2024-05-08T23:02:17.672991+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.672991+0000","last_undegraded":"2024-05-08T23:02:17.672991+0000","last_fullsized":"2024-05-08T23:02:17.672991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132676+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:01:57.132676+0000","last_peered":"2024-05-08T23:01:57.132676+0000","last_clean":"2024-05-08T23:01:57.132676+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:01:57.132676+0000","last_undegraded":"2024-05-08T23:01:57.132676+0000","last_fullsized":"2024-05-08T23:01:57.132676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744371+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:01:56.744371+0000","last_peered":"2024-05-08T23:01:56.744371+0000","last_clean":"2024-05-08T23:01:56.744371+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:01:56.744371+0000","last_undegraded":"2024-05-08T23:01:56.744371+0000","last_fullsized":"2024-05-08T23:01:56.744371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.121362+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:01:57.121362+0000","last_peered":"2024-05-08T23:01:57.121362+0000","last_clean":"2024-05-08T23:01:57.121362+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:01:57.121362+0000","last_undegraded":"2024-05-08T23:01:57.121362+0000","last_fullsized":"2024-05-08T23:01:57.121362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478966+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:01:56.478966+0000","last_peered":"2024-05-08T23:01:56.478966+0000","last_clean":"2024-05-08T23:01:56.478966+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:01:56.478966+0000","last_undegraded":"2024-05-08T23:01:56.478966+0000","last_fullsized":"2024-05-08T23:01:56.478966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.744399+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:01:56.744399+0000","last_peered":"2024-05-08T23:01:56.744399+0000","last_clean":"2024-05-08T23:01:56.744399+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:01:56.744399+0000","last_undegraded":"2024-05-08T23:01:56.744399+0000","last_fullsized":"2024-05-08T23:01:56.744399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653937+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:17.653937+0000","last_peered":"2024-05-08T23:02:17.653937+0000","last_clean":"2024-05-08T23:02:17.653937+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:17.653937+0000","last_undegraded":"2024-05-08T23:02:17.653937+0000","last_fullsized":"2024-05-08T23:02:17.653937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.878492+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:01:57.878492+0000","last_peered":"2024-05-08T23:01:57.878492+0000","last_clean":"2024-05-08T23:01:57.878492+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:01:57.878492+0000","last_undegraded":"2024-05-08T23:01:57.878492+0000","last_fullsized":"2024-05-08T23:01:57.878492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:02:16.658345+0000","last_change":"2024-05-08T23:01:50.962884+0000","last_active":"2024-05-08T23:02:16.658345+0000","last_peered":"2024-05-08T23:02:16.658345+0000","last_clean":"2024-05-08T23:02:16.658345+0000","last_became_active":"2024-05-08T23:01:50.962543+0000","last_became_peered":"2024-05-08T23:01:50.962543+0000","last_unstale":"2024-05-08T23:02:16.658345+0000","last_undegraded":"2024-05-08T23:02:16.658345+0000","last_fullsized":"2024-05-08T23:02:16.658345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:37.332235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.988167+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:01:57.988167+0000","last_peered":"2024-05-08T23:01:57.988167+0000","last_clean":"2024-05-08T23:01:57.988167+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:01:57.988167+0000","last_undegraded":"2024-05-08T23:01:57.988167+0000","last_fullsized":"2024-05-08T23:01:57.988167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.134019+0000","last_change":"2024-05-08T23:01:50.960393+0000","last_active":"2024-05-08T23:01:57.134019+0000","last_peered":"2024-05-08T23:01:57.134019+0000","last_clean":"2024-05-08T23:01:57.134019+0000","last_became_active":"2024-05-08T23:01:50.960061+0000","last_became_peered":"2024-05-08T23:01:50.960061+0000","last_unstale":"2024-05-08T23:01:57.134019+0000","last_undegraded":"2024-05-08T23:01:57.134019+0000","last_fullsized":"2024-05-08T23:01:57.134019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:54.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132548+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:01:57.132548+0000","last_peered":"2024-05-08T23:01:57.132548+0000","last_clean":"2024-05-08T23:01:57.132548+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:01:57.132548+0000","last_undegraded":"2024-05-08T23:01:57.132548+0000","last_fullsized":"2024-05-08T23:01:57.132548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.955938+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:01:55.955938+0000","last_peered":"2024-05-08T23:01:55.955938+0000","last_clean":"2024-05-08T23:01:55.955938+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:01:55.955938+0000","last_undegraded":"2024-05-08T23:01:55.955938+0000","last_fullsized":"2024-05-08T23:01:55.955938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954500+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:01:55.954500+0000","last_peered":"2024-05-08T23:01:55.954500+0000","last_clean":"2024-05-08T23:01:55.954500+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:01:55.954500+0000","last_undegraded":"2024-05-08T23:01:55.954500+0000","last_fullsized":"2024-05-08T23:01:55.954500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132006+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:01:57.132006+0000","last_peered":"2024-05-08T23:01:57.132006+0000","last_clean":"2024-05-08T23:01:57.132006+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:01:57.132006+0000","last_undegraded":"2024-05-08T23:01:57.132006+0000","last_fullsized":"2024-05-08T23:01:57.132006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.962832+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:01:55.962832+0000","last_peered":"2024-05-08T23:01:55.962832+0000","last_clean":"2024-05-08T23:01:55.962832+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:01:55.962832+0000","last_undegraded":"2024-05-08T23:01:55.962832+0000","last_fullsized":"2024-05-08T23:01:55.962832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132832+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:01:57.132832+0000","last_peered":"2024-05-08T23:01:57.132832+0000","last_clean":"2024-05-08T23:01:57.132832+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:01:57.132832+0000","last_undegraded":"2024-05-08T23:01:57.132832+0000","last_fullsized":"2024-05-08T23:01:57.132832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.954356+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:01:55.954356+0000","last_peered":"2024-05-08T23:01:55.954356+0000","last_clean":"2024-05-08T23:01:55.954356+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:01:55.954356+0000","last_undegraded":"2024-05-08T23:01:55.954356+0000","last_fullsized":"2024-05-08T23:01:55.954356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"peering","last_fresh":"2024-05-08T23:02:17.651608+0000","last_change":"2024-05-08T23:02:17.651768+0000","last_active":"2024-05-08T23:02:17.651608+0000","last_peered":"2024-05-08T23:01:56.479359+0000","last_clean":"2024-05-08T23:01:56.479359+0000","last_became_active":"2024-05-08T23:01:51.493814+0000","last_became_peered":"2024-05-08T23:01:51.493814+0000","last_unstale":"2024-05-08T23:02:17.651608+0000","last_undegraded":"2024-05-08T23:02:17.651608+0000","last_fullsized":"2024-05-08T23:02:17.651608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:28.292897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.478747+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:01:56.478747+0000","last_peered":"2024-05-08T23:01:56.478747+0000","last_clean":"2024-05-08T23:01:56.478747+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:01:56.478747+0000","last_undegraded":"2024-05-08T23:01:56.478747+0000","last_fullsized":"2024-05-08T23:01:56.478747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:56.622317+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:01:56.622317+0000","last_peered":"2024-05-08T23:01:56.622317+0000","last_clean":"2024-05-08T23:01:56.622317+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:01:56.622317+0000","last_undegraded":"2024-05-08T23:01:56.622317+0000","last_fullsized":"2024-05-08T23:01:56.622317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:01:55.963505+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:01:55.963505+0000","last_peered":"2024-05-08T23:01:55.963505+0000","last_clean":"2024-05-08T23:01:55.963505+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:01:55.963505+0000","last_undegraded":"2024-05-08T23:01:55.963505+0000","last_fullsized":"2024-05-08T23:01:55.963505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:01:57.132462+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:01:57.132462+0000","last_peered":"2024-05-08T23:01:57.132462+0000","last_clean":"2024-05-08T23:01:57.132462+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:01:57.132462+0000","last_undegraded":"2024-05-08T23:01:57.132462+0000","last_fullsized":"2024-05-08T23:01:57.132462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47946,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":193,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":86637,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612623,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":913408,"data_stored":452932,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.81899999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841554,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095383,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.85099999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":1.2829999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.3680000000000001}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":2.3719999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":2.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.79},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.4099999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.496}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.026}]}]},{"osd":9,"up_from":61,"seq":261993005078,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":443062,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.879},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":8,"up_from":54,"seq":231928234011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69899999999999995}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72899999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.86799999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.755},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":7,"up_from":47,"seq":201863462943,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":995328,"data_stored":1058837,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.499,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.63},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.55200000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.66400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.05},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.69899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.84099999999999997}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":892928,"data_stored":442368,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47599999999999998},"min":{"1min":0.441,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.60499999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78100000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.83499999999999996}]},{"osd":4,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]},{"osd":5,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.746},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.53700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.90400000000000003}]},{"osd":8,"last update":"Wed May 8 23:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.92600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":452949,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed May 8 23:01:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.626,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94999999999999996,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.51000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73399999999999999,"5min":1.03,"15min":1.03},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79400000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.73899999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72599999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":444296,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed May 8 23:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.46899999999999997,"5min":0.25,"15min":0.25},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.38200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed May 8 23:01:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.52800000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.86599999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.751},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.83899999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":446972,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 8 23:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed May 8 23:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.44,"15min":0.435},"min":{"1min":0.32700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.503}]},{"osd":6,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.70999999999999996,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3529999999999998,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.73999999999999999,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":7.3259999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed May 8 23:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed May 8 23:01:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":451909,"data_compressed":14638,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.44,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.443,"15min":0.41499999999999998},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.54700000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.627,"15min":0.59199999999999997},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":442565,"data_compressed":11681,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.442}]},{"osd":5,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.63100000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.84699999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":35,"seq":150323855399,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48999999999999999}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47699999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43943,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:02:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:22 smithi086 ceph-mon[78610]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:22 smithi086 ceph-mon[78610]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:22.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:22 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1105475548' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:02:23.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:22 smithi053 ceph-mon[77587]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:23.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:22 smithi053 ceph-mon[77587]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:23.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:22 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1105475548' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:02:23.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:22 smithi046 ceph-mon[36898]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:22 smithi046 ceph-mon[36898]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:23.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:22 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1105475548' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:02:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:23 smithi086 ceph-mon[78610]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:23.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:23 smithi086 ceph-mon[78610]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 12 KiB/s wr, 25 op/s; 212 B/s, 1 objects/s recovering 2024-05-08T23:02:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:23 smithi053 ceph-mon[77587]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:23 smithi053 ceph-mon[77587]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 12 KiB/s wr, 25 op/s; 212 B/s, 1 objects/s recovering 2024-05-08T23:02:24.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:23 smithi046 ceph-mon[36898]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:24.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:23 smithi046 ceph-mon[36898]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 12 KiB/s wr, 25 op/s; 212 B/s, 1 objects/s recovering 2024-05-08T23:02:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:24 smithi053 ceph-mon[77587]: pgmap v258: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 26 op/s; 215 B/s, 1 objects/s recovering 2024-05-08T23:02:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:24 smithi046 ceph-mon[36898]: pgmap v258: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 26 op/s; 215 B/s, 1 objects/s recovering 2024-05-08T23:02:25.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:24 smithi086 ceph-mon[78610]: pgmap v258: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 26 op/s; 215 B/s, 1 objects/s recovering 2024-05-08T23:02:25.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:02:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 -- 172.21.15.46:0/3039600159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 msgr2=0x7f75f4073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 --2- 172.21.15.46:0/3039600159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4073e80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009990 tx=0x7f75e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 -- 172.21.15.46:0/3039600159 shutdown_connections 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 --2- 172.21.15.46:0/3039600159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f410b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 --2- 172.21.15.46:0/3039600159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 0x7f75f4073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 --2- 172.21.15.46:0/3039600159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4073e80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 -- 172.21.15.46:0/3039600159 >> 172.21.15.46:0/3039600159 conn(0x7f75f40fc410 msgr2=0x7f75f40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 -- 172.21.15.46:0/3039600159 shutdown_connections 2024-05-08T23:02:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.939+0000 7f75fa000700 1 -- 172.21.15.46:0/3039600159 wait complete. 2024-05-08T23:02:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.940+0000 7f75fa000700 1 Processor -- start 2024-05-08T23:02:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 -- start start 2024-05-08T23:02:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 0x7f75f41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a0b40 con 0x7f75f4074fe0 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a0c80 con 0x7f75f4108d40 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75fa000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a0e10 con 0x7f75f4073a00 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75f2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 0x7f75f41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75f3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49006/0 (socket says 172.21.15.46:49006) 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.941+0000 7f75f3e00700 1 -- 172.21.15.46:0/1947326830 learned_addr learned my addr 172.21.15.46:0/1947326830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 -- 172.21.15.46:0/1947326830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 msgr2=0x7f75f4199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 -- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 msgr2=0x7f75f41995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 0x7f75f41995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 -- 172.21.15.46:0/1947326830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4009520 con 0x7f75f4108d40 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3400700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75f3e00700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00ef30 tx=0x7f75ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec018070 con 0x7f75f4108d40 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e00095f0 con 0x7f75f4108d40 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75ec00f040 con 0x7f75f4108d40 2024-05-08T23:02:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00cd00 con 0x7f75f4108d40 2024-05-08T23:02:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.942+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41a1410 con 0x7f75f4108d40 2024-05-08T23:02:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.943+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4107b00 con 0x7f75f4108d40 2024-05-08T23:02:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.945+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f75ec003a00 con 0x7f75f4108d40 2024-05-08T23:02:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.947+0000 7f75ebe00700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 0x7f75d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.947+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75ec0cd2b0 con 0x7f75f4108d40 2024-05-08T23:02:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.947+0000 7f75f3400700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 0x7f75d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.947+0000 7f75f3400700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 0x7f75d80a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009fd0 tx=0x7f75e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:25.950+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f75ec096660 con 0x7f75f4108d40 2024-05-08T23:02:26.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.165+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75f402ce10 con 0x7f75d80a11b0 2024-05-08T23:02:26.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.178+0000 7f75ebe00700 1 -- 172.21.15.46:0/1947326830 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422901 (secure 0 0 0) 0x7f75f402ce10 con 0x7f75d80a11b0 2024-05-08T23:02:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.183+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 msgr2=0x7f75d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.183+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 0x7f75d80a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009fd0 tx=0x7f75e4009260 comp rx=0 tx=0).stop 2024-05-08T23:02:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.183+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 msgr2=0x7f75f41a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.183+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00ef30 tx=0x7f75ec00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 shutdown_connections 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75f4108d40 0x7f75f41a0570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f4073a00 0x7f75f4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f75d80a11b0 0x7f75d80a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 --2- 172.21.15.46:0/1947326830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f4074fe0 0x7f75f41995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 >> 172.21.15.46:0/1947326830 conn(0x7f75f40fc410 msgr2=0x7f75f4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 shutdown_connections 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:26.184+0000 7f75fa000700 1 -- 172.21.15.46:0/1947326830 wait complete. 2024-05-08T23:02:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:02:26.203 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-05-08T23:02:24.925433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643702,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":206,"num_read_kb":216,"num_write":407,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372460,"kb_used_data":11360,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569364,"statfs":{"total":1151940427776,"available":1151559028736,"internally_reserved":0,"allocated":11632640,"data_stored":7493850,"data_compressed":324360,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46988,"num_objects":62,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":70,"num_read_kb":97,"num_write":146,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001901"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661165+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:02:17.661165+0000","last_peered":"2024-05-08T23:02:17.661165+0000","last_clean":"2024-05-08T23:02:17.661165+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:02:17.661165+0000","last_undegraded":"2024-05-08T23:02:17.661165+0000","last_fullsized":"2024-05-08T23:02:17.661165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661559+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:02:17.661559+0000","last_peered":"2024-05-08T23:02:17.661559+0000","last_clean":"2024-05-08T23:02:17.661559+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:02:17.661559+0000","last_undegraded":"2024-05-08T23:02:17.661559+0000","last_fullsized":"2024-05-08T23:02:17.661559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661196+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:02:17.661196+0000","last_peered":"2024-05-08T23:02:17.661196+0000","last_clean":"2024-05-08T23:02:17.661196+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:02:17.661196+0000","last_undegraded":"2024-05-08T23:02:17.661196+0000","last_fullsized":"2024-05-08T23:02:17.661196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654527+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:02:17.654527+0000","last_peered":"2024-05-08T23:02:17.654527+0000","last_clean":"2024-05-08T23:02:17.654527+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:02:17.654527+0000","last_undegraded":"2024-05-08T23:02:17.654527+0000","last_fullsized":"2024-05-08T23:02:17.654527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650061+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:02:17.650061+0000","last_peered":"2024-05-08T23:02:17.650061+0000","last_clean":"2024-05-08T23:02:17.650061+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:02:17.650061+0000","last_undegraded":"2024-05-08T23:02:17.650061+0000","last_fullsized":"2024-05-08T23:02:17.650061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660268+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:02:17.660268+0000","last_peered":"2024-05-08T23:02:17.660268+0000","last_clean":"2024-05-08T23:02:17.660268+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:02:17.660268+0000","last_undegraded":"2024-05-08T23:02:17.660268+0000","last_fullsized":"2024-05-08T23:02:17.660268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654543+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:02:17.654543+0000","last_peered":"2024-05-08T23:02:17.654543+0000","last_clean":"2024-05-08T23:02:17.654543+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:02:17.654543+0000","last_undegraded":"2024-05-08T23:02:17.654543+0000","last_fullsized":"2024-05-08T23:02:17.654543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656492+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:02:17.656492+0000","last_peered":"2024-05-08T23:02:17.656492+0000","last_clean":"2024-05-08T23:02:17.656492+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:02:17.656492+0000","last_undegraded":"2024-05-08T23:02:17.656492+0000","last_fullsized":"2024-05-08T23:02:17.656492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661236+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:02:17.661236+0000","last_peered":"2024-05-08T23:02:17.661236+0000","last_clean":"2024-05-08T23:02:17.661236+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:02:17.661236+0000","last_undegraded":"2024-05-08T23:02:17.661236+0000","last_fullsized":"2024-05-08T23:02:17.661236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661078+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:02:17.661078+0000","last_peered":"2024-05-08T23:02:17.661078+0000","last_clean":"2024-05-08T23:02:17.661078+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:02:17.661078+0000","last_undegraded":"2024-05-08T23:02:17.661078+0000","last_fullsized":"2024-05-08T23:02:17.661078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659040+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:02:17.659040+0000","last_peered":"2024-05-08T23:02:17.659040+0000","last_clean":"2024-05-08T23:02:17.659040+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:02:17.659040+0000","last_undegraded":"2024-05-08T23:02:17.659040+0000","last_fullsized":"2024-05-08T23:02:17.659040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661153+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:02:17.661153+0000","last_peered":"2024-05-08T23:02:17.661153+0000","last_clean":"2024-05-08T23:02:17.661153+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:02:17.661153+0000","last_undegraded":"2024-05-08T23:02:17.661153+0000","last_fullsized":"2024-05-08T23:02:17.661153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656514+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:02:17.656514+0000","last_peered":"2024-05-08T23:02:17.656514+0000","last_clean":"2024-05-08T23:02:17.656514+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:02:17.656514+0000","last_undegraded":"2024-05-08T23:02:17.656514+0000","last_fullsized":"2024-05-08T23:02:17.656514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655453+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:02:17.655453+0000","last_peered":"2024-05-08T23:02:17.655453+0000","last_clean":"2024-05-08T23:02:17.655453+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:02:17.655453+0000","last_undegraded":"2024-05-08T23:02:17.655453+0000","last_fullsized":"2024-05-08T23:02:17.655453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654795+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:02:17.654795+0000","last_peered":"2024-05-08T23:02:17.654795+0000","last_clean":"2024-05-08T23:02:17.654795+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:02:17.654795+0000","last_undegraded":"2024-05-08T23:02:17.654795+0000","last_fullsized":"2024-05-08T23:02:17.654795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.641780+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:02:19.641780+0000","last_peered":"2024-05-08T23:02:19.641780+0000","last_clean":"2024-05-08T23:02:19.641780+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:02:19.641780+0000","last_undegraded":"2024-05-08T23:02:19.641780+0000","last_fullsized":"2024-05-08T23:02:19.641780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651671+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:02:17.651671+0000","last_peered":"2024-05-08T23:02:17.651671+0000","last_clean":"2024-05-08T23:02:17.651671+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:02:17.651671+0000","last_undegraded":"2024-05-08T23:02:17.651671+0000","last_fullsized":"2024-05-08T23:02:17.651671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653802+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:02:17.653802+0000","last_peered":"2024-05-08T23:02:17.653802+0000","last_clean":"2024-05-08T23:02:17.653802+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:02:17.653802+0000","last_undegraded":"2024-05-08T23:02:17.653802+0000","last_fullsized":"2024-05-08T23:02:17.653802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651158+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:17.651158+0000","last_peered":"2024-05-08T23:02:17.651158+0000","last_clean":"2024-05-08T23:02:17.651158+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:17.651158+0000","last_undegraded":"2024-05-08T23:02:17.651158+0000","last_fullsized":"2024-05-08T23:02:17.651158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661525+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:02:17.661525+0000","last_peered":"2024-05-08T23:02:17.661525+0000","last_clean":"2024-05-08T23:02:17.661525+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:02:17.661525+0000","last_undegraded":"2024-05-08T23:02:17.661525+0000","last_fullsized":"2024-05-08T23:02:17.661525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653999+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:02:17.653999+0000","last_peered":"2024-05-08T23:02:17.653999+0000","last_clean":"2024-05-08T23:02:17.653999+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:02:17.653999+0000","last_undegraded":"2024-05-08T23:02:17.653999+0000","last_fullsized":"2024-05-08T23:02:17.653999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636260+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:19.636260+0000","last_peered":"2024-05-08T23:02:19.636260+0000","last_clean":"2024-05-08T23:02:19.636260+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:19.636260+0000","last_undegraded":"2024-05-08T23:02:19.636260+0000","last_fullsized":"2024-05-08T23:02:19.636260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659474+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:02:17.659474+0000","last_peered":"2024-05-08T23:02:17.659474+0000","last_clean":"2024-05-08T23:02:17.659474+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:02:17.659474+0000","last_undegraded":"2024-05-08T23:02:17.659474+0000","last_fullsized":"2024-05-08T23:02:17.659474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653902+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:02:17.653902+0000","last_peered":"2024-05-08T23:02:17.653902+0000","last_clean":"2024-05-08T23:02:17.653902+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:02:17.653902+0000","last_undegraded":"2024-05-08T23:02:17.653902+0000","last_fullsized":"2024-05-08T23:02:17.653902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655207+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:18.655207+0000","last_peered":"2024-05-08T23:02:18.655207+0000","last_clean":"2024-05-08T23:02:18.655207+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:18.655207+0000","last_undegraded":"2024-05-08T23:02:18.655207+0000","last_fullsized":"2024-05-08T23:02:18.655207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655226+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:02:17.655226+0000","last_peered":"2024-05-08T23:02:17.655226+0000","last_clean":"2024-05-08T23:02:17.655226+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:02:17.655226+0000","last_undegraded":"2024-05-08T23:02:17.655226+0000","last_fullsized":"2024-05-08T23:02:17.655226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.640210+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:19.640210+0000","last_peered":"2024-05-08T23:02:19.640210+0000","last_clean":"2024-05-08T23:02:19.640210+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:19.640210+0000","last_undegraded":"2024-05-08T23:02:19.640210+0000","last_fullsized":"2024-05-08T23:02:19.640210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657534+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:02:17.657534+0000","last_peered":"2024-05-08T23:02:17.657534+0000","last_clean":"2024-05-08T23:02:17.657534+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:02:17.657534+0000","last_undegraded":"2024-05-08T23:02:17.657534+0000","last_fullsized":"2024-05-08T23:02:17.657534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661059+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:02:17.661059+0000","last_peered":"2024-05-08T23:02:17.661059+0000","last_clean":"2024-05-08T23:02:17.661059+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:02:17.661059+0000","last_undegraded":"2024-05-08T23:02:17.661059+0000","last_fullsized":"2024-05-08T23:02:17.661059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674762+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:17.674762+0000","last_peered":"2024-05-08T23:02:17.674762+0000","last_clean":"2024-05-08T23:02:17.674762+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:17.674762+0000","last_undegraded":"2024-05-08T23:02:17.674762+0000","last_fullsized":"2024-05-08T23:02:17.674762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649951+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:02:17.649951+0000","last_peered":"2024-05-08T23:02:17.649951+0000","last_clean":"2024-05-08T23:02:17.649951+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:02:17.649951+0000","last_undegraded":"2024-05-08T23:02:17.649951+0000","last_fullsized":"2024-05-08T23:02:17.649951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661283+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:02:17.661283+0000","last_peered":"2024-05-08T23:02:17.661283+0000","last_clean":"2024-05-08T23:02:17.661283+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:02:17.661283+0000","last_undegraded":"2024-05-08T23:02:17.661283+0000","last_fullsized":"2024-05-08T23:02:17.661283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655141+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:02:17.655141+0000","last_peered":"2024-05-08T23:02:17.655141+0000","last_clean":"2024-05-08T23:02:17.655141+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:02:17.655141+0000","last_undegraded":"2024-05-08T23:02:17.655141+0000","last_fullsized":"2024-05-08T23:02:17.655141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651327+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:02:17.651327+0000","last_peered":"2024-05-08T23:02:17.651327+0000","last_clean":"2024-05-08T23:02:17.651327+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:02:17.651327+0000","last_undegraded":"2024-05-08T23:02:17.651327+0000","last_fullsized":"2024-05-08T23:02:17.651327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:23.647283+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:23.647283+0000","last_peered":"2024-05-08T23:02:23.647283+0000","last_clean":"2024-05-08T23:02:23.647283+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:23.647283+0000","last_undegraded":"2024-05-08T23:02:23.647283+0000","last_fullsized":"2024-05-08T23:02:23.647283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660300+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:02:17.660300+0000","last_peered":"2024-05-08T23:02:17.660300+0000","last_clean":"2024-05-08T23:02:17.660300+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:02:17.660300+0000","last_undegraded":"2024-05-08T23:02:17.660300+0000","last_fullsized":"2024-05-08T23:02:17.660300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654915+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:02:17.654915+0000","last_peered":"2024-05-08T23:02:17.654915+0000","last_clean":"2024-05-08T23:02:17.654915+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:02:17.654915+0000","last_undegraded":"2024-05-08T23:02:17.654915+0000","last_fullsized":"2024-05-08T23:02:17.654915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661428+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:02:17.661428+0000","last_peered":"2024-05-08T23:02:17.661428+0000","last_clean":"2024-05-08T23:02:17.661428+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:02:17.661428+0000","last_undegraded":"2024-05-08T23:02:17.661428+0000","last_fullsized":"2024-05-08T23:02:17.661428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654337+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:02:17.654337+0000","last_peered":"2024-05-08T23:02:17.654337+0000","last_clean":"2024-05-08T23:02:17.654337+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:02:17.654337+0000","last_undegraded":"2024-05-08T23:02:17.654337+0000","last_fullsized":"2024-05-08T23:02:17.654337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659646+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:02:17.659646+0000","last_peered":"2024-05-08T23:02:17.659646+0000","last_clean":"2024-05-08T23:02:17.659646+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:02:17.659646+0000","last_undegraded":"2024-05-08T23:02:17.659646+0000","last_fullsized":"2024-05-08T23:02:17.659646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654901+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:02:17.654901+0000","last_peered":"2024-05-08T23:02:17.654901+0000","last_clean":"2024-05-08T23:02:17.654901+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:02:17.654901+0000","last_undegraded":"2024-05-08T23:02:17.654901+0000","last_fullsized":"2024-05-08T23:02:17.654901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'26","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:23.649654+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:23.649654+0000","last_peered":"2024-05-08T23:02:23.649654+0000","last_clean":"2024-05-08T23:02:23.649654+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:23.649654+0000","last_undegraded":"2024-05-08T23:02:23.649654+0000","last_fullsized":"2024-05-08T23:02:23.649654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46477,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":52,"num_write":28,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636478+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:19.636478+0000","last_peered":"2024-05-08T23:02:19.636478+0000","last_clean":"2024-05-08T23:02:19.636478+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:19.636478+0000","last_undegraded":"2024-05-08T23:02:19.636478+0000","last_fullsized":"2024-05-08T23:02:19.636478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655806+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:02:17.655806+0000","last_peered":"2024-05-08T23:02:17.655806+0000","last_clean":"2024-05-08T23:02:17.655806+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:02:17.655806+0000","last_undegraded":"2024-05-08T23:02:17.655806+0000","last_fullsized":"2024-05-08T23:02:17.655806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659759+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:02:17.659759+0000","last_peered":"2024-05-08T23:02:17.659759+0000","last_clean":"2024-05-08T23:02:17.659759+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:02:17.659759+0000","last_undegraded":"2024-05-08T23:02:17.659759+0000","last_fullsized":"2024-05-08T23:02:17.659759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656023+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:02:17.656023+0000","last_peered":"2024-05-08T23:02:17.656023+0000","last_clean":"2024-05-08T23:02:17.656023+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:02:17.656023+0000","last_undegraded":"2024-05-08T23:02:17.656023+0000","last_fullsized":"2024-05-08T23:02:17.656023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:20.642535+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:20.642535+0000","last_peered":"2024-05-08T23:02:20.642535+0000","last_clean":"2024-05-08T23:02:20.642535+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:20.642535+0000","last_undegraded":"2024-05-08T23:02:20.642535+0000","last_fullsized":"2024-05-08T23:02:20.642535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657551+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:02:17.657551+0000","last_peered":"2024-05-08T23:02:17.657551+0000","last_clean":"2024-05-08T23:02:17.657551+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:02:17.657551+0000","last_undegraded":"2024-05-08T23:02:17.657551+0000","last_fullsized":"2024-05-08T23:02:17.657551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656051+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:18.656051+0000","last_peered":"2024-05-08T23:02:18.656051+0000","last_clean":"2024-05-08T23:02:18.656051+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:18.656051+0000","last_undegraded":"2024-05-08T23:02:18.656051+0000","last_fullsized":"2024-05-08T23:02:18.656051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.670834+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:02:18.670834+0000","last_peered":"2024-05-08T23:02:18.670834+0000","last_clean":"2024-05-08T23:02:18.670834+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:02:18.670834+0000","last_undegraded":"2024-05-08T23:02:18.670834+0000","last_fullsized":"2024-05-08T23:02:18.670834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.663365+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:02:18.663365+0000","last_peered":"2024-05-08T23:02:18.663365+0000","last_clean":"2024-05-08T23:02:18.663365+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:02:18.663365+0000","last_undegraded":"2024-05-08T23:02:18.663365+0000","last_fullsized":"2024-05-08T23:02:18.663365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655364+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:02:17.655364+0000","last_peered":"2024-05-08T23:02:17.655364+0000","last_clean":"2024-05-08T23:02:17.655364+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:02:17.655364+0000","last_undegraded":"2024-05-08T23:02:17.655364+0000","last_fullsized":"2024-05-08T23:02:17.655364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656734+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:02:17.656734+0000","last_peered":"2024-05-08T23:02:17.656734+0000","last_clean":"2024-05-08T23:02:17.656734+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:02:17.656734+0000","last_undegraded":"2024-05-08T23:02:17.656734+0000","last_fullsized":"2024-05-08T23:02:17.656734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:22.646651+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:22.646651+0000","last_peered":"2024-05-08T23:02:22.646651+0000","last_clean":"2024-05-08T23:02:22.646651+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:22.646651+0000","last_undegraded":"2024-05-08T23:02:22.646651+0000","last_fullsized":"2024-05-08T23:02:22.646651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657532+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:02:17.657532+0000","last_peered":"2024-05-08T23:02:17.657532+0000","last_clean":"2024-05-08T23:02:17.657532+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:02:17.657532+0000","last_undegraded":"2024-05-08T23:02:17.657532+0000","last_fullsized":"2024-05-08T23:02:17.657532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:20.641872+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:20.641872+0000","last_peered":"2024-05-08T23:02:20.641872+0000","last_clean":"2024-05-08T23:02:20.641872+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:20.641872+0000","last_undegraded":"2024-05-08T23:02:20.641872+0000","last_fullsized":"2024-05-08T23:02:20.641872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650153+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:02:17.650153+0000","last_peered":"2024-05-08T23:02:17.650153+0000","last_clean":"2024-05-08T23:02:17.650153+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:02:17.650153+0000","last_undegraded":"2024-05-08T23:02:17.650153+0000","last_fullsized":"2024-05-08T23:02:17.650153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.639109+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:19.639109+0000","last_peered":"2024-05-08T23:02:19.639109+0000","last_clean":"2024-05-08T23:02:19.639109+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:19.639109+0000","last_undegraded":"2024-05-08T23:02:19.639109+0000","last_fullsized":"2024-05-08T23:02:19.639109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654454+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:02:17.654454+0000","last_peered":"2024-05-08T23:02:17.654454+0000","last_clean":"2024-05-08T23:02:17.654454+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:02:17.654454+0000","last_undegraded":"2024-05-08T23:02:17.654454+0000","last_fullsized":"2024-05-08T23:02:17.654454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667675+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:02:18.667675+0000","last_peered":"2024-05-08T23:02:18.667675+0000","last_clean":"2024-05-08T23:02:18.667675+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:02:18.667675+0000","last_undegraded":"2024-05-08T23:02:18.667675+0000","last_fullsized":"2024-05-08T23:02:18.667675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659637+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:02:17.659637+0000","last_peered":"2024-05-08T23:02:17.659637+0000","last_clean":"2024-05-08T23:02:17.659637+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:02:17.659637+0000","last_undegraded":"2024-05-08T23:02:17.659637+0000","last_fullsized":"2024-05-08T23:02:17.659637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655573+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:18.655573+0000","last_peered":"2024-05-08T23:02:18.655573+0000","last_clean":"2024-05-08T23:02:18.655573+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:18.655573+0000","last_undegraded":"2024-05-08T23:02:18.655573+0000","last_fullsized":"2024-05-08T23:02:18.655573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656812+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:02:17.656812+0000","last_peered":"2024-05-08T23:02:17.656812+0000","last_clean":"2024-05-08T23:02:17.656812+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:02:17.656812+0000","last_undegraded":"2024-05-08T23:02:17.656812+0000","last_fullsized":"2024-05-08T23:02:17.656812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649950+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:02:17.649950+0000","last_peered":"2024-05-08T23:02:17.649950+0000","last_clean":"2024-05-08T23:02:17.649950+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:02:17.649950+0000","last_undegraded":"2024-05-08T23:02:17.649950+0000","last_fullsized":"2024-05-08T23:02:17.649950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.657188+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:18.657188+0000","last_peered":"2024-05-08T23:02:18.657188+0000","last_clean":"2024-05-08T23:02:18.657188+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:18.657188+0000","last_undegraded":"2024-05-08T23:02:18.657188+0000","last_fullsized":"2024-05-08T23:02:18.657188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650266+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:02:17.650266+0000","last_peered":"2024-05-08T23:02:17.650266+0000","last_clean":"2024-05-08T23:02:17.650266+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:02:17.650266+0000","last_undegraded":"2024-05-08T23:02:17.650266+0000","last_fullsized":"2024-05-08T23:02:17.650266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654622+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:02:17.654622+0000","last_peered":"2024-05-08T23:02:17.654622+0000","last_clean":"2024-05-08T23:02:17.654622+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:02:17.654622+0000","last_undegraded":"2024-05-08T23:02:17.654622+0000","last_fullsized":"2024-05-08T23:02:17.654622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656885+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:18.656885+0000","last_peered":"2024-05-08T23:02:18.656885+0000","last_clean":"2024-05-08T23:02:18.656885+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:18.656885+0000","last_undegraded":"2024-05-08T23:02:18.656885+0000","last_fullsized":"2024-05-08T23:02:18.656885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661556+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:02:17.661556+0000","last_peered":"2024-05-08T23:02:17.661556+0000","last_clean":"2024-05-08T23:02:17.661556+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:02:17.661556+0000","last_undegraded":"2024-05-08T23:02:17.661556+0000","last_fullsized":"2024-05-08T23:02:17.661556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654068+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:02:17.654068+0000","last_peered":"2024-05-08T23:02:17.654068+0000","last_clean":"2024-05-08T23:02:17.654068+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:02:17.654068+0000","last_undegraded":"2024-05-08T23:02:17.654068+0000","last_fullsized":"2024-05-08T23:02:17.654068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659168+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:17.659168+0000","last_peered":"2024-05-08T23:02:17.659168+0000","last_clean":"2024-05-08T23:02:17.659168+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:17.659168+0000","last_undegraded":"2024-05-08T23:02:17.659168+0000","last_fullsized":"2024-05-08T23:02:17.659168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657027+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:02:17.657027+0000","last_peered":"2024-05-08T23:02:17.657027+0000","last_clean":"2024-05-08T23:02:17.657027+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:02:17.657027+0000","last_undegraded":"2024-05-08T23:02:17.657027+0000","last_fullsized":"2024-05-08T23:02:17.657027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673909+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.673909+0000","last_peered":"2024-05-08T23:02:17.673909+0000","last_clean":"2024-05-08T23:02:17.673909+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.673909+0000","last_undegraded":"2024-05-08T23:02:17.673909+0000","last_fullsized":"2024-05-08T23:02:17.673909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659636+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:02:17.659636+0000","last_peered":"2024-05-08T23:02:17.659636+0000","last_clean":"2024-05-08T23:02:17.659636+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:02:17.659636+0000","last_undegraded":"2024-05-08T23:02:17.659636+0000","last_fullsized":"2024-05-08T23:02:17.659636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.657221+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:02:18.657221+0000","last_peered":"2024-05-08T23:02:18.657221+0000","last_clean":"2024-05-08T23:02:18.657221+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:02:18.657221+0000","last_undegraded":"2024-05-08T23:02:18.657221+0000","last_fullsized":"2024-05-08T23:02:18.657221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.985257+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:02:18.985257+0000","last_peered":"2024-05-08T23:02:18.985257+0000","last_clean":"2024-05-08T23:02:18.985257+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:02:18.985257+0000","last_undegraded":"2024-05-08T23:02:18.985257+0000","last_fullsized":"2024-05-08T23:02:18.985257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650347+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:02:17.650347+0000","last_peered":"2024-05-08T23:02:17.650347+0000","last_clean":"2024-05-08T23:02:17.650347+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:02:17.650347+0000","last_undegraded":"2024-05-08T23:02:17.650347+0000","last_fullsized":"2024-05-08T23:02:17.650347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654280+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:02:17.654280+0000","last_peered":"2024-05-08T23:02:17.654280+0000","last_clean":"2024-05-08T23:02:17.654280+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:02:17.654280+0000","last_undegraded":"2024-05-08T23:02:17.654280+0000","last_fullsized":"2024-05-08T23:02:17.654280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.654955+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:18.654955+0000","last_peered":"2024-05-08T23:02:18.654955+0000","last_clean":"2024-05-08T23:02:18.654955+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:18.654955+0000","last_undegraded":"2024-05-08T23:02:18.654955+0000","last_fullsized":"2024-05-08T23:02:18.654955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659929+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:02:17.659929+0000","last_peered":"2024-05-08T23:02:17.659929+0000","last_clean":"2024-05-08T23:02:17.659929+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:02:17.659929+0000","last_undegraded":"2024-05-08T23:02:17.659929+0000","last_fullsized":"2024-05-08T23:02:17.659929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:21.644599+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:02:21.644599+0000","last_peered":"2024-05-08T23:02:21.644599+0000","last_clean":"2024-05-08T23:02:21.644599+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:02:21.644599+0000","last_undegraded":"2024-05-08T23:02:21.644599+0000","last_fullsized":"2024-05-08T23:02:21.644599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661348+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:02:17.661348+0000","last_peered":"2024-05-08T23:02:17.661348+0000","last_clean":"2024-05-08T23:02:17.661348+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:02:17.661348+0000","last_undegraded":"2024-05-08T23:02:17.661348+0000","last_fullsized":"2024-05-08T23:02:17.661348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667814+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:02:18.667814+0000","last_peered":"2024-05-08T23:02:18.667814+0000","last_clean":"2024-05-08T23:02:18.667814+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:02:18.667814+0000","last_undegraded":"2024-05-08T23:02:18.667814+0000","last_fullsized":"2024-05-08T23:02:18.667814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659729+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:02:17.659729+0000","last_peered":"2024-05-08T23:02:17.659729+0000","last_clean":"2024-05-08T23:02:17.659729+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:02:17.659729+0000","last_undegraded":"2024-05-08T23:02:17.659729+0000","last_fullsized":"2024-05-08T23:02:17.659729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654952+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:02:17.654952+0000","last_peered":"2024-05-08T23:02:17.654952+0000","last_clean":"2024-05-08T23:02:17.654952+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:02:17.654952+0000","last_undegraded":"2024-05-08T23:02:17.654952+0000","last_fullsized":"2024-05-08T23:02:17.654952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656443+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:02:17.656443+0000","last_peered":"2024-05-08T23:02:17.656443+0000","last_clean":"2024-05-08T23:02:17.656443+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:02:17.656443+0000","last_undegraded":"2024-05-08T23:02:17.656443+0000","last_fullsized":"2024-05-08T23:02:17.656443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661526+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:02:17.661526+0000","last_peered":"2024-05-08T23:02:17.661526+0000","last_clean":"2024-05-08T23:02:17.661526+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:02:17.661526+0000","last_undegraded":"2024-05-08T23:02:17.661526+0000","last_fullsized":"2024-05-08T23:02:17.661526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656909+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:02:17.656909+0000","last_peered":"2024-05-08T23:02:17.656909+0000","last_clean":"2024-05-08T23:02:17.656909+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:02:17.656909+0000","last_undegraded":"2024-05-08T23:02:17.656909+0000","last_fullsized":"2024-05-08T23:02:17.656909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660321+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:02:17.660321+0000","last_peered":"2024-05-08T23:02:17.660321+0000","last_clean":"2024-05-08T23:02:17.660321+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:02:17.660321+0000","last_undegraded":"2024-05-08T23:02:17.660321+0000","last_fullsized":"2024-05-08T23:02:17.660321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655679+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:02:17.655679+0000","last_peered":"2024-05-08T23:02:17.655679+0000","last_clean":"2024-05-08T23:02:17.655679+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:02:17.655679+0000","last_undegraded":"2024-05-08T23:02:17.655679+0000","last_fullsized":"2024-05-08T23:02:17.655679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661450+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:02:18.661450+0000","last_peered":"2024-05-08T23:02:18.661450+0000","last_clean":"2024-05-08T23:02:18.661450+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:02:18.661450+0000","last_undegraded":"2024-05-08T23:02:18.661450+0000","last_fullsized":"2024-05-08T23:02:18.661450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651835+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:02:17.651835+0000","last_peered":"2024-05-08T23:02:17.651835+0000","last_clean":"2024-05-08T23:02:17.651835+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:02:17.651835+0000","last_undegraded":"2024-05-08T23:02:17.651835+0000","last_fullsized":"2024-05-08T23:02:17.651835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659375+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:02:17.659375+0000","last_peered":"2024-05-08T23:02:17.659375+0000","last_clean":"2024-05-08T23:02:17.659375+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:02:17.659375+0000","last_undegraded":"2024-05-08T23:02:17.659375+0000","last_fullsized":"2024-05-08T23:02:17.659375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657091+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:02:17.657091+0000","last_peered":"2024-05-08T23:02:17.657091+0000","last_clean":"2024-05-08T23:02:17.657091+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:02:17.657091+0000","last_undegraded":"2024-05-08T23:02:17.657091+0000","last_fullsized":"2024-05-08T23:02:17.657091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659850+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:02:17.659850+0000","last_peered":"2024-05-08T23:02:17.659850+0000","last_clean":"2024-05-08T23:02:17.659850+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:02:17.659850+0000","last_undegraded":"2024-05-08T23:02:17.659850+0000","last_fullsized":"2024-05-08T23:02:17.659850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53100,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":70,"num_read_kb":97,"num_write":224,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":159300,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612624,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":466334,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.60499999999999998}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.753},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79300000000000004}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.879},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.83899999999999997}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.0169999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.97099999999999997}]}]},{"osd":10,"up_from":66,"seq":283467841555,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1099998,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.496}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67000000000000004}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.626}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.75600000000000001}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.73799999999999999}]}]},{"osd":9,"up_from":61,"seq":261993005079,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":470386,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.752}]},{"osd":5,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.747}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.73199999999999998}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":8,"up_from":54,"seq":231928234012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.749},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.89300000000000002}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94199999999999995}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.877},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":7,"up_from":47,"seq":201863462944,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1003520,"data_stored":1063023,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.45100000000000001,"15min":0.438},"min":{"1min":0.38300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.504}]},{"osd":2,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.499,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.751,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.68799999999999994}]},{"osd":3,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.48299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63500000000000001,"15min":0.626},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.55200000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.66400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.05},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.69899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72499999999999998}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.755}]},{"osd":9,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.76800000000000002}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":6,"up_from":42,"seq":180388626468,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":929792,"data_stored":461746,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47599999999999998},"min":{"1min":0.441,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.60499999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed May 8 23:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.53700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.76800000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.753},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80900000000000005,"15min":0.78000000000000003},"min":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.83199999999999996}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.874},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":983040,"data_stored":509616,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67100000000000004}]},{"osd":2,"last update":"Wed May 8 23:01:26 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.751},{"interface":"front","average":{"1min":0.626,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94999999999999996,"5min":1.012,"15min":1.012},"last":0.95199999999999996}]},{"osd":3,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.79000000000000004}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":1.034}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.03,"15min":1.03},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed May 8 23:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79400000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89100000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.052,"5min":1.071,"15min":1.071},"last":1.1200000000000001}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.93600000000000005}]},{"osd":10,"last update":"Wed May 8 23:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":463202,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed May 8 23:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.46899999999999997,"5min":0.25,"15min":0.25},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Wed May 8 23:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.38200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed May 8 23:01:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.52800000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.52300000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed May 8 23:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.627},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.1080000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0389999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.1519999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.77400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":983040,"data_stored":507325,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.87,"15min":0.87},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.876,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.873,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80600000000000005}]},{"osd":3,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.64000000000000001,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 23:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.35099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45500000000000002,"15min":0.44},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 8 23:01:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.70999999999999996,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":7.3529999999999998,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.73999999999999999,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":7.3259999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.90200000000000002}]},{"osd":7,"last update":"Wed May 8 23:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.92400000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60799999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.747},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.94499999999999995}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":3,"up_from":24,"seq":103079215151,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":461193,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.44,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.443,"15min":0.41499999999999998},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.54700000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.627,"15min":0.59199999999999997},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.69099999999999995}]},{"osd":2,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61599999999999999,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.86799999999999999}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.71399999999999997}]},{"osd":5,"last update":"Wed May 8 23:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71799999999999997,"15min":0.68300000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.169,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 23:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.748}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.80200000000000005}]},{"osd":10,"last update":"Wed May 8 23:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":942080,"data_stored":467026,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67900000000000005}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.622}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.755}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.82599999999999996}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.91500000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.042,"15min":1.042},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.79400000000000004},"min":{"1min":0.59299999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96399999999999997,"5min":1.05,"15min":1.05},"last":0.86499999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.93200000000000005}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":5,"up_from":35,"seq":150323855399,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48999999999999999}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47699999999999998}]},{"osd":9,"last update":"Wed May 8 23:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":48853,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":47231,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":48558,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:02:26.210 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:02:26.211 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T23:02:26.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T23:02:26.615 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:26 smithi046 ceph-mon[36898]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:26 smithi046 ceph-mon[36898]: pgmap v259: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 23 op/s; 186 B/s, 1 objects/s recovering 2024-05-08T23:02:27.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:27 smithi053 ceph-mon[77587]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:27.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:27 smithi053 ceph-mon[77587]: pgmap v259: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 23 op/s; 186 B/s, 1 objects/s recovering 2024-05-08T23:02:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:26 smithi086 ceph-mon[78610]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:02:27.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:27 smithi086 ceph-mon[78610]: pgmap v259: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 23 op/s; 186 B/s, 1 objects/s recovering 2024-05-08T23:02:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 -- 172.21.15.46:0/2186595975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c106af0 msgr2=0x7f836c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 --2- 172.21.15.46:0/2186595975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c106af0 0x7f836c108f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f836400b0a0 tx=0x7f836400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 -- 172.21.15.46:0/2186595975 shutdown_connections 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 --2- 172.21.15.46:0/2186595975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c106af0 0x7f836c108f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 --2- 172.21.15.46:0/2186595975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c101890 0x7f836c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 --2- 172.21.15.46:0/2186595975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 -- 172.21.15.46:0/2186595975 >> 172.21.15.46:0/2186595975 conn(0x7f836c0fb2a0 msgr2=0x7f836c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 -- 172.21.15.46:0/2186595975 shutdown_connections 2024-05-08T23:02:27.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.817+0000 7f8372a00700 1 -- 172.21.15.46:0/2186595975 wait complete. 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.818+0000 7f8372a00700 1 Processor -- start 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.818+0000 7f8372a00700 1 -- start start 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19e960 con 0x7f836c1041c0 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19eaa0 con 0x7f836c101890 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19ec30 con 0x7f836c106af0 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8370c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49024/0 (socket says 172.21.15.46:49024) 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 -- 172.21.15.46:0/4055693114 learned_addr learned my addr 172.21.15.46:0/4055693114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372000700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41532/0 (socket says 172.21.15.46:41532) 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 -- 172.21.15.46:0/4055693114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 msgr2=0x7f836c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 -- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 msgr2=0x7f836c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8371600700 1 -- 172.21.15.46:0/4055693114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009d00 con 0x7f836c101890 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8372000700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.819+0000 7f8370c00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:27.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f8371600700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f836000e9a0 tx=0x7f836000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360009790 con 0x7f836c101890 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8360010430 con 0x7f836c101890 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c0ff6e0 con 0x7f836c101890 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360018680 con 0x7f836c101890 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.820+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c0ffc30 con 0x7f836c101890 2024-05-08T23:02:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.821+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836c0fce20 con 0x7f836c101890 2024-05-08T23:02:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.823+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f83600105a0 con 0x7f836c101890 2024-05-08T23:02:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.824+0000 7f836a000700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 0x7f83540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.824+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8360014070 con 0x7f836c101890 2024-05-08T23:02:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.825+0000 7f8370c00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 0x7f83540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.825+0000 7f8370c00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 0x7f83540a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8358009730 tx=0x7f8358012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:27.827+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8360096630 con 0x7f836c101890 2024-05-08T23:02:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.133+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f836c10c690 con 0x7f836c101890 2024-05-08T23:02:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.139+0000 7f836a000700 1 -- 172.21.15.46:0/4055693114 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8360099c50 con 0x7f836c101890 2024-05-08T23:02:28.139 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-05-08T23:02:28.140 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCEBDxm9kMbCBAAoPI5fvbMjK8uIhBHarP8Sw== 2024-05-08T23:02:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 msgr2=0x7f83540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 0x7f83540a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8358009730 tx=0x7f8358012040 comp rx=0 tx=0).stop 2024-05-08T23:02:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 msgr2=0x7f836c196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f836000e9a0 tx=0x7f836000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 shutdown_connections 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c106af0 0x7f836c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c101890 0x7f836c196e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f83540a1230 0x7f83540a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 --2- 172.21.15.46:0/4055693114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c1041c0 0x7f836c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 >> 172.21.15.46:0/4055693114 conn(0x7f836c0fb2a0 msgr2=0x7f836c1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 shutdown_connections 2024-05-08T23:02:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:28.142+0000 7f8372a00700 1 -- 172.21.15.46:0/4055693114 wait complete. 2024-05-08T23:02:28.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:28 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4055693114' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:28 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:28 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:02:28.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:28 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4055693114' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:28 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:28 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:02:28.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:28 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4055693114' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:28 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:02:28.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:28 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:02:28.529 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:02:28.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T23:02:28.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:02:28.626 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:02:28.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:02:28.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T23:02:29.036 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:29.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:29 smithi086 ceph-mon[78610]: pgmap v260: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 167 B/s, 1 objects/s recovering 2024-05-08T23:02:29.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:29 smithi053 ceph-mon[77587]: pgmap v260: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 167 B/s, 1 objects/s recovering 2024-05-08T23:02:29.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:29 smithi046 ceph-mon[36898]: pgmap v260: 129 pgs: 129 active+clean; 629 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 KiB/s wr, 20 op/s; 167 B/s, 1 objects/s recovering 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.240+0000 7f8eda200700 1 -- 172.21.15.46:0/906740063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 msgr2=0x7f8ed4107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.240+0000 7f8eda200700 1 --2- 172.21.15.46:0/906740063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed4107400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099c0 tx=0x7f8ec4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 -- 172.21.15.46:0/906740063 shutdown_connections 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 --2- 172.21.15.46:0/906740063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 0x7f8ed410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 --2- 172.21.15.46:0/906740063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed4107400 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 --2- 172.21.15.46:0/906740063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed4068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 -- 172.21.15.46:0/906740063 >> 172.21.15.46:0/906740063 conn(0x7f8ed4075f80 msgr2=0x7f8ed4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 -- 172.21.15.46:0/906740063 shutdown_connections 2024-05-08T23:02:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.241+0000 7f8eda200700 1 -- 172.21.15.46:0/906740063 wait complete. 2024-05-08T23:02:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 Processor -- start 2024-05-08T23:02:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 -- start start 2024-05-08T23:02:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 0x7f8ed4199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419e4e0 con 0x7f8ed4107020 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419e650 con 0x7f8ed4068950 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.242+0000 7f8eda200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419e810 con 0x7f8ed40692f0 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49030/0 (socket says 172.21.15.46:49030) 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 -- 172.21.15.46:0/2541426124 learned_addr learned my addr 172.21.15.46:0/2541426124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed2a00700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 0x7f8ed4199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 -- 172.21.15.46:0/2541426124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 msgr2=0x7f8ed4199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 0x7f8ed4199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 -- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 msgr2=0x7f8ed419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 -- 172.21.15.46:0/2541426124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40095f0 con 0x7f8ed4068950 2024-05-08T23:02:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3e00700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.243+0000 7f8ed3400700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b9c0 tx=0x7f8ec4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.244+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec401d070 con 0x7f8ed4068950 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.244+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec400bdb0 con 0x7f8ed4068950 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.244+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed419ea90 con 0x7f8ed4068950 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.244+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400f990 con 0x7f8ed4068950 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.244+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed41a6350 con 0x7f8ed4068950 2024-05-08T23:02:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.245+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed41026d0 con 0x7f8ed4068950 2024-05-08T23:02:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.247+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8ec400fb30 con 0x7f8ed4068950 2024-05-08T23:02:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.248+0000 7f8ecbe00700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 0x7f8eb80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.248+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8ec40cdc80 con 0x7f8ed4068950 2024-05-08T23:02:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.248+0000 7f8ed2a00700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 0x7f8eb80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.249+0000 7f8ed2a00700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 0x7f8eb80a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ed419ab40 tx=0x7f8ec0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.251+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8ec4096fd0 con 0x7f8ed4068950 2024-05-08T23:02:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.525+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8ed419a8f0 con 0x7f8ed4068950 2024-05-08T23:02:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.531+0000 7f8ecbe00700 1 -- 172.21.15.46:0/2541426124 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260285 (secure 0 0 0) 0x7f8ec409a5f0 con 0x7f8ed4068950 2024-05-08T23:02:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.534+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 msgr2=0x7f8eb80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 0x7f8eb80a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ed419ab40 tx=0x7f8ec0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 msgr2=0x7f8ed41995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b9c0 tx=0x7f8ec4009580 comp rx=0 tx=0).stop 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 shutdown_connections 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8eb80a11b0 0x7f8eb80a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed4107020 0x7f8ed419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed4068950 0x7f8ed41995e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 --2- 172.21.15.46:0/2541426124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed40692f0 0x7f8ed4199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 >> 172.21.15.46:0/2541426124 conn(0x7f8ed4075f80 msgr2=0x7f8ed410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 shutdown_connections 2024-05-08T23:02:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:30.535+0000 7f8eda200700 1 -- 172.21.15.46:0/2541426124 wait complete. 2024-05-08T23:02:30.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:30 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2541426124' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:02:30.915 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":2731582335},{"type":"v1","addr":"172.21.15.46:6801","nonce":2731582335}]},"active_addr":"172.21.15.46:6801/2731582335","active_change":"2024-05-08T22:56:16.844340+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1289022895}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":580235593}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2265080110}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1222248037}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3800995796}]}]} 2024-05-08T23:02:30.920 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T23:02:30.920 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T23:02:30.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:02:30.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:30 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2541426124' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:02:31.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:30 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2541426124' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:02:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:31.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:31 smithi046 ceph-mon[36898]: pgmap v261: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 20 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:31.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:31 smithi086 ceph-mon[78610]: pgmap v261: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 20 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:31 smithi053 ceph-mon[77587]: pgmap v261: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 20 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:32.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 -- 172.21.15.46:0/1607014817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c0692f0 msgr2=0x7ff17c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 --2- 172.21.15.46:0/1607014817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c0692f0 0x7ff17c10de60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff17400b0a0 tx=0x7ff17400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 -- 172.21.15.46:0/1607014817 shutdown_connections 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 --2- 172.21.15.46:0/1607014817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c0692f0 0x7ff17c10de60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 --2- 172.21.15.46:0/1607014817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c107020 0x7ff17c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 --2- 172.21.15.46:0/1607014817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 0x7ff17c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 -- 172.21.15.46:0/1607014817 >> 172.21.15.46:0/1607014817 conn(0x7ff17c075f80 msgr2=0x7ff17c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 -- 172.21.15.46:0/1607014817 shutdown_connections 2024-05-08T23:02:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.465+0000 7ff182000700 1 -- 172.21.15.46:0/1607014817 wait complete. 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.466+0000 7ff182000700 1 Processor -- start 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 -- start start 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 0x7ff17c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c0692f0 0x7ff17c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19c410 con 0x7ff17c107020 2024-05-08T23:02:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19c580 con 0x7ff17c068950 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff182000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19c740 con 0x7ff17c0692f0 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff17be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff17be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60238/0 (socket says 172.21.15.46:60238) 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff17be00700 1 -- 172.21.15.46:0/606106019 learned_addr learned my addr 172.21.15.46:0/606106019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff17b400700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 0x7ff17c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.467+0000 7ff173e00700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c0692f0 0x7ff17c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 -- 172.21.15.46:0/606106019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c0692f0 msgr2=0x7ff17c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c0692f0 0x7ff17c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 -- 172.21.15.46:0/606106019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 msgr2=0x7ff17c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 0x7ff17c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 -- 172.21.15.46:0/606106019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174009d00 con 0x7ff17c107020 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff17be00700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff17400b070 tx=0x7ff1740078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400e040 con 0x7ff17c107020 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff17401d470 con 0x7ff17c107020 2024-05-08T23:02:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174005280 con 0x7ff17c107020 2024-05-08T23:02:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c19c9c0 con 0x7ff17c107020 2024-05-08T23:02:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.468+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c1a42a0 con 0x7ff17c107020 2024-05-08T23:02:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.469+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c109540 con 0x7ff17c107020 2024-05-08T23:02:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.471+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff17401d5e0 con 0x7ff17c107020 2024-05-08T23:02:32.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.473+0000 7ff178c00700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 0x7ff1600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:32.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.473+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff1740cdd90 con 0x7ff17c107020 2024-05-08T23:02:32.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.473+0000 7ff17b400700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 0x7ff1600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:32.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.474+0000 7ff17b400700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 0x7ff1600a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0688c0 tx=0x7ff16c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.477+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff1740970e0 con 0x7ff17c107020 2024-05-08T23:02:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.708+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff17c1026d0 con 0x7ff17c107020 2024-05-08T23:02:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.710+0000 7ff178c00700 1 -- 172.21.15.46:0/606106019 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7ff17409a700 con 0x7ff17c107020 2024-05-08T23:02:32.710 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:32.711 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 msgr2=0x7ff1600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 0x7ff1600a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0688c0 tx=0x7ff16c009260 comp rx=0 tx=0).stop 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 msgr2=0x7ff17c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff17400b070 tx=0x7ff1740078c0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 shutdown_connections 2024-05-08T23:02:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1600a11b0 0x7ff1600a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff17c107020 0x7ff17c19be00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff17c068950 0x7ff17c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 --2- 172.21.15.46:0/606106019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c0692f0 0x7ff17c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 >> 172.21.15.46:0/606106019 conn(0x7ff17c075f80 msgr2=0x7ff17c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 shutdown_connections 2024-05-08T23:02:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:32.712+0000 7ff182000700 1 -- 172.21.15.46:0/606106019 wait complete. 2024-05-08T23:02:33.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/606106019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:33.105 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T23:02:33.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:02:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/606106019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/606106019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:33.962 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:33 smithi046 ceph-mon[36898]: pgmap v262: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:33 smithi086 ceph-mon[78610]: pgmap v262: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:33 smithi053 ceph-mon[77587]: pgmap v262: 129 pgs: 129 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 143 B/s, 0 objects/s recovering 2024-05-08T23:02:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 -- 172.21.15.46:0/1825795766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98101f70 msgr2=0x7f4a98102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1825795766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98101f70 0x7f4a98102350 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a880099c0 tx=0x7f4a88009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 -- 172.21.15.46:0/1825795766 shutdown_connections 2024-05-08T23:02:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1825795766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98107500 0x7f4a9810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1825795766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98101f70 0x7f4a98102350 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1825795766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98102920 0x7f4a98106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 -- 172.21.15.46:0/1825795766 >> 172.21.15.46:0/1825795766 conn(0x7f4a980fb7e0 msgr2=0x7f4a980fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.672+0000 7f4a9d600700 1 -- 172.21.15.46:0/1825795766 shutdown_connections 2024-05-08T23:02:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.673+0000 7f4a9d600700 1 -- 172.21.15.46:0/1825795766 wait complete. 2024-05-08T23:02:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 Processor -- start 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 -- start start 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 0x7f4a98199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98107500 0x7f4a9819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819e4e0 con 0x7f4a98107500 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819e650 con 0x7f4a98101f70 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a9d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819e810 con 0x7f4a98102920 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a97e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 0x7f4a98199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.674+0000 7f4a97400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 0x7f4a98199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54768/0 (socket says 172.21.15.46:54768) 2024-05-08T23:02:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52702/0 (socket says 172.21.15.46:52702) 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97e00700 1 -- 172.21.15.46:0/1067569015 learned_addr learned my addr 172.21.15.46:0/1067569015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a9cc00700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98107500 0x7f4a9819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 -- 172.21.15.46:0/1067569015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 msgr2=0x7f4a98199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 0x7f4a98199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 -- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98107500 msgr2=0x7f4a9819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98107500 0x7f4a9819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 -- 172.21.15.46:0/1067569015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880095f0 con 0x7f4a98102920 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a97400700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400e9c0 tx=0x7f4a8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8400c9c0 con 0x7f4a98102920 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9819eaf0 con 0x7f4a98102920 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a84010430 con 0x7f4a98102920 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.675+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84018680 con 0x7f4a98102920 2024-05-08T23:02:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.676+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a981a6350 con 0x7f4a98102920 2024-05-08T23:02:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.677+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a98069230 con 0x7f4a98102920 2024-05-08T23:02:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.679+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4a8400cb20 con 0x7f4a98102920 2024-05-08T23:02:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.681+0000 7f4a94c00700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 0x7f4a7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.681+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a84014070 con 0x7f4a98102920 2024-05-08T23:02:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.681+0000 7f4a97e00700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 0x7f4a7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:34.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.681+0000 7f4a97e00700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 0x7f4a7c0a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a88000c00 tx=0x7f4a88005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.685+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4a84052d60 con 0x7f4a98102920 2024-05-08T23:02:34.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.902+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a9819a8f0 con 0x7f4a98102920 2024-05-08T23:02:34.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.904+0000 7f4a94c00700 1 -- 172.21.15.46:0/1067569015 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7f4a840961d0 con 0x7f4a98102920 2024-05-08T23:02:34.904 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:02:34.904 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:02:34.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 msgr2=0x7f4a7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 0x7f4a7c0a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a88000c00 tx=0x7f4a88005c40 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 msgr2=0x7f4a98199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400e9c0 tx=0x7f4a8400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 shutdown_connections 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a7c0a11b0 0x7f4a7c0a3670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98107500 0x7f4a9819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101f70 0x7f4a98199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 --2- 172.21.15.46:0/1067569015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98102920 0x7f4a98199b70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 >> 172.21.15.46:0/1067569015 conn(0x7f4a980fb7e0 msgr2=0x7f4a980fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 shutdown_connections 2024-05-08T23:02:34.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:34.906+0000 7f4a9d600700 1 -- 172.21.15.46:0/1067569015 wait complete. 2024-05-08T23:02:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:34 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1067569015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:34 smithi086 ceph-mon[78610]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-05-08T23:02:35.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:34 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1067569015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:35.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:34 smithi046 ceph-mon[36898]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-05-08T23:02:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:34 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1067569015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:02:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:34 smithi053 ceph-mon[77587]: pgmap v263: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 KiB/s wr, 12 op/s; 1 B/s, 0 objects/s recovering 2024-05-08T23:02:35.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T23:02:35.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T23:02:35.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T23:02:35.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T23:02:35.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T23:02:35.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T23:02:35.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T23:02:35.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T23:02:35.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T23:02:35.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T23:02:35.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T23:02:35.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T23:02:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:37.212 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:37 smithi046 ceph-mon[36898]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-08T23:02:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:37 smithi053 ceph-mon[77587]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-08T23:02:37.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:37 smithi086 ceph-mon[78610]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-08T23:02:37.908 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.465 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.507 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 -- 172.21.15.46:0/3757854175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 msgr2=0x7ffaa8114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 --2- 172.21.15.46:0/3757854175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa8114040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000b0a0 tx=0x7ffaa000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 -- 172.21.15.46:0/3757854175 shutdown_connections 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 --2- 172.21.15.46:0/3757854175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa8114040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 --2- 172.21.15.46:0/3757854175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8075000 0x7ffaa80731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 --2- 172.21.15.46:0/3757854175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa8073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 -- 172.21.15.46:0/3757854175 >> 172.21.15.46:0/3757854175 conn(0x7ffaa8078190 msgr2=0x7ffaa80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.598+0000 7ffaaf000700 1 -- 172.21.15.46:0/3757854175 shutdown_connections 2024-05-08T23:02:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.599+0000 7ffaaf000700 1 -- 172.21.15.46:0/3757854175 wait complete. 2024-05-08T23:02:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 Processor -- start 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 -- start start 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa81a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8075000 0x7ffaa81a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819e090 con 0x7ffaa8111c30 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819e200 con 0x7ffaa8075000 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.600+0000 7ffaaf000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819e3c0 con 0x7ffaa8073710 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa81a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa81a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52716/0 (socket says 172.21.15.46:52716) 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 -- 172.21.15.46:0/1748555468 learned_addr learned my addr 172.21.15.46:0/1748555468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaad000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7400700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8075000 0x7ffaa81a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 -- 172.21.15.46:0/1748555468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8075000 msgr2=0x7ffaa81a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaa8075000 0x7ffaa81a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 -- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 msgr2=0x7ffaa819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009d00 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaad000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa8111c30 0x7ffaa819da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.601+0000 7ffaa7e00700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa81a38f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800e9a0 tx=0x7ffa9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.602+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98009790 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.602+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa98010430 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.602+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98018680 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.602+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa819e6a0 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.602+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa81aa8e0 con 0x7ffaa8073710 2024-05-08T23:02:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.603+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaa8117790 con 0x7ffaa8073710 2024-05-08T23:02:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.605+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ffa980105a0 con 0x7ffaa8073710 2024-05-08T23:02:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.606+0000 7ffaa4c00700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa8c0a11b0 0x7ffa8c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa98014070 con 0x7ffaa8073710 2024-05-08T23:02:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaa7400700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa8c0a11b0 0x7ffa8c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaa4c00700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] conn(0x7ffa8c0ac1d0 0x7ffa8c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa8c0aed40 con 0x7ffa8c0ac1d0 2024-05-08T23:02:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa980cd540 con 0x7ffaa8073710 2024-05-08T23:02:38.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.607+0000 7ffaad000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] conn(0x7ffa8c0ac1d0 0x7ffa8c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.608+0000 7ffaa7400700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa8c0a11b0 0x7ffa8c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffaa819ebf0 tx=0x7ffa94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.608+0000 7ffaad000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] conn(0x7ffa8c0ac1d0 0x7ffa8c0ae670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.609+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== osd.6 v2:172.21.15.46:6818/1222901662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ffa8c0aed40 con 0x7ffa8c0ac1d0 2024-05-08T23:02:38.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.627+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 --> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffaa8110e00 con 0x7ffa8c0ac1d0 2024-05-08T23:02:38.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.628+0000 7ffaa4c00700 1 -- 172.21.15.46:0/1748555468 <== osd.6 v2:172.21.15.46:6818/1222901662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffaa8110e00 con 0x7ffa8c0ac1d0 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] conn(0x7ffa8c0ac1d0 msgr2=0x7ffa8c0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] conn(0x7ffa8c0ac1d0 0x7ffa8c0ae670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa8c0a11b0 msgr2=0x7ffa8c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ffa8c0a11b0 0x7ffa8c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffaa819ebf0 tx=0x7ffa94009260 comp rx=0 tx=0).stop 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 msgr2=0x7ffaa81a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 --2- 172.21.15.46:0/1748555468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa8073710 0x7ffaa81a38f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800e9a0 tx=0x7ffa9800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaad000700 1 -- 172.21.15.46:0/1748555468 reap_dead start 2024-05-08T23:02:38.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 shutdown_connections 2024-05-08T23:02:38.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 >> 172.21.15.46:0/1748555468 conn(0x7ffaa8078190 msgr2=0x7ffaa81104b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:38.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 shutdown_connections 2024-05-08T23:02:38.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.629+0000 7ffaaf000700 1 -- 172.21.15.46:0/1748555468 wait complete. 2024-05-08T23:02:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 -- 172.21.15.46:0/2075020327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 msgr2=0x7f444c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 --2- 172.21.15.46:0/2075020327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c068db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44380098f0 tx=0x7f4438009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 -- 172.21.15.46:0/2075020327 shutdown_connections 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 --2- 172.21.15.46:0/2075020327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c0692f0 0x7f444c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 --2- 172.21.15.46:0/2075020327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c107020 0x7f444c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 --2- 172.21.15.46:0/2075020327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c068db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 -- 172.21.15.46:0/2075020327 >> 172.21.15.46:0/2075020327 conn(0x7f444c075f80 msgr2=0x7f444c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 -- 172.21.15.46:0/2075020327 shutdown_connections 2024-05-08T23:02:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.723+0000 7f4453000700 1 -- 172.21.15.46:0/2075020327 wait complete. 2024-05-08T23:02:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.724+0000 7f4453000700 1 Processor -- start 2024-05-08T23:02:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 -- start start 2024-05-08T23:02:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c107e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0692f0 0x7f444c1083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c107020 0x7f444c1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1a9cb0 con 0x7f444c107020 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1a9df0 con 0x7f444c068950 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.725+0000 7f4453000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1a9f30 con 0x7f444c0692f0 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c107e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0692f0 0x7f444c1083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c107e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54808/0 (socket says 172.21.15.46:54808) 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444be00700 1 -- 172.21.15.46:0/2503114878 learned_addr learned my addr 172.21.15.46:0/2503114878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f4451000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c107020 0x7f444c1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 -- 172.21.15.46:0/2503114878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 msgr2=0x7f444c107e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f444c068950 0x7f444c107e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 -- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c107020 msgr2=0x7f444c1088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c107020 0x7f444c1088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438009520 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.726+0000 7f444b400700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0692f0 0x7f444c1083b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4438009230 tx=0x7f443800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.727+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443801c070 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.727+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443800ba20 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.727+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c1aa1b0 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.727+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44380038e0 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.727+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c1aa620 con 0x7f444c0692f0 2024-05-08T23:02:38.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.728+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f444c1111a0 con 0x7f444c0692f0 2024-05-08T23:02:38.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.731+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4438020030 con 0x7f444c0692f0 2024-05-08T23:02:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.733+0000 7f4448c00700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44300a1230 0x7f44300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.733+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f44380cdd20 con 0x7f444c0692f0 2024-05-08T23:02:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.733+0000 7f444be00700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44300a1230 0x7f44300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f4448c00700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] conn(0x7f44300ac250 0x7f44300ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44300aedc0 con 0x7f44300ac250 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44380ce090 con 0x7f444c0692f0 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f4451000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] conn(0x7f44300ac250 0x7f44300ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f444be00700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44300a1230 0x7f44300a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f443c005950 tx=0x7f443c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.734+0000 7f4451000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] conn(0x7f44300ac250 0x7f44300ae6f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.735+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== osd.5 v2:172.21.15.86:6808/2217036731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f44300aedc0 con 0x7f44300ac250 2024-05-08T23:02:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.755+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 --> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f444c0ff480 con 0x7f44300ac250 2024-05-08T23:02:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.756+0000 7f4448c00700 1 -- 172.21.15.46:0/2503114878 <== osd.5 v2:172.21.15.86:6808/2217036731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f444c0ff480 con 0x7f44300ac250 2024-05-08T23:02:38.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] conn(0x7f44300ac250 msgr2=0x7f44300ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] conn(0x7f44300ac250 0x7f44300ae6f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44300a1230 msgr2=0x7f44300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f44300a1230 0x7f44300a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f443c005950 tx=0x7f443c009310 comp rx=0 tx=0).stop 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0692f0 msgr2=0x7f444c1083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 --2- 172.21.15.46:0/2503114878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0692f0 0x7f444c1083b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4438009230 tx=0x7f443800fa00 comp rx=0 tx=0).stop 2024-05-08T23:02:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4451000700 1 -- 172.21.15.46:0/2503114878 reap_dead start 2024-05-08T23:02:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 shutdown_connections 2024-05-08T23:02:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.757+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 >> 172.21.15.46:0/2503114878 conn(0x7f444c075f80 msgr2=0x7f444c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.758+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 shutdown_connections 2024-05-08T23:02:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:38.758+0000 7f4453000700 1 -- 172.21.15.46:0/2503114878 wait complete. 2024-05-08T23:02:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:39.063 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:39 smithi053 ceph-mon[77587]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:02:39.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:39 smithi046 ceph-mon[36898]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:02:39.418 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:39.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:39 smithi086 ceph-mon[78610]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T23:02:39.692 INFO:teuthology.orchestra.run.smithi046.stdout:150323855403 2024-05-08T23:02:39.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T23:02:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:41 smithi053 ceph-mon[77587]: pgmap v266: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T23:02:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:41 smithi046 ceph-mon[36898]: pgmap v266: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T23:02:41.395 INFO:teuthology.orchestra.run.smithi046.stdout:180388626472 2024-05-08T23:02:41.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T23:02:41.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:41 smithi086 ceph-mon[78610]: pgmap v266: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T23:02:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:43.108 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:43 smithi046 ceph-mon[36898]: pgmap v267: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T23:02:43.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:43 smithi053 ceph-mon[77587]: pgmap v267: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T23:02:43.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:43 smithi086 ceph-mon[78610]: pgmap v267: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 -- 172.21.15.46:0/1713244643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 msgr2=0x7fdb44069a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 --2- 172.21.15.46:0/1713244643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb44069a40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb300099c0 tx=0x7fdb30009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 -- 172.21.15.46:0/1713244643 shutdown_connections 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 --2- 172.21.15.46:0/1713244643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44069f80 0x7fdb4410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 --2- 172.21.15.46:0/1713244643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44107100 0x7fdb441074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 --2- 172.21.15.46:0/1713244643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb44069a40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 -- 172.21.15.46:0/1713244643 >> 172.21.15.46:0/1713244643 conn(0x7fdb44075f80 msgr2=0x7fdb44076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 -- 172.21.15.46:0/1713244643 shutdown_connections 2024-05-08T23:02:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.068+0000 7fdb4c600700 1 -- 172.21.15.46:0/1713244643 wait complete. 2024-05-08T23:02:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.070+0000 7fdb4c600700 1 Processor -- start 2024-05-08T23:02:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.070+0000 7fdb4c600700 1 -- start start 2024-05-08T23:02:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb441a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44069f80 0x7fdb441a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44107100 0x7fdb4410bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4410c410 con 0x7fdb44069f80 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4410c580 con 0x7fdb44107100 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.071+0000 7fdb4c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4410c740 con 0x7fdb440695c0 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44069f80 0x7fdb441a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb4a600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44107100 0x7fdb4410bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44069f80 0x7fdb441a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54340/0 (socket says 172.21.15.46:54340) 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb4a600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44107100 0x7fdb4410bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59974/0 (socket says 172.21.15.46:59974) 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49200700 1 -- 172.21.15.46:0/4116460846 learned_addr learned my addr 172.21.15.46:0/4116460846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb441a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 -- 172.21.15.46:0/4116460846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44107100 msgr2=0x7fdb4410bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb44107100 0x7fdb4410bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 -- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44069f80 msgr2=0x7fdb441a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb44069f80 0x7fdb441a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.072+0000 7fdb49c00700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300095f0 con 0x7fdb440695c0 2024-05-08T23:02:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.073+0000 7fdb49c00700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb441a3730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9f0 tx=0x7fdb3800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.073+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800c9c0 con 0x7fdb440695c0 2024-05-08T23:02:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.073+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb38010430 con 0x7fdb440695c0 2024-05-08T23:02:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.073+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4410ca20 con 0x7fdb440695c0 2024-05-08T23:02:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.073+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38018680 con 0x7fdb440695c0 2024-05-08T23:02:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.074+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb441aa720 con 0x7fdb440695c0 2024-05-08T23:02:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.074+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb441111c0 con 0x7fdb440695c0 2024-05-08T23:02:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.078+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdb38010860 con 0x7fdb440695c0 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb42000700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdb2c0a11b0 0x7fdb2c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdb38014070 con 0x7fdb440695c0 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb49200700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdb2c0a11b0 0x7fdb2c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb42000700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] conn(0x7fdb2c0ac1d0 0x7fdb2c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb2c0aed40 con 0x7fdb2c0ac1d0 2024-05-08T23:02:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.080+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb380cd810 con 0x7fdb440695c0 2024-05-08T23:02:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.081+0000 7fdb4a600700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] conn(0x7fdb2c0ac1d0 0x7fdb2c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.081+0000 7fdb49200700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdb2c0a11b0 0x7fdb2c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdb4419efb0 tx=0x7fdb30019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.081+0000 7fdb4a600700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] conn(0x7fdb2c0ac1d0 0x7fdb2c0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.082+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== osd.9 v2:172.21.15.46:6826/2347234971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fdb2c0aed40 con 0x7fdb2c0ac1d0 2024-05-08T23:02:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.103+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 --> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb44068e90 con 0x7fdb2c0ac1d0 2024-05-08T23:02:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.104+0000 7fdb42000700 1 -- 172.21.15.46:0/4116460846 <== osd.9 v2:172.21.15.46:6826/2347234971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdb44068e90 con 0x7fdb2c0ac1d0 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] conn(0x7fdb2c0ac1d0 msgr2=0x7fdb2c0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] conn(0x7fdb2c0ac1d0 0x7fdb2c0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdb2c0a11b0 msgr2=0x7fdb2c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdb2c0a11b0 0x7fdb2c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdb4419efb0 tx=0x7fdb30019040 comp rx=0 tx=0).stop 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 msgr2=0x7fdb441a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 --2- 172.21.15.46:0/4116460846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb440695c0 0x7fdb441a3730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9f0 tx=0x7fdb3800ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4a600700 1 -- 172.21.15.46:0/4116460846 reap_dead start 2024-05-08T23:02:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 shutdown_connections 2024-05-08T23:02:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 >> 172.21.15.46:0/4116460846 conn(0x7fdb44075f80 msgr2=0x7fdb44100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 shutdown_connections 2024-05-08T23:02:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.105+0000 7fdb4c600700 1 -- 172.21.15.46:0/4116460846 wait complete. 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 -- 172.21.15.46:0/1904333679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa140ffb00 msgr2=0x7faa140ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 --2- 172.21.15.46:0/1904333679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa140ffb00 0x7faa140ffee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa040098f0 tx=0x7faa04009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 -- 172.21.15.46:0/1904333679 shutdown_connections 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 --2- 172.21.15.46:0/1904333679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 0x7faa14105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 --2- 172.21.15.46:0/1904333679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa140ffb00 0x7faa140ffee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 --2- 172.21.15.46:0/1904333679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14100420 0x7faa14104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.030+0000 7faa19600700 1 -- 172.21.15.46:0/1904333679 >> 172.21.15.46:0/1904333679 conn(0x7faa140fb7e0 msgr2=0x7faa140fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.031+0000 7faa19600700 1 -- 172.21.15.46:0/1904333679 shutdown_connections 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.031+0000 7faa19600700 1 -- 172.21.15.46:0/1904333679 wait complete. 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.032+0000 7faa19600700 1 Processor -- start 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.032+0000 7faa19600700 1 -- start start 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 0x7faa141a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14100420 0x7faa141a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 0x7faa1419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419df30 con 0x7faa14105100 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419e0a0 con 0x7faa14100420 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa19600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419e260 con 0x7faa140ffb00 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa18c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 0x7faa1419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 0x7faa141a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 0x7faa141a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37740/0 (socket says 172.21.15.46:37740) 2024-05-08T23:02:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa13e00700 1 -- 172.21.15.46:0/1714573991 learned_addr learned my addr 172.21.15.46:0/1714573991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.033+0000 7faa13400700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14100420 0x7faa141a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 -- 172.21.15.46:0/1714573991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14100420 msgr2=0x7faa141a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14100420 0x7faa141a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 -- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 msgr2=0x7faa1419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 0x7faa1419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000095f0 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa18c00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa14105100 0x7faa1419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa13e00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 0x7faa141a36b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa04005950 tx=0x7faa0400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0401c070 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0400baa0 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa04009520 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.034+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040038e0 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.035+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141aa510 con 0x7faa140ffb00 2024-05-08T23:02:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.035+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa141176a0 con 0x7faa140ffb00 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.038+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faa04020030 con 0x7faa140ffb00 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa10c00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9f80a1230 0x7fa9f80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faa040cdff0 con 0x7faa140ffb00 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa13400700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9f80a1230 0x7fa9f80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa10c00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] conn(0x7fa9f80ac250 0x7fa9f80ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9f80aedc0 con 0x7fa9f80ac250 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.039+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa040ce400 con 0x7faa140ffb00 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.040+0000 7faa18c00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] conn(0x7fa9f80ac250 0x7fa9f80ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.040+0000 7faa13400700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9f80a1230 0x7fa9f80a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faa140fefb0 tx=0x7faa00009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.040+0000 7faa18c00700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] conn(0x7fa9f80ac250 0x7fa9f80ae6f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.041+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== osd.8 v2:172.21.15.86:6816/4052399727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa9f80aedc0 con 0x7fa9f80ac250 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.062+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 --> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa141044f0 con 0x7fa9f80ac250 2024-05-08T23:02:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa10c00700 1 -- 172.21.15.46:0/1714573991 <== osd.8 v2:172.21.15.86:6816/4052399727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faa141044f0 con 0x7fa9f80ac250 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] conn(0x7fa9f80ac250 msgr2=0x7fa9f80ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] conn(0x7fa9f80ac250 0x7fa9f80ae6f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9f80a1230 msgr2=0x7fa9f80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9f80a1230 0x7fa9f80a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faa140fefb0 tx=0x7faa00009310 comp rx=0 tx=0).stop 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 msgr2=0x7faa141a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.063+0000 7faa19600700 1 --2- 172.21.15.46:0/1714573991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa140ffb00 0x7faa141a36b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa04005950 tx=0x7faa0400fa00 comp rx=0 tx=0).stop 2024-05-08T23:02:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.064+0000 7faa18c00700 1 -- 172.21.15.46:0/1714573991 reap_dead start 2024-05-08T23:02:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.064+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 shutdown_connections 2024-05-08T23:02:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.064+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 >> 172.21.15.46:0/1714573991 conn(0x7faa140fb7e0 msgr2=0x7faa14102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.064+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 shutdown_connections 2024-05-08T23:02:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.064+0000 7faa19600700 1 -- 172.21.15.46:0/1714573991 wait complete. 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 -- 172.21.15.46:0/1841988844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0105100 msgr2=0x7f0ce0105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 --2- 172.21.15.46:0/1841988844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0105100 0x7f0ce0105580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009f90 tx=0x7f0cd800b330 comp rx=0 tx=0).stop 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 -- 172.21.15.46:0/1841988844 shutdown_connections 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 --2- 172.21.15.46:0/1841988844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0105100 0x7f0ce0105580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 --2- 172.21.15.46:0/1841988844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce00ffb00 0x7f0ce00ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 --2- 172.21.15.46:0/1841988844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100420 0x7f0ce0104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.834+0000 7f0ce7000700 1 -- 172.21.15.46:0/1841988844 >> 172.21.15.46:0/1841988844 conn(0x7f0ce00fb7e0 msgr2=0x7f0ce00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.835+0000 7f0ce7000700 1 -- 172.21.15.46:0/1841988844 shutdown_connections 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.835+0000 7f0ce7000700 1 -- 172.21.15.46:0/1841988844 wait complete. 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.836+0000 7f0ce7000700 1 Processor -- start 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.836+0000 7f0ce7000700 1 -- start start 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 0x7f0ce01a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100420 0x7f0ce01a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0105100 0x7f0ce019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019df10 con 0x7f0ce0100420 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019e080 con 0x7f0ce0105100 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.837+0000 7f0ce7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019e240 con 0x7f0ce00ffb00 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 0x7f0ce01a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 0x7f0ce01a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37730/0 (socket says 172.21.15.46:37730) 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce5c00700 1 -- 172.21.15.46:0/3042221681 learned_addr learned my addr 172.21.15.46:0/3042221681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0105100 0x7f0ce019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100420 0x7f0ce01a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 -- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 msgr2=0x7f0ce01a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 0x7f0ce01a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 -- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100420 msgr2=0x7f0ce01a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100420 0x7f0ce01a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce6600700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8009c40 con 0x7f0ce0105100 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.838+0000 7f0ce5c00700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce00ffb00 0x7f0ce01a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0ce6600700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0105100 0x7f0ce019d8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800b6c0 tx=0x7f0cd80094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800e040 con 0x7f0ce0105100 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd8007e10 con 0x7f0ce0105100 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce019e4c0 con 0x7f0ce0105100 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8005040 con 0x7f0ce0105100 2024-05-08T23:02:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.839+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce01aa700 con 0x7f0ce0105100 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.840+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ce01176e0 con 0x7f0ce0105100 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.844+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0cd801b450 con 0x7f0ce0105100 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.845+0000 7f0cde000700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc80a1230 0x7f0cc80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.845+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0cd80ce2f0 con 0x7f0ce0105100 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0ce5c00700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc80a1230 0x7f0cc80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0cde000700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] conn(0x7f0cc80ac120 0x7f0cc80ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc80aec90 con 0x7f0cc80ac120 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd80ce790 con 0x7f0ce0105100 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] conn(0x7f0cc80ac120 0x7f0cc80ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.846+0000 7f0ce5c00700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc80a1230 0x7f0cc80a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4011550 tx=0x7f0cd4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.847+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] conn(0x7f0cc80ac120 0x7f0cc80ae5c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.848+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== osd.11 v2:172.21.15.86:6824/3801696519 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0cc80aec90 con 0x7f0cc80ac120 2024-05-08T23:02:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.868+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 --> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ce01044f0 con 0x7f0cc80ac120 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.869+0000 7f0cde000700 1 -- 172.21.15.46:0/3042221681 <== osd.11 v2:172.21.15.86:6824/3801696519 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ce01044f0 con 0x7f0cc80ac120 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] conn(0x7f0cc80ac120 msgr2=0x7f0cc80ae5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] conn(0x7f0cc80ac120 0x7f0cc80ae5c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc80a1230 msgr2=0x7f0cc80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc80a1230 0x7f0cc80a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4011550 tx=0x7f0cd4009260 comp rx=0 tx=0).stop 2024-05-08T23:02:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0105100 msgr2=0x7f0ce019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 --2- 172.21.15.46:0/3042221681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0105100 0x7f0ce019d8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800b6c0 tx=0x7f0cd80094b0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce6600700 1 -- 172.21.15.46:0/3042221681 reap_dead start 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 shutdown_connections 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.870+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 >> 172.21.15.46:0/3042221681 conn(0x7f0ce00fb7e0 msgr2=0x7f0ce0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.871+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 shutdown_connections 2024-05-08T23:02:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:43.871+0000 7f0ce7000700 1 -- 172.21.15.46:0/3042221681 wait complete. 2024-05-08T23:02:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 -- 172.21.15.46:0/511675228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20075000 msgr2=0x7fdf200731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 --2- 172.21.15.46:0/511675228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20075000 0x7fdf200731d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009990 tx=0x7fdf14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 -- 172.21.15.46:0/511675228 shutdown_connections 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 --2- 172.21.15.46:0/511675228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20111c30 0x7fdf20114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 --2- 172.21.15.46:0/511675228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20075000 0x7fdf200731d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 --2- 172.21.15.46:0/511675228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf20073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.529+0000 7fdf28000700 1 -- 172.21.15.46:0/511675228 >> 172.21.15.46:0/511675228 conn(0x7fdf20078190 msgr2=0x7fdf200785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.530+0000 7fdf28000700 1 -- 172.21.15.46:0/511675228 shutdown_connections 2024-05-08T23:02:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.530+0000 7fdf28000700 1 -- 172.21.15.46:0/511675228 wait complete. 2024-05-08T23:02:44.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.531+0000 7fdf28000700 1 Processor -- start 2024-05-08T23:02:44.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.531+0000 7fdf28000700 1 -- start start 2024-05-08T23:02:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf201a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20075000 0x7fdf201a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20111c30 0x7fdf2019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019e0a0 con 0x7fdf20073710 2024-05-08T23:02:44.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019e210 con 0x7fdf20075000 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf28000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019e3d0 con 0x7fdf20111c30 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf25600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf201a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf25600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf201a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54360/0 (socket says 172.21.15.46:54360) 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf25600700 1 -- 172.21.15.46:0/2053217426 learned_addr learned my addr 172.21.15.46:0/2053217426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf24c00700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20075000 0x7fdf201a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.532+0000 7fdf26000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20111c30 0x7fdf2019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 -- 172.21.15.46:0/2053217426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20111c30 msgr2=0x7fdf2019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20111c30 0x7fdf2019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 -- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 msgr2=0x7fdf201a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf201a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140095f0 con 0x7fdf20075000 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf26000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf20111c30 0x7fdf2019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:44.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf25600700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf20073710 0x7fdf201a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf24c00700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20075000 0x7fdf201a3d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e9f0 tx=0x7fdf0c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.533+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00c9c0 con 0x7fdf20075000 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.534+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2019e6b0 con 0x7fdf20075000 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.534+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0c010430 con 0x7fdf20075000 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.534+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c018680 con 0x7fdf20075000 2024-05-08T23:02:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.534+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf201aa7c0 con 0x7fdf20075000 2024-05-08T23:02:44.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.535+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf20117830 con 0x7fdf20075000 2024-05-08T23:02:44.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.537+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdf0c010860 con 0x7fdf20075000 2024-05-08T23:02:44.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.538+0000 7fdf1e000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdf080a11b0 0x7fdf080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.538+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdf0c014070 con 0x7fdf20075000 2024-05-08T23:02:44.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf25600700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdf080a11b0 0x7fdf080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf1e000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] conn(0x7fdf080ac1d0 0x7fdf080ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf080aed40 con 0x7fdf080ac1d0 2024-05-08T23:02:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf0c0cd500 con 0x7fdf20075000 2024-05-08T23:02:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf26000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] conn(0x7fdf080ac1d0 0x7fdf080ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.539+0000 7fdf25600700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdf080a11b0 0x7fdf080a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf14006010 tx=0x7fdf1400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.540+0000 7fdf26000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] conn(0x7fdf080ac1d0 0x7fdf080ae670 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:44.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.541+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== osd.4 v2:172.21.15.53:6808/2784307263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fdf080aed40 con 0x7fdf080ac1d0 2024-05-08T23:02:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.558+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 --> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf20110570 con 0x7fdf080ac1d0 2024-05-08T23:02:44.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf1e000700 1 -- 172.21.15.46:0/2053217426 <== osd.4 v2:172.21.15.53:6808/2784307263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdf20110570 con 0x7fdf080ac1d0 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] conn(0x7fdf080ac1d0 msgr2=0x7fdf080ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] conn(0x7fdf080ac1d0 0x7fdf080ae670 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdf080a11b0 msgr2=0x7fdf080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdf080a11b0 0x7fdf080a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf14006010 tx=0x7fdf1400b260 comp rx=0 tx=0).stop 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20075000 msgr2=0x7fdf201a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 --2- 172.21.15.46:0/2053217426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf20075000 0x7fdf201a3d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e9f0 tx=0x7fdf0c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf26000700 1 -- 172.21.15.46:0/2053217426 reap_dead start 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 shutdown_connections 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.559+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 >> 172.21.15.46:0/2053217426 conn(0x7fdf20078190 msgr2=0x7fdf2010fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.560+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 shutdown_connections 2024-05-08T23:02:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.560+0000 7fdf28000700 1 -- 172.21.15.46:0/2053217426 wait complete. 2024-05-08T23:02:45.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:45 smithi046 ceph-mon[36898]: pgmap v268: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T23:02:45.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:45 smithi053 ceph-mon[77587]: pgmap v268: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T23:02:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.287+0000 7f6cbe000700 1 -- 172.21.15.46:0/1771619105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 msgr2=0x7f6cb8105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.287+0000 7f6cbe000700 1 --2- 172.21.15.46:0/1771619105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8105a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009990 tx=0x7f6ca4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 -- 172.21.15.46:0/1771619105 shutdown_connections 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 --2- 172.21.15.46:0/1771619105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb81060f0 0x7f6cb810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 --2- 172.21.15.46:0/1771619105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb8101010 0x7f6cb81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 --2- 172.21.15.46:0/1771619105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8105a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 -- 172.21.15.46:0/1771619105 >> 172.21.15.46:0/1771619105 conn(0x7f6cb80fc900 msgr2=0x7f6cb80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 -- 172.21.15.46:0/1771619105 shutdown_connections 2024-05-08T23:02:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.288+0000 7f6cbe000700 1 -- 172.21.15.46:0/1771619105 wait complete. 2024-05-08T23:02:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.289+0000 7f6cbe000700 1 Processor -- start 2024-05-08T23:02:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 -- start start 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb8101010 0x7f6cb8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb81060f0 0x7f6cb819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb819c1d0 con 0x7f6cb81019c0 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb819c340 con 0x7f6cb81060f0 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbe000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb819c500 con 0x7f6cb8101010 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cb7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cb7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54390/0 (socket says 172.21.15.46:54390) 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb8101010 0x7f6cb8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cb7e00700 1 -- 172.21.15.46:0/802268618 learned_addr learned my addr 172.21.15.46:0/802268618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.290+0000 7f6cbd600700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb81060f0 0x7f6cb819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 -- 172.21.15.46:0/802268618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb8101010 msgr2=0x7f6cb8197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb8101010 0x7f6cb8197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 -- 172.21.15.46:0/802268618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb81060f0 msgr2=0x7f6cb819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cb81060f0 0x7f6cb819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca40095f0 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.291+0000 7f6cb7e00700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8197860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.292+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca401d070 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.292+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca4022470 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.292+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb819c780 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.292+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4017650 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.292+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb81a4040 con 0x7f6cb81019c0 2024-05-08T23:02:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.293+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6cb8110ff0 con 0x7f6cb81019c0 2024-05-08T23:02:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.295+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6ca400fad0 con 0x7f6cb81019c0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cb5600700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ca00a11b0 0x7f6ca00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ca40cea90 con 0x7f6cb81019c0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cbcc00700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ca00a11b0 0x7f6ca00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cb5600700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] conn(0x7f6ca00ac1d0 0x7f6ca00ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ca00aed40 con 0x7f6ca00ac1d0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.296+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca40cee00 con 0x7f6cb81019c0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.297+0000 7f6cbd600700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] conn(0x7f6ca00ac1d0 0x7f6ca00ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.297+0000 7f6cbcc00700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ca00a11b0 0x7f6ca00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009c60 tx=0x7f6cac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.297+0000 7f6cbd600700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] conn(0x7f6ca00ac1d0 0x7f6ca00ae670 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.298+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== osd.3 v2:172.21.15.46:6810/1550587514 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6ca00aed40 con 0x7f6ca00ac1d0 2024-05-08T23:02:45.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.316+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 --> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6cb8061620 con 0x7f6ca00ac1d0 2024-05-08T23:02:45.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cb5600700 1 -- 172.21.15.46:0/802268618 <== osd.3 v2:172.21.15.46:6810/1550587514 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6cb8061620 con 0x7f6ca00ac1d0 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] conn(0x7f6ca00ac1d0 msgr2=0x7f6ca00ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] conn(0x7f6ca00ac1d0 0x7f6ca00ae670 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ca00a11b0 msgr2=0x7f6ca00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ca00a11b0 0x7f6ca00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009c60 tx=0x7f6cac009260 comp rx=0 tx=0).stop 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 msgr2=0x7f6cb8197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 --2- 172.21.15.46:0/802268618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb81019c0 0x7f6cb8197860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca4005740 comp rx=0 tx=0).stop 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbd600700 1 -- 172.21.15.46:0/802268618 reap_dead start 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 shutdown_connections 2024-05-08T23:02:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.317+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 >> 172.21.15.46:0/802268618 conn(0x7f6cb80fc900 msgr2=0x7f6cb80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.318+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 shutdown_connections 2024-05-08T23:02:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.318+0000 7f6cbe000700 1 -- 172.21.15.46:0/802268618 wait complete. 2024-05-08T23:02:45.427 INFO:teuthology.orchestra.run.smithi046.stdout:261993005084 2024-05-08T23:02:45.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 -- 172.21.15.46:0/2320990428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f40731d0 msgr2=0x7fa9f40735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 --2- 172.21.15.46:0/2320990428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f40731d0 0x7fa9f40735b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009990 tx=0x7fa9e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 -- 172.21.15.46:0/2320990428 shutdown_connections 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 --2- 172.21.15.46:0/2320990428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f410d9a0 0x7fa9f410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 --2- 172.21.15.46:0/2320990428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f40731d0 0x7fa9f40735b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 --2- 172.21.15.46:0/2320990428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 0x7fa9f410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 -- 172.21.15.46:0/2320990428 >> 172.21.15.46:0/2320990428 conn(0x7fa9f4078190 msgr2=0x7fa9f40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 -- 172.21.15.46:0/2320990428 shutdown_connections 2024-05-08T23:02:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.605+0000 7fa9fc400700 1 -- 172.21.15.46:0/2320990428 wait complete. 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 Processor -- start 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 -- start start 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 0x7fa9f4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 0x7fa9f4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 0x7fa9f419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419e510 con 0x7fa9f410d9a0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419e680 con 0x7fa9f4073af0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.606+0000 7fa9fc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419e840 con 0x7fa9f40731d0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9fa400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 0x7fa9f419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 0x7fa9f4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 0x7fa9f4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 0x7fa9f4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37798/0 (socket says 172.21.15.46:37798) 2024-05-08T23:02:45.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9fa400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 0x7fa9f419df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54370/0 (socket says 172.21.15.46:54370) 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 -- 172.21.15.46:0/3445583509 learned_addr learned my addr 172.21.15.46:0/3445583509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 0x7fa9f4199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60008/0 (socket says 172.21.15.46:60008) 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 -- 172.21.15.46:0/3445583509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 msgr2=0x7fa9f4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9f4073af0 0x7fa9f4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 -- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 msgr2=0x7fa9f419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 0x7fa9f419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9f9a00700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80095f0 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.607+0000 7fa9fa400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f410d9a0 0x7fa9f419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9f9a00700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 0x7fa9f4199660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000c00 tx=0x7fa9e8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e801d070 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f419eac0 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41a6380 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e802b3b0 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.608+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e802bea0 con 0x7fa9f40731d0 2024-05-08T23:02:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.609+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9d4000fc0 con 0x7fa9f40731d0 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.611+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa9e800fb00 con 0x7fa9f40731d0 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.612+0000 7fa9f2000700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9dc0a1230 0x7fa9dc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.612+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa9e80cde10 con 0x7fa9f40731d0 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.612+0000 7fa9f9000700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9dc0a1230 0x7fa9dc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.612+0000 7fa9f2000700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] conn(0x7fa9dc0ac250 0x7fa9dc0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.612+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9dc0aedc0 con 0x7fa9dc0ac250 2024-05-08T23:02:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.613+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e80ce180 con 0x7fa9f40731d0 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.613+0000 7fa9fa400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] conn(0x7fa9dc0ac250 0x7fa9dc0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.613+0000 7fa9f9000700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9dc0a1230 0x7fa9dc0a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9f419ab70 tx=0x7fa9e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.613+0000 7fa9fa400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] conn(0x7fa9dc0ac250 0x7fa9dc0ae6f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.614+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== osd.2 v2:172.21.15.86:6800/1780351335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa9dc0aedc0 con 0x7fa9dc0ac250 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.632+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 --> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9d4002d40 con 0x7fa9dc0ac250 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.633+0000 7fa9f2000700 1 -- 172.21.15.46:0/3445583509 <== osd.2 v2:172.21.15.86:6800/1780351335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa9d4002d40 con 0x7fa9dc0ac250 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] conn(0x7fa9dc0ac250 msgr2=0x7fa9dc0ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] conn(0x7fa9dc0ac250 0x7fa9dc0ae6f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9dc0a1230 msgr2=0x7fa9dc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9dc0a1230 0x7fa9dc0a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9f419ab70 tx=0x7fa9e0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 msgr2=0x7fa9f4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 --2- 172.21.15.46:0/3445583509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f40731d0 0x7fa9f4199660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000c00 tx=0x7fa9e8005740 comp rx=0 tx=0).stop 2024-05-08T23:02:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fa400700 1 -- 172.21.15.46:0/3445583509 reap_dead start 2024-05-08T23:02:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 shutdown_connections 2024-05-08T23:02:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 >> 172.21.15.46:0/3445583509 conn(0x7fa9f4078190 msgr2=0x7fa9f40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 shutdown_connections 2024-05-08T23:02:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:44.634+0000 7fa9fc400700 1 -- 172.21.15.46:0/3445583509 wait complete. 2024-05-08T23:02:45.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:45 smithi086 ceph-mon[78610]: pgmap v268: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T23:02:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 -- 172.21.15.46:0/3946919510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c0692f0 msgr2=0x7f701c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 --2- 172.21.15.46:0/3946919510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c0692f0 0x7f701c10de60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7014009f90 tx=0x7f701400b330 comp rx=0 tx=0).stop 2024-05-08T23:02:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 -- 172.21.15.46:0/3946919510 shutdown_connections 2024-05-08T23:02:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 --2- 172.21.15.46:0/3946919510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c0692f0 0x7f701c10de60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 --2- 172.21.15.46:0/3946919510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f701c107020 0x7f701c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 --2- 172.21.15.46:0/3946919510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c068950 0x7f701c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 -- 172.21.15.46:0/3946919510 >> 172.21.15.46:0/3946919510 conn(0x7f701c075f80 msgr2=0x7f701c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 -- 172.21.15.46:0/3946919510 shutdown_connections 2024-05-08T23:02:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.613+0000 7f7023200700 1 -- 172.21.15.46:0/3946919510 wait complete. 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.614+0000 7f7023200700 1 Processor -- start 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.614+0000 7f7023200700 1 -- start start 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 0x7f701c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f701c0692f0 0x7f701c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c107020 0x7f701c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19d5f0 con 0x7f701c107020 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19d760 con 0x7f701c068950 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7023200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19d920 con 0x7f701c0692f0 2024-05-08T23:02:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c107020 0x7f701c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f701be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 0x7f701c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f701be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 0x7f701c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60054/0 (socket says 172.21.15.46:60054) 2024-05-08T23:02:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f701be00700 1 -- 172.21.15.46:0/4266126789 learned_addr learned my addr 172.21.15.46:0/4266126789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f701b400700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f701c0692f0 0x7f701c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 -- 172.21.15.46:0/4266126789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f701c0692f0 msgr2=0x7f701c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f701c0692f0 0x7f701c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 -- 172.21.15.46:0/4266126789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 msgr2=0x7f701c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 0x7f701c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f7021200700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080095f0 con 0x7f701c107020 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.615+0000 7f701be00700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f701c068950 0x7f701c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7021200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c107020 0x7f701c19cff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f701400b6c0 tx=0x7f70140094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400e040 con 0x7f701c107020 2024-05-08T23:02:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7014009c40 con 0x7f701c107020 2024-05-08T23:02:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f701401b450 con 0x7f701c107020 2024-05-08T23:02:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140052a0 con 0x7f701c107020 2024-05-08T23:02:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.616+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a6380 con 0x7f701c107020 2024-05-08T23:02:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.617+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f701c111160 con 0x7f701c107020 2024-05-08T23:02:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.620+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f701401b5c0 con 0x7f701c107020 2024-05-08T23:02:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f7018c00700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70000a1230 0x7f70000a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f70140ce2b0 con 0x7f701c107020 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f701be00700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70000a1230 0x7f70000a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f7018c00700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] conn(0x7f70000ac250 0x7f70000ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70000aedc0 con 0x7f70000ac250 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.621+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70140ce620 con 0x7f701c107020 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.622+0000 7f701b400700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] conn(0x7f70000ac250 0x7f70000ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.622+0000 7f701be00700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70000a1230 0x7f70000a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f700c011550 tx=0x7f700c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.622+0000 7f701b400700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] conn(0x7f70000ac250 0x7f70000ae6f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.623+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== osd.0 v2:172.21.15.46:6802/2436508778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f70000aedc0 con 0x7f70000ac250 2024-05-08T23:02:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.640+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 --> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f701c1026d0 con 0x7f70000ac250 2024-05-08T23:02:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7018c00700 1 -- 172.21.15.46:0/4266126789 <== osd.0 v2:172.21.15.46:6802/2436508778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f701c1026d0 con 0x7f70000ac250 2024-05-08T23:02:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] conn(0x7f70000ac250 msgr2=0x7f70000ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] conn(0x7f70000ac250 0x7f70000ae6f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70000a1230 msgr2=0x7f70000a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70000a1230 0x7f70000a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f700c011550 tx=0x7f700c009260 comp rx=0 tx=0).stop 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c107020 msgr2=0x7f701c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 --2- 172.21.15.46:0/4266126789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c107020 0x7f701c19cff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f701400b6c0 tx=0x7f70140094b0 comp rx=0 tx=0).stop 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7021200700 1 -- 172.21.15.46:0/4266126789 reap_dead start 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 shutdown_connections 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.641+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 >> 172.21.15.46:0/4266126789 conn(0x7f701c075f80 msgr2=0x7f701c10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.642+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 shutdown_connections 2024-05-08T23:02:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:45.642+0000 7f7023200700 1 -- 172.21.15.46:0/4266126789 wait complete. 2024-05-08T23:02:45.842 INFO:teuthology.orchestra.run.smithi046.stdout:231928234017 2024-05-08T23:02:45.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T23:02:45.953 INFO:teuthology.orchestra.run.smithi046.stdout:313532612629 2024-05-08T23:02:45.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.298+0000 7f8d16a00700 1 -- 172.21.15.46:0/1618159611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10101990 msgr2=0x7f8d1010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.298+0000 7f8d16a00700 1 --2- 172.21.15.46:0/1618159611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10101990 0x7f8d1010de70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800b0a0 tx=0x7f8d0800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 -- 172.21.15.46:0/1618159611 shutdown_connections 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 --2- 172.21.15.46:0/1618159611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10101990 0x7f8d1010de70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 --2- 172.21.15.46:0/1618159611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10107030 0x7f8d10107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 --2- 172.21.15.46:0/1618159611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10100ff0 0x7f8d10101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 -- 172.21.15.46:0/1618159611 >> 172.21.15.46:0/1618159611 conn(0x7f8d10078190 msgr2=0x7f8d100785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 -- 172.21.15.46:0/1618159611 shutdown_connections 2024-05-08T23:02:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.299+0000 7f8d16a00700 1 -- 172.21.15.46:0/1618159611 wait complete. 2024-05-08T23:02:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 Processor -- start 2024-05-08T23:02:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 -- start start 2024-05-08T23:02:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10100ff0 0x7f8d100732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 0x7f8d100737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10107030 0x7f8d10073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101a5970 con 0x7f8d10101990 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101a5ae0 con 0x7f8d10107030 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.300+0000 7f8d16a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101a5ca0 con 0x7f8d10100ff0 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 0x7f8d100737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 0x7f8d100737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54414/0 (socket says 172.21.15.46:54414) 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 -- 172.21.15.46:0/398742769 learned_addr learned my addr 172.21.15.46:0/398742769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d0fe00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10100ff0 0x7f8d100732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d14a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10107030 0x7f8d10073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 -- 172.21.15.46:0/398742769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10100ff0 msgr2=0x7f8d100732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10100ff0 0x7f8d100732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 -- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10107030 msgr2=0x7f8d10073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10107030 0x7f8d10073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d07e00700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00009520 con 0x7f8d10101990 2024-05-08T23:02:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.301+0000 7f8d0fe00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d10100ff0 0x7f8d100732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d07e00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 0x7f8d100737e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9c0 tx=0x7f8cf800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80097b0 con 0x7f8d10101990 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cf800ee60 con 0x7f8d10101990 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d08009d00 con 0x7f8d10101990 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80054f0 con 0x7f8d10101990 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.302+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d101a6210 con 0x7f8d10101990 2024-05-08T23:02:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.303+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d10111220 con 0x7f8d10101990 2024-05-08T23:02:46.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.305+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8cf8004750 con 0x7f8d10101990 2024-05-08T23:02:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.306+0000 7f8d0d600700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8cf40a11b0 0x7f8cf40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.306+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8cf8014070 con 0x7f8d10101990 2024-05-08T23:02:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d0fe00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8cf40a11b0 0x7f8cf40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d0d600700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] conn(0x7f8cf40ac1d0 0x7f8cf40ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cf40aed40 con 0x7f8cf40ac1d0 2024-05-08T23:02:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cf80cd600 con 0x7f8d10101990 2024-05-08T23:02:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d14a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] conn(0x7f8cf40ac1d0 0x7f8cf40ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.307+0000 7f8d0fe00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8cf40a11b0 0x7f8cf40a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011630 tx=0x7f8d00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.308+0000 7f8d14a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] conn(0x7f8cf40ac1d0 0x7f8cf40ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.309+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== osd.7 v2:172.21.15.53:6816/521136929 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8cf40aed40 con 0x7f8cf40ac1d0 2024-05-08T23:02:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.326+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 --> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d10061620 con 0x7f8cf40ac1d0 2024-05-08T23:02:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d0d600700 1 -- 172.21.15.46:0/398742769 <== osd.7 v2:172.21.15.53:6816/521136929 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8d10061620 con 0x7f8cf40ac1d0 2024-05-08T23:02:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] conn(0x7f8cf40ac1d0 msgr2=0x7f8cf40ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] conn(0x7f8cf40ac1d0 0x7f8cf40ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8cf40a11b0 msgr2=0x7f8cf40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8cf40a11b0 0x7f8cf40a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011630 tx=0x7f8d00009260 comp rx=0 tx=0).stop 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 msgr2=0x7f8d100737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 --2- 172.21.15.46:0/398742769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d10101990 0x7f8d100737e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9c0 tx=0x7f8cf800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d14a00700 1 -- 172.21.15.46:0/398742769 reap_dead start 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 shutdown_connections 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 >> 172.21.15.46:0/398742769 conn(0x7f8d10078190 msgr2=0x7f8d100ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 shutdown_connections 2024-05-08T23:02:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:46.327+0000 7f8d16a00700 1 -- 172.21.15.46:0/398742769 wait complete. 2024-05-08T23:02:46.750 INFO:teuthology.orchestra.run.smithi046.stdout:103079215156 2024-05-08T23:02:46.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T23:02:46.934 INFO:teuthology.orchestra.run.smithi046.stdout:77309411383 2024-05-08T23:02:46.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T23:02:47.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:47 smithi046 ceph-mon[36898]: pgmap v269: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:47.207 INFO:teuthology.orchestra.run.smithi046.stdout:201863462949 2024-05-08T23:02:47.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T23:02:47.231 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:47.251 INFO:teuthology.orchestra.run.smithi046.stdout:34359738432 2024-05-08T23:02:47.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T23:02:47.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:47 smithi053 ceph-mon[77587]: pgmap v269: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:47.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:47 smithi086 ceph-mon[78610]: pgmap v269: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:47.675 INFO:teuthology.orchestra.run.smithi046.stdout:133143986224 2024-05-08T23:02:47.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T23:02:47.836 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.851+0000 7f6ee1c00700 1 -- 172.21.15.46:0/17213572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc100ff0 msgr2=0x7f6edc101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.851+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/17213572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc100ff0 0x7f6edc101450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80098f0 tx=0x7f6ec8009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 -- 172.21.15.46:0/17213572 shutdown_connections 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/17213572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6edc101990 0x7f6edc10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/17213572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc107030 0x7f6edc107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/17213572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc100ff0 0x7f6edc101450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 -- 172.21.15.46:0/17213572 >> 172.21.15.46:0/17213572 conn(0x7f6edc078190 msgr2=0x7f6edc0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 -- 172.21.15.46:0/17213572 shutdown_connections 2024-05-08T23:02:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.852+0000 7f6ee1c00700 1 -- 172.21.15.46:0/17213572 wait complete. 2024-05-08T23:02:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.854+0000 7f6ee1c00700 1 Processor -- start 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.854+0000 7f6ee1c00700 1 -- start start 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.854+0000 7f6ee1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc100ff0 0x7f6edc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6ee1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc101990 0x7f6edc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6ee1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6edc107030 0x7f6edc19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6ee1c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc19c3f0 con 0x7f6edc100ff0 2024-05-08T23:02:47.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6ee1c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc19c560 con 0x7f6edc107030 2024-05-08T23:02:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6ee1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc19c720 con 0x7f6edc101990 2024-05-08T23:02:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6edaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc100ff0 0x7f6edc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6eda000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc101990 0x7f6edc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6edaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc100ff0 0x7f6edc1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54434/0 (socket says 172.21.15.46:54434) 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6edaa00700 1 -- 172.21.15.46:0/2326917653 learned_addr learned my addr 172.21.15.46:0/2326917653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.855+0000 7f6edb400700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6edc107030 0x7f6edc19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 -- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6edc107030 msgr2=0x7f6edc19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6edc107030 0x7f6edc19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 -- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc100ff0 msgr2=0x7f6edc1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc100ff0 0x7f6edc1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0095f0 con 0x7f6edc101990 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.856+0000 7f6eda000700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc101990 0x7f6edc197a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009fd0 tx=0x7f6ec800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.857+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8020070 con 0x7f6edc101990 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.857+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec801bd50 con 0x7f6edc101990 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.857+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80052a0 con 0x7f6edc101990 2024-05-08T23:02:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.857+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8009520 con 0x7f6edc101990 2024-05-08T23:02:47.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.857+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc19cbd0 con 0x7f6edc101990 2024-05-08T23:02:47.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.858+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6edc111200 con 0x7f6edc101990 2024-05-08T23:02:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.861+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6ec8024030 con 0x7f6edc101990 2024-05-08T23:02:47.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6ed3400700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ec00a1230 0x7f6ec00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ec80d22d0 con 0x7f6edc101990 2024-05-08T23:02:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6edaa00700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ec00a1230 0x7f6ec00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6ed3400700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] conn(0x7f6ec00ac120 0x7f6ec00ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ec00aec90 con 0x7f6ec00ac120 2024-05-08T23:02:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.863+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ec80d26d0 con 0x7f6edc101990 2024-05-08T23:02:47.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.864+0000 7f6edb400700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] conn(0x7f6ec00ac120 0x7f6ec00ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.864+0000 7f6edaa00700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ec00a1230 0x7f6ec00a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005d10 tx=0x7f6ecc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.864+0000 7f6edb400700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] conn(0x7f6ec00ac120 0x7f6ec00ae5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.865+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== osd.10 v2:172.21.15.53:6824/75118943 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6ec00aec90 con 0x7f6ec00ac120 2024-05-08T23:02:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.884+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 --> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6edc061620 con 0x7f6ec00ac120 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.884+0000 7f6ed3400700 1 -- 172.21.15.46:0/2326917653 <== osd.10 v2:172.21.15.53:6824/75118943 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6edc061620 con 0x7f6ec00ac120 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] conn(0x7f6ec00ac120 msgr2=0x7f6ec00ae5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] conn(0x7f6ec00ac120 0x7f6ec00ae5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ec00a1230 msgr2=0x7f6ec00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ec00a1230 0x7f6ec00a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005d10 tx=0x7f6ecc009310 comp rx=0 tx=0).stop 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc101990 msgr2=0x7f6edc197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6ee1c00700 1 --2- 172.21.15.46:0/2326917653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc101990 0x7f6edc197a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009fd0 tx=0x7f6ec800fa00 comp rx=0 tx=0).stop 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.885+0000 7f6edb400700 1 -- 172.21.15.46:0/2326917653 reap_dead start 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.886+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 shutdown_connections 2024-05-08T23:02:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.886+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 >> 172.21.15.46:0/2326917653 conn(0x7f6edc078190 msgr2=0x7f6edc0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.887+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 shutdown_connections 2024-05-08T23:02:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.887+0000 7f6ee1c00700 1 -- 172.21.15.46:0/2326917653 wait complete. 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 -- 172.21.15.46:0/2034482834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 msgr2=0x7fec1410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 --2- 172.21.15.46:0/2034482834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec1410de70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b0a0 tx=0x7fec0c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 -- 172.21.15.46:0/2034482834 shutdown_connections 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 --2- 172.21.15.46:0/2034482834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec1410de70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 --2- 172.21.15.46:0/2034482834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14107030 0x7fec14107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 --2- 172.21.15.46:0/2034482834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec14100ff0 0x7fec14101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 -- 172.21.15.46:0/2034482834 >> 172.21.15.46:0/2034482834 conn(0x7fec14078190 msgr2=0x7fec140785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 -- 172.21.15.46:0/2034482834 shutdown_connections 2024-05-08T23:02:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.905+0000 7fec1b000700 1 -- 172.21.15.46:0/2034482834 wait complete. 2024-05-08T23:02:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.907+0000 7fec1b000700 1 Processor -- start 2024-05-08T23:02:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.907+0000 7fec1b000700 1 -- start start 2024-05-08T23:02:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.907+0000 7fec1b000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 0x7fec14197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec1b000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec14197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec1b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec14107030 0x7fec1419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec1b000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c440 con 0x7fec14100ff0 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec1b000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c5b0 con 0x7fec14101990 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec1b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c770 con 0x7fec14107030 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 0x7fec14197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 0x7fec14197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54456/0 (socket says 172.21.15.46:54456) 2024-05-08T23:02:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13e00700 1 -- 172.21.15.46:0/1040211884 learned_addr learned my addr 172.21.15.46:0/1040211884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec14197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec19000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec14107030 0x7fec1419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13e00700 1 -- 172.21.15.46:0/1040211884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec14107030 msgr2=0x7fec1419be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.908+0000 7fec13e00700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec14107030 0x7fec1419be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec13e00700 1 -- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 msgr2=0x7fec14197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec13e00700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec14197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec13e00700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009d00 con 0x7fec14100ff0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec13400700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec14101990 0x7fec14197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec13e00700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 0x7fec14197540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9a0 tx=0x7fec0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04009790 con 0x7fec14100ff0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec04010430 con 0x7fec14100ff0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1419ca50 con 0x7fec14100ff0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.909+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04018680 con 0x7fec14100ff0 2024-05-08T23:02:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.910+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec140733a0 con 0x7fec14100ff0 2024-05-08T23:02:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.911+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec141111b0 con 0x7fec14100ff0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.913+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fec040105a0 con 0x7fec14100ff0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.914+0000 7fec10c00700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7febf80a12b0 0x7febf80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec04014070 con 0x7fec14100ff0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec13400700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7febf80a12b0 0x7febf80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec10c00700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] conn(0x7febf80ac2d0 0x7febf80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febf80aee40 con 0x7febf80ac2d0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec040cd530 con 0x7fec14100ff0 2024-05-08T23:02:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec19000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] conn(0x7febf80ac2d0 0x7febf80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.915+0000 7fec13400700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7febf80a12b0 0x7febf80a3770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec14198aa0 tx=0x7fec0000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.916+0000 7fec19000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] conn(0x7febf80ac2d0 0x7febf80ae770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.917+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== osd.1 v2:172.21.15.53:6800/3799790044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7febf80aee40 con 0x7febf80ac2d0 2024-05-08T23:02:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.935+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 --> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec141026e0 con 0x7febf80ac2d0 2024-05-08T23:02:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.936+0000 7fec10c00700 1 -- 172.21.15.46:0/1040211884 <== osd.1 v2:172.21.15.53:6800/3799790044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec141026e0 con 0x7febf80ac2d0 2024-05-08T23:02:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] conn(0x7febf80ac2d0 msgr2=0x7febf80ae770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] conn(0x7febf80ac2d0 0x7febf80ae770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7febf80a12b0 msgr2=0x7febf80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7febf80a12b0 0x7febf80a3770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec14198aa0 tx=0x7fec0000b410 comp rx=0 tx=0).stop 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 msgr2=0x7fec14197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec1b000700 1 --2- 172.21.15.46:0/1040211884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec14100ff0 0x7fec14197540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9a0 tx=0x7fec0400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.937+0000 7fec19000700 1 -- 172.21.15.46:0/1040211884 reap_dead start 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.938+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 shutdown_connections 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.938+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 >> 172.21.15.46:0/1040211884 conn(0x7fec14078190 msgr2=0x7fec140ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.938+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 shutdown_connections 2024-05-08T23:02:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:47.938+0000 7fec1b000700 1 -- 172.21.15.46:0/1040211884 wait complete. 2024-05-08T23:02:48.854 INFO:teuthology.orchestra.run.smithi046.stdout:283467841560 2024-05-08T23:02:48.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T23:02:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:49 smithi053 ceph-mon[77587]: pgmap v270: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:49.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:49 smithi046 ceph-mon[36898]: pgmap v270: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:49.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:49 smithi086 ceph-mon[78610]: pgmap v270: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:49.635 INFO:teuthology.orchestra.run.smithi046.stdout:55834574910 2024-05-08T23:02:49.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T23:02:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.817+0000 7f0550200700 1 -- 172.21.15.46:0/2982485442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 msgr2=0x7f0548073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.817+0000 7f0550200700 1 --2- 172.21.15.46:0/2982485442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f0548073b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0534009990 tx=0x7f0534009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 -- 172.21.15.46:0/2982485442 shutdown_connections 2024-05-08T23:02:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 --2- 172.21.15.46:0/2982485442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548111c30 0x7f0548114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 --2- 172.21.15.46:0/2982485442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548075000 0x7f05480731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 --2- 172.21.15.46:0/2982485442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f0548073b90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 -- 172.21.15.46:0/2982485442 >> 172.21.15.46:0/2982485442 conn(0x7f0548078190 msgr2=0x7f05480785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 -- 172.21.15.46:0/2982485442 shutdown_connections 2024-05-08T23:02:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.818+0000 7f0550200700 1 -- 172.21.15.46:0/2982485442 wait complete. 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 Processor -- start 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 -- start start 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f05481a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548075000 0x7f05481a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 0x7f054819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819e0a0 con 0x7f0548073710 2024-05-08T23:02:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819e210 con 0x7f0548075000 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.819+0000 7f0550200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819e3d0 con 0x7f0548111c30 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f05481a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 0x7f054819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f05481a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54472/0 (socket says 172.21.15.46:54472) 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054d800700 1 -- 172.21.15.46:0/3792309181 learned_addr learned my addr 172.21.15.46:0/3792309181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054ce00700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548075000 0x7f05481a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 -- 172.21.15.46:0/3792309181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548075000 msgr2=0x7f05481a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548075000 0x7f05481a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 -- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 msgr2=0x7f05481a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f05481a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 -- 172.21.15.46:0/3792309181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340095f0 con 0x7f0548111c30 2024-05-08T23:02:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.820+0000 7f054e200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 0x7f054819d8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05481053d0 tx=0x7f054000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.821+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540018070 con 0x7f0548111c30 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.821+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054819e650 con 0x7f0548111c30 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.821+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054000f040 con 0x7f0548111c30 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.821+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000cc10 con 0x7f0548111c30 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.821+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481aa7c0 con 0x7f0548111c30 2024-05-08T23:02:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.822+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05480748b0 con 0x7f0548111c30 2024-05-08T23:02:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.824+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0540003a00 con 0x7f0548111c30 2024-05-08T23:02:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.826+0000 7f0546000700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 0x7f05300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.826+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05400cc960 con 0x7f0548111c30 2024-05-08T23:02:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.826+0000 7f054d800700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 0x7f05300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.827+0000 7f054d800700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 0x7f05300a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d20 tx=0x7f053c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:49.828+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0540095cb0 con 0x7f0548111c30 2024-05-08T23:02:50.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.114+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f054819edd0 con 0x7f0548111c30 2024-05-08T23:02:50.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.115+0000 7f0546000700 1 -- 172.21.15.46:0/3792309181 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05400992d0 con 0x7f0548111c30 2024-05-08T23:02:50.114 INFO:teuthology.orchestra.run.smithi046.stdout:150323855405 2024-05-08T23:02:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 msgr2=0x7f05300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 0x7f05300a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d20 tx=0x7f053c009260 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 msgr2=0x7f054819d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 0x7f054819d8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05481053d0 tx=0x7f054000ef90 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 shutdown_connections 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0548111c30 0x7f054819d8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05300a1230 0x7f05300a36f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548073710 0x7f05481a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 --2- 172.21.15.46:0/3792309181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0548075000 0x7f05481a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 >> 172.21.15.46:0/3792309181 conn(0x7f0548078190 msgr2=0x7f054810fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.118+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 shutdown_connections 2024-05-08T23:02:50.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.119+0000 7f0550200700 1 -- 172.21.15.46:0/3792309181 wait complete. 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 -- 172.21.15.46:0/806101901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644105ec0 msgr2=0x7f0644068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 --2- 172.21.15.46:0/806101901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644105ec0 0x7f0644068950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0634009990 tx=0x7f0634009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 -- 172.21.15.46:0/806101901 shutdown_connections 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 --2- 172.21.15.46:0/806101901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06441039d0 0x7f0644103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 --2- 172.21.15.46:0/806101901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644105ec0 0x7f0644068950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 --2- 172.21.15.46:0/806101901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0644068e90 0x7f0644069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 -- 172.21.15.46:0/806101901 >> 172.21.15.46:0/806101901 conn(0x7f0644075f80 msgr2=0x7f0644076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 -- 172.21.15.46:0/806101901 shutdown_connections 2024-05-08T23:02:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.199+0000 7f064ac00700 1 -- 172.21.15.46:0/806101901 wait complete. 2024-05-08T23:02:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.201+0000 7f064ac00700 1 Processor -- start 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.201+0000 7f064ac00700 1 -- start start 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06441039d0 0x7f064419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0644105ec0 0x7f06441a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a2930 con 0x7f06441039d0 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a2aa0 con 0x7f0644068e90 2024-05-08T23:02:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f064ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a2c60 con 0x7f0644105ec0 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f0643e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f0643e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60126/0 (socket says 172.21.15.46:60126) 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f0643e00700 1 -- 172.21.15.46:0/1871593119 learned_addr learned my addr 172.21.15.46:0/1871593119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.202+0000 7f0643400700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06441039d0 0x7f064419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 -- 172.21.15.46:0/1871593119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0644105ec0 msgr2=0x7f06441a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0644105ec0 0x7f06441a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 -- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06441039d0 msgr2=0x7f064419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06441039d0 0x7f064419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 -- 172.21.15.46:0/1871593119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c009c40 con 0x7f0644068e90 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.203+0000 7f0643e00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.204+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063401d070 con 0x7f0644068e90 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.204+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0634022470 con 0x7f0644068e90 2024-05-08T23:02:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.204+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634017650 con 0x7f0644068e90 2024-05-08T23:02:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.204+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06340095f0 con 0x7f0644068e90 2024-05-08T23:02:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.204+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a3110 con 0x7f0644068e90 2024-05-08T23:02:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.206+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06441004b0 con 0x7f0644068e90 2024-05-08T23:02:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.208+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f06340225e0 con 0x7f0644068e90 2024-05-08T23:02:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.209+0000 7f0640c00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 0x7f06280a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.210+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f06340ce070 con 0x7f0644068e90 2024-05-08T23:02:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.210+0000 7f0643400700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 0x7f06280a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.211+0000 7f0643400700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 0x7f06280a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f064419efa0 tx=0x7f0630009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:50.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.216+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0634097400 con 0x7f0644068e90 2024-05-08T23:02:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:50 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:02:50.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:50 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:02:50.458 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:50.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:50 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:02:50.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.475+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f064419ed50 con 0x7f0644068e90 2024-05-08T23:02:50.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.476+0000 7f0640c00700 1 -- 172.21.15.46:0/1871593119 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f063409aa20 con 0x7f0644068e90 2024-05-08T23:02:50.475 INFO:teuthology.orchestra.run.smithi046.stdout:180388626474 2024-05-08T23:02:50.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 msgr2=0x7f06280a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 0x7f06280a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f064419efa0 tx=0x7f0630009260 comp rx=0 tx=0).stop 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 msgr2=0x7f064419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634005e80 comp rx=0 tx=0).stop 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 shutdown_connections 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0644105ec0 0x7f06441a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0644068e90 0x7f064419da40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06280a1230 0x7f06280a36f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 --2- 172.21.15.46:0/1871593119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06441039d0 0x7f064419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.478+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 >> 172.21.15.46:0/1871593119 conn(0x7f0644075f80 msgr2=0x7f06441020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.479+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 shutdown_connections 2024-05-08T23:02:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:50.479+0000 7f064ac00700 1 -- 172.21.15.46:0/1871593119 wait complete. 2024-05-08T23:02:50.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626472 got 180388626474 for osd.6 2024-05-08T23:02:50.982 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:51.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:51 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3792309181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:02:51.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:51 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1871593119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:02:51.231 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.231 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:51 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.231 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:51 smithi046 ceph-mon[36898]: pgmap v271: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:51 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3792309181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:02:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:51 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1871593119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:02:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:51 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:51 smithi053 ceph-mon[77587]: pgmap v271: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:51.385 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855403 got 150323855405 for osd.5 2024-05-08T23:02:51.385 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:51.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:51 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3792309181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:02:51.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:51 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1871593119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:02:51.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:51 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:51.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:51 smithi086 ceph-mon[78610]: pgmap v271: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-08T23:02:51.673 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:52.334 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:52.334 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:52 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:52.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:52.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:52 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:52.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:52.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:52 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:02:53.419 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:53 smithi046 ceph-mon[36898]: pgmap v272: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-08T23:02:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 -- 172.21.15.46:0/2828521380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 msgr2=0x7f5bb4107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/2828521380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb4107410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009990 tx=0x7f5ba4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 -- 172.21.15.46:0/2828521380 shutdown_connections 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/2828521380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 0x7f5bb410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/2828521380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb4107410 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/2828521380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 0x7f5bb4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 -- 172.21.15.46:0/2828521380 >> 172.21.15.46:0/2828521380 conn(0x7f5bb4078190 msgr2=0x7f5bb40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 -- 172.21.15.46:0/2828521380 shutdown_connections 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.270+0000 7f5bb8e00700 1 -- 172.21.15.46:0/2828521380 wait complete. 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 Processor -- start 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 -- start start 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 0x7f5bb41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 0x7f5bb4199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:53.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419e4e0 con 0x7f5bb4107030 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419e650 con 0x7f5bb4101990 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.271+0000 7f5bb8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb419e810 con 0x7f5bb4100ff0 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 0x7f5bb41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44792/0 (socket says 172.21.15.46:44792) 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 0x7f5bb4199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 -- 172.21.15.46:0/197494421 learned_addr learned my addr 172.21.15.46:0/197494421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 -- 172.21.15.46:0/197494421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 msgr2=0x7f5bb41995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 0x7f5bb41995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 -- 172.21.15.46:0/197494421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 msgr2=0x7f5bb4199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 0x7f5bb4199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 -- 172.21.15.46:0/197494421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40095f0 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.272+0000 7f5bb2800700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00efc0 tx=0x7f5bac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.273+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018070 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.273+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb419ea90 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.273+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac00f040 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.273+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00cc10 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.273+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb41a6350 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.274+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb4104370 con 0x7f5bb4107030 2024-05-08T23:02:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.277+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5bac00cd70 con 0x7f5bb4107030 2024-05-08T23:02:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.279+0000 7f5baaa00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 0x7f5b980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.279+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5bac0cd0f0 con 0x7f5bb4107030 2024-05-08T23:02:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.279+0000 7f5bb1e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 0x7f5b980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.280+0000 7f5bb1e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 0x7f5b980a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098e0 tx=0x7f5ba4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.282+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5bac096440 con 0x7f5bb4107030 2024-05-08T23:02:53.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:53 smithi086 ceph-mon[78610]: pgmap v272: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-08T23:02:53.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:53 smithi053 ceph-mon[77587]: pgmap v272: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-08T23:02:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.538+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f5bb41026e0 con 0x7f5bb4107030 2024-05-08T23:02:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.539+0000 7f5baaa00700 1 -- 172.21.15.46:0/197494421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bac099a60 con 0x7f5bb4107030 2024-05-08T23:02:53.538 INFO:teuthology.orchestra.run.smithi046.stdout:261993005086 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.541+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 msgr2=0x7f5b980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.541+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 0x7f5b980a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098e0 tx=0x7f5ba4005960 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.541+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 msgr2=0x7f5bb419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.541+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00efc0 tx=0x7f5bac00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 shutdown_connections 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b980a11b0 0x7f5b980a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bb4107030 0x7f5bb419ded0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4100ff0 0x7f5bb41995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 --2- 172.21.15.46:0/197494421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bb4101990 0x7f5bb4199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 >> 172.21.15.46:0/197494421 conn(0x7f5bb4078190 msgr2=0x7f5bb410c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 shutdown_connections 2024-05-08T23:02:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:53.542+0000 7f5bb8e00700 1 -- 172.21.15.46:0/197494421 wait complete. 2024-05-08T23:02:54.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/197494421' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:02:54.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/197494421' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:02:54.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/197494421' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:02:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 -- 172.21.15.46:0/1640178108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8068980 msgr2=0x7fcac8068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 --2- 172.21.15.46:0/1640178108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8068980 0x7fcac8068de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcab40098f0 tx=0x7fcab4009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 -- 172.21.15.46:0/1640178108 shutdown_connections 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 --2- 172.21.15.46:0/1640178108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8069320 0x7fcac810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 --2- 172.21.15.46:0/1640178108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8107010 0x7fcac81073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 --2- 172.21.15.46:0/1640178108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8068980 0x7fcac8068de0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 -- 172.21.15.46:0/1640178108 >> 172.21.15.46:0/1640178108 conn(0x7fcac8075f80 msgr2=0x7fcac8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 -- 172.21.15.46:0/1640178108 shutdown_connections 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.990+0000 7fcacde00700 1 -- 172.21.15.46:0/1640178108 wait complete. 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 Processor -- start 2024-05-08T23:02:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 -- start start 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 0x7fcac8197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 0x7fcac81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8107010 0x7fcac819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac819c330 con 0x7fcac8068980 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac819c4a0 con 0x7fcac8069320 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.991+0000 7fcacde00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac819c660 con 0x7fcac8107010 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 0x7fcac8197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8107010 0x7fcac819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 0x7fcac81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 0x7fcac8197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44818/0 (socket says 172.21.15.46:44818) 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6a00700 1 -- 172.21.15.46:0/373888718 learned_addr learned my addr 172.21.15.46:0/373888718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 -- 172.21.15.46:0/373888718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8107010 msgr2=0x7fcac819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8107010 0x7fcac819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 -- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 msgr2=0x7fcac8197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 0x7fcac8197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 -- 172.21.15.46:0/373888718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009520 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.992+0000 7fcac6000700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 0x7fcac81979c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009230 tx=0x7fcab400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.993+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab401c070 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.993+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac819c8e0 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.993+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab400ba20 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.993+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40038e0 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.993+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac81000a0 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.994+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac8109530 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.996+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcab4020030 con 0x7fcac8069320 2024-05-08T23:02:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.997+0000 7fcacca00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 0x7fcaac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.998+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcab40cdf90 con 0x7fcac8069320 2024-05-08T23:02:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.998+0000 7fcac6a00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 0x7fcaac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:54.998+0000 7fcac6a00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 0x7fcaac0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcab8005950 tx=0x7fcab8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.000+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcab40972e0 con 0x7fcac8069320 2024-05-08T23:02:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.228+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcac81026c0 con 0x7fcac8069320 2024-05-08T23:02:55.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.229+0000 7fcacca00700 1 -- 172.21.15.46:0/373888718 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcab409a900 con 0x7fcac8069320 2024-05-08T23:02:55.228 INFO:teuthology.orchestra.run.smithi046.stdout:77309411385 2024-05-08T23:02:55.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 msgr2=0x7fcaac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 0x7fcaac0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcab8005950 tx=0x7fcab8009310 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 msgr2=0x7fcac81979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 0x7fcac81979c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009230 tx=0x7fcab400fa00 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 shutdown_connections 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8107010 0x7fcac819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fcaac0a11b0 0x7fcaac0a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8068980 0x7fcac8197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 --2- 172.21.15.46:0/373888718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac8069320 0x7fcac81979c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.231+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 >> 172.21.15.46:0/373888718 conn(0x7fcac8075f80 msgr2=0x7fcac80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.232+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 shutdown_connections 2024-05-08T23:02:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:55.232+0000 7fcacde00700 1 -- 172.21.15.46:0/373888718 wait complete. 2024-05-08T23:02:55.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:55 smithi046 ceph-mon[36898]: pgmap v273: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:02:55.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:55 smithi086 ceph-mon[78610]: pgmap v273: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:02:55.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:55 smithi053 ceph-mon[77587]: pgmap v273: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:02:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 -- 172.21.15.46:0/202418160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 msgr2=0x7fb4a0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 --2- 172.21.15.46:0/202418160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a0101450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0098f0 tx=0x7fb48c009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 -- 172.21.15.46:0/202418160 shutdown_connections 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 --2- 172.21.15.46:0/202418160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 --2- 172.21.15.46:0/202418160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 0x7fb4a0107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 --2- 172.21.15.46:0/202418160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a0101450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 -- 172.21.15.46:0/202418160 >> 172.21.15.46:0/202418160 conn(0x7fb4a0078190 msgr2=0x7fb4a00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 -- 172.21.15.46:0/202418160 shutdown_connections 2024-05-08T23:02:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.127+0000 7fb4a7000700 1 -- 172.21.15.46:0/202418160 wait complete. 2024-05-08T23:02:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.128+0000 7fb4a7000700 1 Processor -- start 2024-05-08T23:02:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.128+0000 7fb4a7000700 1 -- start start 2024-05-08T23:02:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.128+0000 7fb4a7000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.128+0000 7fb4a7000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.128+0000 7fb4a7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 0x7fb4a0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb4a7000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a5830 con 0x7fb4a0100ff0 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb4a7000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a59a0 con 0x7fb4a0101990 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb4a7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a5b60 con 0x7fb4a0107030 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52090/0 (socket says 172.21.15.46:52090) 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 -- 172.21.15.46:0/3106363298 learned_addr learned my addr 172.21.15.46:0/3106363298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb4a5000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 0x7fb4a0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49fe00700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 -- 172.21.15.46:0/3106363298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 msgr2=0x7fb4a0073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 0x7fb4a0073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 -- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 msgr2=0x7fb4a00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.129+0000 7fb49f400700 1 -- 172.21.15.46:0/3106363298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009520 con 0x7fb4a0101990 2024-05-08T23:02:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb49f400700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005950 tx=0x7fb48c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c01c070 con 0x7fb4a0101990 2024-05-08T23:02:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb48c00bb20 con 0x7fb4a0101990 2024-05-08T23:02:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a01a5de0 con 0x7fb4a0101990 2024-05-08T23:02:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0038e0 con 0x7fb4a0101990 2024-05-08T23:02:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.130+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01a61a0 con 0x7fb4a0101990 2024-05-08T23:02:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.132+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a0109550 con 0x7fb4a0101990 2024-05-08T23:02:56.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.134+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb48c020030 con 0x7fb4a0101990 2024-05-08T23:02:56.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.135+0000 7fb49cc00700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 0x7fb4840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.135+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb48c0cdf30 con 0x7fb4a0101990 2024-05-08T23:02:56.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.136+0000 7fb49fe00700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 0x7fb4840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.136+0000 7fb49fe00700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 0x7fb4840a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb490005fd0 tx=0x7fb490009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:56.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.138+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb48c0fbd60 con 0x7fb4a0101990 2024-05-08T23:02:56.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411385 for osd.2 2024-05-08T23:02:56.210 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:56.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/373888718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:02:56.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005084 got 261993005086 for osd.9 2024-05-08T23:02:56.334 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:56.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.378+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb4a01026e0 con 0x7fb4a0101990 2024-05-08T23:02:56.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.379+0000 7fb49cc00700 1 -- 172.21.15.46:0/3106363298 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb48c097280 con 0x7fb4a0101990 2024-05-08T23:02:56.378 INFO:teuthology.orchestra.run.smithi046.stdout:55834574911 2024-05-08T23:02:56.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 msgr2=0x7fb4840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 0x7fb4840a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb490005fd0 tx=0x7fb490009310 comp rx=0 tx=0).stop 2024-05-08T23:02:56.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 msgr2=0x7fb4a00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005950 tx=0x7fb48c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 shutdown_connections 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0107030 0x7fb4a0073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb4840a11b0 0x7fb4840a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a0100ff0 0x7fb4a00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 --2- 172.21.15.46:0/3106363298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a0101990 0x7fb4a00737e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 >> 172.21.15.46:0/3106363298 conn(0x7fb4a0078190 msgr2=0x7fb4a00fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 shutdown_connections 2024-05-08T23:02:56.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.381+0000 7fb4a7000700 1 -- 172.21.15.46:0/3106363298 wait complete. 2024-05-08T23:02:56.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/373888718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:02:56.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/373888718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:02:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.744+0000 7f4ad1a00700 1 -- 172.21.15.46:0/544672790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc100ff0 msgr2=0x7f4acc101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.744+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/544672790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc100ff0 0x7f4acc101450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009990 tx=0x7f4ab8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 -- 172.21.15.46:0/544672790 shutdown_connections 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/544672790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc101990 0x7f4acc10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/544672790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc107030 0x7f4acc107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/544672790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc100ff0 0x7f4acc101450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 -- 172.21.15.46:0/544672790 >> 172.21.15.46:0/544672790 conn(0x7f4acc078190 msgr2=0x7f4acc0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 -- 172.21.15.46:0/544672790 shutdown_connections 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.745+0000 7f4ad1a00700 1 -- 172.21.15.46:0/544672790 wait complete. 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 Processor -- start 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 -- start start 2024-05-08T23:02:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc101990 0x7f4acc199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc107030 0x7f4acc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19e560 con 0x7f4acc100ff0 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19e6d0 con 0x7f4acc101990 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4ad1a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19e890 con 0x7f4acc107030 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.746+0000 7f4acaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44848/0 (socket says 172.21.15.46:44848) 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 -- 172.21.15.46:0/3501192985 learned_addr learned my addr 172.21.15.46:0/3501192985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acb400700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc107030 0x7f4acc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4aca000700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc101990 0x7f4acc199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 -- 172.21.15.46:0/3501192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc107030 msgr2=0x7f4acc19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc107030 0x7f4acc19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 -- 172.21.15.46:0/3501192985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc101990 msgr2=0x7f4acc199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc101990 0x7f4acc199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:56.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 -- 172.21.15.46:0/3501192985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4009c40 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4acaa00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00eb30 tx=0x7f4abc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc009880 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4abc0044e0 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.747+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc009520 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.748+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc003d80 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.748+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab80095f0 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.749+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc1026e0 con 0x7f4acc100ff0 2024-05-08T23:02:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.752+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4abc003f20 con 0x7f4acc100ff0 2024-05-08T23:02:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.754+0000 7f4ac3400700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 0x7f4ab00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.754+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4abc014070 con 0x7f4acc100ff0 2024-05-08T23:02:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.754+0000 7f4aca000700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 0x7f4ab00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.755+0000 7f4aca000700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 0x7f4ab00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4acc19abc0 tx=0x7f4ab8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:56.758+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4abc0964f0 con 0x7f4acc100ff0 2024-05-08T23:02:57.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.008+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4acc04ec60 con 0x7f4acc100ff0 2024-05-08T23:02:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.009+0000 7f4ac3400700 1 -- 172.21.15.46:0/3501192985 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4abc0041d0 con 0x7f4acc100ff0 2024-05-08T23:02:57.008 INFO:teuthology.orchestra.run.smithi046.stdout:103079215158 2024-05-08T23:02:57.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 msgr2=0x7f4ab00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 0x7f4ab00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4acc19abc0 tx=0x7f4ab8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 msgr2=0x7f4acc199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00eb30 tx=0x7f4abc00ee40 comp rx=0 tx=0).stop 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 shutdown_connections 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4acc107030 0x7f4acc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4ab00a1230 0x7f4ab00a36f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc100ff0 0x7f4acc199660 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 --2- 172.21.15.46:0/3501192985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc101990 0x7f4acc199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.011+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 >> 172.21.15.46:0/3501192985 conn(0x7f4acc078190 msgr2=0x7f4acc0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.012+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 shutdown_connections 2024-05-08T23:02:57.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.012+0000 7f4ad1a00700 1 -- 172.21.15.46:0/3501192985 wait complete. 2024-05-08T23:02:57.304 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3106363298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:02:57.304 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:57 smithi046 ceph-mon[36898]: pgmap v274: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-08T23:02:57.304 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3501192985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:02:57.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3106363298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:02:57.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:57 smithi086 ceph-mon[78610]: pgmap v274: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-08T23:02:57.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3501192985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:02:57.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3106363298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:02:57.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:57 smithi053 ceph-mon[77587]: pgmap v274: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-08T23:02:57.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3501192985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 -- 172.21.15.46:0/3396562452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4101990 msgr2=0x7fb8d410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3396562452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4101990 0x7fb8d410de70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009f90 tx=0x7fb8cc00b330 comp rx=0 tx=0).stop 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 -- 172.21.15.46:0/3396562452 shutdown_connections 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3396562452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4101990 0x7fb8d410de70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3396562452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4107030 0x7fb8d4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3396562452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4100ff0 0x7fb8d4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.646+0000 7fb8dc400700 1 -- 172.21.15.46:0/3396562452 >> 172.21.15.46:0/3396562452 conn(0x7fb8d4078190 msgr2=0x7fb8d40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.647+0000 7fb8dc400700 1 -- 172.21.15.46:0/3396562452 shutdown_connections 2024-05-08T23:02:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.647+0000 7fb8dc400700 1 -- 172.21.15.46:0/3396562452 wait complete. 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 Processor -- start 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 -- start start 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4100ff0 0x7fb8d4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4107030 0x7fb8d419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d419d580 con 0x7fb8d4107030 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d419d6f0 con 0x7fb8d4101990 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.649+0000 7fb8dc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d419d8b0 con 0x7fb8d4100ff0 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.650+0000 7fb8d9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.650+0000 7fb8da400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4107030 0x7fb8d419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.650+0000 7fb8d9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52144/0 (socket says 172.21.15.46:52144) 2024-05-08T23:02:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.650+0000 7fb8d9000700 1 -- 172.21.15.46:0/3456965120 learned_addr learned my addr 172.21.15.46:0/3456965120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.650+0000 7fb8d9a00700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4100ff0 0x7fb8d4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 -- 172.21.15.46:0/3456965120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4100ff0 msgr2=0x7fb8d4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4100ff0 0x7fb8d4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 -- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4107030 msgr2=0x7fb8d419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4107030 0x7fb8d419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 -- 172.21.15.46:0/3456965120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc009c40 con 0x7fb8d4101990 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.651+0000 7fb8d9000700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000e9c0 tx=0x7fb8c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.652+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0004a30 con 0x7fb8d4101990 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.652+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8c000cc80 con 0x7fb8d4101990 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.652+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0005440 con 0x7fb8d4101990 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.652+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d419db90 con 0x7fb8d4101990 2024-05-08T23:02:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.652+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d4107f80 con 0x7fb8d4101990 2024-05-08T23:02:57.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.654+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b40052f0 con 0x7fb8d4101990 2024-05-08T23:02:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.657+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb8c0005600 con 0x7fb8d4101990 2024-05-08T23:02:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.658+0000 7fb8d2000700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 0x7fb8bc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.659+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb8c0014070 con 0x7fb8d4101990 2024-05-08T23:02:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.659+0000 7fb8d9a00700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 0x7fb8bc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.660+0000 7fb8d9a00700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 0x7fb8bc0a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009be0 tx=0x7fb8c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.663+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8c0096090 con 0x7fb8d4101990 2024-05-08T23:02:57.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574910 got 55834574911 for osd.1 2024-05-08T23:02:57.888 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.906+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fb8b4005160 con 0x7fb8d4101990 2024-05-08T23:02:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.907+0000 7fb8d2000700 1 -- 172.21.15.46:0/3456965120 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb8c00996b0 con 0x7fb8d4101990 2024-05-08T23:02:57.906 INFO:teuthology.orchestra.run.smithi046.stdout:313532612631 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.909+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 msgr2=0x7fb8bc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.909+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 0x7fb8bc0a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009be0 tx=0x7fb8c8009260 comp rx=0 tx=0).stop 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.909+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 msgr2=0x7fb8d4199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.909+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000e9c0 tx=0x7fb8c000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 shutdown_connections 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fb8bc0a1230 0x7fb8bc0a36f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d4107030 0x7fb8d419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8d4100ff0 0x7fb8d4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 --2- 172.21.15.46:0/3456965120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8d4101990 0x7fb8d4199c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 >> 172.21.15.46:0/3456965120 conn(0x7fb8d4078190 msgr2=0x7fb8d40ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 shutdown_connections 2024-05-08T23:02:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.910+0000 7fb8dc400700 1 -- 172.21.15.46:0/3456965120 wait complete. 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 -- 172.21.15.46:0/2268077297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 msgr2=0x7f2ca4107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 --2- 172.21.15.46:0/2268077297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca4107400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c940099c0 tx=0x7f2c94009cd0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 -- 172.21.15.46:0/2268077297 shutdown_connections 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 --2- 172.21.15.46:0/2268077297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 0x7f2ca410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 --2- 172.21.15.46:0/2268077297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca4107400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 --2- 172.21.15.46:0/2268077297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4069a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 -- 172.21.15.46:0/2268077297 >> 172.21.15.46:0/2268077297 conn(0x7f2ca4076c50 msgr2=0x7f2ca4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 -- 172.21.15.46:0/2268077297 shutdown_connections 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.530+0000 7f2caa600700 1 -- 172.21.15.46:0/2268077297 wait complete. 2024-05-08T23:02:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.532+0000 7f2caa600700 1 Processor -- start 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.532+0000 7f2caa600700 1 -- start start 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.532+0000 7f2caa600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2caa600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 0x7f2ca4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2caa600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2caa600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca419c300 con 0x7f2ca4107020 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2caa600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca419c470 con 0x7f2ca4069fa0 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2caa600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca419c630 con 0x7f2ca40695e0 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59054/0 (socket says 172.21.15.46:59054) 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 -- 172.21.15.46:0/2543588206 learned_addr learned my addr 172.21.15.46:0/2543588206 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3e00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca2a00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 0x7f2ca4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 -- 172.21.15.46:0/2543588206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 msgr2=0x7f2ca4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 0x7f2ca4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 -- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 msgr2=0x7f2ca419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3400700 1 -- 172.21.15.46:0/2543588206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940095f0 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.533+0000 7f2ca3e00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2ca3400700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005b40 tx=0x7f2c9400ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9401d070 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c94004bd0 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9400fc10 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca419c8b0 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.534+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca41a3fa0 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.536+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca4109520 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.539+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2c9402b440 con 0x7f2ca40695e0 2024-05-08T23:02:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.541+0000 7f2c9be00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 0x7f2c880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.541+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2c940cdad0 con 0x7f2ca40695e0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.541+0000 7f2ca2a00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 0x7f2c880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.542+0000 7f2ca2a00700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 0x7f2c880a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4198960 tx=0x7f2c90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.544+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2c94096e20 con 0x7f2ca40695e0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.808+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2ca4198710 con 0x7f2ca40695e0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.808+0000 7f2c9be00700 1 -- 172.21.15.46:0/2543588206 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c9409a440 con 0x7f2ca40695e0 2024-05-08T23:02:57.944 INFO:teuthology.orchestra.run.smithi046.stdout:133143986227 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 msgr2=0x7f2c880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 0x7f2c880a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4198960 tx=0x7f2c90009260 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 msgr2=0x7f2ca4197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005b40 tx=0x7f2c9400ba70 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 shutdown_connections 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2c880a1230 0x7f2c880a36f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ca4107020 0x7f2ca419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca40695e0 0x7f2ca4197400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 --2- 172.21.15.46:0/2543588206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca4069fa0 0x7f2ca4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 >> 172.21.15.46:0/2543588206 conn(0x7f2ca4076c50 msgr2=0x7f2ca40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 shutdown_connections 2024-05-08T23:02:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.811+0000 7f2caa600700 1 -- 172.21.15.46:0/2543588206 wait complete. 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 -- 172.21.15.46:0/9337219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5098107040 msgr2=0x7f5098107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 --2- 172.21.15.46:0/9337219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5098107040 0x7f5098107420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50880098f0 tx=0x7f5088009c00 comp rx=0 tx=0).stop 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 -- 172.21.15.46:0/9337219 shutdown_connections 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 --2- 172.21.15.46:0/9337219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50981019e0 0x7f509810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 --2- 172.21.15.46:0/9337219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5098107040 0x7f5098107420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 --2- 172.21.15.46:0/9337219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f50981014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.841+0000 7f509ce00700 1 -- 172.21.15.46:0/9337219 >> 172.21.15.46:0/9337219 conn(0x7f5098078100 msgr2=0x7f5098078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.842+0000 7f509ce00700 1 -- 172.21.15.46:0/9337219 shutdown_connections 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.842+0000 7f509ce00700 1 -- 172.21.15.46:0/9337219 wait complete. 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 Processor -- start 2024-05-08T23:02:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 -- start start 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f5098199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 0x7f509819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e510 con 0x7f5098101020 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e680 con 0x7f5098107040 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f509ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e840 con 0x7f50981019e0 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f5095400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f5095400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59094/0 (socket says 172.21.15.46:59094) 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f5095400700 1 -- 172.21.15.46:0/1321336531 learned_addr learned my addr 172.21.15.46:0/1321336531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.843+0000 7f5095e00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f5098199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5096800700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 0x7f509819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 -- 172.21.15.46:0/1321336531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 msgr2=0x7f509819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 0x7f509819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 -- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 msgr2=0x7f5098199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f5098199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 -- 172.21.15.46:0/1321336531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009520 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5096800700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 0x7f509819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095e00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f5098199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.844+0000 7f5095400700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f508400e9c0 tx=0x7f508400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.845+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084004a30 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.845+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508400cc80 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.845+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509819eb20 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.845+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840054e0 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.845+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50981a6380 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.846+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5098102680 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.848+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5084004b90 con 0x7f50981019e0 2024-05-08T23:02:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.850+0000 7f508ea00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 0x7f507c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.851+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5084014070 con 0x7f50981019e0 2024-05-08T23:02:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.851+0000 7f5095e00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 0x7f507c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.851+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f50840962f0 con 0x7f50981019e0 2024-05-08T23:02:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:57.852+0000 7f5095e00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 0x7f507c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f50880098c0 tx=0x7f5088019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.111+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f509804ec60 con 0x7f50981019e0 2024-05-08T23:02:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.111+0000 7f508ea00700 1 -- 172.21.15.46:0/1321336531 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5084099910 con 0x7f50981019e0 2024-05-08T23:02:58.110 INFO:teuthology.orchestra.run.smithi046.stdout:231928234019 2024-05-08T23:02:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 msgr2=0x7f507c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 0x7f507c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f50880098c0 tx=0x7f5088019040 comp rx=0 tx=0).stop 2024-05-08T23:02:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 msgr2=0x7f5098199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f508400e9c0 tx=0x7f508400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 shutdown_connections 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5098107040 0x7f509819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f507c0a1230 0x7f507c0a36f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5098101020 0x7f5098199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 --2- 172.21.15.46:0/1321336531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50981019e0 0x7f5098199b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 >> 172.21.15.46:0/1321336531 conn(0x7f5098078100 msgr2=0x7f509810c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 shutdown_connections 2024-05-08T23:02:58.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.114+0000 7f509ce00700 1 -- 172.21.15.46:0/1321336531 wait complete. 2024-05-08T23:02:58.236 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2543588206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:02:58.237 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3456965120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:02:58.237 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1321336531' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:02:58.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215156 got 103079215158 for osd.3 2024-05-08T23:02:58.258 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:58.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2543588206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:02:58.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3456965120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:02:58.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1321336531' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:02:58.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2543588206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:02:58.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3456965120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:02:58.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1321336531' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:02:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.767+0000 7f77f6600700 1 -- 172.21.15.46:0/4181339576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 msgr2=0x7f77f0069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.767+0000 7f77f6600700 1 --2- 172.21.15.46:0/4181339576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f0069310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009990 tx=0x7f77dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 -- 172.21.15.46:0/4181339576 shutdown_connections 2024-05-08T23:02:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 --2- 172.21.15.46:0/4181339576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f01039d0 0x7f77f0103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 --2- 172.21.15.46:0/4181339576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0105ec0 0x7f77f0068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 --2- 172.21.15.46:0/4181339576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f0069310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 -- 172.21.15.46:0/4181339576 >> 172.21.15.46:0/4181339576 conn(0x7f77f0075f80 msgr2=0x7f77f0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 -- 172.21.15.46:0/4181339576 shutdown_connections 2024-05-08T23:02:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.768+0000 7f77f6600700 1 -- 172.21.15.46:0/4181339576 wait complete. 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 Processor -- start 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 -- start start 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f01a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f0105ec0 0x7f77f019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f019de80 con 0x7f77f0068e90 2024-05-08T23:02:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f019dff0 con 0x7f77f0105ec0 2024-05-08T23:02:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.770+0000 7f77f6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f019e1b0 con 0x7f77f01039d0 2024-05-08T23:02:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77ef400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f01a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59108/0 (socket says 172.21.15.46:59108) 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 -- 172.21.15.46:0/565878397 learned_addr learned my addr 172.21.15.46:0/565878397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77efe00700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f0105ec0 0x7f77f019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 -- 172.21.15.46:0/565878397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f0105ec0 msgr2=0x7f77f019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f0105ec0 0x7f77f019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 -- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 msgr2=0x7f77f01a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f01a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.771+0000 7f77eea00700 1 -- 172.21.15.46:0/565878397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0095f0 con 0x7f77f01039d0 2024-05-08T23:02:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77eea00700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005770 tx=0x7f77dc0057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc01d070 con 0x7f77f01039d0 2024-05-08T23:02:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77dc021410 con 0x7f77f01039d0 2024-05-08T23:02:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f019e430 con 0x7f77f01039d0 2024-05-08T23:02:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f01aa6c0 con 0x7f77f01039d0 2024-05-08T23:02:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.772+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017650 con 0x7f77f01039d0 2024-05-08T23:02:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.774+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f01004b0 con 0x7f77f01039d0 2024-05-08T23:02:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.775+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f77dc0217f0 con 0x7f77f01039d0 2024-05-08T23:02:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.776+0000 7f77e7e00700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 0x7f77d40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.777+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77dc0cdf50 con 0x7f77f01039d0 2024-05-08T23:02:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.777+0000 7f77ef400700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 0x7f77d40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.778+0000 7f77ef400700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 0x7f77d40a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009be0 tx=0x7f77e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.781+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f77dc0972a0 con 0x7f77f01039d0 2024-05-08T23:02:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 -- 172.21.15.46:0/736550682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e4105100 msgr2=0x7f59e4105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 --2- 172.21.15.46:0/736550682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e4105100 0x7f59e4105580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b0a0 tx=0x7f59dc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 -- 172.21.15.46:0/736550682 shutdown_connections 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 --2- 172.21.15.46:0/736550682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e4105100 0x7f59e4105580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 --2- 172.21.15.46:0/736550682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e40ffb00 0x7f59e40ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 --2- 172.21.15.46:0/736550682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 0x7f59e4104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 -- 172.21.15.46:0/736550682 >> 172.21.15.46:0/736550682 conn(0x7f59e40fb7e0 msgr2=0x7f59e40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 -- 172.21.15.46:0/736550682 shutdown_connections 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.869+0000 7f59ea000700 1 -- 172.21.15.46:0/736550682 wait complete. 2024-05-08T23:02:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.871+0000 7f59ea000700 1 Processor -- start 2024-05-08T23:02:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.871+0000 7f59ea000700 1 -- start start 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 0x7f59e419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4105100 0x7f59e41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a2900 con 0x7f59e40ffb00 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a2a70 con 0x7f59e4100420 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59ea000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a2c30 con 0x7f59e4105100 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59e8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59e8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44938/0 (socket says 172.21.15.46:44938) 2024-05-08T23:02:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59e8c00700 1 -- 172.21.15.46:0/2354381255 learned_addr learned my addr 172.21.15.46:0/2354381255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59e3e00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 0x7f59e419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.872+0000 7f59e9600700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4105100 0x7f59e41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 -- 172.21.15.46:0/2354381255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4105100 msgr2=0x7f59e41a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4105100 0x7f59e41a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 -- 172.21.15.46:0/2354381255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 msgr2=0x7f59e419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 0x7f59e419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 -- 172.21.15.46:0/2354381255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0009520 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e8c00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9a0 tx=0x7f59d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8009790 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59d8010430 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc009d00 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e41aa5e0 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.873+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8018680 con 0x7f59e40ffb00 2024-05-08T23:02:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.875+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e4069230 con 0x7f59e40ffb00 2024-05-08T23:02:58.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.877+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f59d80105a0 con 0x7f59e40ffb00 2024-05-08T23:02:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.878+0000 7f59e1600700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 0x7f59cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.878+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f59d8014070 con 0x7f59e40ffb00 2024-05-08T23:02:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.878+0000 7f59e3e00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 0x7f59cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.879+0000 7f59e3e00700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 0x7f59cc0a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59e419ef70 tx=0x7f59d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:58.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:58.882+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f59d8052d60 con 0x7f59e40ffb00 2024-05-08T23:02:59.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612629 got 313532612631 for osd.11 2024-05-08T23:02:59.000 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.029+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77f019ecd0 con 0x7f77f01039d0 2024-05-08T23:02:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.030+0000 7f77e7e00700 1 -- 172.21.15.46:0/565878397 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77dc09a8c0 con 0x7f77f01039d0 2024-05-08T23:02:59.029 INFO:teuthology.orchestra.run.smithi046.stdout:34359738434 2024-05-08T23:02:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 msgr2=0x7f77d40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 0x7f77d40a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009be0 tx=0x7f77e0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 msgr2=0x7f77f01a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005770 tx=0x7f77dc0057a0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 shutdown_connections 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77f0105ec0 0x7f77f019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f77d40a1230 0x7f77d40a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f0068e90 0x7f77f01a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 --2- 172.21.15.46:0/565878397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f01039d0 0x7f77f01a3c10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.032+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 >> 172.21.15.46:0/565878397 conn(0x7f77f0075f80 msgr2=0x7f77f01029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.033+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 shutdown_connections 2024-05-08T23:02:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.033+0000 7f77f6600700 1 -- 172.21.15.46:0/565878397 wait complete. 2024-05-08T23:02:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.119+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f59e419ed20 con 0x7f59e40ffb00 2024-05-08T23:02:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.120+0000 7f59e1600700 1 -- 172.21.15.46:0/2354381255 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f59d8096260 con 0x7f59e40ffb00 2024-05-08T23:02:59.119 INFO:teuthology.orchestra.run.smithi046.stdout:283467841562 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.122+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 msgr2=0x7f59cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.122+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 0x7f59cc0a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59e419ef70 tx=0x7f59d0009260 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.122+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 msgr2=0x7f59e419da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.122+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9a0 tx=0x7f59d800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 shutdown_connections 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4105100 0x7f59e41a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f59cc0a11b0 0x7f59cc0a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59e40ffb00 0x7f59e419da10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 --2- 172.21.15.46:0/2354381255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e4100420 0x7f59e419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 >> 172.21.15.46:0/2354381255 conn(0x7f59e40fb7e0 msgr2=0x7f59e4102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 shutdown_connections 2024-05-08T23:02:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.123+0000 7f59ea000700 1 -- 172.21.15.46:0/2354381255 wait complete. 2024-05-08T23:02:59.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234017 got 231928234019 for osd.8 2024-05-08T23:02:59.403 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:59.710 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738434 for osd.0 2024-05-08T23:02:59.710 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:59.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986224 got 133143986227 for osd.4 2024-05-08T23:02:59.916 DEBUG:teuthology.parallel:result is None 2024-05-08T23:02:59.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 -- 172.21.15.46:0/2754835000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 msgr2=0x7f30ec10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 --2- 172.21.15.46:0/2754835000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec10de60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b0a0 tx=0x7f30e400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 -- 172.21.15.46:0/2754835000 shutdown_connections 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 --2- 172.21.15.46:0/2754835000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec10de60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 --2- 172.21.15.46:0/2754835000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 --2- 172.21.15.46:0/2754835000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 0x7f30ec068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 -- 172.21.15.46:0/2754835000 >> 172.21.15.46:0/2754835000 conn(0x7f30ec075f80 msgr2=0x7f30ec076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 -- 172.21.15.46:0/2754835000 shutdown_connections 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.709+0000 7f30f3800700 1 -- 172.21.15.46:0/2754835000 wait complete. 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.710+0000 7f30f3800700 1 Processor -- start 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 -- start start 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 0x7f30ec19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec199cb0 con 0x7f30ec0692f0 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec199e20 con 0x7f30ec068950 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec199fe0 con 0x7f30ec107020 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30ebe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30f0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 0x7f30ec19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30ebe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec19fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44954/0 (socket says 172.21.15.46:44954) 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.711+0000 7f30ebe00700 1 -- 172.21.15.46:0/2653264210 learned_addr learned my addr 172.21.15.46:0/2653264210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 -- 172.21.15.46:0/2653264210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 msgr2=0x7f30ec19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 0x7f30ec19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 -- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 msgr2=0x7f30ec19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 -- 172.21.15.46:0/2653264210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4009d00 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f1800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec199650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b070 tx=0x7f30e4007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e400e040 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30e401d470 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec19a260 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e40052a0 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.712+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec1a6500 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.713+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec1026d0 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.715+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f30e401d5e0 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.717+0000 7f30e9600700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 0x7f30d40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.717+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f30e40ce390 con 0x7f30ec107020 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.717+0000 7f30f0e00700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 0x7f30d40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:02:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.718+0000 7f30f0e00700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 0x7f30d40a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009a10 tx=0x7f30e0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.719+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f30e4097740 con 0x7f30ec107020 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.940+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f30ec04ec60 con 0x7f30ec107020 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.941+0000 7f30e9600700 1 -- 172.21.15.46:0/2653264210 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30e409ad60 con 0x7f30ec107020 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stdout:201863462951 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 msgr2=0x7f30d40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 0x7f30d40a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009a10 tx=0x7f30e0009190 comp rx=0 tx=0).stop 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 msgr2=0x7f30ec199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:02:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec199650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b070 tx=0x7f30e4007930 comp rx=0 tx=0).stop 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 shutdown_connections 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30ec107020 0x7f30ec199650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec068950 0x7f30ec19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f30d40a11b0 0x7f30d40a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 --2- 172.21.15.46:0/2653264210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ec0692f0 0x7f30ec19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 >> 172.21.15.46:0/2653264210 conn(0x7f30ec075f80 msgr2=0x7f30ec10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 shutdown_connections 2024-05-08T23:02:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:02:59.944+0000 7f30f3800700 1 -- 172.21.15.46:0/2653264210 wait complete. 2024-05-08T23:02:59.966 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:59 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:59.966 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:59 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:02:59.966 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:59 smithi046 ceph-mon[36898]: pgmap v275: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:02:59.966 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/565878397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:02:59.967 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:02:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2354381255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:03:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:59 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:00.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:59 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:00.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:59 smithi053 ceph-mon[77587]: pgmap v275: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:03:00.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/565878397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:03:00.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:02:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2354381255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:03:00.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841560 got 283467841562 for osd.10 2024-05-08T23:03:00.049 DEBUG:teuthology.parallel:result is None 2024-05-08T23:03:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:59 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:59 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:59 smithi086 ceph-mon[78610]: pgmap v275: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:03:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/565878397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:03:00.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:02:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2354381255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:03:00.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462949 got 201863462951 for osd.7 2024-05-08T23:03:00.378 DEBUG:teuthology.parallel:result is None 2024-05-08T23:03:00.378 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T23:03:00.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:03:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:00 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2653264210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:03:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:00 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:00 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:00 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:00 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2653264210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:03:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:00 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:00 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:00 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:00 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2653264210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:03:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:00 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:00 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:00 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 -- 172.21.15.46:0/2899168958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 msgr2=0x7feb5010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 --2- 172.21.15.46:0/2899168958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb5010de70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b0a0 tx=0x7feb4800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 -- 172.21.15.46:0/2899168958 shutdown_connections 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 --2- 172.21.15.46:0/2899168958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb5010de70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 --2- 172.21.15.46:0/2899168958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 0x7feb50107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 --2- 172.21.15.46:0/2899168958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb50101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 -- 172.21.15.46:0/2899168958 >> 172.21.15.46:0/2899168958 conn(0x7feb50078190 msgr2=0x7feb500785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 -- 172.21.15.46:0/2899168958 shutdown_connections 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.995+0000 7feb55800700 1 -- 172.21.15.46:0/2899168958 wait complete. 2024-05-08T23:03:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.996+0000 7feb55800700 1 Processor -- start 2024-05-08T23:03:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.996+0000 7feb55800700 1 -- start start 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.996+0000 7feb55800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.996+0000 7feb55800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb50199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.996+0000 7feb55800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 0x7feb5019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb55800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5019d5e0 con 0x7feb50101990 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb55800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5019d750 con 0x7feb50107030 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb55800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5019d910 con 0x7feb50100ff0 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb50199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35824/0 (socket says 172.21.15.46:35824) 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 -- 172.21.15.46:0/184760762 learned_addr learned my addr 172.21.15.46:0/184760762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4f400700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 0x7feb5019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 -- 172.21.15.46:0/184760762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 msgr2=0x7feb5019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 0x7feb5019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 -- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 msgr2=0x7feb50199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb50199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.997+0000 7feb4ea00700 1 -- 172.21.15.46:0/184760762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009d00 con 0x7feb50100ff0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb4ea00700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb4000e9a0 tx=0x7feb4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40009790 con 0x7feb50100ff0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb40010430 con 0x7feb50100ff0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5019dbf0 con 0x7feb50100ff0 2024-05-08T23:03:01.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40018680 con 0x7feb50100ff0 2024-05-08T23:03:01.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.998+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb501a6540 con 0x7feb50100ff0 2024-05-08T23:03:01.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:01.999+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb50104370 con 0x7feb50100ff0 2024-05-08T23:03:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.001+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7feb400105a0 con 0x7feb50100ff0 2024-05-08T23:03:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.002+0000 7feb47400700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 0x7feb340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.002+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb40014070 con 0x7feb50100ff0 2024-05-08T23:03:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.002+0000 7feb4e000700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 0x7feb340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.003+0000 7feb4e000700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 0x7feb340a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb5019ad00 tx=0x7feb3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.005+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb400961e0 con 0x7feb50100ff0 2024-05-08T23:03:02.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:01 smithi053 ceph-mon[77587]: pgmap v276: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:02.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:01 smithi046 ceph-mon[36898]: pgmap v276: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:01 smithi086 ceph-mon[78610]: pgmap v276: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.215+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb5019aab0 con 0x7feb340a11b0 2024-05-08T23:03:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.230+0000 7feb47400700 1 -- 172.21.15.46:0/184760762 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426450 (secure 0 0 0) 0x7feb5019aab0 con 0x7feb340a11b0 2024-05-08T23:03:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 msgr2=0x7feb340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 0x7feb340a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb5019ad00 tx=0x7feb3c009260 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 msgr2=0x7feb501997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb4000e9a0 tx=0x7feb4000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 shutdown_connections 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb50107030 0x7feb5019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb50100ff0 0x7feb501997a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7feb340a11b0 0x7feb340a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 --2- 172.21.15.46:0/184760762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb50101990 0x7feb50199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 >> 172.21.15.46:0/184760762 conn(0x7feb50078190 msgr2=0x7feb5010c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 shutdown_connections 2024-05-08T23:03:02.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:02.235+0000 7feb55800700 1 -- 172.21.15.46:0/184760762 wait complete. 2024-05-08T23:03:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:03:02.657 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-05-08T23:03:00.935467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658673,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":564,"num_read_kb":407,"num_write":419,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372488,"kb_used_data":11396,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569336,"statfs":{"total":1151940427776,"available":1151559000064,"internally_reserved":0,"allocated":11669504,"data_stored":7538763,"data_compressed":324360,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":53,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003683"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661165+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:02:17.661165+0000","last_peered":"2024-05-08T23:02:17.661165+0000","last_clean":"2024-05-08T23:02:17.661165+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:02:17.661165+0000","last_undegraded":"2024-05-08T23:02:17.661165+0000","last_fullsized":"2024-05-08T23:02:17.661165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661559+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:02:17.661559+0000","last_peered":"2024-05-08T23:02:17.661559+0000","last_clean":"2024-05-08T23:02:17.661559+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:02:17.661559+0000","last_undegraded":"2024-05-08T23:02:17.661559+0000","last_fullsized":"2024-05-08T23:02:17.661559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661196+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:02:17.661196+0000","last_peered":"2024-05-08T23:02:17.661196+0000","last_clean":"2024-05-08T23:02:17.661196+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:02:17.661196+0000","last_undegraded":"2024-05-08T23:02:17.661196+0000","last_fullsized":"2024-05-08T23:02:17.661196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654527+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:02:17.654527+0000","last_peered":"2024-05-08T23:02:17.654527+0000","last_clean":"2024-05-08T23:02:17.654527+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:02:17.654527+0000","last_undegraded":"2024-05-08T23:02:17.654527+0000","last_fullsized":"2024-05-08T23:02:17.654527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650061+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:02:17.650061+0000","last_peered":"2024-05-08T23:02:17.650061+0000","last_clean":"2024-05-08T23:02:17.650061+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:02:17.650061+0000","last_undegraded":"2024-05-08T23:02:17.650061+0000","last_fullsized":"2024-05-08T23:02:17.650061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660268+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:02:17.660268+0000","last_peered":"2024-05-08T23:02:17.660268+0000","last_clean":"2024-05-08T23:02:17.660268+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:02:17.660268+0000","last_undegraded":"2024-05-08T23:02:17.660268+0000","last_fullsized":"2024-05-08T23:02:17.660268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654543+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:02:17.654543+0000","last_peered":"2024-05-08T23:02:17.654543+0000","last_clean":"2024-05-08T23:02:17.654543+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:02:17.654543+0000","last_undegraded":"2024-05-08T23:02:17.654543+0000","last_fullsized":"2024-05-08T23:02:17.654543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656492+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:02:17.656492+0000","last_peered":"2024-05-08T23:02:17.656492+0000","last_clean":"2024-05-08T23:02:17.656492+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:02:17.656492+0000","last_undegraded":"2024-05-08T23:02:17.656492+0000","last_fullsized":"2024-05-08T23:02:17.656492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661236+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:02:17.661236+0000","last_peered":"2024-05-08T23:02:17.661236+0000","last_clean":"2024-05-08T23:02:17.661236+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:02:17.661236+0000","last_undegraded":"2024-05-08T23:02:17.661236+0000","last_fullsized":"2024-05-08T23:02:17.661236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661078+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:02:17.661078+0000","last_peered":"2024-05-08T23:02:17.661078+0000","last_clean":"2024-05-08T23:02:17.661078+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:02:17.661078+0000","last_undegraded":"2024-05-08T23:02:17.661078+0000","last_fullsized":"2024-05-08T23:02:17.661078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659040+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:02:17.659040+0000","last_peered":"2024-05-08T23:02:17.659040+0000","last_clean":"2024-05-08T23:02:17.659040+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:02:17.659040+0000","last_undegraded":"2024-05-08T23:02:17.659040+0000","last_fullsized":"2024-05-08T23:02:17.659040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661153+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:02:17.661153+0000","last_peered":"2024-05-08T23:02:17.661153+0000","last_clean":"2024-05-08T23:02:17.661153+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:02:17.661153+0000","last_undegraded":"2024-05-08T23:02:17.661153+0000","last_fullsized":"2024-05-08T23:02:17.661153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656514+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:02:17.656514+0000","last_peered":"2024-05-08T23:02:17.656514+0000","last_clean":"2024-05-08T23:02:17.656514+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:02:17.656514+0000","last_undegraded":"2024-05-08T23:02:17.656514+0000","last_fullsized":"2024-05-08T23:02:17.656514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655453+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:02:17.655453+0000","last_peered":"2024-05-08T23:02:17.655453+0000","last_clean":"2024-05-08T23:02:17.655453+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:02:17.655453+0000","last_undegraded":"2024-05-08T23:02:17.655453+0000","last_fullsized":"2024-05-08T23:02:17.655453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654795+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:02:17.654795+0000","last_peered":"2024-05-08T23:02:17.654795+0000","last_clean":"2024-05-08T23:02:17.654795+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:02:17.654795+0000","last_undegraded":"2024-05-08T23:02:17.654795+0000","last_fullsized":"2024-05-08T23:02:17.654795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":90,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:54.691835+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:02:54.691835+0000","last_peered":"2024-05-08T23:02:54.691835+0000","last_clean":"2024-05-08T23:02:54.691835+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:02:54.691835+0000","last_undegraded":"2024-05-08T23:02:54.691835+0000","last_fullsized":"2024-05-08T23:02:54.691835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651671+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:02:17.651671+0000","last_peered":"2024-05-08T23:02:17.651671+0000","last_clean":"2024-05-08T23:02:17.651671+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:02:17.651671+0000","last_undegraded":"2024-05-08T23:02:17.651671+0000","last_fullsized":"2024-05-08T23:02:17.651671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653802+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:02:17.653802+0000","last_peered":"2024-05-08T23:02:17.653802+0000","last_clean":"2024-05-08T23:02:17.653802+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:02:17.653802+0000","last_undegraded":"2024-05-08T23:02:17.653802+0000","last_fullsized":"2024-05-08T23:02:17.653802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651158+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:17.651158+0000","last_peered":"2024-05-08T23:02:17.651158+0000","last_clean":"2024-05-08T23:02:17.651158+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:17.651158+0000","last_undegraded":"2024-05-08T23:02:17.651158+0000","last_fullsized":"2024-05-08T23:02:17.651158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661525+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:02:17.661525+0000","last_peered":"2024-05-08T23:02:17.661525+0000","last_clean":"2024-05-08T23:02:17.661525+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:02:17.661525+0000","last_undegraded":"2024-05-08T23:02:17.661525+0000","last_fullsized":"2024-05-08T23:02:17.661525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653999+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:02:17.653999+0000","last_peered":"2024-05-08T23:02:17.653999+0000","last_clean":"2024-05-08T23:02:17.653999+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:02:17.653999+0000","last_undegraded":"2024-05-08T23:02:17.653999+0000","last_fullsized":"2024-05-08T23:02:17.653999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636260+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:19.636260+0000","last_peered":"2024-05-08T23:02:19.636260+0000","last_clean":"2024-05-08T23:02:19.636260+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:19.636260+0000","last_undegraded":"2024-05-08T23:02:19.636260+0000","last_fullsized":"2024-05-08T23:02:19.636260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659474+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:02:17.659474+0000","last_peered":"2024-05-08T23:02:17.659474+0000","last_clean":"2024-05-08T23:02:17.659474+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:02:17.659474+0000","last_undegraded":"2024-05-08T23:02:17.659474+0000","last_fullsized":"2024-05-08T23:02:17.659474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653902+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:02:17.653902+0000","last_peered":"2024-05-08T23:02:17.653902+0000","last_clean":"2024-05-08T23:02:17.653902+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:02:17.653902+0000","last_undegraded":"2024-05-08T23:02:17.653902+0000","last_fullsized":"2024-05-08T23:02:17.653902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655207+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:18.655207+0000","last_peered":"2024-05-08T23:02:18.655207+0000","last_clean":"2024-05-08T23:02:18.655207+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:18.655207+0000","last_undegraded":"2024-05-08T23:02:18.655207+0000","last_fullsized":"2024-05-08T23:02:18.655207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655226+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:02:17.655226+0000","last_peered":"2024-05-08T23:02:17.655226+0000","last_clean":"2024-05-08T23:02:17.655226+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:02:17.655226+0000","last_undegraded":"2024-05-08T23:02:17.655226+0000","last_fullsized":"2024-05-08T23:02:17.655226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.699074+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:58.699074+0000","last_peered":"2024-05-08T23:02:58.699074+0000","last_clean":"2024-05-08T23:02:58.699074+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:58.699074+0000","last_undegraded":"2024-05-08T23:02:58.699074+0000","last_fullsized":"2024-05-08T23:02:58.699074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657534+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:02:17.657534+0000","last_peered":"2024-05-08T23:02:17.657534+0000","last_clean":"2024-05-08T23:02:17.657534+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:02:17.657534+0000","last_undegraded":"2024-05-08T23:02:17.657534+0000","last_fullsized":"2024-05-08T23:02:17.657534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661059+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:02:17.661059+0000","last_peered":"2024-05-08T23:02:17.661059+0000","last_clean":"2024-05-08T23:02:17.661059+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:02:17.661059+0000","last_undegraded":"2024-05-08T23:02:17.661059+0000","last_fullsized":"2024-05-08T23:02:17.661059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674762+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:17.674762+0000","last_peered":"2024-05-08T23:02:17.674762+0000","last_clean":"2024-05-08T23:02:17.674762+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:17.674762+0000","last_undegraded":"2024-05-08T23:02:17.674762+0000","last_fullsized":"2024-05-08T23:02:17.674762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649951+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:02:17.649951+0000","last_peered":"2024-05-08T23:02:17.649951+0000","last_clean":"2024-05-08T23:02:17.649951+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:02:17.649951+0000","last_undegraded":"2024-05-08T23:02:17.649951+0000","last_fullsized":"2024-05-08T23:02:17.649951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661283+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:02:17.661283+0000","last_peered":"2024-05-08T23:02:17.661283+0000","last_clean":"2024-05-08T23:02:17.661283+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:02:17.661283+0000","last_undegraded":"2024-05-08T23:02:17.661283+0000","last_fullsized":"2024-05-08T23:02:17.661283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655141+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:02:17.655141+0000","last_peered":"2024-05-08T23:02:17.655141+0000","last_clean":"2024-05-08T23:02:17.655141+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:02:17.655141+0000","last_undegraded":"2024-05-08T23:02:17.655141+0000","last_fullsized":"2024-05-08T23:02:17.655141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651327+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:02:17.651327+0000","last_peered":"2024-05-08T23:02:17.651327+0000","last_clean":"2024-05-08T23:02:17.651327+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:02:17.651327+0000","last_undegraded":"2024-05-08T23:02:17.651327+0000","last_fullsized":"2024-05-08T23:02:17.651327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.695684+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:58.695684+0000","last_peered":"2024-05-08T23:02:58.695684+0000","last_clean":"2024-05-08T23:02:58.695684+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:58.695684+0000","last_undegraded":"2024-05-08T23:02:58.695684+0000","last_fullsized":"2024-05-08T23:02:58.695684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660300+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:02:17.660300+0000","last_peered":"2024-05-08T23:02:17.660300+0000","last_clean":"2024-05-08T23:02:17.660300+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:02:17.660300+0000","last_undegraded":"2024-05-08T23:02:17.660300+0000","last_fullsized":"2024-05-08T23:02:17.660300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654915+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:02:17.654915+0000","last_peered":"2024-05-08T23:02:17.654915+0000","last_clean":"2024-05-08T23:02:17.654915+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:02:17.654915+0000","last_undegraded":"2024-05-08T23:02:17.654915+0000","last_fullsized":"2024-05-08T23:02:17.654915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661428+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:02:17.661428+0000","last_peered":"2024-05-08T23:02:17.661428+0000","last_clean":"2024-05-08T23:02:17.661428+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:02:17.661428+0000","last_undegraded":"2024-05-08T23:02:17.661428+0000","last_fullsized":"2024-05-08T23:02:17.661428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654337+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:02:17.654337+0000","last_peered":"2024-05-08T23:02:17.654337+0000","last_clean":"2024-05-08T23:02:17.654337+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:02:17.654337+0000","last_undegraded":"2024-05-08T23:02:17.654337+0000","last_fullsized":"2024-05-08T23:02:17.654337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659646+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:02:17.659646+0000","last_peered":"2024-05-08T23:02:17.659646+0000","last_clean":"2024-05-08T23:02:17.659646+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:02:17.659646+0000","last_undegraded":"2024-05-08T23:02:17.659646+0000","last_fullsized":"2024-05-08T23:02:17.659646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654901+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:02:17.654901+0000","last_peered":"2024-05-08T23:02:17.654901+0000","last_clean":"2024-05-08T23:02:17.654901+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:02:17.654901+0000","last_undegraded":"2024-05-08T23:02:17.654901+0000","last_fullsized":"2024-05-08T23:02:17.654901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":137,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.695654+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:58.695654+0000","last_peered":"2024-05-08T23:02:58.695654+0000","last_clean":"2024-05-08T23:02:58.695654+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:58.695654+0000","last_undegraded":"2024-05-08T23:02:58.695654+0000","last_fullsized":"2024-05-08T23:02:58.695654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61448,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":103,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636478+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:19.636478+0000","last_peered":"2024-05-08T23:02:19.636478+0000","last_clean":"2024-05-08T23:02:19.636478+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:19.636478+0000","last_undegraded":"2024-05-08T23:02:19.636478+0000","last_fullsized":"2024-05-08T23:02:19.636478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655806+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:02:17.655806+0000","last_peered":"2024-05-08T23:02:17.655806+0000","last_clean":"2024-05-08T23:02:17.655806+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:02:17.655806+0000","last_undegraded":"2024-05-08T23:02:17.655806+0000","last_fullsized":"2024-05-08T23:02:17.655806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659759+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:02:17.659759+0000","last_peered":"2024-05-08T23:02:17.659759+0000","last_clean":"2024-05-08T23:02:17.659759+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:02:17.659759+0000","last_undegraded":"2024-05-08T23:02:17.659759+0000","last_fullsized":"2024-05-08T23:02:17.659759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656023+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:02:17.656023+0000","last_peered":"2024-05-08T23:02:17.656023+0000","last_clean":"2024-05-08T23:02:17.656023+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:02:17.656023+0000","last_undegraded":"2024-05-08T23:02:17.656023+0000","last_fullsized":"2024-05-08T23:02:17.656023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:55.695583+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:02:55.695583+0000","last_peered":"2024-05-08T23:02:55.695583+0000","last_clean":"2024-05-08T23:02:55.695583+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:02:55.695583+0000","last_undegraded":"2024-05-08T23:02:55.695583+0000","last_fullsized":"2024-05-08T23:02:55.695583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657551+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:02:17.657551+0000","last_peered":"2024-05-08T23:02:17.657551+0000","last_clean":"2024-05-08T23:02:17.657551+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:02:17.657551+0000","last_undegraded":"2024-05-08T23:02:17.657551+0000","last_fullsized":"2024-05-08T23:02:17.657551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656051+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:18.656051+0000","last_peered":"2024-05-08T23:02:18.656051+0000","last_clean":"2024-05-08T23:02:18.656051+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:18.656051+0000","last_undegraded":"2024-05-08T23:02:18.656051+0000","last_fullsized":"2024-05-08T23:02:18.656051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.670834+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:02:18.670834+0000","last_peered":"2024-05-08T23:02:18.670834+0000","last_clean":"2024-05-08T23:02:18.670834+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:02:18.670834+0000","last_undegraded":"2024-05-08T23:02:18.670834+0000","last_fullsized":"2024-05-08T23:02:18.670834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.663365+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:02:18.663365+0000","last_peered":"2024-05-08T23:02:18.663365+0000","last_clean":"2024-05-08T23:02:18.663365+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:02:18.663365+0000","last_undegraded":"2024-05-08T23:02:18.663365+0000","last_fullsized":"2024-05-08T23:02:18.663365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655364+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:02:17.655364+0000","last_peered":"2024-05-08T23:02:17.655364+0000","last_clean":"2024-05-08T23:02:17.655364+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:02:17.655364+0000","last_undegraded":"2024-05-08T23:02:17.655364+0000","last_fullsized":"2024-05-08T23:02:17.655364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656734+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:02:17.656734+0000","last_peered":"2024-05-08T23:02:17.656734+0000","last_clean":"2024-05-08T23:02:17.656734+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:02:17.656734+0000","last_undegraded":"2024-05-08T23:02:17.656734+0000","last_fullsized":"2024-05-08T23:02:17.656734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:57.695998+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:57.695998+0000","last_peered":"2024-05-08T23:02:57.695998+0000","last_clean":"2024-05-08T23:02:57.695998+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:57.695998+0000","last_undegraded":"2024-05-08T23:02:57.695998+0000","last_fullsized":"2024-05-08T23:02:57.695998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657532+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:02:17.657532+0000","last_peered":"2024-05-08T23:02:17.657532+0000","last_clean":"2024-05-08T23:02:17.657532+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:02:17.657532+0000","last_undegraded":"2024-05-08T23:02:17.657532+0000","last_fullsized":"2024-05-08T23:02:17.657532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:55.695101+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:02:55.695101+0000","last_peered":"2024-05-08T23:02:55.695101+0000","last_clean":"2024-05-08T23:02:55.695101+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:02:55.695101+0000","last_undegraded":"2024-05-08T23:02:55.695101+0000","last_fullsized":"2024-05-08T23:02:55.695101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650153+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:02:17.650153+0000","last_peered":"2024-05-08T23:02:17.650153+0000","last_clean":"2024-05-08T23:02:17.650153+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:02:17.650153+0000","last_undegraded":"2024-05-08T23:02:17.650153+0000","last_fullsized":"2024-05-08T23:02:17.650153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.639109+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:19.639109+0000","last_peered":"2024-05-08T23:02:19.639109+0000","last_clean":"2024-05-08T23:02:19.639109+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:19.639109+0000","last_undegraded":"2024-05-08T23:02:19.639109+0000","last_fullsized":"2024-05-08T23:02:19.639109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654454+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:02:17.654454+0000","last_peered":"2024-05-08T23:02:17.654454+0000","last_clean":"2024-05-08T23:02:17.654454+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:02:17.654454+0000","last_undegraded":"2024-05-08T23:02:17.654454+0000","last_fullsized":"2024-05-08T23:02:17.654454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667675+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:02:18.667675+0000","last_peered":"2024-05-08T23:02:18.667675+0000","last_clean":"2024-05-08T23:02:18.667675+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:02:18.667675+0000","last_undegraded":"2024-05-08T23:02:18.667675+0000","last_fullsized":"2024-05-08T23:02:18.667675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659637+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:02:17.659637+0000","last_peered":"2024-05-08T23:02:17.659637+0000","last_clean":"2024-05-08T23:02:17.659637+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:02:17.659637+0000","last_undegraded":"2024-05-08T23:02:17.659637+0000","last_fullsized":"2024-05-08T23:02:17.659637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655573+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:18.655573+0000","last_peered":"2024-05-08T23:02:18.655573+0000","last_clean":"2024-05-08T23:02:18.655573+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:18.655573+0000","last_undegraded":"2024-05-08T23:02:18.655573+0000","last_fullsized":"2024-05-08T23:02:18.655573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656812+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:02:17.656812+0000","last_peered":"2024-05-08T23:02:17.656812+0000","last_clean":"2024-05-08T23:02:17.656812+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:02:17.656812+0000","last_undegraded":"2024-05-08T23:02:17.656812+0000","last_fullsized":"2024-05-08T23:02:17.656812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649950+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:02:17.649950+0000","last_peered":"2024-05-08T23:02:17.649950+0000","last_clean":"2024-05-08T23:02:17.649950+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:02:17.649950+0000","last_undegraded":"2024-05-08T23:02:17.649950+0000","last_fullsized":"2024-05-08T23:02:17.649950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:25.018945+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:25.018945+0000","last_peered":"2024-05-08T23:02:25.018945+0000","last_clean":"2024-05-08T23:02:25.018945+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:25.018945+0000","last_undegraded":"2024-05-08T23:02:25.018945+0000","last_fullsized":"2024-05-08T23:02:25.018945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650266+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:02:17.650266+0000","last_peered":"2024-05-08T23:02:17.650266+0000","last_clean":"2024-05-08T23:02:17.650266+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:02:17.650266+0000","last_undegraded":"2024-05-08T23:02:17.650266+0000","last_fullsized":"2024-05-08T23:02:17.650266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654622+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:02:17.654622+0000","last_peered":"2024-05-08T23:02:17.654622+0000","last_clean":"2024-05-08T23:02:17.654622+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:02:17.654622+0000","last_undegraded":"2024-05-08T23:02:17.654622+0000","last_fullsized":"2024-05-08T23:02:17.654622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656885+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:18.656885+0000","last_peered":"2024-05-08T23:02:18.656885+0000","last_clean":"2024-05-08T23:02:18.656885+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:18.656885+0000","last_undegraded":"2024-05-08T23:02:18.656885+0000","last_fullsized":"2024-05-08T23:02:18.656885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661556+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:02:17.661556+0000","last_peered":"2024-05-08T23:02:17.661556+0000","last_clean":"2024-05-08T23:02:17.661556+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:02:17.661556+0000","last_undegraded":"2024-05-08T23:02:17.661556+0000","last_fullsized":"2024-05-08T23:02:17.661556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654068+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:02:17.654068+0000","last_peered":"2024-05-08T23:02:17.654068+0000","last_clean":"2024-05-08T23:02:17.654068+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:02:17.654068+0000","last_undegraded":"2024-05-08T23:02:17.654068+0000","last_fullsized":"2024-05-08T23:02:17.654068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659168+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:17.659168+0000","last_peered":"2024-05-08T23:02:17.659168+0000","last_clean":"2024-05-08T23:02:17.659168+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:17.659168+0000","last_undegraded":"2024-05-08T23:02:17.659168+0000","last_fullsized":"2024-05-08T23:02:17.659168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657027+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:02:17.657027+0000","last_peered":"2024-05-08T23:02:17.657027+0000","last_clean":"2024-05-08T23:02:17.657027+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:02:17.657027+0000","last_undegraded":"2024-05-08T23:02:17.657027+0000","last_fullsized":"2024-05-08T23:02:17.657027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673909+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.673909+0000","last_peered":"2024-05-08T23:02:17.673909+0000","last_clean":"2024-05-08T23:02:17.673909+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.673909+0000","last_undegraded":"2024-05-08T23:02:17.673909+0000","last_fullsized":"2024-05-08T23:02:17.673909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659636+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:02:17.659636+0000","last_peered":"2024-05-08T23:02:17.659636+0000","last_clean":"2024-05-08T23:02:17.659636+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:02:17.659636+0000","last_undegraded":"2024-05-08T23:02:17.659636+0000","last_fullsized":"2024-05-08T23:02:17.659636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.657221+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:02:18.657221+0000","last_peered":"2024-05-08T23:02:18.657221+0000","last_clean":"2024-05-08T23:02:18.657221+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:02:18.657221+0000","last_undegraded":"2024-05-08T23:02:18.657221+0000","last_fullsized":"2024-05-08T23:02:18.657221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.985257+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:02:18.985257+0000","last_peered":"2024-05-08T23:02:18.985257+0000","last_clean":"2024-05-08T23:02:18.985257+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:02:18.985257+0000","last_undegraded":"2024-05-08T23:02:18.985257+0000","last_fullsized":"2024-05-08T23:02:18.985257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650347+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:02:17.650347+0000","last_peered":"2024-05-08T23:02:17.650347+0000","last_clean":"2024-05-08T23:02:17.650347+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:02:17.650347+0000","last_undegraded":"2024-05-08T23:02:17.650347+0000","last_fullsized":"2024-05-08T23:02:17.650347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654280+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:02:17.654280+0000","last_peered":"2024-05-08T23:02:17.654280+0000","last_clean":"2024-05-08T23:02:17.654280+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:02:17.654280+0000","last_undegraded":"2024-05-08T23:02:17.654280+0000","last_fullsized":"2024-05-08T23:02:17.654280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.654955+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:18.654955+0000","last_peered":"2024-05-08T23:02:18.654955+0000","last_clean":"2024-05-08T23:02:18.654955+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:18.654955+0000","last_undegraded":"2024-05-08T23:02:18.654955+0000","last_fullsized":"2024-05-08T23:02:18.654955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659929+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:02:17.659929+0000","last_peered":"2024-05-08T23:02:17.659929+0000","last_clean":"2024-05-08T23:02:17.659929+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:02:17.659929+0000","last_undegraded":"2024-05-08T23:02:17.659929+0000","last_fullsized":"2024-05-08T23:02:17.659929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:55.692387+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:02:55.692387+0000","last_peered":"2024-05-08T23:02:55.692387+0000","last_clean":"2024-05-08T23:02:55.692387+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:02:55.692387+0000","last_undegraded":"2024-05-08T23:02:55.692387+0000","last_fullsized":"2024-05-08T23:02:55.692387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661348+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:02:17.661348+0000","last_peered":"2024-05-08T23:02:17.661348+0000","last_clean":"2024-05-08T23:02:17.661348+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:02:17.661348+0000","last_undegraded":"2024-05-08T23:02:17.661348+0000","last_fullsized":"2024-05-08T23:02:17.661348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667814+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:02:18.667814+0000","last_peered":"2024-05-08T23:02:18.667814+0000","last_clean":"2024-05-08T23:02:18.667814+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:02:18.667814+0000","last_undegraded":"2024-05-08T23:02:18.667814+0000","last_fullsized":"2024-05-08T23:02:18.667814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659729+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:02:17.659729+0000","last_peered":"2024-05-08T23:02:17.659729+0000","last_clean":"2024-05-08T23:02:17.659729+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:02:17.659729+0000","last_undegraded":"2024-05-08T23:02:17.659729+0000","last_fullsized":"2024-05-08T23:02:17.659729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654952+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:02:17.654952+0000","last_peered":"2024-05-08T23:02:17.654952+0000","last_clean":"2024-05-08T23:02:17.654952+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:02:17.654952+0000","last_undegraded":"2024-05-08T23:02:17.654952+0000","last_fullsized":"2024-05-08T23:02:17.654952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656443+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:02:17.656443+0000","last_peered":"2024-05-08T23:02:17.656443+0000","last_clean":"2024-05-08T23:02:17.656443+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:02:17.656443+0000","last_undegraded":"2024-05-08T23:02:17.656443+0000","last_fullsized":"2024-05-08T23:02:17.656443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661526+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:02:17.661526+0000","last_peered":"2024-05-08T23:02:17.661526+0000","last_clean":"2024-05-08T23:02:17.661526+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:02:17.661526+0000","last_undegraded":"2024-05-08T23:02:17.661526+0000","last_fullsized":"2024-05-08T23:02:17.661526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656909+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:02:17.656909+0000","last_peered":"2024-05-08T23:02:17.656909+0000","last_clean":"2024-05-08T23:02:17.656909+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:02:17.656909+0000","last_undegraded":"2024-05-08T23:02:17.656909+0000","last_fullsized":"2024-05-08T23:02:17.656909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660321+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:02:17.660321+0000","last_peered":"2024-05-08T23:02:17.660321+0000","last_clean":"2024-05-08T23:02:17.660321+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:02:17.660321+0000","last_undegraded":"2024-05-08T23:02:17.660321+0000","last_fullsized":"2024-05-08T23:02:17.660321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655679+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:02:17.655679+0000","last_peered":"2024-05-08T23:02:17.655679+0000","last_clean":"2024-05-08T23:02:17.655679+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:02:17.655679+0000","last_undegraded":"2024-05-08T23:02:17.655679+0000","last_fullsized":"2024-05-08T23:02:17.655679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661450+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:02:18.661450+0000","last_peered":"2024-05-08T23:02:18.661450+0000","last_clean":"2024-05-08T23:02:18.661450+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:02:18.661450+0000","last_undegraded":"2024-05-08T23:02:18.661450+0000","last_fullsized":"2024-05-08T23:02:18.661450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651835+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:02:17.651835+0000","last_peered":"2024-05-08T23:02:17.651835+0000","last_clean":"2024-05-08T23:02:17.651835+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:02:17.651835+0000","last_undegraded":"2024-05-08T23:02:17.651835+0000","last_fullsized":"2024-05-08T23:02:17.651835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659375+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:02:17.659375+0000","last_peered":"2024-05-08T23:02:17.659375+0000","last_clean":"2024-05-08T23:02:17.659375+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:02:17.659375+0000","last_undegraded":"2024-05-08T23:02:17.659375+0000","last_fullsized":"2024-05-08T23:02:17.659375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657091+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:02:17.657091+0000","last_peered":"2024-05-08T23:02:17.657091+0000","last_clean":"2024-05-08T23:02:17.657091+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:02:17.657091+0000","last_undegraded":"2024-05-08T23:02:17.657091+0000","last_fullsized":"2024-05-08T23:02:17.657091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659850+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:02:17.659850+0000","last_peered":"2024-05-08T23:02:17.659850+0000","last_clean":"2024-05-08T23:02:17.659850+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:02:17.659850+0000","last_undegraded":"2024-05-08T23:02:17.659850+0000","last_fullsized":"2024-05-08T23:02:17.659850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68071,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":428,"num_read_kb":288,"num_write":236,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204213,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":466334,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40699999999999997}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.629},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed May 8 23:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82099999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1114969,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.626}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.873},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79300000000000004}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.76300000000000001}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.747},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.0089999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.97099999999999997}]}]},{"osd":9,"up_from":61,"seq":261993005087,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":470386,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0069999999999999,"5min":1.357,"15min":1.357},"last":0.625}]},{"osd":1,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52300000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73799999999999999,"15min":0.749},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.875,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90300000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.57999999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1299999999999999,"5min":1.738,"15min":1.738},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90200000000000002,"15min":0.91200000000000003},"min":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.53,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.86799999999999999}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.63300000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1799999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.95799999999999996,"15min":0.96899999999999997},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.90500000000000003}]},{"osd":8,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.94599999999999995,"15min":0.96199999999999997},"min":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.71499999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.262,"5min":1.766,"15min":1.766},"last":0.89000000000000001}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.93899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.504}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80800000000000005}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.91500000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.097,"15min":1.097},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77000000000000002}]},{"osd":11,"last update":"Wed May 8 23:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.78000000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462952,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1003520,"data_stored":1063023,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.45000000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38}]},{"osd":2,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60599999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":1.109,"15min":1.109},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79400000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.622},"min":{"1min":0.58899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88800000000000001,"5min":1.05,"15min":1.05},"last":0.626},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64200000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed May 8 23:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.65000000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.075,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83899999999999997},"min":{"1min":0.68400000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.03,"5min":1.141,"15min":1.141},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98599999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74199999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82699999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626476,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":929792,"data_stored":461746,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.53200000000000003,"15min":0.48999999999999999},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90000000000000002,"5min":1.103,"15min":1.103},"last":3.9100000000000001}]},{"osd":2,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.443,"15min":0.443},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":3.931}]},{"osd":3,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.61299999999999999,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.581,"15min":1.581},"last":0.5}]},{"osd":4,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.75900000000000001,"15min":0.73699999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.73399999999999999,"15min":0.70399999999999996},"min":{"1min":0.503,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.76500000000000001,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.72099999999999997},"min":{"1min":0.624,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.206,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.77400000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":3.952},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80900000000000005,"15min":0.78000000000000003},"min":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.624}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.90000000000000002,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.8610000000000002,"5min":2.8610000000000002,"15min":2.8610000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.006,"15min":0.98499999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.002,"15min":1.0109999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.748},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.9710000000000001,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":3.9710000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":524587,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60999999999999999}]},{"osd":2,"last update":"Wed May 8 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.53600000000000003},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.437,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81499999999999995}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.622},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.87},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.03,"15min":1.03},"last":0.747}]},{"osd":6,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.73999999999999999,"15min":0.68899999999999995},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.64900000000000002},"min":{"1min":0.53400000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.53400000000000003}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.052,"5min":1.071,"15min":1.071},"last":0.83899999999999997}]},{"osd":9,"last update":"Wed May 8 23:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.82699999999999996,"15min":0.81899999999999995},"min":{"1min":0.66700000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.88500000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.90600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":463202,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.57099999999999995}]},{"osd":2,"last update":"Wed May 8 23:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.62,"15min":0.54000000000000004},"min":{"1min":0.44400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":6.8710000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58699999999999997,"15min":0.53600000000000003},"min":{"1min":0.42199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":6.8849999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.69299999999999995,"15min":0.65700000000000003},"min":{"1min":0.57199999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":6.9139999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":6.9450000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67500000000000004,"15min":0.64100000000000001},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.55,"5min":1.921,"15min":1.921},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.76100000000000001,"15min":0.71499999999999997},"min":{"1min":0.57899999999999996,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.68600000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":6.9630000000000001,"5min":6.9630000000000001,"15min":6.9630000000000001},"last":6.9630000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.76300000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.92400000000000004}]},{"osd":9,"last update":"Wed May 8 23:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":0.96099999999999997},"min":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.673,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.871},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.94099999999999995,"15min":0.92900000000000005},"min":{"1min":0.69299999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.82199999999999995}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.90700000000000003}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":995328,"data_stored":522296,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.87,"15min":0.87},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.876,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.873,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.81599999999999995}]},{"osd":3,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.35099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45500000000000002,"15min":0.44},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.66600000000000004},"min":{"1min":0.55200000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89700000000000002,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.77600000000000002}]},{"osd":7,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.90100000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.879}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.83199999999999996}]}]},{"osd":3,"up_from":24,"seq":103079215159,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":461193,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46999999999999997,"15min":0.42399999999999999},"min":{"1min":0.35899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63700000000000001,"15min":0.56399999999999995},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":13.832000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66000000000000003,"15min":0.60299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":1.046,"15min":1.046},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.67000000000000004,"15min":0.60999999999999999},"min":{"1min":0.49299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.58999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.028,"5min":1.127,"15min":1.127},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71799999999999997,"15min":0.68300000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.169,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.748}]},{"osd":6,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.752,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.077,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed May 8 23:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0580000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":13.807},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.75,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1899999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94699999999999995,"15min":0.95999999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.96999999999999997,"15min":0.98799999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1299999999999999,"5min":1.292,"15min":1.292},"last":0.68799999999999994}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.0609999999999999,"15min":0.97399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.893000000000001,"5min":13.893000000000001,"15min":13.893000000000001},"last":13.893000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.01,"15min":1.014},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.83199999999999996}]},{"osd":11,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":13.855}]}]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":942080,"data_stored":467026,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.628},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.434},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":3.1899999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.45900000000000002}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.75}]},{"osd":6,"last update":"Wed May 8 23:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.65700000000000003},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88300000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89200000000000002,"5min":1.248,"15min":1.248},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed May 8 23:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.042,"15min":1.042},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.79400000000000004},"min":{"1min":0.59299999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96399999999999997,"5min":1.05,"15min":1.05},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed May 8 23:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.90600000000000003,"15min":0.92600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.99299999999999999,"5min":1.391,"15min":1.391},"last":0.79400000000000004}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.86399999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.90500000000000003}]}]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69899999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.746},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.75600000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72899999999999998}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63824,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62202,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63529,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:03:02.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:03:03.084 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:03 smithi053 ceph-mon[77587]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:03 smithi053 ceph-mon[77587]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:03.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:03 smithi046 ceph-mon[36898]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:03.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:03 smithi046 ceph-mon[36898]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:03.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:03 smithi086 ceph-mon[78610]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:03.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:03 smithi086 ceph-mon[78610]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 -- 172.21.15.46:0/638337653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 msgr2=0x7ff16c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 --2- 172.21.15.46:0/638337653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c068db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff158009990 tx=0x7ff158009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 -- 172.21.15.46:0/638337653 shutdown_connections 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 --2- 172.21.15.46:0/638337653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 0x7ff16c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 --2- 172.21.15.46:0/638337653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 --2- 172.21.15.46:0/638337653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c068db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 -- 172.21.15.46:0/638337653 >> 172.21.15.46:0/638337653 conn(0x7ff16c075f80 msgr2=0x7ff16c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 -- 172.21.15.46:0/638337653 shutdown_connections 2024-05-08T23:03:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.302+0000 7ff173400700 1 -- 172.21.15.46:0/638337653 wait complete. 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.303+0000 7ff173400700 1 Processor -- start 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 -- start start 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 0x7ff16c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a57c0 con 0x7ff16c068950 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a5930 con 0x7ff16c107020 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff173400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a5af0 con 0x7ff16c0692f0 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff170a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff171400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff170a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54432/0 (socket says 172.21.15.46:54432) 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff170a00700 1 -- 172.21.15.46:0/3024157560 learned_addr learned my addr 172.21.15.46:0/3024157560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.304+0000 7ff16be00700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 0x7ff16c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 -- 172.21.15.46:0/3024157560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 msgr2=0x7ff16c1004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 0x7ff16c1004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 -- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 msgr2=0x7ff16c0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 -- 172.21.15.46:0/3024157560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580095f0 con 0x7ff16c107020 2024-05-08T23:03:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff171400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c100a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff16400efc0 tx=0x7ff16400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164018070 con 0x7ff16c107020 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16400f040 con 0x7ff16c107020 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400cd40 con 0x7ff16c107020 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.305+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c1a5d70 con 0x7ff16c107020 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.306+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1a6220 con 0x7ff16c107020 2024-05-08T23:03:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.307+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c109540 con 0x7ff16c107020 2024-05-08T23:03:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.309+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff164003a00 con 0x7ff16c107020 2024-05-08T23:03:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.310+0000 7ff169600700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 0x7ff1540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.310+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff1640cd270 con 0x7ff16c107020 2024-05-08T23:03:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.310+0000 7ff170a00700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 0x7ff1540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.311+0000 7ff170a00700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 0x7ff1540a36f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff160009ca0 tx=0x7ff160009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.312+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff1640965c0 con 0x7ff16c107020 2024-05-08T23:03:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.522+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff16c1014f0 con 0x7ff1540a1230 2024-05-08T23:03:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.535+0000 7ff169600700 1 -- 172.21.15.46:0/3024157560 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427011 (secure 0 0 0) 0x7ff16c1014f0 con 0x7ff1540a1230 2024-05-08T23:03:04.541 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 msgr2=0x7ff1540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 0x7ff1540a36f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff160009ca0 tx=0x7ff160009260 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 msgr2=0x7ff16c100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c100a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff16400efc0 tx=0x7ff16400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 shutdown_connections 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16c107020 0x7ff16c100a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff1540a1230 0x7ff1540a36f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c068950 0x7ff16c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 --2- 172.21.15.46:0/3024157560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16c0692f0 0x7ff16c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 >> 172.21.15.46:0/3024157560 conn(0x7ff16c075f80 msgr2=0x7ff16c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:04.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 shutdown_connections 2024-05-08T23:03:04.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:04.541+0000 7ff173400700 1 -- 172.21.15.46:0/3024157560 wait complete. 2024-05-08T23:03:04.546 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:03:04.929 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-05-08T23:03:02.936072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658673,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":584,"num_read_kb":417,"num_write":419,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372488,"kb_used_data":11396,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124569336,"statfs":{"total":1151940427776,"available":1151559000064,"internally_reserved":0,"allocated":11669504,"data_stored":7538763,"data_compressed":324360,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661165+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:02:17.661165+0000","last_peered":"2024-05-08T23:02:17.661165+0000","last_clean":"2024-05-08T23:02:17.661165+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:02:17.661165+0000","last_undegraded":"2024-05-08T23:02:17.661165+0000","last_fullsized":"2024-05-08T23:02:17.661165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661559+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:02:17.661559+0000","last_peered":"2024-05-08T23:02:17.661559+0000","last_clean":"2024-05-08T23:02:17.661559+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:02:17.661559+0000","last_undegraded":"2024-05-08T23:02:17.661559+0000","last_fullsized":"2024-05-08T23:02:17.661559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661196+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:02:17.661196+0000","last_peered":"2024-05-08T23:02:17.661196+0000","last_clean":"2024-05-08T23:02:17.661196+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:02:17.661196+0000","last_undegraded":"2024-05-08T23:02:17.661196+0000","last_fullsized":"2024-05-08T23:02:17.661196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654527+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:02:17.654527+0000","last_peered":"2024-05-08T23:02:17.654527+0000","last_clean":"2024-05-08T23:02:17.654527+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:02:17.654527+0000","last_undegraded":"2024-05-08T23:02:17.654527+0000","last_fullsized":"2024-05-08T23:02:17.654527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659619+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:02:18.659619+0000","last_peered":"2024-05-08T23:02:18.659619+0000","last_clean":"2024-05-08T23:02:18.659619+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:02:18.659619+0000","last_undegraded":"2024-05-08T23:02:18.659619+0000","last_fullsized":"2024-05-08T23:02:18.659619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659366+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:02:18.659366+0000","last_peered":"2024-05-08T23:02:18.659366+0000","last_clean":"2024-05-08T23:02:18.659366+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:02:18.659366+0000","last_undegraded":"2024-05-08T23:02:18.659366+0000","last_fullsized":"2024-05-08T23:02:18.659366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651144+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:02:17.651144+0000","last_peered":"2024-05-08T23:02:17.651144+0000","last_clean":"2024-05-08T23:02:17.651144+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:02:17.651144+0000","last_undegraded":"2024-05-08T23:02:17.651144+0000","last_fullsized":"2024-05-08T23:02:17.651144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650061+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:02:17.650061+0000","last_peered":"2024-05-08T23:02:17.650061+0000","last_clean":"2024-05-08T23:02:17.650061+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:02:17.650061+0000","last_undegraded":"2024-05-08T23:02:17.650061+0000","last_fullsized":"2024-05-08T23:02:17.650061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660268+0000","last_change":"2024-05-08T23:01:51.483419+0000","last_active":"2024-05-08T23:02:17.660268+0000","last_peered":"2024-05-08T23:02:17.660268+0000","last_clean":"2024-05-08T23:02:17.660268+0000","last_became_active":"2024-05-08T23:01:51.482888+0000","last_became_peered":"2024-05-08T23:01:51.482888+0000","last_unstale":"2024-05-08T23:02:17.660268+0000","last_undegraded":"2024-05-08T23:02:17.660268+0000","last_fullsized":"2024-05-08T23:02:17.660268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:15.414096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657657+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:02:17.657657+0000","last_peered":"2024-05-08T23:02:17.657657+0000","last_clean":"2024-05-08T23:02:17.657657+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:02:17.657657+0000","last_undegraded":"2024-05-08T23:02:17.657657+0000","last_fullsized":"2024-05-08T23:02:17.657657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651258+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:02:17.651258+0000","last_peered":"2024-05-08T23:02:17.651258+0000","last_clean":"2024-05-08T23:02:17.651258+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:02:17.651258+0000","last_undegraded":"2024-05-08T23:02:17.651258+0000","last_fullsized":"2024-05-08T23:02:17.651258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654543+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:02:17.654543+0000","last_peered":"2024-05-08T23:02:17.654543+0000","last_clean":"2024-05-08T23:02:17.654543+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:02:17.654543+0000","last_undegraded":"2024-05-08T23:02:17.654543+0000","last_fullsized":"2024-05-08T23:02:17.654543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656492+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:02:17.656492+0000","last_peered":"2024-05-08T23:02:17.656492+0000","last_clean":"2024-05-08T23:02:17.656492+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:02:17.656492+0000","last_undegraded":"2024-05-08T23:02:17.656492+0000","last_fullsized":"2024-05-08T23:02:17.656492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661236+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:02:17.661236+0000","last_peered":"2024-05-08T23:02:17.661236+0000","last_clean":"2024-05-08T23:02:17.661236+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:02:17.661236+0000","last_undegraded":"2024-05-08T23:02:17.661236+0000","last_fullsized":"2024-05-08T23:02:17.661236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661078+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:02:17.661078+0000","last_peered":"2024-05-08T23:02:17.661078+0000","last_clean":"2024-05-08T23:02:17.661078+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:02:17.661078+0000","last_undegraded":"2024-05-08T23:02:17.661078+0000","last_fullsized":"2024-05-08T23:02:17.661078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659040+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:02:17.659040+0000","last_peered":"2024-05-08T23:02:17.659040+0000","last_clean":"2024-05-08T23:02:17.659040+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:02:17.659040+0000","last_undegraded":"2024-05-08T23:02:17.659040+0000","last_fullsized":"2024-05-08T23:02:17.659040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661153+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:02:17.661153+0000","last_peered":"2024-05-08T23:02:17.661153+0000","last_clean":"2024-05-08T23:02:17.661153+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:02:17.661153+0000","last_undegraded":"2024-05-08T23:02:17.661153+0000","last_fullsized":"2024-05-08T23:02:17.661153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656514+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:02:17.656514+0000","last_peered":"2024-05-08T23:02:17.656514+0000","last_clean":"2024-05-08T23:02:17.656514+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:02:17.656514+0000","last_undegraded":"2024-05-08T23:02:17.656514+0000","last_fullsized":"2024-05-08T23:02:17.656514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655453+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:02:17.655453+0000","last_peered":"2024-05-08T23:02:17.655453+0000","last_clean":"2024-05-08T23:02:17.655453+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:02:17.655453+0000","last_undegraded":"2024-05-08T23:02:17.655453+0000","last_fullsized":"2024-05-08T23:02:17.655453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654795+0000","last_change":"2024-05-08T23:01:55.980316+0000","last_active":"2024-05-08T23:02:17.654795+0000","last_peered":"2024-05-08T23:02:17.654795+0000","last_clean":"2024-05-08T23:02:17.654795+0000","last_became_active":"2024-05-08T23:01:55.979891+0000","last_became_peered":"2024-05-08T23:01:55.979891+0000","last_unstale":"2024-05-08T23:02:17.654795+0000","last_undegraded":"2024-05-08T23:02:17.654795+0000","last_fullsized":"2024-05-08T23:02:17.654795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:03.573605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:59.697859+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:02:59.697859+0000","last_peered":"2024-05-08T23:02:59.697859+0000","last_clean":"2024-05-08T23:02:59.697859+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:02:59.697859+0000","last_undegraded":"2024-05-08T23:02:59.697859+0000","last_fullsized":"2024-05-08T23:02:59.697859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651671+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:02:17.651671+0000","last_peered":"2024-05-08T23:02:17.651671+0000","last_clean":"2024-05-08T23:02:17.651671+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:02:17.651671+0000","last_undegraded":"2024-05-08T23:02:17.651671+0000","last_fullsized":"2024-05-08T23:02:17.651671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653802+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:02:17.653802+0000","last_peered":"2024-05-08T23:02:17.653802+0000","last_clean":"2024-05-08T23:02:17.653802+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:02:17.653802+0000","last_undegraded":"2024-05-08T23:02:17.653802+0000","last_fullsized":"2024-05-08T23:02:17.653802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651158+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:02:17.651158+0000","last_peered":"2024-05-08T23:02:17.651158+0000","last_clean":"2024-05-08T23:02:17.651158+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:02:17.651158+0000","last_undegraded":"2024-05-08T23:02:17.651158+0000","last_fullsized":"2024-05-08T23:02:17.651158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661525+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:02:17.661525+0000","last_peered":"2024-05-08T23:02:17.661525+0000","last_clean":"2024-05-08T23:02:17.661525+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:02:17.661525+0000","last_undegraded":"2024-05-08T23:02:17.661525+0000","last_fullsized":"2024-05-08T23:02:17.661525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653999+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:02:17.653999+0000","last_peered":"2024-05-08T23:02:17.653999+0000","last_clean":"2024-05-08T23:02:17.653999+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:02:17.653999+0000","last_undegraded":"2024-05-08T23:02:17.653999+0000","last_fullsized":"2024-05-08T23:02:17.653999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636260+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:02:19.636260+0000","last_peered":"2024-05-08T23:02:19.636260+0000","last_clean":"2024-05-08T23:02:19.636260+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:02:19.636260+0000","last_undegraded":"2024-05-08T23:02:19.636260+0000","last_fullsized":"2024-05-08T23:02:19.636260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659474+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:02:17.659474+0000","last_peered":"2024-05-08T23:02:17.659474+0000","last_clean":"2024-05-08T23:02:17.659474+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:02:17.659474+0000","last_undegraded":"2024-05-08T23:02:17.659474+0000","last_fullsized":"2024-05-08T23:02:17.659474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653902+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:02:17.653902+0000","last_peered":"2024-05-08T23:02:17.653902+0000","last_clean":"2024-05-08T23:02:17.653902+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:02:17.653902+0000","last_undegraded":"2024-05-08T23:02:17.653902+0000","last_fullsized":"2024-05-08T23:02:17.653902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655207+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:02:18.655207+0000","last_peered":"2024-05-08T23:02:18.655207+0000","last_clean":"2024-05-08T23:02:18.655207+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:02:18.655207+0000","last_undegraded":"2024-05-08T23:02:18.655207+0000","last_fullsized":"2024-05-08T23:02:18.655207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654730+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:02:17.654730+0000","last_peered":"2024-05-08T23:02:17.654730+0000","last_clean":"2024-05-08T23:02:17.654730+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:02:17.654730+0000","last_undegraded":"2024-05-08T23:02:17.654730+0000","last_fullsized":"2024-05-08T23:02:17.654730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655226+0000","last_change":"2024-05-08T23:01:55.960741+0000","last_active":"2024-05-08T23:02:17.655226+0000","last_peered":"2024-05-08T23:02:17.655226+0000","last_clean":"2024-05-08T23:02:17.655226+0000","last_became_active":"2024-05-08T23:01:55.960346+0000","last_became_peered":"2024-05-08T23:01:55.960346+0000","last_unstale":"2024-05-08T23:02:17.655226+0000","last_undegraded":"2024-05-08T23:02:17.655226+0000","last_fullsized":"2024-05-08T23:02:17.655226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:02.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.699074+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:02:58.699074+0000","last_peered":"2024-05-08T23:02:58.699074+0000","last_clean":"2024-05-08T23:02:58.699074+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:02:58.699074+0000","last_undegraded":"2024-05-08T23:02:58.699074+0000","last_fullsized":"2024-05-08T23:02:58.699074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655320+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:02:17.655320+0000","last_peered":"2024-05-08T23:02:17.655320+0000","last_clean":"2024-05-08T23:02:17.655320+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:02:17.655320+0000","last_undegraded":"2024-05-08T23:02:17.655320+0000","last_fullsized":"2024-05-08T23:02:17.655320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657534+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:02:17.657534+0000","last_peered":"2024-05-08T23:02:17.657534+0000","last_clean":"2024-05-08T23:02:17.657534+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:02:17.657534+0000","last_undegraded":"2024-05-08T23:02:17.657534+0000","last_fullsized":"2024-05-08T23:02:17.657534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.658288+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:02:17.658288+0000","last_peered":"2024-05-08T23:02:17.658288+0000","last_clean":"2024-05-08T23:02:17.658288+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:02:17.658288+0000","last_undegraded":"2024-05-08T23:02:17.658288+0000","last_fullsized":"2024-05-08T23:02:17.658288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.665953+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:02:18.665953+0000","last_peered":"2024-05-08T23:02:18.665953+0000","last_clean":"2024-05-08T23:02:18.665953+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:02:18.665953+0000","last_undegraded":"2024-05-08T23:02:18.665953+0000","last_fullsized":"2024-05-08T23:02:18.665953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661059+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:02:17.661059+0000","last_peered":"2024-05-08T23:02:17.661059+0000","last_clean":"2024-05-08T23:02:17.661059+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:02:17.661059+0000","last_undegraded":"2024-05-08T23:02:17.661059+0000","last_fullsized":"2024-05-08T23:02:17.661059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.674762+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:02:17.674762+0000","last_peered":"2024-05-08T23:02:17.674762+0000","last_clean":"2024-05-08T23:02:17.674762+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:02:17.674762+0000","last_undegraded":"2024-05-08T23:02:17.674762+0000","last_fullsized":"2024-05-08T23:02:17.674762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649951+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:02:17.649951+0000","last_peered":"2024-05-08T23:02:17.649951+0000","last_clean":"2024-05-08T23:02:17.649951+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:02:17.649951+0000","last_undegraded":"2024-05-08T23:02:17.649951+0000","last_fullsized":"2024-05-08T23:02:17.649951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661283+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:02:17.661283+0000","last_peered":"2024-05-08T23:02:17.661283+0000","last_clean":"2024-05-08T23:02:17.661283+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:02:17.661283+0000","last_undegraded":"2024-05-08T23:02:17.661283+0000","last_fullsized":"2024-05-08T23:02:17.661283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.660664+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:02:18.660664+0000","last_peered":"2024-05-08T23:02:18.660664+0000","last_clean":"2024-05-08T23:02:18.660664+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:02:18.660664+0000","last_undegraded":"2024-05-08T23:02:18.660664+0000","last_fullsized":"2024-05-08T23:02:18.660664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655141+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:02:17.655141+0000","last_peered":"2024-05-08T23:02:17.655141+0000","last_clean":"2024-05-08T23:02:17.655141+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:02:17.655141+0000","last_undegraded":"2024-05-08T23:02:17.655141+0000","last_fullsized":"2024-05-08T23:02:17.655141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651327+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:02:17.651327+0000","last_peered":"2024-05-08T23:02:17.651327+0000","last_clean":"2024-05-08T23:02:17.651327+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:02:17.651327+0000","last_undegraded":"2024-05-08T23:02:17.651327+0000","last_fullsized":"2024-05-08T23:02:17.651327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.695684+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:02:58.695684+0000","last_peered":"2024-05-08T23:02:58.695684+0000","last_clean":"2024-05-08T23:02:58.695684+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:02:58.695684+0000","last_undegraded":"2024-05-08T23:02:58.695684+0000","last_fullsized":"2024-05-08T23:02:58.695684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660300+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:02:17.660300+0000","last_peered":"2024-05-08T23:02:17.660300+0000","last_clean":"2024-05-08T23:02:17.660300+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:02:17.660300+0000","last_undegraded":"2024-05-08T23:02:17.660300+0000","last_fullsized":"2024-05-08T23:02:17.660300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654915+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:02:17.654915+0000","last_peered":"2024-05-08T23:02:17.654915+0000","last_clean":"2024-05-08T23:02:17.654915+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:02:17.654915+0000","last_undegraded":"2024-05-08T23:02:17.654915+0000","last_fullsized":"2024-05-08T23:02:17.654915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.675213+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:02:17.675213+0000","last_peered":"2024-05-08T23:02:17.675213+0000","last_clean":"2024-05-08T23:02:17.675213+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:02:17.675213+0000","last_undegraded":"2024-05-08T23:02:17.675213+0000","last_fullsized":"2024-05-08T23:02:17.675213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661428+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:02:17.661428+0000","last_peered":"2024-05-08T23:02:17.661428+0000","last_clean":"2024-05-08T23:02:17.661428+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:02:17.661428+0000","last_undegraded":"2024-05-08T23:02:17.661428+0000","last_fullsized":"2024-05-08T23:02:17.661428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654337+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:02:17.654337+0000","last_peered":"2024-05-08T23:02:17.654337+0000","last_clean":"2024-05-08T23:02:17.654337+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:02:17.654337+0000","last_undegraded":"2024-05-08T23:02:17.654337+0000","last_fullsized":"2024-05-08T23:02:17.654337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661357+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:02:18.661357+0000","last_peered":"2024-05-08T23:02:18.661357+0000","last_clean":"2024-05-08T23:02:18.661357+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:02:18.661357+0000","last_undegraded":"2024-05-08T23:02:18.661357+0000","last_fullsized":"2024-05-08T23:02:18.661357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659646+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:02:17.659646+0000","last_peered":"2024-05-08T23:02:17.659646+0000","last_clean":"2024-05-08T23:02:17.659646+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:02:17.659646+0000","last_undegraded":"2024-05-08T23:02:17.659646+0000","last_fullsized":"2024-05-08T23:02:17.659646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654901+0000","last_change":"2024-05-08T23:01:55.974904+0000","last_active":"2024-05-08T23:02:17.654901+0000","last_peered":"2024-05-08T23:02:17.654901+0000","last_clean":"2024-05-08T23:02:17.654901+0000","last_became_active":"2024-05-08T23:01:55.972590+0000","last_became_peered":"2024-05-08T23:01:55.972590+0000","last_unstale":"2024-05-08T23:02:17.654901+0000","last_undegraded":"2024-05-08T23:02:17.654901+0000","last_fullsized":"2024-05-08T23:02:17.654901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:19.819371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":137,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:58.695654+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:02:58.695654+0000","last_peered":"2024-05-08T23:02:58.695654+0000","last_clean":"2024-05-08T23:02:58.695654+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:02:58.695654+0000","last_undegraded":"2024-05-08T23:02:58.695654+0000","last_fullsized":"2024-05-08T23:02:58.695654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61448,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":103,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654549+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:02:17.654549+0000","last_peered":"2024-05-08T23:02:17.654549+0000","last_clean":"2024-05-08T23:02:17.654549+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:02:17.654549+0000","last_undegraded":"2024-05-08T23:02:17.654549+0000","last_fullsized":"2024-05-08T23:02:17.654549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655063+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:02:17.655063+0000","last_peered":"2024-05-08T23:02:17.655063+0000","last_clean":"2024-05-08T23:02:17.655063+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:02:17.655063+0000","last_undegraded":"2024-05-08T23:02:17.655063+0000","last_fullsized":"2024-05-08T23:02:17.655063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.636478+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:02:19.636478+0000","last_peered":"2024-05-08T23:02:19.636478+0000","last_clean":"2024-05-08T23:02:19.636478+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:02:19.636478+0000","last_undegraded":"2024-05-08T23:02:19.636478+0000","last_fullsized":"2024-05-08T23:02:19.636478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655806+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:02:17.655806+0000","last_peered":"2024-05-08T23:02:17.655806+0000","last_clean":"2024-05-08T23:02:17.655806+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:02:17.655806+0000","last_undegraded":"2024-05-08T23:02:17.655806+0000","last_fullsized":"2024-05-08T23:02:17.655806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659759+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:02:17.659759+0000","last_peered":"2024-05-08T23:02:17.659759+0000","last_clean":"2024-05-08T23:02:17.659759+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:02:17.659759+0000","last_undegraded":"2024-05-08T23:02:17.659759+0000","last_fullsized":"2024-05-08T23:02:17.659759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.659613+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:02:18.659613+0000","last_peered":"2024-05-08T23:02:18.659613+0000","last_clean":"2024-05-08T23:02:18.659613+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:02:18.659613+0000","last_undegraded":"2024-05-08T23:02:18.659613+0000","last_fullsized":"2024-05-08T23:02:18.659613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651072+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:02:17.651072+0000","last_peered":"2024-05-08T23:02:17.651072+0000","last_clean":"2024-05-08T23:02:17.651072+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:02:17.651072+0000","last_undegraded":"2024-05-08T23:02:17.651072+0000","last_fullsized":"2024-05-08T23:02:17.651072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656023+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:02:17.656023+0000","last_peered":"2024-05-08T23:02:17.656023+0000","last_clean":"2024-05-08T23:02:17.656023+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:02:17.656023+0000","last_undegraded":"2024-05-08T23:02:17.656023+0000","last_fullsized":"2024-05-08T23:02:17.656023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":69,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:03:00.701683+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:03:00.701683+0000","last_peered":"2024-05-08T23:03:00.701683+0000","last_clean":"2024-05-08T23:03:00.701683+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:03:00.701683+0000","last_undegraded":"2024-05-08T23:03:00.701683+0000","last_fullsized":"2024-05-08T23:03:00.701683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650701+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:02:17.650701+0000","last_peered":"2024-05-08T23:02:17.650701+0000","last_clean":"2024-05-08T23:02:17.650701+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:02:17.650701+0000","last_undegraded":"2024-05-08T23:02:17.650701+0000","last_fullsized":"2024-05-08T23:02:17.650701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657551+0000","last_change":"2024-05-08T23:01:55.977597+0000","last_active":"2024-05-08T23:02:17.657551+0000","last_peered":"2024-05-08T23:02:17.657551+0000","last_clean":"2024-05-08T23:02:17.657551+0000","last_became_active":"2024-05-08T23:01:55.976246+0000","last_became_peered":"2024-05-08T23:01:55.976246+0000","last_unstale":"2024-05-08T23:02:17.657551+0000","last_undegraded":"2024-05-08T23:02:17.657551+0000","last_fullsized":"2024-05-08T23:02:17.657551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:45.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656051+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:02:18.656051+0000","last_peered":"2024-05-08T23:02:18.656051+0000","last_clean":"2024-05-08T23:02:18.656051+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:02:18.656051+0000","last_undegraded":"2024-05-08T23:02:18.656051+0000","last_fullsized":"2024-05-08T23:02:18.656051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653548+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:02:17.653548+0000","last_peered":"2024-05-08T23:02:17.653548+0000","last_clean":"2024-05-08T23:02:17.653548+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:02:17.653548+0000","last_undegraded":"2024-05-08T23:02:17.653548+0000","last_fullsized":"2024-05-08T23:02:17.653548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653438+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:02:17.653438+0000","last_peered":"2024-05-08T23:02:17.653438+0000","last_clean":"2024-05-08T23:02:17.653438+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:02:17.653438+0000","last_undegraded":"2024-05-08T23:02:17.653438+0000","last_fullsized":"2024-05-08T23:02:17.653438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.670834+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:02:18.670834+0000","last_peered":"2024-05-08T23:02:18.670834+0000","last_clean":"2024-05-08T23:02:18.670834+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:02:18.670834+0000","last_undegraded":"2024-05-08T23:02:18.670834+0000","last_fullsized":"2024-05-08T23:02:18.670834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.663365+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:02:18.663365+0000","last_peered":"2024-05-08T23:02:18.663365+0000","last_clean":"2024-05-08T23:02:18.663365+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:02:18.663365+0000","last_undegraded":"2024-05-08T23:02:18.663365+0000","last_fullsized":"2024-05-08T23:02:18.663365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653739+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:02:17.653739+0000","last_peered":"2024-05-08T23:02:17.653739+0000","last_clean":"2024-05-08T23:02:17.653739+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:02:17.653739+0000","last_undegraded":"2024-05-08T23:02:17.653739+0000","last_fullsized":"2024-05-08T23:02:17.653739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655364+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:02:17.655364+0000","last_peered":"2024-05-08T23:02:17.655364+0000","last_clean":"2024-05-08T23:02:17.655364+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:02:17.655364+0000","last_undegraded":"2024-05-08T23:02:17.655364+0000","last_fullsized":"2024-05-08T23:02:17.655364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657663+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:02:17.657663+0000","last_peered":"2024-05-08T23:02:17.657663+0000","last_clean":"2024-05-08T23:02:17.657663+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:02:17.657663+0000","last_undegraded":"2024-05-08T23:02:17.657663+0000","last_fullsized":"2024-05-08T23:02:17.657663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656734+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:02:17.656734+0000","last_peered":"2024-05-08T23:02:17.656734+0000","last_clean":"2024-05-08T23:02:17.656734+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:02:17.656734+0000","last_undegraded":"2024-05-08T23:02:17.656734+0000","last_fullsized":"2024-05-08T23:02:17.656734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:57.695998+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:02:57.695998+0000","last_peered":"2024-05-08T23:02:57.695998+0000","last_clean":"2024-05-08T23:02:57.695998+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:02:57.695998+0000","last_undegraded":"2024-05-08T23:02:57.695998+0000","last_fullsized":"2024-05-08T23:02:57.695998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657279+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:02:17.657279+0000","last_peered":"2024-05-08T23:02:17.657279+0000","last_clean":"2024-05-08T23:02:17.657279+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:02:17.657279+0000","last_undegraded":"2024-05-08T23:02:17.657279+0000","last_fullsized":"2024-05-08T23:02:17.657279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657532+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:02:17.657532+0000","last_peered":"2024-05-08T23:02:17.657532+0000","last_clean":"2024-05-08T23:02:17.657532+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:02:17.657532+0000","last_undegraded":"2024-05-08T23:02:17.657532+0000","last_fullsized":"2024-05-08T23:02:17.657532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656991+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:02:17.656991+0000","last_peered":"2024-05-08T23:02:17.656991+0000","last_clean":"2024-05-08T23:02:17.656991+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:02:17.656991+0000","last_undegraded":"2024-05-08T23:02:17.656991+0000","last_fullsized":"2024-05-08T23:02:17.656991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:03:00.701202+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:03:00.701202+0000","last_peered":"2024-05-08T23:03:00.701202+0000","last_clean":"2024-05-08T23:03:00.701202+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:03:00.701202+0000","last_undegraded":"2024-05-08T23:03:00.701202+0000","last_fullsized":"2024-05-08T23:03:00.701202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.697678+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:02:17.697678+0000","last_peered":"2024-05-08T23:02:17.697678+0000","last_clean":"2024-05-08T23:02:17.697678+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:02:17.697678+0000","last_undegraded":"2024-05-08T23:02:17.697678+0000","last_fullsized":"2024-05-08T23:02:17.697678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650153+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:02:17.650153+0000","last_peered":"2024-05-08T23:02:17.650153+0000","last_clean":"2024-05-08T23:02:17.650153+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:02:17.650153+0000","last_undegraded":"2024-05-08T23:02:17.650153+0000","last_fullsized":"2024-05-08T23:02:17.650153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:19.639109+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:02:19.639109+0000","last_peered":"2024-05-08T23:02:19.639109+0000","last_clean":"2024-05-08T23:02:19.639109+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:02:19.639109+0000","last_undegraded":"2024-05-08T23:02:19.639109+0000","last_fullsized":"2024-05-08T23:02:19.639109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650540+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:02:17.650540+0000","last_peered":"2024-05-08T23:02:17.650540+0000","last_clean":"2024-05-08T23:02:17.650540+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:02:17.650540+0000","last_undegraded":"2024-05-08T23:02:17.650540+0000","last_fullsized":"2024-05-08T23:02:17.650540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653597+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:02:17.653597+0000","last_peered":"2024-05-08T23:02:17.653597+0000","last_clean":"2024-05-08T23:02:17.653597+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:02:17.653597+0000","last_undegraded":"2024-05-08T23:02:17.653597+0000","last_fullsized":"2024-05-08T23:02:17.653597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654454+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:02:17.654454+0000","last_peered":"2024-05-08T23:02:17.654454+0000","last_clean":"2024-05-08T23:02:17.654454+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:02:17.654454+0000","last_undegraded":"2024-05-08T23:02:17.654454+0000","last_fullsized":"2024-05-08T23:02:17.654454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667675+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:02:18.667675+0000","last_peered":"2024-05-08T23:02:18.667675+0000","last_clean":"2024-05-08T23:02:18.667675+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:02:18.667675+0000","last_undegraded":"2024-05-08T23:02:18.667675+0000","last_fullsized":"2024-05-08T23:02:18.667675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659637+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:02:17.659637+0000","last_peered":"2024-05-08T23:02:17.659637+0000","last_clean":"2024-05-08T23:02:17.659637+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:02:17.659637+0000","last_undegraded":"2024-05-08T23:02:17.659637+0000","last_fullsized":"2024-05-08T23:02:17.659637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.655573+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:02:18.655573+0000","last_peered":"2024-05-08T23:02:18.655573+0000","last_clean":"2024-05-08T23:02:18.655573+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:02:18.655573+0000","last_undegraded":"2024-05-08T23:02:18.655573+0000","last_fullsized":"2024-05-08T23:02:18.655573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656812+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:02:17.656812+0000","last_peered":"2024-05-08T23:02:17.656812+0000","last_clean":"2024-05-08T23:02:17.656812+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:02:17.656812+0000","last_undegraded":"2024-05-08T23:02:17.656812+0000","last_fullsized":"2024-05-08T23:02:17.656812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.649950+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:02:17.649950+0000","last_peered":"2024-05-08T23:02:17.649950+0000","last_clean":"2024-05-08T23:02:17.649950+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:02:17.649950+0000","last_undegraded":"2024-05-08T23:02:17.649950+0000","last_fullsized":"2024-05-08T23:02:17.649950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:25.018945+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:02:25.018945+0000","last_peered":"2024-05-08T23:02:25.018945+0000","last_clean":"2024-05-08T23:02:25.018945+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:02:25.018945+0000","last_undegraded":"2024-05-08T23:02:25.018945+0000","last_fullsized":"2024-05-08T23:02:25.018945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650266+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:02:17.650266+0000","last_peered":"2024-05-08T23:02:17.650266+0000","last_clean":"2024-05-08T23:02:17.650266+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:02:17.650266+0000","last_undegraded":"2024-05-08T23:02:17.650266+0000","last_fullsized":"2024-05-08T23:02:17.650266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654622+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:02:17.654622+0000","last_peered":"2024-05-08T23:02:17.654622+0000","last_clean":"2024-05-08T23:02:17.654622+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:02:17.654622+0000","last_undegraded":"2024-05-08T23:02:17.654622+0000","last_fullsized":"2024-05-08T23:02:17.654622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.656885+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:02:18.656885+0000","last_peered":"2024-05-08T23:02:18.656885+0000","last_clean":"2024-05-08T23:02:18.656885+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:02:18.656885+0000","last_undegraded":"2024-05-08T23:02:18.656885+0000","last_fullsized":"2024-05-08T23:02:18.656885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661556+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:02:17.661556+0000","last_peered":"2024-05-08T23:02:17.661556+0000","last_clean":"2024-05-08T23:02:17.661556+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:02:17.661556+0000","last_undegraded":"2024-05-08T23:02:17.661556+0000","last_fullsized":"2024-05-08T23:02:17.661556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654068+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:02:17.654068+0000","last_peered":"2024-05-08T23:02:17.654068+0000","last_clean":"2024-05-08T23:02:17.654068+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:02:17.654068+0000","last_undegraded":"2024-05-08T23:02:17.654068+0000","last_fullsized":"2024-05-08T23:02:17.654068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659168+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:02:17.659168+0000","last_peered":"2024-05-08T23:02:17.659168+0000","last_clean":"2024-05-08T23:02:17.659168+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:02:17.659168+0000","last_undegraded":"2024-05-08T23:02:17.659168+0000","last_fullsized":"2024-05-08T23:02:17.659168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.653139+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:02:17.653139+0000","last_peered":"2024-05-08T23:02:17.653139+0000","last_clean":"2024-05-08T23:02:17.653139+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:02:17.653139+0000","last_undegraded":"2024-05-08T23:02:17.653139+0000","last_fullsized":"2024-05-08T23:02:17.653139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657027+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:02:17.657027+0000","last_peered":"2024-05-08T23:02:17.657027+0000","last_clean":"2024-05-08T23:02:17.657027+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:02:17.657027+0000","last_undegraded":"2024-05-08T23:02:17.657027+0000","last_fullsized":"2024-05-08T23:02:17.657027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.673909+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:02:17.673909+0000","last_peered":"2024-05-08T23:02:17.673909+0000","last_clean":"2024-05-08T23:02:17.673909+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:02:17.673909+0000","last_undegraded":"2024-05-08T23:02:17.673909+0000","last_fullsized":"2024-05-08T23:02:17.673909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659636+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:02:17.659636+0000","last_peered":"2024-05-08T23:02:17.659636+0000","last_clean":"2024-05-08T23:02:17.659636+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:02:17.659636+0000","last_undegraded":"2024-05-08T23:02:17.659636+0000","last_fullsized":"2024-05-08T23:02:17.659636+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.657221+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:02:18.657221+0000","last_peered":"2024-05-08T23:02:18.657221+0000","last_clean":"2024-05-08T23:02:18.657221+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:02:18.657221+0000","last_undegraded":"2024-05-08T23:02:18.657221+0000","last_fullsized":"2024-05-08T23:02:18.657221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.985257+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:02:18.985257+0000","last_peered":"2024-05-08T23:02:18.985257+0000","last_clean":"2024-05-08T23:02:18.985257+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:02:18.985257+0000","last_undegraded":"2024-05-08T23:02:18.985257+0000","last_fullsized":"2024-05-08T23:02:18.985257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.650347+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:02:17.650347+0000","last_peered":"2024-05-08T23:02:17.650347+0000","last_clean":"2024-05-08T23:02:17.650347+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:02:17.650347+0000","last_undegraded":"2024-05-08T23:02:17.650347+0000","last_fullsized":"2024-05-08T23:02:17.650347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654280+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:02:17.654280+0000","last_peered":"2024-05-08T23:02:17.654280+0000","last_clean":"2024-05-08T23:02:17.654280+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:02:17.654280+0000","last_undegraded":"2024-05-08T23:02:17.654280+0000","last_fullsized":"2024-05-08T23:02:17.654280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.654955+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:02:18.654955+0000","last_peered":"2024-05-08T23:02:18.654955+0000","last_clean":"2024-05-08T23:02:18.654955+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:02:18.654955+0000","last_undegraded":"2024-05-08T23:02:18.654955+0000","last_fullsized":"2024-05-08T23:02:18.654955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651054+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:02:17.651054+0000","last_peered":"2024-05-08T23:02:17.651054+0000","last_clean":"2024-05-08T23:02:17.651054+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:02:17.651054+0000","last_undegraded":"2024-05-08T23:02:17.651054+0000","last_fullsized":"2024-05-08T23:02:17.651054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659929+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:02:17.659929+0000","last_peered":"2024-05-08T23:02:17.659929+0000","last_clean":"2024-05-08T23:02:17.659929+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:02:17.659929+0000","last_undegraded":"2024-05-08T23:02:17.659929+0000","last_fullsized":"2024-05-08T23:02:17.659929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:03:00.699122+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:03:00.699122+0000","last_peered":"2024-05-08T23:03:00.699122+0000","last_clean":"2024-05-08T23:03:00.699122+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:03:00.699122+0000","last_undegraded":"2024-05-08T23:03:00.699122+0000","last_fullsized":"2024-05-08T23:03:00.699122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651348+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:02:17.651348+0000","last_peered":"2024-05-08T23:02:17.651348+0000","last_clean":"2024-05-08T23:02:17.651348+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:02:17.651348+0000","last_undegraded":"2024-05-08T23:02:17.651348+0000","last_fullsized":"2024-05-08T23:02:17.651348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661348+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:02:17.661348+0000","last_peered":"2024-05-08T23:02:17.661348+0000","last_clean":"2024-05-08T23:02:17.661348+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:02:17.661348+0000","last_undegraded":"2024-05-08T23:02:17.661348+0000","last_fullsized":"2024-05-08T23:02:17.661348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661447+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:02:18.661447+0000","last_peered":"2024-05-08T23:02:18.661447+0000","last_clean":"2024-05-08T23:02:18.661447+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:02:18.661447+0000","last_undegraded":"2024-05-08T23:02:18.661447+0000","last_fullsized":"2024-05-08T23:02:18.661447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651037+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:02:17.651037+0000","last_peered":"2024-05-08T23:02:17.651037+0000","last_clean":"2024-05-08T23:02:17.651037+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:02:17.651037+0000","last_undegraded":"2024-05-08T23:02:17.651037+0000","last_fullsized":"2024-05-08T23:02:17.651037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655866+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:02:17.655866+0000","last_peered":"2024-05-08T23:02:17.655866+0000","last_clean":"2024-05-08T23:02:17.655866+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:02:17.655866+0000","last_undegraded":"2024-05-08T23:02:17.655866+0000","last_fullsized":"2024-05-08T23:02:17.655866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.667814+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:02:18.667814+0000","last_peered":"2024-05-08T23:02:18.667814+0000","last_clean":"2024-05-08T23:02:18.667814+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:02:18.667814+0000","last_undegraded":"2024-05-08T23:02:18.667814+0000","last_fullsized":"2024-05-08T23:02:18.667814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651445+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:02:17.651445+0000","last_peered":"2024-05-08T23:02:17.651445+0000","last_clean":"2024-05-08T23:02:17.651445+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:02:17.651445+0000","last_undegraded":"2024-05-08T23:02:17.651445+0000","last_fullsized":"2024-05-08T23:02:17.651445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659729+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:02:17.659729+0000","last_peered":"2024-05-08T23:02:17.659729+0000","last_clean":"2024-05-08T23:02:17.659729+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:02:17.659729+0000","last_undegraded":"2024-05-08T23:02:17.659729+0000","last_fullsized":"2024-05-08T23:02:17.659729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.654952+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:02:17.654952+0000","last_peered":"2024-05-08T23:02:17.654952+0000","last_clean":"2024-05-08T23:02:17.654952+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:02:17.654952+0000","last_undegraded":"2024-05-08T23:02:17.654952+0000","last_fullsized":"2024-05-08T23:02:17.654952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656443+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:02:17.656443+0000","last_peered":"2024-05-08T23:02:17.656443+0000","last_clean":"2024-05-08T23:02:17.656443+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:02:17.656443+0000","last_undegraded":"2024-05-08T23:02:17.656443+0000","last_fullsized":"2024-05-08T23:02:17.656443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.661526+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:02:17.661526+0000","last_peered":"2024-05-08T23:02:17.661526+0000","last_clean":"2024-05-08T23:02:17.661526+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:02:17.661526+0000","last_undegraded":"2024-05-08T23:02:17.661526+0000","last_fullsized":"2024-05-08T23:02:17.661526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656909+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:02:17.656909+0000","last_peered":"2024-05-08T23:02:17.656909+0000","last_clean":"2024-05-08T23:02:17.656909+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:02:17.656909+0000","last_undegraded":"2024-05-08T23:02:17.656909+0000","last_fullsized":"2024-05-08T23:02:17.656909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.660321+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:02:17.660321+0000","last_peered":"2024-05-08T23:02:17.660321+0000","last_clean":"2024-05-08T23:02:17.660321+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:02:17.660321+0000","last_undegraded":"2024-05-08T23:02:17.660321+0000","last_fullsized":"2024-05-08T23:02:17.660321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.655679+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:02:17.655679+0000","last_peered":"2024-05-08T23:02:17.655679+0000","last_clean":"2024-05-08T23:02:17.655679+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:02:17.655679+0000","last_undegraded":"2024-05-08T23:02:17.655679+0000","last_fullsized":"2024-05-08T23:02:17.655679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:02:18.661450+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:02:18.661450+0000","last_peered":"2024-05-08T23:02:18.661450+0000","last_clean":"2024-05-08T23:02:18.661450+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:02:18.661450+0000","last_undegraded":"2024-05-08T23:02:18.661450+0000","last_fullsized":"2024-05-08T23:02:18.661450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.656262+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:02:17.656262+0000","last_peered":"2024-05-08T23:02:17.656262+0000","last_clean":"2024-05-08T23:02:17.656262+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:02:17.656262+0000","last_undegraded":"2024-05-08T23:02:17.656262+0000","last_fullsized":"2024-05-08T23:02:17.656262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.651835+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:02:17.651835+0000","last_peered":"2024-05-08T23:02:17.651835+0000","last_clean":"2024-05-08T23:02:17.651835+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:02:17.651835+0000","last_undegraded":"2024-05-08T23:02:17.651835+0000","last_fullsized":"2024-05-08T23:02:17.651835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659375+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:02:17.659375+0000","last_peered":"2024-05-08T23:02:17.659375+0000","last_clean":"2024-05-08T23:02:17.659375+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:02:17.659375+0000","last_undegraded":"2024-05-08T23:02:17.659375+0000","last_fullsized":"2024-05-08T23:02:17.659375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.657091+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:02:17.657091+0000","last_peered":"2024-05-08T23:02:17.657091+0000","last_clean":"2024-05-08T23:02:17.657091+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:02:17.657091+0000","last_undegraded":"2024-05-08T23:02:17.657091+0000","last_fullsized":"2024-05-08T23:02:17.657091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:02:17.659850+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:02:17.659850+0000","last_peered":"2024-05-08T23:02:17.659850+0000","last_clean":"2024-05-08T23:02:17.659850+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:02:17.659850+0000","last_undegraded":"2024-05-08T23:02:17.659850+0000","last_fullsized":"2024-05-08T23:02:17.659850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246317,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68071,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":448,"num_read_kb":298,"num_write":236,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204213,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":466334,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40699999999999997}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.629},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed May 8 23:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82099999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1114969,"data_compressed":55335,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.626}]},{"osd":2,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.873},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79300000000000004}]},{"osd":3,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.76300000000000001}]},{"osd":5,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.747},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":8,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.0089999999999999}]},{"osd":9,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.97099999999999997}]}]},{"osd":9,"up_from":61,"seq":261993005088,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":470386,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0069999999999999,"5min":1.357,"15min":1.357},"last":0.46000000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52300000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.747},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73799999999999999,"15min":0.749},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.875,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.95099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.874},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.90300000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.57999999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1299999999999999,"5min":1.738,"15min":1.738},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90200000000000002,"15min":0.91200000000000003},"min":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.53,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.63300000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1799999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.87},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.95799999999999996,"15min":0.96899999999999997},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.94599999999999995,"15min":0.96199999999999997},"min":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.71499999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.262,"5min":1.766,"15min":1.766},"last":0.84399999999999997}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.88200000000000001}]},{"osd":11,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.91600000000000004}]}]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465974,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.504}]},{"osd":1,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.079,"15min":1.079},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":1.139,"15min":1.139},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.069,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80800000000000005}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53800000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.91500000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.097,"15min":1.097},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77000000000000002}]},{"osd":11,"last update":"Wed May 8 23:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.78000000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462952,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1003520,"data_stored":1063023,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.45000000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38}]},{"osd":2,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60599999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":1.109,"15min":1.109},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79400000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.622},"min":{"1min":0.58899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88800000000000001,"5min":1.05,"15min":1.05},"last":0.626},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64200000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72599999999999998}]},{"osd":9,"last update":"Wed May 8 23:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.65000000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.075,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83899999999999997},"min":{"1min":0.68400000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.03,"5min":1.141,"15min":1.141},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98599999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74199999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82699999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626477,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":929792,"data_stored":461746,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.53200000000000003,"15min":0.48999999999999999},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90000000000000002,"5min":1.103,"15min":1.103},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.443,"15min":0.443},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.1559999999999999}]},{"osd":3,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.61299999999999999,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":1.581,"15min":1.581},"last":0.623}]},{"osd":4,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.75900000000000001,"15min":0.73699999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.502},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.73399999999999999,"15min":0.70399999999999996},"min":{"1min":0.503,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.0369999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.76500000000000001,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.72099999999999997},"min":{"1min":0.624,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.206,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed May 8 23:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.79600000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80900000000000005,"15min":0.78000000000000003},"min":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.0780000000000001}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.90000000000000002,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.8610000000000002,"5min":2.8610000000000002,"15min":2.8610000000000002},"last":1.194},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.006,"15min":0.98499999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.002,"15min":1.0109999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.82999999999999996}]},{"osd":11,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.9710000000000001,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":524587,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.46000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed May 8 23:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79800000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.53600000000000003},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.316}]},{"osd":3,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.437,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed May 8 23:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.45300000000000001,"15min":0.41099999999999998},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.49399999999999999,"15min":0.44600000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.03,"15min":1.03},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.73999999999999999,"15min":0.68899999999999995},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.64900000000000002},"min":{"1min":0.53400000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.503}]},{"osd":7,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.53300000000000003}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.052,"5min":1.071,"15min":1.071},"last":0.621}]},{"osd":9,"last update":"Wed May 8 23:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.88500000000000001,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.82699999999999996,"15min":0.81899999999999995},"min":{"1min":0.66700000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.71899999999999997}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.69299999999999995}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":463202,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.59599999999999997,"15min":0.53800000000000003},"min":{"1min":0.47999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.58499999999999996,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.57099999999999995}]},{"osd":2,"last update":"Wed May 8 23:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.62,"15min":0.54000000000000004},"min":{"1min":0.44400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":6.8710000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58699999999999997,"15min":0.53600000000000003},"min":{"1min":0.42199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":6.8849999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed May 8 23:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.69299999999999995,"15min":0.65700000000000003},"min":{"1min":0.57199999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":6.9139999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":6.9450000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67500000000000004,"15min":0.64100000000000001},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.55,"5min":1.921,"15min":1.921},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.76100000000000001,"15min":0.71499999999999997},"min":{"1min":0.57899999999999996,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.68600000000000005}]},{"osd":7,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":6.9630000000000001,"5min":6.9630000000000001,"15min":6.9630000000000001},"last":6.9630000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.76300000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.92400000000000004}]},{"osd":9,"last update":"Wed May 8 23:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":0.96099999999999997},"min":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.673,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.871},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.94099999999999995,"15min":0.92900000000000005},"min":{"1min":0.69299999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.82199999999999995}]},{"osd":10,"last update":"Wed May 8 23:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.90700000000000003}]},{"osd":11,"last update":"Wed May 8 23:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":995328,"data_stored":522296,"data_compressed":19444,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.87,"15min":0.87},"last":0.495},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.876,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.873,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.57799999999999996},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed May 8 23:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.35099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45500000000000002,"15min":0.44},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47199999999999998}]},{"osd":6,"last update":"Wed May 8 23:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.66600000000000004},"min":{"1min":0.55200000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89700000000000002,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.62}]},{"osd":7,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64800000000000002}]},{"osd":8,"last update":"Wed May 8 23:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73699999999999999}]},{"osd":9,"last update":"Wed May 8 23:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.70799999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.75900000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":461193,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46999999999999997,"15min":0.42399999999999999},"min":{"1min":0.35899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63700000000000001,"15min":0.56399999999999995},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66000000000000003,"15min":0.60299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":1.046,"15min":1.046},"last":0.81999999999999995}]},{"osd":2,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.47799999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.67000000000000004,"15min":0.60999999999999999},"min":{"1min":0.49299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.84899999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68799999999999994,"15min":0.64400000000000002},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.71999999999999997,"15min":0.64700000000000002},"min":{"1min":0.58599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Wed May 8 23:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71799999999999997,"15min":0.68300000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.55900000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.169,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.752,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.077,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":13.807,"5min":13.807,"15min":13.807},"last":0.879},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.91700000000000004}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94699999999999995,"15min":0.95999999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.96999999999999997,"15min":0.98799999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1299999999999999,"5min":1.292,"15min":1.292},"last":0.70699999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.0609999999999999,"15min":0.97399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.893000000000001,"5min":13.893000000000001,"15min":13.893000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.01,"15min":1.014},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.98899999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.93300000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":942080,"data_stored":467026,"data_compressed":17595,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.41999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.875}]},{"osd":3,"last update":"Wed May 8 23:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.54000000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78400000000000003}]},{"osd":6,"last update":"Wed May 8 23:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.65700000000000003},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88300000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89200000000000002,"5min":1.248,"15min":1.248},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.57699999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":1.107,"15min":1.107},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0469999999999999,"5min":1.147,"15min":1.147},"last":0.55000000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.042,"15min":1.042},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.79400000000000004},"min":{"1min":0.59299999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96399999999999997,"5min":1.05,"15min":1.05},"last":0.67700000000000005}]},{"osd":9,"last update":"Wed May 8 23:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.90600000000000003,"15min":0.92600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.99299999999999999,"5min":1.391,"15min":1.391},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89700000000000002,"15min":0.90700000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.089,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.91100000000000003}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.95899999999999996}]}]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1058027,"data_compressed":53486,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.41999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.502,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.315,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed May 8 23:02:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.79900000000000004}]},{"osd":6,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69899999999999995}]},{"osd":7,"last update":"Wed May 8 23:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.746},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.75600000000000001}]},{"osd":8,"last update":"Wed May 8 23:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed May 8 23:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72899999999999998}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34251,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63824,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62202,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63529,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:03:04.937 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T23:03:04.937 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:03:04.938 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:03:04.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T23:03:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:05 smithi053 ceph-mon[77587]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:05.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:05 smithi053 ceph-mon[77587]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:05.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:05 smithi046 ceph-mon[36898]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:05.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:05 smithi046 ceph-mon[36898]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:05.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:05 smithi086 ceph-mon[78610]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:05.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:05 smithi086 ceph-mon[78610]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:06.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 -- 172.21.15.46:0/1838972554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 msgr2=0x7fa86410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:06.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 --2- 172.21.15.46:0/1838972554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa86410de60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00b0a0 tx=0x7fa85c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 -- 172.21.15.46:0/1838972554 shutdown_connections 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 --2- 172.21.15.46:0/1838972554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa86410de60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 --2- 172.21.15.46:0/1838972554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 0x7fa864107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 --2- 172.21.15.46:0/1838972554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 0x7fa864068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 -- 172.21.15.46:0/1838972554 >> 172.21.15.46:0/1838972554 conn(0x7fa864075f80 msgr2=0x7fa864076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 -- 172.21.15.46:0/1838972554 shutdown_connections 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.517+0000 7fa86a400700 1 -- 172.21.15.46:0/1838972554 wait complete. 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.518+0000 7fa86a400700 1 Processor -- start 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.518+0000 7fa86a400700 1 -- start start 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 0x7fa864197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 0x7fa86419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:06.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86419c400 con 0x7fa8640692f0 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86419c570 con 0x7fa864107020 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa86a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86419c730 con 0x7fa864068950 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54444/0 (socket says 172.21.15.46:54444) 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 -- 172.21.15.46:0/3043817167 learned_addr learned my addr 172.21.15.46:0/3043817167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa863400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 0x7fa864197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa863e00700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 0x7fa86419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 -- 172.21.15.46:0/3043817167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 msgr2=0x7fa864197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 0x7fa864197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 -- 172.21.15.46:0/3043817167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 msgr2=0x7fa86419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.519+0000 7fa862a00700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 0x7fa86419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa862a00700 1 -- 172.21.15.46:0/3043817167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009d00 con 0x7fa8640692f0 2024-05-08T23:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa862a00700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9a0 tx=0x7fa85000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850009790 con 0x7fa8640692f0 2024-05-08T23:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa850010430 con 0x7fa8640692f0 2024-05-08T23:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86419ca10 con 0x7fa8640692f0 2024-05-08T23:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850018680 con 0x7fa8640692f0 2024-05-08T23:03:06.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.520+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8641a4270 con 0x7fa8640692f0 2024-05-08T23:03:06.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.521+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa864109540 con 0x7fa8640692f0 2024-05-08T23:03:06.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.524+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa8500105a0 con 0x7fa8640692f0 2024-05-08T23:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.525+0000 7fa85be00700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 0x7fa8480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.525+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa850014070 con 0x7fa8640692f0 2024-05-08T23:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.525+0000 7fa863400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 0x7fa8480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.526+0000 7fa863400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 0x7fa8480a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa854007b60 tx=0x7fa8540073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:06.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.530+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa850052d60 con 0x7fa8640692f0 2024-05-08T23:03:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.793+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa86404ec90 con 0x7fa8640692f0 2024-05-08T23:03:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.793+0000 7fa85be00700 1 -- 172.21.15.46:0/3043817167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa850096260 con 0x7fa8640692f0 2024-05-08T23:03:06.792 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:06.793 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 msgr2=0x7fa8480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 0x7fa8480a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa854007b60 tx=0x7fa8540073d0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 msgr2=0x7fa864197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9a0 tx=0x7fa85000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 shutdown_connections 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa864107020 0x7fa86419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864068950 0x7fa864197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa8480a11b0 0x7fa8480a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 --2- 172.21.15.46:0/3043817167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8640692f0 0x7fa864197a90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:06.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 >> 172.21.15.46:0/3043817167 conn(0x7fa864075f80 msgr2=0x7fa8640fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:06.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 shutdown_connections 2024-05-08T23:03:06.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:06.796+0000 7fa86a400700 1 -- 172.21.15.46:0/3043817167 wait complete. 2024-05-08T23:03:07.152 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:06 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3043817167' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:03:07.184 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:03:07.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:07.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:06 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3043817167' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:03:07.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:06 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3043817167' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:03:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 -- 172.21.15.46:0/1425355281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 msgr2=0x7efe401012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 --2- 172.21.15.46:0/1425355281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe401012a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe34009990 tx=0x7efe34009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 -- 172.21.15.46:0/1425355281 shutdown_connections 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 --2- 172.21.15.46:0/1425355281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40102990 0x7efe401092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 --2- 172.21.15.46:0/1425355281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe401012a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 --2- 172.21.15.46:0/1425355281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40101fd0 0x7efe40102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 -- 172.21.15.46:0/1425355281 >> 172.21.15.46:0/1425355281 conn(0x7efe400fc410 msgr2=0x7efe400fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 -- 172.21.15.46:0/1425355281 shutdown_connections 2024-05-08T23:03:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.388+0000 7efe47a00700 1 -- 172.21.15.46:0/1425355281 wait complete. 2024-05-08T23:03:07.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.389+0000 7efe47a00700 1 Processor -- start 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.389+0000 7efe47a00700 1 -- start start 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 0x7efe40197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40102990 0x7efe4019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4019e970 con 0x7efe40100e80 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4019eab0 con 0x7efe40101fd0 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe47a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4019ec40 con 0x7efe40102990 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe45000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe3fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 0x7efe40197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe45000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54464/0 (socket says 172.21.15.46:54464) 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe45000700 1 -- 172.21.15.46:0/2279647957 learned_addr learned my addr 172.21.15.46:0/2279647957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe3fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 0x7efe40197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41826/0 (socket says 172.21.15.46:41826) 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.390+0000 7efe45a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40102990 0x7efe4019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 -- 172.21.15.46:0/2279647957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40102990 msgr2=0x7efe4019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40102990 0x7efe4019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 -- 172.21.15.46:0/2279647957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 msgr2=0x7efe40197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 0x7efe40197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 -- 172.21.15.46:0/2279647957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38009c40 con 0x7efe40100e80 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe45000700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b620 tx=0x7efe34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3401d070 con 0x7efe40100e80 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe34022470 con 0x7efe40100e80 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe340095f0 con 0x7efe40100e80 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34017650 con 0x7efe40100e80 2024-05-08T23:03:07.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.391+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4019f1b0 con 0x7efe40100e80 2024-05-08T23:03:07.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.392+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe40108340 con 0x7efe40100e80 2024-05-08T23:03:07.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.394+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efe340225e0 con 0x7efe40100e80 2024-05-08T23:03:07.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.395+0000 7efe3d600700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 0x7efe280a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.396+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe340ce3b0 con 0x7efe40100e80 2024-05-08T23:03:07.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.396+0000 7efe3fe00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 0x7efe280a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.396+0000 7efe3fe00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 0x7efe280a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009c60 tx=0x7efe2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:07.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.398+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efe34097700 con 0x7efe40100e80 2024-05-08T23:03:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.621+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe40067d10 con 0x7efe40100e80 2024-05-08T23:03:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.623+0000 7efe3d600700 1 -- 172.21.15.46:0/2279647957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7efe3409ad20 con 0x7efe40100e80 2024-05-08T23:03:07.622 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:07.622 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:07.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 msgr2=0x7efe280a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 0x7efe280a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009c60 tx=0x7efe2c009260 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 msgr2=0x7efe40196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b620 tx=0x7efe34004730 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 shutdown_connections 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe40102990 0x7efe4019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe280a1230 0x7efe280a36f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe40100e80 0x7efe40196e40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 --2- 172.21.15.46:0/2279647957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe40101fd0 0x7efe40197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.625+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 >> 172.21.15.46:0/2279647957 conn(0x7efe400fc410 msgr2=0x7efe40105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.626+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 shutdown_connections 2024-05-08T23:03:07.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.626+0000 7efe47a00700 1 -- 172.21.15.46:0/2279647957 wait complete. 2024-05-08T23:03:07.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:07.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 -- 172.21.15.46:0/2451961279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0101740 msgr2=0x7fbec0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 --2- 172.21.15.46:0/2451961279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0101740 0x7fbec0103b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009990 tx=0x7fbeb0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 -- 172.21.15.46:0/2451961279 shutdown_connections 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 --2- 172.21.15.46:0/2451961279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 0x7fbec0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 --2- 172.21.15.46:0/2451961279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0101740 0x7fbec0103b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 --2- 172.21.15.46:0/2451961279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0104070 0x7fbec0106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 -- 172.21.15.46:0/2451961279 >> 172.21.15.46:0/2451961279 conn(0x7fbec00fb130 msgr2=0x7fbec00fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 -- 172.21.15.46:0/2451961279 shutdown_connections 2024-05-08T23:03:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.851+0000 7fbec4a00700 1 -- 172.21.15.46:0/2451961279 wait complete. 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.852+0000 7fbec4a00700 1 Processor -- start 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.852+0000 7fbec4a00700 1 -- start start 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 0x7fbec0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 0x7fbec019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019e640 con 0x7fbec0101740 2024-05-08T23:03:07.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019e780 con 0x7fbec01069a0 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbec4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec019e910 con 0x7fbec0104070 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54482/0 (socket says 172.21.15.46:54482) 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 -- 172.21.15.46:0/1920840222 learned_addr learned my addr 172.21.15.46:0/1920840222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebea00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 0x7fbec0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebfe00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 0x7fbec019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 -- 172.21.15.46:0/1920840222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 msgr2=0x7fbec0197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 0x7fbec0197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 -- 172.21.15.46:0/1920840222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 msgr2=0x7fbec019e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 0x7fbec019e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebf400700 1 -- 172.21.15.46:0/1920840222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00095f0 con 0x7fbec0101740 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.853+0000 7fbebea00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 0x7fbec0197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbebf400700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005910 tx=0x7fbeb0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001d070 con 0x7fbec0101740 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb0021410 con 0x7fbec0101740 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec019eb90 con 0x7fbec0101740 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0005720 con 0x7fbec0101740 2024-05-08T23:03:07.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.854+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec019f010 con 0x7fbec0101740 2024-05-08T23:03:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.856+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec00fccb0 con 0x7fbec0101740 2024-05-08T23:03:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.857+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbeb0021580 con 0x7fbec0101740 2024-05-08T23:03:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.859+0000 7fbeb7e00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 0x7fbea40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.859+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbeb00ce0e0 con 0x7fbec0101740 2024-05-08T23:03:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.859+0000 7fbebea00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 0x7fbea40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.860+0000 7fbebea00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 0x7fbea40a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbeac011490 tx=0x7fbeac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:07.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:07.863+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbeb0097430 con 0x7fbec0101740 2024-05-08T23:03:08.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:07 smithi046 ceph-mon[36898]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2279647957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.085+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbec010c510 con 0x7fbec0101740 2024-05-08T23:03:08.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.085+0000 7fbeb7e00700 1 -- 172.21.15.46:0/1920840222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeb009aa50 con 0x7fbec0101740 2024-05-08T23:03:08.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 msgr2=0x7fbea40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 0x7fbea40a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbeac011490 tx=0x7fbeac009260 comp rx=0 tx=0).stop 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 msgr2=0x7fbec0196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005910 tx=0x7fbeb0009580 comp rx=0 tx=0).stop 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 shutdown_connections 2024-05-08T23:03:08.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec01069a0 0x7fbec019e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fbea40a12e0 0x7fbea40a37a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec0101740 0x7fbec0196b10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 --2- 172.21.15.46:0/1920840222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbec0104070 0x7fbec0197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 >> 172.21.15.46:0/1920840222 conn(0x7fbec00fb130 msgr2=0x7fbec01020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 shutdown_connections 2024-05-08T23:03:08.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.088+0000 7fbec4a00700 1 -- 172.21.15.46:0/1920840222 wait complete. 2024-05-08T23:03:08.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:07 smithi086 ceph-mon[78610]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:07 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2279647957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:07 smithi053 ceph-mon[77587]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:07 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2279647957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 -- 172.21.15.46:0/3963169327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74101fd0 msgr2=0x7f7a74102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 --2- 172.21.15.46:0/3963169327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74101fd0 0x7f7a74102450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009990 tx=0x7f7a60009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 -- 172.21.15.46:0/3963169327 shutdown_connections 2024-05-08T23:03:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 --2- 172.21.15.46:0/3963169327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74102990 0x7f7a741092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 --2- 172.21.15.46:0/3963169327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74100e80 0x7f7a741012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 --2- 172.21.15.46:0/3963169327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74101fd0 0x7f7a74102450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 -- 172.21.15.46:0/3963169327 >> 172.21.15.46:0/3963169327 conn(0x7f7a740fc3b0 msgr2=0x7f7a740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 -- 172.21.15.46:0/3963169327 shutdown_connections 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.311+0000 7f7a79c00700 1 -- 172.21.15.46:0/3963169327 wait complete. 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.312+0000 7f7a79c00700 1 Processor -- start 2024-05-08T23:03:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 -- start start 2024-05-08T23:03:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 0x7f7a74196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74101fd0 0x7f7a74197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419e940 con 0x7f7a74102990 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419ea80 con 0x7f7a74101fd0 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a79c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419ec10 con 0x7f7a74100e80 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a73400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a72a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 0x7f7a74196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a73400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54498/0 (socket says 172.21.15.46:54498) 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a73400700 1 -- 172.21.15.46:0/325607584 learned_addr learned my addr 172.21.15.46:0/325607584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.313+0000 7f7a72000700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74101fd0 0x7f7a74197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 -- 172.21.15.46:0/325607584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 msgr2=0x7f7a74196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 0x7f7a74196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 -- 172.21.15.46:0/325607584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74101fd0 msgr2=0x7f7a74197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74101fd0 0x7f7a74197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 -- 172.21.15.46:0/325607584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009520 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a72a00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 0x7f7a74196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a73400700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00ef30 tx=0x7f7a6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c018070 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6c00f040 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a600095f0 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00cd00 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.314+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7419f1c0 con 0x7f7a74102990 2024-05-08T23:03:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.316+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a741082f0 con 0x7f7a74102990 2024-05-08T23:03:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.317+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7a6c003a00 con 0x7f7a74102990 2024-05-08T23:03:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.319+0000 7f7a6b400700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 0x7f7a580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.319+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7a6c0cc890 con 0x7f7a74102990 2024-05-08T23:03:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.319+0000 7f7a72a00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 0x7f7a580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.320+0000 7f7a72a00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 0x7f7a580a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009fd0 tx=0x7f7a64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:08.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.323+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a6c095be0 con 0x7f7a74102990 2024-05-08T23:03:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.541+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a7410c9b0 con 0x7f7a74102990 2024-05-08T23:03:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.542+0000 7f7a6b400700 1 -- 172.21.15.46:0/325607584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a6c099200 con 0x7f7a74102990 2024-05-08T23:03:08.541 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 msgr2=0x7f7a580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 0x7f7a580a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009fd0 tx=0x7f7a64009260 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 msgr2=0x7f7a7419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00ef30 tx=0x7f7a6c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 shutdown_connections 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7a580a1230 0x7f7a580a36f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a74102990 0x7f7a7419e370 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74100e80 0x7f7a74196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 --2- 172.21.15.46:0/325607584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a74101fd0 0x7f7a74197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 >> 172.21.15.46:0/325607584 conn(0x7f7a740fc3b0 msgr2=0x7f7a74105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 shutdown_connections 2024-05-08T23:03:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.544+0000 7f7a79c00700 1 -- 172.21.15.46:0/325607584 wait complete. 2024-05-08T23:03:08.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 -- 172.21.15.46:0/638012890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 msgr2=0x7f2750102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 --2- 172.21.15.46:0/638012890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750102450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f273c009990 tx=0x7f273c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 -- 172.21.15.46:0/638012890 shutdown_connections 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 --2- 172.21.15.46:0/638012890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750102990 0x7f27501092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 --2- 172.21.15.46:0/638012890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750100e80 0x7f27501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 --2- 172.21.15.46:0/638012890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750102450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.765+0000 7f2757000700 1 -- 172.21.15.46:0/638012890 >> 172.21.15.46:0/638012890 conn(0x7f27500fc3b0 msgr2=0x7f27500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:08.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.766+0000 7f2757000700 1 -- 172.21.15.46:0/638012890 shutdown_connections 2024-05-08T23:03:08.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.766+0000 7f2757000700 1 -- 172.21.15.46:0/638012890 wait complete. 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 Processor -- start 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 -- start start 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750102990 0x7f27500738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2750074070 con 0x7f2750101fd0 2024-05-08T23:03:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500741e0 con 0x7f2750100e80 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2757000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27501a0e30 con 0x7f2750102990 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f274f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f274fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f274f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54514/0 (socket says 172.21.15.46:54514) 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f274fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41876/0 (socket says 172.21.15.46:41876) 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f274f400700 1 -- 172.21.15.46:0/1673753835 learned_addr learned my addr 172.21.15.46:0/1673753835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.767+0000 7f2755000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750102990 0x7f27500738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 -- 172.21.15.46:0/1673753835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750102990 msgr2=0x7f27500738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750102990 0x7f27500738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 -- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 msgr2=0x7f2750073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 -- 172.21.15.46:0/1673753835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009c40 con 0x7f2750100e80 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274f400700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:08.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274fe00700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f274000e9f0 tx=0x7f274000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000c960 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c0095f0 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2740010430 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740018680 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.768+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501a1440 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.769+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27501082a0 con 0x7f2750100e80 2024-05-08T23:03:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.772+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f27400105a0 con 0x7f2750100e80 2024-05-08T23:03:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.773+0000 7f274cc00700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.773+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2740014070 con 0x7f2750100e80 2024-05-08T23:03:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.773+0000 7f274f400700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.774+0000 7f274f400700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 0x7f27340a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f273c00b2e0 tx=0x7f273c00b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.775+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2740096840 con 0x7f2750100e80 2024-05-08T23:03:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.995+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f275010c6d0 con 0x7f2750100e80 2024-05-08T23:03:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.996+0000 7f274cc00700 1 -- 172.21.15.46:0/1673753835 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2740099e60 con 0x7f2750100e80 2024-05-08T23:03:08.995 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:08.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 msgr2=0x7f27340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 0x7f27340a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f273c00b2e0 tx=0x7f273c00b6f0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 msgr2=0x7f2750074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f274000e9f0 tx=0x7f274000ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 shutdown_connections 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750102990 0x7f27500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2750100e80 0x7f2750074d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f27340a11b0 0x7f27340a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 --2- 172.21.15.46:0/1673753835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2750101fd0 0x7f2750073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 >> 172.21.15.46:0/1673753835 conn(0x7f27500fc3b0 msgr2=0x7f27501053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 shutdown_connections 2024-05-08T23:03:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:08.998+0000 7f2757000700 1 -- 172.21.15.46:0/1673753835 wait complete. 2024-05-08T23:03:09.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:08 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1920840222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:08 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/325607584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 -- 172.21.15.46:0/955872098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314073a00 msgr2=0x7f3314073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 --2- 172.21.15.46:0/955872098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314073a00 0x7f3314073e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3300009990 tx=0x7f3300009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 -- 172.21.15.46:0/955872098 shutdown_connections 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 --2- 172.21.15.46:0/955872098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f331410d450 0x7f331410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 --2- 172.21.15.46:0/955872098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3314074fe0 0x7f3314073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 --2- 172.21.15.46:0/955872098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314073a00 0x7f3314073e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 -- 172.21.15.46:0/955872098 >> 172.21.15.46:0/955872098 conn(0x7f33140fc3b0 msgr2=0x7f33140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 -- 172.21.15.46:0/955872098 shutdown_connections 2024-05-08T23:03:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.220+0000 7f3319400700 1 -- 172.21.15.46:0/955872098 wait complete. 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.222+0000 7f3319400700 1 Processor -- start 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.222+0000 7f3319400700 1 -- start start 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.222+0000 7f3319400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 0x7f331419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.222+0000 7f3319400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 0x7f331419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.222+0000 7f3319400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 0x7f33141a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3319400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141a4f10 con 0x7f3314074fe0 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3319400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141a5050 con 0x7f3314073a00 2024-05-08T23:03:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3319400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141a51e0 con 0x7f331410d450 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3311e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 0x7f331419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 0x7f331419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3313200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 0x7f33141a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3313200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 0x7f33141a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35976/0 (socket says 172.21.15.46:35976) 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3313200700 1 -- 172.21.15.46:0/1639954218 learned_addr learned my addr 172.21.15.46:0/1639954218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 -- 172.21.15.46:0/1639954218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 msgr2=0x7f33141a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 0x7f33141a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 -- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 msgr2=0x7f331419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 0x7f331419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3312800700 1 -- 172.21.15.46:0/1639954218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000095f0 con 0x7f3314073a00 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.223+0000 7f3311e00700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 0x7f331419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:09.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f3312800700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 0x7f331419d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f330400c970 tx=0x7f330400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304004440 con 0x7f3314073a00 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3304003b40 con 0x7f3314073a00 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33141a54c0 con 0x7f3314073a00 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304007bc0 con 0x7f3314073a00 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.224+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141a59d0 con 0x7f3314073a00 2024-05-08T23:03:09.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.225+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33141036b0 con 0x7f3314073a00 2024-05-08T23:03:09.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.227+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3304003cb0 con 0x7f3314073a00 2024-05-08T23:03:09.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.228+0000 7f330b400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 0x7f32f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.228+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f33040cccf0 con 0x7f3314073a00 2024-05-08T23:03:09.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.229+0000 7f3311e00700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 0x7f32f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.229+0000 7f3311e00700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 0x7f32f80a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c00 tx=0x7f3300005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:09.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.233+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3304095fb0 con 0x7f3314073a00 2024-05-08T23:03:09.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:08 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1920840222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:08 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/325607584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1920840222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/325607584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:09.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.454+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3314112e70 con 0x7f3314073a00 2024-05-08T23:03:09.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.454+0000 7f330b400700 1 -- 172.21.15.46:0/1639954218 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33040140a0 con 0x7f3314073a00 2024-05-08T23:03:09.454 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:09.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 msgr2=0x7f32f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 0x7f32f80a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c00 tx=0x7f3300005fb0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 msgr2=0x7f331419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 0x7f331419d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f330400c970 tx=0x7f330400cc80 comp rx=0 tx=0).stop 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 shutdown_connections 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331410d450 0x7f33141a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3314073a00 0x7f331419d3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f32f80a11b0 0x7f32f80a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 --2- 172.21.15.46:0/1639954218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3314074fe0 0x7f331419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 >> 172.21.15.46:0/1639954218 conn(0x7f33140fc3b0 msgr2=0x7f33141007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 shutdown_connections 2024-05-08T23:03:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.457+0000 7f3319400700 1 -- 172.21.15.46:0/1639954218 wait complete. 2024-05-08T23:03:09.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 -- 172.21.15.46:0/3128109777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 msgr2=0x7fde68108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 --2- 172.21.15.46:0/3128109777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde68108fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde6000b0a0 tx=0x7fde6000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 -- 172.21.15.46:0/3128109777 shutdown_connections 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 --2- 172.21.15.46:0/3128109777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde68108fd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 --2- 172.21.15.46:0/3128109777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68101900 0x7fde68103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 --2- 172.21.15.46:0/3128109777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68104230 0x7fde68106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.674+0000 7fde6d000700 1 -- 172.21.15.46:0/3128109777 >> 172.21.15.46:0/3128109777 conn(0x7fde680fb310 msgr2=0x7fde680fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.675+0000 7fde6d000700 1 -- 172.21.15.46:0/3128109777 shutdown_connections 2024-05-08T23:03:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.675+0000 7fde6d000700 1 -- 172.21.15.46:0/3128109777 wait complete. 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.676+0000 7fde6d000700 1 Processor -- start 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.676+0000 7fde6d000700 1 -- start start 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68104230 0x7fde68199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde681a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde681a0c20 con 0x7fde68101900 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde681a0d60 con 0x7fde68104230 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde6d000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde681a0ef0 con 0x7fde68106b60 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54558/0 (socket says 172.21.15.46:54558) 2024-05-08T23:03:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 -- 172.21.15.46:0/3188832958 learned_addr learned my addr 172.21.15.46:0/3188832958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde66800700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde681a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65400700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68104230 0x7fde68199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 -- 172.21.15.46:0/3188832958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 msgr2=0x7fde681a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde681a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 -- 172.21.15.46:0/3188832958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68104230 msgr2=0x7fde68199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68104230 0x7fde68199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.677+0000 7fde65e00700 1 -- 172.21.15.46:0/3188832958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde54009520 con 0x7fde68101900 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde65e00700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e9c0 tx=0x7fde5800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580097b0 con 0x7fde68101900 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5800ee60 con 0x7fde68101900 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde60009d00 con 0x7fde68101900 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580054f0 con 0x7fde68101900 2024-05-08T23:03:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.678+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde681a1500 con 0x7fde68101900 2024-05-08T23:03:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.679+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde681086e0 con 0x7fde68101900 2024-05-08T23:03:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.681+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fde58004750 con 0x7fde68101900 2024-05-08T23:03:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.682+0000 7fde5ea00700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 0x7fde4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.682+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fde58014070 con 0x7fde68101900 2024-05-08T23:03:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.683+0000 7fde65400700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 0x7fde4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.683+0000 7fde65400700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 0x7fde4c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde54009df0 tx=0x7fde54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.687+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fde580962f0 con 0x7fde68101900 2024-05-08T23:03:09.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.955+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde68067d10 con 0x7fde68101900 2024-05-08T23:03:09.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.956+0000 7fde5ea00700 1 -- 172.21.15.46:0/3188832958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fde58099910 con 0x7fde68101900 2024-05-08T23:03:09.955 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:09.955 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:09.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 msgr2=0x7fde4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 0x7fde4c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde54009df0 tx=0x7fde54009260 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 msgr2=0x7fde681990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e9c0 tx=0x7fde5800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 shutdown_connections 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde68106b60 0x7fde681a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fde4c0a11b0 0x7fde4c0a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde68101900 0x7fde681990f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 --2- 172.21.15.46:0/3188832958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde68104230 0x7fde68199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 >> 172.21.15.46:0/3188832958 conn(0x7fde680fb310 msgr2=0x7fde68101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 shutdown_connections 2024-05-08T23:03:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:09.958+0000 7fde6d000700 1 -- 172.21.15.46:0/3188832958 wait complete. 2024-05-08T23:03:09.959 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:09.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 -- 172.21.15.46:0/3056835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 msgr2=0x7f57c01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 --2- 172.21.15.46:0/3056835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c01012a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009990 tx=0x7f57b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 -- 172.21.15.46:0/3056835478 shutdown_connections 2024-05-08T23:03:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 --2- 172.21.15.46:0/3056835478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 0x7f57c01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 --2- 172.21.15.46:0/3056835478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c01012a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 --2- 172.21.15.46:0/3056835478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 0x7f57c0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 -- 172.21.15.46:0/3056835478 >> 172.21.15.46:0/3056835478 conn(0x7f57c00fc410 msgr2=0x7f57c00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 -- 172.21.15.46:0/3056835478 shutdown_connections 2024-05-08T23:03:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.176+0000 7f57c7600700 1 -- 172.21.15.46:0/3056835478 wait complete. 2024-05-08T23:03:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.177+0000 7f57c7600700 1 Processor -- start 2024-05-08T23:03:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.177+0000 7f57c7600700 1 -- start start 2024-05-08T23:03:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 0x7f57c0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 0x7f57c00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0073fd0 con 0x7f57c0100e80 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0074140 con 0x7f57c0101fd0 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c7600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a0eb0 con 0x7f57c0102990 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54578/0 (socket says 172.21.15.46:54578) 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 -- 172.21.15.46:0/232351557 learned_addr learned my addr 172.21.15.46:0/232351557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57bfe00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 0x7f57c0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c5600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 0x7f57c00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 -- 172.21.15.46:0/232351557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 msgr2=0x7f57c00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 0x7f57c00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 -- 172.21.15.46:0/232351557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 msgr2=0x7f57c0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 0x7f57c0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.178+0000 7f57c4c00700 1 -- 172.21.15.46:0/232351557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40095f0 con 0x7f57c0100e80 2024-05-08T23:03:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57c4c00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000c00 tx=0x7f57b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b401d070 con 0x7f57c0100e80 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b4021410 con 0x7f57c0100e80 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c01a10b0 con 0x7f57c0100e80 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4017650 con 0x7f57c0100e80 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.179+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c01a14d0 con 0x7f57c0100e80 2024-05-08T23:03:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.180+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c0108240 con 0x7f57c0100e80 2024-05-08T23:03:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.182+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f57b4021580 con 0x7f57c0100e80 2024-05-08T23:03:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.183+0000 7f57bd600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 0x7f57a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.183+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57b40ce160 con 0x7f57c0100e80 2024-05-08T23:03:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.184+0000 7f57bfe00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 0x7f57a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.184+0000 7f57bfe00700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 0x7f57a80a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009c60 tx=0x7f57ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:10.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.187+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57b40974b0 con 0x7f57c0100e80 2024-05-08T23:03:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:09 smithi086 ceph-mon[78610]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:03:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:09 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1673753835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:09 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1639954218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:09 smithi053 ceph-mon[77587]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:03:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:09 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1673753835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:09 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1639954218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:09 smithi046 ceph-mon[36898]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:03:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1673753835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1639954218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:10.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.409+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57c0067d10 con 0x7f57c0100e80 2024-05-08T23:03:10.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.411+0000 7f57bd600700 1 -- 172.21.15.46:0/232351557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7f57b409aad0 con 0x7f57c0100e80 2024-05-08T23:03:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:10.411 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 msgr2=0x7f57a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 0x7f57a80a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009c60 tx=0x7f57ac009260 comp rx=0 tx=0).stop 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 msgr2=0x7f57c0074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000c00 tx=0x7f57b4005740 comp rx=0 tx=0).stop 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 shutdown_connections 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0102990 0x7f57c00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f57a80a11b0 0x7f57a80a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c0100e80 0x7f57c0074cb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 --2- 172.21.15.46:0/232351557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57c0101fd0 0x7f57c0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 >> 172.21.15.46:0/232351557 conn(0x7f57c00fc410 msgr2=0x7f57c0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 shutdown_connections 2024-05-08T23:03:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.414+0000 7f57c7600700 1 -- 172.21.15.46:0/232351557 wait complete. 2024-05-08T23:03:10.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:10.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 -- 172.21.15.46:0/2370619542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec074f80 msgr2=0x7f8fec0733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 --2- 172.21.15.46:0/2370619542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec074f80 0x7f8fec0733d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009990 tx=0x7f8fdc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 -- 172.21.15.46:0/2370619542 shutdown_connections 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 --2- 172.21.15.46:0/2370619542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec108da0 0x7f8fec10b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 --2- 172.21.15.46:0/2370619542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec074f80 0x7f8fec0733d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 --2- 172.21.15.46:0/2370619542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec0739a0 0x7f8fec073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 -- 172.21.15.46:0/2370619542 >> 172.21.15.46:0/2370619542 conn(0x7f8fec0fc460 msgr2=0x7f8fec0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 -- 172.21.15.46:0/2370619542 shutdown_connections 2024-05-08T23:03:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.635+0000 7f8ff2400700 1 -- 172.21.15.46:0/2370619542 wait complete. 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.636+0000 7f8ff2400700 1 Processor -- start 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 -- start start 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 0x7f8fec198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec108da0 0x7f8fec1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a0ac0 con 0x7f8fec108da0 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a0c00 con 0x7f8fec0739a0 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8ff2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a0d90 con 0x7f8fec074f80 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:36052/0 (socket says 172.21.15.46:36052) 2024-05-08T23:03:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8feaa00700 1 -- 172.21.15.46:0/3222669176 learned_addr learned my addr 172.21.15.46:0/3222669176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8febe00700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec108da0 0x7f8fec1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.637+0000 7f8feb400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 0x7f8fec198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 -- 172.21.15.46:0/3222669176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 msgr2=0x7f8fec198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 0x7f8fec198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 -- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec108da0 msgr2=0x7f8fec1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec108da0 0x7f8fec1a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 -- 172.21.15.46:0/3222669176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009520 con 0x7f8fec074f80 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feb400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 0x7f8fec198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8feaa00700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800eb30 tx=0x7f8fd800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8009880 con 0x7f8fec074f80 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd80044e0 con 0x7f8fec074f80 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc0095f0 con 0x7f8fec074f80 2024-05-08T23:03:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8003e90 con 0x7f8fec074f80 2024-05-08T23:03:10.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.638+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1a13f0 con 0x7f8fec074f80 2024-05-08T23:03:10.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.639+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec107bb0 con 0x7f8fec074f80 2024-05-08T23:03:10.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.641+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8fd8004000 con 0x7f8fec074f80 2024-05-08T23:03:10.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.642+0000 7f8fe3e00700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 0x7f8fd00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:10.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.643+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8fd8014070 con 0x7f8fec074f80 2024-05-08T23:03:10.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.643+0000 7f8feb400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 0x7f8fd00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:10.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.643+0000 7f8feb400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 0x7f8fd00a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b2e0 tx=0x7f8fdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.645+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8fd8096680 con 0x7f8fec074f80 2024-05-08T23:03:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.864+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fec10e8d0 con 0x7f8fec074f80 2024-05-08T23:03:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.864+0000 7f8fe3e00700 1 -- 172.21.15.46:0/3222669176 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fd8099ca0 con 0x7f8fec074f80 2024-05-08T23:03:10.863 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 msgr2=0x7f8fd00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 0x7f8fd00a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b2e0 tx=0x7f8fdc005fb0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 msgr2=0x7f8fec1994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800eb30 tx=0x7f8fd800ee40 comp rx=0 tx=0).stop 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 shutdown_connections 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f8fd00a11b0 0x7f8fd00a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fec108da0 0x7f8fec1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fec0739a0 0x7f8fec198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 --2- 172.21.15.46:0/3222669176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec074f80 0x7f8fec1994d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 >> 172.21.15.46:0/3222669176 conn(0x7f8fec0fc460 msgr2=0x7f8fec104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 shutdown_connections 2024-05-08T23:03:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:10.867+0000 7f8ff2400700 1 -- 172.21.15.46:0/3222669176 wait complete. 2024-05-08T23:03:10.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 -- 172.21.15.46:0/2098763936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c101fd0 msgr2=0x7fc30c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 --2- 172.21.15.46:0/2098763936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c101fd0 0x7fc30c102450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009990 tx=0x7fc2f8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 -- 172.21.15.46:0/2098763936 shutdown_connections 2024-05-08T23:03:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 --2- 172.21.15.46:0/2098763936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c102990 0x7fc30c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 --2- 172.21.15.46:0/2098763936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 --2- 172.21.15.46:0/2098763936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c101fd0 0x7fc30c102450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 -- 172.21.15.46:0/2098763936 >> 172.21.15.46:0/2098763936 conn(0x7fc30c0fc3b0 msgr2=0x7fc30c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 -- 172.21.15.46:0/2098763936 shutdown_connections 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.092+0000 7fc312a00700 1 -- 172.21.15.46:0/2098763936 wait complete. 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.093+0000 7fc312a00700 1 Processor -- start 2024-05-08T23:03:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 -- start start 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 0x7fc30c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c102990 0x7fc30c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19e8f0 con 0x7fc30c100e80 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19ea60 con 0x7fc30c101fd0 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc312a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19ec20 con 0x7fc30c102990 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc30be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc30b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 0x7fc30c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc30be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54626/0 (socket says 172.21.15.46:54626) 2024-05-08T23:03:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc30be00700 1 -- 172.21.15.46:0/1109267389 learned_addr learned my addr 172.21.15.46:0/1109267389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.094+0000 7fc310a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c102990 0x7fc30c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 -- 172.21.15.46:0/1109267389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c102990 msgr2=0x7fc30c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c102990 0x7fc30c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 -- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 msgr2=0x7fc30c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 -- 172.21.15.46:0/1109267389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc009520 con 0x7fc30c101fd0 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30be00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc30b400700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 0x7fc30c197340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b680 tx=0x7fc2f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.095+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f801d070 con 0x7fc30c101fd0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.096+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80095f0 con 0x7fc30c101fd0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.096+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2f8021410 con 0x7fc30c101fd0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.096+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8017650 con 0x7fc30c101fd0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.096+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c19f190 con 0x7fc30c101fd0 2024-05-08T23:03:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.097+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c1082f0 con 0x7fc30c101fd0 2024-05-08T23:03:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.099+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc2f800fb10 con 0x7fc30c101fd0 2024-05-08T23:03:11.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.100+0000 7fc308c00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 0x7fc2f00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.100+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc2f80ce2b0 con 0x7fc30c101fd0 2024-05-08T23:03:11.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.100+0000 7fc30be00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 0x7fc2f00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.100+0000 7fc30be00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 0x7fc2f00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc30c198280 tx=0x7fc2fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:11.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.102+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc2f8097600 con 0x7fc30c101fd0 2024-05-08T23:03:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:10 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3188832958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:10 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/232351557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:10 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3188832958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:10 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/232351557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:10 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3188832958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:10 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/232351557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.322+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc30c10c830 con 0x7fc30c101fd0 2024-05-08T23:03:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.322+0000 7fc308c00700 1 -- 172.21.15.46:0/1109267389 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2f8021e60 con 0x7fc30c101fd0 2024-05-08T23:03:11.321 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 msgr2=0x7fc2f00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 0x7fc2f00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc30c198280 tx=0x7fc2fc009260 comp rx=0 tx=0).stop 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 msgr2=0x7fc30c197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 0x7fc30c197340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b680 tx=0x7fc2f8004730 comp rx=0 tx=0).stop 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 shutdown_connections 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30c102990 0x7fc30c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc2f00a11b0 0x7fc2f00a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc30c100e80 0x7fc30c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 --2- 172.21.15.46:0/1109267389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c101fd0 0x7fc30c197340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 >> 172.21.15.46:0/1109267389 conn(0x7fc30c0fc3b0 msgr2=0x7fc30c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 shutdown_connections 2024-05-08T23:03:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.325+0000 7fc312a00700 1 -- 172.21.15.46:0/1109267389 wait complete. 2024-05-08T23:03:11.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 -- 172.21.15.46:0/561893723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 msgr2=0x7f06801012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 --2- 172.21.15.46:0/561893723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f06801012a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0674009990 tx=0x7f0674009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 -- 172.21.15.46:0/561893723 shutdown_connections 2024-05-08T23:03:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 --2- 172.21.15.46:0/561893723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f06801092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 --2- 172.21.15.46:0/561893723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f06801012a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 --2- 172.21.15.46:0/561893723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 -- 172.21.15.46:0/561893723 >> 172.21.15.46:0/561893723 conn(0x7f06800fc3b0 msgr2=0x7f06800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 -- 172.21.15.46:0/561893723 shutdown_connections 2024-05-08T23:03:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.549+0000 7f0687800700 1 -- 172.21.15.46:0/561893723 wait complete. 2024-05-08T23:03:11.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 Processor -- start 2024-05-08T23:03:11.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 -- start start 2024-05-08T23:03:11.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f068019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019e8e0 con 0x7f0680101fd0 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019ea50 con 0x7f0680102990 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.550+0000 7f0687800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019ec10 con 0x7f0680100e80 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0684e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0684e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:36096/0 (socket says 172.21.15.46:36096) 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0684e00700 1 -- 172.21.15.46:0/157088053 learned_addr learned my addr 172.21.15.46:0/157088053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0685800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f068019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 -- 172.21.15.46:0/157088053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 msgr2=0x7f0680196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 -- 172.21.15.46:0/157088053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 msgr2=0x7f068019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f068019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 -- 172.21.15.46:0/157088053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740095f0 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0684e00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f0685800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f068019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.551+0000 7f067fe00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680197380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f066c00e9f0 tx=0x7f066c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.552+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00eee0 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.552+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066c00cbd0 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.552+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c018680 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.552+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068019eef0 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.552+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068019f2c0 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.553+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0680108330 con 0x7f0680101fd0 2024-05-08T23:03:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.555+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f066c00cd40 con 0x7f0680101fd0 2024-05-08T23:03:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.557+0000 7f067d600700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 0x7f06680a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.557+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f066c014070 con 0x7f0680101fd0 2024-05-08T23:03:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.557+0000 7f0684e00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 0x7f06680a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.558+0000 7f0684e00700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 0x7f06680a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f067400b2e0 tx=0x7f067400b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.561+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f066c052d60 con 0x7f0680101fd0 2024-05-08T23:03:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.784+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f068010c6f0 con 0x7f0680101fd0 2024-05-08T23:03:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.784+0000 7f067d600700 1 -- 172.21.15.46:0/157088053 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f066c096200 con 0x7f0680101fd0 2024-05-08T23:03:11.783 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 msgr2=0x7f06680a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 0x7f06680a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f067400b2e0 tx=0x7f067400b750 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 msgr2=0x7f0680197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680197380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f066c00e9f0 tx=0x7f066c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 shutdown_connections 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0680102990 0x7f068019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0680100e80 0x7f0680196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f06680a1230 0x7f06680a36f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 --2- 172.21.15.46:0/157088053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680101fd0 0x7f0680197380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 >> 172.21.15.46:0/157088053 conn(0x7f06800fc3b0 msgr2=0x7f0680105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:11.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 shutdown_connections 2024-05-08T23:03:11.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:11.787+0000 7f0687800700 1 -- 172.21.15.46:0/157088053 wait complete. 2024-05-08T23:03:11.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.009+0000 7ff81a400700 1 -- 172.21.15.46:0/2184203319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814100e80 msgr2=0x7ff8141012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.009+0000 7ff81a400700 1 --2- 172.21.15.46:0/2184203319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814100e80 0x7ff8141012a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff804009990 tx=0x7ff804009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 -- 172.21.15.46:0/2184203319 shutdown_connections 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 --2- 172.21.15.46:0/2184203319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 0x7ff8141092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 --2- 172.21.15.46:0/2184203319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814100e80 0x7ff8141012a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 --2- 172.21.15.46:0/2184203319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814101fd0 0x7ff814102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 -- 172.21.15.46:0/2184203319 >> 172.21.15.46:0/2184203319 conn(0x7ff8140fc3b0 msgr2=0x7ff8140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 -- 172.21.15.46:0/2184203319 shutdown_connections 2024-05-08T23:03:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.010+0000 7ff81a400700 1 -- 172.21.15.46:0/2184203319 wait complete. 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 Processor -- start 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 -- start start 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814101fd0 0x7ff8141995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 0x7ff8141a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8141a0ba0 con 0x7ff814101fd0 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8141a0ce0 con 0x7ff814100e80 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff81a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8141a0e70 con 0x7ff814102990 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.011+0000 7ff813400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52506/0 (socket says 172.21.15.46:52506) 2024-05-08T23:03:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 -- 172.21.15.46:0/30024893 learned_addr learned my addr 172.21.15.46:0/30024893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813e00700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 0x7ff8141a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff812a00700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814101fd0 0x7ff8141995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 -- 172.21.15.46:0/30024893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 msgr2=0x7ff8141a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 0x7ff8141a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 -- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814101fd0 msgr2=0x7ff8141995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814101fd0 0x7ff8141995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 -- 172.21.15.46:0/30024893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040095f0 con 0x7ff814100e80 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff813400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff804006010 tx=0x7ff804004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.012+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80401d070 con 0x7ff814100e80 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.013+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8141a10f0 con 0x7ff814100e80 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.013+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff804005030 con 0x7ff814100e80 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.013+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804004ba0 con 0x7ff814100e80 2024-05-08T23:03:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.013+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8141a1520 con 0x7ff814100e80 2024-05-08T23:03:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.014+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814067d10 con 0x7ff814100e80 2024-05-08T23:03:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.016+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff804004d00 con 0x7ff814100e80 2024-05-08T23:03:12.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.017+0000 7ff80be00700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 0x7ff7f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.017+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff8040cde40 con 0x7ff814100e80 2024-05-08T23:03:12.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.017+0000 7ff812a00700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 0x7ff7f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.018+0000 7ff812a00700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 0x7ff7f80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff800007c60 tx=0x7ff8000073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.019+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff804097190 con 0x7ff814100e80 2024-05-08T23:03:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3222669176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:11 smithi086 ceph-mon[78610]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1109267389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/157088053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.238+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff81410c880 con 0x7ff814100e80 2024-05-08T23:03:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.239+0000 7ff80be00700 1 -- 172.21.15.46:0/30024893 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff80409a7b0 con 0x7ff814100e80 2024-05-08T23:03:12.238 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 msgr2=0x7ff7f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 0x7ff7f80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff800007c60 tx=0x7ff8000073d0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 msgr2=0x7ff814199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff804006010 tx=0x7ff804004730 comp rx=0 tx=0).stop 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 shutdown_connections 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff814102990 0x7ff8141a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff814100e80 0x7ff814199070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7f80a11b0 0x7ff7f80a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 --2- 172.21.15.46:0/30024893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814101fd0 0x7ff8141995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 >> 172.21.15.46:0/30024893 conn(0x7ff8140fc3b0 msgr2=0x7ff8140731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 shutdown_connections 2024-05-08T23:03:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.241+0000 7ff81a400700 1 -- 172.21.15.46:0/30024893 wait complete. 2024-05-08T23:03:12.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3222669176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:11 smithi046 ceph-mon[36898]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:12.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1109267389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/157088053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3222669176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:11 smithi053 ceph-mon[77587]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1109267389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/157088053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 -- 172.21.15.46:0/1151605870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 msgr2=0x7fc708102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1151605870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708102300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40099c0 tx=0x7fc6f4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 -- 172.21.15.46:0/1151605870 shutdown_connections 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1151605870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708102840 0x7fc7081090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1151605870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708100cb0 0x7fc7081010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1151605870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708102300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 -- 172.21.15.46:0/1151605870 >> 172.21.15.46:0/1151605870 conn(0x7fc7080fc220 msgr2=0x7fc7080fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 -- 172.21.15.46:0/1151605870 shutdown_connections 2024-05-08T23:03:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.459+0000 7fc70ce00700 1 -- 172.21.15.46:0/1151605870 wait complete. 2024-05-08T23:03:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.460+0000 7fc70ce00700 1 Processor -- start 2024-05-08T23:03:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.460+0000 7fc70ce00700 1 -- start start 2024-05-08T23:03:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708102840 0x7fc7081a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a09e0 con 0x7fc708101ea0 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a0b20 con 0x7fc708100cb0 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc70ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a0cb0 con 0x7fc708102840 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc706a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc707400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc707400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52538/0 (socket says 172.21.15.46:52538) 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc707400700 1 -- 172.21.15.46:0/1428108712 learned_addr learned my addr 172.21.15.46:0/1428108712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.461+0000 7fc707e00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708102840 0x7fc7081a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 -- 172.21.15.46:0/1428108712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708102840 msgr2=0x7fc7081a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708102840 0x7fc7081a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 -- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 msgr2=0x7fc708199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 -- 172.21.15.46:0/1428108712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40095f0 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc707400700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800e950 tx=0x7fc6f800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8009770 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6f800c7e0 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7081a0f90 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003cf0 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.462+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7081a1360 con 0x7fc708100cb0 2024-05-08T23:03:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.463+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70804ecd0 con 0x7fc708100cb0 2024-05-08T23:03:12.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.465+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc6f800c950 con 0x7fc708100cb0 2024-05-08T23:03:12.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.467+0000 7fc6ffe00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 0x7fc6ec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.467+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc6f8014070 con 0x7fc708100cb0 2024-05-08T23:03:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.467+0000 7fc706a00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 0x7fc6ec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.467+0000 7fc706a00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 0x7fc6ec0a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003730 tx=0x7fc6f400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.469+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6f80962f0 con 0x7fc708100cb0 2024-05-08T23:03:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.739+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7081a15b0 con 0x7fc708100cb0 2024-05-08T23:03:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.741+0000 7fc6ffe00700 1 -- 172.21.15.46:0/1428108712 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fc6f8099910 con 0x7fc708100cb0 2024-05-08T23:03:12.740 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:12.740 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 msgr2=0x7fc6ec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 0x7fc6ec0a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003730 tx=0x7fc6f400b540 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 msgr2=0x7fc708198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800e950 tx=0x7fc6f800ec60 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 shutdown_connections 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc708102840 0x7fc7081a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc708100cb0 0x7fc708198f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6ec0a11b0 0x7fc6ec0a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 --2- 172.21.15.46:0/1428108712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc708101ea0 0x7fc708199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 >> 172.21.15.46:0/1428108712 conn(0x7fc7080fc220 msgr2=0x7fc7080731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 shutdown_connections 2024-05-08T23:03:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.743+0000 7fc70ce00700 1 -- 172.21.15.46:0/1428108712 wait complete. 2024-05-08T23:03:12.744 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:12.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 -- 172.21.15.46:0/1765969446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100e80 msgr2=0x7f0ce01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 --2- 172.21.15.46:0/1765969446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100e80 0x7f0ce01012a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009990 tx=0x7f0cd0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 -- 172.21.15.46:0/1765969446 shutdown_connections 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 --2- 172.21.15.46:0/1765969446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0102990 0x7f0ce01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 --2- 172.21.15.46:0/1765969446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0100e80 0x7f0ce01012a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 --2- 172.21.15.46:0/1765969446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 0x7f0ce0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 -- 172.21.15.46:0/1765969446 >> 172.21.15.46:0/1765969446 conn(0x7f0ce00fc410 msgr2=0x7f0ce00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:12.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 -- 172.21.15.46:0/1765969446 shutdown_connections 2024-05-08T23:03:12.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.963+0000 7f0ce5200700 1 -- 172.21.15.46:0/1765969446 wait complete. 2024-05-08T23:03:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.964+0000 7f0ce5200700 1 Processor -- start 2024-05-08T23:03:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.964+0000 7f0ce5200700 1 -- start start 2024-05-08T23:03:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 0x7f0ce0196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 0x7f0ce0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 0x7f0ce019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019e940 con 0x7f0ce0102990 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019ea80 con 0x7f0ce0101fd0 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0ce5200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce019ec10 con 0x7f0ce0100e80 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0cde800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 0x7f0ce019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0cdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 0x7f0ce0196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0cde800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 0x7f0ce019e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53746/0 (socket says 172.21.15.46:53746) 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0cde800700 1 -- 172.21.15.46:0/3549661239 learned_addr learned my addr 172.21.15.46:0/3549661239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.965+0000 7f0cdd400700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 0x7f0ce0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 -- 172.21.15.46:0/3549661239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 msgr2=0x7f0ce0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 0x7f0ce0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 -- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 msgr2=0x7f0ce019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 0x7f0ce019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 -- 172.21.15.46:0/3549661239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009520 con 0x7f0ce0100e80 2024-05-08T23:03:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cdde00700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 0x7f0ce0196e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001d070 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd0021410 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd00095f0 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0017650 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.966+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce019f180 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.967+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce01082f0 con 0x7f0ce0100e80 2024-05-08T23:03:12.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.969+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0cd0021580 con 0x7f0ce0100e80 2024-05-08T23:03:12.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.970+0000 7f0cd6a00700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 0x7f0cc40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:12.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.970+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0cd00ce140 con 0x7f0ce0100e80 2024-05-08T23:03:12.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.971+0000 7f0cdd400700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 0x7f0cc40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.971+0000 7f0cdd400700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 0x7f0cc40a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009df0 tx=0x7f0ccc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:12.973+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0cd0097490 con 0x7f0ce0100e80 2024-05-08T23:03:13.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/30024893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:13.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1428108712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:13.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.195+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ce010c6f0 con 0x7f0ce0100e80 2024-05-08T23:03:13.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.197+0000 7f0cd6a00700 1 -- 172.21.15.46:0/3549661239 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24387 (secure 0 0 0) 0x7f0cd0005740 con 0x7f0ce0100e80 2024-05-08T23:03:13.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:13.196 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:02:18.647708+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:13.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 msgr2=0x7f0cc40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 0x7f0cc40a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009df0 tx=0x7f0ccc009260 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 msgr2=0x7f0ce0196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 0x7f0ce0196e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd0009580 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 shutdown_connections 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f0cc40a1230 0x7f0cc40a36f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce0102990 0x7f0ce019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ce0100e80 0x7f0ce0196e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 --2- 172.21.15.46:0/3549661239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce0101fd0 0x7f0ce0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 >> 172.21.15.46:0/3549661239 conn(0x7f0ce00fc410 msgr2=0x7f0ce0105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 shutdown_connections 2024-05-08T23:03:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.199+0000 7f0ce5200700 1 -- 172.21.15.46:0/3549661239 wait complete. 2024-05-08T23:03:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/30024893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:13.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1428108712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:13.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/30024893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1428108712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:13.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 -- 172.21.15.46:0/3970997484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4101ea0 msgr2=0x7f5ad4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 --2- 172.21.15.46:0/3970997484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4102300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00099c0 tx=0x7f5ac0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 -- 172.21.15.46:0/3970997484 shutdown_connections 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 --2- 172.21.15.46:0/3970997484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4102840 0x7f5ad41090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 --2- 172.21.15.46:0/3970997484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 --2- 172.21.15.46:0/3970997484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4102300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 -- 172.21.15.46:0/3970997484 >> 172.21.15.46:0/3970997484 conn(0x7f5ad40fc220 msgr2=0x7f5ad40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 -- 172.21.15.46:0/3970997484 shutdown_connections 2024-05-08T23:03:13.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.417+0000 7f5ad9800700 1 -- 172.21.15.46:0/3970997484 wait complete. 2024-05-08T23:03:13.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.418+0000 7f5ad9800700 1 Processor -- start 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.418+0000 7f5ad9800700 1 -- start start 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 0x7f5ad40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4073fd0 con 0x7f5ad4102840 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4074140 con 0x7f5ad4100cb0 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad41a0cf0 con 0x7f5ad4101ea0 2024-05-08T23:03:13.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 0x7f5ad40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52582/0 (socket says 172.21.15.46:52582) 2024-05-08T23:03:13.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 -- 172.21.15.46:0/1621375482 learned_addr learned my addr 172.21.15.46:0/1621375482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3400700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 -- 172.21.15.46:0/1621375482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4101ea0 msgr2=0x7f5ad4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.419+0000 7f5ad3e00700 1 -- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 msgr2=0x7f5ad40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad3e00700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 0x7f5ad40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad3e00700 1 -- 172.21.15.46:0/1621375482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00095f0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad8e00700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 0x7f5ad40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad3e00700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400e970 tx=0x7f5ac400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4009790 con 0x7f5ad4100cb0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad41a0f50 con 0x7f5ad4100cb0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ac400eec0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4003cf0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.420+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad41a1380 con 0x7f5ad4100cb0 2024-05-08T23:03:13.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.421+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad4108080 con 0x7f5ad4100cb0 2024-05-08T23:03:13.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.423+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5ac400ccd0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.424+0000 7f5ad0c00700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 0x7f5ab80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.425+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ac4014070 con 0x7f5ad4100cb0 2024-05-08T23:03:13.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.425+0000 7f5ad3400700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 0x7f5ab80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.425+0000 7f5ad3400700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 0x7f5ab80a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000b580 tx=0x7f5ac001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.427+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5ac4096490 con 0x7f5ad4100cb0 2024-05-08T23:03:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.697+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ad404ecd0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.698+0000 7f5ad0c00700 1 -- 172.21.15.46:0/1621375482 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f5ac4099ab0 con 0x7f5ad4100cb0 2024-05-08T23:03:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:13.697 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 msgr2=0x7f5ab80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 0x7f5ab80a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000b580 tx=0x7f5ac001a040 comp rx=0 tx=0).stop 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 msgr2=0x7f5ad4074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400e970 tx=0x7f5ac400ec80 comp rx=0 tx=0).stop 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 shutdown_connections 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5ab80a11b0 0x7f5ab80a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad4102840 0x7f5ad40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ad4100cb0 0x7f5ad4074cb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 --2- 172.21.15.46:0/1621375482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4101ea0 0x7f5ad4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 >> 172.21.15.46:0/1621375482 conn(0x7f5ad40fc220 msgr2=0x7f5ad41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 shutdown_connections 2024-05-08T23:03:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.701+0000 7f5ad9800700 1 -- 172.21.15.46:0/1621375482 wait complete. 2024-05-08T23:03:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:13.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:01:51.460837+0000', 'modified': '2024-05-08T23:02:19.629003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24583, 'mds_2': 24571, 'mds_3': 24547, 'mds_4': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/3863967478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 3863967478}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 3863967478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/272583724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 272583724}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 272583724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6837/1812878749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6836', 'nonce': 1812878749}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 1812878749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/1845130080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1845130080}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1845130080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/170717769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 170717769}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 170717769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.86:6835/2919086413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2919086413}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2919086413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.86:6837/2553915750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 2553915750}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2553915750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24571': {'gid': 24571, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.86:6839/4115816206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 4115816206}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 4115816206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24583': {'gid': 24583, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.46:6839/1484730309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 1484730309}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 1484730309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/3374696949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3374696949}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3374696949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:03:13.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T23:03:13.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-08T23:03:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 -- 172.21.15.46:0/467513831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8074fe0 msgr2=0x7ff2a8073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 --2- 172.21.15.46:0/467513831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8073430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 -- 172.21.15.46:0/467513831 shutdown_connections 2024-05-08T23:03:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 --2- 172.21.15.46:0/467513831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8108d40 0x7ff2a810b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 --2- 172.21.15.46:0/467513831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8073430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 --2- 172.21.15.46:0/467513831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 0x7ff2a8073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 -- 172.21.15.46:0/467513831 >> 172.21.15.46:0/467513831 conn(0x7ff2a80fc3b0 msgr2=0x7ff2a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 -- 172.21.15.46:0/467513831 shutdown_connections 2024-05-08T23:03:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.920+0000 7ff2aca00700 1 -- 172.21.15.46:0/467513831 wait complete. 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 Processor -- start 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 -- start start 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 0x7ff2a8198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a0af0 con 0x7ff2a8108d40 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a0c30 con 0x7ff2a8073a00 2024-05-08T23:03:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.921+0000 7ff2aca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a0dc0 con 0x7ff2a8074fe0 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53776/0 (socket says 172.21.15.46:53776) 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 -- 172.21.15.46:0/1270732156 learned_addr learned my addr 172.21.15.46:0/1270732156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 0x7ff2a8198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a5400700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 -- 172.21.15.46:0/1270732156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8074fe0 msgr2=0x7ff2a8199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 -- 172.21.15.46:0/1270732156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 msgr2=0x7ff2a8198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 0x7ff2a8198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 -- 172.21.15.46:0/1270732156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980095f0 con 0x7ff2a8108d40 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000ef30 tx=0x7ff2a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.922+0000 7ff29ea00700 1 -- 172.21.15.46:0/1270732156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0018070 con 0x7ff2a8108d40 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.923+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a81a1040 con 0x7ff2a8108d40 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.923+0000 7ff29ea00700 1 -- 172.21.15.46:0/1270732156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a000f040 con 0x7ff2a8108d40 2024-05-08T23:03:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.923+0000 7ff29ea00700 1 -- 172.21.15.46:0/1270732156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000ccc0 con 0x7ff2a8108d40 2024-05-08T23:03:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.923+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a81a1550 con 0x7ff2a8108d40 2024-05-08T23:03:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.926+0000 7ff29ea00700 1 -- 172.21.15.46:0/1270732156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff2a0003a00 con 0x7ff2a8108d40 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff29ea00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 0x7ff28c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 Processor -- start 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 -- start start 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff29ea00700 1 -- 172.21.15.46:0/1270732156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2a00cc8d0 con 0x7ff2a8108d40 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 0x7ff28c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc34f0 0x561013bc3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bc4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013b1a100 con 0x561013bc34f0 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013b17ac0 con 0x561013bc3e50 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.927+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013b133b0 con 0x561013bc4810 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc34f0 0x561013bc3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52608/0 (socket says 172.21.15.46:52608) 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 -- 172.21.15.46:0/179265201 learned_addr learned my addr 172.21.15.46:0/179265201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a6800700 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bc4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 -- 172.21.15.46:0/179265201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 msgr2=0x561013bc4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bc4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 -- 172.21.15.46:0/179265201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc34f0 msgr2=0x561013bc3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc34f0 0x561013bc3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 -- 172.21.15.46:0/179265201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561013994360 con 0x561013bc3e50 2024-05-08T23:03:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a6800700 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bc4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 0x7ff28c0a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b2e0 tx=0x7ff29800bbf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.928+0000 7ff2a5400700 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff294009d60 tx=0x7ff294011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc222a6bddc991f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.929+0000 7ff293000700 1 -- 172.21.15.46:0/179265201 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294004220 con 0x561013bc3e50 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.929+0000 7ff293000700 1 -- 172.21.15.46:0/179265201 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff294004380 con 0x561013bc3e50 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.929+0000 7ff293000700 1 -- 172.21.15.46:0/179265201 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940197f0 con 0x561013bc3e50 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.929+0000 7ff2c0448b80 1 -- 172.21.15.46:0/179265201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 msgr2=0x561013bc42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.929+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff294009d60 tx=0x7ff294011720 comp rx=0 tx=0).stop 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 -- 172.21.15.46:0/179265201 shutdown_connections 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bc4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc34f0 0x561013bc3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/179265201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bc42d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 -- 172.21.15.46:0/179265201 >> 172.21.15.46:0/179265201 conn(0x561013b211c0 msgr2=0x561013bbdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 -- 172.21.15.46:0/179265201 shutdown_connections 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 -- 172.21.15.46:0/179265201 wait complete. 2024-05-08T23:03:13.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 -- start start 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.930+0000 7ff2c0448b80 1 Processor -- start 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc3e50 0x561013bcd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013aaf510 con 0x561013bc4810 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013aaf1b0 con 0x561013bc34f0 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013996650 con 0x561013bc3e50 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2c0448b80 10 client.0 _ll_register_callbacks cb 0x561013ab8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc3e50 0x561013bcd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52622/0 (socket says 172.21.15.46:52622) 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 -- 172.21.15.46:0/509106212 learned_addr learned my addr 172.21.15.46:0/509106212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 -- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc3e50 msgr2=0x561013bcd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc3e50 0x561013bcd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 -- 172.21.15.46:0/509106212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 msgr2=0x561013bd42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.931+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2a5e00700 1 -- 172.21.15.46:0/509106212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a81a1550 con 0x561013bc34f0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2a6800700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12299d2e8b768f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff287400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29801d070 con 0x561013bc34f0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561013a6c220 con 0x561013bc34f0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff298004850 con 0x561013bc34f0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff298004a10 con 0x561013bc34f0 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2c0448b80 10 client.25184 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.932+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff294009960 con 0x561013bc34f0 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561013a6c220 con 0x561013bc34f0 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff29800fc20 con 0x561013bc34f0 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 10 client.25184 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 20 client.25184 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013bd7440 0x561013bdd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 4 client.25184 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561013b927c0 con 0x561013bd7440 2024-05-08T23:03:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.933+0000 7ff2a5400700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013bd7440 0x561013bdd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.934+0000 7ff2a5400700 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013bd7440 0x561013bdd2a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfd381eaaea1a0e3 server_cookie=ae7d69731d39146a in_seq=0 out_seq=0 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.934+0000 7ff287400700 10 client.25184 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff287400700 1 -- 172.21.15.46:0/509106212 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561013b927c0 con 0x561013bd7440 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff287400700 10 client.25184 handle_command_reply: tid=0 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 client.25184 shutdown 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 msgr2=0x561013bccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 shutdown_connections 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013bd7440 0x561013bdd2a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bccd20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff287400700 0 client.25184 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/509106212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc3e50 0x561013bcd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.935+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 >> 172.21.15.46:0/509106212 conn(0x561013b211c0 msgr2=0x561013bbd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.936+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 shutdown_connections 2024-05-08T23:03:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.936+0000 7ff2c0448b80 1 -- 172.21.15.46:0/509106212 wait complete. 2024-05-08T23:03:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.936+0000 7ff2c0448b80 20 client.25184 trim_cache size 0 max 16384 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.955+0000 7ff2c0448b80 1 Processor -- start 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.955+0000 7ff2c0448b80 1 -- start start 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bcd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bd4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd9850 con 0x561013bc3e50 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd5450 con 0x561013bc34f0 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd8dd0 con 0x561013bc4810 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53802/0 (socket says 172.21.15.46:53802) 2024-05-08T23:03:13.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 -- 172.21.15.46:0/1131474151 learned_addr learned my addr 172.21.15.46:0/1131474151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bcd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bd4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 -- 172.21.15.46:0/1131474151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 msgr2=0x561013bd4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bd4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 -- 172.21.15.46:0/1131474151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 msgr2=0x561013bcd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bcd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.956+0000 7ff2a5400700 1 -- 172.21.15.46:0/1131474151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561013940b10 con 0x561013bc3e50 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.957+0000 7ff2a5400700 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c00 tx=0x7ff294005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937d92e9e47c9879 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.957+0000 7ff293a00700 1 -- 172.21.15.46:0/1131474151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294029070 con 0x561013bc3e50 2024-05-08T23:03:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.957+0000 7ff293a00700 1 -- 172.21.15.46:0/1131474151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29402e070 con 0x561013bc3e50 2024-05-08T23:03:13.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.957+0000 7ff293a00700 1 -- 172.21.15.46:0/1131474151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294003970 con 0x561013bc3e50 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1131474151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 msgr2=0x561013bc38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c00 tx=0x7ff294005fa0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1131474151 shutdown_connections 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc4810 0x561013bd4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc34f0 0x561013bcd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1131474151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc3e50 0x561013bc38d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1131474151 >> 172.21.15.46:0/1131474151 conn(0x561013b211c0 msgr2=0x561013bc4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1131474151 shutdown_connections 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1131474151 wait complete. 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 -- start start 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.958+0000 7ff2c0448b80 1 Processor -- start 2024-05-08T23:03:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc34f0 0x561013bce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd3840 con 0x561013bc4810 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd39b0 con 0x561013bc3e50 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013bd3b70 con 0x561013bc34f0 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2c0448b80 10 client.0 _ll_register_callbacks cb 0x561013ab8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53810/0 (socket says 172.21.15.46:53810) 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 -- 172.21.15.46:0/1374477680 learned_addr learned my addr 172.21.15.46:0/1374477680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc34f0 0x561013bce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 -- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc34f0 msgr2=0x561013bce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc34f0 0x561013bce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 -- 172.21.15.46:0/1374477680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 msgr2=0x561013bce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a6800700 1 -- 172.21.15.46:0/1374477680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561013994360 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.959+0000 7ff2a5400700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff2a6800700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0095d90 tx=0x7ff2a0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1669b60d92b7e10b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0084040 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2a00b7e40 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff294004380 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a00b2240 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff2c0448b80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561013ad9510 con 0x561013bc4810 2024-05-08T23:03:13.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff294004380 con 0x561013bc4810 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2a0098dd0 con 0x561013bc4810 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.960+0000 7ff2c0448b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2c0448b80 10 client.15201 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2c0448b80 20 client.15201 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013c331c0 0x561013c2fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2c0448b80 4 client.15201 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561013b0be70 con 0x561013c331c0 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013c331c0 0x561013c2fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff2a5e00700 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013c331c0 0x561013c2fab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7770e6154c61457a server_cookie=b313cbe0e50fd6eb in_seq=0 out_seq=0 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.961+0000 7ff287400700 10 client.15201 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff287400700 1 -- 172.21.15.46:0/1374477680 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561013b0be70 con 0x561013c331c0 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff287400700 10 client.15201 handle_command_reply: tid=0 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 client.15201 shutdown 2024-05-08T23:03:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 msgr2=0x561013bd32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0095d90 tx=0x7ff2a0004b50 comp rx=0 tx=0).stop 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 shutdown_connections 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x561013c331c0 0x561013c2fab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561013bc4810 0x561013bd32d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561013bc34f0 0x561013bce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 --2- 172.21.15.46:0/1374477680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561013bc3e50 0x561013bce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff287400700 0 client.15201 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 >> 172.21.15.46:0/1374477680 conn(0x561013b211c0 msgr2=0x561013bcc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 shutdown_connections 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 1 -- 172.21.15.46:0/1374477680 wait complete. 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.962+0000 7ff2c0448b80 20 client.15201 trim_cache size 0 max 16384 2024-05-08T23:03:13.966 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24541, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 56.297201876000003, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 101.75546887100001 2024-05-08T23:03:13.967 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 msgr2=0x7ff28c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 0x7ff28c0a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b2e0 tx=0x7ff29800bbf0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 msgr2=0x7ff2a81a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000ef30 tx=0x7ff2a000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 shutdown_connections 2024-05-08T23:03:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff28c0a11b0 0x7ff28c0a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8108d40 0x7ff2a81a0520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a8073a00 0x7ff2a8198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 --2- 172.21.15.46:0/1270732156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a8074fe0 0x7ff2a8199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 >> 172.21.15.46:0/1270732156 conn(0x7ff2a80fc3b0 msgr2=0x7ff2a8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 shutdown_connections 2024-05-08T23:03:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:13.963+0000 7ff2aca00700 1 -- 172.21.15.46:0/1270732156 wait complete. 2024-05-08T23:03:13.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:13 smithi046 ceph-mon[36898]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:13.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3549661239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:13.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1621375482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:13.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 -- 172.21.15.46:0/3577158312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 msgr2=0x7f9acc073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 --2- 172.21.15.46:0/3577158312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc073e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80098f0 tx=0x7f9ab8009c00 comp rx=0 tx=0).stop 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 -- 172.21.15.46:0/3577158312 shutdown_connections 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 --2- 172.21.15.46:0/3577158312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 0x7f9acc10b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 --2- 172.21.15.46:0/3577158312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 0x7f9acc073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 --2- 172.21.15.46:0/3577158312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc073e80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 -- 172.21.15.46:0/3577158312 >> 172.21.15.46:0/3577158312 conn(0x7f9acc0fc210 msgr2=0x7f9acc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 -- 172.21.15.46:0/3577158312 shutdown_connections 2024-05-08T23:03:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.180+0000 7f9acb200700 1 -- 172.21.15.46:0/3577158312 wait complete. 2024-05-08T23:03:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.181+0000 7f9acb200700 1 Processor -- start 2024-05-08T23:03:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.181+0000 7f9acb200700 1 -- start start 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 0x7f9acc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 0x7f9acc1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a08f0 con 0x7f9acc074fe0 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a0a60 con 0x7f9acc073a00 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9acb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a0c20 con 0x7f9acc108ba0 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 0x7f9acc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52662/0 (socket says 172.21.15.46:52662) 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 -- 172.21.15.46:0/3305298496 learned_addr learned my addr 172.21.15.46:0/3305298496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9aca800700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 0x7f9acc1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 -- 172.21.15.46:0/3305298496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 msgr2=0x7f9acc1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 0x7f9acc1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 -- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 msgr2=0x7f9acc199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 0x7f9acc199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.182+0000 7f9ac9e00700 1 -- 172.21.15.46:0/3305298496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009520 con 0x7f9acc073a00 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9c0 tx=0x7f9abc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9ac2a00700 1 -- 172.21.15.46:0/3305298496 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc004a30 con 0x7f9acc073a00 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc1a0f00 con 0x7f9acc073a00 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9ac2a00700 1 -- 172.21.15.46:0/3305298496 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9abc00cc80 con 0x7f9acc073a00 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9ac2a00700 1 -- 172.21.15.46:0/3305298496 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0054e0 con 0x7f9acc073a00 2024-05-08T23:03:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.183+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1a1330 con 0x7f9acc073a00 2024-05-08T23:03:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.186+0000 7f9ac2a00700 1 -- 172.21.15.46:0/3305298496 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9abc004b90 con 0x7f9acc073a00 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.187+0000 7f9ae4b99b80 1 Processor -- start 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.187+0000 7f9ae4b99b80 1 -- start start 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.187+0000 7f9ac2a00700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 0x7f9ab00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac2a00700 1 -- 172.21.15.46:0/3305298496 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9abc014070 con 0x7f9acc073a00 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653168ff100 con 0x5653169a9950 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653168fcac0 con 0x5653169a8f30 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653168f83b0 con 0x5653169a85d0 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9aca800700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 0x7f9ab00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52678/0 (socket says 172.21.15.46:52678) 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 -- 172.21.15.46:0/799353369 learned_addr learned my addr 172.21.15.46:0/799353369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9aca800700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 -- 172.21.15.46:0/799353369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 msgr2=0x5653169a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 -- 172.21.15.46:0/799353369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 msgr2=0x5653169a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9400700 1 -- 172.21.15.46:0/799353369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565316779360 con 0x5653169a8f30 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.188+0000 7f9aca800700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.189+0000 7f9ac9400700 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800fa40 tx=0x7f9ab8009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a6612499cf749d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.189+0000 7f9aca800700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 0x7f9ab00a3720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400f6e0 tx=0x7f9ac4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.189+0000 7f9ab7400700 1 -- 172.21.15.46:0/799353369 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800f660 con 0x5653169a8f30 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.189+0000 7f9ab7400700 1 -- 172.21.15.46:0/799353369 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ab801d040 con 0x5653169a8f30 2024-05-08T23:03:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.189+0000 7f9ab7400700 1 -- 172.21.15.46:0/799353369 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8005470 con 0x5653169a8f30 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/799353369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 msgr2=0x5653169a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800fa40 tx=0x7f9ab8009f50 comp rx=0 tx=0).stop 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/799353369 shutdown_connections 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/799353369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169a93c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/799353369 >> 172.21.15.46:0/799353369 conn(0x5653169061c0 msgr2=0x5653169a30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/799353369 shutdown_connections 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/799353369 wait complete. 2024-05-08T23:03:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.190+0000 7f9ae4b99b80 1 -- start start 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 Processor -- start 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565316894510 con 0x5653169a9950 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653168941b0 con 0x5653169a8f30 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531677b650 con 0x5653169a85d0 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ae4b99b80 10 client.0 _ll_register_callbacks cb 0x56531689dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ac9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ac9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.191+0000 7f9ac9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52686/0 (socket says 172.21.15.46:52686) 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 -- 172.21.15.46:0/756893026 learned_addr learned my addr 172.21.15.46:0/756893026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 -- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 msgr2=0x5653169b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 -- 172.21.15.46:0/756893026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 msgr2=0x5653169b94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 -- 172.21.15.46:0/756893026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1a1330 con 0x5653169a8f30 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ac9400700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009810 tx=0x7f9ab80053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78bf04b626a2845b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9aabe00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800b760 con 0x5653169a8f30 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.192+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565316851220 con 0x5653169a8f30 2024-05-08T23:03:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9ab800b8c0 con 0x5653169a8f30 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab800ba80 con 0x5653169a8f30 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9ae4b99b80 10 client.25202 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f9acc10bc50 con 0x5653169a8f30 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565316851220 con 0x5653169a8f30 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.193+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9ab8015020 con 0x5653169a8f30 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 10 client.25202 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 20 client.25202 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5653169bcc10 0x5653169c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 4 client.25202 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653169777c0 con 0x5653169bcc10 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.194+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5653169bcc10 0x5653169c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.195+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5653169bcc10 0x5653169c1f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f173006f247ab6e3 server_cookie=73f73b9908fa935 in_seq=0 out_seq=0 2024-05-08T23:03:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.195+0000 7f9aabe00700 10 client.25202 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.195+0000 7f9aabe00700 1 -- 172.21.15.46:0/756893026 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653169777c0 con 0x5653169bcc10 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.195+0000 7f9aabe00700 10 client.25202 handle_command_reply: tid=0 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 client.25202 shutdown 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 msgr2=0x5653169b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009810 tx=0x7f9ab80053e0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 shutdown_connections 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5653169bcc10 0x5653169c1f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9aabe00700 0 client.25202 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/756893026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b24b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 >> 172.21.15.46:0/756893026 conn(0x5653169061c0 msgr2=0x5653169a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 shutdown_connections 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/756893026 wait complete. 2024-05-08T23:03:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.196+0000 7f9ae4b99b80 20 client.25202 trim_cache size 0 max 16384 2024-05-08T23:03:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:13 smithi086 ceph-mon[78610]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3549661239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1621375482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.214+0000 7f9ae4b99b80 1 Processor -- start 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 -- start start 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531699fd70 con 0x5653169a9950 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653169bb170 con 0x5653169a8f30 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531696dc10 con 0x5653169a85d0 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53842/0 (socket says 172.21.15.46:53842) 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 -- 172.21.15.46:0/2575754491 learned_addr learned my addr 172.21.15.46:0/2575754491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9ac9400700 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 -- 172.21.15.46:0/2575754491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 msgr2=0x5653169a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 -- 172.21.15.46:0/2575754491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 msgr2=0x5653169b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.215+0000 7f9aca800700 1 -- 172.21.15.46:0/2575754491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565316725b10 con 0x5653169a9950 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.216+0000 7f9aca800700 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009440 tx=0x7f9ac4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60766ea67cdc992c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.216+0000 7f9ab7e00700 1 -- 172.21.15.46:0/2575754491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac402a070 con 0x5653169a9950 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.216+0000 7f9ab7e00700 1 -- 172.21.15.46:0/2575754491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac400f040 con 0x5653169a9950 2024-05-08T23:03:14.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.216+0000 7f9ab7e00700 1 -- 172.21.15.46:0/2575754491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e040 con 0x5653169a9950 2024-05-08T23:03:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/2575754491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 msgr2=0x5653169b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009440 tx=0x7f9ac4011000 comp rx=0 tx=0).stop 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/2575754491 shutdown_connections 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b9410 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/2575754491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/2575754491 >> 172.21.15.46:0/2575754491 conn(0x5653169061c0 msgr2=0x56531699f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/2575754491 shutdown_connections 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/2575754491 wait complete. 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 -- start start 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.217+0000 7f9ae4b99b80 1 Processor -- start 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653169b9160 con 0x5653169a9950 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653169b92d0 con 0x5653169a8f30 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653169b9440 con 0x5653169a85d0 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ae4b99b80 10 client.0 _ll_register_callbacks cb 0x56531689dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53858/0 (socket says 172.21.15.46:53858) 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 -- 172.21.15.46:0/96502599 learned_addr learned my addr 172.21.15.46:0/96502599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9ac9400700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 -- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 msgr2=0x5653169b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 -- 172.21.15.46:0/96502599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 msgr2=0x5653169b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.218+0000 7f9aca800700 1 -- 172.21.15.46:0/96502599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4009120 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aca800700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400f460 tx=0x7f9ac4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef5a0864427b963 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aabe00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac402a040 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ab801d040 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9ac4004200 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ac400e040 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9ae4b99b80 10 client.15216 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5653168be510 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.219+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ab801d040 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9ac4026ba0 con 0x5653169a9950 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 10 client.15216 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 20 client.15216 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565316a18370 0x565316a14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 4 client.15216 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653168f0e70 con 0x565316a18370 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.220+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565316a18370 0x565316a14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ac9e00700 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565316a18370 0x565316a14cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c7ffc66fb286c3 server_cookie=45fa0df42f6d32e8 in_seq=0 out_seq=0 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9aabe00700 10 client.15216 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9aabe00700 1 -- 172.21.15.46:0/96502599 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5653168f0e70 con 0x565316a18370 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9aabe00700 10 client.15216 handle_command_reply: tid=0 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 client.15216 shutdown 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 msgr2=0x5653169b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400f460 tx=0x7f9ac4004d00 comp rx=0 tx=0).stop 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 shutdown_connections 2024-05-08T23:03:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565316a18370 0x565316a14cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653169a9950 0x5653169b8c20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653169a85d0 0x5653169b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 --2- 172.21.15.46:0/96502599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653169a8f30 0x5653169b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9aabe00700 0 client.15216 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 >> 172.21.15.46:0/96502599 conn(0x5653169061c0 msgr2=0x5653169a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 shutdown_connections 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 1 -- 172.21.15.46:0/96502599 wait complete. 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.221+0000 7f9ae4b99b80 20 client.15216 trim_cache size 0 max 16384 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24547, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 57.574762724000003, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 100.240645235 2024-05-08T23:03:14.226 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 msgr2=0x7f9ab00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 0x7f9ab00a3720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400f6e0 tx=0x7f9ac4009920 comp rx=0 tx=0).stop 2024-05-08T23:03:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 msgr2=0x7f9acc198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9c0 tx=0x7f9abc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 shutdown_connections 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc108ba0 0x7f9acc1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9acc073a00 0x7f9acc198e00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ab00a1260 0x7f9ab00a3720 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 --2- 172.21.15.46:0/3305298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9acc074fe0 0x7f9acc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 >> 172.21.15.46:0/3305298496 conn(0x7f9acc0fc210 msgr2=0x7f9acc104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 shutdown_connections 2024-05-08T23:03:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.222+0000 7f9acb200700 1 -- 172.21.15.46:0/3305298496 wait complete. 2024-05-08T23:03:14.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:03:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:13 smithi053 ceph-mon[77587]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3549661239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1621375482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 -- 172.21.15.46:0/2961942326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8100e20 msgr2=0x7f24b8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 --2- 172.21.15.46:0/2961942326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8100e20 0x7f24b8101240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009990 tx=0x7f24a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 -- 172.21.15.46:0/2961942326 shutdown_connections 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 --2- 172.21.15.46:0/2961942326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 0x7f24b8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 --2- 172.21.15.46:0/2961942326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8100e20 0x7f24b8101240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 --2- 172.21.15.46:0/2961942326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8101f70 0x7f24b81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 -- 172.21.15.46:0/2961942326 >> 172.21.15.46:0/2961942326 conn(0x7f24b80fc350 msgr2=0x7f24b80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.442+0000 7f24bd400700 1 -- 172.21.15.46:0/2961942326 shutdown_connections 2024-05-08T23:03:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.443+0000 7f24bd400700 1 -- 172.21.15.46:0/2961942326 wait complete. 2024-05-08T23:03:14.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.443+0000 7f24bd400700 1 Processor -- start 2024-05-08T23:03:14.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 -- start start 2024-05-08T23:03:14.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8101f70 0x7f24b81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 0x7f24b81a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a0ac0 con 0x7f24b8102930 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a0c00 con 0x7f24b8100e20 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24bd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a0d90 con 0x7f24b8101f70 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8101f70 0x7f24b81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24b7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 0x7f24b81a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52728/0 (socket says 172.21.15.46:52728) 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.444+0000 7f24b6800700 1 -- 172.21.15.46:0/630329780 learned_addr learned my addr 172.21.15.46:0/630329780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 -- 172.21.15.46:0/630329780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8101f70 msgr2=0x7f24b81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8101f70 0x7f24b81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 -- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 msgr2=0x7f24b81a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 0x7f24b81a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 -- 172.21.15.46:0/630329780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80095f0 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24b6800700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24a80098e0 tx=0x7f24a800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24af400700 1 -- 172.21.15.46:0/630329780 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a801d070 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b81a1010 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24af400700 1 -- 172.21.15.46:0/630329780 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a8005050 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24af400700 1 -- 172.21.15.46:0/630329780 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8004ae0 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.445+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81a1490 con 0x7f24b8100e20 2024-05-08T23:03:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.448+0000 7f24af400700 1 -- 172.21.15.46:0/630329780 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f24a80054a0 con 0x7f24b8100e20 2024-05-08T23:03:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.450+0000 7f24d0f59b80 1 Processor -- start 2024-05-08T23:03:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.450+0000 7f24d0f59b80 1 -- start start 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a6c100 con 0x5608d6b154f0 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a69ac0 con 0x5608d6b15e50 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a653b0 con 0x5608d6b16810 2024-05-08T23:03:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24af400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 0x7f249c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24af400700 1 -- 172.21.15.46:0/630329780 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24a80cdce0 con 0x7f24b8100e20 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52738/0 (socket says 172.21.15.46:52738) 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 -- 172.21.15.46:0/1337976040 learned_addr learned my addr 172.21.15.46:0/1337976040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b7200700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b5e00700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b7200700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 0x7f249c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 -- 172.21.15.46:0/1337976040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 msgr2=0x5608d6b16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 -- 172.21.15.46:0/1337976040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 msgr2=0x5608d6b15910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b15910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b6800700 1 -- 172.21.15.46:0/1337976040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d68e6360 con 0x5608d6b15e50 2024-05-08T23:03:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.451+0000 7f24b7200700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.452+0000 7f24b6800700 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097760 tx=0x7f24a8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164ae6dec5cb73b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.452+0000 7f24a3400700 1 -- 172.21.15.46:0/1337976040 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80051c0 con 0x5608d6b15e50 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.452+0000 7f24a3400700 1 -- 172.21.15.46:0/1337976040 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a809b3f0 con 0x5608d6b15e50 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.452+0000 7f24a3400700 1 -- 172.21.15.46:0/1337976040 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8004ed0 con 0x5608d6b15e50 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- 172.21.15.46:0/1337976040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 msgr2=0x5608d6b162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24b7200700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 0x7f249c0a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009550 tx=0x7f24b0009130 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097760 tx=0x7f24a8009310 comp rx=0 tx=0).stop 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- 172.21.15.46:0/1337976040 shutdown_connections 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/1337976040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b162d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- 172.21.15.46:0/1337976040 >> 172.21.15.46:0/1337976040 conn(0x5608d6a731c0 msgr2=0x5608d6b0fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- 172.21.15.46:0/1337976040 shutdown_connections 2024-05-08T23:03:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- 172.21.15.46:0/1337976040 wait complete. 2024-05-08T23:03:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 -- start start 2024-05-08T23:03:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.453+0000 7f24d0f59b80 1 Processor -- start 2024-05-08T23:03:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b26030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a01510 con 0x5608d6b154f0 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a011b0 con 0x5608d6b15e50 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68e8650 con 0x5608d6b16810 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24d0f59b80 10 client.0 _ll_register_callbacks cb 0x5608d6a0aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24b6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24b6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53894/0 (socket says 172.21.15.46:53894) 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24b6800700 1 -- 172.21.15.46:0/2705405767 learned_addr learned my addr 172.21.15.46:0/2705405767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24b5e00700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.454+0000 7f24b7200700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b26030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 -- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 msgr2=0x5608d6b26030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b26030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 -- 172.21.15.46:0/2705405767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 msgr2=0x5608d6b1eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b1eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 -- 172.21.15.46:0/2705405767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81a1490 con 0x5608d6b154f0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b5e00700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b1eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24b6800700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f24a809ac30 tx=0x7f24a809b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32140b303b85c430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f2497e00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80051c0 con 0x5608d6b154f0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24a809acf0 con 0x5608d6b154f0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608d69be220 con 0x5608d6b154f0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a809a3f0 con 0x5608d6b154f0 2024-05-08T23:03:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24d0f59b80 10 client.15228 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.455+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608d6892b10 con 0x5608d6b154f0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608d69be220 con 0x5608d6b154f0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24a8096ae0 con 0x5608d6b154f0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 20 client.15228 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b297e0 0x5608d6b2eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 4 client.15228 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608d6ae47c0 con 0x5608d6b297e0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.456+0000 7f24b5e00700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b297e0 0x5608d6b2eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.457+0000 7f24b5e00700 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b297e0 0x5608d6b2eac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=161f6980781198fd server_cookie=d55b715c77748594 in_seq=0 out_seq=0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.457+0000 7f2497e00700 10 client.15228 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f2497e00700 1 -- 172.21.15.46:0/2705405767 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608d6ae47c0 con 0x5608d6b297e0 2024-05-08T23:03:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f2497e00700 10 client.15228 handle_command_reply: tid=0 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 client.15228 shutdown 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 msgr2=0x5608d6b1eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f24a809ac30 tx=0x7f24a809b180 comp rx=0 tx=0).stop 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 shutdown_connections 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b26030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b297e0 0x5608d6b2eac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b154f0 0x5608d6b1eab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/2705405767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b1eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f2497e00700 0 client.15228 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 >> 172.21.15.46:0/2705405767 conn(0x5608d6a731c0 msgr2=0x5608d6b0f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 shutdown_connections 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 1 -- 172.21.15.46:0/2705405767 wait complete. 2024-05-08T23:03:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.458+0000 7f24d0f59b80 20 client.15228 trim_cache size 0 max 16384 2024-05-08T23:03:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.477+0000 7f24d0f59b80 1 Processor -- start 2024-05-08T23:03:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.477+0000 7f24d0f59b80 1 -- start start 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 0x5608d6b1d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b25ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6ae47f0 con 0x5608d6b16810 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69be250 con 0x5608d6b15e50 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6a232b0 con 0x5608d6b154f0 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53896/0 (socket says 172.21.15.46:53896) 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b7200700 1 -- 172.21.15.46:0/3717563380 learned_addr learned my addr 172.21.15.46:0/3717563380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b6800700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 0x5608d6b1d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b25ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 -- 172.21.15.46:0/3717563380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 msgr2=0x5608d6b1d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 0x5608d6b1d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 -- 172.21.15.46:0/3717563380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 msgr2=0x5608d6b1dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b5e00700 1 -- 172.21.15.46:0/3717563380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d6a2b510 con 0x5608d6b15e50 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b6800700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 0x5608d6b1d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.478+0000 7f24b7200700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.479+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b25ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5608d6b2b040 tx=0x7f24a400b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476a4cde0a112206 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.479+0000 7f24a3e00700 1 -- 172.21.15.46:0/3717563380 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4016030 con 0x5608d6b15e50 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.479+0000 7f24a3e00700 1 -- 172.21.15.46:0/3717563380 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a4017040 con 0x5608d6b15e50 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.479+0000 7f24a3e00700 1 -- 172.21.15.46:0/3717563380 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4005280 con 0x5608d6b15e50 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3717563380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 msgr2=0x5608d6b25ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b25ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5608d6b2b040 tx=0x7f24a400b3c0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3717563380 shutdown_connections 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b16810 0x5608d6b1dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b154f0 0x5608d6b1d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3717563380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b15e50 0x5608d6b25ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3717563380 >> 172.21.15.46:0/3717563380 conn(0x5608d6a731c0 msgr2=0x5608d6b1ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3717563380 shutdown_connections 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3717563380 wait complete. 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 -- start start 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.480+0000 7f24d0f59b80 1 Processor -- start 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 0x5608d6b22ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b25ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6b26200 con 0x5608d6b15e50 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6b26370 con 0x5608d6b154f0 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6b80780 con 0x5608d6b16810 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24d0f59b80 10 client.0 _ll_register_callbacks cb 0x5608d6a0aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24b5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 0x5608d6b22ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24b5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53910/0 (socket says 172.21.15.46:53910) 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24b5e00700 1 -- 172.21.15.46:0/3587678186 learned_addr learned my addr 172.21.15.46:0/3587678186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.481+0000 7f24b7200700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b25ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 -- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 msgr2=0x5608d6b25ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b25ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 -- 172.21.15.46:0/3587678186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 msgr2=0x5608d6b22ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 0x5608d6b22ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 -- 172.21.15.46:0/3587678186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608d6892b10 con 0x5608d6b15e50 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b6800700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 0x5608d6b22ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24b5e00700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24a4022040 tx=0x7f24a40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad65e2d8428ad97d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f2497e00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4016040 con 0x5608d6b15e50 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24a40049c0 con 0x5608d6b15e50 2024-05-08T23:03:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.482+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a809b3f0 con 0x5608d6b15e50 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a4004b80 con 0x5608d6b15e50 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 10 client.15234 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f24b0011040 con 0x5608d6b15e50 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a809b3f0 con 0x5608d6b15e50 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24a4028810 con 0x5608d6b15e50 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 20 client.15234 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b854c0 0x5608d6b81dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 4 client.15234 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.483+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608d6a5de70 con 0x5608d6b854c0 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f24b6800700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b854c0 0x5608d6b81dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f24b6800700 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b854c0 0x5608d6b81dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec4e207dc939614d server_cookie=4899db8637d057cd in_seq=0 out_seq=0 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f2497e00700 10 client.15234 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f2497e00700 1 -- 172.21.15.46:0/3587678186 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5608d6a5de70 con 0x5608d6b854c0 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f2497e00700 10 client.15234 handle_command_reply: tid=0 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f24d0f59b80 1 client.15234 shutdown 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 msgr2=0x5608d6b25540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.484+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24a4022040 tx=0x7f24a40039e0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 shutdown_connections 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608d6b16810 0x5608d6b25ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5608d6b154f0 0x5608d6b22ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5608d6b854c0 0x5608d6b81dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 --2- 172.21.15.46:0/3587678186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608d6b15e50 0x5608d6b25540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f2497e00700 0 client.15234 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 >> 172.21.15.46:0/3587678186 conn(0x5608d6a731c0 msgr2=0x5608d6b1dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 shutdown_connections 2024-05-08T23:03:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 1 -- 172.21.15.46:0/3587678186 wait complete. 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24d0f59b80 20 client.15234 trim_cache size 0 max 16384 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24571, 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 58.844984154999999, 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 96.769527870000005 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:14.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 msgr2=0x7f249c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 0x7f249c0a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009550 tx=0x7f24b0009130 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 msgr2=0x7f24b8198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.485+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24a80098e0 tx=0x7f24a800f670 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 shutdown_connections 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f249c0a11b0 0x7f249c0a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b8102930 0x7f24b81a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b8100e20 0x7f24b8198f90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 --2- 172.21.15.46:0/630329780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b8101f70 0x7f24b81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 >> 172.21.15.46:0/630329780 conn(0x7f24b80fc350 msgr2=0x7f24b80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 shutdown_connections 2024-05-08T23:03:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.486+0000 7f24bd400700 1 -- 172.21.15.46:0/630329780 wait complete. 2024-05-08T23:03:14.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 -- 172.21.15.46:0/2357301033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa41029b0 msgr2=0x7f4aa41092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 --2- 172.21.15.46:0/2357301033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa41029b0 0x7f4aa41092d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00b0a0 tx=0x7f4a9c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 -- 172.21.15.46:0/2357301033 shutdown_connections 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 --2- 172.21.15.46:0/2357301033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa41029b0 0x7f4aa41092d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 --2- 172.21.15.46:0/2357301033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4100ea0 0x7f4aa41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 --2- 172.21.15.46:0/2357301033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.706+0000 7f4aab800700 1 -- 172.21.15.46:0/2357301033 >> 172.21.15.46:0/2357301033 conn(0x7f4aa40fc430 msgr2=0x7f4aa40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.707+0000 7f4aab800700 1 -- 172.21.15.46:0/2357301033 shutdown_connections 2024-05-08T23:03:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.707+0000 7f4aab800700 1 -- 172.21.15.46:0/2357301033 wait complete. 2024-05-08T23:03:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 Processor -- start 2024-05-08T23:03:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 -- start start 2024-05-08T23:03:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 0x7f4aa4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 0x7f4aa41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41a0c30 con 0x7f4aa4101ff0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41a0d70 con 0x7f4aa4100ea0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aab800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41a0f00 con 0x7f4aa41029b0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 0x7f4aa4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53942/0 (socket says 172.21.15.46:53942) 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aa3e00700 1 -- 172.21.15.46:0/513229502 learned_addr learned my addr 172.21.15.46:0/513229502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.708+0000 7f4aa9800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 0x7f4aa41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 -- 172.21.15.46:0/513229502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 msgr2=0x7f4aa41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 0x7f4aa41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 -- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 msgr2=0x7f4aa4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 -- 172.21.15.46:0/513229502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98009520 con 0x7f4aa4100ea0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa9800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 0x7f4aa41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 0x7f4aa4199100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800e9f0 tx=0x7f4a9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa1600700 1 -- 172.21.15.46:0/513229502 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800cb70 con 0x7f4aa4100ea0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c009d00 con 0x7f4aa4100ea0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa1600700 1 -- 172.21.15.46:0/513229502 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9800ee90 con 0x7f4aa4100ea0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aa1600700 1 -- 172.21.15.46:0/513229502 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003cf0 con 0x7f4aa4100ea0 2024-05-08T23:03:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.709+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa41a15b0 con 0x7f4aa4100ea0 2024-05-08T23:03:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.712+0000 7f4aa1600700 1 -- 172.21.15.46:0/513229502 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4a9800ccd0 con 0x7f4aa4100ea0 2024-05-08T23:03:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.713+0000 7f4abf40eb80 1 Processor -- start 2024-05-08T23:03:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 -- start start 2024-05-08T23:03:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa1600700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 0x7f4a8c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 0x5597998d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa1600700 1 -- 172.21.15.46:0/513229502 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a98014070 con 0x7f4aa4100ea0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559799829100 con 0x5597998d3810 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559799826ac0 con 0x5597998d2e50 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597998223b0 con 0x5597998d24f0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa9800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 0x7f4a8c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52796/0 (socket says 172.21.15.46:52796) 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa3e00700 1 -- 172.21.15.46:0/1695185729 learned_addr learned my addr 172.21.15.46:0/1695185729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.714+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 0x5597998d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 -- 172.21.15.46:0/1695185729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 msgr2=0x5597998d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 0x5597998d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 -- 172.21.15.46:0/1695185729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 msgr2=0x5597998d3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998d3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 -- 172.21.15.46:0/1695185729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597996a3360 con 0x5597998d2e50 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 0x5597998d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa9800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 0x7f4a8c0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009610 tx=0x7f4a9c009340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ed0 tx=0x7f4a90011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1dc79271be1594 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4a96000700 1 -- 172.21.15.46:0/1695185729 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019630 con 0x5597998d2e50 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4a96000700 1 -- 172.21.15.46:0/1695185729 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90019c50 con 0x5597998d2e50 2024-05-08T23:03:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.715+0000 7f4a96000700 1 -- 172.21.15.46:0/1695185729 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000fbc0 con 0x5597998d2e50 2024-05-08T23:03:14.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1695185729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 msgr2=0x5597998d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ed0 tx=0x7f4a90011720 comp rx=0 tx=0).stop 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1695185729 shutdown_connections 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998d3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d24f0 0x5597998d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1695185729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998d32d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1695185729 >> 172.21.15.46:0/1695185729 conn(0x5597998301c0 msgr2=0x5597998ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1695185729 shutdown_connections 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.716+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1695185729 wait complete. 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.717+0000 7f4abf40eb80 1 -- start start 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.717+0000 7f4abf40eb80 1 Processor -- start 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.717+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d2e50 0x5597998dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597997be510 con 0x5597998d2e50 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597997be1b0 con 0x5597998d24f0 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597996a5650 con 0x5597998d3810 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4abf40eb80 10 client.0 _ll_register_callbacks cb 0x5597997c7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d2e50 0x5597998dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47906/0 (socket says 172.21.15.46:47906) 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 -- 172.21.15.46:0/923969098 learned_addr learned my addr 172.21.15.46:0/923969098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 -- 172.21.15.46:0/923969098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 msgr2=0x5597998dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 -- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d2e50 msgr2=0x5597998dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d2e50 0x5597998dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 -- 172.21.15.46:0/923969098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90009b00 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4aa9800700 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00b0a0 tx=0x7f4a9c00b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f34ad6e99fa6129 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.718+0000 7f4a8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c025070 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55979977b220 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a9c020660 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9c020820 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4abf40eb80 10 client.25252 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55979964fb10 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55979977b220 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a9c02b9e0 con 0x5597998d3810 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.719+0000 7f4abf40eb80 10 client.25252 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4abf40eb80 10 client.25252 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T23:03:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4abf40eb80 20 client.25252 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5597998e64f0 0x5597998ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4abf40eb80 4 client.25252 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597998a17c0 con 0x5597998e64f0 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5597998e64f0 0x5597998ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5597998e64f0 0x5597998ec330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fc296ad615b0af1 server_cookie=c5ee68cc368e0a7c in_seq=0 out_seq=0 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.720+0000 7f4a8aa00700 10 client.25252 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4a8aa00700 1 -- 172.21.15.46:0/923969098 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597998a17c0 con 0x5597998e64f0 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4a8aa00700 10 client.25252 handle_command_reply: tid=0 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 client.25252 shutdown 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 msgr2=0x5597998e3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00b0a0 tx=0x7f4a9c00b700 comp rx=0 tx=0).stop 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 shutdown_connections 2024-05-08T23:03:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3350 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5597998e64f0 0x5597998ec330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/923969098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d2e50 0x5597998dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 >> 172.21.15.46:0/923969098 conn(0x5597998301c0 msgr2=0x5597998cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.721+0000 7f4a8aa00700 0 client.25252 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.722+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 shutdown_connections 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.722+0000 7f4abf40eb80 1 -- 172.21.15.46:0/923969098 wait complete. 2024-05-08T23:03:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.722+0000 7f4abf40eb80 20 client.25252 trim_cache size 0 max 16384 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 Processor -- start 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 -- start start 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979981f040 con 0x5597998d24f0 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597997c15d0 con 0x5597998d2e50 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597998c9d70 con 0x5597998d3810 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53970/0 (socket says 172.21.15.46:53970) 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.741+0000 7f4aa8e00700 1 -- 172.21.15.46:0/3132990316 learned_addr learned my addr 172.21.15.46:0/3132990316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa9800700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 -- 172.21.15.46:0/3132990316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 msgr2=0x5597998e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 -- 172.21.15.46:0/3132990316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 msgr2=0x5597998e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 -- 172.21.15.46:0/3132990316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597997e8510 con 0x5597998d24f0 2024-05-08T23:03:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa9800700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a98096a10 tx=0x7f4a980cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace27105d94d7c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4a96a00700 1 -- 172.21.15.46:0/3132990316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9809ddd0 con 0x5597998d24f0 2024-05-08T23:03:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4a96a00700 1 -- 172.21.15.46:0/3132990316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a980a2420 con 0x5597998d24f0 2024-05-08T23:03:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.742+0000 7f4a96a00700 1 -- 172.21.15.46:0/3132990316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a980996f0 con 0x5597998d24f0 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- 172.21.15.46:0/3132990316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 msgr2=0x5597998d28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a98096a10 tx=0x7f4a980cc040 comp rx=0 tx=0).stop 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- 172.21.15.46:0/3132990316 shutdown_connections 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d3810 0x5597998e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d24f0 0x5597998d28d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/3132990316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d2e50 0x5597998e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- 172.21.15.46:0/3132990316 >> 172.21.15.46:0/3132990316 conn(0x5597998301c0 msgr2=0x5597998d3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- 172.21.15.46:0/3132990316 shutdown_connections 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- 172.21.15.46:0/3132990316 wait complete. 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.743+0000 7f4abf40eb80 1 -- start start 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 Processor -- start 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d2e50 0x5597998e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597998e3780 con 0x5597998d3810 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979993dff0 con 0x5597998d24f0 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979993e160 con 0x5597998d2e50 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4abf40eb80 10 client.0 _ll_register_callbacks cb 0x5597997c7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d2e50 0x5597998e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52816/0 (socket says 172.21.15.46:52816) 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa8e00700 1 -- 172.21.15.46:0/1238291586 learned_addr learned my addr 172.21.15.46:0/1238291586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.744+0000 7f4aa9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53984/0 (socket says 172.21.15.46:53984) 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 -- 172.21.15.46:0/1238291586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d2e50 msgr2=0x5597998e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d2e50 0x5597998e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 -- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 msgr2=0x5597998e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 -- 172.21.15.46:0/1238291586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa41a15b0 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa9800700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4aa8e00700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a980cc010 tx=0x7f4a9809a810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49419ea541279c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4a8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9809ddd0 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a98099490 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a90019c50 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a98099650 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4abf40eb80 10 client.25244 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.745+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5597996a7c80 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90019c50 con 0x5597998d24f0 2024-05-08T23:03:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a9807eb90 con 0x5597998d24f0 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 10 client.25244 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 20 client.25244 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559799942c60 0x55979993f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 4 client.25244 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55979981ae70 con 0x559799942c60 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.746+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559799942c60 0x55979993f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4aa3e00700 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559799942c60 0x55979993f610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0c6eb6043f5e273 server_cookie=987919e40b438fd9 in_seq=0 out_seq=0 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4a8aa00700 10 client.25244 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4a8aa00700 1 -- 172.21.15.46:0/1238291586 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55979981ae70 con 0x559799942c60 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4a8aa00700 10 client.25244 handle_command_reply: tid=0 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 client.25244 shutdown 2024-05-08T23:03:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 msgr2=0x5597998e04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a980cc010 tx=0x7f4a9809a810 comp rx=0 tx=0).stop 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 shutdown_connections 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559799942c60 0x55979993f610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597998d3810 0x5597998e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597998d24f0 0x5597998e04f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4a8aa00700 0 client.25244 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 --2- 172.21.15.46:0/1238291586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597998d2e50 0x5597998e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 >> 172.21.15.46:0/1238291586 conn(0x5597998301c0 msgr2=0x5597998db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 shutdown_connections 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 1 -- 172.21.15.46:0/1238291586 wait complete. 2024-05-08T23:03:14.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.747+0000 7f4abf40eb80 20 client.25244 trim_cache size 0 max 16384 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24583, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 60.116103694000003, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 94.414718820999994 2024-05-08T23:03:14.752 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 msgr2=0x7f4a8c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 0x7f4a8c0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009610 tx=0x7f4a9c009340 comp rx=0 tx=0).stop 2024-05-08T23:03:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 msgr2=0x7f4aa4199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 0x7f4aa4199100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800e9f0 tx=0x7f4a9800ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 shutdown_connections 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa41029b0 0x7f4aa41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4aa4100ea0 0x7f4aa4199100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4a8c0a1230 0x7f4a8c0a36f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 --2- 172.21.15.46:0/513229502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa4101ff0 0x7f4aa4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 >> 172.21.15.46:0/513229502 conn(0x7f4aa40fc430 msgr2=0x7f4aa40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 shutdown_connections 2024-05-08T23:03:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.748+0000 7f4aab800700 1 -- 172.21.15.46:0/513229502 wait complete. 2024-05-08T23:03:14.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 -- 172.21.15.46:0/2557506782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 msgr2=0x7efe88073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2557506782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88073e80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efe74009990 tx=0x7efe74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 -- 172.21.15.46:0/2557506782 shutdown_connections 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2557506782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88108d40 0x7efe8810b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2557506782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88074fe0 0x7efe88073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2557506782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88073e80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 -- 172.21.15.46:0/2557506782 >> 172.21.15.46:0/2557506782 conn(0x7efe880fc3b0 msgr2=0x7efe880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 -- 172.21.15.46:0/2557506782 shutdown_connections 2024-05-08T23:03:14.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.966+0000 7efe8fc00700 1 -- 172.21.15.46:0/2557506782 wait complete. 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.967+0000 7efe8fc00700 1 Processor -- start 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 -- start start 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88074fe0 0x7efe881995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88108d40 0x7efe881a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881a0b60 con 0x7efe88073a00 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881a0ca0 con 0x7efe88108d40 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881a0e30 con 0x7efe88074fe0 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53988/0 (socket says 172.21.15.46:53988) 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8d200700 1 -- 172.21.15.46:0/2113660841 learned_addr learned my addr 172.21.15.46:0/2113660841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe87e00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88074fe0 0x7efe881995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.968+0000 7efe8dc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88108d40 0x7efe881a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 -- 172.21.15.46:0/2113660841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88074fe0 msgr2=0x7efe881995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88074fe0 0x7efe881995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 -- 172.21.15.46:0/2113660841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88108d40 msgr2=0x7efe881a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88108d40 0x7efe881a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 -- 172.21.15.46:0/2113660841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c009520 con 0x7efe88073a00 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8d200700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eb30 tx=0x7efe7c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe85600700 1 -- 172.21.15.46:0/2113660841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c009880 con 0x7efe88073a00 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe85600700 1 -- 172.21.15.46:0/2113660841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe7c0044e0 con 0x7efe88073a00 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe85600700 1 -- 172.21.15.46:0/2113660841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c003d80 con 0x7efe88073a00 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe740095f0 con 0x7efe88073a00 2024-05-08T23:03:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.969+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe881a1440 con 0x7efe88073a00 2024-05-08T23:03:14.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.973+0000 7efe85600700 1 -- 172.21.15.46:0/2113660841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efe7c003f20 con 0x7efe88073a00 2024-05-08T23:03:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.974+0000 7efe85600700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 0x7efe700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.974+0000 7efea3635b80 1 Processor -- start 2024-05-08T23:03:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe85600700 1 -- 172.21.15.46:0/2113660841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe7c014070 con 0x7efe88073a00 2024-05-08T23:03:14.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe87e00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 0x7efe700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 -- start start 2024-05-08T23:03:14.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb59910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb5a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 0x55f67bb5ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67bab0100 con 0x55f67bb594f0 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67baadac0 con 0x55f67bb5a810 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efea3635b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67baa93b0 con 0x55f67bb59e50 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb59910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb59910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54000/0 (socket says 172.21.15.46:54000) 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe8d200700 1 -- 172.21.15.46:0/4266162289 learned_addr learned my addr 172.21.15.46:0/4266162289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe8dc00700 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 0x55f67bb5ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.975+0000 7efe87e00700 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb5a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe87e00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 0x7efe700a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b680 tx=0x7efe74005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 -- 172.21.15.46:0/4266162289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 msgr2=0x55f67bb5a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb5a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 -- 172.21.15.46:0/4266162289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 msgr2=0x55f67bb59910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb59910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 -- 172.21.15.46:0/4266162289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f67b92a360 con 0x55f67bb5a810 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe8dc00700 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 0x55f67bb5ac90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe800115b0 tx=0x7efe80009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fa5191521a154a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe7a000700 1 -- 172.21.15.46:0/4266162289 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8000e040 con 0x55f67bb5a810 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe7a000700 1 -- 172.21.15.46:0/4266162289 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8000f040 con 0x55f67bb5a810 2024-05-08T23:03:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.976+0000 7efe7a000700 1 -- 172.21.15.46:0/4266162289 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80004430 con 0x55f67bb5a810 2024-05-08T23:03:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 -- 172.21.15.46:0/4266162289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 msgr2=0x55f67bb5ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 0x55f67bb5ac90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe800115b0 tx=0x7efe80009920 comp rx=0 tx=0).stop 2024-05-08T23:03:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 -- 172.21.15.46:0/4266162289 shutdown_connections 2024-05-08T23:03:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb5a810 0x55f67bb5ac90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb59910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 --2- 172.21.15.46:0/4266162289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb5a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 -- 172.21.15.46:0/4266162289 >> 172.21.15.46:0/4266162289 conn(0x55f67bab71c0 msgr2=0x55f67bb53d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 -- 172.21.15.46:0/4266162289 shutdown_connections 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.977+0000 7efea3635b80 1 -- 172.21.15.46:0/4266162289 wait complete. 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 -- start start 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 Processor -- start 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb62d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ba45510 con 0x55f67bb5a810 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ba451b0 con 0x55f67bb594f0 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67b92c650 con 0x55f67bb59e50 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.978+0000 7efea3635b80 10 client.0 _ll_register_callbacks cb 0x55f67ba4eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe8dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe8d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb62d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe8d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb62d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52848/0 (socket says 172.21.15.46:52848) 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe8d200700 1 -- 172.21.15.46:0/4066741691 learned_addr learned my addr 172.21.15.46:0/4066741691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 -- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 msgr2=0x55f67bb62d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb62d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 -- 172.21.15.46:0/4066741691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 msgr2=0x55f67bb6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 -- 172.21.15.46:0/4066741691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe80011230 con 0x55f67bb59e50 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.979+0000 7efe87e00700 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb632a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efe74000c00 tx=0x7efe74009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa83a4a147500e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74004dd0 con 0x55f67bb59e50 2024-05-08T23:03:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f67ba02220 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe7400f460 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7400f620 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efea3635b80 10 client.25270 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f67b8d6b10 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f67ba02220 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.980+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe740306c0 con 0x55f67bb59e50 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 10 client.25270 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 10 client.25270 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 20 client.25270 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bb6df40 0x55f67bb73240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 4 client.25270 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f67bb287c0 con 0x55f67bb6df40 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.981+0000 7efe8d200700 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bb6df40 0x55f67bb73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.982+0000 7efe8d200700 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bb6df40 0x55f67bb73240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aecee7ee689412a server_cookie=2a5dcc05be8478d8 in_seq=0 out_seq=0 2024-05-08T23:03:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.982+0000 7efe6ea00700 10 client.25270 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.982+0000 7efe6ea00700 1 -- 172.21.15.46:0/4066741691 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f67bb287c0 con 0x55f67bb6df40 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.982+0000 7efe6ea00700 10 client.25270 handle_command_reply: tid=0 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.982+0000 7efea3635b80 1 client.25270 shutdown 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 msgr2=0x55f67bb632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb632a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efe74000c00 tx=0x7efe74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 shutdown_connections 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bb6df40 0x55f67bb73240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb62d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 --2- 172.21.15.46:0/4066741691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb632a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:14.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efe6ea00700 0 client.25270 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 >> 172.21.15.46:0/4066741691 conn(0x55f67bab71c0 msgr2=0x55f67bb53550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 shutdown_connections 2024-05-08T23:03:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 1 -- 172.21.15.46:0/4066741691 wait complete. 2024-05-08T23:03:14.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:14.983+0000 7efea3635b80 20 client.25270 trim_cache size 0 max 16384 2024-05-08T23:03:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 Processor -- start 2024-05-08T23:03:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 -- start start 2024-05-08T23:03:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb6a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67baa2150 con 0x55f67bb5a810 2024-05-08T23:03:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67baa6010 con 0x55f67bb594f0 2024-05-08T23:03:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.004+0000 7efea3635b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67ba485a0 con 0x55f67bb59e50 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52850/0 (socket says 172.21.15.46:52850) 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb6a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 -- 172.21.15.46:0/1432772487 learned_addr learned my addr 172.21.15.46:0/1432772487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 -- 172.21.15.46:0/1432772487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 msgr2=0x55f67bb6a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb6a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 -- 172.21.15.46:0/1432772487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 msgr2=0x55f67bb60d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb60d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 -- 172.21.15.46:0/1432772487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f67ba6f510 con 0x55f67bb594f0 2024-05-08T23:03:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8dc00700 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb60d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.005+0000 7efe8d200700 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0a2230 tx=0x7efe7c0966d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db96ef65b638d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.008+0000 7efe7aa00700 1 -- 172.21.15.46:0/1432772487 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0cd120 con 0x55f67bb594f0 2024-05-08T23:03:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.008+0000 7efe7aa00700 1 -- 172.21.15.46:0/1432772487 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe7c0b2e90 con 0x55f67bb594f0 2024-05-08T23:03:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.008+0000 7efe7aa00700 1 -- 172.21.15.46:0/1432772487 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0041d0 con 0x55f67bb594f0 2024-05-08T23:03:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 -- 172.21.15.46:0/1432772487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 msgr2=0x55f67bb598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0a2230 tx=0x7efe7c0966d0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 -- 172.21.15.46:0/1432772487 shutdown_connections 2024-05-08T23:03:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb5a810 0x55f67bb60d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb594f0 0x55f67bb598d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 --2- 172.21.15.46:0/1432772487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb59e50 0x55f67bb6a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 -- 172.21.15.46:0/1432772487 >> 172.21.15.46:0/1432772487 conn(0x55f67bab71c0 msgr2=0x55f67bb5abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 -- 172.21.15.46:0/1432772487 shutdown_connections 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.009+0000 7efea3635b80 1 -- 172.21.15.46:0/1432772487 wait complete. 2024-05-08T23:03:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 -- start start 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 Processor -- start 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 0x55f67bb692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb5a810 0x55f67bb69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67bb6a040 con 0x55f67bb594f0 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67bb6a1b0 con 0x55f67bb59e50 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67bb6a370 con 0x55f67bb5a810 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.010+0000 7efea3635b80 10 client.0 _ll_register_callbacks cb 0x55f67ba4eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 0x55f67bb692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe8d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54028/0 (socket says 172.21.15.46:54028) 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe8d200700 1 -- 172.21.15.46:0/1967543279 learned_addr learned my addr 172.21.15.46:0/1967543279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe8dc00700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb5a810 0x55f67bb69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 -- 172.21.15.46:0/1967543279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb5a810 msgr2=0x55f67bb69960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb5a810 0x55f67bb69960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 -- 172.21.15.46:0/1967543279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 msgr2=0x55f67bb53780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 -- 172.21.15.46:0/1967543279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f67b92a360 con 0x55f67bb59e50 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe8d200700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe87e00700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 0x55f67bb692e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b7b0 tx=0x7efe740058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e559896f9d39861f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:15.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74029040 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe8000f040 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.011+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe74030420 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe740305e0 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 10 client.25262 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7efe881a1440 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8000f040 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe74046070 con 0x55f67bb59e50 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 10 client.25262 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 10 client.25262 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 20 client.25262 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bbc9440 0x55f67bbc5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 4 client.25262 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.012+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f67baa1e70 con 0x55f67bbc9440 2024-05-08T23:03:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efe8d200700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bbc9440 0x55f67bbc5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efe8d200700 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bbc9440 0x55f67bbc5dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa3608ed7a33eff1 server_cookie=973ef3d26ef99908 in_seq=0 out_seq=0 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efe6ea00700 10 client.25262 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efe6ea00700 1 -- 172.21.15.46:0/1967543279 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f67baa1e70 con 0x55f67bbc9440 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efe6ea00700 10 client.25262 handle_command_reply: tid=0 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.013+0000 7efea3635b80 1 client.25262 shutdown 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 msgr2=0x55f67bb692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 0x55f67bb692e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b7b0 tx=0x7efe740058e0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 shutdown_connections 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f67bb5a810 0x55f67bb69960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f67bbc9440 0x55f67bbc5dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f67bb594f0 0x55f67bb53780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 --2- 172.21.15.46:0/1967543279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f67bb59e50 0x55f67bb692e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 >> 172.21.15.46:0/1967543279 conn(0x55f67bab71c0 msgr2=0x55f67bb5d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efe6ea00700 0 client.25262 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 shutdown_connections 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 1 -- 172.21.15.46:0/1967543279 wait complete. 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efea3635b80 20 client.25262 trim_cache size 0 max 16384 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24587, 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 83.549349294999999, 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 90.000310024000001 2024-05-08T23:03:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.014+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 msgr2=0x7efe700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 0x7efe700a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b680 tx=0x7efe74005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 msgr2=0x7efe88199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eb30 tx=0x7efe7c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 shutdown_connections 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe88108d40 0x7efe881a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7efe700a11b0 0x7efe700a3670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe88073a00 0x7efe88199080 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 --2- 172.21.15.46:0/2113660841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88074fe0 0x7efe881995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 >> 172.21.15.46:0/2113660841 conn(0x7efe880fc3b0 msgr2=0x7efe88104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 shutdown_connections 2024-05-08T23:03:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:15.015+0000 7efe8fc00700 1 -- 172.21.15.46:0/2113660841 wait complete. 2024-05-08T23:03:15.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:15 smithi046 ceph-mon[36898]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:15.032 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T23:03:15.032 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:03:15.041 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:03:15.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T23:03:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:15 smithi053 ceph-mon[77587]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:15.407 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:15.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:15 smithi086 ceph-mon[78610]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:16.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.560+0000 7f568ec00700 1 -- 172.21.15.46:0/3954429989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688073a00 msgr2=0x7f5688073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:16.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.560+0000 7f568ec00700 1 --2- 172.21.15.46:0/3954429989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688073a00 0x7f5688073e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5674009990 tx=0x7f5674009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.560+0000 7f568ec00700 1 -- 172.21.15.46:0/3954429989 shutdown_connections 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 --2- 172.21.15.46:0/3954429989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688108ed0 0x7f568810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 --2- 172.21.15.46:0/3954429989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688074fd0 0x7f5688073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 --2- 172.21.15.46:0/3954429989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688073a00 0x7f5688073e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 -- 172.21.15.46:0/3954429989 >> 172.21.15.46:0/3954429989 conn(0x7f56880fc5a0 msgr2=0x7f56880fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 -- 172.21.15.46:0/3954429989 shutdown_connections 2024-05-08T23:03:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.561+0000 7f568ec00700 1 -- 172.21.15.46:0/3954429989 wait complete. 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 Processor -- start 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 -- start start 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 0x7f5688104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688074fd0 0x7f56881047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881015b0 con 0x7f5688108ed0 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688101720 con 0x7f5688073a00 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881018e0 con 0x7f5688074fd0 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f5687e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 0x7f5688104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54054/0 (socket says 172.21.15.46:54054) 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f5687400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688074fd0 0x7f56881047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.562+0000 7f568cc00700 1 -- 172.21.15.46:0/816029794 learned_addr learned my addr 172.21.15.46:0/816029794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 -- 172.21.15.46:0/816029794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688074fd0 msgr2=0x7f56881047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688074fd0 0x7f56881047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 -- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 msgr2=0x7f5688100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 -- 172.21.15.46:0/816029794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740095f0 con 0x7f5688073a00 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f568cc00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5687e00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 0x7f5688104290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f567800e9c0 tx=0x7f567800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800c9c0 con 0x7f5688073a00 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5678010430 con 0x7f5688073a00 2024-05-08T23:03:16.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678018680 con 0x7f5688073a00 2024-05-08T23:03:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.563+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56881a5680 con 0x7f5688073a00 2024-05-08T23:03:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.564+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881a5b50 con 0x7f5688073a00 2024-05-08T23:03:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.565+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688107c90 con 0x7f5688073a00 2024-05-08T23:03:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.567+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f567800cb20 con 0x7f5688073a00 2024-05-08T23:03:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.568+0000 7f5684c00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 0x7f566c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.568+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5678014070 con 0x7f5688073a00 2024-05-08T23:03:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.568+0000 7f5687400700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 0x7f566c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:16.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.569+0000 7f5687400700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 0x7f566c0a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f567400b680 tx=0x7f5674005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.571+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56780963c0 con 0x7f5688073a00 2024-05-08T23:03:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.801+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56880613e0 con 0x7f566c0a1230 2024-05-08T23:03:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.803+0000 7f5684c00700 1 -- 172.21.15.46:0/816029794 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f56880613e0 con 0x7f566c0a1230 2024-05-08T23:03:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-05-08T23:03:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-05-08T23:03:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 msgr2=0x7f566c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 0x7f566c0a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f567400b680 tx=0x7f5674005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 msgr2=0x7f5688104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 0x7f5688104290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f567800e9c0 tx=0x7f567800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 shutdown_connections 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f566c0a1230 0x7f566c0a36f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5688108ed0 0x7f5688100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5688073a00 0x7f5688104290 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 --2- 172.21.15.46:0/816029794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5688074fd0 0x7f56881047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 >> 172.21.15.46:0/816029794 conn(0x7f56880fc5a0 msgr2=0x7f5688104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 shutdown_connections 2024-05-08T23:03:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:16.805+0000 7f568ec00700 1 -- 172.21.15.46:0/816029794 wait complete. 2024-05-08T23:03:17.169 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:17 smithi046 ceph-mon[36898]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:17.169 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:17 smithi046 ceph-mon[36898]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:17.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T23:03:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:17 smithi053 ceph-mon[77587]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:17 smithi053 ceph-mon[77587]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:17.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:17 smithi086 ceph-mon[78610]: from='client.25268 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:17.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:17 smithi086 ceph-mon[78610]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-08T23:03:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T23:03:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:18 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T23:03:18.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-08T23:03:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T23:03:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:18 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T23:03:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-08T23:03:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T23:03:18.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T23:03:18.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:18 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T23:03:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1891107789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38101000 msgr2=0x7f1f38101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1891107789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38101000 0x7f1f38101420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009990 tx=0x7f1f2c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1891107789 shutdown_connections 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1891107789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102ba0 0x7f1f38109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1891107789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38101000 0x7f1f38101420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1891107789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38102200 0x7f1f38102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1891107789 >> 172.21.15.46:0/1891107789 conn(0x7f1f380fc5a0 msgr2=0x7f1f380fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.887+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1891107789 shutdown_connections 2024-05-08T23:03:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.888+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1891107789 wait complete. 2024-05-08T23:03:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.888+0000 7f1f3fc00700 1 Processor -- start 2024-05-08T23:03:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 -- start start 2024-05-08T23:03:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 0x7f1f381974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38102ba0 0x7f1f3819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819eab0 con 0x7f1f38101000 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819ebf0 con 0x7f1f38102200 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819ed80 con 0x7f1f38102ba0 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54080/0 (socket says 172.21.15.46:54080) 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3d200700 1 -- 172.21.15.46:0/1506827590 learned_addr learned my addr 172.21.15.46:0/1506827590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f37e00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 0x7f1f381974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.889+0000 7f1f3dc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38102ba0 0x7f1f3819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 -- 172.21.15.46:0/1506827590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38102ba0 msgr2=0x7f1f3819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38102ba0 0x7f1f3819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 -- 172.21.15.46:0/1506827590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 msgr2=0x7f1f381974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 0x7f1f381974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 -- 172.21.15.46:0/1506827590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0095f0 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f37e00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 0x7f1f381974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3d200700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c005b40 tx=0x7f1f2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c01d070 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2c021410 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3819f000 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.890+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c017650 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.891+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3819f410 con 0x7f1f38101000 2024-05-08T23:03:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.892+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f381084e0 con 0x7f1f38101000 2024-05-08T23:03:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.893+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1f2c021580 con 0x7f1f38101000 2024-05-08T23:03:18.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.895+0000 7f1f35600700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 0x7f1f200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.895+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1f2c0ce1d0 con 0x7f1f38101000 2024-05-08T23:03:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.895+0000 7f1f37e00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 0x7f1f200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.896+0000 7f1f37e00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 0x7f1f200a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011490 tx=0x7f1f24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:18.899+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f2c097490 con 0x7f1f38101000 2024-05-08T23:03:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.134+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f3802cdf0 con 0x7f1f200a11b0 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.148+0000 7f1f35600700 1 -- 172.21.15.46:0/1506827590 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1f3802cdf0 con 0x7f1f200a11b0 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (96s) 20s ago 96s 13.8M - 18.2.2-1322-gd21e4f96 af23193ff691 365257641f44 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi053 running (94s) 28s ago 94s 30.9M - 18.2.2-1322-gd21e4f96 af23193ff691 24090139d1d8 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi086 running (105s) 28s ago 105s 18.2M - 18.2.2-1322-gd21e4f96 af23193ff691 cae6151f52c9 2024-05-08T23:03:19.147 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (110s) 20s ago 110s 16.4M - 18.2.2-1322-gd21e4f96 af23193ff691 c34a970844a8 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi053 running (108s) 28s ago 108s 18.8M - 18.2.2-1322-gd21e4f96 af23193ff691 4780d7bb5d5c 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi086 running (107s) 28s ago 106s 17.3M - 18.2.2-1322-gd21e4f96 af23193ff691 537202b25f78 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (99s) 20s ago 98s 17.3M - 18.2.2-1322-gd21e4f96 af23193ff691 fb8e37e01512 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi053 running (103s) 28s ago 103s 20.1M - 18.2.2-1322-gd21e4f96 af23193ff691 c3ba96742247 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi086 running (101s) 28s ago 101s 18.7M - 18.2.2-1322-gd21e4f96 af23193ff691 690363dbe5da 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (113s) 20s ago 113s 13.8M - 18.2.2-1322-gd21e4f96 af23193ff691 47cfb5ecfb8b 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi053 running (115s) 28s ago 115s 17.5M - 18.2.2-1322-gd21e4f96 af23193ff691 0c7cc412a6df 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi086 running (117s) 28s ago 117s 18.9M - 18.2.2-1322-gd21e4f96 af23193ff691 559cf1242a9f 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (7m) 20s ago 7m 539M - 18.2.2-1322-gd21e4f96 af23193ff691 b5657c15bd3d 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi053 *:8443,8765 running (6m) 28s ago 6m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 5976b94f9304 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi086 *:8443,8765 running (6m) 28s ago 6m 458M - 18.2.2-1322-gd21e4f96 af23193ff691 0a28832d27c4 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (7m) 20s ago 7m 90.2M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 4d191837d87e 2024-05-08T23:03:19.148 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi053 running (6m) 28s ago 6m 65.1M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 849d003eb504 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi086 running (6m) 28s ago 6m 64.1M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 99da62d85626 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (5m) 20s ago 5m 69.0M 4318M 18.2.2-1322-gd21e4f96 af23193ff691 78b9874d1d4e 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi053 running (5m) 28s ago 5m 152M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 b3fae9ce0d5a 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi086 running (5m) 28s ago 5m 142M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 fb98eb7abd5b 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (4m) 20s ago 4m 64.9M 4318M 18.2.2-1322-gd21e4f96 af23193ff691 7483a3ead5db 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi053 running (4m) 28s ago 4m 137M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 0d3d18019d12 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi086 running (4m) 28s ago 4m 69.9M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 2f53a5108058 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (3m) 20s ago 3m 133M 4318M 18.2.2-1322-gd21e4f96 af23193ff691 587e2caa49ed 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi053 running (3m) 28s ago 3m 139M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 2464491466cf 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi086 running (3m) 28s ago 3m 62.2M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 77edb54e4eab 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (2m) 20s ago 2m 62.9M 4318M 18.2.2-1322-gd21e4f96 af23193ff691 6416d5bca676 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi053 running (2m) 28s ago 2m 62.6M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 21551e3654af 2024-05-08T23:03:19.149 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi086 running (2m) 28s ago 2m 128M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 45a2141e5ee3 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 msgr2=0x7f1f200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 0x7f1f200a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011490 tx=0x7f1f24009260 comp rx=0 tx=0).stop 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 msgr2=0x7f1f38196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c005b40 tx=0x7f1f2c004730 comp rx=0 tx=0).stop 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 shutdown_connections 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f38102ba0 0x7f1f3819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f1f200a11b0 0x7f1f200a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:19.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38101000 0x7f1f38196f70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:19.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 --2- 172.21.15.46:0/1506827590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f38102200 0x7f1f381974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:19.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 >> 172.21.15.46:0/1506827590 conn(0x7f1f380fc5a0 msgr2=0x7f1f38105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:19.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.150+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 shutdown_connections 2024-05-08T23:03:19.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:19.151+0000 7f1f3fc00700 1 -- 172.21.15.46:0/1506827590 wait complete. 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:03:19.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:19 smithi046 ceph-mon[36898]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T23:03:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:03:19.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:19 smithi086 ceph-mon[78610]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: from='mgr.14150 172.21.15.46:0/1021985164' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T23:03:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:03:19.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:19 smithi053 ceph-mon[77587]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:19.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T23:03:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:20 smithi046 ceph-mon[36898]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:03:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:20 smithi046 ceph-mon[36898]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:20.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:20 smithi086 ceph-mon[78610]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:03:20.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:20 smithi086 ceph-mon[78610]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:20.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:20 smithi053 ceph-mon[77587]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:03:20.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:20 smithi053 ceph-mon[77587]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:21.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 -- 172.21.15.46:0/1820521581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102200 msgr2=0x7ff990102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 --2- 172.21.15.46:0/1820521581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102200 0x7ff990102660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0099c0 tx=0x7ff97c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 -- 172.21.15.46:0/1820521581 shutdown_connections 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 --2- 172.21.15.46:0/1820521581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990102ba0 0x7ff990109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 --2- 172.21.15.46:0/1820521581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990101000 0x7ff990101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 --2- 172.21.15.46:0/1820521581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102200 0x7ff990102660 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 -- 172.21.15.46:0/1820521581 >> 172.21.15.46:0/1820521581 conn(0x7ff9900fc5a0 msgr2=0x7ff9900fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 -- 172.21.15.46:0/1820521581 shutdown_connections 2024-05-08T23:03:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.064+0000 7ff998000700 1 -- 172.21.15.46:0/1820521581 wait complete. 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.065+0000 7ff998000700 1 Processor -- start 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.065+0000 7ff998000700 1 -- start start 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.065+0000 7ff998000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990101000 0x7ff9901990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff998000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990102200 0x7ff990199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff998000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff998000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a0c30 con 0x7ff990102ba0 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff998000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a0d70 con 0x7ff990102200 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff998000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a0f00 con 0x7ff990101000 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff995600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990101000 0x7ff9901990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:21.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54090/0 (socket says 172.21.15.46:54090) 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff994c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990102200 0x7ff990199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 -- 172.21.15.46:0/2616629500 learned_addr learned my addr 172.21.15.46:0/2616629500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 -- 172.21.15.46:0/2616629500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990101000 msgr2=0x7ff9901990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990101000 0x7ff9901990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 -- 172.21.15.46:0/2616629500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990102200 msgr2=0x7ff990199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990102200 0x7ff990199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.066+0000 7ff996000700 1 -- 172.21.15.46:0/2616629500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0095f0 con 0x7ff990102ba0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff996000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff98800ef60 tx=0x7ff98800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988018070 con 0x7ff990102ba0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98800f040 con 0x7ff990102ba0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901a1180 con 0x7ff990102ba0 2024-05-08T23:03:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800ccc0 con 0x7ff990102ba0 2024-05-08T23:03:21.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.067+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901a1630 con 0x7ff990102ba0 2024-05-08T23:03:21.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.068+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99004ec60 con 0x7ff990102ba0 2024-05-08T23:03:21.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.070+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff988003a00 con 0x7ff990102ba0 2024-05-08T23:03:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.072+0000 7ff98e000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 0x7ff9780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.073+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7ff9880cd2b0 con 0x7ff990102ba0 2024-05-08T23:03:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.073+0000 7ff995600700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 0x7ff9780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.073+0000 7ff995600700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 0x7ff9780a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff990102060 tx=0x7ff984009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.076+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9880965d0 con 0x7ff990102ba0 2024-05-08T23:03:21.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:21 smithi046 ceph-mon[36898]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.299+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9900748f0 con 0x7ff9780a11b0 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.305+0000 7ff98e000700 1 -- 172.21.15.46:0/2616629500 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ff9900748f0 con 0x7ff9780a11b0 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 30s ago 2m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi086=c;smithi086=f;smithi086=i;smithi086=l;count:12 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 30s ago 6m smithi046=x;smithi053=y;smithi086=z;count:3 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 30s ago 6m smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi086:172.21.15.86=c;count:3 2024-05-08T23:03:21.304 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 30s ago - 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 msgr2=0x7ff9780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 0x7ff9780a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff990102060 tx=0x7ff984009260 comp rx=0 tx=0).stop 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 msgr2=0x7ff9901a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff98800ef60 tx=0x7ff98800ef90 comp rx=0 tx=0).stop 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 shutdown_connections 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff9780a11b0 0x7ff9780a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff990102ba0 0x7ff9901a0660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff990101000 0x7ff9901990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 --2- 172.21.15.46:0/2616629500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff990102200 0x7ff990199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 >> 172.21.15.46:0/2616629500 conn(0x7ff9900fc5a0 msgr2=0x7ff9900731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 shutdown_connections 2024-05-08T23:03:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:21.307+0000 7ff998000700 1 -- 172.21.15.46:0/2616629500 wait complete. 2024-05-08T23:03:21.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:21 smithi086 ceph-mon[78610]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:21 smithi053 ceph-mon[77587]: pgmap v288: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:21.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T23:03:22.040 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:22.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:22 smithi046 ceph-mon[36898]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-08T23:03:22.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:22 smithi046 ceph-mon[36898]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:22 smithi086 ceph-mon[78610]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-08T23:03:22.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:22 smithi086 ceph-mon[78610]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:22.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:22 smithi053 ceph-mon[77587]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-08T23:03:22.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:22 smithi053 ceph-mon[77587]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 -- 172.21.15.46:0/785775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 msgr2=0x7fc8e4109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 --2- 172.21.15.46:0/785775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4109180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b0a0 tx=0x7fc8dc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 -- 172.21.15.46:0/785775192 shutdown_connections 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 --2- 172.21.15.46:0/785775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4109180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 --2- 172.21.15.46:0/785775192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4101b30 0x7fc8e4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 --2- 172.21.15.46:0/785775192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4104460 0x7fc8e4106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 -- 172.21.15.46:0/785775192 >> 172.21.15.46:0/785775192 conn(0x7fc8e40fb4a0 msgr2=0x7fc8e40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 -- 172.21.15.46:0/785775192 shutdown_connections 2024-05-08T23:03:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.237+0000 7fc8ec200700 1 -- 172.21.15.46:0/785775192 wait complete. 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.238+0000 7fc8ec200700 1 Processor -- start 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.238+0000 7fc8ec200700 1 -- start start 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4104460 0x7fc8e40ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41007c0 con 0x7fc8e4106d90 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4100930 con 0x7fc8e4101b30 2024-05-08T23:03:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ec200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41a1040 con 0x7fc8e4104460 2024-05-08T23:03:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8e9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8ea200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8e9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34528/0 (socket says 172.21.15.46:34528) 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8e9800700 1 -- 172.21.15.46:0/3463490523 learned_addr learned my addr 172.21.15.46:0/3463490523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.239+0000 7fc8e8e00700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4104460 0x7fc8e40ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 -- 172.21.15.46:0/3463490523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4104460 msgr2=0x7fc8e40ffae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4104460 0x7fc8e40ffae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 -- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 msgr2=0x7fc8e4100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 -- 172.21.15.46:0/3463490523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc009d00 con 0x7fc8e4101b30 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e9800700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e9a0 tx=0x7fc8d800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8009790 con 0x7fc8e4101b30 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d8010430 con 0x7fc8e4101b30 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e41a12d0 con 0x7fc8e4101b30 2024-05-08T23:03:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.240+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8018680 con 0x7fc8e4101b30 2024-05-08T23:03:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.241+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e41a1820 con 0x7fc8e4101b30 2024-05-08T23:03:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.242+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e40fd020 con 0x7fc8e4101b30 2024-05-08T23:03:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.244+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc8d80105a0 con 0x7fc8e4101b30 2024-05-08T23:03:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.245+0000 7fc8e2000700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 0x7fc8cc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.245+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc8d8014070 con 0x7fc8e4101b30 2024-05-08T23:03:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.245+0000 7fc8e8e00700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 0x7fc8cc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.246+0000 7fc8e8e00700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 0x7fc8cc0a36f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009c60 tx=0x7fc8d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.248+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8d8096220 con 0x7fc8e4101b30 2024-05-08T23:03:23.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:23 smithi046 ceph-mon[36898]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:23.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:23 smithi086 ceph-mon[78610]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.469+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e40613e0 con 0x7fc8cc0a1230 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.471+0000 7fc8e2000700 1 -- 172.21.15.46:0/3463490523 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7fc8e40613e0 con 0x7fc8cc0a1230 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stdout:smithi053 172.21.15.53 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stdout:smithi086 172.21.15.86 2024-05-08T23:03:23.471 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-05-08T23:03:23.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 msgr2=0x7fc8cc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 0x7fc8cc0a36f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009c60 tx=0x7fc8d0009260 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 msgr2=0x7fc8e4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e9a0 tx=0x7fc8d800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 shutdown_connections 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc8cc0a1230 0x7fc8cc0a36f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e4106d90 0x7fc8e4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8e4101b30 0x7fc8e4101440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 --2- 172.21.15.46:0/3463490523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e4104460 0x7fc8e40ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 >> 172.21.15.46:0/3463490523 conn(0x7fc8e40fb4a0 msgr2=0x7fc8e4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 shutdown_connections 2024-05-08T23:03:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:23.473+0000 7fc8ec200700 1 -- 172.21.15.46:0/3463490523 wait complete. 2024-05-08T23:03:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:23 smithi053 ceph-mon[77587]: pgmap v289: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-08T23:03:23.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T23:03:24.232 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:24 smithi046 ceph-mon[36898]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:24.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:24 smithi086 ceph-mon[78610]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:24.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:24 smithi053 ceph-mon[77587]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:25.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 -- 172.21.15.46:0/3959287872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36941043c0 msgr2=0x7f36941067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:25.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 --2- 172.21.15.46:0/3959287872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36941043c0 0x7f36941067b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3680009990 tx=0x7f3680009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 -- 172.21.15.46:0/3959287872 shutdown_connections 2024-05-08T23:03:25.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 --2- 172.21.15.46:0/3959287872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3694106cf0 0x7f3694109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 --2- 172.21.15.46:0/3959287872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f3694103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 --2- 172.21.15.46:0/3959287872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36941043c0 0x7f36941067b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 -- 172.21.15.46:0/3959287872 >> 172.21.15.46:0/3959287872 conn(0x7f36940fb4a0 msgr2=0x7f36940fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 -- 172.21.15.46:0/3959287872 shutdown_connections 2024-05-08T23:03:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.452+0000 7f369ac00700 1 -- 172.21.15.46:0/3959287872 wait complete. 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.453+0000 7f369ac00700 1 Processor -- start 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.453+0000 7f369ac00700 1 -- start start 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.453+0000 7f369ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f36941013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.453+0000 7f369ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.453+0000 7f369ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3694106cf0 0x7f36940fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f369ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694100720 con 0x7f36941043c0 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f369ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694100890 con 0x7f3694106cf0 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f369ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36941a0fd0 con 0x7f3694101a90 2024-05-08T23:03:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59934/0 (socket says 172.21.15.46:59934) 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693400700 1 -- 172.21.15.46:0/2449430367 learned_addr learned my addr 172.21.15.46:0/2449430367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f36941013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3698c00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3694106cf0 0x7f36940fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 -- 172.21.15.46:0/2449430367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3694106cf0 msgr2=0x7f36940fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3694106cf0 0x7f36940fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 -- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 msgr2=0x7f36940ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693e00700 1 -- 172.21.15.46:0/2449430367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684009520 con 0x7f3694101a90 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.454+0000 7f3693400700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f3693e00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f36941013a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f368400e9f0 tx=0x7f368400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368400cb70 con 0x7f3694101a90 2024-05-08T23:03:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f368400ee90 con 0x7f3694101a90 2024-05-08T23:03:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800095f0 con 0x7f3694101a90 2024-05-08T23:03:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684003cf0 con 0x7f3694101a90 2024-05-08T23:03:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.455+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36941a15d0 con 0x7f3694101a90 2024-05-08T23:03:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.456+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36940fd020 con 0x7f3694101a90 2024-05-08T23:03:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.458+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f368400ccd0 con 0x7f3694101a90 2024-05-08T23:03:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.459+0000 7f3690c00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 0x7f36780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.460+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3684014070 con 0x7f3694101a90 2024-05-08T23:03:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.460+0000 7f3693400700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 0x7f36780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.461+0000 7f3693400700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 0x7f36780a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f368000b2e0 tx=0x7f368000b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.462+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f36840964a0 con 0x7f3694101a90 2024-05-08T23:03:25.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.685+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 --> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f36940613e0 con 0x7f36780a11b0 2024-05-08T23:03:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.686+0000 7f3690c00700 1 -- 172.21.15.46:0/2449430367 <== mgr.14150 v2:172.21.15.46:6800/2731582335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f36940613e0 con 0x7f36780a11b0 2024-05-08T23:03:25.687 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T23:03:25.687 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:03:25.687 INFO:teuthology.orchestra.run.smithi046.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:03:25.687 INFO:teuthology.orchestra.run.smithi046.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 msgr2=0x7f36780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 0x7f36780a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f368000b2e0 tx=0x7f368000b750 comp rx=0 tx=0).stop 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 msgr2=0x7f36941013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f36941013a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f368400e9f0 tx=0x7f368400ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 shutdown_connections 2024-05-08T23:03:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3694106cf0 0x7f36940fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3694101a90 0x7f36941013a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f36780a11b0 0x7f36780a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 --2- 172.21.15.46:0/2449430367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36941043c0 0x7f36940ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 >> 172.21.15.46:0/2449430367 conn(0x7f36940fb4a0 msgr2=0x7f3694103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 shutdown_connections 2024-05-08T23:03:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:25.689+0000 7f369ac00700 1 -- 172.21.15.46:0/2449430367 wait complete. 2024-05-08T23:03:25.902 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:25 smithi046 ceph-mon[36898]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-05-08T23:03:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:25 smithi086 ceph-mon[78610]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-05-08T23:03:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:25 smithi053 ceph-mon[77587]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-05-08T23:03:26.120 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:03:26.130 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:03:26.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T23:03:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:26.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:26 smithi086 ceph-mon[78610]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-08T23:03:26.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:26 smithi086 ceph-mon[78610]: Cluster is now healthy 2024-05-08T23:03:26.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:26 smithi086 ceph-mon[78610]: from='client.25306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:27.006 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:26 smithi046 ceph-mon[36898]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-08T23:03:27.006 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:26 smithi046 ceph-mon[36898]: Cluster is now healthy 2024-05-08T23:03:27.006 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:26 smithi046 ceph-mon[36898]: from='client.25306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:27.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:26 smithi053 ceph-mon[77587]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-08T23:03:27.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:26 smithi053 ceph-mon[77587]: Cluster is now healthy 2024-05-08T23:03:27.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:26 smithi053 ceph-mon[77587]: from='client.25306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 -- 172.21.15.46:0/449109273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0102ba0 msgr2=0x7f64c0109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 --2- 172.21.15.46:0/449109273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0102ba0 0x7f64c0109450 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f64b800b0d0 tx=0x7f64b800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 -- 172.21.15.46:0/449109273 shutdown_connections 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 --2- 172.21.15.46:0/449109273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0102ba0 0x7f64c0109450 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 --2- 172.21.15.46:0/449109273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0101000 0x7f64c0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 --2- 172.21.15.46:0/449109273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 0x7f64c0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 -- 172.21.15.46:0/449109273 >> 172.21.15.46:0/449109273 conn(0x7f64c00fc5a0 msgr2=0x7f64c00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.720+0000 7f64c7200700 1 -- 172.21.15.46:0/449109273 shutdown_connections 2024-05-08T23:03:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.721+0000 7f64c7200700 1 -- 172.21.15.46:0/449109273 wait complete. 2024-05-08T23:03:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 Processor -- start 2024-05-08T23:03:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 -- start start 2024-05-08T23:03:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 0x7f64c0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 0x7f64c01a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01a0d10 con 0x7f64c0101000 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01a0e50 con 0x7f64c0102ba0 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01a0fe0 con 0x7f64c0102200 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59958/0 (socket says 172.21.15.46:59958) 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64bfe00700 1 -- 172.21.15.46:0/2876784698 learned_addr learned my addr 172.21.15.46:0/2876784698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64c5200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 0x7f64c01a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.722+0000 7f64bf400700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 0x7f64c0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 -- 172.21.15.46:0/2876784698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 msgr2=0x7f64c0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 0x7f64c0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 -- 172.21.15.46:0/2876784698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 msgr2=0x7f64c01a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 0x7f64c01a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 -- 172.21.15.46:0/2876784698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009d00 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64c5200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 0x7f64c01a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bfe00700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64b000e9a0 tx=0x7f64b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0009790 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64b000c810 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c01a12c0 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.723+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0003c70 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.724+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c01a1810 con 0x7f64c0101000 2024-05-08T23:03:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.724+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c004ec60 con 0x7f64c0101000 2024-05-08T23:03:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.728+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f64b000cc80 con 0x7f64c0101000 2024-05-08T23:03:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.730+0000 7f64bcc00700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 0x7f64a40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.730+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f64b0014070 con 0x7f64c0101000 2024-05-08T23:03:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.730+0000 7f64bf400700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 0x7f64a40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:27.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.731+0000 7f64bf400700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 0x7f64a40a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009c60 tx=0x7f64ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.732+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64b0096270 con 0x7f64c0101000 2024-05-08T23:03:27.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:27 smithi046 ceph-mon[36898]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-08T23:03:27.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:27 smithi086 ceph-mon[78610]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-08T23:03:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.998+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64c010ca50 con 0x7f64c0101000 2024-05-08T23:03:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:27.998+0000 7f64bcc00700 1 -- 172.21.15.46:0/2876784698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2919 (secure 0 0 0) 0x7f64b0099890 con 0x7f64c0101000 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:e26 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 26 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-08T23:01:51.460837+0000 2024-05-08T23:03:27.999 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-08T23:02:19.629003+0000 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 5 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2,3,4 2024-05-08T23:03:28.000 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24587,1=24583,2=24571,3=24547,4=24541} 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3,4] 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{0:24539} state up:standby-replay seq 1 addr [v2:172.21.15.53:6836/170717769,v1:172.21.15.53:6837/170717769] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{1:24583} state up:active seq 10 addr [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.001 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{1:24512} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/1845130080,v1:172.21.15.53:6835/1845130080] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{2:24571} state up:active seq 11 addr [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{2:24500} state up:standby-replay seq 1 addr [v2:172.21.15.46:6836/1812878749,v1:172.21.15.46:6837/1812878749] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{3:24547} state up:active seq 12 addr [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{3:24490} state up:standby-replay seq 1 addr [v2:172.21.15.86:6832/272583724,v1:172.21.15.86:6833/272583724] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{4:24541} state up:active seq 13 addr [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{4:24476} state up:standby-replay seq 1 addr [v2:172.21.15.53:6832/3863967478,v1:172.21.15.53:6833/3863967478] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.46:6834/3787979221,v1:172.21.15.46:6835/3787979221] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.002 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{-1:14586} state up:standby seq 1 addr [v2:172.21.15.46:6840/1402919380,v1:172.21.15.46:6841/1402919380] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:03:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 msgr2=0x7f64a40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 0x7f64a40a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009c60 tx=0x7f64ac009260 comp rx=0 tx=0).stop 2024-05-08T23:03:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 msgr2=0x7f64c01991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64b000e9a0 tx=0x7f64b000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 shutdown_connections 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64c0102ba0 0x7f64c01a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f64a40a1230 0x7f64a40a36f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c0101000 0x7f64c01991d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 --2- 172.21.15.46:0/2876784698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c0102200 0x7f64c0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 >> 172.21.15.46:0/2876784698 conn(0x7f64c00fc5a0 msgr2=0x7f64c00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 shutdown_connections 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:28.001+0000 7f64c7200700 1 -- 172.21.15.46:0/2876784698 wait complete. 2024-05-08T23:03:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:28.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:27 smithi053 ceph-mon[77587]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-08T23:03:28.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T23:03:28.805 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:28.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:28 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2876784698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:03:28.981 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:28 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2876784698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:03:29.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:28 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2876784698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:03:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:29 smithi086 ceph-mon[78610]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:30.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:29 smithi053 ceph-mon[77587]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:29 smithi046 ceph-mon[36898]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.076+0000 7f9af5c00700 1 -- 172.21.15.46:0/1182072838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0101000 msgr2=0x7f9af0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.076+0000 7f9af5c00700 1 --2- 172.21.15.46:0/1182072838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0101000 0x7f9af0101420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00099c0 tx=0x7f9ae0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 -- 172.21.15.46:0/1182072838 shutdown_connections 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 --2- 172.21.15.46:0/1182072838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0102ba0 0x7f9af0109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 --2- 172.21.15.46:0/1182072838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0101000 0x7f9af0101420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 --2- 172.21.15.46:0/1182072838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102200 0x7f9af0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 -- 172.21.15.46:0/1182072838 >> 172.21.15.46:0/1182072838 conn(0x7f9af00fc5a0 msgr2=0x7f9af00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 -- 172.21.15.46:0/1182072838 shutdown_connections 2024-05-08T23:03:30.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.077+0000 7f9af5c00700 1 -- 172.21.15.46:0/1182072838 wait complete. 2024-05-08T23:03:30.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 Processor -- start 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 -- start start 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 0x7f9af0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102ba0 0x7f9af019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af019ea80 con 0x7f9af0102200 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af019ebc0 con 0x7f9af0101000 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.078+0000 7f9af5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af019ed50 con 0x7f9af0102ba0 2024-05-08T23:03:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aee000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aee000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59988/0 (socket says 172.21.15.46:59988) 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aee000700 1 -- 172.21.15.46:0/78410658 learned_addr learned my addr 172.21.15.46:0/78410658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 0x7f9af0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aef400700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102ba0 0x7f9af019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 -- 172.21.15.46:0/78410658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102ba0 msgr2=0x7f9af019e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102ba0 0x7f9af019e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 -- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 msgr2=0x7f9af0197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 -- 172.21.15.46:0/78410658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009c40 con 0x7f9af0101000 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aee000700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.079+0000 7f9aeea00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 0x7f9af0196f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00058e0 tx=0x7f9ae000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.080+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001d070 con 0x7f9af0101000 2024-05-08T23:03:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.080+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae00095f0 con 0x7f9af0101000 2024-05-08T23:03:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.080+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ae00059a0 con 0x7f9af0101000 2024-05-08T23:03:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.080+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004970 con 0x7f9af0101000 2024-05-08T23:03:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.080+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af00734b0 con 0x7f9af0101000 2024-05-08T23:03:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.081+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af01084b0 con 0x7f9af0101000 2024-05-08T23:03:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.083+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9ae0005050 con 0x7f9af0101000 2024-05-08T23:03:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.084+0000 7f9ae7400700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 0x7f9ad40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.084+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9ae00cd980 con 0x7f9af0101000 2024-05-08T23:03:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.084+0000 7f9aee000700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 0x7f9ad40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.085+0000 7f9aee000700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 0x7f9ad40a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9adc011550 tx=0x7f9adc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.087+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ae0096c40 con 0x7f9af0101000 2024-05-08T23:03:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.303+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f9af010c990 con 0x7f9af0101000 2024-05-08T23:03:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.304+0000 7f9ae7400700 1 -- 172.21.15.46:0/78410658 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) v1 ==== 56+0+5696 (secure 0 0 0) 0x7f9ae00b8e60 con 0x7f9af0101000 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 86 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:fsid ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-08T22:55:20.134668+0000 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-08T23:03:19.015441+0000 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 27 2024-05-08T23:03:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release reef 2024-05-08T23:03:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/2436508778,v1:172.21.15.46:6803/2436508778] [v2:172.21.15.46:6804/2436508778,v1:172.21.15.46:6805/2436508778] exists,up f77bd5d5-1794-4093-becb-18c682bf3da2 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/3799790044,v1:172.21.15.53:6801/3799790044] [v2:172.21.15.53:6802/3799790044,v1:172.21.15.53:6803/3799790044] exists,up c389d713-72f0-4a78-854e-85a7a352ee10 2024-05-08T23:03:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/1780351335,v1:172.21.15.86:6801/1780351335] [v2:172.21.15.86:6802/1780351335,v1:172.21.15.86:6803/1780351335] exists,up 5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/1550587514,v1:172.21.15.46:6811/1550587514] [v2:172.21.15.46:6812/1550587514,v1:172.21.15.46:6813/1550587514] exists,up d63af76a-815c-4170-ab60-6ed7dbe596ff 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 31 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2784307263,v1:172.21.15.53:6809/2784307263] [v2:172.21.15.53:6810/2784307263,v1:172.21.15.53:6811/2784307263] exists,up 3ca7a742-9c35-448a-9287-63b6fe77a632 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 35 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/2217036731,v1:172.21.15.86:6809/2217036731] [v2:172.21.15.86:6810/2217036731,v1:172.21.15.86:6811/2217036731] exists,up c0ef3604-5955-42ac-b4f7-5d211ef7d553 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 42 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/1222901662,v1:172.21.15.46:6819/1222901662] [v2:172.21.15.46:6820/1222901662,v1:172.21.15.46:6821/1222901662] exists,up 9977b050-bdba-4adc-8d0a-8ee2935643bf 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 47 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/521136929,v1:172.21.15.53:6817/521136929] [v2:172.21.15.53:6818/521136929,v1:172.21.15.53:6819/521136929] exists,up 1305daf7-451a-42b5-bc27-04165d5de6eb 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 54 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/4052399727,v1:172.21.15.86:6817/4052399727] [v2:172.21.15.86:6818/4052399727,v1:172.21.15.86:6819/4052399727] exists,up a79a15f0-16af-47ba-af13-ceb2e1df5f43 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/2347234971,v1:172.21.15.46:6827/2347234971] [v2:172.21.15.46:6828/2347234971,v1:172.21.15.46:6829/2347234971] exists,up 885ac456-a429-4688-b3f7-c66c1d785c1c 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 66 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/75118943,v1:172.21.15.53:6825/75118943] [v2:172.21.15.53:6826/75118943,v1:172.21.15.53:6827/75118943] exists,up 66e06a38-ad3e-4b9d-af1f-8f62447ec12a 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 73 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/3801696519,v1:172.21.15.86:6825/3801696519] [v2:172.21.15.86:6826/3801696519,v1:172.21.15.86:6827/3801696519] exists,up 2b97781b-2ef0-485c-b42b-47099ce3ff54 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.0 [7,11] 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.c [2,4] 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.e [2,4] 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.f [8,11] 2024-05-08T23:03:30.308 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.1f [8,1] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.e [5,9] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1b [4,8] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.27 [4,6] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.37 [1,9] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3a [4,7] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3b [3,9] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.12 [9,10] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.16 [9,8] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.19 [9,8] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.1d [5,8] 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2015473581 expires 2024-05-09T22:56:16.844189+0000 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/664114694 expires 2024-05-09T22:55:52.116773+0000 2024-05-08T23:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/664114694 expires 2024-05-09T22:55:52.116773+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/387189775 expires 2024-05-09T22:55:52.116773+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/652393065 expires 2024-05-09T22:56:16.844189+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/2056989721 expires 2024-05-09T22:56:16.844189+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1980881468 expires 2024-05-09T22:55:52.116773+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3841409271 expires 2024-05-09T22:55:52.116773+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/4047035481 expires 2024-05-09T22:56:16.844189+0000 2024-05-08T23:03:30.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/2056989721 expires 2024-05-09T22:56:16.844189+0000 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 msgr2=0x7f9ad40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 0x7f9ad40a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9adc011550 tx=0x7f9adc009260 comp rx=0 tx=0).stop 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 msgr2=0x7f9af0196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 0x7f9af0196f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00058e0 tx=0x7f9ae000f690 comp rx=0 tx=0).stop 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 shutdown_connections 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af0102ba0 0x7f9af019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af0101000 0x7f9af0196f40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9ad40a11b0 0x7f9ad40a3670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 --2- 172.21.15.46:0/78410658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af0102200 0x7f9af0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 >> 172.21.15.46:0/78410658 conn(0x7f9af00fc5a0 msgr2=0x7f9af01055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 shutdown_connections 2024-05-08T23:03:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.307+0000 7f9af5c00700 1 -- 172.21.15.46:0/78410658 wait complete. 2024-05-08T23:03:30.731 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T23:03:30.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 -- 172.21.15.46:0/4277740605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70108b60 msgr2=0x7f7d7010afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 --2- 172.21.15.46:0/4277740605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70108b60 0x7f7d7010afd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b0a0 tx=0x7f7d6800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 -- 172.21.15.46:0/4277740605 shutdown_connections 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 --2- 172.21.15.46:0/4277740605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70108b60 0x7f7d7010afd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 --2- 172.21.15.46:0/4277740605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d700733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 --2- 172.21.15.46:0/4277740605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d700739a0 0x7f7d70073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.949+0000 7f7d75800700 1 -- 172.21.15.46:0/4277740605 >> 172.21.15.46:0/4277740605 conn(0x7f7d700fc220 msgr2=0x7f7d700fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.950+0000 7f7d75800700 1 -- 172.21.15.46:0/4277740605 shutdown_connections 2024-05-08T23:03:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.950+0000 7f7d75800700 1 -- 172.21.15.46:0/4277740605 wait complete. 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 Processor -- start 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 -- start start 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d70108b60 0x7f7d701a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701a0a40 con 0x7f7d700739a0 2024-05-08T23:03:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701a0b80 con 0x7f7d70074f80 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d75800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701a0d10 con 0x7f7d70108b60 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d6fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d6f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d6fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60006/0 (socket says 172.21.15.46:60006) 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d6f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34586/0 (socket says 172.21.15.46:34586) 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d6fe00700 1 -- 172.21.15.46:0/1709703691 learned_addr learned my addr 172.21.15.46:0/1709703691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.951+0000 7f7d74e00700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d70108b60 0x7f7d701a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 -- 172.21.15.46:0/1709703691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d70108b60 msgr2=0x7f7d701a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d70108b60 0x7f7d701a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 -- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 msgr2=0x7f7d70198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 -- 172.21.15.46:0/1709703691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68009d00 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6fe00700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6f400700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e9f0 tx=0x7f7d5c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004220 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d5c004cf0 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d701a0ff0 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c005020 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.952+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d701a1420 con 0x7f7d70074f80 2024-05-08T23:03:30.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.953+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d70107970 con 0x7f7d70074f80 2024-05-08T23:03:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.956+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7d5c004750 con 0x7f7d70074f80 2024-05-08T23:03:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.957+0000 7f7d6cc00700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 0x7f7d540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.957+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7d5c014070 con 0x7f7d70074f80 2024-05-08T23:03:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.957+0000 7f7d6fe00700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 0x7f7d540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.958+0000 7f7d6fe00700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 0x7f7d540a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d60011550 tx=0x7f7d60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:30.959+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d5c096090 con 0x7f7d70074f80 2024-05-08T23:03:30.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:30 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/78410658' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:03:31.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:30 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/78410658' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:03:31.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:30 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/78410658' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:03:31.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.180+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d7004ecd0 con 0x7f7d70074f80 2024-05-08T23:03:31.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.182+0000 7f7d6cc00700 1 -- 172.21.15.46:0/1709703691 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24623 (secure 0 0 0) 0x7f7d5c004e60 con 0x7f7d70074f80 2024-05-08T23:03:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:31.183 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:19.015441+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 msgr2=0x7f7d540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 0x7f7d540a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d60011550 tx=0x7f7d60009260 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 msgr2=0x7f7d70199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e9f0 tx=0x7f7d5c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 shutdown_connections 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d70108b60 0x7f7d701a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f7d540a11b0 0x7f7d540a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d700739a0 0x7f7d70198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 --2- 172.21.15.46:0/1709703691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70074f80 0x7f7d70199450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.184+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 >> 172.21.15.46:0/1709703691 conn(0x7f7d700fc220 msgr2=0x7f7d70104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.185+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 shutdown_connections 2024-05-08T23:03:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.185+0000 7f7d75800700 1 -- 172.21.15.46:0/1709703691 wait complete. 2024-05-08T23:03:31.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:31.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 -- 172.21.15.46:0/1347312162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650102990 msgr2=0x7fc6501092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 --2- 172.21.15.46:0/1347312162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650102990 0x7fc6501092b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc64800b0a0 tx=0x7fc64800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 -- 172.21.15.46:0/1347312162 shutdown_connections 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 --2- 172.21.15.46:0/1347312162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650102990 0x7fc6501092b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 --2- 172.21.15.46:0/1347312162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650100e80 0x7fc6501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 --2- 172.21.15.46:0/1347312162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650101fd0 0x7fc650102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 -- 172.21.15.46:0/1347312162 >> 172.21.15.46:0/1347312162 conn(0x7fc6500fc3b0 msgr2=0x7fc6500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 -- 172.21.15.46:0/1347312162 shutdown_connections 2024-05-08T23:03:31.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.405+0000 7fc656a00700 1 -- 172.21.15.46:0/1347312162 wait complete. 2024-05-08T23:03:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.406+0000 7fc656a00700 1 Processor -- start 2024-05-08T23:03:31.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.406+0000 7fc656a00700 1 -- start start 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650100e80 0x7fc650074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650102990 0x7fc6500738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc650074070 con 0x7fc650101fd0 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500741e0 con 0x7fc650100e80 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc656a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6501a0f80 con 0x7fc650102990 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60020/0 (socket says 172.21.15.46:60020) 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 -- 172.21.15.46:0/3670572650 learned_addr learned my addr 172.21.15.46:0/3670572650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64fe00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650100e80 0x7fc650074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc654a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650102990 0x7fc6500738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 -- 172.21.15.46:0/3670572650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650102990 msgr2=0x7fc6500738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650102990 0x7fc6500738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 -- 172.21.15.46:0/3670572650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650100e80 msgr2=0x7fc650074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650100e80 0x7fc650074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.407+0000 7fc64f400700 1 -- 172.21.15.46:0/3670572650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648009d00 con 0x7fc650101fd0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc64f400700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9a0 tx=0x7fc63c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c009790 con 0x7fc650101fd0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63c00ce70 con 0x7fc650101fd0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c018420 con 0x7fc650101fd0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6501a1210 con 0x7fc650101fd0 2024-05-08T23:03:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.408+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6501a1720 con 0x7fc650101fd0 2024-05-08T23:03:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.409+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650108220 con 0x7fc650101fd0 2024-05-08T23:03:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.411+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc63c0106f0 con 0x7fc650101fd0 2024-05-08T23:03:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.412+0000 7fc64cc00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 0x7fc6340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.412+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc63c014070 con 0x7fc650101fd0 2024-05-08T23:03:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.413+0000 7fc64fe00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 0x7fc6340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.413+0000 7fc64fe00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 0x7fc6340a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc640009c60 tx=0x7fc640009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.417+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc63c052d60 con 0x7fc650101fd0 2024-05-08T23:03:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.634+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc65010c6f0 con 0x7fc650101fd0 2024-05-08T23:03:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.634+0000 7fc64cc00700 1 -- 172.21.15.46:0/3670572650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc63c099c20 con 0x7fc650101fd0 2024-05-08T23:03:31.634 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 msgr2=0x7fc6340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 0x7fc6340a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc640009c60 tx=0x7fc640009260 comp rx=0 tx=0).stop 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 msgr2=0x7fc650073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9a0 tx=0x7fc63c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 shutdown_connections 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc650102990 0x7fc6500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc650100e80 0x7fc650074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fc6340a1230 0x7fc6340a36f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 --2- 172.21.15.46:0/3670572650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc650101fd0 0x7fc650073390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 >> 172.21.15.46:0/3670572650 conn(0x7fc6500fc3b0 msgr2=0x7fc650105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 shutdown_connections 2024-05-08T23:03:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.637+0000 7fc656a00700 1 -- 172.21.15.46:0/3670572650 wait complete. 2024-05-08T23:03:31.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 -- 172.21.15.46:0/3058554640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4102870 msgr2=0x7f92b4109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 --2- 172.21.15.46:0/3058554640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4102870 0x7f92b4109120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b0d0 tx=0x7f92ac00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 -- 172.21.15.46:0/3058554640 shutdown_connections 2024-05-08T23:03:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 --2- 172.21.15.46:0/3058554640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4102870 0x7f92b4109120 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 --2- 172.21.15.46:0/3058554640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4100ce0 0x7f92b4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 --2- 172.21.15.46:0/3058554640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 0x7f92b4102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.859+0000 7f92b3200700 1 -- 172.21.15.46:0/3058554640 >> 172.21.15.46:0/3058554640 conn(0x7f92b40fc210 msgr2=0x7f92b40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.860+0000 7f92b3200700 1 -- 172.21.15.46:0/3058554640 shutdown_connections 2024-05-08T23:03:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.860+0000 7f92b3200700 1 -- 172.21.15.46:0/3058554640 wait complete. 2024-05-08T23:03:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 Processor -- start 2024-05-08T23:03:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 -- start start 2024-05-08T23:03:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 0x7f92b4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4102870 0x7f92b41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41a09d0 con 0x7f92b4100ce0 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41a0b10 con 0x7f92b4102870 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41a0ca0 con 0x7f92b4101ed0 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53952/0 (socket says 172.21.15.46:53952) 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b1e00700 1 -- 172.21.15.46:0/4121185638 learned_addr learned my addr 172.21.15.46:0/4121185638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.861+0000 7f92b1400700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 0x7f92b4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b2800700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4102870 0x7f92b41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 -- 172.21.15.46:0/4121185638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 msgr2=0x7f92b4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 0x7f92b4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 -- 172.21.15.46:0/4121185638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4102870 msgr2=0x7f92b41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4102870 0x7f92b41a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 -- 172.21.15.46:0/4121185638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009d00 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b1e00700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e970 tx=0x7f92a400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4009790 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a400eec0 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4003cf0 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41a0f80 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.862+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b41a13a0 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.863+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404ecd0 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.865+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f92a400ccd0 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.867+0000 7f92aaa00700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 0x7f92980a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.867+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f92a4014070 con 0x7f92b4100ce0 2024-05-08T23:03:31.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.867+0000 7f92b1400700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 0x7f92980a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.868+0000 7f92b1400700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 0x7f92980a37a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009c60 tx=0x7f92a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:31.871+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f92b404ecd0 con 0x7f92b4100ce0 2024-05-08T23:03:31.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:31 smithi086 ceph-mon[78610]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T23:03:31.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:31 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1709703691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:31.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:31 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3670572650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:31 smithi053 ceph-mon[77587]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T23:03:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:31 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1709703691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:31 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3670572650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:32.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:31 smithi046 ceph-mon[36898]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T23:03:32.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:31 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1709703691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:32.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:31 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3670572650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.094+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92b41a16a0 con 0x7f92b4100ce0 2024-05-08T23:03:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.095+0000 7f92aaa00700 1 -- 172.21.15.46:0/4121185638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92a4096270 con 0x7f92b4100ce0 2024-05-08T23:03:32.095 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 msgr2=0x7f92980a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 0x7f92980a37a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009c60 tx=0x7f92a0009260 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 msgr2=0x7f92b4198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e970 tx=0x7f92a400ec80 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 shutdown_connections 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92b4102870 0x7f92b41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f92980a12e0 0x7f92980a37a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b4100ce0 0x7f92b4198ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 --2- 172.21.15.46:0/4121185638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b4101ed0 0x7f92b4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 >> 172.21.15.46:0/4121185638 conn(0x7f92b40fc210 msgr2=0x7f92b40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.097+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 shutdown_connections 2024-05-08T23:03:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.098+0000 7f92b3200700 1 -- 172.21.15.46:0/4121185638 wait complete. 2024-05-08T23:03:32.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 -- 172.21.15.46:0/599443459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac100e80 msgr2=0x7f5bac1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 --2- 172.21.15.46:0/599443459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac100e80 0x7f5bac1012a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009990 tx=0x7f5ba0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 -- 172.21.15.46:0/599443459 shutdown_connections 2024-05-08T23:03:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 --2- 172.21.15.46:0/599443459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac102990 0x7f5bac1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 --2- 172.21.15.46:0/599443459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac100e80 0x7f5bac1012a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 --2- 172.21.15.46:0/599443459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac101fd0 0x7f5bac102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 -- 172.21.15.46:0/599443459 >> 172.21.15.46:0/599443459 conn(0x7f5bac0fc3b0 msgr2=0x7f5bac0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 -- 172.21.15.46:0/599443459 shutdown_connections 2024-05-08T23:03:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.318+0000 7f5bb3800700 1 -- 172.21.15.46:0/599443459 wait complete. 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 Processor -- start 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 -- start start 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 0x7f5bac199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 0x7f5bac1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1a0b70 con 0x7f5bac101fd0 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1a0cb0 con 0x7f5bac100e80 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.319+0000 7f5bb3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1a0e40 con 0x7f5bac102990 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5babe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 0x7f5bac199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 0x7f5bac1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50222/0 (socket says 172.21.15.46:50222) 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb0e00700 1 -- 172.21.15.46:0/472005887 learned_addr learned my addr 172.21.15.46:0/472005887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 -- 172.21.15.46:0/472005887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 msgr2=0x7f5bac199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 -- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 msgr2=0x7f5bac199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 0x7f5bac199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 -- 172.21.15.46:0/472005887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00095f0 con 0x7f5bac102990 2024-05-08T23:03:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb0e00700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5babe00700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 0x7f5bac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.320+0000 7f5bb1800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 0x7f5bac1a05a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400efc0 tx=0x7f5ba400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.321+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4018070 con 0x7f5bac102990 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.321+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ba400f040 con 0x7f5bac102990 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.321+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac1a10c0 con 0x7f5bac102990 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.321+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400cd40 con 0x7f5bac102990 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.321+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac1a1620 con 0x7f5bac102990 2024-05-08T23:03:32.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.322+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac067d10 con 0x7f5bac102990 2024-05-08T23:03:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.324+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5ba4003a00 con 0x7f5bac102990 2024-05-08T23:03:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.325+0000 7f5ba9600700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 0x7f5b940a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.325+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f5ba40cd340 con 0x7f5bac102990 2024-05-08T23:03:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.325+0000 7f5bb0e00700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 0x7f5b940a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.326+0000 7f5bb0e00700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 0x7f5b940a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b2e0 tx=0x7f5ba000b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:32.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.328+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5ba4096600 con 0x7f5bac102990 2024-05-08T23:03:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.543+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bac10c6f0 con 0x7f5bac102990 2024-05-08T23:03:32.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.544+0000 7f5ba9600700 1 -- 172.21.15.46:0/472005887 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ba4099c20 con 0x7f5bac102990 2024-05-08T23:03:32.545 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 msgr2=0x7f5b940a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 0x7f5b940a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b2e0 tx=0x7f5ba000b6f0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 msgr2=0x7f5bac1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 0x7f5bac1a05a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400efc0 tx=0x7f5ba400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 shutdown_connections 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bac102990 0x7f5bac1a05a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bac100e80 0x7f5bac199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f5b940a1230 0x7f5b940a36f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 --2- 172.21.15.46:0/472005887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bac101fd0 0x7f5bac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 >> 172.21.15.46:0/472005887 conn(0x7f5bac0fc3b0 msgr2=0x7f5bac0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 shutdown_connections 2024-05-08T23:03:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.547+0000 7f5bb3800700 1 -- 172.21.15.46:0/472005887 wait complete. 2024-05-08T23:03:32.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 -- 172.21.15.46:0/3756286912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4101eb0 msgr2=0x7fe8a4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 --2- 172.21.15.46:0/3756286912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4101eb0 0x7fe8a4102310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe8900099c0 tx=0x7fe890009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 -- 172.21.15.46:0/3756286912 shutdown_connections 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 --2- 172.21.15.46:0/3756286912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4102850 0x7fe8a4109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 --2- 172.21.15.46:0/3756286912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4100cc0 0x7fe8a41010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 --2- 172.21.15.46:0/3756286912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4101eb0 0x7fe8a4102310 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 -- 172.21.15.46:0/3756286912 >> 172.21.15.46:0/3756286912 conn(0x7fe8a40fc250 msgr2=0x7fe8a40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 -- 172.21.15.46:0/3756286912 shutdown_connections 2024-05-08T23:03:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.768+0000 7fe8a3200700 1 -- 172.21.15.46:0/3756286912 wait complete. 2024-05-08T23:03:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.769+0000 7fe8a3200700 1 Processor -- start 2024-05-08T23:03:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 -- start start 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4101eb0 0x7fe8a41970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4102850 0x7fe8a419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a419e6b0 con 0x7fe8a4100cc0 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a419e7f0 con 0x7fe8a4102850 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a419e980 con 0x7fe8a4101eb0 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53978/0 (socket says 172.21.15.46:53978) 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a1e00700 1 -- 172.21.15.46:0/283318498 learned_addr learned my addr 172.21.15.46:0/283318498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a2800700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4102850 0x7fe8a419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.770+0000 7fe8a1400700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4101eb0 0x7fe8a41970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 -- 172.21.15.46:0/283318498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4101eb0 msgr2=0x7fe8a41970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4101eb0 0x7fe8a41970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 -- 172.21.15.46:0/283318498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4102850 msgr2=0x7fe8a419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4102850 0x7fe8a419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 -- 172.21.15.46:0/283318498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900095f0 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a1e00700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e970 tx=0x7fe89400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894009790 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89400c7e0 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a419ec60 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894003c70 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.771+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a419f170 con 0x7fe8a4100cc0 2024-05-08T23:03:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.772+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4108010 con 0x7fe8a4100cc0 2024-05-08T23:03:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.774+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe89400cc50 con 0x7fe8a4100cc0 2024-05-08T23:03:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.775+0000 7fe89aa00700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 0x7fe8880a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.776+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe894014070 con 0x7fe8a4100cc0 2024-05-08T23:03:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.776+0000 7fe8a1400700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 0x7fe8880a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:32.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.776+0000 7fe8a1400700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 0x7fe8880a3720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe890009990 tx=0x7fe890000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:32.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:32.779+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe894096290 con 0x7fe8a4100cc0 2024-05-08T23:03:32.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4121185638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:32.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/472005887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.003+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8a410c530 con 0x7fe8a4100cc0 2024-05-08T23:03:33.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.004+0000 7fe89aa00700 1 -- 172.21.15.46:0/283318498 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe894003dd0 con 0x7fe8a4100cc0 2024-05-08T23:03:33.004 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 msgr2=0x7fe8880a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 0x7fe8880a3720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe890009990 tx=0x7fe890000bc0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 msgr2=0x7fe8a4196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e970 tx=0x7fe89400ec80 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 shutdown_connections 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8a4102850 0x7fe8a419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fe8880a1260 0x7fe8880a3720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8a4100cc0 0x7fe8a4196b80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 --2- 172.21.15.46:0/283318498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a4101eb0 0x7fe8a41970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.006+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 >> 172.21.15.46:0/283318498 conn(0x7fe8a40fc250 msgr2=0x7fe8a4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.007+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 shutdown_connections 2024-05-08T23:03:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.007+0000 7fe8a3200700 1 -- 172.21.15.46:0/283318498 wait complete. 2024-05-08T23:03:33.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4121185638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/472005887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.023 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4121185638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.023 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/472005887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.223+0000 7f4c5be00700 1 -- 172.21.15.46:0/3400446912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54100e80 msgr2=0x7f4c541012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.223+0000 7f4c5be00700 1 --2- 172.21.15.46:0/3400446912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54100e80 0x7f4c541012a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009990 tx=0x7f4c48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 -- 172.21.15.46:0/3400446912 shutdown_connections 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 --2- 172.21.15.46:0/3400446912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c541092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 --2- 172.21.15.46:0/3400446912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54100e80 0x7f4c541012a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 --2- 172.21.15.46:0/3400446912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54101fd0 0x7f4c54102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 -- 172.21.15.46:0/3400446912 >> 172.21.15.46:0/3400446912 conn(0x7f4c540fc410 msgr2=0x7f4c540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:33.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 -- 172.21.15.46:0/3400446912 shutdown_connections 2024-05-08T23:03:33.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.224+0000 7f4c5be00700 1 -- 172.21.15.46:0/3400446912 wait complete. 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 Processor -- start 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 -- start start 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54074070 con 0x7f4c54101fd0 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540741e0 con 0x7f4c54100e80 2024-05-08T23:03:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.225+0000 7f4c5be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541a0ea0 con 0x7f4c54102990 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c59400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53994/0 (socket says 172.21.15.46:53994) 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c59400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50254/0 (socket says 172.21.15.46:50254) 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 -- 172.21.15.46:0/1034232281 learned_addr learned my addr 172.21.15.46:0/1034232281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c59e00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 -- 172.21.15.46:0/1034232281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 msgr2=0x7f4c540738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c540738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 -- 172.21.15.46:0/1034232281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 msgr2=0x7f4c54074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 -- 172.21.15.46:0/1034232281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c40009520 con 0x7f4c54101fd0 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c59400700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c59e00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c540738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.226+0000 7f4c58a00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000eb30 tx=0x7f4c4000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.227+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40009880 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.227+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c009c40 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.227+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c400044e0 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.227+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40003df0 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.227+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c480095f0 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.228+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c541082f0 con 0x7f4c54101fd0 2024-05-08T23:03:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.230+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4c40004000 con 0x7f4c54101fd0 2024-05-08T23:03:33.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.231+0000 7f4c52000700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 0x7f4c3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.231+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4c40014070 con 0x7f4c54101fd0 2024-05-08T23:03:33.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.231+0000 7f4c59400700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 0x7f4c3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.232+0000 7f4c59400700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 0x7f4c3c0a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b2e0 tx=0x7f4c48005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.235+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4c400964f0 con 0x7f4c54101fd0 2024-05-08T23:03:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.501+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c54067d10 con 0x7f4c54101fd0 2024-05-08T23:03:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.501+0000 7f4c52000700 1 -- 172.21.15.46:0/1034232281 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f4c40099b10 con 0x7f4c54101fd0 2024-05-08T23:03:33.502 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 msgr2=0x7f4c3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 0x7f4c3c0a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b2e0 tx=0x7f4c48005fb0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 msgr2=0x7f4c54073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000eb30 tx=0x7f4c4000ee40 comp rx=0 tx=0).stop 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 shutdown_connections 2024-05-08T23:03:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c54102990 0x7f4c540738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c54100e80 0x7f4c54074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f4c3c0a11b0 0x7f4c3c0a3670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 --2- 172.21.15.46:0/1034232281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c54101fd0 0x7f4c54073390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 >> 172.21.15.46:0/1034232281 conn(0x7f4c540fc410 msgr2=0x7f4c54105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 shutdown_connections 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.504+0000 7f4c5be00700 1 -- 172.21.15.46:0/1034232281 wait complete. 2024-05-08T23:03:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:33.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 -- 172.21.15.46:0/1767329618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4101fd0 msgr2=0x7f00c4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 --2- 172.21.15.46:0/1767329618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4101fd0 0x7f00c4102450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009990 tx=0x7f00b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 -- 172.21.15.46:0/1767329618 shutdown_connections 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 --2- 172.21.15.46:0/1767329618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4102990 0x7f00c41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 --2- 172.21.15.46:0/1767329618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4100e80 0x7f00c41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 --2- 172.21.15.46:0/1767329618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4101fd0 0x7f00c4102450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 -- 172.21.15.46:0/1767329618 >> 172.21.15.46:0/1767329618 conn(0x7f00c40fc410 msgr2=0x7f00c40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 -- 172.21.15.46:0/1767329618 shutdown_connections 2024-05-08T23:03:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.723+0000 7f00ca000700 1 -- 172.21.15.46:0/1767329618 wait complete. 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.724+0000 7f00ca000700 1 Processor -- start 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.724+0000 7f00ca000700 1 -- start start 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 0x7f00c4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4102990 0x7f00c41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41a0b40 con 0x7f00c4100e80 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41a0c80 con 0x7f00c4102990 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00ca000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41a0e10 con 0x7f00c4101fd0 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54012/0 (socket says 172.21.15.46:54012) 2024-05-08T23:03:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 -- 172.21.15.46:0/2457747244 learned_addr learned my addr 172.21.15.46:0/2457747244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c2a00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 0x7f00c4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3e00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4102990 0x7f00c41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 -- 172.21.15.46:0/2457747244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 msgr2=0x7f00c4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 0x7f00c4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 -- 172.21.15.46:0/2457747244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4102990 msgr2=0x7f00c41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4102990 0x7f00c41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c3400700 1 -- 172.21.15.46:0/2457747244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00095f0 con 0x7f00c4100e80 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.725+0000 7f00c2a00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 0x7f00c4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00c3400700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e9a0 tx=0x7f00b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4009790 con 0x7f00c4100e80 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b400cbd0 con 0x7f00c4100e80 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c41a10f0 con 0x7f00c4100e80 2024-05-08T23:03:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4018680 con 0x7f00c4100e80 2024-05-08T23:03:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.726+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c41a1650 con 0x7f00c4100e80 2024-05-08T23:03:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.727+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c4067d10 con 0x7f00c4100e80 2024-05-08T23:03:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.729+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f00b400cd40 con 0x7f00c4100e80 2024-05-08T23:03:33.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.730+0000 7f00bbe00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 0x7f00a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:33.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.730+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f00b4014070 con 0x7f00c4100e80 2024-05-08T23:03:33.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.730+0000 7f00c2a00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 0x7f00a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:33.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.732+0000 7f00c2a00700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 0x7f00a80a36f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00b000b2e0 tx=0x7f00b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:33.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.735+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00b4052d60 con 0x7f00c4100e80 2024-05-08T23:03:33.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:33 smithi046 ceph-mon[36898]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:33.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:33 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/283318498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:33 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1034232281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.956+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00c410c6d0 con 0x7f00c4100e80 2024-05-08T23:03:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.958+0000 7f00bbe00700 1 -- 172.21.15.46:0/2457747244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24623 (secure 0 0 0) 0x7f00b4099e20 con 0x7f00c4100e80 2024-05-08T23:03:33.958 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:33.958 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:19.015441+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 msgr2=0x7f00a80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 0x7f00a80a36f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00b000b2e0 tx=0x7f00b0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 msgr2=0x7f00c4199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e9a0 tx=0x7f00b400ed60 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 shutdown_connections 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00c4102990 0x7f00c41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f00a80a1230 0x7f00a80a36f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4100e80 0x7f00c4199010 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 --2- 172.21.15.46:0/2457747244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c4101fd0 0x7f00c4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.960+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 >> 172.21.15.46:0/2457747244 conn(0x7f00c40fc410 msgr2=0x7f00c40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.961+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 shutdown_connections 2024-05-08T23:03:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:33.961+0000 7f00ca000700 1 -- 172.21.15.46:0/2457747244 wait complete. 2024-05-08T23:03:33.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:33 smithi086 ceph-mon[78610]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:33.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:33 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/283318498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:33.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:33 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1034232281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:33.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:33.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:34.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:33 smithi053 ceph-mon[77587]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T23:03:34.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:33 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/283318498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:34.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:33 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1034232281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 -- 172.21.15.46:0/240714129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 msgr2=0x7facf0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 --2- 172.21.15.46:0/240714129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0102450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facdc009990 tx=0x7facdc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 -- 172.21.15.46:0/240714129 shutdown_connections 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 --2- 172.21.15.46:0/240714129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 --2- 172.21.15.46:0/240714129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 --2- 172.21.15.46:0/240714129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0102450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 -- 172.21.15.46:0/240714129 >> 172.21.15.46:0/240714129 conn(0x7facf00fc3b0 msgr2=0x7facf00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 -- 172.21.15.46:0/240714129 shutdown_connections 2024-05-08T23:03:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.182+0000 7facf5a00700 1 -- 172.21.15.46:0/240714129 wait complete. 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 Processor -- start 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 -- start start 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019e940 con 0x7facf0101fd0 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019ea80 con 0x7facf0100e80 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.183+0000 7facf5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019ec10 con 0x7facf0102990 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7facee000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50292/0 (socket says 172.21.15.46:50292) 2024-05-08T23:03:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7facee000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54026/0 (socket says 172.21.15.46:54026) 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 -- 172.21.15.46:0/2575893528 learned_addr learned my addr 172.21.15.46:0/2575893528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7facef400700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 -- 172.21.15.46:0/2575893528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 msgr2=0x7facf019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 -- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 msgr2=0x7facf0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 -- 172.21.15.46:0/2575893528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0095f0 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7facee000700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7facef400700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.184+0000 7faceea00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7face000e9a0 tx=0x7face000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.185+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0009790 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.185+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face0010430 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.185+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf019eef0 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.185+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0018680 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.185+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf019f310 con 0x7facf0100e80 2024-05-08T23:03:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.186+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf01082c0 con 0x7facf0100e80 2024-05-08T23:03:34.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.188+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7face00105a0 con 0x7facf0100e80 2024-05-08T23:03:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.189+0000 7face7400700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 0x7facd40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.189+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7face0014070 con 0x7facf0100e80 2024-05-08T23:03:34.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.189+0000 7facee000700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 0x7facd40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.190+0000 7facee000700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 0x7facd40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b2e0 tx=0x7facdc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:34.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.192+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7face0096440 con 0x7facf0100e80 2024-05-08T23:03:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.409+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facf010c9b0 con 0x7facf0100e80 2024-05-08T23:03:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.410+0000 7face7400700 1 -- 172.21.15.46:0/2575893528 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7facf010c9b0 con 0x7facf0100e80 2024-05-08T23:03:34.410 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 msgr2=0x7facd40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 0x7facd40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b2e0 tx=0x7facdc005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 msgr2=0x7facf0196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7face000e9a0 tx=0x7face000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 shutdown_connections 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf0102990 0x7facf019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facf0100e80 0x7facf0196e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7facd40a11b0 0x7facd40a3670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 --2- 172.21.15.46:0/2575893528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facf0101fd0 0x7facf0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 >> 172.21.15.46:0/2575893528 conn(0x7facf00fc3b0 msgr2=0x7facf0105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 shutdown_connections 2024-05-08T23:03:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.413+0000 7facf5a00700 1 -- 172.21.15.46:0/2575893528 wait complete. 2024-05-08T23:03:34.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.633+0000 7fd77d200700 1 -- 172.21.15.46:0/1176585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778108d40 msgr2=0x7fd77810b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.633+0000 7fd77d200700 1 --2- 172.21.15.46:0/1176585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778108d40 0x7fd77810b1d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd770009f90 tx=0x7fd77000b330 comp rx=0 tx=0).stop 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 -- 172.21.15.46:0/1176585890 shutdown_connections 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 --2- 172.21.15.46:0/1176585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778108d40 0x7fd77810b1d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 --2- 172.21.15.46:0/1176585890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778074fe0 0x7fd778073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 --2- 172.21.15.46:0/1176585890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 -- 172.21.15.46:0/1176585890 >> 172.21.15.46:0/1176585890 conn(0x7fd7780fc3b0 msgr2=0x7fd7780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 -- 172.21.15.46:0/1176585890 shutdown_connections 2024-05-08T23:03:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.634+0000 7fd77d200700 1 -- 172.21.15.46:0/1176585890 wait complete. 2024-05-08T23:03:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 Processor -- start 2024-05-08T23:03:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 -- start start 2024-05-08T23:03:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778074fe0 0x7fd778199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 0x7fd7781a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781a0c50 con 0x7fd778073a00 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781a0d90 con 0x7fd778108d40 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd77d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781a0f20 con 0x7fd778074fe0 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd775e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd775400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778074fe0 0x7fd778199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd775e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54036/0 (socket says 172.21.15.46:54036) 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.635+0000 7fd775e00700 1 -- 172.21.15.46:0/2131096801 learned_addr learned my addr 172.21.15.46:0/2131096801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd776800700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 0x7fd7781a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 -- 172.21.15.46:0/2131096801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778074fe0 msgr2=0x7fd778199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778074fe0 0x7fd778199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 -- 172.21.15.46:0/2131096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 msgr2=0x7fd7781a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 0x7fd7781a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 -- 172.21.15.46:0/2131096801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009c40 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd776800700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 0x7fd7781a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd775e00700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9c0 tx=0x7fd76800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680097b0 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7781a1200 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd768004cf0 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.636+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768005590 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.637+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7781a1710 con 0x7fd778073a00 2024-05-08T23:03:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.638+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd778107b00 con 0x7fd778073a00 2024-05-08T23:03:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.640+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd76800cc60 con 0x7fd778073a00 2024-05-08T23:03:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.642+0000 7fd76ea00700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 0x7fd75c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.642+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd768014070 con 0x7fd778073a00 2024-05-08T23:03:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.643+0000 7fd775400700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 0x7fd75c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.643+0000 7fd775400700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 0x7fd75c0a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd778074d00 tx=0x7fd764009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.647+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7680962d0 con 0x7fd778073a00 2024-05-08T23:03:34.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.873+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd77810e8e0 con 0x7fd778073a00 2024-05-08T23:03:34.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.873+0000 7fd76ea00700 1 -- 172.21.15.46:0/2131096801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7680998f0 con 0x7fd778073a00 2024-05-08T23:03:34.873 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 msgr2=0x7fd75c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 0x7fd75c0a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd778074d00 tx=0x7fd764009310 comp rx=0 tx=0).stop 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 msgr2=0x7fd778199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9c0 tx=0x7fd76800ed80 comp rx=0 tx=0).stop 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 shutdown_connections 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd778108d40 0x7fd7781a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fd75c0a1230 0x7fd75c0a36f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd778073a00 0x7fd778199120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 --2- 172.21.15.46:0/2131096801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778074fe0 0x7fd778199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:34.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 >> 172.21.15.46:0/2131096801 conn(0x7fd7780fc3b0 msgr2=0x7fd778104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:34.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 shutdown_connections 2024-05-08T23:03:34.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:34.876+0000 7fd77d200700 1 -- 172.21.15.46:0/2131096801 wait complete. 2024-05-08T23:03:34.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:34.952 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:34 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2457747244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:34.953 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:34 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2575893528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:34.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:34 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2457747244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:34.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:34 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2575893528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:35.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:34 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2457747244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:35.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:34 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2575893528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 -- 172.21.15.46:0/753643241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 msgr2=0x7fa9981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 --2- 172.21.15.46:0/753643241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa9981012a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009990 tx=0x7fa98c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 -- 172.21.15.46:0/753643241 shutdown_connections 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 --2- 172.21.15.46:0/753643241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 0x7fa9981092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 --2- 172.21.15.46:0/753643241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa9981012a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 --2- 172.21.15.46:0/753643241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa998102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 -- 172.21.15.46:0/753643241 >> 172.21.15.46:0/753643241 conn(0x7fa9980fc3b0 msgr2=0x7fa9980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 -- 172.21.15.46:0/753643241 shutdown_connections 2024-05-08T23:03:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.101+0000 7fa99f600700 1 -- 172.21.15.46:0/753643241 wait complete. 2024-05-08T23:03:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 Processor -- start 2024-05-08T23:03:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 -- start start 2024-05-08T23:03:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa998199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 0x7fa9981a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0bc0 con 0x7fa998100e80 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0d00 con 0x7fa998102990 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.102+0000 7fa99f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0e90 con 0x7fa998101fd0 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa99cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa998199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53600/0 (socket says 172.21.15.46:53600) 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 -- 172.21.15.46:0/290911403 learned_addr learned my addr 172.21.15.46:0/290911403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa99d600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 0x7fa9981a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 -- 172.21.15.46:0/290911403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 msgr2=0x7fa9981a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 0x7fa9981a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 -- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 msgr2=0x7fa998199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa998199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 -- 172.21.15.46:0/290911403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0095f0 con 0x7fa998101fd0 2024-05-08T23:03:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa99cc00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa998199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.103+0000 7fa997e00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9c0 tx=0x7fa98400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.104+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840097b0 con 0x7fa998101fd0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.104+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981a1170 con 0x7fa998101fd0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.104+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa98400ee60 con 0x7fa998101fd0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.104+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840054f0 con 0x7fa998101fd0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.104+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a1630 con 0x7fa998101fd0 2024-05-08T23:03:35.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.105+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998067d10 con 0x7fa998101fd0 2024-05-08T23:03:35.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.107+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa984004750 con 0x7fa998101fd0 2024-05-08T23:03:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.108+0000 7fa995600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 0x7fa9800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.108+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fa984014070 con 0x7fa998101fd0 2024-05-08T23:03:35.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.108+0000 7fa99cc00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 0x7fa9800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.109+0000 7fa99cc00700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 0x7fa9800a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b2e0 tx=0x7fa98c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:35.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.110+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9840964a0 con 0x7fa998101fd0 2024-05-08T23:03:35.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.328+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa99810c6f0 con 0x7fa998101fd0 2024-05-08T23:03:35.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.329+0000 7fa995600700 1 -- 172.21.15.46:0/290911403 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa984099ac0 con 0x7fa998101fd0 2024-05-08T23:03:35.329 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:35.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 msgr2=0x7fa9800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 0x7fa9800a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b2e0 tx=0x7fa98c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 msgr2=0x7fa9981995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9c0 tx=0x7fa98400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 shutdown_connections 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa998102990 0x7fa9981a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fa9800a11b0 0x7fa9800a3670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa998100e80 0x7fa998199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 --2- 172.21.15.46:0/290911403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa998101fd0 0x7fa9981995d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 >> 172.21.15.46:0/290911403 conn(0x7fa9980fc3b0 msgr2=0x7fa9980731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 shutdown_connections 2024-05-08T23:03:35.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.331+0000 7fa99f600700 1 -- 172.21.15.46:0/290911403 wait complete. 2024-05-08T23:03:35.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.553+0000 7f6b04200700 1 -- 172.21.15.46:0/371518081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc10d450 msgr2=0x7f6afc10f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.553+0000 7f6b04200700 1 --2- 172.21.15.46:0/371518081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc10d450 0x7f6afc10f840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b0a0 tx=0x7f6af400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 -- 172.21.15.46:0/371518081 shutdown_connections 2024-05-08T23:03:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 --2- 172.21.15.46:0/371518081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc10d450 0x7f6afc10f840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 --2- 172.21.15.46:0/371518081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 0x7f6afc073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 --2- 172.21.15.46:0/371518081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc073a00 0x7f6afc073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 -- 172.21.15.46:0/371518081 >> 172.21.15.46:0/371518081 conn(0x7f6afc0fc3b0 msgr2=0x7f6afc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 -- 172.21.15.46:0/371518081 shutdown_connections 2024-05-08T23:03:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.554+0000 7f6b04200700 1 -- 172.21.15.46:0/371518081 wait complete. 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 Processor -- start 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 -- start start 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 0x7f6afc19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 0x7f6afc1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1a5050 con 0x7f6afc073a00 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1a5190 con 0x7f6afc10d450 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b04200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1a5320 con 0x7f6afc074fe0 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b01800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.555+0000 7f6b00e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 0x7f6afc19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54092/0 (socket says 172.21.15.46:54092) 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 -- 172.21.15.46:0/438729174 learned_addr learned my addr 172.21.15.46:0/438729174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:35.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b02200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 0x7f6afc1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 -- 172.21.15.46:0/438729174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 msgr2=0x7f6afc19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 0x7f6afc19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 -- 172.21.15.46:0/438729174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 msgr2=0x7f6afc1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 0x7f6afc1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 -- 172.21.15.46:0/438729174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4009d00 con 0x7f6afc073a00 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b02200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 0x7f6afc1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b01800700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9c0 tx=0x7f6af000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00097b0 con 0x7f6afc073a00 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af000ee60 con 0x7f6afc073a00 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00054f0 con 0x7f6afc073a00 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.556+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc1a5600 con 0x7f6afc073a00 2024-05-08T23:03:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.557+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc1a5ac0 con 0x7f6afc073a00 2024-05-08T23:03:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.558+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc1036b0 con 0x7f6afc073a00 2024-05-08T23:03:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.560+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6af0004750 con 0x7f6afc073a00 2024-05-08T23:03:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.561+0000 7f6afa000700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 0x7f6ae40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.561+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f6af0014070 con 0x7f6afc073a00 2024-05-08T23:03:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.561+0000 7f6b00e00700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 0x7f6ae40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.562+0000 7f6b00e00700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 0x7f6ae40a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6afc074d00 tx=0x7f6ae8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.565+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6af0096250 con 0x7f6afc073a00 2024-05-08T23:03:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.787+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6afc112f90 con 0x7f6afc073a00 2024-05-08T23:03:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.787+0000 7f6afa000700 1 -- 172.21.15.46:0/438729174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6af0099870 con 0x7f6afc073a00 2024-05-08T23:03:35.787 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 msgr2=0x7f6ae40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 0x7f6ae40a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6afc074d00 tx=0x7f6ae8009260 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 msgr2=0x7f6afc19d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9c0 tx=0x7f6af000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 shutdown_connections 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6afc10d450 0x7f6afc1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f6ae40a1230 0x7f6ae40a36f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6afc073a00 0x7f6afc19d520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 --2- 172.21.15.46:0/438729174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6afc074fe0 0x7f6afc19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 >> 172.21.15.46:0/438729174 conn(0x7f6afc0fc3b0 msgr2=0x7f6afc1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 shutdown_connections 2024-05-08T23:03:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:35.790+0000 7f6b04200700 1 -- 172.21.15.46:0/438729174 wait complete. 2024-05-08T23:03:35.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:35 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2131096801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:35 smithi086 ceph-mon[78610]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:35.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:35 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/290911403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 -- 172.21.15.46:0/124459428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 msgr2=0x7f056c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 --2- 172.21.15.46:0/124459428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c1010d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05600099c0 tx=0x7f0560009cd0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 -- 172.21.15.46:0/124459428 shutdown_connections 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 --2- 172.21.15.46:0/124459428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c102840 0x7f056c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 --2- 172.21.15.46:0/124459428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c1010d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 --2- 172.21.15.46:0/124459428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c101ea0 0x7f056c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 -- 172.21.15.46:0/124459428 >> 172.21.15.46:0/124459428 conn(0x7f056c0fc220 msgr2=0x7f056c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 -- 172.21.15.46:0/124459428 shutdown_connections 2024-05-08T23:03:36.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.006+0000 7f0572a00700 1 -- 172.21.15.46:0/124459428 wait complete. 2024-05-08T23:03:36.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.007+0000 7f0572a00700 1 Processor -- start 2024-05-08T23:03:36.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.007+0000 7f0572a00700 1 -- start start 2024-05-08T23:03:36.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c101ea0 0x7f056c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c102840 0x7f056c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1a09f0 con 0x7f056c100cb0 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1a0b30 con 0x7f056c101ea0 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1a0cc0 con 0x7f056c102840 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54104/0 (socket says 172.21.15.46:54104) 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 -- 172.21.15.46:0/3469635716 learned_addr learned my addr 172.21.15.46:0/3469635716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0570c00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c101ea0 0x7f056c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0572000700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c102840 0x7f056c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 -- 172.21.15.46:0/3469635716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c102840 msgr2=0x7f056c1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c102840 0x7f056c1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 -- 172.21.15.46:0/3469635716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c101ea0 msgr2=0x7f056c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c101ea0 0x7f056c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.008+0000 7f0571600700 1 -- 172.21.15.46:0/3469635716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600095f0 con 0x7f056c100cb0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f0571600700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0560006010 tx=0x7f056000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056001d070 con 0x7f056c100cb0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0560005020 con 0x7f056c100cb0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c1a0f40 con 0x7f056c100cb0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560004b10 con 0x7f056c100cb0 2024-05-08T23:03:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.009+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1a12b0 con 0x7f056c100cb0 2024-05-08T23:03:36.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:35 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2131096801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:35 smithi053 ceph-mon[77587]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:36.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:35 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/290911403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.010+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c04ed40 con 0x7f056c100cb0 2024-05-08T23:03:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.012+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0560005190 con 0x7f056c100cb0 2024-05-08T23:03:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.013+0000 7f056a000700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 0x7f05540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.014+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f05600cd880 con 0x7f056c100cb0 2024-05-08T23:03:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.014+0000 7f0570c00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 0x7f05540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.014+0000 7f0570c00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 0x7f05540a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0558009c60 tx=0x7f0558009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.017+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f056c04ed40 con 0x7f056c100cb0 2024-05-08T23:03:36.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:35 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2131096801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:35 smithi046 ceph-mon[36898]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T23:03:36.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:35 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/290911403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.285+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056c04ecd0 con 0x7f056c100cb0 2024-05-08T23:03:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.286+0000 7f056a000700 1 -- 172.21.15.46:0/3469635716 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f0560096b40 con 0x7f056c100cb0 2024-05-08T23:03:36.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:36.286 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:02:19.629003+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.288+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 msgr2=0x7f05540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.288+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 0x7f05540a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0558009c60 tx=0x7f0558009260 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.288+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 msgr2=0x7f056c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.288+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0560006010 tx=0x7f056000f690 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 shutdown_connections 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c102840 0x7f056c1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f05540a11b0 0x7f05540a3670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f056c100cb0 0x7f056c198f10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 --2- 172.21.15.46:0/3469635716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f056c101ea0 0x7f056c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 >> 172.21.15.46:0/3469635716 conn(0x7f056c0fc220 msgr2=0x7f056c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 shutdown_connections 2024-05-08T23:03:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.289+0000 7f0572a00700 1 -- 172.21.15.46:0/3469635716 wait complete. 2024-05-08T23:03:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-08T23:03:36.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 -- 172.21.15.46:0/2179676122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 msgr2=0x7fad380733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 --2- 172.21.15.46:0/2179676122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad380733d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad28009990 tx=0x7fad28009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 -- 172.21.15.46:0/2179676122 shutdown_connections 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 --2- 172.21.15.46:0/2179676122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad3810aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 --2- 172.21.15.46:0/2179676122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad380733d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 --2- 172.21.15.46:0/2179676122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 0x7fad38073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 -- 172.21.15.46:0/2179676122 >> 172.21.15.46:0/2179676122 conn(0x7fad380fc200 msgr2=0x7fad380fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 -- 172.21.15.46:0/2179676122 shutdown_connections 2024-05-08T23:03:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.506+0000 7fad3ca00700 1 -- 172.21.15.46:0/2179676122 wait complete. 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.507+0000 7fad3ca00700 1 Processor -- start 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.507+0000 7fad3ca00700 1 -- start start 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 0x7fad38198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a0960 con 0x7fad38108b80 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a0aa0 con 0x7fad380739a0 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad3ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a0c30 con 0x7fad38074f80 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad37e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad37e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54112/0 (socket says 172.21.15.46:54112) 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad37400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 0x7fad38198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad37e00700 1 -- 172.21.15.46:0/4151987647 learned_addr learned my addr 172.21.15.46:0/4151987647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 -- 172.21.15.46:0/4151987647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 msgr2=0x7fad38198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 0x7fad38198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 -- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 msgr2=0x7fad381a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad36a00700 1 -- 172.21.15.46:0/4151987647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280095f0 con 0x7fad38074f80 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.508+0000 7fad37e00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad36a00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad381993c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad2400e9f0 tx=0x7fad2400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24004220 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad24004cf0 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad381a0f10 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24005020 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.509+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad381a1340 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.510+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38107990 con 0x7fad38074f80 2024-05-08T23:03:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.512+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fad24004750 con 0x7fad38074f80 2024-05-08T23:03:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.513+0000 7fad2fe00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 0x7fad1c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.514+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fad24014070 con 0x7fad38074f80 2024-05-08T23:03:36.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.514+0000 7fad37400700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 0x7fad1c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.514+0000 7fad37400700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 0x7fad1c0a3670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b2e0 tx=0x7fad28005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.516+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad24096090 con 0x7fad38074f80 2024-05-08T23:03:36.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.735+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad3810e700 con 0x7fad38074f80 2024-05-08T23:03:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.737+0000 7fad2fe00700 1 -- 172.21.15.46:0/4151987647 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24623 (secure 0 0 0) 0x7fad24004e60 con 0x7fad38074f80 2024-05-08T23:03:36.738 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:36.738 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:19.015441+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 msgr2=0x7fad1c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 0x7fad1c0a3670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b2e0 tx=0x7fad28005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 msgr2=0x7fad381993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad381993c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad2400e9f0 tx=0x7fad2400ed00 comp rx=0 tx=0).stop 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 shutdown_connections 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fad1c0a11b0 0x7fad1c0a3670 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad38108b80 0x7fad381a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad380739a0 0x7fad38198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 --2- 172.21.15.46:0/4151987647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad38074f80 0x7fad381993c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 >> 172.21.15.46:0/4151987647 conn(0x7fad380fc200 msgr2=0x7fad38104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 shutdown_connections 2024-05-08T23:03:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.740+0000 7fad3ca00700 1 -- 172.21.15.46:0/4151987647 wait complete. 2024-05-08T23:03:36.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:01:51.460837+0000', 'modified': '2024-05-08T23:02:19.629003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24583, 'mds_2': 24571, 'mds_3': 24547, 'mds_4': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/3863967478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 3863967478}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 3863967478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/272583724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 272583724}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 272583724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6837/1812878749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6836', 'nonce': 1812878749}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 1812878749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/1845130080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1845130080}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1845130080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/170717769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 170717769}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 170717769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.86:6835/2919086413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2919086413}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2919086413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.86:6837/2553915750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 2553915750}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2553915750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24571': {'gid': 24571, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.86:6839/4115816206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 4115816206}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 4115816206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24583': {'gid': 24583, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.46:6839/1484730309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 1484730309}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 1484730309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/3374696949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3374696949}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3374696949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:03:36.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T23:03:36.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.960+0000 7f2db8a00700 1 -- 172.21.15.46:0/908067164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4102990 msgr2=0x7f2db41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.960+0000 7f2db8a00700 1 --2- 172.21.15.46:0/908067164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4102990 0x7f2db41092b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b0a0 tx=0x7f2dac00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.960+0000 7f2db8a00700 1 -- 172.21.15.46:0/908067164 shutdown_connections 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.960+0000 7f2db8a00700 1 --2- 172.21.15.46:0/908067164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4102990 0x7f2db41092b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.961+0000 7f2db8a00700 1 --2- 172.21.15.46:0/908067164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4100e80 0x7f2db41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.961+0000 7f2db8a00700 1 --2- 172.21.15.46:0/908067164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4101fd0 0x7f2db4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.961+0000 7f2db8a00700 1 -- 172.21.15.46:0/908067164 >> 172.21.15.46:0/908067164 conn(0x7f2db40fc3b0 msgr2=0x7f2db40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.961+0000 7f2db8a00700 1 -- 172.21.15.46:0/908067164 shutdown_connections 2024-05-08T23:03:36.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.961+0000 7f2db8a00700 1 -- 172.21.15.46:0/908067164 wait complete. 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 Processor -- start 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 -- start start 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4101fd0 0x7f2db4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4102990 0x7f2db41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41a0c30 con 0x7f2db4100e80 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41a0d70 con 0x7f2db4102990 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41a0f00 con 0x7f2db4101fd0 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.962+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54132/0 (socket says 172.21.15.46:54132) 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 -- 172.21.15.46:0/1111836972 learned_addr learned my addr 172.21.15.46:0/1111836972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1400700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4101fd0 0x7f2db4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db2800700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4102990 0x7f2db41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 -- 172.21.15.46:0/1111836972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4101fd0 msgr2=0x7f2db4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4101fd0 0x7f2db4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 -- 172.21.15.46:0/1111836972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4102990 msgr2=0x7f2db41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4102990 0x7f2db41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 -- 172.21.15.46:0/1111836972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009d00 con 0x7f2db4100e80 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2da400e9c0 tx=0x7f2da400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2daaa00700 1 -- 172.21.15.46:0/1111836972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40097b0 con 0x7f2db4100e80 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2daaa00700 1 -- 172.21.15.46:0/1111836972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da400ee60 con 0x7f2db4100e80 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db41a11e0 con 0x7f2db4100e80 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.963+0000 7f2daaa00700 1 -- 172.21.15.46:0/1111836972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40054f0 con 0x7f2db4100e80 2024-05-08T23:03:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.964+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db41a16a0 con 0x7f2db4100e80 2024-05-08T23:03:36.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:36 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/438729174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:36.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:36 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3469635716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:36.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.967+0000 7f2daaa00700 1 -- 172.21.15.46:0/1111836972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2da4004750 con 0x7f2db4100e80 2024-05-08T23:03:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.968+0000 7f2dcc429b80 1 Processor -- start 2024-05-08T23:03:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.968+0000 7f2dcc429b80 1 -- start start 2024-05-08T23:03:36.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.968+0000 7f2daaa00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 0x7f2d980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ad82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ad8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3a2e100 con 0x5564e3ad74f0 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3a2bac0 con 0x5564e3ad8810 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3a273b0 con 0x5564e3ad7e50 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2daaa00700 1 -- 172.21.15.46:0/1111836972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2da4014070 con 0x7f2db4100e80 2024-05-08T23:03:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db2800700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 0x7f2d980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54134/0 (socket says 172.21.15.46:54134) 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 -- 172.21.15.46:0/1634917463 learned_addr learned my addr 172.21.15.46:0/1634917463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db2800700 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ad8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1400700 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ad82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 -- 172.21.15.46:0/1634917463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 msgr2=0x5564e3ad82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ad82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 -- 172.21.15.46:0/1634917463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 msgr2=0x5564e3ad8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ad8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.969+0000 7f2db1e00700 1 -- 172.21.15.46:0/1634917463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564e38a8360 con 0x5564e3ad74f0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.970+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2da40a1f10 tx=0x7f2da409a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e82c069ba444ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.970+0000 7f2db2800700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 0x7f2d980a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009fd0 tx=0x7f2dac007da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.970+0000 7f2d9f000700 1 -- 172.21.15.46:0/1634917463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da409ddf0 con 0x5564e3ad74f0 2024-05-08T23:03:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.970+0000 7f2d9f000700 1 -- 172.21.15.46:0/1634917463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da40a2440 con 0x5564e3ad74f0 2024-05-08T23:03:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.970+0000 7f2d9f000700 1 -- 172.21.15.46:0/1634917463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4016330 con 0x5564e3ad74f0 2024-05-08T23:03:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1634917463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 msgr2=0x5564e3ad7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2da40a1f10 tx=0x7f2da409a740 comp rx=0 tx=0).stop 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1634917463 shutdown_connections 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ad8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad74f0 0x5564e3ad7910 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1634917463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ad82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1634917463 >> 172.21.15.46:0/1634917463 conn(0x5564e3a351c0 msgr2=0x5564e3ad1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1634917463 shutdown_connections 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1634917463 wait complete. 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.971+0000 7f2dcc429b80 1 -- start start 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 Processor -- start 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e39c3510 con 0x5564e3ad7e50 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e39c31b0 con 0x5564e3ad8810 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e38aa650 con 0x5564e3ad74f0 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2dcc429b80 10 client.0 _ll_register_callbacks cb 0x5564e39ccae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53684/0 (socket says 172.21.15.46:53684) 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db1e00700 1 -- 172.21.15.46:0/1796043868 learned_addr learned my addr 172.21.15.46:0/1796043868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50380/0 (socket says 172.21.15.46:50380) 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.972+0000 7f2db1400700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 -- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 msgr2=0x5564e3ae0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 -- 172.21.15.46:0/1796043868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 msgr2=0x5564e3ae1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 -- 172.21.15.46:0/1796043868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db41a16a0 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2db2800700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b070 tx=0x7f2dac00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9e876aef21b59f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2d93400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00e040 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564e3980220 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.973+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2dac01f440 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.974+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dac009b80 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.974+0000 7f2dcc429b80 10 client.25394 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.974+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5564e3854b10 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.974+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564e3980220 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.974+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dac019030 con 0x5564e3ad8810 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 10 client.25394 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 20 client.25394 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3aebf80 0x5564e3af1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 4 client.25394 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564e3aa67c0 con 0x5564e3aebf80 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3aebf80 0x5564e3af1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2db1e00700 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3aebf80 0x5564e3af1280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b50dc1b5e01ce784 server_cookie=c0c882235490dd5a in_seq=0 out_seq=0 2024-05-08T23:03:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.975+0000 7f2d93400700 10 client.25394 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2d93400700 1 -- 172.21.15.46:0/1796043868 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5564e3aa67c0 con 0x5564e3aebf80 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2d93400700 10 client.25394 handle_command_reply: tid=0 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 client.25394 shutdown 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 msgr2=0x5564e3ae8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b070 tx=0x7f2dac00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 shutdown_connections 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3aebf80 0x5564e3af1280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae8340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.976+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ae0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/1796043868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 >> 172.21.15.46:0/1796043868 conn(0x5564e3a351c0 msgr2=0x5564e3ad1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2d93400700 0 client.25394 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 shutdown_connections 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2dcc429b80 1 -- 172.21.15.46:0/1796043868 wait complete. 2024-05-08T23:03:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.977+0000 7f2dcc429b80 20 client.25394 trim_cache size 0 max 16384 2024-05-08T23:03:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 Processor -- start 2024-05-08T23:03:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 -- start start 2024-05-08T23:03:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e39c65a0 con 0x5564e3ad7e50 2024-05-08T23:03:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e39e52b0 con 0x5564e3ad8810 2024-05-08T23:03:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.996+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3ae94c0 con 0x5564e3ad74f0 2024-05-08T23:03:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53698/0 (socket says 172.21.15.46:53698) 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 -- 172.21.15.46:0/2625242709 learned_addr learned my addr 172.21.15.46:0/2625242709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db2800700 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 -- 172.21.15.46:0/2625242709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 msgr2=0x5564e3ae89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 -- 172.21.15.46:0/2625242709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 msgr2=0x5564e3ae8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 -- 172.21.15.46:0/2625242709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564e39ed510 con 0x5564e3ad74f0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db2800700 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.997+0000 7f2db1e00700 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2da4099f50 tx=0x7f2da40ca8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c0f771806847d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2d9fa00700 1 -- 172.21.15.46:0/2625242709 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40ca9b0 con 0x5564e3ad74f0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2d9fa00700 1 -- 172.21.15.46:0/2625242709 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da40ade90 con 0x5564e3ad74f0 2024-05-08T23:03:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2d9fa00700 1 -- 172.21.15.46:0/2625242709 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4016330 con 0x5564e3ad74f0 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 -- 172.21.15.46:0/2625242709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 msgr2=0x5564e3ad78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2da4099f50 tx=0x7f2da40ca8d0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 -- 172.21.15.46:0/2625242709 shutdown_connections 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad8810 0x5564e3ae89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad74f0 0x5564e3ad78d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/2625242709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad7e50 0x5564e3ae8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 -- 172.21.15.46:0/2625242709 >> 172.21.15.46:0/2625242709 conn(0x5564e3a351c0 msgr2=0x5564e3ace1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 -- 172.21.15.46:0/2625242709 shutdown_connections 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.998+0000 7f2dcc429b80 1 -- 172.21.15.46:0/2625242709 wait complete. 2024-05-08T23:03:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.999+0000 7f2dcc429b80 1 -- start start 2024-05-08T23:03:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:36.999+0000 7f2dcc429b80 1 Processor -- start 2024-05-08T23:03:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 0x5564e3ae2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ae2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 0x5564e3ae7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3ae7970 con 0x5564e3ad8810 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3ae7ae0 con 0x5564e3ad74f0 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3ae7c50 con 0x5564e3ad7e50 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2dcc429b80 10 client.0 _ll_register_callbacks cb 0x5564e39ccae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ae2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 0x5564e3ae7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ae2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53700/0 (socket says 172.21.15.46:53700) 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db1400700 1 -- 172.21.15.46:0/3570080575 learned_addr learned my addr 172.21.15.46:0/3570080575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db1e00700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 0x5564e3ae2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 -- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 msgr2=0x5564e3ae2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ae2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 -- 172.21.15.46:0/3570080575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 msgr2=0x5564e3ae2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 0x5564e3ae2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db2800700 1 -- 172.21.15.46:0/3570080575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564e38a8360 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.000+0000 7f2db1e00700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 0x5564e3ae2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2db2800700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 0x5564e3ae7430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b0a0 tx=0x7f2dac0094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4d29da74bc5c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2d93400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac01f440 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2dac02ac60 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2da40a2440 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dac01f810 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2dcc429b80 10 client.15408 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5564e38acc80 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.001+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da40a2440 con 0x5564e3ad8810 2024-05-08T23:03:37.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dac032040 con 0x5564e3ad8810 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 10 client.15408 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 20 client.15408 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3b46e50 0x5564e3b437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 4 client.15408 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564e3a1fe70 con 0x5564e3b46e50 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2db1e00700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3b46e50 0x5564e3b437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2db1e00700 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3b46e50 0x5564e3b437b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f09cb0c73bc4b777 server_cookie=7dac3881f4b4a4bb in_seq=0 out_seq=0 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.002+0000 7f2d93400700 10 client.15408 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2d93400700 1 -- 172.21.15.46:0/3570080575 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5564e3a1fe70 con 0x5564e3b46e50 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2d93400700 10 client.15408 handle_command_reply: tid=0 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 client.15408 shutdown 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 msgr2=0x5564e3ae7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 0x5564e3ae7430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b0a0 tx=0x7f2dac0094b0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 shutdown_connections 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5564e3b46e50 0x5564e3b437b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3ad8810 0x5564e3ae7430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564e3ad74f0 0x5564e3ae2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 --2- 172.21.15.46:0/3570080575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564e3ad7e50 0x5564e3ae2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2d93400700 0 client.15408 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 >> 172.21.15.46:0/3570080575 conn(0x5564e3a351c0 msgr2=0x5564e3ade990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 shutdown_connections 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 1 -- 172.21.15.46:0/3570080575 wait complete. 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.003+0000 7f2dcc429b80 20 client.15408 trim_cache size 0 max 16384 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:37.008 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24541, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 79.339215326000001, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 86, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 124.797481973 2024-05-08T23:03:37.009 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 msgr2=0x7f2d980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 0x7f2d980a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009fd0 tx=0x7f2dac007da0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 msgr2=0x7f2db4199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2da400e9c0 tx=0x7f2da400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 shutdown_connections 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2db4102990 0x7f2db41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f2d980a11b0 0x7f2d980a3670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db4100e80 0x7f2db4199100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 --2- 172.21.15.46:0/1111836972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2db4101fd0 0x7f2db4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 >> 172.21.15.46:0/1111836972 conn(0x7f2db40fc3b0 msgr2=0x7f2db40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 shutdown_connections 2024-05-08T23:03:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.004+0000 7f2db8a00700 1 -- 172.21.15.46:0/1111836972 wait complete. 2024-05-08T23:03:37.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:36 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/438729174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:37.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:36 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3469635716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:37.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:36 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/438729174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:37.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:36 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3469635716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:37.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.220+0000 7f9abe800700 1 -- 172.21.15.46:0/2393854797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8102990 msgr2=0x7f9ab81092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.220+0000 7f9abe800700 1 --2- 172.21.15.46:0/2393854797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8102990 0x7f9ab81092b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f90 tx=0x7f9ab000b330 comp rx=0 tx=0).stop 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 -- 172.21.15.46:0/2393854797 shutdown_connections 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 --2- 172.21.15.46:0/2393854797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8102990 0x7f9ab81092b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 --2- 172.21.15.46:0/2393854797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 0x7f9ab81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 --2- 172.21.15.46:0/2393854797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 -- 172.21.15.46:0/2393854797 >> 172.21.15.46:0/2393854797 conn(0x7f9ab80fc3d0 msgr2=0x7f9ab80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 -- 172.21.15.46:0/2393854797 shutdown_connections 2024-05-08T23:03:37.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.221+0000 7f9abe800700 1 -- 172.21.15.46:0/2393854797 wait complete. 2024-05-08T23:03:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 Processor -- start 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 -- start start 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 0x7f9ab8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8102990 0x7f9ab80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8074070 con 0x7f9ab8101fd0 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80741e0 con 0x7f9ab8100e80 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9abe800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81a0f60 con 0x7f9ab8102990 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54194/0 (socket says 172.21.15.46:54194) 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9ab6a00700 1 -- 172.21.15.46:0/1647024892 learned_addr learned my addr 172.21.15.46:0/1647024892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9ab7400700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 0x7f9ab8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.222+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8102990 0x7f9ab80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 -- 172.21.15.46:0/1647024892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8102990 msgr2=0x7f9ab80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8102990 0x7f9ab80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 -- 172.21.15.46:0/1647024892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 msgr2=0x7f9ab8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 0x7f9ab8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 -- 172.21.15.46:0/1647024892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0009c40 con 0x7f9ab8101fd0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400e9c0 tx=0x7f9aa400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9aafe00700 1 -- 172.21.15.46:0/1647024892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004a30 con 0x7f9ab8101fd0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9aafe00700 1 -- 172.21.15.46:0/1647024892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa40054e0 con 0x7f9ab8101fd0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab81a11f0 con 0x7f9ab8101fd0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.223+0000 7f9aafe00700 1 -- 172.21.15.46:0/1647024892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400cbf0 con 0x7f9ab8101fd0 2024-05-08T23:03:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.224+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab81a1700 con 0x7f9ab8101fd0 2024-05-08T23:03:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.226+0000 7f9aafe00700 1 -- 172.21.15.46:0/1647024892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9aa400cd50 con 0x7f9ab8101fd0 2024-05-08T23:03:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9aafe00700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 0x7f9a9c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 Processor -- start 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9aafe00700 1 -- 172.21.15.46:0/1647024892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9aa4014070 con 0x7f9ab8101fd0 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 -- start start 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ab7400700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 0x7f9a9c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c704f0 0x55ff36c70910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c71810 0x55ff36c71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36bc7100 con 0x55ff36c704f0 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36bc4ac0 con 0x55ff36c70e50 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.228+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36bc03b0 con 0x55ff36c71810 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c71810 0x55ff36c71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50432/0 (socket says 172.21.15.46:50432) 2024-05-08T23:03:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 -- 172.21.15.46:0/327209312 learned_addr learned my addr 172.21.15.46:0/327209312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab7400700 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c704f0 0x55ff36c70910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 -- 172.21.15.46:0/327209312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c71810 msgr2=0x55ff36c71c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c71810 0x55ff36c71c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 -- 172.21.15.46:0/327209312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c704f0 msgr2=0x55ff36c70910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c704f0 0x55ff36c70910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 -- 172.21.15.46:0/327209312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff36a41360 con 0x55ff36c70e50 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab7400700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 0x7f9a9c0a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8005950 tx=0x7f9aa80106b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40a1db0 tx=0x7f9aa40998c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1449377bd48ed44e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.229+0000 7f9aacc00700 1 -- 172.21.15.46:0/327209312 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40b1070 con 0x55ff36c70e50 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9aacc00700 1 -- 172.21.15.46:0/327209312 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa4099e60 con 0x55ff36c70e50 2024-05-08T23:03:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9aacc00700 1 -- 172.21.15.46:0/327209312 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004b90 con 0x55ff36c70e50 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 -- 172.21.15.46:0/327209312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 msgr2=0x55ff36c712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40a1db0 tx=0x7f9aa40998c0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 -- 172.21.15.46:0/327209312 shutdown_connections 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c71810 0x55ff36c71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c704f0 0x55ff36c70910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/327209312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c712d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 -- 172.21.15.46:0/327209312 >> 172.21.15.46:0/327209312 conn(0x55ff36bce1c0 msgr2=0x55ff36c6ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 -- 172.21.15.46:0/327209312 shutdown_connections 2024-05-08T23:03:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.230+0000 7f9ad225eb80 1 -- 172.21.15.46:0/327209312 wait complete. 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.231+0000 7f9ad225eb80 1 -- start start 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.231+0000 7f9ad225eb80 1 Processor -- start 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c79e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c71810 0x55ff36c813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36b5c510 con 0x55ff36c70e50 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36b5c1b0 con 0x55ff36c71810 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36a43650 con 0x55ff36c704f0 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ad225eb80 10 client.0 _ll_register_callbacks cb 0x55ff36b65ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54212/0 (socket says 172.21.15.46:54212) 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 -- 172.21.15.46:0/183273169 learned_addr learned my addr 172.21.15.46:0/183273169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab7400700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c71810 0x55ff36c813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 -- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 msgr2=0x55ff36c79e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c79e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 -- 172.21.15.46:0/183273169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c71810 msgr2=0x55ff36c813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c71810 0x55ff36c813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab6a00700 1 -- 172.21.15.46:0/183273169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab81a1700 con 0x55ff36c70e50 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.232+0000 7f9ab7400700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c79e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4099b60 tx=0x7f9aa409dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cf787d70a37e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40b6070 con 0x55ff36c70e50 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9aa409a2d0 con 0x55ff36c70e50 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff36b19220 con 0x55ff36c70e50 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa40b1070 con 0x55ff36c70e50 2024-05-08T23:03:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9ad225eb80 10 client.15420 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff369edb10 con 0x55ff36c70e50 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff36b19220 con 0x55ff36c70e50 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.233+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9aa40a1e50 con 0x55ff36c70e50 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 10 client.15420 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 20 client.15420 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36c84ca0 0x55ff36c89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 4 client.15420 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff36c3f7c0 con 0x55ff36c84ca0 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.234+0000 7f9ab7400700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36c84ca0 0x55ff36c89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.235+0000 7f9ab7400700 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36c84ca0 0x55ff36c89f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af9e54f8c944464 server_cookie=821111f95523edef in_seq=0 out_seq=0 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.235+0000 7f9aa1600700 10 client.15420 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.235+0000 7f9aa1600700 1 -- 172.21.15.46:0/183273169 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff36c3f7c0 con 0x55ff36c84ca0 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.235+0000 7f9aa1600700 10 client.15420 handle_command_reply: tid=0 2024-05-08T23:03:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.235+0000 7f9ad225eb80 1 client.15420 shutdown 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 msgr2=0x55ff36c7a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4099b60 tx=0x7f9aa409dc90 comp rx=0 tx=0).stop 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 shutdown_connections 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36c84ca0 0x55ff36c89f20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c71810 0x55ff36c813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c79e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9aa1600700 0 client.15420 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/183273169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c70e50 0x55ff36c7a360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 >> 172.21.15.46:0/183273169 conn(0x55ff36bce1c0 msgr2=0x55ff36c6a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 shutdown_connections 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 1 -- 172.21.15.46:0/183273169 wait complete. 2024-05-08T23:03:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.236+0000 7f9ad225eb80 20 client.15420 trim_cache size 0 max 16384 2024-05-08T23:03:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 Processor -- start 2024-05-08T23:03:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 -- start start 2024-05-08T23:03:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c70e50 0x55ff36c812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c79330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36bb9150 con 0x55ff36c71810 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36bbd010 con 0x55ff36c704f0 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36c7ad20 con 0x55ff36c70e50 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.255+0000 7f9ab7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50438/0 (socket says 172.21.15.46:50438) 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c70e50 0x55ff36c812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 -- 172.21.15.46:0/3008064380 learned_addr learned my addr 172.21.15.46:0/3008064380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c79330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 -- 172.21.15.46:0/3008064380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c70e50 msgr2=0x55ff36c812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c70e50 0x55ff36c812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 -- 172.21.15.46:0/3008064380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 msgr2=0x55ff36c79330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c79330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 -- 172.21.15.46:0/3008064380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff36b86510 con 0x55ff36c704f0 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c79330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.256+0000 7f9ab7400700 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bbb0 tx=0x7f9aa8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c8871eeb1dfc02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9aad600700 1 -- 172.21.15.46:0/3008064380 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8025070 con 0x55ff36c704f0 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9aad600700 1 -- 172.21.15.46:0/3008064380 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa8026040 con 0x55ff36c704f0 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9aad600700 1 -- 172.21.15.46:0/3008064380 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8029a90 con 0x55ff36c704f0 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3008064380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 msgr2=0x55ff36c78a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800bbb0 tx=0x7f9aa8015000 comp rx=0 tx=0).stop 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3008064380 shutdown_connections 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c79330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c704f0 0x55ff36c78a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3008064380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c70e50 0x55ff36c812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3008064380 >> 172.21.15.46:0/3008064380 conn(0x55ff36bce1c0 msgr2=0x55ff36c71bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3008064380 shutdown_connections 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.257+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3008064380 wait complete. 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 -- start start 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 Processor -- start 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c7baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36c81000 con 0x55ff36c71810 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36c81170 con 0x55ff36c70e50 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff36c81330 con 0x55ff36c704f0 2024-05-08T23:03:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.258+0000 7f9ad225eb80 10 client.0 _ll_register_callbacks cb 0x55ff36b65ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c7baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54234/0 (socket says 172.21.15.46:54234) 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 -- 172.21.15.46:0/3268668164 learned_addr learned my addr 172.21.15.46:0/3268668164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 -- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 msgr2=0x55ff36c7b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c7b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 -- 172.21.15.46:0/3268668164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 msgr2=0x55ff36c7baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c7baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 -- 172.21.15.46:0/3268668164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab8109be0 con 0x55ff36c71810 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab6a00700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c7baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9ab7e00700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f90 tx=0x7f9ab0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3b4cefeb85cc42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.259+0000 7f9aa1600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000e040 con 0x55ff36c71810 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9ab000bb90 con 0x55ff36c71810 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9aa4099e60 con 0x55ff36c71810 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab0007720 con 0x55ff36c71810 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 10 client.15426 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff36a45c80 con 0x55ff36c71810 2024-05-08T23:03:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa4099e60 con 0x55ff36c71810 2024-05-08T23:03:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9ab000b050 con 0x55ff36c71810 2024-05-08T23:03:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 10 client.15426 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:03:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 20 client.15426 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36ce03b0 0x55ff36cdcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.260+0000 7f9ad225eb80 4 client.15426 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.261+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff36bb8e70 con 0x55ff36ce03b0 2024-05-08T23:03:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.261+0000 7f9ab7400700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36ce03b0 0x55ff36cdcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.261+0000 7f9ab7400700 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36ce03b0 0x55ff36cdcd40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35e9057d532b8960 server_cookie=e654d68a7e52a1ef in_seq=0 out_seq=0 2024-05-08T23:03:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.261+0000 7f9aa1600700 10 client.15426 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9aa1600700 1 -- 172.21.15.46:0/3268668164 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ff36bb8e70 con 0x55ff36ce03b0 2024-05-08T23:03:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9aa1600700 10 client.15426 handle_command_reply: tid=0 2024-05-08T23:03:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 client.15426 shutdown 2024-05-08T23:03:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 msgr2=0x55ff36c80ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f90 tx=0x7f9ab0003c30 comp rx=0 tx=0).stop 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 shutdown_connections 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55ff36ce03b0 0x55ff36cdcd40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff36c71810 0x55ff36c80ac0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff36c704f0 0x55ff36c7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9aa1600700 0 client.15426 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 --2- 172.21.15.46:0/3268668164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff36c70e50 0x55ff36c7baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 >> 172.21.15.46:0/3268668164 conn(0x55ff36bce1c0 msgr2=0x55ff36c74820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 shutdown_connections 2024-05-08T23:03:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 1 -- 172.21.15.46:0/3268668164 wait complete. 2024-05-08T23:03:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.262+0000 7f9ad225eb80 20 client.15426 trim_cache size 0 max 16384 2024-05-08T23:03:37.272 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24547, 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:37.273 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 80.616546835999998, 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 86, 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 123.282428944 2024-05-08T23:03:37.274 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:37.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 msgr2=0x7f9a9c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 0x7f9a9c0a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8005950 tx=0x7f9aa80106b0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 msgr2=0x7f9ab8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400e9c0 tx=0x7f9aa400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 shutdown_connections 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab8102990 0x7f9ab80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab8100e80 0x7f9ab8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f9a9c0a11b0 0x7f9a9c0a3670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 --2- 172.21.15.46:0/1647024892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ab8101fd0 0x7f9ab8073390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 >> 172.21.15.46:0/1647024892 conn(0x7f9ab80fc3d0 msgr2=0x7f9ab8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 shutdown_connections 2024-05-08T23:03:37.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.263+0000 7f9abe800700 1 -- 172.21.15.46:0/1647024892 wait complete. 2024-05-08T23:03:37.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.480+0000 7f5240000700 1 -- 172.21.15.46:0/2882444383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238100e80 msgr2=0x7f52381012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.480+0000 7f5240000700 1 --2- 172.21.15.46:0/2882444383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238100e80 0x7f52381012a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f522c009990 tx=0x7f522c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 -- 172.21.15.46:0/2882444383 shutdown_connections 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 --2- 172.21.15.46:0/2882444383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238102990 0x7f52381092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 --2- 172.21.15.46:0/2882444383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238100e80 0x7f52381012a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 --2- 172.21.15.46:0/2882444383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238101fd0 0x7f5238102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 -- 172.21.15.46:0/2882444383 >> 172.21.15.46:0/2882444383 conn(0x7f52380fc3b0 msgr2=0x7f52380fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 -- 172.21.15.46:0/2882444383 shutdown_connections 2024-05-08T23:03:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.481+0000 7f5240000700 1 -- 172.21.15.46:0/2882444383 wait complete. 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 Processor -- start 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 -- start start 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238101fd0 0x7f5238199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 0x7f52381a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a0ba0 con 0x7f5238102990 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a0ce0 con 0x7f5238101fd0 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f5240000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a0e70 con 0x7f5238100e80 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f523e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 0x7f52381a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53752/0 (socket says 172.21.15.46:53752) 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.482+0000 7f523d600700 1 -- 172.21.15.46:0/413061659 learned_addr learned my addr 172.21.15.46:0/413061659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523cc00700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238101fd0 0x7f5238199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 -- 172.21.15.46:0/413061659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238101fd0 msgr2=0x7f5238199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238101fd0 0x7f5238199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 -- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 msgr2=0x7f52381a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 0x7f52381a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 -- 172.21.15.46:0/413061659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0095f0 con 0x7f5238100e80 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523e000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 0x7f52381a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f523d600700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b620 tx=0x7f522c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f5236000700 1 -- 172.21.15.46:0/413061659 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c01d070 con 0x7f5238100e80 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f5236000700 1 -- 172.21.15.46:0/413061659 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522c022470 con 0x7f5238100e80 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.483+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52381a10f0 con 0x7f5238100e80 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.484+0000 7f5236000700 1 -- 172.21.15.46:0/413061659 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c017650 con 0x7f5238100e80 2024-05-08T23:03:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.484+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381a1510 con 0x7f5238100e80 2024-05-08T23:03:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.487+0000 7f5236000700 1 -- 172.21.15.46:0/413061659 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f522c0225e0 con 0x7f5238100e80 2024-05-08T23:03:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 Processor -- start 2024-05-08T23:03:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 -- start start 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc27014910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc270152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f6b100 con 0x55dc270144f0 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f68ac0 con 0x55dc27014e50 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5253affb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f643b0 con 0x55dc27015810 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.488+0000 7f5236000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 0x7f52200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc270152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc27014910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53768/0 (socket says 172.21.15.46:53768) 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 -- 172.21.15.46:0/4243045312 learned_addr learned my addr 172.21.15.46:0/4243045312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f5236000700 1 -- 172.21.15.46:0/413061659 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f522c0ce0f0 con 0x7f5238100e80 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523e000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 0x7f52200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 -- 172.21.15.46:0/4243045312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 msgr2=0x55dc270152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc270152d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 -- 172.21.15.46:0/4243045312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 msgr2=0x55dc27014910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc27014910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 -- 172.21.15.46:0/4243045312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc26de5360 con 0x55dc27015810 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523d600700 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc270152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523cc00700 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5224009db0 tx=0x7f5224011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818f4c7b47915cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.489+0000 7f523e000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 0x7f52200a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ec0 tx=0x7f523000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f522a600700 1 -- 172.21.15.46:0/4243045312 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224004230 con 0x55dc27015810 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f522a600700 1 -- 172.21.15.46:0/4243045312 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5224004390 con 0x55dc27015810 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f522a600700 1 -- 172.21.15.46:0/4243045312 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240198c0 con 0x55dc27015810 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 -- 172.21.15.46:0/4243045312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 msgr2=0x55dc27015c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5224009db0 tx=0x7f5224011720 comp rx=0 tx=0).stop 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 -- 172.21.15.46:0/4243045312 shutdown_connections 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27015810 0x55dc27015c90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc27014910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 --2- 172.21.15.46:0/4243045312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc270152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 -- 172.21.15.46:0/4243045312 >> 172.21.15.46:0/4243045312 conn(0x55dc26f721c0 msgr2=0x55dc2700ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 -- 172.21.15.46:0/4243045312 shutdown_connections 2024-05-08T23:03:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.490+0000 7f5253affb80 1 -- 172.21.15.46:0/4243045312 wait complete. 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.491+0000 7f5253affb80 1 -- start start 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.491+0000 7f5253affb80 1 Processor -- start 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27025360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f00510 con 0x55dc27015810 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f001b0 con 0x55dc27014e50 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26de7650 con 0x55dc270144f0 2024-05-08T23:03:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f5253affb80 10 client.0 _ll_register_callbacks cb 0x55dc26f09ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27025360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53782/0 (socket says 172.21.15.46:53782) 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 -- 172.21.15.46:0/1659249014 learned_addr learned my addr 172.21.15.46:0/1659249014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523cc00700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 -- 172.21.15.46:0/1659249014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 msgr2=0x55dc2701e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 -- 172.21.15.46:0/1659249014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 msgr2=0x55dc27025360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27025360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.492+0000 7f523d600700 1 -- 172.21.15.46:0/1659249014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381a1510 con 0x55dc270144f0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f523d600700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f522c022890 tx=0x7f522c06a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260715f51e496220 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f521ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c09b070 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc26ebd220 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f522c09eea0 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f522c0a3460 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f5253affb80 10 client.25444 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f52240099e0 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.493+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc26ebd220 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f522c066670 con 0x55dc270144f0 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 10 client.25444 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 10 client.25444 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:03:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 20 client.25444 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27028c50 0x55dc2702df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 4 client.25444 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc26fe37c0 con 0x55dc27028c50 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.494+0000 7f523cc00700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27028c50 0x55dc2702df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.495+0000 7f523cc00700 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27028c50 0x55dc2702df50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2fb868fd2b730d server_cookie=efe08ef540174c3 in_seq=0 out_seq=0 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.495+0000 7f521ea00700 10 client.25444 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.495+0000 7f521ea00700 1 -- 172.21.15.46:0/1659249014 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc26fe37c0 con 0x55dc27028c50 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.495+0000 7f521ea00700 10 client.25444 handle_command_reply: tid=0 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.495+0000 7f5253affb80 1 client.25444 shutdown 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 msgr2=0x55dc2701dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f522c022890 tx=0x7f522c06a380 comp rx=0 tx=0).stop 2024-05-08T23:03:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 shutdown_connections 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27028c50 0x55dc2702df50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27025360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701dde0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f521ea00700 0 client.25444 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 --2- 172.21.15.46:0/1659249014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 >> 172.21.15.46:0/1659249014 conn(0x55dc26f721c0 msgr2=0x55dc2700e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 shutdown_connections 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 1 -- 172.21.15.46:0/1659249014 wait complete. 2024-05-08T23:03:37.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.496+0000 7f5253affb80 20 client.25444 trim_cache size 0 max 16384 2024-05-08T23:03:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 Processor -- start 2024-05-08T23:03:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 -- start start 2024-05-08T23:03:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc2701d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27015810 0x55dc2701cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26ebd220 con 0x55dc270144f0 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f222b0 con 0x55dc27015810 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f5253affb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc26f61010 con 0x55dc27014e50 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f523cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f523d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc2701d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.515+0000 7f523cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53792/0 (socket says 172.21.15.46:53792) 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 -- 172.21.15.46:0/3298586304 learned_addr learned my addr 172.21.15.46:0/3298586304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523e000700 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27015810 0x55dc2701cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 -- 172.21.15.46:0/3298586304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27015810 msgr2=0x55dc2701cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27015810 0x55dc2701cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 -- 172.21.15.46:0/3298586304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 msgr2=0x55dc2701d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc2701d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 -- 172.21.15.46:0/3298586304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc26d91b10 con 0x55dc27014e50 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f523cc00700 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55dc2700fcf0 tx=0x7f52240058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c1a6ae4246868 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f522b000700 1 -- 172.21.15.46:0/3298586304 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240041c0 con 0x55dc27014e50 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f522b000700 1 -- 172.21.15.46:0/3298586304 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5224004500 con 0x55dc27014e50 2024-05-08T23:03:37.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.516+0000 7f522b000700 1 -- 172.21.15.46:0/3298586304 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522402b250 con 0x55dc27014e50 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 -- 172.21.15.46:0/3298586304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 msgr2=0x55dc27025320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55dc2700fcf0 tx=0x7f52240058e0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 -- 172.21.15.46:0/3298586304 shutdown_connections 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27015810 0x55dc2701cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc270144f0 0x55dc2701d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 --2- 172.21.15.46:0/3298586304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc27014e50 0x55dc27025320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 -- 172.21.15.46:0/3298586304 >> 172.21.15.46:0/3298586304 conn(0x55dc26f721c0 msgr2=0x55dc2701bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 -- 172.21.15.46:0/3298586304 shutdown_connections 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.517+0000 7f5253affb80 1 -- 172.21.15.46:0/3298586304 wait complete. 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 -- start start 2024-05-08T23:03:37.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 Processor -- start 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27024e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc270253b0 con 0x55dc27015810 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc27025520 con 0x55dc27014e50 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc270256e0 con 0x55dc270144f0 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f5253affb80 10 client.0 _ll_register_callbacks cb 0x55dc26f09ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f523d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53796/0 (socket says 172.21.15.46:53796) 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f523d600700 1 -- 172.21.15.46:0/3196340203 learned_addr learned my addr 172.21.15.46:0/3196340203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.518+0000 7f523e000700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27024e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523cc00700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 -- 172.21.15.46:0/3196340203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 msgr2=0x55dc2701fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 -- 172.21.15.46:0/3196340203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 msgr2=0x55dc27024e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27024e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 -- 172.21.15.46:0/3196340203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc26de5360 con 0x55dc270144f0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f523d600700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f522c097570 tx=0x7f522c0cb750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9739eb9cda51d90c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f521ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0973b0 con 0x55dc270144f0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f522c06a1c0 con 0x55dc270144f0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.519+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5224004390 con 0x55dc270144f0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f522c09eea0 con 0x55dc270144f0 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 10 client.25450 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc26f2a510 con 0x55dc270144f0 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5224004390 con 0x55dc270144f0 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f522c0b3cb0 con 0x55dc270144f0 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 10 client.25450 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 10 client.25450 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 20 client.25450 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27084420 0x55dc27080d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 4 client.25450 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dc26f5ce70 con 0x55dc27084420 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.520+0000 7f523cc00700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27084420 0x55dc27080d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f523cc00700 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27084420 0x55dc27080d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dbcd4e2598729c8 server_cookie=923add8ee46699e0 in_seq=0 out_seq=0 2024-05-08T23:03:37.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f521ea00700 10 client.25450 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f521ea00700 1 -- 172.21.15.46:0/3196340203 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55dc26f5ce70 con 0x55dc27084420 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f521ea00700 10 client.25450 handle_command_reply: tid=0 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 client.25450 shutdown 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 msgr2=0x55dc2701f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f522c097570 tx=0x7f522c0cb750 comp rx=0 tx=0).stop 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 shutdown_connections 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55dc27084420 0x55dc27080d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc27015810 0x55dc27024e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc270144f0 0x55dc2701f950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 --2- 172.21.15.46:0/3196340203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dc27014e50 0x55dc2701fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f521ea00700 0 client.25450 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 >> 172.21.15.46:0/3196340203 conn(0x55dc26f721c0 msgr2=0x55dc27031f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 shutdown_connections 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.521+0000 7f5253affb80 1 -- 172.21.15.46:0/3196340203 wait complete. 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5253affb80 20 client.25450 trim_cache size 0 max 16384 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24571, 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 81.882713777999996, 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 86, 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 119.807260011 2024-05-08T23:03:37.529 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 msgr2=0x7f52200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 0x7f52200a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ec0 tx=0x7f523000b040 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 msgr2=0x7f52381990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b620 tx=0x7f522c004730 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 shutdown_connections 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f52200a11b0 0x7f52200a3670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5238102990 0x7f52381a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5238100e80 0x7f52381990c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 --2- 172.21.15.46:0/413061659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5238101fd0 0x7f5238199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 >> 172.21.15.46:0/413061659 conn(0x7f52380fc3b0 msgr2=0x7f52380731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 shutdown_connections 2024-05-08T23:03:37.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.522+0000 7f5240000700 1 -- 172.21.15.46:0/413061659 wait complete. 2024-05-08T23:03:37.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-08T23:03:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.740+0000 7f228b400700 1 -- 172.21.15.46:0/1146898385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 msgr2=0x7f2284073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.740+0000 7f228b400700 1 --2- 172.21.15.46:0/1146898385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f2284073430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2278009990 tx=0x7f2278009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 -- 172.21.15.46:0/1146898385 shutdown_connections 2024-05-08T23:03:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 --2- 172.21.15.46:0/1146898385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 0x7f228410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 --2- 172.21.15.46:0/1146898385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f2284073430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 --2- 172.21.15.46:0/1146898385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f2284073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 -- 172.21.15.46:0/1146898385 >> 172.21.15.46:0/1146898385 conn(0x7f22840fc3b0 msgr2=0x7f22840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 -- 172.21.15.46:0/1146898385 shutdown_connections 2024-05-08T23:03:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.741+0000 7f228b400700 1 -- 172.21.15.46:0/1146898385 wait complete. 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 Processor -- start 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 -- start start 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 0x7f22841a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841a4f30 con 0x7f2284074fe0 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841a5070 con 0x7f228410d450 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.742+0000 7f228b400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841a5200 con 0x7f2284073a00 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2288a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54322/0 (socket says 172.21.15.46:54322) 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 -- 172.21.15.46:0/1525115265 learned_addr learned my addr 172.21.15.46:0/1525115265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2288a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53808/0 (socket says 172.21.15.46:53808) 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2289400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 0x7f22841a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 -- 172.21.15.46:0/1525115265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 msgr2=0x7f228419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 -- 172.21.15.46:0/1525115265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 msgr2=0x7f22841a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 0x7f22841a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2283e00700 1 -- 172.21.15.46:0/1525115265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780095f0 con 0x7f2284074fe0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.743+0000 7f2288a00700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f2283e00700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f227000e950 tx=0x7f227000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f2281600700 1 -- 172.21.15.46:0/1525115265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270009770 con 0x7f2284074fe0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f2281600700 1 -- 172.21.15.46:0/1525115265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227000c7e0 con 0x7f2284074fe0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22841a54e0 con 0x7f2284074fe0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f2281600700 1 -- 172.21.15.46:0/1525115265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270003cf0 con 0x7f2284074fe0 2024-05-08T23:03:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.744+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22841a59f0 con 0x7f2284074fe0 2024-05-08T23:03:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.747+0000 7f2281600700 1 -- 172.21.15.46:0/1525115265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f227000c950 con 0x7f2284074fe0 2024-05-08T23:03:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.748+0000 7f229eeb1b80 1 Processor -- start 2024-05-08T23:03:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.748+0000 7f229eeb1b80 1 -- start start 2024-05-08T23:03:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.748+0000 7f2281600700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 0x7f226c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d93071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2281600700 1 -- 172.21.15.46:0/1525115265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2270014070 con 0x7f2284074fe0 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93072c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92fc8100 con 0x558d930714f0 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92fc5ac0 con 0x558d93071e50 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92fc13b0 con 0x558d93072810 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2289400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 0x7f226c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2289400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2283e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2283e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50554/0 (socket says 172.21.15.46:50554) 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2283e00700 1 -- 172.21.15.46:0/4044726789 learned_addr learned my addr 172.21.15.46:0/4044726789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.749+0000 7f2288a00700 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d93071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 -- 172.21.15.46:0/4044726789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 msgr2=0x558d93072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 -- 172.21.15.46:0/4044726789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 msgr2=0x558d93071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d93071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 -- 172.21.15.46:0/4044726789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d92e42360 con 0x558d93071e50 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2289400700 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2289400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 0x7f226c0a36f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f227c004230 tx=0x7f227c003ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2283e00700 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f22700963e0 tx=0x7f22700998c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239575264426f6c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2275c00700 1 -- 172.21.15.46:0/4044726789 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700b1070 con 0x558d93071e50 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2275c00700 1 -- 172.21.15.46:0/4044726789 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2270099ee0 con 0x558d93071e50 2024-05-08T23:03:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.750+0000 7f2275c00700 1 -- 172.21.15.46:0/4044726789 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000cc30 con 0x558d93071e50 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- 172.21.15.46:0/4044726789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 msgr2=0x558d930722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f22700963e0 tx=0x7f22700998c0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- 172.21.15.46:0/4044726789 shutdown_connections 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d93071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/4044726789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d930722d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- 172.21.15.46:0/4044726789 >> 172.21.15.46:0/4044726789 conn(0x558d92fcf1c0 msgr2=0x558d9306bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- 172.21.15.46:0/4044726789 shutdown_connections 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- 172.21.15.46:0/4044726789 wait complete. 2024-05-08T23:03:37.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.751+0000 7f229eeb1b80 1 -- start start 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 Processor -- start 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d9307b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93082360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92f5d510 con 0x558d930714f0 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92f5d1b0 con 0x558d93071e50 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92e44650 con 0x558d93072810 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f229eeb1b80 10 client.0 _ll_register_callbacks cb 0x558d92f66ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f2288a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f2288a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54336/0 (socket says 172.21.15.46:54336) 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f2288a00700 1 -- 172.21.15.46:0/604664040 learned_addr learned my addr 172.21.15.46:0/604664040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f2289400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93082360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.752+0000 7f2283e00700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d9307b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 -- 172.21.15.46:0/604664040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 msgr2=0x558d9307b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d9307b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 -- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 msgr2=0x558d9307ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 -- 172.21.15.46:0/604664040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c007560 con 0x558d93072810 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2288a00700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f2289400700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93082360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f227c003fd0 tx=0x7f227c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b5c0cab632a302 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f226a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c009b20 con 0x558d93072810 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d92f1a220 con 0x558d93072810 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.753+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f227c021070 con 0x558d93072810 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f227c007e20 con 0x558d93072810 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 10 client.25462 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f22841a59f0 con 0x558d93072810 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d92f1a220 con 0x558d93072810 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f227c01a470 con 0x558d93072810 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 10 client.25462 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 10 client.25462 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 20 client.25462 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d93085fd0 0x558d9308b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 4 client.25462 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.754+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d930407c0 con 0x558d93085fd0 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.755+0000 7f2288a00700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d93085fd0 0x558d9308b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.755+0000 7f2288a00700 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d93085fd0 0x558d9308b2d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b4b19dc11f1cdb6 server_cookie=e3c09dab665fd02e in_seq=0 out_seq=0 2024-05-08T23:03:37.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.755+0000 7f226a000700 10 client.25462 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f226a000700 1 -- 172.21.15.46:0/604664040 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d930407c0 con 0x558d93085fd0 2024-05-08T23:03:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f226a000700 10 client.25462 handle_command_reply: tid=0 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 client.25462 shutdown 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 msgr2=0x558d93082360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93082360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f227c003fd0 tx=0x7f227c004b40 comp rx=0 tx=0).stop 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 shutdown_connections 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93072810 0x558d93082360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d93085fd0 0x558d9308b2d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/604664040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93071e50 0x558d9307b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 >> 172.21.15.46:0/604664040 conn(0x558d92fcf1c0 msgr2=0x558d9306b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f226a000700 0 client.25462 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 shutdown_connections 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 1 -- 172.21.15.46:0/604664040 wait complete. 2024-05-08T23:03:37.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.756+0000 7f229eeb1b80 20 client.25462 trim_cache size 0 max 16384 2024-05-08T23:03:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.775+0000 7f229eeb1b80 1 Processor -- start 2024-05-08T23:03:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.775+0000 7f229eeb1b80 1 -- start start 2024-05-08T23:03:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.775+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d930714f0 0x558d93082a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d93082320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92fba150 con 0x558d93072810 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92fbe010 con 0x558d930714f0 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9306cb40 con 0x558d93071e50 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54346/0 (socket says 172.21.15.46:54346) 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 -- 172.21.15.46:0/2642088931 learned_addr learned my addr 172.21.15.46:0/2642088931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2283e00700 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d93082320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2288a00700 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d930714f0 0x558d93082a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 -- 172.21.15.46:0/2642088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 msgr2=0x558d93082320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d93082320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 -- 172.21.15.46:0/2642088931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d930714f0 msgr2=0x558d93082a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d930714f0 0x558d93082a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 -- 172.21.15.46:0/2642088931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d92deeb10 con 0x558d93072810 2024-05-08T23:03:37.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.776+0000 7f2289400700 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f227c000f80 tx=0x7f227c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8522fe0c641fe793 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f2276600700 1 -- 172.21.15.46:0/2642088931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c01ba10 con 0x558d93072810 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f2276600700 1 -- 172.21.15.46:0/2642088931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227c00f040 con 0x558d93072810 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f2276600700 1 -- 172.21.15.46:0/2642088931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c009b60 con 0x558d93072810 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 -- 172.21.15.46:0/2642088931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 msgr2=0x558d93078de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f227c000f80 tx=0x7f227c004040 comp rx=0 tx=0).stop 2024-05-08T23:03:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 -- 172.21.15.46:0/2642088931 shutdown_connections 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d93072810 0x558d93078de0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d930714f0 0x558d93082a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/2642088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d93082320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.777+0000 7f229eeb1b80 1 -- 172.21.15.46:0/2642088931 >> 172.21.15.46:0/2642088931 conn(0x558d92fcf1c0 msgr2=0x558d930681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.778+0000 7f229eeb1b80 1 -- 172.21.15.46:0/2642088931 shutdown_connections 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.778+0000 7f229eeb1b80 1 -- 172.21.15.46:0/2642088931 wait complete. 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.778+0000 7f229eeb1b80 1 -- start start 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.778+0000 7f229eeb1b80 1 Processor -- start 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d9307ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93072810 0x558d930817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d93081d20 con 0x558d930714f0 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d93081e90 con 0x558d93072810 2024-05-08T23:03:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d93082000 con 0x558d93071e50 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f229eeb1b80 10 client.0 _ll_register_callbacks cb 0x558d92f66ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54358/0 (socket says 172.21.15.46:54358) 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 -- 172.21.15.46:0/1903745887 learned_addr learned my addr 172.21.15.46:0/1903745887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2289400700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93072810 0x558d930817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2283e00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d9307ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 -- 172.21.15.46:0/1903745887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 msgr2=0x558d9307ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d9307ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 -- 172.21.15.46:0/1903745887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93072810 msgr2=0x558d930817e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93072810 0x558d930817e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 -- 172.21.15.46:0/1903745887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22841a59f0 con 0x558d930714f0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f2288a00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f227800b680 tx=0x7f2278026410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adc012b5832fcdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.779+0000 7f226a000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227801d070 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2278026cb0 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2270099ee0 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2278026e70 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f229eeb1b80 10 client.15462 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558d92f87510 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2270099ee0 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.780+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22780174f0 con 0x558d930714f0 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 10 client.15462 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 20 client.15462 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d930e1460 0x558d930dde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 4 client.15462 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558d92fb9e70 con 0x558d930e1460 2024-05-08T23:03:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f2283e00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d930e1460 0x558d930dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f2283e00700 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d930e1460 0x558d930dde10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed8b85230f78e95 server_cookie=d7b07cfda90019b in_seq=0 out_seq=0 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.781+0000 7f226a000700 10 client.15462 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f226a000700 1 -- 172.21.15.46:0/1903745887 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558d92fb9e70 con 0x558d930e1460 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f226a000700 10 client.15462 handle_command_reply: tid=0 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 client.15462 shutdown 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 msgr2=0x558d9307c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f227800b680 tx=0x7f2278026410 comp rx=0 tx=0).stop 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 shutdown_connections 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d93072810 0x558d930817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x558d930e1460 0x558d930dde10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d930714f0 0x558d9307c4a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 --2- 172.21.15.46:0/1903745887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d93071e50 0x558d9307ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f226a000700 0 client.15462 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:03:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 >> 172.21.15.46:0/1903745887 conn(0x558d92fcf1c0 msgr2=0x558d93079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 shutdown_connections 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 1 -- 172.21.15.46:0/1903745887 wait complete. 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.782+0000 7f229eeb1b80 20 client.15462 trim_cache size 0 max 16384 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24583, 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 83.151838290000001, 2024-05-08T23:03:37.788 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:37.789 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 86, 2024-05-08T23:03:37.789 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:37.789 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 117.450453971 2024-05-08T23:03:37.789 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 msgr2=0x7f226c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 0x7f226c0a36f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f227c004230 tx=0x7f227c003ea0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 msgr2=0x7f228419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f227000e950 tx=0x7f227000ec60 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 shutdown_connections 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f228410d450 0x7f22841a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284073a00 0x7f228419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f226c0a1230 0x7f226c0a36f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 --2- 172.21.15.46:0/1525115265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2284074fe0 0x7f228419d940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 >> 172.21.15.46:0/1525115265 conn(0x7f22840fc3b0 msgr2=0x7f22841007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 shutdown_connections 2024-05-08T23:03:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:37.783+0000 7f228b400700 1 -- 172.21.15.46:0/1525115265 wait complete. 2024-05-08T23:03:37.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:37 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4151987647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:37.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:37 smithi046 ceph-mon[36898]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:37.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:03:37.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:37 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4151987647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:37.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:37 smithi086 ceph-mon[78610]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.000+0000 7ff7c8400700 1 -- 172.21.15.46:0/1469896849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 msgr2=0x7ff7c01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.000+0000 7ff7c8400700 1 --2- 172.21.15.46:0/1469896849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c01012a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009990 tx=0x7ff7b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 -- 172.21.15.46:0/1469896849 shutdown_connections 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 --2- 172.21.15.46:0/1469896849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 0x7ff7c01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 --2- 172.21.15.46:0/1469896849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c01012a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 --2- 172.21.15.46:0/1469896849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 0x7ff7c0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 -- 172.21.15.46:0/1469896849 >> 172.21.15.46:0/1469896849 conn(0x7ff7c00fc3b0 msgr2=0x7ff7c00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 -- 172.21.15.46:0/1469896849 shutdown_connections 2024-05-08T23:03:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.001+0000 7ff7c8400700 1 -- 172.21.15.46:0/1469896849 wait complete. 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 Processor -- start 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 -- start start 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 0x7ff7c0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 0x7ff7c01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a0b60 con 0x7ff7c0100e80 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a0ca0 con 0x7ff7c0101fd0 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a0e30 con 0x7ff7c0102990 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54372/0 (socket says 172.21.15.46:54372) 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2284581432 learned_addr learned my addr 172.21.15.46:0/2284581432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.002+0000 7ff7c5000700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 0x7ff7c0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c6400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 0x7ff7c01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2284581432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 msgr2=0x7ff7c01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 0x7ff7c01a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2284581432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 msgr2=0x7ff7c0199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 0x7ff7c0199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2284581432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40095f0 con 0x7ff7c0100e80 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4005910 tx=0x7ff7b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7be000700 1 -- 172.21.15.46:0/2284581432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b401d070 con 0x7ff7c0100e80 2024-05-08T23:03:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7be000700 1 -- 172.21.15.46:0/2284581432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b4021410 con 0x7ff7c0100e80 2024-05-08T23:03:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7be000700 1 -- 172.21.15.46:0/2284581432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4017650 con 0x7ff7c0100e80 2024-05-08T23:03:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.003+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c01a10b0 con 0x7ff7c0100e80 2024-05-08T23:03:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.004+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01a1480 con 0x7ff7c0100e80 2024-05-08T23:03:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.007+0000 7ff7be000700 1 -- 172.21.15.46:0/2284581432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff7b4021580 con 0x7ff7c0100e80 2024-05-08T23:03:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 Processor -- start 2024-05-08T23:03:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 -- start start 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7be000700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 0x7ff7a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062ea0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062df6100 con 0x563062ea0810 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062df3ac0 con 0x563062e9fe50 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.008+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062def3b0 con 0x563062e9f4f0 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7be000700 1 -- 172.21.15.46:0/2284581432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7ff7b40ce1f0 con 0x7ff7c0100e80 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53874/0 (socket says 172.21.15.46:53874) 2024-05-08T23:03:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 -- 172.21.15.46:0/1848795708 learned_addr learned my addr 172.21.15.46:0/1848795708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c6400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 0x7ff7a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5000700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c6400700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062ea0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 -- 172.21.15.46:0/1848795708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 msgr2=0x563062ea02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 -- 172.21.15.46:0/1848795708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 msgr2=0x563062ea0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062ea0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 -- 172.21.15.46:0/1848795708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563062c70360 con 0x563062e9f4f0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5000700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.009+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409ae70 tx=0x7ff7b409aad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebc83d199f6683 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7c6400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 0x7ff7a80a3670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x563062e9acf0 tx=0x7ff7b8009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7b2a00700 1 -- 172.21.15.46:0/1848795708 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40ce020 con 0x563062e9f4f0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7b2a00700 1 -- 172.21.15.46:0/1848795708 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b409b870 con 0x563062e9f4f0 2024-05-08T23:03:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7b2a00700 1 -- 172.21.15.46:0/1848795708 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b43d0 con 0x563062e9f4f0 2024-05-08T23:03:38.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:37 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4151987647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:38.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:37 smithi053 ceph-mon[77587]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/1848795708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 msgr2=0x563062e9f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409ae70 tx=0x7ff7b409aad0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/1848795708 shutdown_connections 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062ea0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9f4f0 0x563062e9f910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/1848795708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/1848795708 >> 172.21.15.46:0/1848795708 conn(0x563062dfd1c0 msgr2=0x563062e99d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.010+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/1848795708 shutdown_connections 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.011+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/1848795708 wait complete. 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.011+0000 7ff7dbf80b80 1 -- start start 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.011+0000 7ff7dbf80b80 1 Processor -- start 2024-05-08T23:03:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062ea0810 0x563062eaff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062d8b510 con 0x563062e9f4f0 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062d8b1b0 con 0x563062e9fe50 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062c72650 con 0x563062ea0810 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7dbf80b80 10 client.0 _ll_register_callbacks cb 0x563062d94ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54388/0 (socket says 172.21.15.46:54388) 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2897639648 learned_addr learned my addr 172.21.15.46:0/2897639648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c6400700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062ea0810 0x563062eaff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2897639648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062ea0810 msgr2=0x563062eaff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062ea0810 0x563062eaff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 msgr2=0x563062ea8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.012+0000 7ff7c5a00700 1 -- 172.21.15.46:0/2897639648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01a1480 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40b56b0 tx=0x7ff7b409b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c60646ca742c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40af1c0 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7ff7b409b170 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563062d48220 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b4021860 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7dbf80b80 10 client.15474 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563062c1cb10 con 0x563062e9f4f0 2024-05-08T23:03:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563062d48220 con 0x563062e9f4f0 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.013+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7b409a660 con 0x563062e9f4f0 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 10 client.15474 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 20 client.15474 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062eb3740 0x563062eb89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 4 client.15474 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563062e6e7c0 con 0x563062eb3740 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.014+0000 7ff7c5000700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062eb3740 0x563062eb89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.015+0000 7ff7c5000700 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062eb3740 0x563062eb89e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f35919caf01391d server_cookie=c072b77ed5121bf5 in_seq=0 out_seq=0 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.015+0000 7ff7a7400700 10 client.15474 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.015+0000 7ff7a7400700 1 -- 172.21.15.46:0/2897639648 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563062e6e7c0 con 0x563062eb3740 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.015+0000 7ff7a7400700 10 client.15474 handle_command_reply: tid=0 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 client.15474 shutdown 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 msgr2=0x563062ea89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40b56b0 tx=0x7ff7b409b070 comp rx=0 tx=0).stop 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 shutdown_connections 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062ea0810 0x563062eaff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062eb3740 0x563062eb89e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea89f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/2897639648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9fe50 0x563062ea8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7a7400700 0 client.15474 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 >> 172.21.15.46:0/2897639648 conn(0x563062dfd1c0 msgr2=0x563062e99550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 shutdown_connections 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/2897639648 wait complete. 2024-05-08T23:03:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.016+0000 7ff7dbf80b80 20 client.15474 trim_cache size 0 max 16384 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.035+0000 7ff7dbf80b80 1 Processor -- start 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.035+0000 7ff7dbf80b80 1 -- start start 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062ea0810 0x563062ea3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062eb4f90 con 0x563062e9f4f0 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062ea98f0 con 0x563062ea0810 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062e6e7c0 con 0x563062e9fe50 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54400/0 (socket says 172.21.15.46:54400) 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 -- 172.21.15.46:0/544748228 learned_addr learned my addr 172.21.15.46:0/544748228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5000700 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c6400700 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062ea0810 0x563062ea3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 -- 172.21.15.46:0/544748228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 msgr2=0x563062eaff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 -- 172.21.15.46:0/544748228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062ea0810 msgr2=0x563062ea3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062ea0810 0x563062ea3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.036+0000 7ff7c5a00700 1 -- 172.21.15.46:0/544748228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563062db5510 con 0x563062e9f4f0 2024-05-08T23:03:38.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cb7d0 tx=0x7ff7b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57330bb0d3be5435 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7b3400700 1 -- 172.21.15.46:0/544748228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b3e10 con 0x563062e9f4f0 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7b3400700 1 -- 172.21.15.46:0/544748228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b40a3560 con 0x563062e9f4f0 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7b3400700 1 -- 172.21.15.46:0/544748228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b409a660 con 0x563062e9f4f0 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/544748228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 msgr2=0x563062ea7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.037+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cb7d0 tx=0x7ff7b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/544748228 shutdown_connections 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062ea0810 0x563062ea3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062e9f4f0 0x563062ea7f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/544748228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/544748228 >> 172.21.15.46:0/544748228 conn(0x563062dfd1c0 msgr2=0x563062ea0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/544748228 shutdown_connections 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/544748228 wait complete. 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 -- start start 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.038+0000 7ff7dbf80b80 1 Processor -- start 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062eaff20 con 0x563062ea0810 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062eb0090 con 0x563062e9f4f0 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563062eb0250 con 0x563062e9fe50 2024-05-08T23:03:38.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7dbf80b80 10 client.0 _ll_register_callbacks cb 0x563062d94ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54416/0 (socket says 172.21.15.46:54416) 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c5a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50638/0 (socket says 172.21.15.46:50638) 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c6400700 1 -- 172.21.15.46:0/3291374147 learned_addr learned my addr 172.21.15.46:0/3291374147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.039+0000 7ff7c5000700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 -- 172.21.15.46:0/3291374147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 msgr2=0x563062eaf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 -- 172.21.15.46:0/3291374147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 msgr2=0x563062eaf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 -- 172.21.15.46:0/3291374147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563062c1cb10 con 0x563062e9f4f0 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c6400700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7c5a00700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006010 tx=0x7ff7b40975e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64603a9541ccdd46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7a7400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b3e10 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7ff7b409b170 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7b409b870 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b409a870 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7dbf80b80 10 client.25472 fetch_fsmap learned FSMap version 26 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.040+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563062c74c80 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b409b870 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7b4096f60 con 0x563062e9f4f0 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 10 client.25472 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 10 client.25472 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 20 client.25472 populate_metadata read hostname 'smithi046' 2024-05-08T23:03:38.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062f0e940 0x563062f0b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 4 client.25472 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563062de7e70 con 0x563062f0e940 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.041+0000 7ff7c5000700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062f0e940 0x563062f0b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7c5000700 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062f0e940 0x563062f0b710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69191c0e3a9eb88 server_cookie=84f499f99981828b in_seq=0 out_seq=0 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7a7400700 10 client.25472 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7a7400700 1 -- 172.21.15.46:0/3291374147 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x563062de7e70 con 0x563062f0e940 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7a7400700 10 client.25472 handle_command_reply: tid=0 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 client.25472 shutdown 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 msgr2=0x563062eacec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006010 tx=0x7ff7b40975e0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 shutdown_connections 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563062f0e940 0x563062f0b710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563062ea0810 0x563062eaf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563062e9f4f0 0x563062eacec0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 --2- 172.21.15.46:0/3291374147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563062e9fe50 0x563062eaf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7a7400700 0 client.25472 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 >> 172.21.15.46:0/3291374147 conn(0x563062dfd1c0 msgr2=0x563062ea7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 shutdown_connections 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.042+0000 7ff7dbf80b80 1 -- 172.21.15.46:0/3291374147 wait complete. 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7dbf80b80 20 client.25472 trim_cache size 0 max 16384 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ebe57712-0d8d-11ef-bc97-c7b262605968", 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24587, 2024-05-08T23:03:38.048 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 106.57904304, 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 86, 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 113.030001145 2024-05-08T23:03:38.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 msgr2=0x7ff7a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 0x7ff7a80a3670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x563062e9acf0 tx=0x7ff7b8009920 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 msgr2=0x7ff7c0199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4005910 tx=0x7ff7b4005740 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 shutdown_connections 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0102990 0x7ff7c01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7ff7a80a11b0 0x7ff7a80a3670 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0100e80 0x7ff7c0199030 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 --2- 172.21.15.46:0/2284581432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c0101fd0 0x7ff7c0199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 >> 172.21.15.46:0/2284581432 conn(0x7ff7c00fc3b0 msgr2=0x7ff7c00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.043+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 shutdown_connections 2024-05-08T23:03:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.044+0000 7ff7c8400700 1 -- 172.21.15.46:0/2284581432 wait complete. 2024-05-08T23:03:38.062 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:03:38.072 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:03:38.072 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:03:38.072 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:03:38.072 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T23:03:38.073 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:03:38.073 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:03:38.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 -- 172.21.15.46:0/4053923048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102040 msgr2=0x7f70981024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 --2- 172.21.15.46:0/4053923048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102040 0x7f70981024c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7084009990 tx=0x7f7084009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 -- 172.21.15.46:0/4053923048 shutdown_connections 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 --2- 172.21.15.46:0/4053923048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098102a00 0x7f7098109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 --2- 172.21.15.46:0/4053923048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098100e60 0x7f7098101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 --2- 172.21.15.46:0/4053923048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102040 0x7f70981024c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 -- 172.21.15.46:0/4053923048 >> 172.21.15.46:0/4053923048 conn(0x7f70980fc400 msgr2=0x7f70980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 -- 172.21.15.46:0/4053923048 shutdown_connections 2024-05-08T23:03:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.266+0000 7f70a0400700 1 -- 172.21.15.46:0/4053923048 wait complete. 2024-05-08T23:03:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.267+0000 7f70a0400700 1 Processor -- start 2024-05-08T23:03:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 -- start start 2024-05-08T23:03:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 0x7f7098074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 0x7f70980738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7098073fd0 con 0x7f7098102040 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7098074140 con 0x7f7098100e60 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f70a0400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981a0ee0 con 0x7f7098102a00 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f709d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f709da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 0x7f7098074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f709e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 0x7f70980738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f709d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54446/0 (socket says 172.21.15.46:54446) 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.268+0000 7f709d000700 1 -- 172.21.15.46:0/1081969518 learned_addr learned my addr 172.21.15.46:0/1081969518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 -- 172.21.15.46:0/1081969518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 msgr2=0x7f7098074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 0x7f7098074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 -- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 msgr2=0x7f7098073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 -- 172.21.15.46:0/1081969518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840095f0 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709da00700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 0x7f7098074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709d000700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f709e400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 0x7f70980738d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f709000efc0 tx=0x7f709000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090018070 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709000f040 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70981a10e0 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000cd40 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.269+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70981a15f0 con 0x7f7098102a00 2024-05-08T23:03:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.270+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7098108380 con 0x7f7098102a00 2024-05-08T23:03:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.272+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7090003a00 con 0x7f7098102a00 2024-05-08T23:03:38.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.274+0000 7f7096000700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 0x7f70800a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.274+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f70900cd360 con 0x7f7098102a00 2024-05-08T23:03:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.274+0000 7f709da00700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 0x7f70800a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.274+0000 7f709da00700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 0x7f70800a36e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7098101bb0 tx=0x7f708c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.276+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7090096620 con 0x7f7098102a00 2024-05-08T23:03:38.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.557+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f709810c8e0 con 0x7f7098102a00 2024-05-08T23:03:38.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.714+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f7090003cb0 con 0x7f7098102a00 2024-05-08T23:03:38.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.715+0000 7f7096000700 1 -- 172.21.15.46:0/1081969518 <== mon.1 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f709009a0b0 con 0x7f7098102a00 2024-05-08T23:03:38.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.719+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 msgr2=0x7f70800a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.719+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 0x7f70800a36e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7098101bb0 tx=0x7f708c009260 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.719+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 msgr2=0x7f70980738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.719+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 0x7f70980738d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f709000efc0 tx=0x7f709000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 shutdown_connections 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098102a00 0x7f70980738d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7098100e60 0x7f7098074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f70800a1220 0x7f70800a36e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 --2- 172.21.15.46:0/1081969518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7098102040 0x7f7098073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 >> 172.21.15.46:0/1081969518 conn(0x7f70980fc400 msgr2=0x7f70981054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 shutdown_connections 2024-05-08T23:03:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.720+0000 7f70a0400700 1 -- 172.21.15.46:0/1081969518 wait complete. 2024-05-08T23:03:38.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T23:03:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 -- 172.21.15.46:0/2480660742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 msgr2=0x7fdc381024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2480660742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc381024c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009990 tx=0x7fdc24009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 -- 172.21.15.46:0/2480660742 shutdown_connections 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2480660742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38102a00 0x7fdc38109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2480660742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38100e60 0x7fdc38101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2480660742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc381024c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 -- 172.21.15.46:0/2480660742 >> 172.21.15.46:0/2480660742 conn(0x7fdc380fc400 msgr2=0x7fdc380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 -- 172.21.15.46:0/2480660742 shutdown_connections 2024-05-08T23:03:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.946+0000 7fdc3d400700 1 -- 172.21.15.46:0/2480660742 wait complete. 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.947+0000 7fdc3d400700 1 Processor -- start 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.947+0000 7fdc3d400700 1 -- start start 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38100e60 0x7fdc38074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 0x7fdc380738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38074070 con 0x7fdc38100e60 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380741e0 con 0x7fdc38102a00 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc3d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381a0e80 con 0x7fdc38102040 2024-05-08T23:03:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc36800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38100e60 0x7fdc38074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc37200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 0x7fdc380738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53952/0 (socket says 172.21.15.46:53952) 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc37200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 0x7fdc380738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50662/0 (socket says 172.21.15.46:50662) 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 -- 172.21.15.46:0/2037240921 learned_addr learned my addr 172.21.15.46:0/2037240921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 msgr2=0x7fdc380738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 0x7fdc380738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38100e60 msgr2=0x7fdc38074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38100e60 0x7fdc38074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.948+0000 7fdc35e00700 1 -- 172.21.15.46:0/2037240921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240095f0 con 0x7fdc38102040 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc35e00700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000c00 tx=0x7fdc24009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2401d070 con 0x7fdc38102040 2024-05-08T23:03:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc24021410 con 0x7fdc38102040 2024-05-08T23:03:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc381a10b0 con 0x7fdc38102040 2024-05-08T23:03:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24017650 con 0x7fdc38102040 2024-05-08T23:03:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.949+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381a1470 con 0x7fdc38102040 2024-05-08T23:03:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.950+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc381083b0 con 0x7fdc38102040 2024-05-08T23:03:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.952+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7fdc24021580 con 0x7fdc38102040 2024-05-08T23:03:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.953+0000 7fdc2f400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 0x7fdc1c0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.953+0000 7fdc36800700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 msgr2=0x7fdc1c0a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.953+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fdc240ce170 con 0x7fdc38102040 2024-05-08T23:03:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.953+0000 7fdc36800700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 0x7fdc1c0a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:03:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:38.957+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc24097430 con 0x7fdc38102040 2024-05-08T23:03:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1081969518' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:38 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: ignoring --setuser ceph since I am not root 2024-05-08T23:03:39.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: ignoring --setgroup ceph since I am not root 2024-05-08T23:03:39.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:38 smithi053 ceph-mgr[79065]: -- 172.21.15.53:0/1026967295 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5601dd3481a0 con 0x5601dd31e800 2024-05-08T23:03:39.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1081969518' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:38 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setuser ceph since I am not root 2024-05-08T23:03:39.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: ignoring --setgroup ceph since I am not root 2024-05-08T23:03:39.144 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1081969518' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.144 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:38 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:03:39.144 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: ignoring --setuser ceph since I am not root 2024-05-08T23:03:39.144 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: ignoring --setgroup ceph since I am not root 2024-05-08T23:03:39.144 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:38 smithi086 ceph-mgr[80462]: -- 172.21.15.86:0/1091434775 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f8fd461a0 con 0x555f8fd1cc00 2024-05-08T23:03:39.145 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:39.143+0000 7fca969b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:03:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.154+0000 7fdc36800700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 msgr2=0x7fdc1c0a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.154+0000 7fdc36800700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 0x7fdc1c0a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:03:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.179+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fdc3804ec60 con 0x7fdc38102040 2024-05-08T23:03:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.185+0000 7fdc2f400700 1 -- 172.21.15.46:0/2037240921 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc24005740 con 0x7fdc38102040 2024-05-08T23:03:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 msgr2=0x7fdc1c0a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:03:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 0x7fdc1c0a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 msgr2=0x7fdc38073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000c00 tx=0x7fdc24009580 comp rx=0 tx=0).stop 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 shutdown_connections 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc38102a00 0x7fdc380738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fdc1c0a11f0 0x7fdc1c0a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc38100e60 0x7fdc38074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 --2- 172.21.15.46:0/2037240921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc38102040 0x7fdc38073390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 >> 172.21.15.46:0/2037240921 conn(0x7fdc380fc400 msgr2=0x7fdc381054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 shutdown_connections 2024-05-08T23:03:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.187+0000 7fdc3d400700 1 -- 172.21.15.46:0/2037240921 wait complete. 2024-05-08T23:03:39.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T23:03:39.407 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:39.237+0000 7fca969b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:03:39.407 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:39.323+0000 7fca969b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.409+0000 7fddd6000700 1 -- 172.21.15.46:0/3293214338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0073a00 msgr2=0x7fddd0073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.409+0000 7fddd6000700 1 --2- 172.21.15.46:0/3293214338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0073a00 0x7fddd0073e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009990 tx=0x7fddbc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 -- 172.21.15.46:0/3293214338 shutdown_connections 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 --2- 172.21.15.46:0/3293214338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0108d70 0x7fddd010b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 --2- 172.21.15.46:0/3293214338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 0x7fddd0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 --2- 172.21.15.46:0/3293214338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0073a00 0x7fddd0073e80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 -- 172.21.15.46:0/3293214338 >> 172.21.15.46:0/3293214338 conn(0x7fddd00fc400 msgr2=0x7fddd00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 -- 172.21.15.46:0/3293214338 shutdown_connections 2024-05-08T23:03:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.410+0000 7fddd6000700 1 -- 172.21.15.46:0/3293214338 wait complete. 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 Processor -- start 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 -- start start 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 0x7fddd01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0108d70 0x7fddd01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a0b80 con 0x7fddd0074fe0 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a0cc0 con 0x7fddd0108d70 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddd6000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a0e50 con 0x7fddd0073a00 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddcf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddcf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53962/0 (socket says 172.21.15.46:53962) 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddcf400700 1 -- 172.21.15.46:0/3409588190 learned_addr learned my addr 172.21.15.46:0/3409588190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.411+0000 7fddcea00700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 0x7fddd01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcfe00700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0108d70 0x7fddd01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0108d70 msgr2=0x7fddd01a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0108d70 0x7fddd01a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 msgr2=0x7fddd01995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 0x7fddd01995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 -- 172.21.15.46:0/3409588190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc8009c40 con 0x7fddd0073a00 2024-05-08T23:03:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddcf400700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9a0 tx=0x7fddc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0009790 con 0x7fddd0073a00 2024-05-08T23:03:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddc000ce80 con 0x7fddd0073a00 2024-05-08T23:03:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc0095f0 con 0x7fddd0073a00 2024-05-08T23:03:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.412+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0018680 con 0x7fddd0073a00 2024-05-08T23:03:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.413+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd01a14b0 con 0x7fddd0073a00 2024-05-08T23:03:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.413+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0107b30 con 0x7fddd0073a00 2024-05-08T23:03:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.415+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7fddc0010430 con 0x7fddd0073a00 2024-05-08T23:03:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.416+0000 7fddc7e00700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 0x7fddb40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.417+0000 7fddcea00700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 msgr2=0x7fddb40a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.417+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fddc0014070 con 0x7fddd0073a00 2024-05-08T23:03:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.417+0000 7fddcea00700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 0x7fddb40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:03:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.420+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fddc0052d60 con 0x7fddd0073a00 2024-05-08T23:03:39.481 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:39.352+0000 7fcffc0a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:03:39.510 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:39.209+0000 7fe18e080200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:03:39.510 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:39.315+0000 7fe18e080200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:03:39.510 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:39.414+0000 7fe18e080200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:03:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.618+0000 7fddcea00700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 msgr2=0x7fddb40a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.618+0000 7fddcea00700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 0x7fddb40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:03:39.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.636+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fddd004ec60 con 0x7fddd0073a00 2024-05-08T23:03:39.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.644+0000 7fddc7e00700 1 -- 172.21.15.46:0/3409588190 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fddc0099d90 con 0x7fddd0073a00 2024-05-08T23:03:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 msgr2=0x7fddb40a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:03:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 0x7fddb40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 msgr2=0x7fddd01990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9a0 tx=0x7fddc000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 shutdown_connections 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0108d70 0x7fddd01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd0073a00 0x7fddd01990a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7fddb40a11f0 0x7fddb40a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 --2- 172.21.15.46:0/3409588190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd0074fe0 0x7fddd01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 >> 172.21.15.46:0/3409588190 conn(0x7fddd00fc400 msgr2=0x7fddd0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 shutdown_connections 2024-05-08T23:03:39.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:39.647+0000 7fddd6000700 1 -- 172.21.15.46:0/3409588190 wait complete. 2024-05-08T23:03:39.667 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T23:03:39.681 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T23:03:39.681 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T23:03:39.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:39.682 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-08T23:03:39.713 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:39.407+0000 7fca969b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:03:39.724 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T23:03:39.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:39.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T23:03:39.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:39.510+0000 7fe18e080200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:03:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:39 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:03:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:39 smithi053 ceph-mon[77587]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T23:03:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:39 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' 2024-05-08T23:03:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:39 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' 2024-05-08T23:03:39.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:39 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:03:39.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:39 smithi046 ceph-mon[36898]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T23:03:39.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:39 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' 2024-05-08T23:03:39.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:39 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' 2024-05-08T23:03:39.777 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:39.481+0000 7fcffc0a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:03:39.777 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:39.598+0000 7fcffc0a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:03:39.777 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:39.720+0000 7fcffc0a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:03:39.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:39.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-08T23:03:39.967 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-08T23:03:39.967 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:03:39.967 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:03:39.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T23:03:40.035 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:39 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:03:40.035 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:39 smithi086 ceph-mon[78610]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T23:03:40.035 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:39 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' 2024-05-08T23:03:40.035 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:39 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' 2024-05-08T23:03:40.287 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.035+0000 7fca969b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:03:40.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:40.238+0000 7fe18e080200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:40.639 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.287+0000 7fca969b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:03:40.639 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.372+0000 7fca969b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:03:40.639 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.481+0000 7fca969b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:40.912 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:40.655+0000 7fcffc0a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:40.931 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:40.526+0000 7fe18e080200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:03:40.931 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:40.622+0000 7fe18e080200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:03:40.931 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:40.747+0000 7fe18e080200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:40.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.639+0000 7fca969b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:03:40.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.768+0000 7fca969b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:03:40.964 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:40.853+0000 7fca969b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:03:41.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:40.930+0000 7fe18e080200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:03:41.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:41.076+0000 7fe18e080200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:03:41.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:41.173+0000 7fe18e080200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:03:41.268 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:40.995+0000 7fcffc0a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:03:41.268 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:41.112+0000 7fcffc0a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:03:41.527 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:41.268+0000 7fcffc0a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:41.527 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:41.486+0000 7fcffc0a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:03:41.605 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:41.219+0000 7fca969b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 -- 172.21.15.46:0/3982857199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 msgr2=0x7f79e410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 --2- 172.21.15.46:0/3982857199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e410de20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00b0a0 tx=0x7f79dc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 -- 172.21.15.46:0/3982857199 shutdown_connections 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 --2- 172.21.15.46:0/3982857199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e410de20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 --2- 172.21.15.46:0/3982857199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 --2- 172.21.15.46:0/3982857199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 0x7f79e4105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 -- 172.21.15.46:0/3982857199 >> 172.21.15.46:0/3982857199 conn(0x7f79e40fc9c0 msgr2=0x7f79e40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 -- 172.21.15.46:0/3982857199 shutdown_connections 2024-05-08T23:03:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.667+0000 7f79eb600700 1 -- 172.21.15.46:0/3982857199 wait complete. 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 Processor -- start 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 -- start start 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 0x7f79e41083e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e4108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e41a9d10 con 0x7f79e4101b20 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e41a9e50 con 0x7f79e4106250 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.668+0000 7f79eb600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e41a9f90 con 0x7f79e4101170 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:41.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 0x7f79e41083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53982/0 (socket says 172.21.15.46:53982) 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 -- 172.21.15.46:0/182066422 learned_addr learned my addr 172.21.15.46:0/182066422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e9600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e4108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 msgr2=0x7f79e4108920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e4108920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 msgr2=0x7f79e41083e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 0x7f79e41083e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 -- 172.21.15.46:0/182066422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009d00 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.669+0000 7f79e8c00700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9a0 tx=0x7f79d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.670+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8009790 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.670+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d8010430 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.670+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e41aa270 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.670+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8018680 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.670+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e41aa7c0 con 0x7f79e4101170 2024-05-08T23:03:41.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.671+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e4102650 con 0x7f79e4101170 2024-05-08T23:03:41.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.674+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f79d80105a0 con 0x7f79e4101170 2024-05-08T23:03:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.675+0000 7f79e1600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 0x7f79cc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.675+0000 7f79e3e00700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 msgr2=0x7f79cc0a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.675+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f79d8014070 con 0x7f79e4101170 2024-05-08T23:03:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.675+0000 7f79e3e00700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 0x7f79cc0a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:03:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.679+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f79d8052d60 con 0x7f79e4101170 2024-05-08T23:03:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.876+0000 7f79e3e00700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 msgr2=0x7f79cc0a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2731582335 2024-05-08T23:03:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.876+0000 7f79e3e00700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 0x7f79cc0a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:03:41.893 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:41.605+0000 7fca969b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:03:41.893 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:41.800+0000 7fca969b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:03:41.893 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:41.893+0000 7fca969b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:03:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.941+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79e404ec60 con 0x7f79e4101170 2024-05-08T23:03:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.941+0000 7f79e1600700 1 -- 172.21.15.46:0/182066422 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+88 (secure 0 0 0) 0x7f79d8096240 con 0x7f79e4101170 2024-05-08T23:03:41.942 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T23:03:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 msgr2=0x7f79cc0a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:03:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 0x7f79cc0a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 msgr2=0x7f79e4107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9a0 tx=0x7f79d800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 shutdown_connections 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e4106250 0x7f79e4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e4101170 0x7f79e4107ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:6800/2731582335,v1:172.21.15.46:6801/2731582335] conn(0x7f79cc0a1280 0x7f79cc0a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 --2- 172.21.15.46:0/182066422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79e4101b20 0x7f79e41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 >> 172.21.15.46:0/182066422 conn(0x7f79e40fc9c0 msgr2=0x7f79e40fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 shutdown_connections 2024-05-08T23:03:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:41.944+0000 7f79eb600700 1 -- 172.21.15.46:0/182066422 wait complete. 2024-05-08T23:03:41.968 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:41.669+0000 7fcffc0a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:03:41.968 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:41.790+0000 7fcffc0a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:03:41.993 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:41.598+0000 7fe18e080200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:03:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:41 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/182066422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:03:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:41 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/182066422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:03:42.261 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:42.046+0000 7fe18e080200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:03:42.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:41 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/182066422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:03:42.358 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:03:42.358 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:03:42.358 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-05-08T23:03:42.358 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:03:42.358 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:03:42.359 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:03:42.359 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:03:42.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:03:42.359 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:03:42.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.359 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-05-08T23:03:42.403 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:03:42.403 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:03:42.403 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:03:42.403 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:03:42.404 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41596sec preferred_lft 41596sec 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:03:42.405 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-05-08T23:03:42.406 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:03:42.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.406 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:03:42.406 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:03:42.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:03:42.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-05-08T23:03:42.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-05-08T23:03:42.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:03:42.407 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:03:42.550 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:42.294+0000 7fcffc0a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:03:42.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.656 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:03:42.725 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-05-08T23:03:42.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.728 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-05-08T23:03:42.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:42.267+0000 7fe18e080200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:03:42.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:42.373+0000 7fe18e080200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:03:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-05-08T23:03:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:03:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:03:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:03:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:03:42.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:03:42.776 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:03:42.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.950 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-08T23:03:42.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:42.653+0000 7fca969b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:03:42.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:42.751+0000 7fca969b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:03:42.963 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:42.898+0000 7fca969b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:03:42.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:42.995 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-05-08T23:03:43.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:43.079 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:03:43.079 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:03:43.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:03:43.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:03:43.079 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:03:43.125 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:42.847+0000 7fcffc0a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:03:43.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:03:43.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:03:43.259 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:03:43.397 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:43.125+0000 7fcffc0a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:03:43.397 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:43.267+0000 7fcffc0a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:03:43.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:43.254+0000 7fe18e080200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:03:43.511 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:43.364+0000 7fe18e080200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:03:43.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:43.552 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-05-08T23:03:43.552 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-08T23:03:43.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-05-08T23:03:43.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:03:43.552 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-08T23:03:43.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:03:43.723 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:03:43.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:03:43.867 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:03:43.899 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:43.525+0000 7fca969b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:03:43.899 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:43.708+0000 7fca969b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:03:43.909 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T23:03:43.909 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T23:03:43.910 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/d6af9f64-4946-4177-ac36-39bf5e3d4b15 2024-05-08T23:03:43.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:43.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/d6af9f64-4946-4177-ac36-39bf5e3d4b15 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T23:03:44.011 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:43.538+0000 7fe18e080200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T23:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T23:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-08T23:03:44.673 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:44.331+0000 7fcffc0a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:03:44.673 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:44.468+0000 7fcffc0a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:03:44.694 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:44.263+0000 7fe18e080200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:03:44.694 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:44.472+0000 7fe18e080200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:03:44.809 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T23:03:44.809 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T23:03:44.810 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-08T23:03:44.810 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.53:6789,172.21.15.86:6789" 2024-05-08T23:03:44.810 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.53:6789,172.21.15.86:6789:/volumes/qa/sv_1/d6af9f64-4946-4177-ac36-39bf5e3d4b15 2024-05-08T23:03:44.810 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=ebe57712-0d8d-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T23:03:44.816 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:03:44.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:44.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:03:45.011 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:44.694+0000 7fe18e080200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:03:45.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:44.672+0000 7fcffc0a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:03:45.108 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:43.900+0000 7fca969b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:03:45.108 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:44.822+0000 7fca969b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:03:45.386 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:45.108+0000 7fca969b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:45.386 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:45.196+0000 7fca969b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:45.713 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:03:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z[80439]: 2024-05-08T23:03:45.386+0000 7fca969b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:03:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: Standby manager daemon z restarted 2024-05-08T23:03:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: Standby manager daemon z started 2024-05-08T23:03:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:03:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:03:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:45 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: Standby manager daemon z restarted 2024-05-08T23:03:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: Standby manager daemon z started 2024-05-08T23:03:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:03:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:03:45.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:45 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:45.776 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:45.537+0000 7fcffc0a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: Standby manager daemon z restarted 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: Standby manager daemon z started 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:03:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:45 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.86:0/1137095481' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:46.061 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:45.790+0000 7fcffc0a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:03:46.097 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:45.771+0000 7fe18e080200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:03:46.417 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:46.097+0000 7fe18e080200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:46.417 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:46.200+0000 7fe18e080200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:46.464 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:46.061+0000 7fcffc0a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:03:46.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:03:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y[79042]: 2024-05-08T23:03:46.417+0000 7fe18e080200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:03:46.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T23:03:46.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: Standby manager daemon y restarted 2024-05-08T23:03:46.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: Standby manager daemon y started 2024-05-08T23:03:46.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:03:46.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:46.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:03:46.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:46 smithi053 ceph-mon[77587]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:46.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T23:03:46.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: Standby manager daemon y restarted 2024-05-08T23:03:46.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: Standby manager daemon y started 2024-05-08T23:03:46.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:03:46.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:46.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:03:46.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:46 smithi046 ceph-mon[36898]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: mgrmap e19: x(active, since 7m), standbys: y, z 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: Standby manager daemon y restarted 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: Standby manager daemon y started 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:03:46.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:03:46.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:46 smithi086 ceph-mon[78610]: from='mgr.? 172.21.15.53:0/523475405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:03:47.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:47 smithi046 ceph-mon[36898]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-05-08T23:03:47.527 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:47.270+0000 7fcffc0a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:03:47.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:47 smithi053 ceph-mon[77587]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-05-08T23:03:47.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:47 smithi086 ceph-mon[78610]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-05-08T23:03:48.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:47.668+0000 7fcffc0a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:03:48.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:47.795+0000 7fcffc0a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:03:48.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:48.142 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:03:48.244 INFO:teuthology.orchestra.run.smithi046.stdout:client15522 2024-05-08T23:03:48.245 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:03:48.245 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:03:48.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:48.304 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:03:48.414 INFO:teuthology.orchestra.run.smithi046.stdout:client15522 2024-05-08T23:03:48.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:48.415 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:03:48.462 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.066+0000 7fcffc0a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:03:48.512 INFO:teuthology.orchestra.run.smithi046.stdout:client15522 2024-05-08T23:03:48.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:03:48.513 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968.client15522/status') 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: Active manager daemon x restarted 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: Activating manager daemon x 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: mgrmap e21: x(active, starting, since 0.0121582s), standbys: y, z 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:03:48.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:03:48.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:03:48.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:03:48.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: Manager daemon x is now available 2024-05-08T23:03:48.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:48.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:03:48.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:48 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:03:48.631 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.15522 (3)192.168.0.1:0/1735214183 2024-05-08T23:03:48.631 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-05-08T23:03:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-05-08T23:03:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-05-08T23:03:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:70 bytes copied, 5.4883e-05 s, 1.3 MB/s 2024-05-08T23:03:48.635 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:03:48.649 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:03:48.650 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:03:48.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-08T23:03:48.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: Active manager daemon x restarted 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: Activating manager daemon x 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: mgrmap e21: x(active, starting, since 0.0121582s), standbys: y, z 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:03:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:03:48.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: Manager daemon x is now available 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:03:48.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:48 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.461+0000 7fcf4ec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.461+0000 7fcf4ec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.461+0000 7fcf4ec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.461+0000 7fcf4ec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.461+0000 7fcf4ec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.463+0000 7fcf51400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.463+0000 7fcf51400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.717 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.463+0000 7fcf51400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.463+0000 7fcf51400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:48.463+0000 7fcf51400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: Active manager daemon x restarted 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: Activating manager daemon x 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: mgrmap e21: x(active, starting, since 0.0121582s), standbys: y, z 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:03:48.718 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:03:48.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:03:48.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: Manager daemon x is now available 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:03:48.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:48 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:03:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 -- 172.21.15.46:0/924909665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 msgr2=0x7f5650073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 --2- 172.21.15.46:0/924909665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f5650073430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5640009990 tx=0x7f5640009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 -- 172.21.15.46:0/924909665 shutdown_connections 2024-05-08T23:03:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 --2- 172.21.15.46:0/924909665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 0x7f565010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 --2- 172.21.15.46:0/924909665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f5650073430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 --2- 172.21.15.46:0/924909665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 0x7f5650073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 -- 172.21.15.46:0/924909665 >> 172.21.15.46:0/924909665 conn(0x7f56500fc400 msgr2=0x7f56500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 -- 172.21.15.46:0/924909665 shutdown_connections 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.863+0000 7f5654e00700 1 -- 172.21.15.46:0/924909665 wait complete. 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.864+0000 7f5654e00700 1 Processor -- start 2024-05-08T23:03:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 -- start start 2024-05-08T23:03:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 0x7f565019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 0x7f56501a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a4f60 con 0x7f565010d480 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a50a0 con 0x7f5650073a00 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f5654e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a5230 con 0x7f5650074fe0 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f564d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f564de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 0x7f565019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f564d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38704/0 (socket says 172.21.15.46:38704) 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f564d400700 1 -- 172.21.15.46:0/1611026190 learned_addr learned my addr 172.21.15.46:0/1611026190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.865+0000 7f564e800700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 0x7f56501a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 -- 172.21.15.46:0/1611026190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 msgr2=0x7f565019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 0x7f565019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 -- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 msgr2=0x7f56501a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 0x7f56501a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 -- 172.21.15.46:0/1611026190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400095f0 con 0x7f5650074fe0 2024-05-08T23:03:48.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f564d400700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d980 tx=0x7f563c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00f890 con 0x7f5650074fe0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f563c010460 con 0x7f5650074fe0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00e610 con 0x7f5650074fe0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501a5510 con 0x7f5650074fe0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.866+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501a59d0 con 0x7f5650074fe0 2024-05-08T23:03:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.868+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56501036e0 con 0x7f5650074fe0 2024-05-08T23:03:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.869+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) v1 ==== 135558+0+0 (secure 0 0 0) 0x7f563c0108d0 con 0x7f5650074fe0 2024-05-08T23:03:48.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.871+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f563c0cc210 con 0x7f5650074fe0 2024-05-08T23:03:48.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:48.873+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f563c095870 con 0x7f5650074fe0 2024-05-08T23:03:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:49.089+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f563c0105d0 con 0x7f5650074fe0 2024-05-08T23:03:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:49.090+0000 7f5646a00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 0x7f56340aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:49.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:49.093+0000 7f564de00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 0x7f56340aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:49.102+0000 7f564de00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 0x7f56340aba40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f564000b2e0 tx=0x7f5640005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:49.173+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f565004ec60 con 0x7f5650074fe0 2024-05-08T23:03:49.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:49 smithi053 ceph-mon[77587]: [08/May/2024:23:03:48] ENGINE Bus STARTING 2024-05-08T23:03:49.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:49 smithi053 ceph-mon[77587]: mgrmap e22: x(active, since 1.01889s), standbys: y, z 2024-05-08T23:03:49.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:49 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1611026190' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:49 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:49 smithi046 ceph-mon[36898]: [08/May/2024:23:03:48] ENGINE Bus STARTING 2024-05-08T23:03:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:49 smithi046 ceph-mon[36898]: mgrmap e22: x(active, since 1.01889s), standbys: y, z 2024-05-08T23:03:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:49 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1611026190' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:49 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:49 smithi086 ceph-mon[78610]: [08/May/2024:23:03:48] ENGINE Bus STARTING 2024-05-08T23:03:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:49 smithi086 ceph-mon[78610]: mgrmap e22: x(active, since 1.01889s), standbys: y, z 2024-05-08T23:03:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:49 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1611026190' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:49.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:49 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T23:03:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.086+0000 7f5646a00700 1 -- 172.21.15.46:0/1611026190 <== mon.1 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f563c0200b0 con 0x7f5650074fe0 2024-05-08T23:03:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.088+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 msgr2=0x7f56340aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.088+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 0x7f56340aba40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f564000b2e0 tx=0x7f5640005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.088+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 msgr2=0x7f565019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.088+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d980 tx=0x7f563c00dc90 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 shutdown_connections 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56340a9650 0x7f56340aba40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565010d480 0x7f56501a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5650073a00 0x7f565019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 --2- 172.21.15.46:0/1611026190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5650074fe0 0x7f565019d970 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 >> 172.21.15.46:0/1611026190 conn(0x7f56500fc400 msgr2=0x7f5650100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 shutdown_connections 2024-05-08T23:03:50.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.089+0000 7f5654e00700 1 -- 172.21.15.46:0/1611026190 wait complete. 2024-05-08T23:03:50.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-08T23:03:50.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.325+0000 7f8c06a00700 1 -- 172.21.15.46:0/4032341523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001042d0 msgr2=0x7f8c00104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.325+0000 7f8c06a00700 1 --2- 172.21.15.46:0/4032341523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001042d0 0x7f8c00104730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0098f0 tx=0x7f8bec009c00 comp rx=0 tx=0).stop 2024-05-08T23:03:50.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 -- 172.21.15.46:0/4032341523 shutdown_connections 2024-05-08T23:03:50.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 --2- 172.21.15.46:0/4032341523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c00104c70 0x7f8c0010b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 --2- 172.21.15.46:0/4032341523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001030e0 0x7f8c00103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 --2- 172.21.15.46:0/4032341523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001042d0 0x7f8c00104730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 -- 172.21.15.46:0/4032341523 >> 172.21.15.46:0/4032341523 conn(0x7f8c000fe670 msgr2=0x7f8c00100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 -- 172.21.15.46:0/4032341523 shutdown_connections 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.326+0000 7f8c06a00700 1 -- 172.21.15.46:0/4032341523 wait complete. 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 Processor -- start 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 -- start start 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c001042d0 0x7f8c00071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00104c70 0x7f8c00072070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00072640 con 0x7f8c00104c70 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000727b0 con 0x7f8c001042d0 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.327+0000 7f8c06a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00072970 con 0x7f8c001030e0 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38732/0 (socket says 172.21.15.46:38732) 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 -- 172.21.15.46:0/1378648886 learned_addr learned my addr 172.21.15.46:0/1378648886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bff400700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c001042d0 0x7f8c00071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 -- 172.21.15.46:0/1378648886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c001042d0 msgr2=0x7f8c00071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c001042d0 0x7f8c00071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 -- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00104c70 msgr2=0x7f8c00072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00104c70 0x7f8c00072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 -- 172.21.15.46:0/1378648886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00095f0 con 0x7f8c001030e0 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.328+0000 7f8bffe00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000ebf0 tx=0x7f8bf000ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.329+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0009a10 con 0x7f8c001030e0 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.329+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec009520 con 0x7f8c001030e0 2024-05-08T23:03:50.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.329+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf000cd40 con 0x7f8c001030e0 2024-05-08T23:03:50.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.329+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0018790 con 0x7f8c001030e0 2024-05-08T23:03:50.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.329+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c00072f70 con 0x7f8c001030e0 2024-05-08T23:03:50.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.330+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0010a500 con 0x7f8c001030e0 2024-05-08T23:03:50.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.332+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f8bf0010430 con 0x7f8c001030e0 2024-05-08T23:03:50.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.334+0000 7f8bfcc00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 0x7f8be40a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.334+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8bf0014070 con 0x7f8c001030e0 2024-05-08T23:03:50.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.334+0000 7f8bff400700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 0x7f8be40a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.335+0000 7f8bff400700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 0x7f8be40a33c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00fa70 tx=0x7f8bec019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:50.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.338+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bf00965f0 con 0x7f8c001030e0 2024-05-08T23:03:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.644+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c00061480 con 0x7f8be40a0f00 2024-05-08T23:03:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.667+0000 7f8bfcc00700 1 -- 172.21.15.46:0/1378648886 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f8c00061480 con 0x7f8be40a0f00 2024-05-08T23:03:50.667 INFO:teuthology.orchestra.run.smithi046.stdout:{} 2024-05-08T23:03:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 msgr2=0x7f8be40a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 0x7f8be40a33c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00fa70 tx=0x7f8bec019040 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 msgr2=0x7f8c00070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000ebf0 tx=0x7f8bf000ef00 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 shutdown_connections 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8be40a0f00 0x7f8be40a33c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00104c70 0x7f8c00072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c001030e0 0x7f8c00070b40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 --2- 172.21.15.46:0/1378648886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c001042d0 0x7f8c00071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 >> 172.21.15.46:0/1378648886 conn(0x7f8c000fe670 msgr2=0x7f8c00107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 shutdown_connections 2024-05-08T23:03:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.669+0000 7f8c06a00700 1 -- 172.21.15.46:0/1378648886 wait complete. 2024-05-08T23:03:50.689 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T23:03:50.700 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T23:03:50.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.930+0000 7f2c38a00700 1 -- 172.21.15.46:0/2947344035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34073a00 msgr2=0x7f2c34073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.930+0000 7f2c38a00700 1 --2- 172.21.15.46:0/2947344035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34073a00 0x7f2c34073e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009990 tx=0x7f2c20009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 -- 172.21.15.46:0/2947344035 shutdown_connections 2024-05-08T23:03:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 --2- 172.21.15.46:0/2947344035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34108d40 0x7f2c3410b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 --2- 172.21.15.46:0/2947344035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34074fe0 0x7f2c34073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 --2- 172.21.15.46:0/2947344035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34073a00 0x7f2c34073e80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 -- 172.21.15.46:0/2947344035 >> 172.21.15.46:0/2947344035 conn(0x7f2c340fc410 msgr2=0x7f2c340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 -- 172.21.15.46:0/2947344035 shutdown_connections 2024-05-08T23:03:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.931+0000 7f2c38a00700 1 -- 172.21.15.46:0/2947344035 wait complete. 2024-05-08T23:03:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 Processor -- start 2024-05-08T23:03:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 -- start start 2024-05-08T23:03:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34073a00 0x7f2c34199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34108d40 0x7f2c341a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a0b40 con 0x7f2c34073a00 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a0c80 con 0x7f2c34074fe0 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.932+0000 7f2c38a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a0e10 con 0x7f2c34108d40 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34073a00 0x7f2c34199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49924/0 (socket says 172.21.15.46:49924) 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 -- 172.21.15.46:0/1877170745 learned_addr learned my addr 172.21.15.46:0/1877170745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c32800700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34108d40 0x7f2c341a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 -- 172.21.15.46:0/1877170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34108d40 msgr2=0x7f2c341a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34108d40 0x7f2c341a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 -- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34073a00 msgr2=0x7f2c34199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34073a00 0x7f2c34199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 -- 172.21.15.46:0/1877170745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200095f0 con 0x7f2c34074fe0 2024-05-08T23:03:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.933+0000 7f2c31400700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c20009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.934+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2001d070 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.934+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c341a1090 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.934+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c20021410 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.934+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20017650 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.934+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c341a14b0 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.935+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c34107b00 con 0x7f2c34074fe0 2024-05-08T23:03:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.936+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 23) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f2c20021580 con 0x7f2c34074fe0 2024-05-08T23:03:50.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.938+0000 7f2c2aa00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 0x7f2c180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:50.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.938+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2c200ce310 con 0x7f2c34074fe0 2024-05-08T23:03:50.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.938+0000 7f2c31e00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 0x7f2c180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:50.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.939+0000 7f2c31e00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 0x7f2c180a3750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009c60 tx=0x7f2c24009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:50.942+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2c20097440 con 0x7f2c34074fe0 2024-05-08T23:03:50.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: [08/May/2024:23:03:48] ENGINE Serving on https://172.21.15.46:7150 2024-05-08T23:03:50.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: [08/May/2024:23:03:49] ENGINE Serving on http://172.21.15.46:8765 2024-05-08T23:03:50.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: [08/May/2024:23:03:49] ENGINE Bus STARTED 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:50.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:50 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: [08/May/2024:23:03:48] ENGINE Serving on https://172.21.15.46:7150 2024-05-08T23:03:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: [08/May/2024:23:03:49] ENGINE Serving on http://172.21.15.46:8765 2024-05-08T23:03:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: [08/May/2024:23:03:49] ENGINE Bus STARTED 2024-05-08T23:03:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:51.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.014 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.014 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.014 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.014 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.014 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:50 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.026 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:50.651+0000 7fcf4b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:50.651+0000 7fcf4b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:50.651+0000 7fcf4b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:50.651+0000 7fcf4b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:03:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:03:50.651+0000 7fcf4b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: [08/May/2024:23:03:48] ENGINE Serving on https://172.21.15.46:7150 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: [08/May/2024:23:03:49] ENGINE Serving on http://172.21.15.46:8765 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: [08/May/2024:23:03:49] ENGINE Bus STARTED 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:51.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:50 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.185+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c3410e900 con 0x7f2c34074fe0 2024-05-08T23:03:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.187+0000 7f2c2aa00700 1 -- 172.21.15.46:0/1877170745 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+25062 (secure 0 0 0) 0x7f2c20005740 con 0x7f2c34074fe0 2024-05-08T23:03:51.187 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:51.187 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:48.069712+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 msgr2=0x7f2c180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 0x7f2c180a3750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009c60 tx=0x7f2c24009260 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 msgr2=0x7f2c341995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c20009580 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 shutdown_connections 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c34108d40 0x7f2c341a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2c180a1290 0x7f2c180a3750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c34073a00 0x7f2c34199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 --2- 172.21.15.46:0/1877170745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c34074fe0 0x7f2c341995a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.189+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 >> 172.21.15.46:0/1877170745 conn(0x7f2c340fc410 msgr2=0x7f2c34104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.190+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 shutdown_connections 2024-05-08T23:03:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.190+0000 7f2c38a00700 1 -- 172.21.15.46:0/1877170745 wait complete. 2024-05-08T23:03:51.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:51.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 -- 172.21.15.46:0/1050054351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5800104210 msgr2=0x7f5800106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 --2- 172.21.15.46:0/1050054351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5800104210 0x7f5800106600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009990 tx=0x7f57ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 -- 172.21.15.46:0/1050054351 shutdown_connections 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 --2- 172.21.15.46:0/1050054351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800106b40 0x7f5800108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 --2- 172.21.15.46:0/1050054351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58001018e0 0x7f5800103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 --2- 172.21.15.46:0/1050054351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5800104210 0x7f5800106600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 -- 172.21.15.46:0/1050054351 >> 172.21.15.46:0/1050054351 conn(0x7f58000fb290 msgr2=0x7f58000fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 -- 172.21.15.46:0/1050054351 shutdown_connections 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.438+0000 7f5808600700 1 -- 172.21.15.46:0/1050054351 wait complete. 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 Processor -- start 2024-05-08T23:03:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 -- start start 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800104210 0x7f58000ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5800106b40 0x7f58000ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800100570 con 0x7f58001018e0 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58001006e0 con 0x7f5800106b40 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.440+0000 7f5808600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58001a0dd0 con 0x7f5800104210 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33708/0 (socket says 172.21.15.46:33708) 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 -- 172.21.15.46:0/1499890349 learned_addr learned my addr 172.21.15.46:0/1499890349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805200700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800104210 0x7f58000ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5806600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5800106b40 0x7f58000ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 -- 172.21.15.46:0/1499890349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800104210 msgr2=0x7f58000ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800104210 0x7f58000ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 -- 172.21.15.46:0/1499890349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5800106b40 msgr2=0x7f58000ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5800106b40 0x7f58000ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 -- 172.21.15.46:0/1499890349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0095f0 con 0x7f58001018e0 2024-05-08T23:03:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.441+0000 7f5805c00700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e9a0 tx=0x7f57f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.442+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4009790 con 0x7f58001018e0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.442+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f400ce70 con 0x7f58001018e0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.442+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58001a1060 con 0x7f58001018e0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.442+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4018420 con 0x7f58001018e0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.442+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58001a15c0 con 0x7f58001018e0 2024-05-08T23:03:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.443+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58000fce10 con 0x7f58001018e0 2024-05-08T23:03:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.445+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f57f40106f0 con 0x7f58001018e0 2024-05-08T23:03:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.447+0000 7f57fe000700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 0x7f57e80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.447+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f57f4014070 con 0x7f58001018e0 2024-05-08T23:03:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.447+0000 7f5805200700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 0x7f57e80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.448+0000 7f5805200700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 0x7f57e80a37d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b2e0 tx=0x7f57ec00b750 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.449+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57f4096690 con 0x7f58001018e0 2024-05-08T23:03:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.685+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f580010c800 con 0x7f58001018e0 2024-05-08T23:03:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.686+0000 7f57fe000700 1 -- 172.21.15.46:0/1499890349 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57f4099cb0 con 0x7f58001018e0 2024-05-08T23:03:51.686 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 msgr2=0x7f57e80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 0x7f57e80a37d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b2e0 tx=0x7f57ec00b750 comp rx=0 tx=0).stop 2024-05-08T23:03:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 msgr2=0x7f5800101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e9a0 tx=0x7f57f400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 shutdown_connections 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5800106b40 0x7f58000ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e80a1310 0x7f57e80a37d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58001018e0 0x7f5800101200 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 --2- 172.21.15.46:0/1499890349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800104210 0x7f58000ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 >> 172.21.15.46:0/1499890349 conn(0x7f58000fb290 msgr2=0x7f58001037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 shutdown_connections 2024-05-08T23:03:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.688+0000 7f5808600700 1 -- 172.21.15.46:0/1499890349 wait complete. 2024-05-08T23:03:51.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 -- 172.21.15.46:0/852323724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 msgr2=0x7ff4b8101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 --2- 172.21.15.46:0/852323724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b8101960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009990 tx=0x7ff4ac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 -- 172.21.15.46:0/852323724 shutdown_connections 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 --2- 172.21.15.46:0/852323724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8104870 0x7ff4b8104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 --2- 172.21.15.46:0/852323724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b8101960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 --2- 172.21.15.46:0/852323724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8101ea0 0x7ff4b8104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 -- 172.21.15.46:0/852323724 >> 172.21.15.46:0/852323724 conn(0x7ff4b80fb130 msgr2=0x7ff4b80fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 -- 172.21.15.46:0/852323724 shutdown_connections 2024-05-08T23:03:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.932+0000 7ff4be400700 1 -- 172.21.15.46:0/852323724 wait complete. 2024-05-08T23:03:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.933+0000 7ff4be400700 1 Processor -- start 2024-05-08T23:03:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.933+0000 7ff4be400700 1 -- start start 2024-05-08T23:03:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.933+0000 7ff4be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.933+0000 7ff4be400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101ea0 0x7ff4b819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4be400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8104870 0x7ff4b81a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4be400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a4c30 con 0x7ff4b8101ea0 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4be400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a4d70 con 0x7ff4b8104870 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81a4f00 con 0x7ff4b80ff530 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34756/0 (socket says 172.21.15.46:34756) 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 -- 172.21.15.46:0/767920553 learned_addr learned my addr 172.21.15.46:0/767920553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bda00700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8104870 0x7ff4b81a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4b7e00700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101ea0 0x7ff4b819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 -- 172.21.15.46:0/767920553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8104870 msgr2=0x7ff4b81a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8104870 0x7ff4b81a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 -- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101ea0 msgr2=0x7ff4b819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101ea0 0x7ff4b819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.934+0000 7ff4bd000700 1 -- 172.21.15.46:0/767920553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac0095f0 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4bd000700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0098e0 tx=0x7ff4ac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01d070 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ac022470 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81a5180 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac017650 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.935+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81a55a0 con 0x7ff4b80ff530 2024-05-08T23:03:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.936+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b80fccb0 con 0x7ff4b80ff530 2024-05-08T23:03:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.938+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) v1 ==== 135993+0+0 (secure 0 0 0) 0x7ff4ac022850 con 0x7ff4b80ff530 2024-05-08T23:03:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.939+0000 7ff4b5600700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 0x7ff4a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.940+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff4ac0ce7b0 con 0x7ff4b80ff530 2024-05-08T23:03:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.940+0000 7ff4b7e00700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 0x7ff4a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.940+0000 7ff4b7e00700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 0x7ff4a00a37f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009c60 tx=0x7ff4a4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:51.944+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4ac0979a0 con 0x7ff4b80ff530 2024-05-08T23:03:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:03:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:03:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:03:51.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:03:51.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:51.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T23:03:51.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:51 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1877170745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T23:03:52.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:51 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1877170745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: from='client.25579 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T23:03:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:51 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1877170745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.185+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4b8112c00 con 0x7ff4b80ff530 2024-05-08T23:03:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.186+0000 7ff4b5600700 1 -- 172.21.15.46:0/767920553 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4ac0270a0 con 0x7ff4b80ff530 2024-05-08T23:03:52.186 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 msgr2=0x7ff4a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 0x7ff4a00a37f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009c60 tx=0x7ff4a4009260 comp rx=0 tx=0).stop 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 msgr2=0x7ff4b819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0098e0 tx=0x7ff4ac005740 comp rx=0 tx=0).stop 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 shutdown_connections 2024-05-08T23:03:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b8104870 0x7ff4b81a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b80ff530 0x7ff4b819d100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4a00a1330 0x7ff4a00a37f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 --2- 172.21.15.46:0/767920553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b8101ea0 0x7ff4b819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 >> 172.21.15.46:0/767920553 conn(0x7ff4b80fb130 msgr2=0x7ff4b8102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 shutdown_connections 2024-05-08T23:03:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.189+0000 7ff4be400700 1 -- 172.21.15.46:0/767920553 wait complete. 2024-05-08T23:03:52.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 -- 172.21.15.46:0/1681438050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100030 msgr2=0x7febf01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 --2- 172.21.15.46:0/1681438050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100030 0x7febf01004b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098f0 tx=0x7febdc009c00 comp rx=0 tx=0).stop 2024-05-08T23:03:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 -- 172.21.15.46:0/1681438050 shutdown_connections 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 --2- 172.21.15.46:0/1681438050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf01009f0 0x7febf0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 --2- 172.21.15.46:0/1681438050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 --2- 172.21.15.46:0/1681438050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100030 0x7febf01004b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 -- 172.21.15.46:0/1681438050 >> 172.21.15.46:0/1681438050 conn(0x7febf00fb290 msgr2=0x7febf00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 -- 172.21.15.46:0/1681438050 shutdown_connections 2024-05-08T23:03:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.442+0000 7febf4e00700 1 -- 172.21.15.46:0/1681438050 wait complete. 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 Processor -- start 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 -- start start 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 0x7febf01995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf01009f0 0x7febf01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01a0ba0 con 0x7febf01009f0 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01a0ce0 con 0x7febf0100030 2024-05-08T23:03:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.443+0000 7febf4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01a0e70 con 0x7febf00ff6d0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 0x7febf01995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34766/0 (socket says 172.21.15.46:34766) 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 -- 172.21.15.46:0/2549403596 learned_addr learned my addr 172.21.15.46:0/2549403596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febee800700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf01009f0 0x7febf01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 -- 172.21.15.46:0/2549403596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 msgr2=0x7febf01995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 0x7febf01995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 -- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf01009f0 msgr2=0x7febf01a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf01009f0 0x7febf01a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febede00700 1 -- 172.21.15.46:0/2549403596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc009520 con 0x7febf00ff6d0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.444+0000 7febed400700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 0x7febf01995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febede00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febe000e9a0 tx=0x7febe000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0009790 con 0x7febf00ff6d0 2024-05-08T23:03:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf0101940 con 0x7febf00ff6d0 2024-05-08T23:03:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe000ce50 con 0x7febf00ff6d0 2024-05-08T23:03:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0018680 con 0x7febf00ff6d0 2024-05-08T23:03:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.445+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf0101ea0 con 0x7febf00ff6d0 2024-05-08T23:03:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.446+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf010a4b0 con 0x7febf00ff6d0 2024-05-08T23:03:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.448+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) v1 ==== 135993+0+0 (secure 0 0 0) 0x7febe0010430 con 0x7febf00ff6d0 2024-05-08T23:03:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.450+0000 7febe6a00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 0x7febd40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.450+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7febe0014070 con 0x7febf00ff6d0 2024-05-08T23:03:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.450+0000 7febed400700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 0x7febd40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.451+0000 7febed400700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 0x7febd40a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febdc00fa40 tx=0x7febdc019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:52.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.454+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7febe0052d60 con 0x7febf00ff6d0 2024-05-08T23:03:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.680+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7febf010e8a0 con 0x7febf00ff6d0 2024-05-08T23:03:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.680+0000 7febe6a00700 1 -- 172.21.15.46:0/2549403596 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7febe0096290 con 0x7febf00ff6d0 2024-05-08T23:03:52.681 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 msgr2=0x7febd40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 0x7febd40a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febdc00fa40 tx=0x7febdc019040 comp rx=0 tx=0).stop 2024-05-08T23:03:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 msgr2=0x7febf0199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febe000e9a0 tx=0x7febe000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 shutdown_connections 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7febd40a1290 0x7febd40a3750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf01009f0 0x7febf01a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf00ff6d0 0x7febf0199070 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febe000e9a0 tx=0x7febe000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 --2- 172.21.15.46:0/2549403596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febf0100030 0x7febf01995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 >> 172.21.15.46:0/2549403596 conn(0x7febf00fb290 msgr2=0x7febf00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.684+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 shutdown_connections 2024-05-08T23:03:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.685+0000 7febf4e00700 1 -- 172.21.15.46:0/2549403596 wait complete. 2024-05-08T23:03:52.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 -- 172.21.15.46:0/3237636974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70104210 msgr2=0x7fdf70106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 --2- 172.21.15.46:0/3237636974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70104210 0x7fdf70106600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0098f0 tx=0x7fdf5c009c00 comp rx=0 tx=0).stop 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 -- 172.21.15.46:0/3237636974 shutdown_connections 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 --2- 172.21.15.46:0/3237636974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70106b40 0x7fdf70108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 --2- 172.21.15.46:0/3237636974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 --2- 172.21.15.46:0/3237636974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70104210 0x7fdf70106600 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 -- 172.21.15.46:0/3237636974 >> 172.21.15.46:0/3237636974 conn(0x7fdf700fb290 msgr2=0x7fdf700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.941+0000 7fdf74e00700 1 -- 172.21.15.46:0/3237636974 shutdown_connections 2024-05-08T23:03:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.942+0000 7fdf74e00700 1 -- 172.21.15.46:0/3237636974 wait complete. 2024-05-08T23:03:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.942+0000 7fdf74e00700 1 Processor -- start 2024-05-08T23:03:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 -- start start 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70104210 0x7fdf70199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70106b40 0x7fdf701a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701a0b60 con 0x7fdf70106b40 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701a0ca0 con 0x7fdf701018e0 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf74e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701a0e30 con 0x7fdf70104210 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf6de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf6de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56786/0 (socket says 172.21.15.46:56786) 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf6de00700 1 -- 172.21.15.46:0/1289746825 learned_addr learned my addr 172.21.15.46:0/1289746825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf6d400700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70104210 0x7fdf70199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.943+0000 7fdf6e800700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70106b40 0x7fdf701a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 -- 172.21.15.46:0/1289746825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70104210 msgr2=0x7fdf70199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70104210 0x7fdf70199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 -- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70106b40 msgr2=0x7fdf701a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70106b40 0x7fdf701a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 -- 172.21.15.46:0/1289746825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600095f0 con 0x7fdf701018e0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf6de00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000ec10 tx=0x7fdf6000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000cd40 con 0x7fdf701018e0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c009520 con 0x7fdf701018e0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf600044e0 con 0x7fdf701018e0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60010690 con 0x7fdf701018e0 2024-05-08T23:03:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.944+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf701a1490 con 0x7fdf701018e0 2024-05-08T23:03:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.946+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70108710 con 0x7fdf701018e0 2024-05-08T23:03:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.948+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fdf60003710 con 0x7fdf701018e0 2024-05-08T23:03:52.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.949+0000 7fdf66a00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 0x7fdf540a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:52.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.949+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf60014070 con 0x7fdf701018e0 2024-05-08T23:03:52.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.949+0000 7fdf6d400700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 0x7fdf540a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:52.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.950+0000 7fdf6d400700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 0x7fdf540a3870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00fa40 tx=0x7fdf5c019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:52.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:52.954+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf60096740 con 0x7fdf701018e0 2024-05-08T23:03:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1499890349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:52.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/767920553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:52.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:52 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2549403596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1499890349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/767920553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:52 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2549403596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1499890349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/767920553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:52 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2549403596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:53.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.216+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdf7010c900 con 0x7fdf701018e0 2024-05-08T23:03:53.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.216+0000 7fdf66a00700 1 -- 172.21.15.46:0/1289746825 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdf60099d60 con 0x7fdf701018e0 2024-05-08T23:03:53.216 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.219+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 msgr2=0x7fdf540a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.219+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 0x7fdf540a3870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00fa40 tx=0x7fdf5c019040 comp rx=0 tx=0).stop 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.219+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 msgr2=0x7fdf70199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.219+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000ec10 tx=0x7fdf6000ef20 comp rx=0 tx=0).stop 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 shutdown_connections 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf540a13b0 0x7fdf540a3870 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf70106b40 0x7fdf701a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf701018e0 0x7fdf70199030 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 --2- 172.21.15.46:0/1289746825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf70104210 0x7fdf70199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 >> 172.21.15.46:0/1289746825 conn(0x7fdf700fb290 msgr2=0x7fdf70101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:53.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 shutdown_connections 2024-05-08T23:03:53.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.220+0000 7fdf74e00700 1 -- 172.21.15.46:0/1289746825 wait complete. 2024-05-08T23:03:53.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 -- 172.21.15.46:0/2655560145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 msgr2=0x7fbc701012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 --2- 172.21.15.46:0/2655560145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc701012a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 -- 172.21.15.46:0/2655560145 shutdown_connections 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 --2- 172.21.15.46:0/2655560145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70102990 0x7fbc701092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 --2- 172.21.15.46:0/2655560145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc701012a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 --2- 172.21.15.46:0/2655560145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70101fd0 0x7fbc70102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 -- 172.21.15.46:0/2655560145 >> 172.21.15.46:0/2655560145 conn(0x7fbc700fc410 msgr2=0x7fbc700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 -- 172.21.15.46:0/2655560145 shutdown_connections 2024-05-08T23:03:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.442+0000 7fbc78800700 1 -- 172.21.15.46:0/2655560145 wait complete. 2024-05-08T23:03:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.443+0000 7fbc78800700 1 Processor -- start 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.443+0000 7fbc78800700 1 -- start start 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70101fd0 0x7fbc70073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70102990 0x7fbc700738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70074070 con 0x7fbc70100e80 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700741e0 con 0x7fbc70102990 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc78800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701a0ea0 con 0x7fbc70101fd0 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33656/0 (socket says 172.21.15.46:33656) 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 -- 172.21.15.46:0/1367806047 learned_addr learned my addr 172.21.15.46:0/1367806047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75400700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70101fd0 0x7fbc70073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc76800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70102990 0x7fbc700738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 -- 172.21.15.46:0/1367806047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70101fd0 msgr2=0x7fbc70073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70101fd0 0x7fbc70073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 -- 172.21.15.46:0/1367806047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70102990 msgr2=0x7fbc700738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70102990 0x7fbc700738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.444+0000 7fbc75e00700 1 -- 172.21.15.46:0/1367806047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009c40 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc75e00700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc64005910 tx=0x7fbc64005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6401d070 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc64021410 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc640095f0 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64017650 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.445+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc701a1360 con 0x7fbc70100e80 2024-05-08T23:03:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.447+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc701082f0 con 0x7fbc70100e80 2024-05-08T23:03:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.449+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fbc64021580 con 0x7fbc70100e80 2024-05-08T23:03:53.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.450+0000 7fbc6aa00700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 0x7fbc580a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.451+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fbc640cee10 con 0x7fbc70100e80 2024-05-08T23:03:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.451+0000 7fbc75400700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 0x7fbc580a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.451+0000 7fbc75400700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 0x7fbc580a38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009c60 tx=0x7fbc60009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:53.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.454+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc64097c90 con 0x7fbc70100e80 2024-05-08T23:03:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.683+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc7010d010 con 0x7fbc70100e80 2024-05-08T23:03:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.685+0000 7fbc6aa00700 1 -- 172.21.15.46:0/1367806047 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+25062 (secure 0 0 0) 0x7fbc6409b2b0 con 0x7fbc70100e80 2024-05-08T23:03:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:53.686 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:48.069712+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.689+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 msgr2=0x7fbc580a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.689+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 0x7fbc580a38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009c60 tx=0x7fbc60009260 comp rx=0 tx=0).stop 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 msgr2=0x7fbc70074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc64005910 tx=0x7fbc64005740 comp rx=0 tx=0).stop 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 shutdown_connections 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc70102990 0x7fbc700738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc580a1430 0x7fbc580a38f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc70100e80 0x7fbc70074d00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 --2- 172.21.15.46:0/1367806047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc70101fd0 0x7fbc70073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 >> 172.21.15.46:0/1367806047 conn(0x7fbc700fc410 msgr2=0x7fbc70105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:53.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 shutdown_connections 2024-05-08T23:03:53.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.690+0000 7fbc78800700 1 -- 172.21.15.46:0/1367806047 wait complete. 2024-05-08T23:03:53.712 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:53.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:53.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 -- 172.21.15.46:0/1350368940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c073c60 msgr2=0x7f0f3c1118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 --2- 172.21.15.46:0/1350368940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1118e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009990 tx=0x7f0f28009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 -- 172.21.15.46:0/1350368940 shutdown_connections 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 --2- 172.21.15.46:0/1350368940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c111e20 0x7f0f3c114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 --2- 172.21.15.46:0/1350368940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073340 0x7f0f3c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 --2- 172.21.15.46:0/1350368940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1118e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 -- 172.21.15.46:0/1350368940 >> 172.21.15.46:0/1350368940 conn(0x7f0f3c0fc8e0 msgr2=0x7f0f3c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 -- 172.21.15.46:0/1350368940 shutdown_connections 2024-05-08T23:03:53.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.927+0000 7f0f41a00700 1 -- 172.21.15.46:0/1350368940 wait complete. 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.928+0000 7f0f41a00700 1 Processor -- start 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.928+0000 7f0f41a00700 1 -- start start 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c111e20 0x7f0f3c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c19de20 con 0x7f0f3c111e20 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c19df90 con 0x7f0f3c073c60 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f41a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c19e150 con 0x7f0f3c073340 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34816/0 (socket says 172.21.15.46:34816) 2024-05-08T23:03:53.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f3b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c111e20 0x7f0f3c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f3aa00700 1 -- 172.21.15.46:0/2508252969 learned_addr learned my addr 172.21.15.46:0/2508252969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.929+0000 7f0f3a000700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 -- 172.21.15.46:0/2508252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 msgr2=0x7f0f3c1a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 -- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c111e20 msgr2=0x7f0f3c19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c111e20 0x7f0f3c19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 -- 172.21.15.46:0/2508252969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280095f0 con 0x7f0f3c073340 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3a000700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f3aa00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00e9a0 tx=0x7f0f2c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c009790 con 0x7f0f3c073340 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c19e430 con 0x7f0f3c073340 2024-05-08T23:03:53.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2c010430 con 0x7f0f3c073340 2024-05-08T23:03:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.930+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c018680 con 0x7f0f3c073340 2024-05-08T23:03:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.931+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c1aa670 con 0x7f0f3c073340 2024-05-08T23:03:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.932+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c104300 con 0x7f0f3c073340 2024-05-08T23:03:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.933+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f0f2c0105a0 con 0x7f0f3c073340 2024-05-08T23:03:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.935+0000 7f0f33400700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 0x7f0f200a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.935+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0f2c014070 con 0x7f0f3c073340 2024-05-08T23:03:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.935+0000 7f0f3a000700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 0x7f0f200a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.936+0000 7f0f3a000700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 0x7f0f200a3870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c19ebd0 tx=0x7f0f28005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:53.939+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f2c052d60 con 0x7f0f3c073340 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T23:03:53.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:53 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1289746825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:03:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:03:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T23:03:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:53 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1289746825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:54.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:03:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: Updating smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:53 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1289746825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.164+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f3c074a80 con 0x7f0f3c073340 2024-05-08T23:03:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.165+0000 7f0f33400700 1 -- 172.21.15.46:0/2508252969 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f2c096360 con 0x7f0f3c073340 2024-05-08T23:03:54.165 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 msgr2=0x7f0f200a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 0x7f0f200a3870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c19ebd0 tx=0x7f0f28005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 msgr2=0x7f0f3c1a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00e9a0 tx=0x7f0f2c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 shutdown_connections 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0f200a13b0 0x7f0f200a3870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f3c111e20 0x7f0f3c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3c073340 0x7f0f3c1a3680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 --2- 172.21.15.46:0/2508252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f3c073c60 0x7f0f3c1a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.167+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 >> 172.21.15.46:0/2508252969 conn(0x7f0f3c0fc8e0 msgr2=0x7f0f3c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.168+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 shutdown_connections 2024-05-08T23:03:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.168+0000 7f0f41a00700 1 -- 172.21.15.46:0/2508252969 wait complete. 2024-05-08T23:03:54.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 -- 172.21.15.46:0/1213439306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101a30 msgr2=0x7f3fec105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 --2- 172.21.15.46:0/1213439306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101a30 0x7f3fec105a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 -- 172.21.15.46:0/1213439306 shutdown_connections 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 --2- 172.21.15.46:0/1213439306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 --2- 172.21.15.46:0/1213439306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec101080 0x7f3fec101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 --2- 172.21.15.46:0/1213439306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101a30 0x7f3fec105a70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 -- 172.21.15.46:0/1213439306 >> 172.21.15.46:0/1213439306 conn(0x7f3fec0fc8e0 msgr2=0x7f3fec0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 -- 172.21.15.46:0/1213439306 shutdown_connections 2024-05-08T23:03:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.395+0000 7f3ff2400700 1 -- 172.21.15.46:0/1213439306 wait complete. 2024-05-08T23:03:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.396+0000 7f3ff2400700 1 Processor -- start 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.396+0000 7f3ff2400700 1 -- start start 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 0x7f3fec19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19e880 con 0x7f3fec101080 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19e9f0 con 0x7f3fec19de00 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3ff2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19ebb0 con 0x7f3fec1061a0 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33692/0 (socket says 172.21.15.46:33692) 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 -- 172.21.15.46:0/664020295 learned_addr learned my addr 172.21.15.46:0/664020295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feaa00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3febe00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 0x7f3fec19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 -- 172.21.15.46:0/664020295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 msgr2=0x7f3fec199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 -- 172.21.15.46:0/664020295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 msgr2=0x7f3fec19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 0x7f3fec19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feb400700 1 -- 172.21.15.46:0/664020295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80095f0 con 0x7f3fec101080 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3feaa00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.397+0000 7f3febe00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 0x7f3fec19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3feb400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc009790 con 0x7f3fec101080 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fdc00cbd0 con 0x7f3fec101080 2024-05-08T23:03:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec1a6110 con 0x7f3fec101080 2024-05-08T23:03:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc018680 con 0x7f3fec101080 2024-05-08T23:03:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.398+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec1a6640 con 0x7f3fec101080 2024-05-08T23:03:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.399+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec102550 con 0x7f3fec101080 2024-05-08T23:03:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.401+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f3fdc00cd40 con 0x7f3fec101080 2024-05-08T23:03:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.402+0000 7f3fe3e00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 0x7f3fd00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.402+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3fdc014070 con 0x7f3fec101080 2024-05-08T23:03:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.402+0000 7f3feaa00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 0x7f3fd00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.403+0000 7f3feaa00700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 0x7f3fd00a38f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006010 tx=0x7f3fd800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.406+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fdc052d60 con 0x7f3fec101080 2024-05-08T23:03:54.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.625+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fec101a30 con 0x7f3fec101080 2024-05-08T23:03:54.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.626+0000 7f3fe3e00700 1 -- 172.21.15.46:0/664020295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fdc096490 con 0x7f3fec101080 2024-05-08T23:03:54.626 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 msgr2=0x7f3fd00a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 0x7f3fd00a38f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006010 tx=0x7f3fd800b260 comp rx=0 tx=0).stop 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 msgr2=0x7f3fec199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).stop 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 shutdown_connections 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec19de00 0x7f3fec19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3fd00a1430 0x7f3fd00a38f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fec101080 0x7f3fec199520 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 --2- 172.21.15.46:0/664020295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec1061a0 0x7f3fec199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 >> 172.21.15.46:0/664020295 conn(0x7f3fec0fc8e0 msgr2=0x7f3fec0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 shutdown_connections 2024-05-08T23:03:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.628+0000 7f3ff2400700 1 -- 172.21.15.46:0/664020295 wait complete. 2024-05-08T23:03:54.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.850+0000 7ff326200700 1 -- 172.21.15.46:0/4062374730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 msgr2=0x7ff320101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.850+0000 7ff326200700 1 --2- 172.21.15.46:0/4062374730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff320101460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff310009990 tx=0x7ff310009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 -- 172.21.15.46:0/4062374730 shutdown_connections 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 --2- 172.21.15.46:0/4062374730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3201061a0 0x7ff32010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 --2- 172.21.15.46:0/4062374730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff320101460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 --2- 172.21.15.46:0/4062374730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff320101a30 0x7ff320105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 -- 172.21.15.46:0/4062374730 >> 172.21.15.46:0/4062374730 conn(0x7ff3200fc8e0 msgr2=0x7ff3200fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 -- 172.21.15.46:0/4062374730 shutdown_connections 2024-05-08T23:03:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.851+0000 7ff326200700 1 -- 172.21.15.46:0/4062374730 wait complete. 2024-05-08T23:03:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 Processor -- start 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 -- start start 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff3201995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3201061a0 0x7ff320199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32019e8b0 con 0x7ff320101080 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32019ea20 con 0x7ff32019de30 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.852+0000 7ff326200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32019ebe0 con 0x7ff3201061a0 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff3201995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56862/0 (socket says 172.21.15.46:56862) 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 -- 172.21.15.46:0/3112506670 learned_addr learned my addr 172.21.15.46:0/3112506670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31ea00700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3201061a0 0x7ff320199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 -- 172.21.15.46:0/3112506670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3201061a0 msgr2=0x7ff320199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3201061a0 0x7ff320199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 -- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 msgr2=0x7ff3201995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff3201995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 -- 172.21.15.46:0/3112506670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100095f0 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff31fe00700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff31800ef30 tx=0x7ff31800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.853+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318018070 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.854+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff31800f040 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.854+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3201a60e0 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.854+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800cd00 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.854+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3201a6610 con 0x7ff32019de30 2024-05-08T23:03:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.855+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff320102550 con 0x7ff32019de30 2024-05-08T23:03:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.857+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7ff318003a00 con 0x7ff32019de30 2024-05-08T23:03:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.858+0000 7ff317e00700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 0x7ff3040a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.858+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff3180ccf20 con 0x7ff32019de30 2024-05-08T23:03:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.858+0000 7ff31f400700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 0x7ff3040a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.859+0000 7ff31f400700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 0x7ff3040a38f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff31000b2e0 tx=0x7ff310005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:54.861+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff318095da0 con 0x7ff32019de30 2024-05-08T23:03:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1367806047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:54.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2508252969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:54.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/664020295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:55.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1367806047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2508252969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/664020295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: Updating smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1367806047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2508252969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:03:55.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:03:55.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/664020295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.084+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff320101a30 con 0x7ff32019de30 2024-05-08T23:03:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.085+0000 7ff317e00700 1 -- 172.21.15.46:0/3112506670 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3180993c0 con 0x7ff32019de30 2024-05-08T23:03:55.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.087+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 msgr2=0x7ff3040a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.087+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 0x7ff3040a38f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff31000b2e0 tx=0x7ff310005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.087+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 msgr2=0x7ff32019e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.087+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff31800ef30 tx=0x7ff31800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 shutdown_connections 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff32019de30 0x7ff32019e2b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3040a1430 0x7ff3040a38f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff320101080 0x7ff3201995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 --2- 172.21.15.46:0/3112506670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3201061a0 0x7ff320199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 >> 172.21.15.46:0/3112506670 conn(0x7ff3200fc8e0 msgr2=0x7ff3200fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 shutdown_connections 2024-05-08T23:03:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.088+0000 7ff326200700 1 -- 172.21.15.46:0/3112506670 wait complete. 2024-05-08T23:03:55.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 -- 172.21.15.46:0/1192639487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4100910 msgr2=0x7f90f4104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 --2- 172.21.15.46:0/1192639487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4100910 0x7f90f4104950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009990 tx=0x7f90e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 -- 172.21.15.46:0/1192639487 shutdown_connections 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 --2- 172.21.15.46:0/1192639487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f4105080 0x7f90f410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 --2- 172.21.15.46:0/1192639487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f40fff60 0x7f90f4100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 --2- 172.21.15.46:0/1192639487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4100910 0x7f90f4104950 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 -- 172.21.15.46:0/1192639487 >> 172.21.15.46:0/1192639487 conn(0x7f90f40fb7a0 msgr2=0x7f90f40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 -- 172.21.15.46:0/1192639487 shutdown_connections 2024-05-08T23:03:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.311+0000 7f90fc200700 1 -- 172.21.15.46:0/1192639487 wait complete. 2024-05-08T23:03:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 Processor -- start 2024-05-08T23:03:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 -- start start 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 0x7f90f4101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 0x7f90f41a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41a5be0 con 0x7f90f4102470 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41a5d50 con 0x7f90f4105080 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.312+0000 7f90fc200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41a5f10 con 0x7f90f40fff60 2024-05-08T23:03:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90fa200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 0x7f90f41a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34878/0 (socket says 172.21.15.46:34878) 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 -- 172.21.15.46:0/1176642564 learned_addr learned my addr 172.21.15.46:0/1176642564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90fa200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 0x7f90f41a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33718/0 (socket says 172.21.15.46:33718) 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f8e00700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 0x7f90f4101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 -- 172.21.15.46:0/1176642564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 msgr2=0x7f90f4101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 0x7f90f4101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 -- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 msgr2=0x7f90f41a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 0x7f90f41a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 -- 172.21.15.46:0/1176642564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00095f0 con 0x7f90f40fff60 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f8e00700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 0x7f90f4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.313+0000 7f90f9800700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e9a0 tx=0x7f90e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.314+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8009790 con 0x7f90f40fff60 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.314+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f41a61f0 con 0x7f90f40fff60 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.314+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90e8010430 con 0x7f90f40fff60 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.314+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8018680 con 0x7f90f40fff60 2024-05-08T23:03:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.314+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f41a66d0 con 0x7f90f40fff60 2024-05-08T23:03:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.315+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f4109360 con 0x7f90f40fff60 2024-05-08T23:03:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.317+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f90e80105a0 con 0x7f90f40fff60 2024-05-08T23:03:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.318+0000 7f90f2000700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 0x7f90dc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.318+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90e8014070 con 0x7f90f40fff60 2024-05-08T23:03:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.318+0000 7f90f8e00700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 0x7f90dc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.319+0000 7f90f8e00700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 0x7f90dc0a3870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90f41034a0 tx=0x7f90e000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.321+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f90e8096340 con 0x7f90f40fff60 2024-05-08T23:03:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.540+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90f4102ed0 con 0x7f90f40fff60 2024-05-08T23:03:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.540+0000 7f90f2000700 1 -- 172.21.15.46:0/1176642564 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90e8099960 con 0x7f90f40fff60 2024-05-08T23:03:55.540 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 msgr2=0x7f90dc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 0x7f90dc0a3870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90f41034a0 tx=0x7f90e000b260 comp rx=0 tx=0).stop 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 msgr2=0x7f90f41019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e9a0 tx=0x7f90e800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 shutdown_connections 2024-05-08T23:03:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90dc0a13b0 0x7f90dc0a3870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4102470 0x7f90f41a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f40fff60 0x7f90f41019f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 --2- 172.21.15.46:0/1176642564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f4105080 0x7f90f4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 >> 172.21.15.46:0/1176642564 conn(0x7f90f40fb7a0 msgr2=0x7f90f4069970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 shutdown_connections 2024-05-08T23:03:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.543+0000 7f90fc200700 1 -- 172.21.15.46:0/1176642564 wait complete. 2024-05-08T23:03:55.562 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T23:03:55.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.761+0000 7ff194000700 1 -- 172.21.15.46:0/4270434001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c101070 msgr2=0x7ff18c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.761+0000 7ff194000700 1 --2- 172.21.15.46:0/4270434001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c101070 0x7ff18c101450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1800098f0 tx=0x7ff180009c00 comp rx=0 tx=0).stop 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 -- 172.21.15.46:0/4270434001 shutdown_connections 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 --2- 172.21.15.46:0/4270434001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c106190 0x7ff18c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 --2- 172.21.15.46:0/4270434001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c101070 0x7ff18c101450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 --2- 172.21.15.46:0/4270434001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c101a20 0x7ff18c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 -- 172.21.15.46:0/4270434001 >> 172.21.15.46:0/4270434001 conn(0x7ff18c0fc930 msgr2=0x7ff18c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 -- 172.21.15.46:0/4270434001 shutdown_connections 2024-05-08T23:03:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.762+0000 7ff194000700 1 -- 172.21.15.46:0/4270434001 wait complete. 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 Processor -- start 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 -- start start 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 0x7ff18c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c073d20 0x7ff18c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1a5be0 con 0x7ff18c073d20 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1a5d50 con 0x7ff18c101070 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff194000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1a5f10 con 0x7ff18c106190 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff191600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff191600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56906/0 (socket says 172.21.15.46:56906) 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff191600700 1 -- 172.21.15.46:0/3876262474 learned_addr learned my addr 172.21.15.46:0/3876262474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff192000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c073d20 0x7ff18c1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.763+0000 7ff190c00700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 0x7ff18c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 -- 172.21.15.46:0/3876262474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 msgr2=0x7ff18c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 0x7ff18c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 -- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c073d20 msgr2=0x7ff18c1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c073d20 0x7ff18c1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 -- 172.21.15.46:0/3876262474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009520 con 0x7ff18c101070 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff190c00700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 0x7ff18c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff191600700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff180005950 tx=0x7ff18000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18001c070 con 0x7ff18c101070 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c1a6190 con 0x7ff18c101070 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.764+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18000bac0 con 0x7ff18c101070 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.765+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800038e0 con 0x7ff18c101070 2024-05-08T23:03:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.765+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c1a6610 con 0x7ff18c101070 2024-05-08T23:03:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.765+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c1093b0 con 0x7ff18c101070 2024-05-08T23:03:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.768+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7ff180020030 con 0x7ff18c101070 2024-05-08T23:03:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.769+0000 7ff18a000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 0x7ff1740a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.769+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff1800ce5f0 con 0x7ff18c101070 2024-05-08T23:03:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.769+0000 7ff190c00700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 0x7ff1740a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.770+0000 7ff190c00700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 0x7ff1740a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0749d0 tx=0x7ff178009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:55.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:55.771+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff180097470 con 0x7ff18c101070 2024-05-08T23:03:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:55 smithi086 ceph-mon[78610]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:55 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3112506670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:55.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:55 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1176642564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:55 smithi053 ceph-mon[77587]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:55 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3112506670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:56.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:55 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1176642564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:55 smithi046 ceph-mon[36898]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:03:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:55 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3112506670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:03:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:55 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1176642564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:03:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.042+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff18c102540 con 0x7ff18c101070 2024-05-08T23:03:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.043+0000 7ff18a000700 1 -- 172.21.15.46:0/3876262474 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10904 (secure 0 0 0) 0x7ff18009aa90 con 0x7ff18c101070 2024-05-08T23:03:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:56.045 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:03:50.080154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 msgr2=0x7ff1740a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 0x7ff1740a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0749d0 tx=0x7ff178009310 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 msgr2=0x7ff18c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff180005950 tx=0x7ff18000fa00 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 shutdown_connections 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1740a13b0 0x7ff1740a3870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c073d20 0x7ff18c1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff18c101070 0x7ff18c0732a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 --2- 172.21.15.46:0/3876262474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff18c106190 0x7ff18c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 >> 172.21.15.46:0/3876262474 conn(0x7ff18c0fc930 msgr2=0x7ff18c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 shutdown_connections 2024-05-08T23:03:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.046+0000 7ff194000700 1 -- 172.21.15.46:0/3876262474 wait complete. 2024-05-08T23:03:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-05-08T23:03:56.065 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T23:03:56.065 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T23:03:56.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:03:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 -- 172.21.15.46:0/935647972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0101070 msgr2=0x7f3ac0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/935647972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0101070 0x7f3ac0101450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009990 tx=0x7f3ab4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 -- 172.21.15.46:0/935647972 shutdown_connections 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/935647972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac0106190 0x7f3ac010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/935647972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0101070 0x7f3ac0101450 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/935647972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101a20 0x7f3ac0105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 -- 172.21.15.46:0/935647972 >> 172.21.15.46:0/935647972 conn(0x7f3ac00fc930 msgr2=0x7f3ac00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 -- 172.21.15.46:0/935647972 shutdown_connections 2024-05-08T23:03:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.265+0000 7f3ac7a00700 1 -- 172.21.15.46:0/935647972 wait complete. 2024-05-08T23:03:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.266+0000 7f3ac7a00700 1 Processor -- start 2024-05-08T23:03:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.266+0000 7f3ac7a00700 1 -- start start 2024-05-08T23:03:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 0x7f3ac010acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac01a3440 0x7f3ac01a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a3ec0 con 0x7f3ac0106190 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a4030 con 0x7f3ac0101070 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac7a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a41f0 con 0x7f3ac01a3440 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 0x7f3ac010acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56922/0 (socket says 172.21.15.46:56922) 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac5000700 1 -- 172.21.15.46:0/1215825404 learned_addr learned my addr 172.21.15.46:0/1215825404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3ac5a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac01a3440 0x7f3ac01a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 -- 172.21.15.46:0/1215825404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac01a3440 msgr2=0x7f3ac01a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac01a3440 0x7f3ac01a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 -- 172.21.15.46:0/1215825404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 msgr2=0x7f3ac010a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.267+0000 7f3abfe00700 1 -- 172.21.15.46:0/1215825404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40095f0 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3ac5000700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3abfe00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 0x7f3ac010acd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9c0 tx=0x7f3aac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00c9c0 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac01aa550 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac010430 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac018680 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.268+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac01aaa30 con 0x7f3ac0106190 2024-05-08T23:03:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.269+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac0102540 con 0x7f3ac0106190 2024-05-08T23:03:56.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.271+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f3aac00cb20 con 0x7f3ac0106190 2024-05-08T23:03:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.272+0000 7f3abd600700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 0x7f3aa80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.272+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3aac014070 con 0x7f3ac0106190 2024-05-08T23:03:56.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.273+0000 7f3ac5000700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 0x7f3aa80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.273+0000 7f3ac5000700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 0x7f3aa80a3870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4006010 tx=0x7f3ab400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.277+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3aac052d60 con 0x7f3ac0106190 2024-05-08T23:03:56.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.491+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ac0101a20 con 0x7f3aa80a13b0 2024-05-08T23:03:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.508+0000 7f3abd600700 1 -- 172.21.15.46:0/1215825404 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426193 (secure 0 0 0) 0x7f3ac0101a20 con 0x7f3aa80a13b0 2024-05-08T23:03:56.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 msgr2=0x7f3aa80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 0x7f3aa80a3870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4006010 tx=0x7f3ab400b260 comp rx=0 tx=0).stop 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 msgr2=0x7f3ac010acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 0x7f3ac010acd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9c0 tx=0x7f3aac00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 shutdown_connections 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac01a3440 0x7f3ac01a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0101070 0x7f3ac010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3aa80a13b0 0x7f3aa80a3870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.513+0000 7f3ac7a00700 1 --2- 172.21.15.46:0/1215825404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac0106190 0x7f3ac010acd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.514+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 >> 172.21.15.46:0/1215825404 conn(0x7f3ac00fc930 msgr2=0x7f3ac00fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.514+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 shutdown_connections 2024-05-08T23:03:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.514+0000 7f3ac7a00700 1 -- 172.21.15.46:0/1215825404 wait complete. 2024-05-08T23:03:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:03:56.533 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T23:03:56.092339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192743,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1323,"num_read_kb":990,"num_write":574,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":951,"ondisk_log_size":951,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373068,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568756,"statfs":{"total":1151940427776,"available":1151558406144,"internally_reserved":0,"allocated":12271616,"data_stored":9487461,"data_compressed":507234,"data_compressed_allocated":688128,"data_compressed_original":4227072,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":239,"num_write":29,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.984004"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142984+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:03:48.142984+0000","last_peered":"2024-05-08T23:03:48.142984+0000","last_clean":"2024-05-08T23:03:48.142984+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:03:48.142984+0000","last_undegraded":"2024-05-08T23:03:48.142984+0000","last_fullsized":"2024-05-08T23:03:48.142984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137182+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:03:48.137182+0000","last_peered":"2024-05-08T23:03:48.137182+0000","last_clean":"2024-05-08T23:03:48.137182+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:03:48.137182+0000","last_undegraded":"2024-05-08T23:03:48.137182+0000","last_fullsized":"2024-05-08T23:03:48.137182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134416+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:03:48.134416+0000","last_peered":"2024-05-08T23:03:48.134416+0000","last_clean":"2024-05-08T23:03:48.134416+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:03:48.134416+0000","last_undegraded":"2024-05-08T23:03:48.134416+0000","last_fullsized":"2024-05-08T23:03:48.134416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129246+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:03:48.129246+0000","last_peered":"2024-05-08T23:03:48.129246+0000","last_clean":"2024-05-08T23:03:48.129246+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:03:48.129246+0000","last_undegraded":"2024-05-08T23:03:48.129246+0000","last_fullsized":"2024-05-08T23:03:48.129246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128870+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:03:48.128870+0000","last_peered":"2024-05-08T23:03:48.128870+0000","last_clean":"2024-05-08T23:03:48.128870+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:03:48.128870+0000","last_undegraded":"2024-05-08T23:03:48.128870+0000","last_fullsized":"2024-05-08T23:03:48.128870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123525+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:03:48.123525+0000","last_peered":"2024-05-08T23:03:48.123525+0000","last_clean":"2024-05-08T23:03:48.123525+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:03:48.123525+0000","last_undegraded":"2024-05-08T23:03:48.123525+0000","last_fullsized":"2024-05-08T23:03:48.123525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120109+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:03:48.120109+0000","last_peered":"2024-05-08T23:03:48.120109+0000","last_clean":"2024-05-08T23:03:48.120109+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:03:48.120109+0000","last_undegraded":"2024-05-08T23:03:48.120109+0000","last_fullsized":"2024-05-08T23:03:48.120109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143018+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:03:48.143018+0000","last_peered":"2024-05-08T23:03:48.143018+0000","last_clean":"2024-05-08T23:03:48.143018+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:03:48.143018+0000","last_undegraded":"2024-05-08T23:03:48.143018+0000","last_fullsized":"2024-05-08T23:03:48.143018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.116240+0000","last_change":"2024-05-08T23:03:19.568519+0000","last_active":"2024-05-08T23:03:48.116240+0000","last_peered":"2024-05-08T23:03:48.116240+0000","last_clean":"2024-05-08T23:03:48.116240+0000","last_became_active":"2024-05-08T23:03:19.567932+0000","last_became_peered":"2024-05-08T23:03:19.567932+0000","last_unstale":"2024-05-08T23:03:48.116240+0000","last_undegraded":"2024-05-08T23:03:48.116240+0000","last_fullsized":"2024-05-08T23:03:48.116240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:18.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123210+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:03:48.123210+0000","last_peered":"2024-05-08T23:03:48.123210+0000","last_clean":"2024-05-08T23:03:48.123210+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:03:48.123210+0000","last_undegraded":"2024-05-08T23:03:48.123210+0000","last_fullsized":"2024-05-08T23:03:48.123210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132775+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:03:48.132775+0000","last_peered":"2024-05-08T23:03:48.132775+0000","last_clean":"2024-05-08T23:03:48.132775+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:03:48.132775+0000","last_undegraded":"2024-05-08T23:03:48.132775+0000","last_fullsized":"2024-05-08T23:03:48.132775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128936+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:03:48.128936+0000","last_peered":"2024-05-08T23:03:48.128936+0000","last_clean":"2024-05-08T23:03:48.128936+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:03:48.128936+0000","last_undegraded":"2024-05-08T23:03:48.128936+0000","last_fullsized":"2024-05-08T23:03:48.128936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151844+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:03:48.151844+0000","last_peered":"2024-05-08T23:03:48.151844+0000","last_clean":"2024-05-08T23:03:48.151844+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:03:48.151844+0000","last_undegraded":"2024-05-08T23:03:48.151844+0000","last_fullsized":"2024-05-08T23:03:48.151844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.136583+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:03:48.136583+0000","last_peered":"2024-05-08T23:03:48.136583+0000","last_clean":"2024-05-08T23:03:48.136583+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:03:48.136583+0000","last_undegraded":"2024-05-08T23:03:48.136583+0000","last_fullsized":"2024-05-08T23:03:48.136583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141698+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:03:48.141698+0000","last_peered":"2024-05-08T23:03:48.141698+0000","last_clean":"2024-05-08T23:03:48.141698+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:03:48.141698+0000","last_undegraded":"2024-05-08T23:03:48.141698+0000","last_fullsized":"2024-05-08T23:03:48.141698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135586+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:03:48.135586+0000","last_peered":"2024-05-08T23:03:48.135586+0000","last_clean":"2024-05-08T23:03:48.135586+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:03:48.135586+0000","last_undegraded":"2024-05-08T23:03:48.135586+0000","last_fullsized":"2024-05-08T23:03:48.135586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143079+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:03:48.143079+0000","last_peered":"2024-05-08T23:03:48.143079+0000","last_clean":"2024-05-08T23:03:48.143079+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:03:48.143079+0000","last_undegraded":"2024-05-08T23:03:48.143079+0000","last_fullsized":"2024-05-08T23:03:48.143079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129439+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:03:48.129439+0000","last_peered":"2024-05-08T23:03:48.129439+0000","last_clean":"2024-05-08T23:03:48.129439+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:03:48.129439+0000","last_undegraded":"2024-05-08T23:03:48.129439+0000","last_fullsized":"2024-05-08T23:03:48.129439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137675+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:03:48.137675+0000","last_peered":"2024-05-08T23:03:48.137675+0000","last_clean":"2024-05-08T23:03:48.137675+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:03:48.137675+0000","last_undegraded":"2024-05-08T23:03:48.137675+0000","last_fullsized":"2024-05-08T23:03:48.137675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649303+0000","last_change":"2024-05-08T23:03:19.648891+0000","last_active":"2024-05-08T23:03:19.649303+0000","last_peered":"2024-05-08T23:03:19.649303+0000","last_clean":"2024-05-08T23:03:19.649303+0000","last_became_active":"2024-05-08T23:03:19.648436+0000","last_became_peered":"2024-05-08T23:03:19.648436+0000","last_unstale":"2024-05-08T23:03:19.649303+0000","last_undegraded":"2024-05-08T23:03:19.649303+0000","last_fullsized":"2024-05-08T23:03:19.649303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:39.693182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771820+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:03:53.771820+0000","last_peered":"2024-05-08T23:03:53.771820+0000","last_clean":"2024-05-08T23:03:53.771820+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:03:53.771820+0000","last_undegraded":"2024-05-08T23:03:53.771820+0000","last_fullsized":"2024-05-08T23:03:53.771820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143528+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:03:48.143528+0000","last_peered":"2024-05-08T23:03:48.143528+0000","last_clean":"2024-05-08T23:03:48.143528+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:03:48.143528+0000","last_undegraded":"2024-05-08T23:03:48.143528+0000","last_fullsized":"2024-05-08T23:03:48.143528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648069+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:03:19.648069+0000","last_peered":"2024-05-08T23:03:19.648069+0000","last_clean":"2024-05-08T23:03:19.648069+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:03:19.648069+0000","last_undegraded":"2024-05-08T23:03:19.648069+0000","last_fullsized":"2024-05-08T23:03:19.648069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.088831+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:03:48.088831+0000","last_peered":"2024-05-08T23:03:48.088831+0000","last_clean":"2024-05-08T23:03:48.088831+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:03:48.088831+0000","last_undegraded":"2024-05-08T23:03:48.088831+0000","last_fullsized":"2024-05-08T23:03:48.088831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.139631+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:03:48.139631+0000","last_peered":"2024-05-08T23:03:48.139631+0000","last_clean":"2024-05-08T23:03:48.139631+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:03:48.139631+0000","last_undegraded":"2024-05-08T23:03:48.139631+0000","last_fullsized":"2024-05-08T23:03:48.139631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648708+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:03:19.648708+0000","last_peered":"2024-05-08T23:03:19.648708+0000","last_clean":"2024-05-08T23:03:19.648708+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:03:19.648708+0000","last_undegraded":"2024-05-08T23:03:19.648708+0000","last_fullsized":"2024-05-08T23:03:19.648708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:47.213419+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:03:47.213419+0000","last_peered":"2024-05-08T23:03:47.213419+0000","last_clean":"2024-05-08T23:03:47.213419+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:03:47.213419+0000","last_undegraded":"2024-05-08T23:03:47.213419+0000","last_fullsized":"2024-05-08T23:03:47.213419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145570+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:03:48.145570+0000","last_peered":"2024-05-08T23:03:48.145570+0000","last_clean":"2024-05-08T23:03:48.145570+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:03:48.145570+0000","last_undegraded":"2024-05-08T23:03:48.145570+0000","last_fullsized":"2024-05-08T23:03:48.145570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648176+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:03:19.648176+0000","last_peered":"2024-05-08T23:03:19.648176+0000","last_clean":"2024-05-08T23:03:19.648176+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:03:19.648176+0000","last_undegraded":"2024-05-08T23:03:19.648176+0000","last_fullsized":"2024-05-08T23:03:19.648176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319178+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:03:19.319178+0000","last_peered":"2024-05-08T23:03:19.319178+0000","last_clean":"2024-05-08T23:03:19.319178+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:03:19.319178+0000","last_undegraded":"2024-05-08T23:03:19.319178+0000","last_fullsized":"2024-05-08T23:03:19.319178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132478+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:03:48.132478+0000","last_peered":"2024-05-08T23:03:48.132478+0000","last_clean":"2024-05-08T23:03:48.132478+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:03:48.132478+0000","last_undegraded":"2024-05-08T23:03:48.132478+0000","last_fullsized":"2024-05-08T23:03:48.132478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569695+0000","last_change":"2024-05-08T23:03:19.568929+0000","last_active":"2024-05-08T23:03:19.569695+0000","last_peered":"2024-05-08T23:03:19.569695+0000","last_clean":"2024-05-08T23:03:19.569695+0000","last_became_active":"2024-05-08T23:03:19.568460+0000","last_became_peered":"2024-05-08T23:03:19.568460+0000","last_unstale":"2024-05-08T23:03:19.569695+0000","last_undegraded":"2024-05-08T23:03:19.569695+0000","last_fullsized":"2024-05-08T23:03:19.569695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:30.975740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771668+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:03:52.771668+0000","last_peered":"2024-05-08T23:03:52.771668+0000","last_clean":"2024-05-08T23:03:52.771668+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:03:52.771668+0000","last_undegraded":"2024-05-08T23:03:52.771668+0000","last_fullsized":"2024-05-08T23:03:52.771668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.111635+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:03:48.111635+0000","last_peered":"2024-05-08T23:03:48.111635+0000","last_clean":"2024-05-08T23:03:48.111635+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:03:48.111635+0000","last_undegraded":"2024-05-08T23:03:48.111635+0000","last_fullsized":"2024-05-08T23:03:48.111635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026600+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:03:19.026600+0000","last_peered":"2024-05-08T23:03:19.026600+0000","last_clean":"2024-05-08T23:03:19.026600+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:03:19.026600+0000","last_undegraded":"2024-05-08T23:03:19.026600+0000","last_fullsized":"2024-05-08T23:03:19.026600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029107+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:03:19.029107+0000","last_peered":"2024-05-08T23:03:19.029107+0000","last_clean":"2024-05-08T23:03:19.029107+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:03:19.029107+0000","last_undegraded":"2024-05-08T23:03:19.029107+0000","last_fullsized":"2024-05-08T23:03:19.029107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.144393+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:03:48.144393+0000","last_peered":"2024-05-08T23:03:48.144393+0000","last_clean":"2024-05-08T23:03:48.144393+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:03:48.144393+0000","last_undegraded":"2024-05-08T23:03:48.144393+0000","last_fullsized":"2024-05-08T23:03:48.144393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.567021+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:03:19.567021+0000","last_peered":"2024-05-08T23:03:19.567021+0000","last_clean":"2024-05-08T23:03:19.567021+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:03:19.567021+0000","last_undegraded":"2024-05-08T23:03:19.567021+0000","last_fullsized":"2024-05-08T23:03:19.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026129+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:03:19.026129+0000","last_peered":"2024-05-08T23:03:19.026129+0000","last_clean":"2024-05-08T23:03:19.026129+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:03:19.026129+0000","last_undegraded":"2024-05-08T23:03:19.026129+0000","last_fullsized":"2024-05-08T23:03:19.026129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142934+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:03:48.142934+0000","last_peered":"2024-05-08T23:03:48.142934+0000","last_clean":"2024-05-08T23:03:48.142934+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:03:48.142934+0000","last_undegraded":"2024-05-08T23:03:48.142934+0000","last_fullsized":"2024-05-08T23:03:48.142934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647859+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:03:19.647859+0000","last_peered":"2024-05-08T23:03:19.647859+0000","last_clean":"2024-05-08T23:03:19.647859+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:03:19.647859+0000","last_undegraded":"2024-05-08T23:03:19.647859+0000","last_fullsized":"2024-05-08T23:03:19.647859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228997+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:03:19.228997+0000","last_peered":"2024-05-08T23:03:19.228997+0000","last_clean":"2024-05-08T23:03:19.228997+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:03:19.228997+0000","last_undegraded":"2024-05-08T23:03:19.228997+0000","last_fullsized":"2024-05-08T23:03:19.228997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.127009+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:03:48.127009+0000","last_peered":"2024-05-08T23:03:48.127009+0000","last_clean":"2024-05-08T23:03:48.127009+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:03:48.127009+0000","last_undegraded":"2024-05-08T23:03:48.127009+0000","last_fullsized":"2024-05-08T23:03:48.127009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024203+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:03:19.024203+0000","last_peered":"2024-05-08T23:03:19.024203+0000","last_clean":"2024-05-08T23:03:19.024203+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:03:19.024203+0000","last_undegraded":"2024-05-08T23:03:19.024203+0000","last_fullsized":"2024-05-08T23:03:19.024203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":217,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771761+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:03:53.771761+0000","last_peered":"2024-05-08T23:03:53.771761+0000","last_clean":"2024-05-08T23:03:53.771761+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:03:53.771761+0000","last_undegraded":"2024-05-08T23:03:53.771761+0000","last_fullsized":"2024-05-08T23:03:53.771761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133767+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:03:48.133767+0000","last_peered":"2024-05-08T23:03:48.133767+0000","last_clean":"2024-05-08T23:03:48.133767+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:03:48.133767+0000","last_undegraded":"2024-05-08T23:03:48.133767+0000","last_fullsized":"2024-05-08T23:03:48.133767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029046+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:03:19.029046+0000","last_peered":"2024-05-08T23:03:19.029046+0000","last_clean":"2024-05-08T23:03:19.029046+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:03:19.029046+0000","last_undegraded":"2024-05-08T23:03:19.029046+0000","last_fullsized":"2024-05-08T23:03:19.029046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025153+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:03:19.025153+0000","last_peered":"2024-05-08T23:03:19.025153+0000","last_clean":"2024-05-08T23:03:19.025153+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:03:19.025153+0000","last_undegraded":"2024-05-08T23:03:19.025153+0000","last_fullsized":"2024-05-08T23:03:19.025153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.140089+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:03:48.140089+0000","last_peered":"2024-05-08T23:03:48.140089+0000","last_clean":"2024-05-08T23:03:48.140089+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:03:48.140089+0000","last_undegraded":"2024-05-08T23:03:48.140089+0000","last_fullsized":"2024-05-08T23:03:48.140089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648854+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:03:19.648854+0000","last_peered":"2024-05-08T23:03:19.648854+0000","last_clean":"2024-05-08T23:03:19.648854+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:03:19.648854+0000","last_undegraded":"2024-05-08T23:03:19.648854+0000","last_fullsized":"2024-05-08T23:03:19.648854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229387+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:03:19.229387+0000","last_peered":"2024-05-08T23:03:19.229387+0000","last_clean":"2024-05-08T23:03:19.229387+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:03:19.229387+0000","last_undegraded":"2024-05-08T23:03:19.229387+0000","last_fullsized":"2024-05-08T23:03:19.229387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134729+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:03:48.134729+0000","last_peered":"2024-05-08T23:03:48.134729+0000","last_clean":"2024-05-08T23:03:48.134729+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:03:48.134729+0000","last_undegraded":"2024-05-08T23:03:48.134729+0000","last_fullsized":"2024-05-08T23:03:48.134729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.568583+0000","last_change":"2024-05-08T23:03:19.568159+0000","last_active":"2024-05-08T23:03:19.568583+0000","last_peered":"2024-05-08T23:03:19.568583+0000","last_clean":"2024-05-08T23:03:19.568583+0000","last_became_active":"2024-05-08T23:03:19.567769+0000","last_became_peered":"2024-05-08T23:03:19.567769+0000","last_unstale":"2024-05-08T23:03:19.568583+0000","last_undegraded":"2024-05-08T23:03:19.568583+0000","last_fullsized":"2024-05-08T23:03:19.568583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:53.373375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'40","reported_seq":259,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772557+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:03:53.772557+0000","last_peered":"2024-05-08T23:03:53.772557+0000","last_clean":"2024-05-08T23:03:53.772557+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:03:53.772557+0000","last_undegraded":"2024-05-08T23:03:53.772557+0000","last_fullsized":"2024-05-08T23:03:53.772557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67500,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":165,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.118936+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:03:48.118936+0000","last_peered":"2024-05-08T23:03:48.118936+0000","last_clean":"2024-05-08T23:03:48.118936+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:03:48.118936+0000","last_undegraded":"2024-05-08T23:03:48.118936+0000","last_fullsized":"2024-05-08T23:03:48.118936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229834+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:03:19.229834+0000","last_peered":"2024-05-08T23:03:19.229834+0000","last_clean":"2024-05-08T23:03:19.229834+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:03:19.229834+0000","last_undegraded":"2024-05-08T23:03:19.229834+0000","last_fullsized":"2024-05-08T23:03:19.229834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026610+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:03:19.026610+0000","last_peered":"2024-05-08T23:03:19.026610+0000","last_clean":"2024-05-08T23:03:19.026610+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:03:19.026610+0000","last_undegraded":"2024-05-08T23:03:19.026610+0000","last_fullsized":"2024-05-08T23:03:19.026610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149247+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:03:48.149247+0000","last_peered":"2024-05-08T23:03:48.149247+0000","last_clean":"2024-05-08T23:03:48.149247+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:03:48.149247+0000","last_undegraded":"2024-05-08T23:03:48.149247+0000","last_fullsized":"2024-05-08T23:03:48.149247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566993+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:03:19.566993+0000","last_peered":"2024-05-08T23:03:19.566993+0000","last_clean":"2024-05-08T23:03:19.566993+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:03:19.566993+0000","last_undegraded":"2024-05-08T23:03:19.566993+0000","last_fullsized":"2024-05-08T23:03:19.566993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229573+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:03:19.229573+0000","last_peered":"2024-05-08T23:03:19.229573+0000","last_clean":"2024-05-08T23:03:19.229573+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:03:19.229573+0000","last_undegraded":"2024-05-08T23:03:19.229573+0000","last_fullsized":"2024-05-08T23:03:19.229573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150295+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:03:48.150295+0000","last_peered":"2024-05-08T23:03:48.150295+0000","last_clean":"2024-05-08T23:03:48.150295+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:03:48.150295+0000","last_undegraded":"2024-05-08T23:03:48.150295+0000","last_fullsized":"2024-05-08T23:03:48.150295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030071+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:03:19.030071+0000","last_peered":"2024-05-08T23:03:19.030071+0000","last_clean":"2024-05-08T23:03:19.030071+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:03:19.030071+0000","last_undegraded":"2024-05-08T23:03:19.030071+0000","last_fullsized":"2024-05-08T23:03:19.030071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'10","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771913+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:03:52.771913+0000","last_peered":"2024-05-08T23:03:52.771913+0000","last_clean":"2024-05-08T23:03:52.771913+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:03:52.771913+0000","last_undegraded":"2024-05-08T23:03:52.771913+0000","last_fullsized":"2024-05-08T23:03:52.771913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2253,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125794+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:03:48.125794+0000","last_peered":"2024-05-08T23:03:48.125794+0000","last_clean":"2024-05-08T23:03:48.125794+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:03:48.125794+0000","last_undegraded":"2024-05-08T23:03:48.125794+0000","last_fullsized":"2024-05-08T23:03:48.125794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569288+0000","last_change":"2024-05-08T23:03:19.568544+0000","last_active":"2024-05-08T23:03:19.569288+0000","last_peered":"2024-05-08T23:03:19.569288+0000","last_clean":"2024-05-08T23:03:19.569288+0000","last_became_active":"2024-05-08T23:03:19.568087+0000","last_became_peered":"2024-05-08T23:03:19.568087+0000","last_unstale":"2024-05-08T23:03:19.569288+0000","last_undegraded":"2024-05-08T23:03:19.569288+0000","last_fullsized":"2024-05-08T23:03:19.569288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:45.970629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.318742+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:03:19.318742+0000","last_peered":"2024-05-08T23:03:19.318742+0000","last_clean":"2024-05-08T23:03:19.318742+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:03:19.318742+0000","last_undegraded":"2024-05-08T23:03:19.318742+0000","last_fullsized":"2024-05-08T23:03:19.318742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.122962+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:03:48.122962+0000","last_peered":"2024-05-08T23:03:48.122962+0000","last_clean":"2024-05-08T23:03:48.122962+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:03:48.122962+0000","last_undegraded":"2024-05-08T23:03:48.122962+0000","last_fullsized":"2024-05-08T23:03:48.122962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228903+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:03:19.228903+0000","last_peered":"2024-05-08T23:03:19.228903+0000","last_clean":"2024-05-08T23:03:19.228903+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:03:19.228903+0000","last_undegraded":"2024-05-08T23:03:19.228903+0000","last_fullsized":"2024-05-08T23:03:19.228903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093426+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:03:48.093426+0000","last_peered":"2024-05-08T23:03:48.093426+0000","last_clean":"2024-05-08T23:03:48.093426+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:03:48.093426+0000","last_undegraded":"2024-05-08T23:03:48.093426+0000","last_fullsized":"2024-05-08T23:03:48.093426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132064+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:03:48.132064+0000","last_peered":"2024-05-08T23:03:48.132064+0000","last_clean":"2024-05-08T23:03:48.132064+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:03:48.132064+0000","last_undegraded":"2024-05-08T23:03:48.132064+0000","last_fullsized":"2024-05-08T23:03:48.132064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319091+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:03:19.319091+0000","last_peered":"2024-05-08T23:03:19.319091+0000","last_clean":"2024-05-08T23:03:19.319091+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:03:19.319091+0000","last_undegraded":"2024-05-08T23:03:19.319091+0000","last_fullsized":"2024-05-08T23:03:19.319091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029438+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:03:19.029438+0000","last_peered":"2024-05-08T23:03:19.029438+0000","last_clean":"2024-05-08T23:03:19.029438+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:03:19.029438+0000","last_undegraded":"2024-05-08T23:03:19.029438+0000","last_fullsized":"2024-05-08T23:03:19.029438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134037+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:03:48.134037+0000","last_peered":"2024-05-08T23:03:48.134037+0000","last_clean":"2024-05-08T23:03:48.134037+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:03:48.134037+0000","last_undegraded":"2024-05-08T23:03:48.134037+0000","last_fullsized":"2024-05-08T23:03:48.134037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026364+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:03:19.026364+0000","last_peered":"2024-05-08T23:03:19.026364+0000","last_clean":"2024-05-08T23:03:19.026364+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:03:19.026364+0000","last_undegraded":"2024-05-08T23:03:19.026364+0000","last_fullsized":"2024-05-08T23:03:19.026364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":126,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771144+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:03:52.771144+0000","last_peered":"2024-05-08T23:03:52.771144+0000","last_clean":"2024-05-08T23:03:52.771144+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:03:52.771144+0000","last_undegraded":"2024-05-08T23:03:52.771144+0000","last_fullsized":"2024-05-08T23:03:52.771144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1243,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135922+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:03:48.135922+0000","last_peered":"2024-05-08T23:03:48.135922+0000","last_clean":"2024-05-08T23:03:48.135922+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:03:48.135922+0000","last_undegraded":"2024-05-08T23:03:48.135922+0000","last_fullsized":"2024-05-08T23:03:48.135922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026697+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:03:19.026697+0000","last_peered":"2024-05-08T23:03:19.026697+0000","last_clean":"2024-05-08T23:03:19.026697+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:03:19.026697+0000","last_undegraded":"2024-05-08T23:03:19.026697+0000","last_fullsized":"2024-05-08T23:03:19.026697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120851+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:03:48.120851+0000","last_peered":"2024-05-08T23:03:48.120851+0000","last_clean":"2024-05-08T23:03:48.120851+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:03:48.120851+0000","last_undegraded":"2024-05-08T23:03:48.120851+0000","last_fullsized":"2024-05-08T23:03:48.120851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":124,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772498+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:03:53.772498+0000","last_peered":"2024-05-08T23:03:53.772498+0000","last_clean":"2024-05-08T23:03:53.772498+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:03:53.772498+0000","last_undegraded":"2024-05-08T23:03:53.772498+0000","last_fullsized":"2024-05-08T23:03:53.772498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024808+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:03:19.024808+0000","last_peered":"2024-05-08T23:03:19.024808+0000","last_clean":"2024-05-08T23:03:19.024808+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:03:19.024808+0000","last_undegraded":"2024-05-08T23:03:19.024808+0000","last_fullsized":"2024-05-08T23:03:19.024808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137285+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:03:48.137285+0000","last_peered":"2024-05-08T23:03:48.137285+0000","last_clean":"2024-05-08T23:03:48.137285+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:03:48.137285+0000","last_undegraded":"2024-05-08T23:03:48.137285+0000","last_fullsized":"2024-05-08T23:03:48.137285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648728+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:03:19.648728+0000","last_peered":"2024-05-08T23:03:19.648728+0000","last_clean":"2024-05-08T23:03:19.648728+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:03:19.648728+0000","last_undegraded":"2024-05-08T23:03:19.648728+0000","last_fullsized":"2024-05-08T23:03:19.648728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":385,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:50.222340+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:03:50.222340+0000","last_peered":"2024-05-08T23:03:50.222340+0000","last_clean":"2024-05-08T23:03:50.222340+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:03:50.222340+0000","last_undegraded":"2024-05-08T23:03:50.222340+0000","last_fullsized":"2024-05-08T23:03:50.222340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229833+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:03:19.229833+0000","last_peered":"2024-05-08T23:03:19.229833+0000","last_clean":"2024-05-08T23:03:19.229833+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:03:19.229833+0000","last_undegraded":"2024-05-08T23:03:19.229833+0000","last_fullsized":"2024-05-08T23:03:19.229833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141189+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:03:48.141189+0000","last_peered":"2024-05-08T23:03:48.141189+0000","last_clean":"2024-05-08T23:03:48.141189+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:03:48.141189+0000","last_undegraded":"2024-05-08T23:03:48.141189+0000","last_fullsized":"2024-05-08T23:03:48.141189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566669+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:03:19.566669+0000","last_peered":"2024-05-08T23:03:19.566669+0000","last_clean":"2024-05-08T23:03:19.566669+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:03:19.566669+0000","last_undegraded":"2024-05-08T23:03:19.566669+0000","last_fullsized":"2024-05-08T23:03:19.566669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566734+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:03:19.566734+0000","last_peered":"2024-05-08T23:03:19.566734+0000","last_clean":"2024-05-08T23:03:19.566734+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:03:19.566734+0000","last_undegraded":"2024-05-08T23:03:19.566734+0000","last_fullsized":"2024-05-08T23:03:19.566734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319413+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:03:19.319413+0000","last_peered":"2024-05-08T23:03:19.319413+0000","last_clean":"2024-05-08T23:03:19.319413+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:03:19.319413+0000","last_undegraded":"2024-05-08T23:03:19.319413+0000","last_fullsized":"2024-05-08T23:03:19.319413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142913+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:03:48.142913+0000","last_peered":"2024-05-08T23:03:48.142913+0000","last_clean":"2024-05-08T23:03:48.142913+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:03:48.142913+0000","last_undegraded":"2024-05-08T23:03:48.142913+0000","last_fullsized":"2024-05-08T23:03:48.142913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023762+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:03:19.023762+0000","last_peered":"2024-05-08T23:03:19.023762+0000","last_clean":"2024-05-08T23:03:19.023762+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:03:19.023762+0000","last_undegraded":"2024-05-08T23:03:19.023762+0000","last_fullsized":"2024-05-08T23:03:19.023762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'45","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.383589+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:03:48.383589+0000","last_peered":"2024-05-08T23:03:48.383589+0000","last_clean":"2024-05-08T23:03:48.383589+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:03:48.383589+0000","last_undegraded":"2024-05-08T23:03:48.383589+0000","last_fullsized":"2024-05-08T23:03:48.383589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143365+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:03:48.143365+0000","last_peered":"2024-05-08T23:03:48.143365+0000","last_clean":"2024-05-08T23:03:48.143365+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:03:48.143365+0000","last_undegraded":"2024-05-08T23:03:48.143365+0000","last_fullsized":"2024-05-08T23:03:48.143365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025789+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:03:19.025789+0000","last_peered":"2024-05-08T23:03:19.025789+0000","last_clean":"2024-05-08T23:03:19.025789+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:03:19.025789+0000","last_undegraded":"2024-05-08T23:03:19.025789+0000","last_fullsized":"2024-05-08T23:03:19.025789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649256+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:03:19.649256+0000","last_peered":"2024-05-08T23:03:19.649256+0000","last_clean":"2024-05-08T23:03:19.649256+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:03:19.649256+0000","last_undegraded":"2024-05-08T23:03:19.649256+0000","last_fullsized":"2024-05-08T23:03:19.649256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145687+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:03:48.145687+0000","last_peered":"2024-05-08T23:03:48.145687+0000","last_clean":"2024-05-08T23:03:48.145687+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:03:48.145687+0000","last_undegraded":"2024-05-08T23:03:48.145687+0000","last_fullsized":"2024-05-08T23:03:48.145687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649029+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:03:19.649029+0000","last_peered":"2024-05-08T23:03:19.649029+0000","last_clean":"2024-05-08T23:03:19.649029+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:03:19.649029+0000","last_undegraded":"2024-05-08T23:03:19.649029+0000","last_fullsized":"2024-05-08T23:03:19.649029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648003+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:03:19.648003+0000","last_peered":"2024-05-08T23:03:19.648003+0000","last_clean":"2024-05-08T23:03:19.648003+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:03:19.648003+0000","last_undegraded":"2024-05-08T23:03:19.648003+0000","last_fullsized":"2024-05-08T23:03:19.648003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093132+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:03:48.093132+0000","last_peered":"2024-05-08T23:03:48.093132+0000","last_clean":"2024-05-08T23:03:48.093132+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:03:48.093132+0000","last_undegraded":"2024-05-08T23:03:48.093132+0000","last_fullsized":"2024-05-08T23:03:48.093132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030035+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:03:19.030035+0000","last_peered":"2024-05-08T23:03:19.030035+0000","last_clean":"2024-05-08T23:03:19.030035+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:03:19.030035+0000","last_undegraded":"2024-05-08T23:03:19.030035+0000","last_fullsized":"2024-05-08T23:03:19.030035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023898+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:03:19.023898+0000","last_peered":"2024-05-08T23:03:19.023898+0000","last_clean":"2024-05-08T23:03:19.023898+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:03:19.023898+0000","last_undegraded":"2024-05-08T23:03:19.023898+0000","last_fullsized":"2024-05-08T23:03:19.023898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142729+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:03:48.142729+0000","last_peered":"2024-05-08T23:03:48.142729+0000","last_clean":"2024-05-08T23:03:48.142729+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:03:48.142729+0000","last_undegraded":"2024-05-08T23:03:48.142729+0000","last_fullsized":"2024-05-08T23:03:48.142729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648771+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:03:19.648771+0000","last_peered":"2024-05-08T23:03:19.648771+0000","last_clean":"2024-05-08T23:03:19.648771+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:03:19.648771+0000","last_undegraded":"2024-05-08T23:03:19.648771+0000","last_fullsized":"2024-05-08T23:03:19.648771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026057+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:03:19.026057+0000","last_peered":"2024-05-08T23:03:19.026057+0000","last_clean":"2024-05-08T23:03:19.026057+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:03:19.026057+0000","last_undegraded":"2024-05-08T23:03:19.026057+0000","last_fullsized":"2024-05-08T23:03:19.026057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150336+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:03:48.150336+0000","last_peered":"2024-05-08T23:03:48.150336+0000","last_clean":"2024-05-08T23:03:48.150336+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:03:48.150336+0000","last_undegraded":"2024-05-08T23:03:48.150336+0000","last_fullsized":"2024-05-08T23:03:48.150336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648702+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:03:19.648702+0000","last_peered":"2024-05-08T23:03:19.648702+0000","last_clean":"2024-05-08T23:03:19.648702+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:03:19.648702+0000","last_undegraded":"2024-05-08T23:03:19.648702+0000","last_fullsized":"2024-05-08T23:03:19.648702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319031+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:03:19.319031+0000","last_peered":"2024-05-08T23:03:19.319031+0000","last_clean":"2024-05-08T23:03:19.319031+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:03:19.319031+0000","last_undegraded":"2024-05-08T23:03:19.319031+0000","last_fullsized":"2024-05-08T23:03:19.319031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.138554+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:03:48.138554+0000","last_peered":"2024-05-08T23:03:48.138554+0000","last_clean":"2024-05-08T23:03:48.138554+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:03:48.138554+0000","last_undegraded":"2024-05-08T23:03:48.138554+0000","last_fullsized":"2024-05-08T23:03:48.138554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649390+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:03:19.649390+0000","last_peered":"2024-05-08T23:03:19.649390+0000","last_clean":"2024-05-08T23:03:19.649390+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:03:19.649390+0000","last_undegraded":"2024-05-08T23:03:19.649390+0000","last_fullsized":"2024-05-08T23:03:19.649390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.770901+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:03:52.770901+0000","last_peered":"2024-05-08T23:03:52.770901+0000","last_clean":"2024-05-08T23:03:52.770901+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:03:52.770901+0000","last_undegraded":"2024-05-08T23:03:52.770901+0000","last_fullsized":"2024-05-08T23:03:52.770901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132905+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:03:48.132905+0000","last_peered":"2024-05-08T23:03:48.132905+0000","last_clean":"2024-05-08T23:03:48.132905+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:03:48.132905+0000","last_undegraded":"2024-05-08T23:03:48.132905+0000","last_fullsized":"2024-05-08T23:03:48.132905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647481+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:03:19.647481+0000","last_peered":"2024-05-08T23:03:19.647481+0000","last_clean":"2024-05-08T23:03:19.647481+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:03:19.647481+0000","last_undegraded":"2024-05-08T23:03:19.647481+0000","last_fullsized":"2024-05-08T23:03:19.647481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.377743+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:03:48.377743+0000","last_peered":"2024-05-08T23:03:48.377743+0000","last_clean":"2024-05-08T23:03:48.377743+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:03:48.377743+0000","last_undegraded":"2024-05-08T23:03:48.377743+0000","last_fullsized":"2024-05-08T23:03:48.377743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126672+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:03:48.126672+0000","last_peered":"2024-05-08T23:03:48.126672+0000","last_clean":"2024-05-08T23:03:48.126672+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:03:48.126672+0000","last_undegraded":"2024-05-08T23:03:48.126672+0000","last_fullsized":"2024-05-08T23:03:48.126672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228724+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:03:19.228724+0000","last_peered":"2024-05-08T23:03:19.228724+0000","last_clean":"2024-05-08T23:03:19.228724+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:03:19.228724+0000","last_undegraded":"2024-05-08T23:03:19.228724+0000","last_fullsized":"2024-05-08T23:03:19.228724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026628+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:03:19.026628+0000","last_peered":"2024-05-08T23:03:19.026628+0000","last_clean":"2024-05-08T23:03:19.026628+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:03:19.026628+0000","last_undegraded":"2024-05-08T23:03:19.026628+0000","last_fullsized":"2024-05-08T23:03:19.026628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126885+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:03:48.126885+0000","last_peered":"2024-05-08T23:03:48.126885+0000","last_clean":"2024-05-08T23:03:48.126885+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:03:48.126885+0000","last_undegraded":"2024-05-08T23:03:48.126885+0000","last_fullsized":"2024-05-08T23:03:48.126885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133238+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:03:48.133238+0000","last_peered":"2024-05-08T23:03:48.133238+0000","last_clean":"2024-05-08T23:03:48.133238+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:03:48.133238+0000","last_undegraded":"2024-05-08T23:03:48.133238+0000","last_fullsized":"2024-05-08T23:03:48.133238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124622+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:03:48.124622+0000","last_peered":"2024-05-08T23:03:48.124622+0000","last_clean":"2024-05-08T23:03:48.124622+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:03:48.124622+0000","last_undegraded":"2024-05-08T23:03:48.124622+0000","last_fullsized":"2024-05-08T23:03:48.124622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124829+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:03:48.124829+0000","last_peered":"2024-05-08T23:03:48.124829+0000","last_clean":"2024-05-08T23:03:48.124829+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:03:48.124829+0000","last_undegraded":"2024-05-08T23:03:48.124829+0000","last_fullsized":"2024-05-08T23:03:48.124829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150811+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:03:48.150811+0000","last_peered":"2024-05-08T23:03:48.150811+0000","last_clean":"2024-05-08T23:03:48.150811+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:03:48.150811+0000","last_undegraded":"2024-05-08T23:03:48.150811+0000","last_fullsized":"2024-05-08T23:03:48.150811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149706+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:03:48.149706+0000","last_peered":"2024-05-08T23:03:48.149706+0000","last_clean":"2024-05-08T23:03:48.149706+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:03:48.149706+0000","last_undegraded":"2024-05-08T23:03:48.149706+0000","last_fullsized":"2024-05-08T23:03:48.149706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.130871+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:03:48.130871+0000","last_peered":"2024-05-08T23:03:48.130871+0000","last_clean":"2024-05-08T23:03:48.130871+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:03:48.130871+0000","last_undegraded":"2024-05-08T23:03:48.130871+0000","last_fullsized":"2024-05-08T23:03:48.130871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125971+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:03:48.125971+0000","last_peered":"2024-05-08T23:03:48.125971+0000","last_clean":"2024-05-08T23:03:48.125971+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:03:48.125971+0000","last_undegraded":"2024-05-08T23:03:48.125971+0000","last_fullsized":"2024-05-08T23:03:48.125971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137011+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:03:48.137011+0000","last_peered":"2024-05-08T23:03:48.137011+0000","last_clean":"2024-05-08T23:03:48.137011+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:03:48.137011+0000","last_undegraded":"2024-05-08T23:03:48.137011+0000","last_fullsized":"2024-05-08T23:03:48.137011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143616+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:03:48.143616+0000","last_peered":"2024-05-08T23:03:48.143616+0000","last_clean":"2024-05-08T23:03:48.143616+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:03:48.143616+0000","last_undegraded":"2024-05-08T23:03:48.143616+0000","last_fullsized":"2024-05-08T23:03:48.143616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143122+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:03:48.143122+0000","last_peered":"2024-05-08T23:03:48.143122+0000","last_clean":"2024-05-08T23:03:48.143122+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:03:48.143122+0000","last_undegraded":"2024-05-08T23:03:48.143122+0000","last_fullsized":"2024-05-08T23:03:48.143122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134728+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:03:48.134728+0000","last_peered":"2024-05-08T23:03:48.134728+0000","last_clean":"2024-05-08T23:03:48.134728+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:03:48.134728+0000","last_undegraded":"2024-05-08T23:03:48.134728+0000","last_fullsized":"2024-05-08T23:03:48.134728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151641+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:03:48.151641+0000","last_peered":"2024-05-08T23:03:48.151641+0000","last_clean":"2024-05-08T23:03:48.151641+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:03:48.151641+0000","last_undegraded":"2024-05-08T23:03:48.151641+0000","last_fullsized":"2024-05-08T23:03:48.151641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.131074+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:03:48.131074+0000","last_peered":"2024-05-08T23:03:48.131074+0000","last_clean":"2024-05-08T23:03:48.131074+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:03:48.131074+0000","last_undegraded":"2024-05-08T23:03:48.131074+0000","last_fullsized":"2024-05-08T23:03:48.131074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107673,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602141,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1086,"num_read_kb":660,"num_write":371,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806423,"data_compressed":81453,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612642,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":970752,"data_stored":495994,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.755,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":1.095,"15min":1.095},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.72099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.56399999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85899999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.499},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93100000000000005,"5min":1.218,"15min":1.218},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0169999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.628,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.196,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.63100000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68500000000000005}]},{"osd":10,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.622,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":1.387,"15min":1.387},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.66000000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97099999999999997,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.66200000000000003}]}]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1282815,"data_compressed":70405,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.505}]},{"osd":1,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83799999999999997,"15min":0.82999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.69599999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.624}]},{"osd":9,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.85899999999999999},"min":{"1min":0.63,"5min":0.622,"15min":0.622},"max":{"1min":1.4279999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.90100000000000002}]},{"osd":11,"last update":"Wed May 8 23:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.82199999999999995},"min":{"1min":0.70999999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.008,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0700000000000001,"5min":1.091,"15min":1.091},"last":0.754}]}]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":892476,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":1.919,"15min":1.919},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84099999999999997,"5min":1.357,"15min":1.357},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71599999999999997,"15min":0.72799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.006,"5min":1.639,"15min":1.639},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93500000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.755}]},{"osd":2,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96499999999999997,"5min":1.488,"15min":1.488},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93899999999999995,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":1.371,"15min":1.371},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.71199999999999997,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95099999999999996,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.80400000000000005,"15min":0.84499999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89100000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.81899999999999995,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.077,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82799999999999996,"15min":0.86399999999999999},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.738,"15min":1.738},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.86799999999999999,"15min":0.90100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.038,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.89100000000000001,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95799999999999996,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.92000000000000004,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.008,"5min":1.673,"15min":1.673},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.89700000000000002,"15min":0.94499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.881,"15min":1.881},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.157,"5min":1.766,"15min":1.766},"last":0.77600000000000002}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.746}]}]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":888064,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.441},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.55200000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.47199999999999998},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.621},"min":{"1min":0.48499999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78500000000000003,"5min":1.079,"15min":1.079},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.372}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0660000000000001,"5min":1.139,"15min":1.139},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.629,"5min":0.441,"15min":0.441},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.497},{"interface":"front","average":{"1min":0.745,"5min":0.70399999999999996,"15min":0.69099999999999995},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89700000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.59299999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.53300000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.58699999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.877,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.62,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85499999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54000000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50700000000000001}]}]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":1094313,"data_compressed":62400,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.45000000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":1.109,"15min":1.109},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79400000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.622},"min":{"1min":0.58899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88800000000000001,"5min":1.05,"15min":1.05},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.89100000000000001}]},{"osd":8,"last update":"Wed May 8 23:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89000000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84899999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.91000000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96699999999999997,"5min":1.141,"15min":1.141},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98599999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.877}]},{"osd":11,"last update":"Wed May 8 23:03:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.01,"5min":1.052,"15min":1.052},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.85899999999999999,"15min":0.86099999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0329999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.95499999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":491323,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.504,"15min":0.46700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.503},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.0579999999999998,"5min":3.0579999999999998,"15min":3.0579999999999998},"last":0.91700000000000004}]},{"osd":1,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71899999999999997,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.67500000000000004},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.97699999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71499999999999997,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.093,"5min":0.76100000000000001,"15min":0.66800000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.67200000000000004,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68300000000000005,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.417,"5min":1.581,"15min":1.581},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.77100000000000002,"15min":0.74099999999999999},"min":{"1min":0.499,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.78100000000000003,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.343,"5min":3.343,"15min":3.343},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78900000000000003,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.72599999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3380000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.879}]},{"osd":7,"last update":"Wed May 8 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.83199999999999996,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.81699999999999995,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.377,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":0.91000000000000003,"15min":0.83399999999999996},"min":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.86899999999999999,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":0.61799999999999999}]},{"osd":9,"last update":"Wed May 8 23:03:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.93200000000000005,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.268,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.93700000000000006}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.006,"15min":0.98499999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.002,"15min":1.0109999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0569999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.9710000000000001,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":1040384,"data_stored":560216,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57599999999999996,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.60199999999999998,"15min":0.54200000000000004},"min":{"1min":0.44600000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 23:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.878,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.626,"15min":0.54900000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.99099999999999999,"5min":1.069,"15min":1.069},"last":0.751}]},{"osd":3,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64500000000000002,"15min":0.57999999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.49199999999999999,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.46300000000000002},"min":{"1min":0.42699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.70599999999999996,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76300000000000001,"15min":0.69699999999999995},"min":{"1min":0.61899999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93200000000000005,"5min":1.21,"15min":1.21},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83199999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.83799999999999997,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.66400000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0429999999999999,"5min":1.482,"15min":1.482},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1890000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.93799999999999994}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.96199999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.089}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":492862,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.081,"5min":1.431,"15min":1.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61799999999999999,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.129,"5min":2.125,"15min":2.125},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.73199999999999998,"15min":0.57799999999999996},"min":{"1min":0.503,"5min":0.25,"15min":0.25},"max":{"1min":6.8710000000000004,"5min":6.8710000000000004,"15min":6.8710000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.022,"5min":0.68899999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2729999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2529999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.67100000000000004,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.28,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.79000000000000004,"15min":0.68899999999999995},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.78500000000000003,"15min":0.70099999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69999999999999996,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.093,"5min":1.921,"15min":1.921},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.58199999999999996,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.347,"5min":1.871,"15min":1.871},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":6.9630000000000001,"5min":6.9630000000000001,"15min":6.9630000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.98099999999999998}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5780000000000001,"5min":2.177,"15min":2.177},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.82299999999999995,"15min":0.80000000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.409,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.998}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95099999999999996,"15min":0.93300000000000005},"min":{"1min":0.65200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4199999999999999,"5min":1.964,"15min":1.964},"last":0.89700000000000002}]},{"osd":10,"last update":"Wed May 8 23:02:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.97199999999999998,"15min":0.96299999999999997},"min":{"1min":0.71199999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.716,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.079},{"interface":"front","average":{"1min":1.05,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5680000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.006}]},{"osd":11,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.82499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.462,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.038},"min":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6819999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0189999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1040384,"data_stored":557615,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77900000000000003,"5min":0.87,"15min":0.87},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77800000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85899999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67400000000000004,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.871,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.496},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79300000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47299999999999998,"15min":0.44600000000000001},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39100000000000001}]},{"osd":6,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.71499999999999997,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84399999999999997,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35199999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.747},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.05,"15min":1.05},"last":0.48499999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1014883,"data_compressed":51811,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.48999999999999999,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69099999999999995,"5min":1.242,"15min":1.242},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.80100000000000005,"15min":0.61899999999999999},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":13.832000000000001,"5min":13.832000000000001,"15min":13.832000000000001},"last":1.032},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.61199999999999999},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.046,"15min":1.046},"last":0.73099999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.52400000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.0049999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.61899999999999999},"min":{"1min":0.47099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1040000000000001,"5min":1.845,"15min":1.845},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68799999999999994,"15min":0.64400000000000002},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.71999999999999997,"15min":0.64700000000000002},"min":{"1min":0.58599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.105}]},{"osd":5,"last update":"Wed May 8 23:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76200000000000001,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.70999999999999996},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.80400000000000005}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1339999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":13.807,"5min":13.807,"15min":13.807},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.0680000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.95099999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1459999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.92800000000000005,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97199999999999998,"5min":1.292,"15min":1.292},"last":0.82999999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.0609999999999999,"15min":0.97399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.893000000000001,"5min":13.893000000000001,"15min":13.893000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.01,"15min":1.014},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.1439999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":498006,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.46700000000000003,"15min":0.433},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.374,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63900000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90100000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.622,"15min":0.57399999999999995},"min":{"1min":0.48399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65600000000000003,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94499999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.70399999999999996},"min":{"1min":0.60999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.90600000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.57699999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":1.107,"15min":1.107},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0469999999999999,"5min":1.147,"15min":1.147},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.67400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.147,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.871},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.66700000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.391,"15min":1.391},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89700000000000002,"15min":0.90700000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.089,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.002}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89800000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1218894,"data_compressed":68556,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.51100000000000001},"min":{"1min":0.41699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.372},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56399999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.74099999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.57599999999999996},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.47799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.45000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59099999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.50900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.751}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70579,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68647,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134765,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394022,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202501,"data_compressed":8005,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:03:56.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:03:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.773+0000 7f3685000700 1 -- 172.21.15.46:0/1738756413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 msgr2=0x7f368010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.773+0000 7f3685000700 1 --2- 172.21.15.46:0/1738756413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f368010adc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f366c009990 tx=0x7f366c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 -- 172.21.15.46:0/1738756413 shutdown_connections 2024-05-08T23:03:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 --2- 172.21.15.46:0/1738756413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f368010b3e0 0x7f368010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 --2- 172.21.15.46:0/1738756413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680101fc0 0x7f36801023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 --2- 172.21.15.46:0/1738756413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f368010adc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 -- 172.21.15.46:0/1738756413 >> 172.21.15.46:0/1738756413 conn(0x7f36800fb820 msgr2=0x7f36800fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:56.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 -- 172.21.15.46:0/1738756413 shutdown_connections 2024-05-08T23:03:56.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.774+0000 7f3685000700 1 -- 172.21.15.46:0/1738756413 wait complete. 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 Processor -- start 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 -- start start 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f3680197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368010b3e0 0x7f368019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368019c200 con 0x7f36801028e0 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368019c370 con 0x7f368010b3e0 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.775+0000 7f3685000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368019c530 con 0x7f3680101fc0 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f3680197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34930/0 (socket says 172.21.15.46:34930) 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368010b3e0 0x7f368019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 -- 172.21.15.46:0/2855005279 learned_addr learned my addr 172.21.15.46:0/2855005279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 -- 172.21.15.46:0/2855005279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368010b3e0 msgr2=0x7f368019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368010b3e0 0x7f368019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 -- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 msgr2=0x7f3680197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f3680197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 -- 172.21.15.46:0/2855005279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670009520 con 0x7f3680101fc0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f367de00700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f367000eb30 tx=0x7f367000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.776+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670009880 con 0x7f3680101fc0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.777+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36700044e0 con 0x7f3680101fc0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.777+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c0095f0 con 0x7f3680101fc0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.777+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670003d80 con 0x7f3680101fc0 2024-05-08T23:03:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.777+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368019ca00 con 0x7f3680101fc0 2024-05-08T23:03:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.778+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36801064b0 con 0x7f3680101fc0 2024-05-08T23:03:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.779+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f3670003f20 con 0x7f3680101fc0 2024-05-08T23:03:56.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.781+0000 7f3676a00700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 0x7f36640a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:56.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.781+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3670014070 con 0x7f3680101fc0 2024-05-08T23:03:56.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.781+0000 7f367d400700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 0x7f36640a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:56.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.781+0000 7f367d400700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 0x7f36640a38f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3680198830 tx=0x7f366c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:56.785+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3670096650 con 0x7f3680101fc0 2024-05-08T23:03:56.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3876262474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:57.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.000+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36801986c0 con 0x7f36640a1430 2024-05-08T23:03:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3876262474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:57.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.014+0000 7f3676a00700 1 -- 172.21.15.46:0/2855005279 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426193 (secure 0 0 0) 0x7f36801986c0 con 0x7f36640a1430 2024-05-08T23:03:57.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 msgr2=0x7f36640a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 0x7f36640a38f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3680198830 tx=0x7f366c00b260 comp rx=0 tx=0).stop 2024-05-08T23:03:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 msgr2=0x7f36801972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f367000eb30 tx=0x7f367000ee40 comp rx=0 tx=0).stop 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 shutdown_connections 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368010b3e0 0x7f368019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3680101fc0 0x7f36801972e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36640a1430 0x7f36640a38f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 --2- 172.21.15.46:0/2855005279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36801028e0 0x7f3680197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 >> 172.21.15.46:0/2855005279 conn(0x7f36800fb820 msgr2=0x7f36800fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 shutdown_connections 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.019+0000 7f3685000700 1 -- 172.21.15.46:0/2855005279 wait complete. 2024-05-08T23:03:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:03:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3876262474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:03:57.039 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T23:03:56.092339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192743,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1323,"num_read_kb":990,"num_write":574,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":951,"ondisk_log_size":951,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373068,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568756,"statfs":{"total":1151940427776,"available":1151558406144,"internally_reserved":0,"allocated":12271616,"data_stored":9487461,"data_compressed":507234,"data_compressed_allocated":688128,"data_compressed_original":4227072,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":239,"num_write":29,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.984004"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142984+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:03:48.142984+0000","last_peered":"2024-05-08T23:03:48.142984+0000","last_clean":"2024-05-08T23:03:48.142984+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:03:48.142984+0000","last_undegraded":"2024-05-08T23:03:48.142984+0000","last_fullsized":"2024-05-08T23:03:48.142984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137182+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:03:48.137182+0000","last_peered":"2024-05-08T23:03:48.137182+0000","last_clean":"2024-05-08T23:03:48.137182+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:03:48.137182+0000","last_undegraded":"2024-05-08T23:03:48.137182+0000","last_fullsized":"2024-05-08T23:03:48.137182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134416+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:03:48.134416+0000","last_peered":"2024-05-08T23:03:48.134416+0000","last_clean":"2024-05-08T23:03:48.134416+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:03:48.134416+0000","last_undegraded":"2024-05-08T23:03:48.134416+0000","last_fullsized":"2024-05-08T23:03:48.134416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129246+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:03:48.129246+0000","last_peered":"2024-05-08T23:03:48.129246+0000","last_clean":"2024-05-08T23:03:48.129246+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:03:48.129246+0000","last_undegraded":"2024-05-08T23:03:48.129246+0000","last_fullsized":"2024-05-08T23:03:48.129246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128870+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:03:48.128870+0000","last_peered":"2024-05-08T23:03:48.128870+0000","last_clean":"2024-05-08T23:03:48.128870+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:03:48.128870+0000","last_undegraded":"2024-05-08T23:03:48.128870+0000","last_fullsized":"2024-05-08T23:03:48.128870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123525+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:03:48.123525+0000","last_peered":"2024-05-08T23:03:48.123525+0000","last_clean":"2024-05-08T23:03:48.123525+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:03:48.123525+0000","last_undegraded":"2024-05-08T23:03:48.123525+0000","last_fullsized":"2024-05-08T23:03:48.123525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120109+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:03:48.120109+0000","last_peered":"2024-05-08T23:03:48.120109+0000","last_clean":"2024-05-08T23:03:48.120109+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:03:48.120109+0000","last_undegraded":"2024-05-08T23:03:48.120109+0000","last_fullsized":"2024-05-08T23:03:48.120109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143018+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:03:48.143018+0000","last_peered":"2024-05-08T23:03:48.143018+0000","last_clean":"2024-05-08T23:03:48.143018+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:03:48.143018+0000","last_undegraded":"2024-05-08T23:03:48.143018+0000","last_fullsized":"2024-05-08T23:03:48.143018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.116240+0000","last_change":"2024-05-08T23:03:19.568519+0000","last_active":"2024-05-08T23:03:48.116240+0000","last_peered":"2024-05-08T23:03:48.116240+0000","last_clean":"2024-05-08T23:03:48.116240+0000","last_became_active":"2024-05-08T23:03:19.567932+0000","last_became_peered":"2024-05-08T23:03:19.567932+0000","last_unstale":"2024-05-08T23:03:48.116240+0000","last_undegraded":"2024-05-08T23:03:48.116240+0000","last_fullsized":"2024-05-08T23:03:48.116240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:18.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123210+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:03:48.123210+0000","last_peered":"2024-05-08T23:03:48.123210+0000","last_clean":"2024-05-08T23:03:48.123210+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:03:48.123210+0000","last_undegraded":"2024-05-08T23:03:48.123210+0000","last_fullsized":"2024-05-08T23:03:48.123210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132775+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:03:48.132775+0000","last_peered":"2024-05-08T23:03:48.132775+0000","last_clean":"2024-05-08T23:03:48.132775+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:03:48.132775+0000","last_undegraded":"2024-05-08T23:03:48.132775+0000","last_fullsized":"2024-05-08T23:03:48.132775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128936+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:03:48.128936+0000","last_peered":"2024-05-08T23:03:48.128936+0000","last_clean":"2024-05-08T23:03:48.128936+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:03:48.128936+0000","last_undegraded":"2024-05-08T23:03:48.128936+0000","last_fullsized":"2024-05-08T23:03:48.128936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151844+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:03:48.151844+0000","last_peered":"2024-05-08T23:03:48.151844+0000","last_clean":"2024-05-08T23:03:48.151844+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:03:48.151844+0000","last_undegraded":"2024-05-08T23:03:48.151844+0000","last_fullsized":"2024-05-08T23:03:48.151844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.136583+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:03:48.136583+0000","last_peered":"2024-05-08T23:03:48.136583+0000","last_clean":"2024-05-08T23:03:48.136583+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:03:48.136583+0000","last_undegraded":"2024-05-08T23:03:48.136583+0000","last_fullsized":"2024-05-08T23:03:48.136583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141698+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:03:48.141698+0000","last_peered":"2024-05-08T23:03:48.141698+0000","last_clean":"2024-05-08T23:03:48.141698+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:03:48.141698+0000","last_undegraded":"2024-05-08T23:03:48.141698+0000","last_fullsized":"2024-05-08T23:03:48.141698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135586+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:03:48.135586+0000","last_peered":"2024-05-08T23:03:48.135586+0000","last_clean":"2024-05-08T23:03:48.135586+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:03:48.135586+0000","last_undegraded":"2024-05-08T23:03:48.135586+0000","last_fullsized":"2024-05-08T23:03:48.135586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143079+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:03:48.143079+0000","last_peered":"2024-05-08T23:03:48.143079+0000","last_clean":"2024-05-08T23:03:48.143079+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:03:48.143079+0000","last_undegraded":"2024-05-08T23:03:48.143079+0000","last_fullsized":"2024-05-08T23:03:48.143079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129439+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:03:48.129439+0000","last_peered":"2024-05-08T23:03:48.129439+0000","last_clean":"2024-05-08T23:03:48.129439+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:03:48.129439+0000","last_undegraded":"2024-05-08T23:03:48.129439+0000","last_fullsized":"2024-05-08T23:03:48.129439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137675+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:03:48.137675+0000","last_peered":"2024-05-08T23:03:48.137675+0000","last_clean":"2024-05-08T23:03:48.137675+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:03:48.137675+0000","last_undegraded":"2024-05-08T23:03:48.137675+0000","last_fullsized":"2024-05-08T23:03:48.137675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649303+0000","last_change":"2024-05-08T23:03:19.648891+0000","last_active":"2024-05-08T23:03:19.649303+0000","last_peered":"2024-05-08T23:03:19.649303+0000","last_clean":"2024-05-08T23:03:19.649303+0000","last_became_active":"2024-05-08T23:03:19.648436+0000","last_became_peered":"2024-05-08T23:03:19.648436+0000","last_unstale":"2024-05-08T23:03:19.649303+0000","last_undegraded":"2024-05-08T23:03:19.649303+0000","last_fullsized":"2024-05-08T23:03:19.649303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:39.693182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771820+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:03:53.771820+0000","last_peered":"2024-05-08T23:03:53.771820+0000","last_clean":"2024-05-08T23:03:53.771820+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:03:53.771820+0000","last_undegraded":"2024-05-08T23:03:53.771820+0000","last_fullsized":"2024-05-08T23:03:53.771820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143528+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:03:48.143528+0000","last_peered":"2024-05-08T23:03:48.143528+0000","last_clean":"2024-05-08T23:03:48.143528+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:03:48.143528+0000","last_undegraded":"2024-05-08T23:03:48.143528+0000","last_fullsized":"2024-05-08T23:03:48.143528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648069+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:03:19.648069+0000","last_peered":"2024-05-08T23:03:19.648069+0000","last_clean":"2024-05-08T23:03:19.648069+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:03:19.648069+0000","last_undegraded":"2024-05-08T23:03:19.648069+0000","last_fullsized":"2024-05-08T23:03:19.648069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.088831+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:03:48.088831+0000","last_peered":"2024-05-08T23:03:48.088831+0000","last_clean":"2024-05-08T23:03:48.088831+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:03:48.088831+0000","last_undegraded":"2024-05-08T23:03:48.088831+0000","last_fullsized":"2024-05-08T23:03:48.088831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.139631+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:03:48.139631+0000","last_peered":"2024-05-08T23:03:48.139631+0000","last_clean":"2024-05-08T23:03:48.139631+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:03:48.139631+0000","last_undegraded":"2024-05-08T23:03:48.139631+0000","last_fullsized":"2024-05-08T23:03:48.139631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648708+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:03:19.648708+0000","last_peered":"2024-05-08T23:03:19.648708+0000","last_clean":"2024-05-08T23:03:19.648708+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:03:19.648708+0000","last_undegraded":"2024-05-08T23:03:19.648708+0000","last_fullsized":"2024-05-08T23:03:19.648708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:47.213419+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:03:47.213419+0000","last_peered":"2024-05-08T23:03:47.213419+0000","last_clean":"2024-05-08T23:03:47.213419+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:03:47.213419+0000","last_undegraded":"2024-05-08T23:03:47.213419+0000","last_fullsized":"2024-05-08T23:03:47.213419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145570+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:03:48.145570+0000","last_peered":"2024-05-08T23:03:48.145570+0000","last_clean":"2024-05-08T23:03:48.145570+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:03:48.145570+0000","last_undegraded":"2024-05-08T23:03:48.145570+0000","last_fullsized":"2024-05-08T23:03:48.145570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648176+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:03:19.648176+0000","last_peered":"2024-05-08T23:03:19.648176+0000","last_clean":"2024-05-08T23:03:19.648176+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:03:19.648176+0000","last_undegraded":"2024-05-08T23:03:19.648176+0000","last_fullsized":"2024-05-08T23:03:19.648176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319178+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:03:19.319178+0000","last_peered":"2024-05-08T23:03:19.319178+0000","last_clean":"2024-05-08T23:03:19.319178+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:03:19.319178+0000","last_undegraded":"2024-05-08T23:03:19.319178+0000","last_fullsized":"2024-05-08T23:03:19.319178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132478+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:03:48.132478+0000","last_peered":"2024-05-08T23:03:48.132478+0000","last_clean":"2024-05-08T23:03:48.132478+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:03:48.132478+0000","last_undegraded":"2024-05-08T23:03:48.132478+0000","last_fullsized":"2024-05-08T23:03:48.132478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569695+0000","last_change":"2024-05-08T23:03:19.568929+0000","last_active":"2024-05-08T23:03:19.569695+0000","last_peered":"2024-05-08T23:03:19.569695+0000","last_clean":"2024-05-08T23:03:19.569695+0000","last_became_active":"2024-05-08T23:03:19.568460+0000","last_became_peered":"2024-05-08T23:03:19.568460+0000","last_unstale":"2024-05-08T23:03:19.569695+0000","last_undegraded":"2024-05-08T23:03:19.569695+0000","last_fullsized":"2024-05-08T23:03:19.569695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:30.975740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771668+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:03:52.771668+0000","last_peered":"2024-05-08T23:03:52.771668+0000","last_clean":"2024-05-08T23:03:52.771668+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:03:52.771668+0000","last_undegraded":"2024-05-08T23:03:52.771668+0000","last_fullsized":"2024-05-08T23:03:52.771668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.111635+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:03:48.111635+0000","last_peered":"2024-05-08T23:03:48.111635+0000","last_clean":"2024-05-08T23:03:48.111635+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:03:48.111635+0000","last_undegraded":"2024-05-08T23:03:48.111635+0000","last_fullsized":"2024-05-08T23:03:48.111635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026600+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:03:19.026600+0000","last_peered":"2024-05-08T23:03:19.026600+0000","last_clean":"2024-05-08T23:03:19.026600+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:03:19.026600+0000","last_undegraded":"2024-05-08T23:03:19.026600+0000","last_fullsized":"2024-05-08T23:03:19.026600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029107+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:03:19.029107+0000","last_peered":"2024-05-08T23:03:19.029107+0000","last_clean":"2024-05-08T23:03:19.029107+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:03:19.029107+0000","last_undegraded":"2024-05-08T23:03:19.029107+0000","last_fullsized":"2024-05-08T23:03:19.029107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.144393+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:03:48.144393+0000","last_peered":"2024-05-08T23:03:48.144393+0000","last_clean":"2024-05-08T23:03:48.144393+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:03:48.144393+0000","last_undegraded":"2024-05-08T23:03:48.144393+0000","last_fullsized":"2024-05-08T23:03:48.144393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.567021+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:03:19.567021+0000","last_peered":"2024-05-08T23:03:19.567021+0000","last_clean":"2024-05-08T23:03:19.567021+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:03:19.567021+0000","last_undegraded":"2024-05-08T23:03:19.567021+0000","last_fullsized":"2024-05-08T23:03:19.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026129+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:03:19.026129+0000","last_peered":"2024-05-08T23:03:19.026129+0000","last_clean":"2024-05-08T23:03:19.026129+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:03:19.026129+0000","last_undegraded":"2024-05-08T23:03:19.026129+0000","last_fullsized":"2024-05-08T23:03:19.026129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142934+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:03:48.142934+0000","last_peered":"2024-05-08T23:03:48.142934+0000","last_clean":"2024-05-08T23:03:48.142934+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:03:48.142934+0000","last_undegraded":"2024-05-08T23:03:48.142934+0000","last_fullsized":"2024-05-08T23:03:48.142934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647859+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:03:19.647859+0000","last_peered":"2024-05-08T23:03:19.647859+0000","last_clean":"2024-05-08T23:03:19.647859+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:03:19.647859+0000","last_undegraded":"2024-05-08T23:03:19.647859+0000","last_fullsized":"2024-05-08T23:03:19.647859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228997+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:03:19.228997+0000","last_peered":"2024-05-08T23:03:19.228997+0000","last_clean":"2024-05-08T23:03:19.228997+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:03:19.228997+0000","last_undegraded":"2024-05-08T23:03:19.228997+0000","last_fullsized":"2024-05-08T23:03:19.228997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.127009+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:03:48.127009+0000","last_peered":"2024-05-08T23:03:48.127009+0000","last_clean":"2024-05-08T23:03:48.127009+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:03:48.127009+0000","last_undegraded":"2024-05-08T23:03:48.127009+0000","last_fullsized":"2024-05-08T23:03:48.127009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024203+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:03:19.024203+0000","last_peered":"2024-05-08T23:03:19.024203+0000","last_clean":"2024-05-08T23:03:19.024203+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:03:19.024203+0000","last_undegraded":"2024-05-08T23:03:19.024203+0000","last_fullsized":"2024-05-08T23:03:19.024203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":217,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771761+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:03:53.771761+0000","last_peered":"2024-05-08T23:03:53.771761+0000","last_clean":"2024-05-08T23:03:53.771761+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:03:53.771761+0000","last_undegraded":"2024-05-08T23:03:53.771761+0000","last_fullsized":"2024-05-08T23:03:53.771761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133767+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:03:48.133767+0000","last_peered":"2024-05-08T23:03:48.133767+0000","last_clean":"2024-05-08T23:03:48.133767+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:03:48.133767+0000","last_undegraded":"2024-05-08T23:03:48.133767+0000","last_fullsized":"2024-05-08T23:03:48.133767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029046+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:03:19.029046+0000","last_peered":"2024-05-08T23:03:19.029046+0000","last_clean":"2024-05-08T23:03:19.029046+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:03:19.029046+0000","last_undegraded":"2024-05-08T23:03:19.029046+0000","last_fullsized":"2024-05-08T23:03:19.029046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025153+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:03:19.025153+0000","last_peered":"2024-05-08T23:03:19.025153+0000","last_clean":"2024-05-08T23:03:19.025153+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:03:19.025153+0000","last_undegraded":"2024-05-08T23:03:19.025153+0000","last_fullsized":"2024-05-08T23:03:19.025153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.140089+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:03:48.140089+0000","last_peered":"2024-05-08T23:03:48.140089+0000","last_clean":"2024-05-08T23:03:48.140089+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:03:48.140089+0000","last_undegraded":"2024-05-08T23:03:48.140089+0000","last_fullsized":"2024-05-08T23:03:48.140089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648854+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:03:19.648854+0000","last_peered":"2024-05-08T23:03:19.648854+0000","last_clean":"2024-05-08T23:03:19.648854+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:03:19.648854+0000","last_undegraded":"2024-05-08T23:03:19.648854+0000","last_fullsized":"2024-05-08T23:03:19.648854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229387+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:03:19.229387+0000","last_peered":"2024-05-08T23:03:19.229387+0000","last_clean":"2024-05-08T23:03:19.229387+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:03:19.229387+0000","last_undegraded":"2024-05-08T23:03:19.229387+0000","last_fullsized":"2024-05-08T23:03:19.229387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134729+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:03:48.134729+0000","last_peered":"2024-05-08T23:03:48.134729+0000","last_clean":"2024-05-08T23:03:48.134729+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:03:48.134729+0000","last_undegraded":"2024-05-08T23:03:48.134729+0000","last_fullsized":"2024-05-08T23:03:48.134729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.568583+0000","last_change":"2024-05-08T23:03:19.568159+0000","last_active":"2024-05-08T23:03:19.568583+0000","last_peered":"2024-05-08T23:03:19.568583+0000","last_clean":"2024-05-08T23:03:19.568583+0000","last_became_active":"2024-05-08T23:03:19.567769+0000","last_became_peered":"2024-05-08T23:03:19.567769+0000","last_unstale":"2024-05-08T23:03:19.568583+0000","last_undegraded":"2024-05-08T23:03:19.568583+0000","last_fullsized":"2024-05-08T23:03:19.568583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:53.373375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'40","reported_seq":259,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772557+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:03:53.772557+0000","last_peered":"2024-05-08T23:03:53.772557+0000","last_clean":"2024-05-08T23:03:53.772557+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:03:53.772557+0000","last_undegraded":"2024-05-08T23:03:53.772557+0000","last_fullsized":"2024-05-08T23:03:53.772557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67500,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":165,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.118936+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:03:48.118936+0000","last_peered":"2024-05-08T23:03:48.118936+0000","last_clean":"2024-05-08T23:03:48.118936+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:03:48.118936+0000","last_undegraded":"2024-05-08T23:03:48.118936+0000","last_fullsized":"2024-05-08T23:03:48.118936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229834+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:03:19.229834+0000","last_peered":"2024-05-08T23:03:19.229834+0000","last_clean":"2024-05-08T23:03:19.229834+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:03:19.229834+0000","last_undegraded":"2024-05-08T23:03:19.229834+0000","last_fullsized":"2024-05-08T23:03:19.229834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026610+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:03:19.026610+0000","last_peered":"2024-05-08T23:03:19.026610+0000","last_clean":"2024-05-08T23:03:19.026610+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:03:19.026610+0000","last_undegraded":"2024-05-08T23:03:19.026610+0000","last_fullsized":"2024-05-08T23:03:19.026610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149247+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:03:48.149247+0000","last_peered":"2024-05-08T23:03:48.149247+0000","last_clean":"2024-05-08T23:03:48.149247+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:03:48.149247+0000","last_undegraded":"2024-05-08T23:03:48.149247+0000","last_fullsized":"2024-05-08T23:03:48.149247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566993+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:03:19.566993+0000","last_peered":"2024-05-08T23:03:19.566993+0000","last_clean":"2024-05-08T23:03:19.566993+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:03:19.566993+0000","last_undegraded":"2024-05-08T23:03:19.566993+0000","last_fullsized":"2024-05-08T23:03:19.566993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229573+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:03:19.229573+0000","last_peered":"2024-05-08T23:03:19.229573+0000","last_clean":"2024-05-08T23:03:19.229573+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:03:19.229573+0000","last_undegraded":"2024-05-08T23:03:19.229573+0000","last_fullsized":"2024-05-08T23:03:19.229573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150295+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:03:48.150295+0000","last_peered":"2024-05-08T23:03:48.150295+0000","last_clean":"2024-05-08T23:03:48.150295+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:03:48.150295+0000","last_undegraded":"2024-05-08T23:03:48.150295+0000","last_fullsized":"2024-05-08T23:03:48.150295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030071+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:03:19.030071+0000","last_peered":"2024-05-08T23:03:19.030071+0000","last_clean":"2024-05-08T23:03:19.030071+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:03:19.030071+0000","last_undegraded":"2024-05-08T23:03:19.030071+0000","last_fullsized":"2024-05-08T23:03:19.030071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'10","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771913+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:03:52.771913+0000","last_peered":"2024-05-08T23:03:52.771913+0000","last_clean":"2024-05-08T23:03:52.771913+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:03:52.771913+0000","last_undegraded":"2024-05-08T23:03:52.771913+0000","last_fullsized":"2024-05-08T23:03:52.771913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2253,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125794+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:03:48.125794+0000","last_peered":"2024-05-08T23:03:48.125794+0000","last_clean":"2024-05-08T23:03:48.125794+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:03:48.125794+0000","last_undegraded":"2024-05-08T23:03:48.125794+0000","last_fullsized":"2024-05-08T23:03:48.125794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569288+0000","last_change":"2024-05-08T23:03:19.568544+0000","last_active":"2024-05-08T23:03:19.569288+0000","last_peered":"2024-05-08T23:03:19.569288+0000","last_clean":"2024-05-08T23:03:19.569288+0000","last_became_active":"2024-05-08T23:03:19.568087+0000","last_became_peered":"2024-05-08T23:03:19.568087+0000","last_unstale":"2024-05-08T23:03:19.569288+0000","last_undegraded":"2024-05-08T23:03:19.569288+0000","last_fullsized":"2024-05-08T23:03:19.569288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:45.970629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.318742+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:03:19.318742+0000","last_peered":"2024-05-08T23:03:19.318742+0000","last_clean":"2024-05-08T23:03:19.318742+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:03:19.318742+0000","last_undegraded":"2024-05-08T23:03:19.318742+0000","last_fullsized":"2024-05-08T23:03:19.318742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.122962+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:03:48.122962+0000","last_peered":"2024-05-08T23:03:48.122962+0000","last_clean":"2024-05-08T23:03:48.122962+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:03:48.122962+0000","last_undegraded":"2024-05-08T23:03:48.122962+0000","last_fullsized":"2024-05-08T23:03:48.122962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228903+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:03:19.228903+0000","last_peered":"2024-05-08T23:03:19.228903+0000","last_clean":"2024-05-08T23:03:19.228903+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:03:19.228903+0000","last_undegraded":"2024-05-08T23:03:19.228903+0000","last_fullsized":"2024-05-08T23:03:19.228903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093426+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:03:48.093426+0000","last_peered":"2024-05-08T23:03:48.093426+0000","last_clean":"2024-05-08T23:03:48.093426+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:03:48.093426+0000","last_undegraded":"2024-05-08T23:03:48.093426+0000","last_fullsized":"2024-05-08T23:03:48.093426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132064+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:03:48.132064+0000","last_peered":"2024-05-08T23:03:48.132064+0000","last_clean":"2024-05-08T23:03:48.132064+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:03:48.132064+0000","last_undegraded":"2024-05-08T23:03:48.132064+0000","last_fullsized":"2024-05-08T23:03:48.132064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319091+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:03:19.319091+0000","last_peered":"2024-05-08T23:03:19.319091+0000","last_clean":"2024-05-08T23:03:19.319091+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:03:19.319091+0000","last_undegraded":"2024-05-08T23:03:19.319091+0000","last_fullsized":"2024-05-08T23:03:19.319091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029438+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:03:19.029438+0000","last_peered":"2024-05-08T23:03:19.029438+0000","last_clean":"2024-05-08T23:03:19.029438+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:03:19.029438+0000","last_undegraded":"2024-05-08T23:03:19.029438+0000","last_fullsized":"2024-05-08T23:03:19.029438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134037+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:03:48.134037+0000","last_peered":"2024-05-08T23:03:48.134037+0000","last_clean":"2024-05-08T23:03:48.134037+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:03:48.134037+0000","last_undegraded":"2024-05-08T23:03:48.134037+0000","last_fullsized":"2024-05-08T23:03:48.134037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026364+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:03:19.026364+0000","last_peered":"2024-05-08T23:03:19.026364+0000","last_clean":"2024-05-08T23:03:19.026364+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:03:19.026364+0000","last_undegraded":"2024-05-08T23:03:19.026364+0000","last_fullsized":"2024-05-08T23:03:19.026364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":126,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771144+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:03:52.771144+0000","last_peered":"2024-05-08T23:03:52.771144+0000","last_clean":"2024-05-08T23:03:52.771144+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:03:52.771144+0000","last_undegraded":"2024-05-08T23:03:52.771144+0000","last_fullsized":"2024-05-08T23:03:52.771144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1243,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135922+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:03:48.135922+0000","last_peered":"2024-05-08T23:03:48.135922+0000","last_clean":"2024-05-08T23:03:48.135922+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:03:48.135922+0000","last_undegraded":"2024-05-08T23:03:48.135922+0000","last_fullsized":"2024-05-08T23:03:48.135922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026697+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:03:19.026697+0000","last_peered":"2024-05-08T23:03:19.026697+0000","last_clean":"2024-05-08T23:03:19.026697+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:03:19.026697+0000","last_undegraded":"2024-05-08T23:03:19.026697+0000","last_fullsized":"2024-05-08T23:03:19.026697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120851+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:03:48.120851+0000","last_peered":"2024-05-08T23:03:48.120851+0000","last_clean":"2024-05-08T23:03:48.120851+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:03:48.120851+0000","last_undegraded":"2024-05-08T23:03:48.120851+0000","last_fullsized":"2024-05-08T23:03:48.120851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":124,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772498+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:03:53.772498+0000","last_peered":"2024-05-08T23:03:53.772498+0000","last_clean":"2024-05-08T23:03:53.772498+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:03:53.772498+0000","last_undegraded":"2024-05-08T23:03:53.772498+0000","last_fullsized":"2024-05-08T23:03:53.772498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024808+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:03:19.024808+0000","last_peered":"2024-05-08T23:03:19.024808+0000","last_clean":"2024-05-08T23:03:19.024808+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:03:19.024808+0000","last_undegraded":"2024-05-08T23:03:19.024808+0000","last_fullsized":"2024-05-08T23:03:19.024808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137285+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:03:48.137285+0000","last_peered":"2024-05-08T23:03:48.137285+0000","last_clean":"2024-05-08T23:03:48.137285+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:03:48.137285+0000","last_undegraded":"2024-05-08T23:03:48.137285+0000","last_fullsized":"2024-05-08T23:03:48.137285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648728+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:03:19.648728+0000","last_peered":"2024-05-08T23:03:19.648728+0000","last_clean":"2024-05-08T23:03:19.648728+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:03:19.648728+0000","last_undegraded":"2024-05-08T23:03:19.648728+0000","last_fullsized":"2024-05-08T23:03:19.648728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":385,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:50.222340+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:03:50.222340+0000","last_peered":"2024-05-08T23:03:50.222340+0000","last_clean":"2024-05-08T23:03:50.222340+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:03:50.222340+0000","last_undegraded":"2024-05-08T23:03:50.222340+0000","last_fullsized":"2024-05-08T23:03:50.222340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229833+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:03:19.229833+0000","last_peered":"2024-05-08T23:03:19.229833+0000","last_clean":"2024-05-08T23:03:19.229833+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:03:19.229833+0000","last_undegraded":"2024-05-08T23:03:19.229833+0000","last_fullsized":"2024-05-08T23:03:19.229833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141189+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:03:48.141189+0000","last_peered":"2024-05-08T23:03:48.141189+0000","last_clean":"2024-05-08T23:03:48.141189+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:03:48.141189+0000","last_undegraded":"2024-05-08T23:03:48.141189+0000","last_fullsized":"2024-05-08T23:03:48.141189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566669+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:03:19.566669+0000","last_peered":"2024-05-08T23:03:19.566669+0000","last_clean":"2024-05-08T23:03:19.566669+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:03:19.566669+0000","last_undegraded":"2024-05-08T23:03:19.566669+0000","last_fullsized":"2024-05-08T23:03:19.566669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566734+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:03:19.566734+0000","last_peered":"2024-05-08T23:03:19.566734+0000","last_clean":"2024-05-08T23:03:19.566734+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:03:19.566734+0000","last_undegraded":"2024-05-08T23:03:19.566734+0000","last_fullsized":"2024-05-08T23:03:19.566734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319413+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:03:19.319413+0000","last_peered":"2024-05-08T23:03:19.319413+0000","last_clean":"2024-05-08T23:03:19.319413+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:03:19.319413+0000","last_undegraded":"2024-05-08T23:03:19.319413+0000","last_fullsized":"2024-05-08T23:03:19.319413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142913+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:03:48.142913+0000","last_peered":"2024-05-08T23:03:48.142913+0000","last_clean":"2024-05-08T23:03:48.142913+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:03:48.142913+0000","last_undegraded":"2024-05-08T23:03:48.142913+0000","last_fullsized":"2024-05-08T23:03:48.142913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023762+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:03:19.023762+0000","last_peered":"2024-05-08T23:03:19.023762+0000","last_clean":"2024-05-08T23:03:19.023762+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:03:19.023762+0000","last_undegraded":"2024-05-08T23:03:19.023762+0000","last_fullsized":"2024-05-08T23:03:19.023762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'45","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.383589+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:03:48.383589+0000","last_peered":"2024-05-08T23:03:48.383589+0000","last_clean":"2024-05-08T23:03:48.383589+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:03:48.383589+0000","last_undegraded":"2024-05-08T23:03:48.383589+0000","last_fullsized":"2024-05-08T23:03:48.383589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143365+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:03:48.143365+0000","last_peered":"2024-05-08T23:03:48.143365+0000","last_clean":"2024-05-08T23:03:48.143365+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:03:48.143365+0000","last_undegraded":"2024-05-08T23:03:48.143365+0000","last_fullsized":"2024-05-08T23:03:48.143365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025789+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:03:19.025789+0000","last_peered":"2024-05-08T23:03:19.025789+0000","last_clean":"2024-05-08T23:03:19.025789+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:03:19.025789+0000","last_undegraded":"2024-05-08T23:03:19.025789+0000","last_fullsized":"2024-05-08T23:03:19.025789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649256+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:03:19.649256+0000","last_peered":"2024-05-08T23:03:19.649256+0000","last_clean":"2024-05-08T23:03:19.649256+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:03:19.649256+0000","last_undegraded":"2024-05-08T23:03:19.649256+0000","last_fullsized":"2024-05-08T23:03:19.649256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145687+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:03:48.145687+0000","last_peered":"2024-05-08T23:03:48.145687+0000","last_clean":"2024-05-08T23:03:48.145687+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:03:48.145687+0000","last_undegraded":"2024-05-08T23:03:48.145687+0000","last_fullsized":"2024-05-08T23:03:48.145687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649029+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:03:19.649029+0000","last_peered":"2024-05-08T23:03:19.649029+0000","last_clean":"2024-05-08T23:03:19.649029+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:03:19.649029+0000","last_undegraded":"2024-05-08T23:03:19.649029+0000","last_fullsized":"2024-05-08T23:03:19.649029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648003+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:03:19.648003+0000","last_peered":"2024-05-08T23:03:19.648003+0000","last_clean":"2024-05-08T23:03:19.648003+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:03:19.648003+0000","last_undegraded":"2024-05-08T23:03:19.648003+0000","last_fullsized":"2024-05-08T23:03:19.648003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093132+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:03:48.093132+0000","last_peered":"2024-05-08T23:03:48.093132+0000","last_clean":"2024-05-08T23:03:48.093132+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:03:48.093132+0000","last_undegraded":"2024-05-08T23:03:48.093132+0000","last_fullsized":"2024-05-08T23:03:48.093132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030035+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:03:19.030035+0000","last_peered":"2024-05-08T23:03:19.030035+0000","last_clean":"2024-05-08T23:03:19.030035+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:03:19.030035+0000","last_undegraded":"2024-05-08T23:03:19.030035+0000","last_fullsized":"2024-05-08T23:03:19.030035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023898+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:03:19.023898+0000","last_peered":"2024-05-08T23:03:19.023898+0000","last_clean":"2024-05-08T23:03:19.023898+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:03:19.023898+0000","last_undegraded":"2024-05-08T23:03:19.023898+0000","last_fullsized":"2024-05-08T23:03:19.023898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142729+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:03:48.142729+0000","last_peered":"2024-05-08T23:03:48.142729+0000","last_clean":"2024-05-08T23:03:48.142729+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:03:48.142729+0000","last_undegraded":"2024-05-08T23:03:48.142729+0000","last_fullsized":"2024-05-08T23:03:48.142729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648771+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:03:19.648771+0000","last_peered":"2024-05-08T23:03:19.648771+0000","last_clean":"2024-05-08T23:03:19.648771+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:03:19.648771+0000","last_undegraded":"2024-05-08T23:03:19.648771+0000","last_fullsized":"2024-05-08T23:03:19.648771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026057+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:03:19.026057+0000","last_peered":"2024-05-08T23:03:19.026057+0000","last_clean":"2024-05-08T23:03:19.026057+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:03:19.026057+0000","last_undegraded":"2024-05-08T23:03:19.026057+0000","last_fullsized":"2024-05-08T23:03:19.026057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150336+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:03:48.150336+0000","last_peered":"2024-05-08T23:03:48.150336+0000","last_clean":"2024-05-08T23:03:48.150336+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:03:48.150336+0000","last_undegraded":"2024-05-08T23:03:48.150336+0000","last_fullsized":"2024-05-08T23:03:48.150336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648702+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:03:19.648702+0000","last_peered":"2024-05-08T23:03:19.648702+0000","last_clean":"2024-05-08T23:03:19.648702+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:03:19.648702+0000","last_undegraded":"2024-05-08T23:03:19.648702+0000","last_fullsized":"2024-05-08T23:03:19.648702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319031+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:03:19.319031+0000","last_peered":"2024-05-08T23:03:19.319031+0000","last_clean":"2024-05-08T23:03:19.319031+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:03:19.319031+0000","last_undegraded":"2024-05-08T23:03:19.319031+0000","last_fullsized":"2024-05-08T23:03:19.319031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.138554+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:03:48.138554+0000","last_peered":"2024-05-08T23:03:48.138554+0000","last_clean":"2024-05-08T23:03:48.138554+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:03:48.138554+0000","last_undegraded":"2024-05-08T23:03:48.138554+0000","last_fullsized":"2024-05-08T23:03:48.138554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649390+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:03:19.649390+0000","last_peered":"2024-05-08T23:03:19.649390+0000","last_clean":"2024-05-08T23:03:19.649390+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:03:19.649390+0000","last_undegraded":"2024-05-08T23:03:19.649390+0000","last_fullsized":"2024-05-08T23:03:19.649390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.770901+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:03:52.770901+0000","last_peered":"2024-05-08T23:03:52.770901+0000","last_clean":"2024-05-08T23:03:52.770901+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:03:52.770901+0000","last_undegraded":"2024-05-08T23:03:52.770901+0000","last_fullsized":"2024-05-08T23:03:52.770901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132905+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:03:48.132905+0000","last_peered":"2024-05-08T23:03:48.132905+0000","last_clean":"2024-05-08T23:03:48.132905+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:03:48.132905+0000","last_undegraded":"2024-05-08T23:03:48.132905+0000","last_fullsized":"2024-05-08T23:03:48.132905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647481+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:03:19.647481+0000","last_peered":"2024-05-08T23:03:19.647481+0000","last_clean":"2024-05-08T23:03:19.647481+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:03:19.647481+0000","last_undegraded":"2024-05-08T23:03:19.647481+0000","last_fullsized":"2024-05-08T23:03:19.647481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.377743+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:03:48.377743+0000","last_peered":"2024-05-08T23:03:48.377743+0000","last_clean":"2024-05-08T23:03:48.377743+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:03:48.377743+0000","last_undegraded":"2024-05-08T23:03:48.377743+0000","last_fullsized":"2024-05-08T23:03:48.377743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126672+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:03:48.126672+0000","last_peered":"2024-05-08T23:03:48.126672+0000","last_clean":"2024-05-08T23:03:48.126672+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:03:48.126672+0000","last_undegraded":"2024-05-08T23:03:48.126672+0000","last_fullsized":"2024-05-08T23:03:48.126672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228724+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:03:19.228724+0000","last_peered":"2024-05-08T23:03:19.228724+0000","last_clean":"2024-05-08T23:03:19.228724+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:03:19.228724+0000","last_undegraded":"2024-05-08T23:03:19.228724+0000","last_fullsized":"2024-05-08T23:03:19.228724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026628+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:03:19.026628+0000","last_peered":"2024-05-08T23:03:19.026628+0000","last_clean":"2024-05-08T23:03:19.026628+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:03:19.026628+0000","last_undegraded":"2024-05-08T23:03:19.026628+0000","last_fullsized":"2024-05-08T23:03:19.026628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126885+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:03:48.126885+0000","last_peered":"2024-05-08T23:03:48.126885+0000","last_clean":"2024-05-08T23:03:48.126885+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:03:48.126885+0000","last_undegraded":"2024-05-08T23:03:48.126885+0000","last_fullsized":"2024-05-08T23:03:48.126885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133238+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:03:48.133238+0000","last_peered":"2024-05-08T23:03:48.133238+0000","last_clean":"2024-05-08T23:03:48.133238+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:03:48.133238+0000","last_undegraded":"2024-05-08T23:03:48.133238+0000","last_fullsized":"2024-05-08T23:03:48.133238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124622+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:03:48.124622+0000","last_peered":"2024-05-08T23:03:48.124622+0000","last_clean":"2024-05-08T23:03:48.124622+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:03:48.124622+0000","last_undegraded":"2024-05-08T23:03:48.124622+0000","last_fullsized":"2024-05-08T23:03:48.124622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124829+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:03:48.124829+0000","last_peered":"2024-05-08T23:03:48.124829+0000","last_clean":"2024-05-08T23:03:48.124829+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:03:48.124829+0000","last_undegraded":"2024-05-08T23:03:48.124829+0000","last_fullsized":"2024-05-08T23:03:48.124829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150811+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:03:48.150811+0000","last_peered":"2024-05-08T23:03:48.150811+0000","last_clean":"2024-05-08T23:03:48.150811+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:03:48.150811+0000","last_undegraded":"2024-05-08T23:03:48.150811+0000","last_fullsized":"2024-05-08T23:03:48.150811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149706+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:03:48.149706+0000","last_peered":"2024-05-08T23:03:48.149706+0000","last_clean":"2024-05-08T23:03:48.149706+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:03:48.149706+0000","last_undegraded":"2024-05-08T23:03:48.149706+0000","last_fullsized":"2024-05-08T23:03:48.149706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.130871+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:03:48.130871+0000","last_peered":"2024-05-08T23:03:48.130871+0000","last_clean":"2024-05-08T23:03:48.130871+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:03:48.130871+0000","last_undegraded":"2024-05-08T23:03:48.130871+0000","last_fullsized":"2024-05-08T23:03:48.130871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125971+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:03:48.125971+0000","last_peered":"2024-05-08T23:03:48.125971+0000","last_clean":"2024-05-08T23:03:48.125971+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:03:48.125971+0000","last_undegraded":"2024-05-08T23:03:48.125971+0000","last_fullsized":"2024-05-08T23:03:48.125971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137011+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:03:48.137011+0000","last_peered":"2024-05-08T23:03:48.137011+0000","last_clean":"2024-05-08T23:03:48.137011+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:03:48.137011+0000","last_undegraded":"2024-05-08T23:03:48.137011+0000","last_fullsized":"2024-05-08T23:03:48.137011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143616+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:03:48.143616+0000","last_peered":"2024-05-08T23:03:48.143616+0000","last_clean":"2024-05-08T23:03:48.143616+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:03:48.143616+0000","last_undegraded":"2024-05-08T23:03:48.143616+0000","last_fullsized":"2024-05-08T23:03:48.143616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143122+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:03:48.143122+0000","last_peered":"2024-05-08T23:03:48.143122+0000","last_clean":"2024-05-08T23:03:48.143122+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:03:48.143122+0000","last_undegraded":"2024-05-08T23:03:48.143122+0000","last_fullsized":"2024-05-08T23:03:48.143122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134728+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:03:48.134728+0000","last_peered":"2024-05-08T23:03:48.134728+0000","last_clean":"2024-05-08T23:03:48.134728+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:03:48.134728+0000","last_undegraded":"2024-05-08T23:03:48.134728+0000","last_fullsized":"2024-05-08T23:03:48.134728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151641+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:03:48.151641+0000","last_peered":"2024-05-08T23:03:48.151641+0000","last_clean":"2024-05-08T23:03:48.151641+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:03:48.151641+0000","last_undegraded":"2024-05-08T23:03:48.151641+0000","last_fullsized":"2024-05-08T23:03:48.151641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.131074+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:03:48.131074+0000","last_peered":"2024-05-08T23:03:48.131074+0000","last_clean":"2024-05-08T23:03:48.131074+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:03:48.131074+0000","last_undegraded":"2024-05-08T23:03:48.131074+0000","last_fullsized":"2024-05-08T23:03:48.131074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107673,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602141,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1086,"num_read_kb":660,"num_write":371,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806423,"data_compressed":81453,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612642,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":970752,"data_stored":495994,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.755,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":1.095,"15min":1.095},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.72099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.56399999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85899999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.499},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93100000000000005,"5min":1.218,"15min":1.218},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0169999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.628,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.196,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.63100000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68500000000000005}]},{"osd":10,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.622,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":1.387,"15min":1.387},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.66000000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97099999999999997,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.66200000000000003}]}]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1282815,"data_compressed":70405,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.505}]},{"osd":1,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83799999999999997,"15min":0.82999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.69599999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.624}]},{"osd":9,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.85899999999999999},"min":{"1min":0.63,"5min":0.622,"15min":0.622},"max":{"1min":1.4279999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.90100000000000002}]},{"osd":11,"last update":"Wed May 8 23:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.82199999999999995},"min":{"1min":0.70999999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.008,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0700000000000001,"5min":1.091,"15min":1.091},"last":0.754}]}]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":892476,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":1.919,"15min":1.919},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84099999999999997,"5min":1.357,"15min":1.357},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71599999999999997,"15min":0.72799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.006,"5min":1.639,"15min":1.639},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93500000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.755}]},{"osd":2,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96499999999999997,"5min":1.488,"15min":1.488},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93899999999999995,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":1.371,"15min":1.371},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.71199999999999997,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95099999999999996,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.80400000000000005,"15min":0.84499999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89100000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.81899999999999995,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.077,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82799999999999996,"15min":0.86399999999999999},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.738,"15min":1.738},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.86799999999999999,"15min":0.90100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.038,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.89100000000000001,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95799999999999996,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.92000000000000004,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.008,"5min":1.673,"15min":1.673},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.89700000000000002,"15min":0.94499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.881,"15min":1.881},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.157,"5min":1.766,"15min":1.766},"last":0.77600000000000002}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.746}]}]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":888064,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.441},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.55200000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.47199999999999998},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.621},"min":{"1min":0.48499999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78500000000000003,"5min":1.079,"15min":1.079},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.372}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0660000000000001,"5min":1.139,"15min":1.139},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.629,"5min":0.441,"15min":0.441},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.497},{"interface":"front","average":{"1min":0.745,"5min":0.70399999999999996,"15min":0.69099999999999995},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89700000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.59299999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.53300000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.58699999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.877,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.62,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85499999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54000000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50700000000000001}]}]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":1094313,"data_compressed":62400,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.45000000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":1.109,"15min":1.109},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79400000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.622},"min":{"1min":0.58899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88800000000000001,"5min":1.05,"15min":1.05},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.89100000000000001}]},{"osd":8,"last update":"Wed May 8 23:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89000000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84899999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.91000000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96699999999999997,"5min":1.141,"15min":1.141},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98599999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.877}]},{"osd":11,"last update":"Wed May 8 23:03:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.01,"5min":1.052,"15min":1.052},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.85899999999999999,"15min":0.86099999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0329999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.95499999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":491323,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.504,"15min":0.46700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.503},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.0579999999999998,"5min":3.0579999999999998,"15min":3.0579999999999998},"last":0.91700000000000004}]},{"osd":1,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71899999999999997,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.67500000000000004},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.97699999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71499999999999997,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.093,"5min":0.76100000000000001,"15min":0.66800000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.67200000000000004,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68300000000000005,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.417,"5min":1.581,"15min":1.581},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.77100000000000002,"15min":0.74099999999999999},"min":{"1min":0.499,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.78100000000000003,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.343,"5min":3.343,"15min":3.343},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78900000000000003,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.72599999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3380000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.879}]},{"osd":7,"last update":"Wed May 8 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.83199999999999996,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.81699999999999995,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.377,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":0.91000000000000003,"15min":0.83399999999999996},"min":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.86899999999999999,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":0.61799999999999999}]},{"osd":9,"last update":"Wed May 8 23:03:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.93200000000000005,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.268,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.93700000000000006}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.006,"15min":0.98499999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.002,"15min":1.0109999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0569999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.9710000000000001,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":1040384,"data_stored":560216,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57599999999999996,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.60199999999999998,"15min":0.54200000000000004},"min":{"1min":0.44600000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 23:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.878,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.626,"15min":0.54900000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.99099999999999999,"5min":1.069,"15min":1.069},"last":0.751}]},{"osd":3,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64500000000000002,"15min":0.57999999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.49199999999999999,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.46300000000000002},"min":{"1min":0.42699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.70599999999999996,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76300000000000001,"15min":0.69699999999999995},"min":{"1min":0.61899999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93200000000000005,"5min":1.21,"15min":1.21},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83199999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.83799999999999997,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.66400000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0429999999999999,"5min":1.482,"15min":1.482},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1890000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.93799999999999994}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.96199999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.089}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":492862,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.081,"5min":1.431,"15min":1.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61799999999999999,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.129,"5min":2.125,"15min":2.125},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.73199999999999998,"15min":0.57799999999999996},"min":{"1min":0.503,"5min":0.25,"15min":0.25},"max":{"1min":6.8710000000000004,"5min":6.8710000000000004,"15min":6.8710000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.022,"5min":0.68899999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2729999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2529999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.67100000000000004,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.28,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.79000000000000004,"15min":0.68899999999999995},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.78500000000000003,"15min":0.70099999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69999999999999996,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.093,"5min":1.921,"15min":1.921},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.58199999999999996,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.347,"5min":1.871,"15min":1.871},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":6.9630000000000001,"5min":6.9630000000000001,"15min":6.9630000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.98099999999999998}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5780000000000001,"5min":2.177,"15min":2.177},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.82299999999999995,"15min":0.80000000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.409,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.998}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95099999999999996,"15min":0.93300000000000005},"min":{"1min":0.65200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4199999999999999,"5min":1.964,"15min":1.964},"last":0.89700000000000002}]},{"osd":10,"last update":"Wed May 8 23:02:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.97199999999999998,"15min":0.96299999999999997},"min":{"1min":0.71199999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.716,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.079},{"interface":"front","average":{"1min":1.05,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5680000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.006}]},{"osd":11,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.82499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.462,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.038},"min":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6819999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0189999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1040384,"data_stored":557615,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77900000000000003,"5min":0.87,"15min":0.87},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77800000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85899999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67400000000000004,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.871,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.496},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79300000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47299999999999998,"15min":0.44600000000000001},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39100000000000001}]},{"osd":6,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.71499999999999997,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84399999999999997,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35199999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.747},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.05,"15min":1.05},"last":0.48499999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1014883,"data_compressed":51811,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.48999999999999999,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69099999999999995,"5min":1.242,"15min":1.242},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.80100000000000005,"15min":0.61899999999999999},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":13.832000000000001,"5min":13.832000000000001,"15min":13.832000000000001},"last":1.032},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.61199999999999999},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.046,"15min":1.046},"last":0.73099999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.52400000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.0049999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.61899999999999999},"min":{"1min":0.47099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1040000000000001,"5min":1.845,"15min":1.845},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68799999999999994,"15min":0.64400000000000002},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.71999999999999997,"15min":0.64700000000000002},"min":{"1min":0.58599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.105}]},{"osd":5,"last update":"Wed May 8 23:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76200000000000001,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.70999999999999996},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.80400000000000005}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1339999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":13.807,"5min":13.807,"15min":13.807},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.0680000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.95099999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1459999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.92800000000000005,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97199999999999998,"5min":1.292,"15min":1.292},"last":0.82999999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.0609999999999999,"15min":0.97399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.893000000000001,"5min":13.893000000000001,"15min":13.893000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.01,"15min":1.014},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.1439999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":498006,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.46700000000000003,"15min":0.433},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.374,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63900000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90100000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.622,"15min":0.57399999999999995},"min":{"1min":0.48399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65600000000000003,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94499999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.70399999999999996},"min":{"1min":0.60999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.90600000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.57699999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":1.107,"15min":1.107},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0469999999999999,"5min":1.147,"15min":1.147},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.67400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.147,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.871},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.66700000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.391,"15min":1.391},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89700000000000002,"15min":0.90700000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.089,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.002}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89800000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1218894,"data_compressed":68556,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.51100000000000001},"min":{"1min":0.41699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.372},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56399999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.74099999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.57599999999999996},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.47799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.45000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59099999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.50900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.751}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70579,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68647,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134765,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394022,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202501,"data_compressed":8005,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:03:57.046 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T23:03:57.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:03:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 -- 172.21.15.46:0/3249956552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0101a20 msgr2=0x7f90b0105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 --2- 172.21.15.46:0/3249956552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0101a20 0x7f90b0105a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f909c009990 tx=0x7f909c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 -- 172.21.15.46:0/3249956552 shutdown_connections 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 --2- 172.21.15.46:0/3249956552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0106190 0x7f90b010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 --2- 172.21.15.46:0/3249956552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 --2- 172.21.15.46:0/3249956552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0101a20 0x7f90b0105a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 -- 172.21.15.46:0/3249956552 >> 172.21.15.46:0/3249956552 conn(0x7f90b00fc930 msgr2=0x7f90b00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 -- 172.21.15.46:0/3249956552 shutdown_connections 2024-05-08T23:03:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.278+0000 7f90b6200700 1 -- 172.21.15.46:0/3249956552 wait complete. 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.279+0000 7f90b6200700 1 Processor -- start 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.279+0000 7f90b6200700 1 -- start start 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 0x7f90b00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 0x7f90b0073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01aa1a0 con 0x7f90b0101070 2024-05-08T23:03:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01aa310 con 0x7f90b0074f60 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90b6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01aa4d0 con 0x7f90b0106190 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90af400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 0x7f90b00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90af400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33782/0 (socket says 172.21.15.46:33782) 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90af400700 1 -- 172.21.15.46:0/2848073768 learned_addr learned my addr 172.21.15.46:0/2848073768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90afe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 0x7f90b0073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 -- 172.21.15.46:0/2848073768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 msgr2=0x7f90b0073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 0x7f90b0073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 -- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 msgr2=0x7f90b00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90aea00700 1 -- 172.21.15.46:0/2848073768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0095f0 con 0x7f90b0106190 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.280+0000 7f90afe00700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 0x7f90b0073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90af400700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90aea00700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 0x7f90b00737e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90b006a4d0 tx=0x7f909c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c01d070 con 0x7f90b0106190 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b01aa750 con 0x7f90b0106190 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c00fb30 con 0x7f90b0106190 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c022ba0 con 0x7f90b0106190 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.281+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b01aabd0 con 0x7f90b0106190 2024-05-08T23:03:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.282+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b0109460 con 0x7f90b0106190 2024-05-08T23:03:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.284+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f909c00fca0 con 0x7f90b0106190 2024-05-08T23:03:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.285+0000 7f90a7e00700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 0x7f90940a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.286+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f909c0cee70 con 0x7f90b0106190 2024-05-08T23:03:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.286+0000 7f90af400700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 0x7f90940a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.286+0000 7f90af400700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 0x7f90940a38f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009ea0 tx=0x7f90a0009290 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.288+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f909c097cf0 con 0x7f90b0106190 2024-05-08T23:03:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.501+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90b0067d10 con 0x7f90940a1430 2024-05-08T23:03:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.514+0000 7f90a7e00700 1 -- 172.21.15.46:0/2848073768 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426193 (secure 0 0 0) 0x7f90b0067d10 con 0x7f90940a1430 2024-05-08T23:03:57.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 msgr2=0x7f90940a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 0x7f90940a38f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009ea0 tx=0x7f90a0009290 comp rx=0 tx=0).stop 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 msgr2=0x7f90b00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 0x7f90b00737e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90b006a4d0 tx=0x7f909c005740 comp rx=0 tx=0).stop 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 shutdown_connections 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b0074f60 0x7f90b0073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90940a1430 0x7f90940a38f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90b0101070 0x7f90b00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 --2- 172.21.15.46:0/2848073768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90b0106190 0x7f90b00737e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 >> 172.21.15.46:0/2848073768 conn(0x7f90b00fc930 msgr2=0x7f90b00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 shutdown_connections 2024-05-08T23:03:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.520+0000 7f90b6200700 1 -- 172.21.15.46:0/2848073768 wait complete. 2024-05-08T23:03:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T23:03:57.539 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T23:03:56.092339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192743,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1323,"num_read_kb":990,"num_write":574,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182462,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":951,"ondisk_log_size":951,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373068,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124568756,"statfs":{"total":1151940427776,"available":1151558406144,"internally_reserved":0,"allocated":12271616,"data_stored":9487461,"data_compressed":507234,"data_compressed_allocated":688128,"data_compressed_original":4227072,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":239,"num_write":29,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.984004"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142984+0000","last_change":"2024-05-08T23:01:51.482814+0000","last_active":"2024-05-08T23:03:48.142984+0000","last_peered":"2024-05-08T23:03:48.142984+0000","last_clean":"2024-05-08T23:03:48.142984+0000","last_became_active":"2024-05-08T23:01:51.482004+0000","last_became_peered":"2024-05-08T23:01:51.482004+0000","last_unstale":"2024-05-08T23:03:48.142984+0000","last_undegraded":"2024-05-08T23:03:48.142984+0000","last_fullsized":"2024-05-08T23:03:48.142984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:30.980687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137182+0000","last_change":"2024-05-08T23:01:51.499311+0000","last_active":"2024-05-08T23:03:48.137182+0000","last_peered":"2024-05-08T23:03:48.137182+0000","last_clean":"2024-05-08T23:03:48.137182+0000","last_became_active":"2024-05-08T23:01:51.493444+0000","last_became_peered":"2024-05-08T23:01:51.493444+0000","last_unstale":"2024-05-08T23:03:48.137182+0000","last_undegraded":"2024-05-08T23:03:48.137182+0000","last_fullsized":"2024-05-08T23:03:48.137182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:26.426765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134416+0000","last_change":"2024-05-08T23:01:51.483764+0000","last_active":"2024-05-08T23:03:48.134416+0000","last_peered":"2024-05-08T23:03:48.134416+0000","last_clean":"2024-05-08T23:03:48.134416+0000","last_became_active":"2024-05-08T23:01:51.483396+0000","last_became_peered":"2024-05-08T23:01:51.483396+0000","last_unstale":"2024-05-08T23:03:48.134416+0000","last_undegraded":"2024-05-08T23:03:48.134416+0000","last_fullsized":"2024-05-08T23:03:48.134416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:57.103866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129246+0000","last_change":"2024-05-08T23:01:51.497075+0000","last_active":"2024-05-08T23:03:48.129246+0000","last_peered":"2024-05-08T23:03:48.129246+0000","last_clean":"2024-05-08T23:03:48.129246+0000","last_became_active":"2024-05-08T23:01:51.496744+0000","last_became_peered":"2024-05-08T23:01:51.496744+0000","last_unstale":"2024-05-08T23:03:48.129246+0000","last_undegraded":"2024-05-08T23:03:48.129246+0000","last_fullsized":"2024-05-08T23:03:48.129246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:45.384956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128870+0000","last_change":"2024-05-08T23:02:18.659619+0000","last_active":"2024-05-08T23:03:48.128870+0000","last_peered":"2024-05-08T23:03:48.128870+0000","last_clean":"2024-05-08T23:03:48.128870+0000","last_became_active":"2024-05-08T23:02:18.659226+0000","last_became_peered":"2024-05-08T23:02:18.659226+0000","last_unstale":"2024-05-08T23:03:48.128870+0000","last_undegraded":"2024-05-08T23:03:48.128870+0000","last_fullsized":"2024-05-08T23:03:48.128870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:35.156032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123525+0000","last_change":"2024-05-08T23:02:18.659366+0000","last_active":"2024-05-08T23:03:48.123525+0000","last_peered":"2024-05-08T23:03:48.123525+0000","last_clean":"2024-05-08T23:03:48.123525+0000","last_became_active":"2024-05-08T23:02:18.658877+0000","last_became_peered":"2024-05-08T23:02:18.658877+0000","last_unstale":"2024-05-08T23:03:48.123525+0000","last_undegraded":"2024-05-08T23:03:48.123525+0000","last_fullsized":"2024-05-08T23:03:48.123525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:48.173250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120109+0000","last_change":"2024-05-08T23:01:51.478285+0000","last_active":"2024-05-08T23:03:48.120109+0000","last_peered":"2024-05-08T23:03:48.120109+0000","last_clean":"2024-05-08T23:03:48.120109+0000","last_became_active":"2024-05-08T23:01:51.477900+0000","last_became_peered":"2024-05-08T23:01:51.477900+0000","last_unstale":"2024-05-08T23:03:48.120109+0000","last_undegraded":"2024-05-08T23:03:48.120109+0000","last_fullsized":"2024-05-08T23:03:48.120109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:26.559761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143018+0000","last_change":"2024-05-08T23:01:51.493862+0000","last_active":"2024-05-08T23:03:48.143018+0000","last_peered":"2024-05-08T23:03:48.143018+0000","last_clean":"2024-05-08T23:03:48.143018+0000","last_became_active":"2024-05-08T23:01:51.487886+0000","last_became_peered":"2024-05-08T23:01:51.487886+0000","last_unstale":"2024-05-08T23:03:48.143018+0000","last_undegraded":"2024-05-08T23:03:48.143018+0000","last_fullsized":"2024-05-08T23:03:48.143018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:38.979321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.116240+0000","last_change":"2024-05-08T23:03:19.568519+0000","last_active":"2024-05-08T23:03:48.116240+0000","last_peered":"2024-05-08T23:03:48.116240+0000","last_clean":"2024-05-08T23:03:48.116240+0000","last_became_active":"2024-05-08T23:03:19.567932+0000","last_became_peered":"2024-05-08T23:03:19.567932+0000","last_unstale":"2024-05-08T23:03:48.116240+0000","last_undegraded":"2024-05-08T23:03:48.116240+0000","last_fullsized":"2024-05-08T23:03:48.116240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:18.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.123210+0000","last_change":"2024-05-08T23:01:51.492039+0000","last_active":"2024-05-08T23:03:48.123210+0000","last_peered":"2024-05-08T23:03:48.123210+0000","last_clean":"2024-05-08T23:03:48.123210+0000","last_became_active":"2024-05-08T23:01:51.491713+0000","last_became_peered":"2024-05-08T23:01:51.491713+0000","last_unstale":"2024-05-08T23:03:48.123210+0000","last_undegraded":"2024-05-08T23:03:48.123210+0000","last_fullsized":"2024-05-08T23:03:48.123210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:18.075605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132775+0000","last_change":"2024-05-08T23:01:51.479852+0000","last_active":"2024-05-08T23:03:48.132775+0000","last_peered":"2024-05-08T23:03:48.132775+0000","last_clean":"2024-05-08T23:03:48.132775+0000","last_became_active":"2024-05-08T23:01:51.479247+0000","last_became_peered":"2024-05-08T23:01:51.479247+0000","last_unstale":"2024-05-08T23:03:48.132775+0000","last_undegraded":"2024-05-08T23:03:48.132775+0000","last_fullsized":"2024-05-08T23:03:48.132775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:07.576724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.128936+0000","last_change":"2024-05-08T23:01:51.498341+0000","last_active":"2024-05-08T23:03:48.128936+0000","last_peered":"2024-05-08T23:03:48.128936+0000","last_clean":"2024-05-08T23:03:48.128936+0000","last_became_active":"2024-05-08T23:01:51.497887+0000","last_became_peered":"2024-05-08T23:01:51.497887+0000","last_unstale":"2024-05-08T23:03:48.128936+0000","last_undegraded":"2024-05-08T23:03:48.128936+0000","last_fullsized":"2024-05-08T23:03:48.128936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:06.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151844+0000","last_change":"2024-05-08T23:01:51.492739+0000","last_active":"2024-05-08T23:03:48.151844+0000","last_peered":"2024-05-08T23:03:48.151844+0000","last_clean":"2024-05-08T23:03:48.151844+0000","last_became_active":"2024-05-08T23:01:51.491317+0000","last_became_peered":"2024-05-08T23:01:51.491317+0000","last_unstale":"2024-05-08T23:03:48.151844+0000","last_undegraded":"2024-05-08T23:03:48.151844+0000","last_fullsized":"2024-05-08T23:03:48.151844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:07.345676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.136583+0000","last_change":"2024-05-08T23:01:51.496580+0000","last_active":"2024-05-08T23:03:48.136583+0000","last_peered":"2024-05-08T23:03:48.136583+0000","last_clean":"2024-05-08T23:03:48.136583+0000","last_became_active":"2024-05-08T23:01:51.492822+0000","last_became_peered":"2024-05-08T23:01:51.492822+0000","last_unstale":"2024-05-08T23:03:48.136583+0000","last_undegraded":"2024-05-08T23:03:48.136583+0000","last_fullsized":"2024-05-08T23:03:48.136583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:49.103440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141698+0000","last_change":"2024-05-08T23:01:51.499898+0000","last_active":"2024-05-08T23:03:48.141698+0000","last_peered":"2024-05-08T23:03:48.141698+0000","last_clean":"2024-05-08T23:03:48.141698+0000","last_became_active":"2024-05-08T23:01:51.496598+0000","last_became_peered":"2024-05-08T23:01:51.496598+0000","last_unstale":"2024-05-08T23:03:48.141698+0000","last_undegraded":"2024-05-08T23:03:48.141698+0000","last_fullsized":"2024-05-08T23:03:48.141698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:03.300202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135586+0000","last_change":"2024-05-08T23:01:51.482662+0000","last_active":"2024-05-08T23:03:48.135586+0000","last_peered":"2024-05-08T23:03:48.135586+0000","last_clean":"2024-05-08T23:03:48.135586+0000","last_became_active":"2024-05-08T23:01:51.482276+0000","last_became_peered":"2024-05-08T23:01:51.482276+0000","last_unstale":"2024-05-08T23:03:48.135586+0000","last_undegraded":"2024-05-08T23:03:48.135586+0000","last_fullsized":"2024-05-08T23:03:48.135586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:59.966873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143079+0000","last_change":"2024-05-08T23:01:51.485908+0000","last_active":"2024-05-08T23:03:48.143079+0000","last_peered":"2024-05-08T23:03:48.143079+0000","last_clean":"2024-05-08T23:03:48.143079+0000","last_became_active":"2024-05-08T23:01:51.485587+0000","last_became_peered":"2024-05-08T23:01:51.485587+0000","last_unstale":"2024-05-08T23:03:48.143079+0000","last_undegraded":"2024-05-08T23:03:48.143079+0000","last_fullsized":"2024-05-08T23:03:48.143079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:10.058558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.129439+0000","last_change":"2024-05-08T23:01:51.492465+0000","last_active":"2024-05-08T23:03:48.129439+0000","last_peered":"2024-05-08T23:03:48.129439+0000","last_clean":"2024-05-08T23:03:48.129439+0000","last_became_active":"2024-05-08T23:01:51.492087+0000","last_became_peered":"2024-05-08T23:01:51.492087+0000","last_unstale":"2024-05-08T23:03:48.129439+0000","last_undegraded":"2024-05-08T23:03:48.129439+0000","last_fullsized":"2024-05-08T23:03:48.129439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:17.130554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137675+0000","last_change":"2024-05-08T23:01:51.490381+0000","last_active":"2024-05-08T23:03:48.137675+0000","last_peered":"2024-05-08T23:03:48.137675+0000","last_clean":"2024-05-08T23:03:48.137675+0000","last_became_active":"2024-05-08T23:01:51.490076+0000","last_became_peered":"2024-05-08T23:01:51.490076+0000","last_unstale":"2024-05-08T23:03:48.137675+0000","last_undegraded":"2024-05-08T23:03:48.137675+0000","last_fullsized":"2024-05-08T23:03:48.137675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:40.733698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649303+0000","last_change":"2024-05-08T23:03:19.648891+0000","last_active":"2024-05-08T23:03:19.649303+0000","last_peered":"2024-05-08T23:03:19.649303+0000","last_clean":"2024-05-08T23:03:19.649303+0000","last_became_active":"2024-05-08T23:03:19.648436+0000","last_became_peered":"2024-05-08T23:03:19.648436+0000","last_unstale":"2024-05-08T23:03:19.649303+0000","last_undegraded":"2024-05-08T23:03:19.649303+0000","last_fullsized":"2024-05-08T23:03:19.649303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:39.693182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'14","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771820+0000","last_change":"2024-05-08T23:02:18.677888+0000","last_active":"2024-05-08T23:03:53.771820+0000","last_peered":"2024-05-08T23:03:53.771820+0000","last_clean":"2024-05-08T23:03:53.771820+0000","last_became_active":"2024-05-08T23:02:18.668493+0000","last_became_peered":"2024-05-08T23:02:18.668493+0000","last_unstale":"2024-05-08T23:03:53.771820+0000","last_undegraded":"2024-05-08T23:03:53.771820+0000","last_fullsized":"2024-05-08T23:03:53.771820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:04.920257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143528+0000","last_change":"2024-05-08T23:01:51.500943+0000","last_active":"2024-05-08T23:03:48.143528+0000","last_peered":"2024-05-08T23:03:48.143528+0000","last_clean":"2024-05-08T23:03:48.143528+0000","last_became_active":"2024-05-08T23:01:51.499371+0000","last_became_peered":"2024-05-08T23:01:51.499371+0000","last_unstale":"2024-05-08T23:03:48.143528+0000","last_undegraded":"2024-05-08T23:03:48.143528+0000","last_fullsized":"2024-05-08T23:03:48.143528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:01.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648069+0000","last_change":"2024-05-08T23:01:55.977048+0000","last_active":"2024-05-08T23:03:19.648069+0000","last_peered":"2024-05-08T23:03:19.648069+0000","last_clean":"2024-05-08T23:03:19.648069+0000","last_became_active":"2024-05-08T23:01:55.976617+0000","last_became_peered":"2024-05-08T23:01:55.976617+0000","last_unstale":"2024-05-08T23:03:19.648069+0000","last_undegraded":"2024-05-08T23:03:19.648069+0000","last_fullsized":"2024-05-08T23:03:19.648069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:31.866198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.088831+0000","last_change":"2024-05-08T23:01:50.964173+0000","last_active":"2024-05-08T23:03:48.088831+0000","last_peered":"2024-05-08T23:03:48.088831+0000","last_clean":"2024-05-08T23:03:48.088831+0000","last_became_active":"2024-05-08T23:01:50.962020+0000","last_became_peered":"2024-05-08T23:01:50.962020+0000","last_unstale":"2024-05-08T23:03:48.088831+0000","last_undegraded":"2024-05-08T23:03:48.088831+0000","last_fullsized":"2024-05-08T23:03:48.088831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:13.676991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.139631+0000","last_change":"2024-05-08T23:01:51.480907+0000","last_active":"2024-05-08T23:03:48.139631+0000","last_peered":"2024-05-08T23:03:48.139631+0000","last_clean":"2024-05-08T23:03:48.139631+0000","last_became_active":"2024-05-08T23:01:51.480596+0000","last_became_peered":"2024-05-08T23:01:51.480596+0000","last_unstale":"2024-05-08T23:03:48.139631+0000","last_undegraded":"2024-05-08T23:03:48.139631+0000","last_fullsized":"2024-05-08T23:03:48.139631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:37.456434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648708+0000","last_change":"2024-05-08T23:01:55.968890+0000","last_active":"2024-05-08T23:03:19.648708+0000","last_peered":"2024-05-08T23:03:19.648708+0000","last_clean":"2024-05-08T23:03:19.648708+0000","last_became_active":"2024-05-08T23:01:55.968206+0000","last_became_peered":"2024-05-08T23:01:55.968206+0000","last_unstale":"2024-05-08T23:03:19.648708+0000","last_undegraded":"2024-05-08T23:03:19.648708+0000","last_fullsized":"2024-05-08T23:03:19.648708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:37.385565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:47.213419+0000","last_change":"2024-05-08T23:01:50.959442+0000","last_active":"2024-05-08T23:03:47.213419+0000","last_peered":"2024-05-08T23:03:47.213419+0000","last_clean":"2024-05-08T23:03:47.213419+0000","last_became_active":"2024-05-08T23:01:50.958650+0000","last_became_peered":"2024-05-08T23:01:50.958650+0000","last_unstale":"2024-05-08T23:03:47.213419+0000","last_undegraded":"2024-05-08T23:03:47.213419+0000","last_fullsized":"2024-05-08T23:03:47.213419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:07:34.247673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145570+0000","last_change":"2024-05-08T23:01:51.492123+0000","last_active":"2024-05-08T23:03:48.145570+0000","last_peered":"2024-05-08T23:03:48.145570+0000","last_clean":"2024-05-08T23:03:48.145570+0000","last_became_active":"2024-05-08T23:01:51.491492+0000","last_became_peered":"2024-05-08T23:01:51.491492+0000","last_unstale":"2024-05-08T23:03:48.145570+0000","last_undegraded":"2024-05-08T23:03:48.145570+0000","last_fullsized":"2024-05-08T23:03:48.145570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:17.269055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648176+0000","last_change":"2024-05-08T23:01:55.962864+0000","last_active":"2024-05-08T23:03:19.648176+0000","last_peered":"2024-05-08T23:03:19.648176+0000","last_clean":"2024-05-08T23:03:19.648176+0000","last_became_active":"2024-05-08T23:01:55.962437+0000","last_became_peered":"2024-05-08T23:01:55.962437+0000","last_unstale":"2024-05-08T23:03:19.648176+0000","last_undegraded":"2024-05-08T23:03:19.648176+0000","last_fullsized":"2024-05-08T23:03:19.648176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:47.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319178+0000","last_change":"2024-05-08T23:01:50.962465+0000","last_active":"2024-05-08T23:03:19.319178+0000","last_peered":"2024-05-08T23:03:19.319178+0000","last_clean":"2024-05-08T23:03:19.319178+0000","last_became_active":"2024-05-08T23:01:50.962124+0000","last_became_peered":"2024-05-08T23:01:50.962124+0000","last_unstale":"2024-05-08T23:03:19.319178+0000","last_undegraded":"2024-05-08T23:03:19.319178+0000","last_fullsized":"2024-05-08T23:03:19.319178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:18.382276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132478+0000","last_change":"2024-05-08T23:01:51.489322+0000","last_active":"2024-05-08T23:03:48.132478+0000","last_peered":"2024-05-08T23:03:48.132478+0000","last_clean":"2024-05-08T23:03:48.132478+0000","last_became_active":"2024-05-08T23:01:51.488636+0000","last_became_peered":"2024-05-08T23:01:51.488636+0000","last_unstale":"2024-05-08T23:03:48.132478+0000","last_undegraded":"2024-05-08T23:03:48.132478+0000","last_fullsized":"2024-05-08T23:03:48.132478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569695+0000","last_change":"2024-05-08T23:03:19.568929+0000","last_active":"2024-05-08T23:03:19.569695+0000","last_peered":"2024-05-08T23:03:19.569695+0000","last_clean":"2024-05-08T23:03:19.569695+0000","last_became_active":"2024-05-08T23:03:19.568460+0000","last_became_peered":"2024-05-08T23:03:19.568460+0000","last_unstale":"2024-05-08T23:03:19.569695+0000","last_undegraded":"2024-05-08T23:03:19.569695+0000","last_fullsized":"2024-05-08T23:03:19.569695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:30.975740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771668+0000","last_change":"2024-05-08T23:01:50.958454+0000","last_active":"2024-05-08T23:03:52.771668+0000","last_peered":"2024-05-08T23:03:52.771668+0000","last_clean":"2024-05-08T23:03:52.771668+0000","last_became_active":"2024-05-08T23:01:50.958085+0000","last_became_peered":"2024-05-08T23:01:50.958085+0000","last_unstale":"2024-05-08T23:03:52.771668+0000","last_undegraded":"2024-05-08T23:03:52.771668+0000","last_fullsized":"2024-05-08T23:03:52.771668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:16.502696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.111635+0000","last_change":"2024-05-08T23:01:51.482979+0000","last_active":"2024-05-08T23:03:48.111635+0000","last_peered":"2024-05-08T23:03:48.111635+0000","last_clean":"2024-05-08T23:03:48.111635+0000","last_became_active":"2024-05-08T23:01:51.482683+0000","last_became_peered":"2024-05-08T23:01:51.482683+0000","last_unstale":"2024-05-08T23:03:48.111635+0000","last_undegraded":"2024-05-08T23:03:48.111635+0000","last_fullsized":"2024-05-08T23:03:48.111635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:37.058750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026600+0000","last_change":"2024-05-08T23:01:55.972939+0000","last_active":"2024-05-08T23:03:19.026600+0000","last_peered":"2024-05-08T23:03:19.026600+0000","last_clean":"2024-05-08T23:03:19.026600+0000","last_became_active":"2024-05-08T23:01:55.972584+0000","last_became_peered":"2024-05-08T23:01:55.972584+0000","last_unstale":"2024-05-08T23:03:19.026600+0000","last_undegraded":"2024-05-08T23:03:19.026600+0000","last_fullsized":"2024-05-08T23:03:19.026600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:50.211326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029107+0000","last_change":"2024-05-08T23:01:50.966471+0000","last_active":"2024-05-08T23:03:19.029107+0000","last_peered":"2024-05-08T23:03:19.029107+0000","last_clean":"2024-05-08T23:03:19.029107+0000","last_became_active":"2024-05-08T23:01:50.966151+0000","last_became_peered":"2024-05-08T23:01:50.966151+0000","last_unstale":"2024-05-08T23:03:19.029107+0000","last_undegraded":"2024-05-08T23:03:19.029107+0000","last_fullsized":"2024-05-08T23:03:19.029107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:12.224345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.144393+0000","last_change":"2024-05-08T23:02:18.665953+0000","last_active":"2024-05-08T23:03:48.144393+0000","last_peered":"2024-05-08T23:03:48.144393+0000","last_clean":"2024-05-08T23:03:48.144393+0000","last_became_active":"2024-05-08T23:02:18.665607+0000","last_became_peered":"2024-05-08T23:02:18.665607+0000","last_unstale":"2024-05-08T23:03:48.144393+0000","last_undegraded":"2024-05-08T23:03:48.144393+0000","last_fullsized":"2024-05-08T23:03:48.144393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:04.532948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.567021+0000","last_change":"2024-05-08T23:01:55.977116+0000","last_active":"2024-05-08T23:03:19.567021+0000","last_peered":"2024-05-08T23:03:19.567021+0000","last_clean":"2024-05-08T23:03:19.567021+0000","last_became_active":"2024-05-08T23:01:55.976712+0000","last_became_peered":"2024-05-08T23:01:55.976712+0000","last_unstale":"2024-05-08T23:03:19.567021+0000","last_undegraded":"2024-05-08T23:03:19.567021+0000","last_fullsized":"2024-05-08T23:03:19.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:58.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026129+0000","last_change":"2024-05-08T23:01:50.955336+0000","last_active":"2024-05-08T23:03:19.026129+0000","last_peered":"2024-05-08T23:03:19.026129+0000","last_clean":"2024-05-08T23:03:19.026129+0000","last_became_active":"2024-05-08T23:01:50.954992+0000","last_became_peered":"2024-05-08T23:01:50.954992+0000","last_unstale":"2024-05-08T23:03:19.026129+0000","last_undegraded":"2024-05-08T23:03:19.026129+0000","last_fullsized":"2024-05-08T23:03:19.026129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:01.944283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142934+0000","last_change":"2024-05-08T23:01:51.483558+0000","last_active":"2024-05-08T23:03:48.142934+0000","last_peered":"2024-05-08T23:03:48.142934+0000","last_clean":"2024-05-08T23:03:48.142934+0000","last_became_active":"2024-05-08T23:01:51.483092+0000","last_became_peered":"2024-05-08T23:01:51.483092+0000","last_unstale":"2024-05-08T23:03:48.142934+0000","last_undegraded":"2024-05-08T23:03:48.142934+0000","last_fullsized":"2024-05-08T23:03:48.142934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:43.647126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647859+0000","last_change":"2024-05-08T23:01:55.972389+0000","last_active":"2024-05-08T23:03:19.647859+0000","last_peered":"2024-05-08T23:03:19.647859+0000","last_clean":"2024-05-08T23:03:19.647859+0000","last_became_active":"2024-05-08T23:01:55.971633+0000","last_became_peered":"2024-05-08T23:01:55.971633+0000","last_unstale":"2024-05-08T23:03:19.647859+0000","last_undegraded":"2024-05-08T23:03:19.647859+0000","last_fullsized":"2024-05-08T23:03:19.647859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:07.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228997+0000","last_change":"2024-05-08T23:01:50.961934+0000","last_active":"2024-05-08T23:03:19.228997+0000","last_peered":"2024-05-08T23:03:19.228997+0000","last_clean":"2024-05-08T23:03:19.228997+0000","last_became_active":"2024-05-08T23:01:50.961547+0000","last_became_peered":"2024-05-08T23:01:50.961547+0000","last_unstale":"2024-05-08T23:03:19.228997+0000","last_undegraded":"2024-05-08T23:03:19.228997+0000","last_fullsized":"2024-05-08T23:03:19.228997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:03.890879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.127009+0000","last_change":"2024-05-08T23:01:51.495232+0000","last_active":"2024-05-08T23:03:48.127009+0000","last_peered":"2024-05-08T23:03:48.127009+0000","last_clean":"2024-05-08T23:03:48.127009+0000","last_became_active":"2024-05-08T23:01:51.494042+0000","last_became_peered":"2024-05-08T23:01:51.494042+0000","last_unstale":"2024-05-08T23:03:48.127009+0000","last_undegraded":"2024-05-08T23:03:48.127009+0000","last_fullsized":"2024-05-08T23:03:48.127009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024203+0000","last_change":"2024-05-08T23:01:55.961625+0000","last_active":"2024-05-08T23:03:19.024203+0000","last_peered":"2024-05-08T23:03:19.024203+0000","last_clean":"2024-05-08T23:03:19.024203+0000","last_became_active":"2024-05-08T23:01:55.961204+0000","last_became_peered":"2024-05-08T23:01:55.961204+0000","last_unstale":"2024-05-08T23:03:19.024203+0000","last_undegraded":"2024-05-08T23:03:19.024203+0000","last_fullsized":"2024-05-08T23:03:19.024203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:26.133511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":217,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.771761+0000","last_change":"2024-05-08T23:01:50.956339+0000","last_active":"2024-05-08T23:03:53.771761+0000","last_peered":"2024-05-08T23:03:53.771761+0000","last_clean":"2024-05-08T23:03:53.771761+0000","last_became_active":"2024-05-08T23:01:50.955875+0000","last_became_peered":"2024-05-08T23:01:50.955875+0000","last_unstale":"2024-05-08T23:03:53.771761+0000","last_undegraded":"2024-05-08T23:03:53.771761+0000","last_fullsized":"2024-05-08T23:03:53.771761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:30.863021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133767+0000","last_change":"2024-05-08T23:01:51.492078+0000","last_active":"2024-05-08T23:03:48.133767+0000","last_peered":"2024-05-08T23:03:48.133767+0000","last_clean":"2024-05-08T23:03:48.133767+0000","last_became_active":"2024-05-08T23:01:51.491448+0000","last_became_peered":"2024-05-08T23:01:51.491448+0000","last_unstale":"2024-05-08T23:03:48.133767+0000","last_undegraded":"2024-05-08T23:03:48.133767+0000","last_fullsized":"2024-05-08T23:03:48.133767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:41.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029046+0000","last_change":"2024-05-08T23:01:55.976157+0000","last_active":"2024-05-08T23:03:19.029046+0000","last_peered":"2024-05-08T23:03:19.029046+0000","last_clean":"2024-05-08T23:03:19.029046+0000","last_became_active":"2024-05-08T23:01:55.972122+0000","last_became_peered":"2024-05-08T23:01:55.972122+0000","last_unstale":"2024-05-08T23:03:19.029046+0000","last_undegraded":"2024-05-08T23:03:19.029046+0000","last_fullsized":"2024-05-08T23:03:19.029046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:08.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025153+0000","last_change":"2024-05-08T23:01:50.957874+0000","last_active":"2024-05-08T23:03:19.025153+0000","last_peered":"2024-05-08T23:03:19.025153+0000","last_clean":"2024-05-08T23:03:19.025153+0000","last_became_active":"2024-05-08T23:01:50.957518+0000","last_became_peered":"2024-05-08T23:01:50.957518+0000","last_unstale":"2024-05-08T23:03:19.025153+0000","last_undegraded":"2024-05-08T23:03:19.025153+0000","last_fullsized":"2024-05-08T23:03:19.025153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:22.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.140089+0000","last_change":"2024-05-08T23:01:51.489552+0000","last_active":"2024-05-08T23:03:48.140089+0000","last_peered":"2024-05-08T23:03:48.140089+0000","last_clean":"2024-05-08T23:03:48.140089+0000","last_became_active":"2024-05-08T23:01:51.489209+0000","last_became_peered":"2024-05-08T23:01:51.489209+0000","last_unstale":"2024-05-08T23:03:48.140089+0000","last_undegraded":"2024-05-08T23:03:48.140089+0000","last_fullsized":"2024-05-08T23:03:48.140089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:59.188138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648854+0000","last_change":"2024-05-08T23:01:55.968574+0000","last_active":"2024-05-08T23:03:19.648854+0000","last_peered":"2024-05-08T23:03:19.648854+0000","last_clean":"2024-05-08T23:03:19.648854+0000","last_became_active":"2024-05-08T23:01:55.967694+0000","last_became_peered":"2024-05-08T23:01:55.967694+0000","last_unstale":"2024-05-08T23:03:19.648854+0000","last_undegraded":"2024-05-08T23:03:19.648854+0000","last_fullsized":"2024-05-08T23:03:19.648854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:55.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229387+0000","last_change":"2024-05-08T23:01:50.957713+0000","last_active":"2024-05-08T23:03:19.229387+0000","last_peered":"2024-05-08T23:03:19.229387+0000","last_clean":"2024-05-08T23:03:19.229387+0000","last_became_active":"2024-05-08T23:01:50.957327+0000","last_became_peered":"2024-05-08T23:01:50.957327+0000","last_unstale":"2024-05-08T23:03:19.229387+0000","last_undegraded":"2024-05-08T23:03:19.229387+0000","last_fullsized":"2024-05-08T23:03:19.229387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:01.971976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134729+0000","last_change":"2024-05-08T23:01:51.491841+0000","last_active":"2024-05-08T23:03:48.134729+0000","last_peered":"2024-05-08T23:03:48.134729+0000","last_clean":"2024-05-08T23:03:48.134729+0000","last_became_active":"2024-05-08T23:01:51.490939+0000","last_became_peered":"2024-05-08T23:01:51.490939+0000","last_unstale":"2024-05-08T23:03:48.134729+0000","last_undegraded":"2024-05-08T23:03:48.134729+0000","last_fullsized":"2024-05-08T23:03:48.134729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:11.536107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.568583+0000","last_change":"2024-05-08T23:03:19.568159+0000","last_active":"2024-05-08T23:03:19.568583+0000","last_peered":"2024-05-08T23:03:19.568583+0000","last_clean":"2024-05-08T23:03:19.568583+0000","last_became_active":"2024-05-08T23:03:19.567769+0000","last_became_peered":"2024-05-08T23:03:19.567769+0000","last_unstale":"2024-05-08T23:03:19.568583+0000","last_undegraded":"2024-05-08T23:03:19.568583+0000","last_fullsized":"2024-05-08T23:03:19.568583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:53.373375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'40","reported_seq":259,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772557+0000","last_change":"2024-05-08T23:01:50.957973+0000","last_active":"2024-05-08T23:03:53.772557+0000","last_peered":"2024-05-08T23:03:53.772557+0000","last_clean":"2024-05-08T23:03:53.772557+0000","last_became_active":"2024-05-08T23:01:50.957616+0000","last_became_peered":"2024-05-08T23:01:50.957616+0000","last_unstale":"2024-05-08T23:03:53.772557+0000","last_undegraded":"2024-05-08T23:03:53.772557+0000","last_fullsized":"2024-05-08T23:03:53.772557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:34.305390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67500,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":165,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.118936+0000","last_change":"2024-05-08T23:01:51.479792+0000","last_active":"2024-05-08T23:03:48.118936+0000","last_peered":"2024-05-08T23:03:48.118936+0000","last_clean":"2024-05-08T23:03:48.118936+0000","last_became_active":"2024-05-08T23:01:51.479451+0000","last_became_peered":"2024-05-08T23:01:51.479451+0000","last_unstale":"2024-05-08T23:03:48.118936+0000","last_undegraded":"2024-05-08T23:03:48.118936+0000","last_fullsized":"2024-05-08T23:03:48.118936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:23.619126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229834+0000","last_change":"2024-05-08T23:01:55.961558+0000","last_active":"2024-05-08T23:03:19.229834+0000","last_peered":"2024-05-08T23:03:19.229834+0000","last_clean":"2024-05-08T23:03:19.229834+0000","last_became_active":"2024-05-08T23:01:55.961034+0000","last_became_peered":"2024-05-08T23:01:55.961034+0000","last_unstale":"2024-05-08T23:03:19.229834+0000","last_undegraded":"2024-05-08T23:03:19.229834+0000","last_fullsized":"2024-05-08T23:03:19.229834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:13:06.992822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026610+0000","last_change":"2024-05-08T23:01:50.966842+0000","last_active":"2024-05-08T23:03:19.026610+0000","last_peered":"2024-05-08T23:03:19.026610+0000","last_clean":"2024-05-08T23:03:19.026610+0000","last_became_active":"2024-05-08T23:01:50.966514+0000","last_became_peered":"2024-05-08T23:01:50.966514+0000","last_unstale":"2024-05-08T23:03:19.026610+0000","last_undegraded":"2024-05-08T23:03:19.026610+0000","last_fullsized":"2024-05-08T23:03:19.026610+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:58.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149247+0000","last_change":"2024-05-08T23:01:51.476237+0000","last_active":"2024-05-08T23:03:48.149247+0000","last_peered":"2024-05-08T23:03:48.149247+0000","last_clean":"2024-05-08T23:03:48.149247+0000","last_became_active":"2024-05-08T23:01:51.475898+0000","last_became_peered":"2024-05-08T23:01:51.475898+0000","last_unstale":"2024-05-08T23:03:48.149247+0000","last_undegraded":"2024-05-08T23:03:48.149247+0000","last_fullsized":"2024-05-08T23:03:48.149247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:12.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566993+0000","last_change":"2024-05-08T23:01:55.976413+0000","last_active":"2024-05-08T23:03:19.566993+0000","last_peered":"2024-05-08T23:03:19.566993+0000","last_clean":"2024-05-08T23:03:19.566993+0000","last_became_active":"2024-05-08T23:01:55.975957+0000","last_became_peered":"2024-05-08T23:01:55.975957+0000","last_unstale":"2024-05-08T23:03:19.566993+0000","last_undegraded":"2024-05-08T23:03:19.566993+0000","last_fullsized":"2024-05-08T23:03:19.566993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:50.015072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229573+0000","last_change":"2024-05-08T23:01:50.965480+0000","last_active":"2024-05-08T23:03:19.229573+0000","last_peered":"2024-05-08T23:03:19.229573+0000","last_clean":"2024-05-08T23:03:19.229573+0000","last_became_active":"2024-05-08T23:01:50.965145+0000","last_became_peered":"2024-05-08T23:01:50.965145+0000","last_unstale":"2024-05-08T23:03:19.229573+0000","last_undegraded":"2024-05-08T23:03:19.229573+0000","last_fullsized":"2024-05-08T23:03:19.229573+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:37:23.514620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150295+0000","last_change":"2024-05-08T23:01:51.490081+0000","last_active":"2024-05-08T23:03:48.150295+0000","last_peered":"2024-05-08T23:03:48.150295+0000","last_clean":"2024-05-08T23:03:48.150295+0000","last_became_active":"2024-05-08T23:01:51.489747+0000","last_became_peered":"2024-05-08T23:01:51.489747+0000","last_unstale":"2024-05-08T23:03:48.150295+0000","last_undegraded":"2024-05-08T23:03:48.150295+0000","last_fullsized":"2024-05-08T23:03:48.150295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:27.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030071+0000","last_change":"2024-05-08T23:01:55.975648+0000","last_active":"2024-05-08T23:03:19.030071+0000","last_peered":"2024-05-08T23:03:19.030071+0000","last_clean":"2024-05-08T23:03:19.030071+0000","last_became_active":"2024-05-08T23:01:55.972885+0000","last_became_peered":"2024-05-08T23:01:55.972885+0000","last_unstale":"2024-05-08T23:03:19.030071+0000","last_undegraded":"2024-05-08T23:03:19.030071+0000","last_fullsized":"2024-05-08T23:03:19.030071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:14:11.871483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'10","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771913+0000","last_change":"2024-05-08T23:01:50.953898+0000","last_active":"2024-05-08T23:03:52.771913+0000","last_peered":"2024-05-08T23:03:52.771913+0000","last_clean":"2024-05-08T23:03:52.771913+0000","last_became_active":"2024-05-08T23:01:50.953511+0000","last_became_peered":"2024-05-08T23:01:50.953511+0000","last_unstale":"2024-05-08T23:03:52.771913+0000","last_undegraded":"2024-05-08T23:03:52.771913+0000","last_fullsized":"2024-05-08T23:03:52.771913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:33.675057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2253,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125794+0000","last_change":"2024-05-08T23:01:51.488666+0000","last_active":"2024-05-08T23:03:48.125794+0000","last_peered":"2024-05-08T23:03:48.125794+0000","last_clean":"2024-05-08T23:03:48.125794+0000","last_became_active":"2024-05-08T23:01:51.488325+0000","last_became_peered":"2024-05-08T23:01:51.488325+0000","last_unstale":"2024-05-08T23:03:48.125794+0000","last_undegraded":"2024-05-08T23:03:48.125794+0000","last_fullsized":"2024-05-08T23:03:48.125794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:58.975304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.569288+0000","last_change":"2024-05-08T23:03:19.568544+0000","last_active":"2024-05-08T23:03:19.569288+0000","last_peered":"2024-05-08T23:03:19.569288+0000","last_clean":"2024-05-08T23:03:19.569288+0000","last_became_active":"2024-05-08T23:03:19.568087+0000","last_became_peered":"2024-05-08T23:03:19.568087+0000","last_unstale":"2024-05-08T23:03:19.569288+0000","last_undegraded":"2024-05-08T23:03:19.569288+0000","last_fullsized":"2024-05-08T23:03:19.569288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:45.970629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.318742+0000","last_change":"2024-05-08T23:01:50.958084+0000","last_active":"2024-05-08T23:03:19.318742+0000","last_peered":"2024-05-08T23:03:19.318742+0000","last_clean":"2024-05-08T23:03:19.318742+0000","last_became_active":"2024-05-08T23:01:50.957591+0000","last_became_peered":"2024-05-08T23:01:50.957591+0000","last_unstale":"2024-05-08T23:03:19.318742+0000","last_undegraded":"2024-05-08T23:03:19.318742+0000","last_fullsized":"2024-05-08T23:03:19.318742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:25.036748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.122962+0000","last_change":"2024-05-08T23:01:51.494214+0000","last_active":"2024-05-08T23:03:48.122962+0000","last_peered":"2024-05-08T23:03:48.122962+0000","last_clean":"2024-05-08T23:03:48.122962+0000","last_became_active":"2024-05-08T23:01:51.493903+0000","last_became_peered":"2024-05-08T23:01:51.493903+0000","last_unstale":"2024-05-08T23:03:48.122962+0000","last_undegraded":"2024-05-08T23:03:48.122962+0000","last_fullsized":"2024-05-08T23:03:48.122962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:16.051577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228903+0000","last_change":"2024-05-08T23:01:55.961383+0000","last_active":"2024-05-08T23:03:19.228903+0000","last_peered":"2024-05-08T23:03:19.228903+0000","last_clean":"2024-05-08T23:03:19.228903+0000","last_became_active":"2024-05-08T23:01:55.960935+0000","last_became_peered":"2024-05-08T23:01:55.960935+0000","last_unstale":"2024-05-08T23:03:19.228903+0000","last_undegraded":"2024-05-08T23:03:19.228903+0000","last_fullsized":"2024-05-08T23:03:19.228903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:04.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093426+0000","last_change":"2024-05-08T23:02:18.670405+0000","last_active":"2024-05-08T23:03:48.093426+0000","last_peered":"2024-05-08T23:03:48.093426+0000","last_clean":"2024-05-08T23:03:48.093426+0000","last_became_active":"2024-05-08T23:02:18.662441+0000","last_became_peered":"2024-05-08T23:02:18.662441+0000","last_unstale":"2024-05-08T23:03:48.093426+0000","last_undegraded":"2024-05-08T23:03:48.093426+0000","last_fullsized":"2024-05-08T23:03:48.093426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:54.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132064+0000","last_change":"2024-05-08T23:02:18.663365+0000","last_active":"2024-05-08T23:03:48.132064+0000","last_peered":"2024-05-08T23:03:48.132064+0000","last_clean":"2024-05-08T23:03:48.132064+0000","last_became_active":"2024-05-08T23:02:18.662956+0000","last_became_peered":"2024-05-08T23:02:18.662956+0000","last_unstale":"2024-05-08T23:03:48.132064+0000","last_undegraded":"2024-05-08T23:03:48.132064+0000","last_fullsized":"2024-05-08T23:03:48.132064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:51:09.432106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319091+0000","last_change":"2024-05-08T23:01:56.262066+0000","last_active":"2024-05-08T23:03:19.319091+0000","last_peered":"2024-05-08T23:03:19.319091+0000","last_clean":"2024-05-08T23:03:19.319091+0000","last_became_active":"2024-05-08T23:01:56.261601+0000","last_became_peered":"2024-05-08T23:01:56.261601+0000","last_unstale":"2024-05-08T23:03:19.319091+0000","last_undegraded":"2024-05-08T23:03:19.319091+0000","last_fullsized":"2024-05-08T23:03:19.319091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:35.225110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.029438+0000","last_change":"2024-05-08T23:01:50.965672+0000","last_active":"2024-05-08T23:03:19.029438+0000","last_peered":"2024-05-08T23:03:19.029438+0000","last_clean":"2024-05-08T23:03:19.029438+0000","last_became_active":"2024-05-08T23:01:50.965297+0000","last_became_peered":"2024-05-08T23:01:50.965297+0000","last_unstale":"2024-05-08T23:03:19.029438+0000","last_undegraded":"2024-05-08T23:03:19.029438+0000","last_fullsized":"2024-05-08T23:03:19.029438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:19.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134037+0000","last_change":"2024-05-08T23:01:51.503433+0000","last_active":"2024-05-08T23:03:48.134037+0000","last_peered":"2024-05-08T23:03:48.134037+0000","last_clean":"2024-05-08T23:03:48.134037+0000","last_became_active":"2024-05-08T23:01:51.503112+0000","last_became_peered":"2024-05-08T23:01:51.503112+0000","last_unstale":"2024-05-08T23:03:48.134037+0000","last_undegraded":"2024-05-08T23:03:48.134037+0000","last_fullsized":"2024-05-08T23:03:48.134037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:14.887844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026364+0000","last_change":"2024-05-08T23:01:55.966259+0000","last_active":"2024-05-08T23:03:19.026364+0000","last_peered":"2024-05-08T23:03:19.026364+0000","last_clean":"2024-05-08T23:03:19.026364+0000","last_became_active":"2024-05-08T23:01:55.965802+0000","last_became_peered":"2024-05-08T23:01:55.965802+0000","last_unstale":"2024-05-08T23:03:19.026364+0000","last_undegraded":"2024-05-08T23:03:19.026364+0000","last_fullsized":"2024-05-08T23:03:19.026364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:57:53.195074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":126,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.771144+0000","last_change":"2024-05-08T23:01:50.961631+0000","last_active":"2024-05-08T23:03:52.771144+0000","last_peered":"2024-05-08T23:03:52.771144+0000","last_clean":"2024-05-08T23:03:52.771144+0000","last_became_active":"2024-05-08T23:01:50.961234+0000","last_became_peered":"2024-05-08T23:01:50.961234+0000","last_unstale":"2024-05-08T23:03:52.771144+0000","last_undegraded":"2024-05-08T23:03:52.771144+0000","last_fullsized":"2024-05-08T23:03:52.771144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:34.759338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1243,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.135922+0000","last_change":"2024-05-08T23:01:51.502889+0000","last_active":"2024-05-08T23:03:48.135922+0000","last_peered":"2024-05-08T23:03:48.135922+0000","last_clean":"2024-05-08T23:03:48.135922+0000","last_became_active":"2024-05-08T23:01:51.502536+0000","last_became_peered":"2024-05-08T23:01:51.502536+0000","last_unstale":"2024-05-08T23:03:48.135922+0000","last_undegraded":"2024-05-08T23:03:48.135922+0000","last_fullsized":"2024-05-08T23:03:48.135922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:43.679561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026697+0000","last_change":"2024-05-08T23:01:56.262552+0000","last_active":"2024-05-08T23:03:19.026697+0000","last_peered":"2024-05-08T23:03:19.026697+0000","last_clean":"2024-05-08T23:03:19.026697+0000","last_became_active":"2024-05-08T23:01:56.261959+0000","last_became_peered":"2024-05-08T23:01:56.261959+0000","last_unstale":"2024-05-08T23:03:19.026697+0000","last_undegraded":"2024-05-08T23:03:19.026697+0000","last_fullsized":"2024-05-08T23:03:19.026697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:56.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.120851+0000","last_change":"2024-05-08T23:01:51.485313+0000","last_active":"2024-05-08T23:03:48.120851+0000","last_peered":"2024-05-08T23:03:48.120851+0000","last_clean":"2024-05-08T23:03:48.120851+0000","last_became_active":"2024-05-08T23:01:51.484991+0000","last_became_peered":"2024-05-08T23:01:51.484991+0000","last_unstale":"2024-05-08T23:03:48.120851+0000","last_undegraded":"2024-05-08T23:03:48.120851+0000","last_fullsized":"2024-05-08T23:03:48.120851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:34.359093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":124,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:53.772498+0000","last_change":"2024-05-08T23:01:50.964042+0000","last_active":"2024-05-08T23:03:53.772498+0000","last_peered":"2024-05-08T23:03:53.772498+0000","last_clean":"2024-05-08T23:03:53.772498+0000","last_became_active":"2024-05-08T23:01:50.963731+0000","last_became_peered":"2024-05-08T23:01:50.963731+0000","last_unstale":"2024-05-08T23:03:53.772498+0000","last_undegraded":"2024-05-08T23:03:53.772498+0000","last_fullsized":"2024-05-08T23:03:53.772498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:50.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.024808+0000","last_change":"2024-05-08T23:01:55.965109+0000","last_active":"2024-05-08T23:03:19.024808+0000","last_peered":"2024-05-08T23:03:19.024808+0000","last_clean":"2024-05-08T23:03:19.024808+0000","last_became_active":"2024-05-08T23:01:55.964673+0000","last_became_peered":"2024-05-08T23:01:55.964673+0000","last_unstale":"2024-05-08T23:03:19.024808+0000","last_undegraded":"2024-05-08T23:03:19.024808+0000","last_fullsized":"2024-05-08T23:03:19.024808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:00:15.011260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137285+0000","last_change":"2024-05-08T23:01:51.492679+0000","last_active":"2024-05-08T23:03:48.137285+0000","last_peered":"2024-05-08T23:03:48.137285+0000","last_clean":"2024-05-08T23:03:48.137285+0000","last_became_active":"2024-05-08T23:01:51.492373+0000","last_became_peered":"2024-05-08T23:01:51.492373+0000","last_unstale":"2024-05-08T23:03:48.137285+0000","last_undegraded":"2024-05-08T23:03:48.137285+0000","last_fullsized":"2024-05-08T23:03:48.137285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:15.942222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648728+0000","last_change":"2024-05-08T23:01:50.953665+0000","last_active":"2024-05-08T23:03:19.648728+0000","last_peered":"2024-05-08T23:03:19.648728+0000","last_clean":"2024-05-08T23:03:19.648728+0000","last_became_active":"2024-05-08T23:01:50.953241+0000","last_became_peered":"2024-05-08T23:01:50.953241+0000","last_unstale":"2024-05-08T23:03:19.648728+0000","last_undegraded":"2024-05-08T23:03:19.648728+0000","last_fullsized":"2024-05-08T23:03:19.648728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:35.482635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":385,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:50.222340+0000","last_change":"2024-05-08T23:00:56.873533+0000","last_active":"2024-05-08T23:03:50.222340+0000","last_peered":"2024-05-08T23:03:50.222340+0000","last_clean":"2024-05-08T23:03:50.222340+0000","last_became_active":"2024-05-08T23:00:56.873176+0000","last_became_peered":"2024-05-08T23:00:56.873176+0000","last_unstale":"2024-05-08T23:03:50.222340+0000","last_undegraded":"2024-05-08T23:03:50.222340+0000","last_fullsized":"2024-05-08T23:03:50.222340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:58:17.520065+0000","last_clean_scrub_stamp":"2024-05-08T22:58:17.520065+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:12.313768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.229833+0000","last_change":"2024-05-08T23:01:56.262193+0000","last_active":"2024-05-08T23:03:19.229833+0000","last_peered":"2024-05-08T23:03:19.229833+0000","last_clean":"2024-05-08T23:03:19.229833+0000","last_became_active":"2024-05-08T23:01:56.261761+0000","last_became_peered":"2024-05-08T23:01:56.261761+0000","last_unstale":"2024-05-08T23:03:19.229833+0000","last_undegraded":"2024-05-08T23:03:19.229833+0000","last_fullsized":"2024-05-08T23:03:19.229833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:17.800402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.141189+0000","last_change":"2024-05-08T23:01:51.495372+0000","last_active":"2024-05-08T23:03:48.141189+0000","last_peered":"2024-05-08T23:03:48.141189+0000","last_clean":"2024-05-08T23:03:48.141189+0000","last_became_active":"2024-05-08T23:01:51.494909+0000","last_became_peered":"2024-05-08T23:01:51.494909+0000","last_unstale":"2024-05-08T23:03:48.141189+0000","last_undegraded":"2024-05-08T23:03:48.141189+0000","last_fullsized":"2024-05-08T23:03:48.141189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:35.917255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566669+0000","last_change":"2024-05-08T23:02:18.667675+0000","last_active":"2024-05-08T23:03:19.566669+0000","last_peered":"2024-05-08T23:03:19.566669+0000","last_clean":"2024-05-08T23:03:19.566669+0000","last_became_active":"2024-05-08T23:02:18.662151+0000","last_became_peered":"2024-05-08T23:02:18.662151+0000","last_unstale":"2024-05-08T23:03:19.566669+0000","last_undegraded":"2024-05-08T23:03:19.566669+0000","last_fullsized":"2024-05-08T23:03:19.566669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.861138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.566734+0000","last_change":"2024-05-08T23:01:55.963740+0000","last_active":"2024-05-08T23:03:19.566734+0000","last_peered":"2024-05-08T23:03:19.566734+0000","last_clean":"2024-05-08T23:03:19.566734+0000","last_became_active":"2024-05-08T23:01:55.963155+0000","last_became_peered":"2024-05-08T23:01:55.963155+0000","last_unstale":"2024-05-08T23:03:19.566734+0000","last_undegraded":"2024-05-08T23:03:19.566734+0000","last_fullsized":"2024-05-08T23:03:19.566734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:34.673422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319413+0000","last_change":"2024-05-08T23:01:50.959690+0000","last_active":"2024-05-08T23:03:19.319413+0000","last_peered":"2024-05-08T23:03:19.319413+0000","last_clean":"2024-05-08T23:03:19.319413+0000","last_became_active":"2024-05-08T23:01:50.959146+0000","last_became_peered":"2024-05-08T23:01:50.959146+0000","last_unstale":"2024-05-08T23:03:19.319413+0000","last_undegraded":"2024-05-08T23:03:19.319413+0000","last_fullsized":"2024-05-08T23:03:19.319413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.141555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142913+0000","last_change":"2024-05-08T23:01:51.487105+0000","last_active":"2024-05-08T23:03:48.142913+0000","last_peered":"2024-05-08T23:03:48.142913+0000","last_clean":"2024-05-08T23:03:48.142913+0000","last_became_active":"2024-05-08T23:01:51.486771+0000","last_became_peered":"2024-05-08T23:01:51.486771+0000","last_unstale":"2024-05-08T23:03:48.142913+0000","last_undegraded":"2024-05-08T23:03:48.142913+0000","last_fullsized":"2024-05-08T23:03:48.142913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:26.220149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023762+0000","last_change":"2024-05-08T23:01:56.261692+0000","last_active":"2024-05-08T23:03:19.023762+0000","last_peered":"2024-05-08T23:03:19.023762+0000","last_clean":"2024-05-08T23:03:19.023762+0000","last_became_active":"2024-05-08T23:01:56.261233+0000","last_became_peered":"2024-05-08T23:01:56.261233+0000","last_unstale":"2024-05-08T23:03:19.023762+0000","last_undegraded":"2024-05-08T23:03:19.023762+0000","last_fullsized":"2024-05-08T23:03:19.023762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:12.583696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'45","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.383589+0000","last_change":"2024-05-08T23:01:50.953917+0000","last_active":"2024-05-08T23:03:48.383589+0000","last_peered":"2024-05-08T23:03:48.383589+0000","last_clean":"2024-05-08T23:03:48.383589+0000","last_became_active":"2024-05-08T23:01:50.953063+0000","last_became_peered":"2024-05-08T23:01:50.953063+0000","last_unstale":"2024-05-08T23:03:48.383589+0000","last_undegraded":"2024-05-08T23:03:48.383589+0000","last_fullsized":"2024-05-08T23:03:48.383589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:38.029829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143365+0000","last_change":"2024-05-08T23:01:51.487013+0000","last_active":"2024-05-08T23:03:48.143365+0000","last_peered":"2024-05-08T23:03:48.143365+0000","last_clean":"2024-05-08T23:03:48.143365+0000","last_became_active":"2024-05-08T23:01:51.486715+0000","last_became_peered":"2024-05-08T23:01:51.486715+0000","last_unstale":"2024-05-08T23:03:48.143365+0000","last_undegraded":"2024-05-08T23:03:48.143365+0000","last_fullsized":"2024-05-08T23:03:48.143365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:06.497486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.025789+0000","last_change":"2024-05-08T23:01:55.961357+0000","last_active":"2024-05-08T23:03:19.025789+0000","last_peered":"2024-05-08T23:03:19.025789+0000","last_clean":"2024-05-08T23:03:19.025789+0000","last_became_active":"2024-05-08T23:01:55.960940+0000","last_became_peered":"2024-05-08T23:01:55.960940+0000","last_unstale":"2024-05-08T23:03:19.025789+0000","last_undegraded":"2024-05-08T23:03:19.025789+0000","last_fullsized":"2024-05-08T23:03:19.025789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:21.390341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649256+0000","last_change":"2024-05-08T23:01:50.956414+0000","last_active":"2024-05-08T23:03:19.649256+0000","last_peered":"2024-05-08T23:03:19.649256+0000","last_clean":"2024-05-08T23:03:19.649256+0000","last_became_active":"2024-05-08T23:01:50.956059+0000","last_became_peered":"2024-05-08T23:01:50.956059+0000","last_unstale":"2024-05-08T23:03:19.649256+0000","last_undegraded":"2024-05-08T23:03:19.649256+0000","last_fullsized":"2024-05-08T23:03:19.649256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:01.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.145687+0000","last_change":"2024-05-08T23:01:51.492518+0000","last_active":"2024-05-08T23:03:48.145687+0000","last_peered":"2024-05-08T23:03:48.145687+0000","last_clean":"2024-05-08T23:03:48.145687+0000","last_became_active":"2024-05-08T23:01:51.490647+0000","last_became_peered":"2024-05-08T23:01:51.490647+0000","last_unstale":"2024-05-08T23:03:48.145687+0000","last_undegraded":"2024-05-08T23:03:48.145687+0000","last_fullsized":"2024-05-08T23:03:48.145687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:18.920500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649029+0000","last_change":"2024-05-08T23:01:56.263661+0000","last_active":"2024-05-08T23:03:19.649029+0000","last_peered":"2024-05-08T23:03:19.649029+0000","last_clean":"2024-05-08T23:03:19.649029+0000","last_became_active":"2024-05-08T23:01:56.263156+0000","last_became_peered":"2024-05-08T23:01:56.263156+0000","last_unstale":"2024-05-08T23:03:19.649029+0000","last_undegraded":"2024-05-08T23:03:19.649029+0000","last_fullsized":"2024-05-08T23:03:19.649029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.578476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648003+0000","last_change":"2024-05-08T23:01:50.960792+0000","last_active":"2024-05-08T23:03:19.648003+0000","last_peered":"2024-05-08T23:03:19.648003+0000","last_clean":"2024-05-08T23:03:19.648003+0000","last_became_active":"2024-05-08T23:01:50.960203+0000","last_became_peered":"2024-05-08T23:01:50.960203+0000","last_unstale":"2024-05-08T23:03:19.648003+0000","last_undegraded":"2024-05-08T23:03:19.648003+0000","last_fullsized":"2024-05-08T23:03:19.648003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.003002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.093132+0000","last_change":"2024-05-08T23:01:51.488348+0000","last_active":"2024-05-08T23:03:48.093132+0000","last_peered":"2024-05-08T23:03:48.093132+0000","last_clean":"2024-05-08T23:03:48.093132+0000","last_became_active":"2024-05-08T23:01:51.487980+0000","last_became_peered":"2024-05-08T23:01:51.487980+0000","last_unstale":"2024-05-08T23:03:48.093132+0000","last_undegraded":"2024-05-08T23:03:48.093132+0000","last_fullsized":"2024-05-08T23:03:48.093132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:39.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.030035+0000","last_change":"2024-05-08T23:01:55.972379+0000","last_active":"2024-05-08T23:03:19.030035+0000","last_peered":"2024-05-08T23:03:19.030035+0000","last_clean":"2024-05-08T23:03:19.030035+0000","last_became_active":"2024-05-08T23:01:55.971847+0000","last_became_peered":"2024-05-08T23:01:55.971847+0000","last_unstale":"2024-05-08T23:03:19.030035+0000","last_undegraded":"2024-05-08T23:03:19.030035+0000","last_fullsized":"2024-05-08T23:03:19.030035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:33.027204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.023898+0000","last_change":"2024-05-08T23:01:50.953661+0000","last_active":"2024-05-08T23:03:19.023898+0000","last_peered":"2024-05-08T23:03:19.023898+0000","last_clean":"2024-05-08T23:03:19.023898+0000","last_became_active":"2024-05-08T23:01:50.953314+0000","last_became_peered":"2024-05-08T23:01:50.953314+0000","last_unstale":"2024-05-08T23:03:19.023898+0000","last_undegraded":"2024-05-08T23:03:19.023898+0000","last_fullsized":"2024-05-08T23:03:19.023898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:04.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.142729+0000","last_change":"2024-05-08T23:01:51.486638+0000","last_active":"2024-05-08T23:03:48.142729+0000","last_peered":"2024-05-08T23:03:48.142729+0000","last_clean":"2024-05-08T23:03:48.142729+0000","last_became_active":"2024-05-08T23:01:51.486133+0000","last_became_peered":"2024-05-08T23:01:51.486133+0000","last_unstale":"2024-05-08T23:03:48.142729+0000","last_undegraded":"2024-05-08T23:03:48.142729+0000","last_fullsized":"2024-05-08T23:03:48.142729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:08.369215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648771+0000","last_change":"2024-05-08T23:01:55.971549+0000","last_active":"2024-05-08T23:03:19.648771+0000","last_peered":"2024-05-08T23:03:19.648771+0000","last_clean":"2024-05-08T23:03:19.648771+0000","last_became_active":"2024-05-08T23:01:55.971069+0000","last_became_peered":"2024-05-08T23:01:55.971069+0000","last_unstale":"2024-05-08T23:03:19.648771+0000","last_undegraded":"2024-05-08T23:03:19.648771+0000","last_fullsized":"2024-05-08T23:03:19.648771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:58.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026057+0000","last_change":"2024-05-08T23:01:50.964439+0000","last_active":"2024-05-08T23:03:19.026057+0000","last_peered":"2024-05-08T23:03:19.026057+0000","last_clean":"2024-05-08T23:03:19.026057+0000","last_became_active":"2024-05-08T23:01:50.964079+0000","last_became_peered":"2024-05-08T23:01:50.964079+0000","last_unstale":"2024-05-08T23:03:19.026057+0000","last_undegraded":"2024-05-08T23:03:19.026057+0000","last_fullsized":"2024-05-08T23:03:19.026057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:11.878549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150336+0000","last_change":"2024-05-08T23:01:51.498197+0000","last_active":"2024-05-08T23:03:48.150336+0000","last_peered":"2024-05-08T23:03:48.150336+0000","last_clean":"2024-05-08T23:03:48.150336+0000","last_became_active":"2024-05-08T23:01:51.487507+0000","last_became_peered":"2024-05-08T23:01:51.487507+0000","last_unstale":"2024-05-08T23:03:48.150336+0000","last_undegraded":"2024-05-08T23:03:48.150336+0000","last_fullsized":"2024-05-08T23:03:48.150336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:22.429812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.648702+0000","last_change":"2024-05-08T23:01:56.262314+0000","last_active":"2024-05-08T23:03:19.648702+0000","last_peered":"2024-05-08T23:03:19.648702+0000","last_clean":"2024-05-08T23:03:19.648702+0000","last_became_active":"2024-05-08T23:01:56.261797+0000","last_became_peered":"2024-05-08T23:01:56.261797+0000","last_unstale":"2024-05-08T23:03:19.648702+0000","last_undegraded":"2024-05-08T23:03:19.648702+0000","last_fullsized":"2024-05-08T23:03:19.648702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:07.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.319031+0000","last_change":"2024-05-08T23:01:50.958228+0000","last_active":"2024-05-08T23:03:19.319031+0000","last_peered":"2024-05-08T23:03:19.319031+0000","last_clean":"2024-05-08T23:03:19.319031+0000","last_became_active":"2024-05-08T23:01:50.957516+0000","last_became_peered":"2024-05-08T23:01:50.957516+0000","last_unstale":"2024-05-08T23:03:19.319031+0000","last_undegraded":"2024-05-08T23:03:19.319031+0000","last_fullsized":"2024-05-08T23:03:19.319031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:47.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.138554+0000","last_change":"2024-05-08T23:01:51.483685+0000","last_active":"2024-05-08T23:03:48.138554+0000","last_peered":"2024-05-08T23:03:48.138554+0000","last_clean":"2024-05-08T23:03:48.138554+0000","last_became_active":"2024-05-08T23:01:51.483359+0000","last_became_peered":"2024-05-08T23:01:51.483359+0000","last_unstale":"2024-05-08T23:03:48.138554+0000","last_undegraded":"2024-05-08T23:03:48.138554+0000","last_fullsized":"2024-05-08T23:03:48.138554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:59:07.451866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.649390+0000","last_change":"2024-05-08T23:01:55.969124+0000","last_active":"2024-05-08T23:03:19.649390+0000","last_peered":"2024-05-08T23:03:19.649390+0000","last_clean":"2024-05-08T23:03:19.649390+0000","last_became_active":"2024-05-08T23:01:55.968680+0000","last_became_peered":"2024-05-08T23:01:55.968680+0000","last_unstale":"2024-05-08T23:03:19.649390+0000","last_undegraded":"2024-05-08T23:03:19.649390+0000","last_fullsized":"2024-05-08T23:03:19.649390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:44.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:52.770901+0000","last_change":"2024-05-08T23:02:18.672839+0000","last_active":"2024-05-08T23:03:52.770901+0000","last_peered":"2024-05-08T23:03:52.770901+0000","last_clean":"2024-05-08T23:03:52.770901+0000","last_became_active":"2024-05-08T23:02:18.665850+0000","last_became_peered":"2024-05-08T23:02:18.665850+0000","last_unstale":"2024-05-08T23:03:52.770901+0000","last_undegraded":"2024-05-08T23:03:52.770901+0000","last_fullsized":"2024-05-08T23:03:52.770901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:40.422775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1122,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.132905+0000","last_change":"2024-05-08T23:01:51.482291+0000","last_active":"2024-05-08T23:03:48.132905+0000","last_peered":"2024-05-08T23:03:48.132905+0000","last_clean":"2024-05-08T23:03:48.132905+0000","last_became_active":"2024-05-08T23:01:51.481931+0000","last_became_peered":"2024-05-08T23:01:51.481931+0000","last_unstale":"2024-05-08T23:03:48.132905+0000","last_undegraded":"2024-05-08T23:03:48.132905+0000","last_fullsized":"2024-05-08T23:03:48.132905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:15.583282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.647481+0000","last_change":"2024-05-08T23:01:55.974597+0000","last_active":"2024-05-08T23:03:19.647481+0000","last_peered":"2024-05-08T23:03:19.647481+0000","last_clean":"2024-05-08T23:03:19.647481+0000","last_became_active":"2024-05-08T23:01:55.973443+0000","last_became_peered":"2024-05-08T23:01:55.973443+0000","last_unstale":"2024-05-08T23:03:19.647481+0000","last_undegraded":"2024-05-08T23:03:19.647481+0000","last_fullsized":"2024-05-08T23:03:19.647481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:18.352042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.377743+0000","last_change":"2024-05-08T23:01:50.965423+0000","last_active":"2024-05-08T23:03:48.377743+0000","last_peered":"2024-05-08T23:03:48.377743+0000","last_clean":"2024-05-08T23:03:48.377743+0000","last_became_active":"2024-05-08T23:01:50.965055+0000","last_became_peered":"2024-05-08T23:01:50.965055+0000","last_unstale":"2024-05-08T23:03:48.377743+0000","last_undegraded":"2024-05-08T23:03:48.377743+0000","last_fullsized":"2024-05-08T23:03:48.377743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:27.167883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126672+0000","last_change":"2024-05-08T23:01:51.485884+0000","last_active":"2024-05-08T23:03:48.126672+0000","last_peered":"2024-05-08T23:03:48.126672+0000","last_clean":"2024-05-08T23:03:48.126672+0000","last_became_active":"2024-05-08T23:01:51.485312+0000","last_became_peered":"2024-05-08T23:01:51.485312+0000","last_unstale":"2024-05-08T23:03:48.126672+0000","last_undegraded":"2024-05-08T23:03:48.126672+0000","last_fullsized":"2024-05-08T23:03:48.126672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:01.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.228724+0000","last_change":"2024-05-08T23:01:55.961501+0000","last_active":"2024-05-08T23:03:19.228724+0000","last_peered":"2024-05-08T23:03:19.228724+0000","last_clean":"2024-05-08T23:03:19.228724+0000","last_became_active":"2024-05-08T23:01:55.961038+0000","last_became_peered":"2024-05-08T23:01:55.961038+0000","last_unstale":"2024-05-08T23:03:19.228724+0000","last_undegraded":"2024-05-08T23:03:19.228724+0000","last_fullsized":"2024-05-08T23:03:19.228724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:54.942532+0000","last_clean_scrub_stamp":"2024-05-08T23:01:54.942532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:47.094203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T23:03:19.026628+0000","last_change":"2024-05-08T23:02:18.667814+0000","last_active":"2024-05-08T23:03:19.026628+0000","last_peered":"2024-05-08T23:03:19.026628+0000","last_clean":"2024-05-08T23:03:19.026628+0000","last_became_active":"2024-05-08T23:02:18.658985+0000","last_became_peered":"2024-05-08T23:02:18.658985+0000","last_unstale":"2024-05-08T23:03:19.026628+0000","last_undegraded":"2024-05-08T23:03:19.026628+0000","last_fullsized":"2024-05-08T23:03:19.026628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:49.924676+0000","last_clean_scrub_stamp":"2024-05-08T23:01:49.924676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:34.618155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.126885+0000","last_change":"2024-05-08T23:01:51.476552+0000","last_active":"2024-05-08T23:03:48.126885+0000","last_peered":"2024-05-08T23:03:48.126885+0000","last_clean":"2024-05-08T23:03:48.126885+0000","last_became_active":"2024-05-08T23:01:51.476181+0000","last_became_peered":"2024-05-08T23:01:51.476181+0000","last_unstale":"2024-05-08T23:03:48.126885+0000","last_undegraded":"2024-05-08T23:03:48.126885+0000","last_fullsized":"2024-05-08T23:03:48.126885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:10.741356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.133238+0000","last_change":"2024-05-08T23:01:51.491778+0000","last_active":"2024-05-08T23:03:48.133238+0000","last_peered":"2024-05-08T23:03:48.133238+0000","last_clean":"2024-05-08T23:03:48.133238+0000","last_became_active":"2024-05-08T23:01:51.491415+0000","last_became_peered":"2024-05-08T23:01:51.491415+0000","last_unstale":"2024-05-08T23:03:48.133238+0000","last_undegraded":"2024-05-08T23:03:48.133238+0000","last_fullsized":"2024-05-08T23:03:48.133238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:27.872468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124622+0000","last_change":"2024-05-08T23:01:51.496982+0000","last_active":"2024-05-08T23:03:48.124622+0000","last_peered":"2024-05-08T23:03:48.124622+0000","last_clean":"2024-05-08T23:03:48.124622+0000","last_became_active":"2024-05-08T23:01:51.496636+0000","last_became_peered":"2024-05-08T23:01:51.496636+0000","last_unstale":"2024-05-08T23:03:48.124622+0000","last_undegraded":"2024-05-08T23:03:48.124622+0000","last_fullsized":"2024-05-08T23:03:48.124622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:38.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.124829+0000","last_change":"2024-05-08T23:01:51.485367+0000","last_active":"2024-05-08T23:03:48.124829+0000","last_peered":"2024-05-08T23:03:48.124829+0000","last_clean":"2024-05-08T23:03:48.124829+0000","last_became_active":"2024-05-08T23:01:51.485061+0000","last_became_peered":"2024-05-08T23:01:51.485061+0000","last_unstale":"2024-05-08T23:03:48.124829+0000","last_undegraded":"2024-05-08T23:03:48.124829+0000","last_fullsized":"2024-05-08T23:03:48.124829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:50.119714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.150811+0000","last_change":"2024-05-08T23:01:51.490728+0000","last_active":"2024-05-08T23:03:48.150811+0000","last_peered":"2024-05-08T23:03:48.150811+0000","last_clean":"2024-05-08T23:03:48.150811+0000","last_became_active":"2024-05-08T23:01:51.490442+0000","last_became_peered":"2024-05-08T23:01:51.490442+0000","last_unstale":"2024-05-08T23:03:48.150811+0000","last_undegraded":"2024-05-08T23:03:48.150811+0000","last_fullsized":"2024-05-08T23:03:48.150811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:44.027171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.149706+0000","last_change":"2024-05-08T23:01:51.492021+0000","last_active":"2024-05-08T23:03:48.149706+0000","last_peered":"2024-05-08T23:03:48.149706+0000","last_clean":"2024-05-08T23:03:48.149706+0000","last_became_active":"2024-05-08T23:01:51.490444+0000","last_became_peered":"2024-05-08T23:01:51.490444+0000","last_unstale":"2024-05-08T23:03:48.149706+0000","last_undegraded":"2024-05-08T23:03:48.149706+0000","last_fullsized":"2024-05-08T23:03:48.149706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:40.282545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.130871+0000","last_change":"2024-05-08T23:01:51.480859+0000","last_active":"2024-05-08T23:03:48.130871+0000","last_peered":"2024-05-08T23:03:48.130871+0000","last_clean":"2024-05-08T23:03:48.130871+0000","last_became_active":"2024-05-08T23:01:51.480383+0000","last_became_peered":"2024-05-08T23:01:51.480383+0000","last_unstale":"2024-05-08T23:03:48.130871+0000","last_undegraded":"2024-05-08T23:03:48.130871+0000","last_fullsized":"2024-05-08T23:03:48.130871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:55.156061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.125971+0000","last_change":"2024-05-08T23:01:51.489369+0000","last_active":"2024-05-08T23:03:48.125971+0000","last_peered":"2024-05-08T23:03:48.125971+0000","last_clean":"2024-05-08T23:03:48.125971+0000","last_became_active":"2024-05-08T23:01:51.488891+0000","last_became_peered":"2024-05-08T23:01:51.488891+0000","last_unstale":"2024-05-08T23:03:48.125971+0000","last_undegraded":"2024-05-08T23:03:48.125971+0000","last_fullsized":"2024-05-08T23:03:48.125971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:13.859296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.137011+0000","last_change":"2024-05-08T23:02:18.661450+0000","last_active":"2024-05-08T23:03:48.137011+0000","last_peered":"2024-05-08T23:03:48.137011+0000","last_clean":"2024-05-08T23:03:48.137011+0000","last_became_active":"2024-05-08T23:02:18.661041+0000","last_became_peered":"2024-05-08T23:02:18.661041+0000","last_unstale":"2024-05-08T23:03:48.137011+0000","last_undegraded":"2024-05-08T23:03:48.137011+0000","last_fullsized":"2024-05-08T23:03:48.137011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:06.393479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143616+0000","last_change":"2024-05-08T23:01:51.492498+0000","last_active":"2024-05-08T23:03:48.143616+0000","last_peered":"2024-05-08T23:03:48.143616+0000","last_clean":"2024-05-08T23:03:48.143616+0000","last_became_active":"2024-05-08T23:01:51.492093+0000","last_became_peered":"2024-05-08T23:01:51.492093+0000","last_unstale":"2024-05-08T23:03:48.143616+0000","last_undegraded":"2024-05-08T23:03:48.143616+0000","last_fullsized":"2024-05-08T23:03:48.143616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:31.438578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.143122+0000","last_change":"2024-05-08T23:01:51.493207+0000","last_active":"2024-05-08T23:03:48.143122+0000","last_peered":"2024-05-08T23:03:48.143122+0000","last_clean":"2024-05-08T23:03:48.143122+0000","last_became_active":"2024-05-08T23:01:51.492292+0000","last_became_peered":"2024-05-08T23:01:51.492292+0000","last_unstale":"2024-05-08T23:03:48.143122+0000","last_undegraded":"2024-05-08T23:03:48.143122+0000","last_fullsized":"2024-05-08T23:03:48.143122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:24.365799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.134728+0000","last_change":"2024-05-08T23:01:51.498276+0000","last_active":"2024-05-08T23:03:48.134728+0000","last_peered":"2024-05-08T23:03:48.134728+0000","last_clean":"2024-05-08T23:03:48.134728+0000","last_became_active":"2024-05-08T23:01:51.497878+0000","last_became_peered":"2024-05-08T23:01:51.497878+0000","last_unstale":"2024-05-08T23:03:48.134728+0000","last_undegraded":"2024-05-08T23:03:48.134728+0000","last_fullsized":"2024-05-08T23:03:48.134728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:21.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.151641+0000","last_change":"2024-05-08T23:01:51.482601+0000","last_active":"2024-05-08T23:03:48.151641+0000","last_peered":"2024-05-08T23:03:48.151641+0000","last_clean":"2024-05-08T23:03:48.151641+0000","last_became_active":"2024-05-08T23:01:51.481638+0000","last_became_peered":"2024-05-08T23:01:51.481638+0000","last_unstale":"2024-05-08T23:03:48.151641+0000","last_undegraded":"2024-05-08T23:03:48.151641+0000","last_fullsized":"2024-05-08T23:03:48.151641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.221841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T23:03:48.131074+0000","last_change":"2024-05-08T23:01:51.490087+0000","last_active":"2024-05-08T23:03:48.131074+0000","last_peered":"2024-05-08T23:03:48.131074+0000","last_clean":"2024-05-08T23:03:48.131074+0000","last_became_active":"2024-05-08T23:01:51.489772+0000","last_became_peered":"2024-05-08T23:01:51.489772+0000","last_unstale":"2024-05-08T23:03:48.131074+0000","last_undegraded":"2024-05-08T23:03:48.131074+0000","last_fullsized":"2024-05-08T23:03:48.131074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:01:50.930938+0000","last_clean_scrub_stamp":"2024-05-08T23:01:50.930938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:47.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107673,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602141,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1086,"num_read_kb":660,"num_write":371,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1726,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806423,"data_compressed":81453,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612642,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":970752,"data_stored":495994,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.755,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":1.095,"15min":1.095},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.72099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.56399999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85899999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.499},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93100000000000005,"5min":1.218,"15min":1.218},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0169999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed May 8 23:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.628,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.196,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.621},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.63100000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68500000000000005}]},{"osd":10,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.622,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":1.387,"15min":1.387},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.66000000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97099999999999997,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.66200000000000003}]}]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":1282815,"data_compressed":70405,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.505}]},{"osd":1,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83799999999999997,"15min":0.82999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.69599999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.624}]},{"osd":9,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86099999999999999,"15min":0.85899999999999999},"min":{"1min":0.63,"5min":0.622,"15min":0.622},"max":{"1min":1.4279999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.90100000000000002}]},{"osd":11,"last update":"Wed May 8 23:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.82199999999999995},"min":{"1min":0.70999999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.008,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0700000000000001,"5min":1.091,"15min":1.091},"last":0.754}]}]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":892476,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":1.919,"15min":1.919},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84099999999999997,"5min":1.357,"15min":1.357},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71599999999999997,"15min":0.72799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.006,"5min":1.639,"15min":1.639},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93500000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.755}]},{"osd":2,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.46800000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96499999999999997,"5min":1.488,"15min":1.488},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93899999999999995,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":1.371,"15min":1.371},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.71199999999999997,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95099999999999996,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.80400000000000005,"15min":0.84499999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89100000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.81899999999999995,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.077,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82799999999999996,"15min":0.86399999999999999},"min":{"1min":0.53200000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.738,"15min":1.738},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.86799999999999999,"15min":0.90100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.038,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed May 8 23:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.89100000000000001,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95799999999999996,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.92000000000000004,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.008,"5min":1.673,"15min":1.673},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.89700000000000002,"15min":0.94499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99299999999999999,"5min":1.881,"15min":1.881},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.57299999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.157,"5min":1.766,"15min":1.766},"last":0.77600000000000002}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94599999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.78500000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.746}]}]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":888064,"data_compressed":45655,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.441},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.55200000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48799999999999999,"15min":0.47199999999999998},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed May 8 23:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.621},"min":{"1min":0.48499999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78500000000000003,"5min":1.079,"15min":1.079},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.372}]},{"osd":5,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0660000000000001,"5min":1.139,"15min":1.139},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.629,"5min":0.441,"15min":0.441},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.497},{"interface":"front","average":{"1min":0.745,"5min":0.70399999999999996,"15min":0.69099999999999995},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89700000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.59299999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.53300000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.58699999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.877,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.62,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85499999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54000000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50700000000000001}]}]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":1040384,"data_stored":1094313,"data_compressed":62400,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.45000000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":1.109,"15min":1.109},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79400000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.622},"min":{"1min":0.58899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88800000000000001,"5min":1.05,"15min":1.05},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.748},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.89100000000000001}]},{"osd":8,"last update":"Wed May 8 23:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89000000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84899999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:46 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.91000000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96699999999999997,"5min":1.141,"15min":1.141},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.85599999999999998,"15min":0.86599999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98599999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.877}]},{"osd":11,"last update":"Wed May 8 23:03:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.01,"5min":1.052,"15min":1.052},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.85899999999999999,"15min":0.86099999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0329999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.95499999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":966656,"data_stored":491323,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.504,"15min":0.46700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.503},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.0579999999999998,"5min":3.0579999999999998,"15min":3.0579999999999998},"last":0.91700000000000004}]},{"osd":1,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71899999999999997,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.73299999999999998,"15min":0.67500000000000004},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.97699999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71499999999999997,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.093,"5min":0.76100000000000001,"15min":0.66800000000000004},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.67200000000000004,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68300000000000005,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.417,"5min":1.581,"15min":1.581},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.77100000000000002,"15min":0.74099999999999999},"min":{"1min":0.499,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.78100000000000003,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.343,"5min":3.343,"15min":3.343},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78900000000000003,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.72599999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3380000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.879}]},{"osd":7,"last update":"Wed May 8 23:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.83199999999999996,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.81699999999999995,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.377,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":0.91000000000000003,"15min":0.83399999999999996},"min":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.86899999999999999,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":0.61799999999999999}]},{"osd":9,"last update":"Wed May 8 23:03:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.93200000000000005,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.268,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.93700000000000006}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.006,"15min":0.98499999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.002,"15min":1.0109999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0569999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.9710000000000001,"5min":3.9710000000000001,"15min":3.9710000000000001},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":1040384,"data_stored":560216,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57599999999999996,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.60199999999999998,"15min":0.54200000000000004},"min":{"1min":0.44600000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed May 8 23:03:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.878,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.626,"15min":0.54900000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.99099999999999999,"5min":1.069,"15min":1.069},"last":0.751}]},{"osd":3,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64500000000000002,"15min":0.57999999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.49199999999999999,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.46300000000000002},"min":{"1min":0.42699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.70599999999999996,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 8 23:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76300000000000001,"15min":0.69699999999999995},"min":{"1min":0.61899999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93200000000000005,"5min":1.21,"15min":1.21},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.65500000000000003},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83199999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.83799999999999997,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.66400000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0429999999999999,"5min":1.482,"15min":1.482},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1890000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.93799999999999994}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.96199999999999997}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.089}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":492862,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.081,"5min":1.431,"15min":1.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61799999999999999,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.129,"5min":2.125,"15min":2.125},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.73199999999999998,"15min":0.57799999999999996},"min":{"1min":0.503,"5min":0.25,"15min":0.25},"max":{"1min":6.8710000000000004,"5min":6.8710000000000004,"15min":6.8710000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.022,"5min":0.68899999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":6.8849999999999998,"5min":6.8849999999999998,"15min":6.8849999999999998},"last":0.61199999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2729999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2529999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.67100000000000004,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.28,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed May 8 23:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.79000000000000004,"15min":0.68899999999999995},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.78500000000000003,"15min":0.70099999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69999999999999996,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.093,"5min":1.921,"15min":1.921},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.58199999999999996,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.347,"5min":1.871,"15min":1.871},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":6.9630000000000001,"5min":6.9630000000000001,"15min":6.9630000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.98099999999999998}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5780000000000001,"5min":2.177,"15min":2.177},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.82299999999999995,"15min":0.80000000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.409,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.998}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5189999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.95099999999999996,"15min":0.93300000000000005},"min":{"1min":0.65200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4199999999999999,"5min":1.964,"15min":1.964},"last":0.89700000000000002}]},{"osd":10,"last update":"Wed May 8 23:02:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.97199999999999998,"15min":0.96299999999999997},"min":{"1min":0.71199999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.716,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.079},{"interface":"front","average":{"1min":1.05,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5680000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.006}]},{"osd":11,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.82499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.462,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.038},"min":{"1min":0.65900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6819999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0189999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1040384,"data_stored":557615,"data_compressed":28358,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77900000000000003,"5min":0.87,"15min":0.87},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.48499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77800000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85899999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67400000000000004,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.871,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.496},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67400000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.85999999999999999,"5min":0.879,"15min":0.879},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79300000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47299999999999998,"15min":0.44600000000000001},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39100000000000001}]},{"osd":6,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.71499999999999997,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84399999999999997,"5min":7.3529999999999998,"15min":7.3529999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90300000000000002,"5min":7.3259999999999996,"15min":7.3259999999999996},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70899999999999996}]},{"osd":8,"last update":"Wed May 8 23:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.35199999999999998}]},{"osd":9,"last update":"Wed May 8 23:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.747},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Wed May 8 23:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.05,"15min":1.05},"last":0.48499999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1014883,"data_compressed":51811,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.48999999999999999,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69099999999999995,"5min":1.242,"15min":1.242},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.80100000000000005,"15min":0.61899999999999999},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":13.832000000000001,"5min":13.832000000000001,"15min":13.832000000000001},"last":1.032},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.61199999999999999},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.012,"5min":1.046,"15min":1.046},"last":0.73099999999999998}]},{"osd":2,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.52400000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.0049999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.61899999999999999},"min":{"1min":0.47099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1040000000000001,"5min":1.845,"15min":1.845},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68799999999999994,"15min":0.64400000000000002},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.71999999999999997,"15min":0.64700000000000002},"min":{"1min":0.58599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.105}]},{"osd":5,"last update":"Wed May 8 23:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.76200000000000001,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.70999999999999996},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.80400000000000005}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1339999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed May 8 23:03:01 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":13.807,"5min":13.807,"15min":13.807},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.0680000000000001}]},{"osd":8,"last update":"Wed May 8 23:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.628},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.95099999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1459999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.92800000000000005,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97199999999999998,"5min":1.292,"15min":1.292},"last":0.82999999999999996}]},{"osd":10,"last update":"Wed May 8 23:02:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.0609999999999999,"15min":0.97399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.893000000000001,"5min":13.893000000000001,"15min":13.893000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.01,"15min":1.014},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.1439999999999999}]},{"osd":11,"last update":"Wed May 8 23:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":498006,"data_compressed":26509,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.46700000000000003,"15min":0.433},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.374,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63900000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed May 8 23:03:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90100000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.622,"15min":0.57399999999999995},"min":{"1min":0.48399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65600000000000003,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94499999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.70399999999999996},"min":{"1min":0.60999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.90600000000000003}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.57699999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90100000000000002,"5min":1.107,"15min":1.107},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0469999999999999,"5min":1.147,"15min":1.147},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed May 8 23:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.67400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":9,"last update":"Wed May 8 23:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.147,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.871},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.66700000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.391,"15min":1.391},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed May 8 23:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.86099999999999999,"15min":0.872},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89700000000000002,"15min":0.90700000000000003},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.089,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.002}]},{"osd":11,"last update":"Wed May 8 23:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89800000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1218894,"data_compressed":68556,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.51100000000000001},"min":{"1min":0.41699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.372},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56399999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed May 8 23:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.441,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.74099999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed May 8 23:03:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.57599999999999996},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.47799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed May 8 23:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51100000000000001,"15min":0.49199999999999999},"min":{"1min":0.45000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59099999999999997}]},{"osd":9,"last update":"Wed May 8 23:03:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.50900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.751}]},{"osd":10,"last update":"Wed May 8 23:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35891,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70579,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68647,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134765,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394022,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202501,"data_compressed":8005,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:03:57.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.781+0000 7fd709a00700 1 -- 172.21.15.46:0/1440031806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd704103270 msgr2=0x7fd704103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.781+0000 7fd709a00700 1 --2- 172.21.15.46:0/1440031806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd704103270 0x7fd704103650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009990 tx=0x7fd6f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.781+0000 7fd709a00700 1 -- 172.21.15.46:0/1440031806 shutdown_connections 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.781+0000 7fd709a00700 1 --2- 172.21.15.46:0/1440031806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 0x7fd70410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.782+0000 7fd709a00700 1 --2- 172.21.15.46:0/1440031806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd704103270 0x7fd704103650 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.782+0000 7fd709a00700 1 --2- 172.21.15.46:0/1440031806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103c20 0x7fd704107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.782+0000 7fd709a00700 1 -- 172.21.15.46:0/1440031806 >> 172.21.15.46:0/1440031806 conn(0x7fd7040fead0 msgr2=0x7fd704100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:57.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.782+0000 7fd709a00700 1 -- 172.21.15.46:0/1440031806 shutdown_connections 2024-05-08T23:03:57.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.782+0000 7fd709a00700 1 -- 172.21.15.46:0/1440031806 wait complete. 2024-05-08T23:03:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 Processor -- start 2024-05-08T23:03:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 -- start start 2024-05-08T23:03:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 0x7fd704199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd70419de40 0x7fd70419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419e8c0 con 0x7fd704108390 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419ea30 con 0x7fd704103270 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.783+0000 7fd709a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419ebf0 con 0x7fd70419de40 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 0x7fd704199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56984/0 (socket says 172.21.15.46:56984) 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 -- 172.21.15.46:0/2502357628 learned_addr learned my addr 172.21.15.46:0/2502357628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd703400700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd70419de40 0x7fd70419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 -- 172.21.15.46:0/2502357628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd70419de40 msgr2=0x7fd70419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd70419de40 0x7fd70419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 -- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 msgr2=0x7fd704199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 0x7fd704199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 -- 172.21.15.46:0/2502357628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40095f0 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.784+0000 7fd702a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000c00 tx=0x7fd6f4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.785+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f401d070 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.785+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6f4022470 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.785+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7041a60f0 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.785+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4017650 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.785+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041a6570 con 0x7fd704103270 2024-05-08T23:03:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.786+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70410b600 con 0x7fd704103270 2024-05-08T23:03:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.788+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fd6f40225e0 con 0x7fd704103270 2024-05-08T23:03:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.789+0000 7fd6fb400700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 0x7fd6e80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.789+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd6f40cec60 con 0x7fd704103270 2024-05-08T23:03:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.789+0000 7fd702000700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 0x7fd6e80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.790+0000 7fd702000700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 0x7fd6e80a38f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd70419aab0 tx=0x7fd6f0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:57.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:57.792+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6f4097b40 con 0x7fd704103270 2024-05-08T23:03:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:57 smithi086 ceph-mon[78610]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 879 B/s wr, 26 op/s 2024-05-08T23:03:57.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:57 smithi086 ceph-mon[78610]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:57 smithi053 ceph-mon[77587]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 879 B/s wr, 26 op/s 2024-05-08T23:03:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:57 smithi053 ceph-mon[77587]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.014+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd704104740 con 0x7fd704103270 2024-05-08T23:03:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.016+0000 7fd6fb400700 1 -- 172.21.15.46:0/2502357628 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+25062 (secure 0 0 0) 0x7fd6f409b160 con 0x7fd704103270 2024-05-08T23:03:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:03:58.016 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:48.069712+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 msgr2=0x7fd6e80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 0x7fd6e80a38f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd70419aab0 tx=0x7fd6f0009260 comp rx=0 tx=0).stop 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 msgr2=0x7fd7041995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000c00 tx=0x7fd6f4005740 comp rx=0 tx=0).stop 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 shutdown_connections 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd70419de40 0x7fd70419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd704103270 0x7fd7041995b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd6e80a1430 0x7fd6e80a38f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 --2- 172.21.15.46:0/2502357628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd704108390 0x7fd704199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 >> 172.21.15.46:0/2502357628 conn(0x7fd7040fead0 msgr2=0x7fd7040fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 shutdown_connections 2024-05-08T23:03:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.019+0000 7fd709a00700 1 -- 172.21.15.46:0/2502357628 wait complete. 2024-05-08T23:03:58.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:57 smithi046 ceph-mon[36898]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 879 B/s wr, 26 op/s 2024-05-08T23:03:58.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:57 smithi046 ceph-mon[36898]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:03:58.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 -- 172.21.15.46:0/1995514325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c101080 msgr2=0x7fb83c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 --2- 172.21.15.46:0/1995514325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c101080 0x7fb83c101460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009990 tx=0x7fb82c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 -- 172.21.15.46:0/1995514325 shutdown_connections 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 --2- 172.21.15.46:0/1995514325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c1061a0 0x7fb83c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 --2- 172.21.15.46:0/1995514325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c101080 0x7fb83c101460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 --2- 172.21.15.46:0/1995514325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c101a30 0x7fb83c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 -- 172.21.15.46:0/1995514325 >> 172.21.15.46:0/1995514325 conn(0x7fb83c0fc8e0 msgr2=0x7fb83c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 -- 172.21.15.46:0/1995514325 shutdown_connections 2024-05-08T23:03:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.241+0000 7fb840e00700 1 -- 172.21.15.46:0/1995514325 wait complete. 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.242+0000 7fb840e00700 1 Processor -- start 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 -- start start 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c101080 0x7fb83c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 0x7fb83c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c19c650 con 0x7fb83c1061a0 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c19c7c0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb840e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c19c980 con 0x7fb83c101080 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb839400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb83a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 0x7fb83c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb839e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c101080 0x7fb83c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb839400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33824/0 (socket says 172.21.15.46:33824) 2024-05-08T23:03:58.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.243+0000 7fb839400700 1 -- 172.21.15.46:0/471615941 learned_addr learned my addr 172.21.15.46:0/471615941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 -- 172.21.15.46:0/471615941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c101080 msgr2=0x7fb83c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c101080 0x7fb83c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 -- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 msgr2=0x7fb83c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 -- 172.21.15.46:0/471615941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0095f0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb839400700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb83a800700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 0x7fb83c19c050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb83400efc0 tx=0x7fb83400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834018070 con 0x7fb83c19bbd0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb83400f040 con 0x7fb83c19bbd0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400cd40 con 0x7fb83c19bbd0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c0731d0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.244+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c0736e0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.245+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c1093c0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.248+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fb834003a00 con 0x7fb83c19bbd0 2024-05-08T23:03:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.249+0000 7fb832a00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 0x7fb8200a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.249+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb8340cd840 con 0x7fb83c19bbd0 2024-05-08T23:03:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.249+0000 7fb839e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 0x7fb8200a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.250+0000 7fb839e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 0x7fb8200a38f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006010 tx=0x7fb82c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.252+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8340966c0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.468+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb83c1985f0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.469+0000 7fb832a00700 1 -- 172.21.15.46:0/471615941 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb834099ce0 con 0x7fb83c19bbd0 2024-05-08T23:03:58.469 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:03:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 msgr2=0x7fb8200a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 0x7fb8200a38f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006010 tx=0x7fb82c00b260 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 msgr2=0x7fb83c19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 0x7fb83c19c050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb83400efc0 tx=0x7fb83400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 shutdown_connections 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb83c19bbd0 0x7fb83c19c050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c101080 0x7fb83c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb8200a1430 0x7fb8200a38f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 --2- 172.21.15.46:0/471615941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb83c1061a0 0x7fb83c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.471+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 >> 172.21.15.46:0/471615941 conn(0x7fb83c0fc8e0 msgr2=0x7fb83c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.472+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 shutdown_connections 2024-05-08T23:03:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.472+0000 7fb840e00700 1 -- 172.21.15.46:0/471615941 wait complete. 2024-05-08T23:03:58.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:03:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 -- 172.21.15.46:0/2477082089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5708101080 msgr2=0x7f5708101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 --2- 172.21.15.46:0/2477082089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5708101080 0x7f5708101460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009990 tx=0x7f56fc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 -- 172.21.15.46:0/2477082089 shutdown_connections 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 --2- 172.21.15.46:0/2477082089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57081061a0 0x7f570810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 --2- 172.21.15.46:0/2477082089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5708101080 0x7f5708101460 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 --2- 172.21.15.46:0/2477082089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5708101a30 0x7f5708105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 -- 172.21.15.46:0/2477082089 >> 172.21.15.46:0/2477082089 conn(0x7f57080fc8e0 msgr2=0x7f57080fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 -- 172.21.15.46:0/2477082089 shutdown_connections 2024-05-08T23:03:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.696+0000 7f5710800700 1 -- 172.21.15.46:0/2477082089 wait complete. 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.697+0000 7f5710800700 1 Processor -- start 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.697+0000 7f5710800700 1 -- start start 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 0x7f5708197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f570819bc50 0x7f570819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570819c6d0 con 0x7f570819bc50 2024-05-08T23:03:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570819c840 con 0x7f5708101080 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f5710800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570819ca00 con 0x7f57081061a0 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 0x7f5708197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57018/0 (socket says 172.21.15.46:57018) 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 -- 172.21.15.46:0/3544920900 learned_addr learned my addr 172.21.15.46:0/3544920900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570e800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f570819bc50 0x7f570819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 -- 172.21.15.46:0/3544920900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 msgr2=0x7f5708197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 0x7f5708197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 -- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f570819bc50 msgr2=0x7f570819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f570819bc50 0x7f570819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570de00700 1 -- 172.21.15.46:0/3544920900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc0095f0 con 0x7f5708101080 2024-05-08T23:03:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.698+0000 7f570d400700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 0x7f5708197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.699+0000 7f570de00700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56fc000c00 tx=0x7f56fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.701+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc01d070 con 0x7f5708101080 2024-05-08T23:03:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.701+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56fc022470 con 0x7f5708101080 2024-05-08T23:03:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.701+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc017650 con 0x7f5708101080 2024-05-08T23:03:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.701+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57081a3f00 con 0x7f5708101080 2024-05-08T23:03:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.701+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57081a4270 con 0x7f5708101080 2024-05-08T23:03:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.702+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57081093c0 con 0x7f5708101080 2024-05-08T23:03:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.704+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f56fc0225e0 con 0x7f5708101080 2024-05-08T23:03:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.705+0000 7f5702a00700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 0x7f56f00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.706+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f56fc0ce610 con 0x7f5708101080 2024-05-08T23:03:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.706+0000 7f570d400700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 0x7f56f00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.706+0000 7f570d400700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 0x7f56f00a38f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57081988c0 tx=0x7f56f8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.710+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56fc097490 con 0x7f5708101080 2024-05-08T23:03:58.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:58 smithi046 ceph-mon[36898]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:58 smithi046 ceph-mon[36898]: from='client.25669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2502357628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:58 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/471615941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.933+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57081019a0 con 0x7f5708101080 2024-05-08T23:03:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.934+0000 7f5702a00700 1 -- 172.21.15.46:0/3544920900 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56fc09aab0 con 0x7f5708101080 2024-05-08T23:03:58.934 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:58.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 msgr2=0x7f56f00a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 0x7f56f00a38f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57081988c0 tx=0x7f56f8009260 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 msgr2=0x7f57081973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56fc000c00 tx=0x7f56fc005740 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 shutdown_connections 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56f00a1430 0x7f56f00a38f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f570819bc50 0x7f570819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708101080 0x7f57081973c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 --2- 172.21.15.46:0/3544920900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57081061a0 0x7f5708197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 >> 172.21.15.46:0/3544920900 conn(0x7f57080fc8e0 msgr2=0x7f57080fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 shutdown_connections 2024-05-08T23:03:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:58.936+0000 7f5710800700 1 -- 172.21.15.46:0/3544920900 wait complete. 2024-05-08T23:03:58.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:03:58.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:58 smithi086 ceph-mon[78610]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:58 smithi086 ceph-mon[78610]: from='client.25669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:58.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2502357628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:58.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:58 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/471615941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:59.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:58 smithi053 ceph-mon[77587]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:59.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:58 smithi053 ceph-mon[77587]: from='client.25669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:03:59.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2502357628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:03:59.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:58 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/471615941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:03:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 -- 172.21.15.46:0/1665378020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073390 msgr2=0x7ffb58073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1665378020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073390 0x7ffb58073770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009990 tx=0x7ffb48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 -- 172.21.15.46:0/1665378020 shutdown_connections 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1665378020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1665378020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073390 0x7ffb58073770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1665378020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073cb0 0x7ffb5810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 -- 172.21.15.46:0/1665378020 >> 172.21.15.46:0/1665378020 conn(0x7ffb580fc8e0 msgr2=0x7ffb580fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 -- 172.21.15.46:0/1665378020 shutdown_connections 2024-05-08T23:03:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.160+0000 7ffb5e400700 1 -- 172.21.15.46:0/1665378020 wait complete. 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.161+0000 7ffb5e400700 1 Processor -- start 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.161+0000 7ffb5e400700 1 -- start start 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5819e450 con 0x7ffb58073cb0 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5819e5c0 con 0x7ffb58073390 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb5e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5819e780 con 0x7ffb5810d690 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb56a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35014/0 (socket says 172.21.15.46:35014) 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57036/0 (socket says 172.21.15.46:57036) 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 -- 172.21.15.46:0/1862761354 learned_addr learned my addr 172.21.15.46:0/1862761354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb56a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33858/0 (socket says 172.21.15.46:33858) 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 -- 172.21.15.46:0/1862761354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 msgr2=0x7ffb5819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 -- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 msgr2=0x7ffb58199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.162+0000 7ffb57400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb57400700 1 -- 172.21.15.46:0/1862761354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480095f0 con 0x7ffb58073390 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb57e00700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb56a00700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb57400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4801d070 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb48022470 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48017650 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5819ea00 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.163+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5819ec80 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.164+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb581088a0 con 0x7ffb58073390 2024-05-08T23:03:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.167+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7ffb480225e0 con 0x7ffb58073390 2024-05-08T23:03:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.168+0000 7ffb4fe00700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 0x7ffb3c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.168+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ffb480ceb60 con 0x7ffb58073390 2024-05-08T23:03:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.168+0000 7ffb56a00700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 0x7ffb3c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.169+0000 7ffb56a00700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 0x7ffb3c0a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb5819aa80 tx=0x7ffb44009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.170+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffb480979e0 con 0x7ffb58073390 2024-05-08T23:03:59.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.387+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb5819a8d0 con 0x7ffb58073390 2024-05-08T23:03:59.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.388+0000 7ffb4fe00700 1 -- 172.21.15.46:0/1862761354 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb4809b000 con 0x7ffb58073390 2024-05-08T23:03:59.388 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:03:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 msgr2=0x7ffb3c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 0x7ffb3c0a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb5819aa80 tx=0x7ffb44009260 comp rx=0 tx=0).stop 2024-05-08T23:03:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 msgr2=0x7ffb58199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48005740 comp rx=0 tx=0).stop 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 shutdown_connections 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb5810d690 0x7ffb5819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb58073390 0x7ffb58199580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb3c0a13b0 0x7ffb3c0a3870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 --2- 172.21.15.46:0/1862761354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb58073cb0 0x7ffb58199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 >> 172.21.15.46:0/1862761354 conn(0x7ffb580fc8e0 msgr2=0x7ffb580fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.390+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 shutdown_connections 2024-05-08T23:03:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.391+0000 7ffb5e400700 1 -- 172.21.15.46:0/1862761354 wait complete. 2024-05-08T23:03:59.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 -- 172.21.15.46:0/359331321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d8101080 msgr2=0x7f69d8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 --2- 172.21.15.46:0/359331321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d8101080 0x7f69d8101460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009990 tx=0x7f69cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 -- 172.21.15.46:0/359331321 shutdown_connections 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 --2- 172.21.15.46:0/359331321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d81061a0 0x7f69d810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 --2- 172.21.15.46:0/359331321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d8101080 0x7f69d8101460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 --2- 172.21.15.46:0/359331321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8101a30 0x7f69d8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 -- 172.21.15.46:0/359331321 >> 172.21.15.46:0/359331321 conn(0x7f69d80fc8e0 msgr2=0x7f69d80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.615+0000 7f69e0800700 1 -- 172.21.15.46:0/359331321 shutdown_connections 2024-05-08T23:03:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.616+0000 7f69e0800700 1 -- 172.21.15.46:0/359331321 wait complete. 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.616+0000 7f69e0800700 1 Processor -- start 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 -- start start 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 0x7f69d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a5a90 con 0x7f69d8073d20 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a5c00 con 0x7f69d81061a0 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69e0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a5dc0 con 0x7f69d8101080 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69de800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69dd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69de800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33870/0 (socket says 172.21.15.46:33870) 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69dde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 0x7f69d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69de800700 1 -- 172.21.15.46:0/1068157 learned_addr learned my addr 172.21.15.46:0/1068157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.617+0000 7f69dd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57056/0 (socket says 172.21.15.46:57056) 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 -- 172.21.15.46:0/1068157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 msgr2=0x7f69d80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 0x7f69d80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 -- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 msgr2=0x7f69d81a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 -- 172.21.15.46:0/1068157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0095f0 con 0x7f69d81061a0 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dde00700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 0x7f69d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69de800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69dd400700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e9c0 tx=0x7f69c800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800c9c0 con 0x7f69d81061a0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69c8010430 con 0x7f69d81061a0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d81a60a0 con 0x7f69d81061a0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.618+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8018680 con 0x7f69d81061a0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.619+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d81a65b0 con 0x7f69d81061a0 2024-05-08T23:03:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.620+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d81093c0 con 0x7f69d81061a0 2024-05-08T23:03:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.622+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f69c800cb20 con 0x7f69d81061a0 2024-05-08T23:03:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.623+0000 7f69d2a00700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 0x7f69c00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:03:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.624+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f69c8014070 con 0x7f69d81061a0 2024-05-08T23:03:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.624+0000 7f69dde00700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 0x7f69c00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:03:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.624+0000 7f69dde00700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 0x7f69c00a3870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b2e0 tx=0x7f69cc005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:03:59.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.625+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f69c8096500 con 0x7f69d81061a0 2024-05-08T23:03:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.845+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f69d8074780 con 0x7f69d81061a0 2024-05-08T23:03:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.846+0000 7f69d2a00700 1 -- 172.21.15.46:0/1068157 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f69c8099b20 con 0x7f69d81061a0 2024-05-08T23:03:59.846 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:03:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 msgr2=0x7f69c00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 0x7f69c00a3870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b2e0 tx=0x7f69cc005fd0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 msgr2=0x7f69d80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e9c0 tx=0x7f69c800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 shutdown_connections 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f69c00a13b0 0x7f69c00a3870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69d8073d20 0x7f69d81a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69d8101080 0x7f69d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 --2- 172.21.15.46:0/1068157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69d81061a0 0x7f69d80737e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:03:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.848+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 >> 172.21.15.46:0/1068157 conn(0x7f69d80fc8e0 msgr2=0x7f69d80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:03:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.849+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 shutdown_connections 2024-05-08T23:03:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:03:59.849+0000 7f69e0800700 1 -- 172.21.15.46:0/1068157 wait complete. 2024-05-08T23:03:59.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:03:59.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:59 smithi086 ceph-mon[78610]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 20 op/s 2024-05-08T23:03:59.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3544920900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:03:59.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:03:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1862761354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:59 smithi053 ceph-mon[77587]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 20 op/s 2024-05-08T23:04:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3544920900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:03:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1862761354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:59 smithi046 ceph-mon[36898]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 20 op/s 2024-05-08T23:04:00.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3544920900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:03:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1862761354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 -- 172.21.15.46:0/70647579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c106190 msgr2=0x7f042c10dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 --2- 172.21.15.46:0/70647579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c106190 0x7f042c10dd10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f042400b0a0 tx=0x7f042400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 -- 172.21.15.46:0/70647579 shutdown_connections 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 --2- 172.21.15.46:0/70647579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c106190 0x7f042c10dd10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 --2- 172.21.15.46:0/70647579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c101070 0x7f042c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 --2- 172.21.15.46:0/70647579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c101a20 0x7f042c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.066+0000 7f0433200700 1 -- 172.21.15.46:0/70647579 >> 172.21.15.46:0/70647579 conn(0x7f042c0fc930 msgr2=0x7f042c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.067+0000 7f0433200700 1 -- 172.21.15.46:0/70647579 shutdown_connections 2024-05-08T23:04:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.067+0000 7f0433200700 1 -- 172.21.15.46:0/70647579 wait complete. 2024-05-08T23:04:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 Processor -- start 2024-05-08T23:04:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 -- start start 2024-05-08T23:04:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c106190 0x7f042c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c073d20 0x7f042c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1a5c40 con 0x7f042c101070 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1a5d80 con 0x7f042c106190 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0433200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1a5f40 con 0x7f042c073d20 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f042be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f042be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33890/0 (socket says 172.21.15.46:33890) 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f042be00700 1 -- 172.21.15.46:0/1867719957 learned_addr learned my addr 172.21.15.46:0/1867719957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f042b400700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c106190 0x7f042c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.068+0000 7f0431200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c073d20 0x7f042c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 -- 172.21.15.46:0/1867719957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c073d20 msgr2=0x7f042c1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c073d20 0x7f042c1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 -- 172.21.15.46:0/1867719957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c106190 msgr2=0x7f042c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c106190 0x7f042c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 -- 172.21.15.46:0/1867719957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418009520 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f042be00700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e9c0 tx=0x7f041c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c0097b0 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041c00ee60 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0424009d00 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c0054f0 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.069+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c1a6550 con 0x7f042c101070 2024-05-08T23:04:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.070+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c109460 con 0x7f042c101070 2024-05-08T23:04:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.072+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f041c004750 con 0x7f042c101070 2024-05-08T23:04:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.073+0000 7f0428c00700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 0x7f04100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.074+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f041c014070 con 0x7f042c101070 2024-05-08T23:04:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.074+0000 7f042b400700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 0x7f04100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.074+0000 7f042b400700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 0x7f04100a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f042c074980 tx=0x7f0418009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.078+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f041c099e80 con 0x7f042c101070 2024-05-08T23:04:00.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.347+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f042c0747f0 con 0x7f042c101070 2024-05-08T23:04:00.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.348+0000 7f0428c00700 1 -- 172.21.15.46:0/1867719957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f041c099a10 con 0x7f042c101070 2024-05-08T23:04:00.348 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:04:00.348 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:03:50.080154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:04:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 msgr2=0x7f04100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 0x7f04100a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f042c074980 tx=0x7f0418009260 comp rx=0 tx=0).stop 2024-05-08T23:04:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 msgr2=0x7f042c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e9c0 tx=0x7f041c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 shutdown_connections 2024-05-08T23:04:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042c073d20 0x7f042c1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f04100a13b0 0x7f04100a3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f042c101070 0x7f042c0732a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 --2- 172.21.15.46:0/1867719957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042c106190 0x7f042c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 >> 172.21.15.46:0/1867719957 conn(0x7f042c0fc930 msgr2=0x7f042c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 shutdown_connections 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.351+0000 7f0433200700 1 -- 172.21.15.46:0/1867719957 wait complete. 2024-05-08T23:04:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-05-08T23:04:00.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 -- 172.21.15.46:0/3546128176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68101070 msgr2=0x7fcb68101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 --2- 172.21.15.46:0/3546128176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68101070 0x7fcb68101450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009990 tx=0x7fcb5c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 -- 172.21.15.46:0/3546128176 shutdown_connections 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 --2- 172.21.15.46:0/3546128176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 0x7fcb6810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 --2- 172.21.15.46:0/3546128176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68101070 0x7fcb68101450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 --2- 172.21.15.46:0/3546128176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101a20 0x7fcb68105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 -- 172.21.15.46:0/3546128176 >> 172.21.15.46:0/3546128176 conn(0x7fcb680fc930 msgr2=0x7fcb680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 -- 172.21.15.46:0/3546128176 shutdown_connections 2024-05-08T23:04:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.569+0000 7fcb70800700 1 -- 172.21.15.46:0/3546128176 wait complete. 2024-05-08T23:04:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.570+0000 7fcb70800700 1 Processor -- start 2024-05-08T23:04:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 -- start start 2024-05-08T23:04:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 0x7fcb680732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 0x7fcb680737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 0x7fcb681a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681a5c10 con 0x7fcb68073d20 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681a5d80 con 0x7fcb68101070 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb70800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681a5f40 con 0x7fcb68106190 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb6e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 0x7fcb681a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb6de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 0x7fcb680732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb6e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 0x7fcb681a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33910/0 (socket says 172.21.15.46:33910) 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb6e800700 1 -- 172.21.15.46:0/2959733429 learned_addr learned my addr 172.21.15.46:0/2959733429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.571+0000 7fcb6d400700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 0x7fcb680737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 -- 172.21.15.46:0/2959733429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 msgr2=0x7fcb680737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 0x7fcb680737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 -- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 msgr2=0x7fcb681a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 0x7fcb681a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 -- 172.21.15.46:0/2959733429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009520 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb6de00700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 0x7fcb680732a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c01d070 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5c021410 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c017650 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.572+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c0095f0 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.573+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb681a6400 con 0x7fcb68101070 2024-05-08T23:04:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.574+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb68109400 con 0x7fcb68101070 2024-05-08T23:04:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.576+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fcb5c021580 con 0x7fcb68101070 2024-05-08T23:04:00.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.577+0000 7fcb62a00700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 0x7fcb500a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:00.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.577+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fcb5c0cec20 con 0x7fcb68101070 2024-05-08T23:04:00.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.577+0000 7fcb6d400700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 0x7fcb500a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:00.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.578+0000 7fcb6d400700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 0x7fcb500a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb680749d0 tx=0x7fcb58009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:00.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.579+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb5c097aa0 con 0x7fcb68101070 2024-05-08T23:04:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.806+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb68074780 con 0x7fcb68101070 2024-05-08T23:04:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.809+0000 7fcb62a00700 1 -- 172.21.15.46:0/2959733429 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+25062 (secure 0 0 0) 0x7fcb5c09b0c0 con 0x7fcb68101070 2024-05-08T23:04:00.809 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:04:00.809 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:03:48.069712+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 msgr2=0x7fcb500a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 0x7fcb500a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb680749d0 tx=0x7fcb58009260 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 msgr2=0x7fcb680732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 0x7fcb680732a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000c00 tx=0x7fcb5c004730 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 shutdown_connections 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb500a1430 0x7fcb500a38f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb68073d20 0x7fcb681a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb68101070 0x7fcb680732a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 --2- 172.21.15.46:0/2959733429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb68106190 0x7fcb680737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.811+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 >> 172.21.15.46:0/2959733429 conn(0x7fcb680fc930 msgr2=0x7fcb680fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.812+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 shutdown_connections 2024-05-08T23:04:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:00.812+0000 7fcb70800700 1 -- 172.21.15.46:0/2959733429 wait complete. 2024-05-08T23:04:00.831 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T23:04:00.831 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:04:00.842 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:04:00.842 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:04:00.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T23:04:00.843 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T23:04:00.843 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:04:00.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:04:00.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:00 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1068157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:04:00.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:00 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1867719957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:00 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1068157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:04:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:00 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1867719957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:01.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:00 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1068157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:04:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:00 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1867719957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 -- 172.21.15.46:0/1617171886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da4101090 msgr2=0x7f8da4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 --2- 172.21.15.46:0/1617171886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da4101090 0x7f8da4101470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009990 tx=0x7f8d98009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 -- 172.21.15.46:0/1617171886 shutdown_connections 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 --2- 172.21.15.46:0/1617171886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da41061b0 0x7f8da410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 --2- 172.21.15.46:0/1617171886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da4101090 0x7f8da4101470 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 --2- 172.21.15.46:0/1617171886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101a40 0x7f8da4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 -- 172.21.15.46:0/1617171886 >> 172.21.15.46:0/1617171886 conn(0x7f8da40fc970 msgr2=0x7f8da40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 -- 172.21.15.46:0/1617171886 shutdown_connections 2024-05-08T23:04:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.045+0000 7f8dab600700 1 -- 172.21.15.46:0/1617171886 wait complete. 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.046+0000 7f8dab600700 1 Processor -- start 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.046+0000 7f8dab600700 1 -- start start 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da41061b0 0x7f8da40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da4073d20 0x7f8da41a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41a5b50 con 0x7f8da41061b0 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41a5cc0 con 0x7f8da4073d20 2024-05-08T23:04:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8dab600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41a5e80 con 0x7f8da4101090 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da41061b0 0x7f8da40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35080/0 (socket says 172.21.15.46:35080) 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 -- 172.21.15.46:0/3552197801 learned_addr learned my addr 172.21.15.46:0/3552197801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da9600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da4073d20 0x7f8da41a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 -- 172.21.15.46:0/3552197801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da4073d20 msgr2=0x7f8da41a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da4073d20 0x7f8da41a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 -- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da41061b0 msgr2=0x7f8da40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da41061b0 0x7f8da40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.047+0000 7f8da8c00700 1 -- 172.21.15.46:0/3552197801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90009520 con 0x7f8da4101090 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8da8c00700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005910 tx=0x7f8d98004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9801d070 con 0x7f8da4101090 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d98022470 con 0x7f8da4101090 2024-05-08T23:04:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98017650 con 0x7f8da4101090 2024-05-08T23:04:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d980095f0 con 0x7f8da4101090 2024-05-08T23:04:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.048+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da41a6340 con 0x7f8da4101090 2024-05-08T23:04:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.049+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da41093d0 con 0x7f8da4101090 2024-05-08T23:04:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1830328674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c78101fc0 msgr2=0x7f4c781023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1830328674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c78101fc0 0x7f4c781023a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009990 tx=0x7f4c68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1830328674 shutdown_connections 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1830328674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 0x7f4c7810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1830328674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c78101fc0 0x7f4c781023a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1830328674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c781028e0 0x7f4c7810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1830328674 >> 172.21.15.46:0/1830328674 conn(0x7f4c780fb7e0 msgr2=0x7f4c780fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1830328674 shutdown_connections 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.051+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1830328674 wait complete. 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.052+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f8d980225e0 con 0x7f8da4101090 2024-05-08T23:04:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f4c7ec00700 1 Processor -- start 2024-05-08T23:04:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f8da1600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 0x7f8d8c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f4c7ec00700 1 -- start start 2024-05-08T23:04:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8d980ce560 con 0x7f8da4101090 2024-05-08T23:04:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f8da3e00700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 0x7f8d8c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.053+0000 7f4c7ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c781028e0 0x7f4c78100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 0x7f4c78100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a56d0 con 0x7f4c781028e0 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a5840 con 0x7f4c7810b3e0 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a5a00 con 0x7f4c78101fc0 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f8da3e00700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 0x7f8d8c0a3860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8da40749d0 tx=0x7f8d90009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35094/0 (socket says 172.21.15.46:35094) 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 -- 172.21.15.46:0/1899502368 learned_addr learned my addr 172.21.15.46:0/1899502368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77400700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c781028e0 0x7f4c78100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c7cc00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 0x7f4c78100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 -- 172.21.15.46:0/1899502368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 msgr2=0x7f4c78100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 0x7f4c78100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 -- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c781028e0 msgr2=0x7f4c78100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c781028e0 0x7f4c78100150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.054+0000 7f4c77e00700 1 -- 172.21.15.46:0/1899502368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009520 con 0x7f4c78101fc0 2024-05-08T23:04:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.055+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000c00 tx=0x7f4c68005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f4c74c00700 1 -- 172.21.15.46:0/1899502368 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801d070 con 0x7f4c78101fc0 2024-05-08T23:04:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f4c74c00700 1 -- 172.21.15.46:0/1899502368 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c68022470 con 0x7f4c78101fc0 2024-05-08T23:04:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c70009c40 con 0x7f4c78101fc0 2024-05-08T23:04:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f4c74c00700 1 -- 172.21.15.46:0/1899502368 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68017650 con 0x7f4c78101fc0 2024-05-08T23:04:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680095f0 con 0x7f4c78101fc0 2024-05-08T23:04:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.057+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d98097440 con 0x7f8da4101090 2024-05-08T23:04:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.060+0000 7f4c74c00700 1 -- 172.21.15.46:0/1899502368 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f4c680225e0 con 0x7f4c78101fc0 2024-05-08T23:04:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.061+0000 7f4c74c00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 0x7f4c5c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.061+0000 7f4c74c00700 1 -- 172.21.15.46:0/1899502368 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4c680ce520 con 0x7f4c78101fc0 2024-05-08T23:04:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 Processor -- start 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c77400700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 0x7f4c5c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 -- start start 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 0x560947d22d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947c791d0 con 0x560947d27e80 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947c76b90 con 0x560947d22910 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c92818b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947c72480 con 0x560947d287a0 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.062+0000 7f4c77400700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 0x7f4c5c0a39c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c78101340 tx=0x7f4c64009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33962/0 (socket says 172.21.15.46:33962) 2024-05-08T23:04:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 -- 172.21.15.46:0/3066563743 learned_addr learned my addr 172.21.15.46:0/3066563743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c7cc00700 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 0x560947d22d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 -- 172.21.15.46:0/3066563743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 msgr2=0x560947d223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 -- 172.21.15.46:0/3066563743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 msgr2=0x560947d22d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 0x560947d22d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 -- 172.21.15.46:0/3066563743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560947af3430 con 0x560947d27e80 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c7cc00700 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 0x560947d22d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.063+0000 7f4c77e00700 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4c68097bf0 tx=0x7f4c68005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da04eb2b89cc932 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c6d200700 1 -- 172.21.15.46:0/3066563743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6809aae0 con 0x560947d27e80 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c6d200700 1 -- 172.21.15.46:0/3066563743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c6809b880 con 0x560947d27e80 2024-05-08T23:04:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c6d200700 1 -- 172.21.15.46:0/3066563743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680a1070 con 0x560947d27e80 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 -- 172.21.15.46:0/3066563743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 msgr2=0x560947d28260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4c68097bf0 tx=0x7f4c68005840 comp rx=0 tx=0).stop 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 -- 172.21.15.46:0/3066563743 shutdown_connections 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d22910 0x560947d22d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d27e80 0x560947d28260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.064+0000 7f4c92818b80 1 --2- 172.21.15.46:0/3066563743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.065+0000 7f4c92818b80 1 -- 172.21.15.46:0/3066563743 >> 172.21.15.46:0/3066563743 conn(0x560947c80290 msgr2=0x560947d1ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.065+0000 7f4c92818b80 1 -- 172.21.15.46:0/3066563743 shutdown_connections 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.065+0000 7f4c92818b80 1 -- 172.21.15.46:0/3066563743 wait complete. 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.065+0000 7f4c92818b80 1 -- start start 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.065+0000 7f4c92818b80 1 Processor -- start 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 0x560947d2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 0x560947d2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947c0e5e0 con 0x560947d287a0 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947c0e280 con 0x560947d27e80 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947af5720 con 0x560947d22910 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c92818b80 10 client.0 _ll_register_callbacks cb 0x560947c17bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c7cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 0x560947d2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 0x560947d2bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c7cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 0x560947d2c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33966/0 (socket says 172.21.15.46:33966) 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c7cc00700 1 -- 172.21.15.46:0/1036477740 learned_addr learned my addr 172.21.15.46:0/1036477740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.066+0000 7f4c77400700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 -- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 msgr2=0x560947d2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 -- 172.21.15.46:0/1036477740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 msgr2=0x560947d2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 0x560947d2c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 -- 172.21.15.46:0/1036477740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7000e330 con 0x560947d22910 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 0x560947d2bb50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800f740 tx=0x7f4c68022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac74d9ee254e86e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c61600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6809b180 con 0x560947d22910 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560947bcb2f0 con 0x560947d22910 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4c6809d030 con 0x560947d22910 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c680a3570 con 0x560947d22910 2024-05-08T23:04:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c92818b80 10 client.25729 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.067+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4c680095f0 con 0x560947d22910 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560947bcb2f0 con 0x560947d22910 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c6809a670 con 0x560947d22910 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 20 client.25729 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d3ba80 0x560947d3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 4 client.25729 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.068+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560947cf1890 con 0x560947d3ba80 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.069+0000 7f4c77400700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d3ba80 0x560947d3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.069+0000 7f4c77400700 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d3ba80 0x560947d3de70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc32bff59eec620 server_cookie=477e48eec9d4705f in_seq=0 out_seq=0 2024-05-08T23:04:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.069+0000 7f4c61600700 10 client.25729 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c61600700 1 -- 172.21.15.46:0/1036477740 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560947cf1890 con 0x560947d3ba80 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c61600700 10 client.25729 handle_command_reply: tid=0 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 client.25729 shutdown 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 msgr2=0x560947d2bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 0x560947d2bb50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800f740 tx=0x7f4c68022af0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 shutdown_connections 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d3ba80 0x560947d3de70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d287a0 0x560947d2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d22910 0x560947d2bb50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c61600700 0 client.25729 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1036477740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 >> 172.21.15.46:0/1036477740 conn(0x560947c80290 msgr2=0x560947d1c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 shutdown_connections 2024-05-08T23:04:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.070+0000 7f4c92818b80 1 -- 172.21.15.46:0/1036477740 wait complete. 2024-05-08T23:04:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.071+0000 7f4c92818b80 20 client.25729 trim_cache size 0 max 16384 2024-05-08T23:04:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 Processor -- start 2024-05-08T23:04:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 -- start start 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d287a0 0x560947d1cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947bcb2f0 con 0x560947d22910 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947d2e650 con 0x560947d287a0 2024-05-08T23:04:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.108+0000 7f4c92818b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947d1e0f0 con 0x560947d27e80 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35118/0 (socket says 172.21.15.46:35118) 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 -- 172.21.15.46:0/4134066140 learned_addr learned my addr 172.21.15.46:0/4134066140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c7cc00700 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d287a0 0x560947d1cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 -- 172.21.15.46:0/4134066140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d287a0 msgr2=0x560947d1cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d287a0 0x560947d1cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 -- 172.21.15.46:0/4134066140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 msgr2=0x560947d2c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d2c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 -- 172.21.15.46:0/4134066140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560947a9fbe0 con 0x560947d27e80 2024-05-08T23:04:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.109+0000 7f4c77400700 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x560947d28c00 tx=0x7f4c64014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7439a39f178fb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c6dc00700 1 -- 172.21.15.46:0/4134066140 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64022030 con 0x560947d27e80 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c6dc00700 1 -- 172.21.15.46:0/4134066140 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c64023040 con 0x560947d27e80 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c6dc00700 1 -- 172.21.15.46:0/4134066140 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640038e0 con 0x560947d27e80 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 -- 172.21.15.46:0/4134066140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 msgr2=0x560947d274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x560947d28c00 tx=0x7f4c64014000 comp rx=0 tx=0).stop 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 -- 172.21.15.46:0/4134066140 shutdown_connections 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d287a0 0x560947d1cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d2c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 --2- 172.21.15.46:0/4134066140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d27e80 0x560947d274c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.110+0000 7f4c92818b80 1 -- 172.21.15.46:0/4134066140 >> 172.21.15.46:0/4134066140 conn(0x560947c80290 msgr2=0x560947d2d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.111+0000 7f4c92818b80 1 -- 172.21.15.46:0/4134066140 shutdown_connections 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.111+0000 7f4c92818b80 1 -- 172.21.15.46:0/4134066140 wait complete. 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.111+0000 7f4c92818b80 1 -- start start 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.111+0000 7f4c92818b80 1 Processor -- start 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d8c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d86550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947d86b30 con 0x560947d22910 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947d86ca0 con 0x560947d27e80 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560947d86e60 con 0x560947d287a0 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c92818b80 10 client.0 _ll_register_callbacks cb 0x560947c17bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d8c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d8c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d8c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57158/0 (socket says 172.21.15.46:57158) 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77400700 1 -- 172.21.15.46:0/1775003443 learned_addr learned my addr 172.21.15.46:0/1775003443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c7cc00700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d86550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 -- 172.21.15.46:0/1775003443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 msgr2=0x560947d86550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d86550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 -- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 msgr2=0x560947d8c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d8c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.112+0000 7f4c77e00700 1 -- 172.21.15.46:0/1775003443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c64017070 con 0x560947d22910 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c77e00700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d8c0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c6809b110 tx=0x7f4c680cb820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b844c592785d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c61600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680a3570 con 0x560947d22910 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4c6809d040 con 0x560947d22910 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c6809b880 con 0x560947d22910 2024-05-08T23:04:01.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c680a12d0 con 0x560947d22910 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c92818b80 10 client.15723 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560947c385e0 con 0x560947d22910 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.113+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c6809b880 con 0x560947d22910 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c680b92c0 con 0x560947d22910 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 20 client.15723 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d93b00 0x560947d95ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 4 client.15723 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560947c6af40 con 0x560947d93b00 2024-05-08T23:04:01.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c77400700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d93b00 0x560947d95ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c77400700 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d93b00 0x560947d95ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=883d0b391e65c0c4 server_cookie=52645135a68f3513 in_seq=0 out_seq=0 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.114+0000 7f4c61600700 10 client.15723 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.116+0000 7f4c61600700 1 -- 172.21.15.46:0/1775003443 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560947c6af40 con 0x560947d93b00 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.116+0000 7f4c61600700 10 client.15723 handle_command_reply: tid=0 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.116+0000 7f4c92818b80 1 client.15723 shutdown 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.116+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 msgr2=0x560947d8c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.116+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d8c0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c6809b110 tx=0x7f4c680cb820 comp rx=0 tx=0).stop 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 shutdown_connections 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560947d287a0 0x560947d86550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560947d93b00 0x560947d95ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560947d22910 0x560947d8c0a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 --2- 172.21.15.46:0/1775003443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560947d27e80 0x560947d8c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 >> 172.21.15.46:0/1775003443 conn(0x560947c80290 msgr2=0x560947d28c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c61600700 0 client.15723 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 shutdown_connections 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 1 -- 172.21.15.46:0/1775003443 wait complete. 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.117+0000 7f4c92818b80 20 client.15723 trim_cache size 0 max 16384 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "1a5c2e5f-54de-4303-b368-85ccd813bd8a", 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:04:01.120 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 msgr2=0x7f4c5c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 0x7f4c5c0a39c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c78101340 tx=0x7f4c64009260 comp rx=0 tx=0).stop 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 msgr2=0x7f4c780ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000c00 tx=0x7f4c68005740 comp rx=0 tx=0).stop 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 shutdown_connections 2024-05-08T23:04:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7810b3e0 0x7f4c78100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c78101fc0 0x7f4c780ffc10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4c5c0a1500 0x7f4c5c0a39c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 --2- 172.21.15.46:0/1899502368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c781028e0 0x7f4c78100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 >> 172.21.15.46:0/1899502368 conn(0x7f4c780fb7e0 msgr2=0x7f4c780fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 shutdown_connections 2024-05-08T23:04:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.118+0000 7f4c7ec00700 1 -- 172.21.15.46:0/1899502368 wait complete. 2024-05-08T23:04:01.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:04:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.340+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8da40747f0 con 0x7f8d8c0a13a0 2024-05-08T23:04:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 -- 172.21.15.46:0/1921607672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c101a30 msgr2=0x7fb64c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 --2- 172.21.15.46:0/1921607672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c101a30 0x7fb64c105a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098f0 tx=0x7fb638009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 -- 172.21.15.46:0/1921607672 shutdown_connections 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 --2- 172.21.15.46:0/1921607672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c1061a0 0x7fb64c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 --2- 172.21.15.46:0/1921607672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c101080 0x7fb64c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 --2- 172.21.15.46:0/1921607672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c101a30 0x7fb64c105a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 -- 172.21.15.46:0/1921607672 >> 172.21.15.46:0/1921607672 conn(0x7fb64c0fc8e0 msgr2=0x7fb64c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 -- 172.21.15.46:0/1921607672 shutdown_connections 2024-05-08T23:04:01.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.344+0000 7fb652000700 1 -- 172.21.15.46:0/1921607672 wait complete. 2024-05-08T23:04:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.345+0000 7fb652000700 1 Processor -- start 2024-05-08T23:04:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 -- start start 2024-05-08T23:04:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c1061a0 0x7fb64c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 0x7fb64c19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c19e8b0 con 0x7fb64c101080 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c19ea20 con 0x7fb64c1061a0 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb652000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c19ebe0 con 0x7fb64c19de30 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb64b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb64be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 0x7fb64c19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb64b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33994/0 (socket says 172.21.15.46:33994) 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb64b400700 1 -- 172.21.15.46:0/2911947281 learned_addr learned my addr 172.21.15.46:0/2911947281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.346+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c1061a0 0x7fb64c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 -- 172.21.15.46:0/2911947281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c1061a0 msgr2=0x7fb64c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c1061a0 0x7fb64c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 -- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 msgr2=0x7fb64c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 -- 172.21.15.46:0/2911947281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0095f0 con 0x7fb64c19de30 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64b400700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb64be00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 0x7fb64c19e2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb64400efc0 tx=0x7fb64400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb643e00700 1 -- 172.21.15.46:0/2911947281 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644018070 con 0x7fb64c19de30 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb643e00700 1 -- 172.21.15.46:0/2911947281 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64400f040 con 0x7fb64c19de30 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb638009520 con 0x7fb64c19de30 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.347+0000 7fb643e00700 1 -- 172.21.15.46:0/2911947281 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400cd40 con 0x7fb64c19de30 2024-05-08T23:04:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.348+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c1a6430 con 0x7fb64c19de30 2024-05-08T23:04:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.351+0000 7fb643e00700 1 -- 172.21.15.46:0/2911947281 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fb644003a00 con 0x7fb64c19de30 2024-05-08T23:04:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 Processor -- start 2024-05-08T23:04:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 -- start start 2024-05-08T23:04:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b281c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b276d150 con 0x5639b2816890 2024-05-08T23:04:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b276ab10 con 0x5639b281c720 2024-05-08T23:04:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.353+0000 7fb665a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2766400 con 0x5639b281be00 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb643e00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 0x7fb6300a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57172/0 (socket says 172.21.15.46:57172) 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 -- 172.21.15.46:0/2221479895 learned_addr learned my addr 172.21.15.46:0/2221479895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b281c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 -- 172.21.15.46:0/2221479895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 msgr2=0x5639b281c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b281c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 -- 172.21.15.46:0/2221479895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 msgr2=0x5639b2816cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64b400700 1 -- 172.21.15.46:0/2221479895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b25e73b0 con 0x5639b281c720 2024-05-08T23:04:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.354+0000 7fb64be00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 0x7fb6300a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.355+0000 7fb64b400700 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0118d0 tx=0x7fb63c011be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefc38eb660c5b61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.355+0000 7fb643e00700 1 -- 172.21.15.46:0/2911947281 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb6440cceb0 con 0x7fb64c19de30 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.355+0000 7fb637e00700 1 -- 172.21.15.46:0/2221479895 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c011250 con 0x5639b281c720 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.355+0000 7fb637e00700 1 -- 172.21.15.46:0/2221479895 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c004cf0 con 0x5639b281c720 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.355+0000 7fb637e00700 1 -- 172.21.15.46:0/2221479895 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00f8a0 con 0x5639b281c720 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 -- 172.21.15.46:0/2221479895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 msgr2=0x5639b2816350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0118d0 tx=0x7fb63c011be0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb64be00700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 0x7fb6300a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ca0d0 tx=0x7fb6440c3ea0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 -- 172.21.15.46:0/2221479895 shutdown_connections 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b281c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2221479895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b281c720 0x5639b2816350 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0118d0 tx=0x7fb63c011be0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.356+0000 7fb665a81b80 1 -- 172.21.15.46:0/2221479895 >> 172.21.15.46:0/2221479895 conn(0x5639b2774210 msgr2=0x5639b2810df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.357+0000 7fb665a81b80 1 -- 172.21.15.46:0/2221479895 shutdown_connections 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.357+0000 7fb665a81b80 1 -- 172.21.15.46:0/2221479895 wait complete. 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.357+0000 7fb665a81b80 1 -- start start 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.357+0000 7fb665a81b80 1 Processor -- start 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b282c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2702560 con 0x5639b281be00 2024-05-08T23:04:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2702200 con 0x5639b2820820 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b25e96a0 con 0x5639b2816890 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb665a81b80 10 client.0 _ll_register_callbacks cb 0x5639b270bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb64b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35144/0 (socket says 172.21.15.46:35144) 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb64aa00700 1 -- 172.21.15.46:0/3316676003 learned_addr learned my addr 172.21.15.46:0/3316676003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.358+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b282c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 -- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 msgr2=0x5639b282c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b282c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 -- 172.21.15.46:0/3316676003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 msgr2=0x5639b28202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 -- 172.21.15.46:0/3316676003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c011500 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64b400700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098c0 tx=0x7fb638009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477ad48084407e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.359+0000 7fb634c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380050d0 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb638005230 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639b26bf270 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6380053f0 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb665a81b80 10 client.25747 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5639b2593b60 con 0x5639b2816890 2024-05-08T23:04:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.360+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639b26bf270 con 0x5639b2816890 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb638015020 con 0x5639b2816890 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 20 client.25747 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2830830 0x5639b2832c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 4 client.25747 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639b27e5810 con 0x5639b2830830 2024-05-08T23:04:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.361+0000 7fb64b400700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2830830 0x5639b2832c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.362+0000 7fb64b400700 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2830830 0x5639b2832c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e748c5b11964d0 server_cookie=da88199d507670ad in_seq=0 out_seq=0 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.362+0000 7fb634c00700 10 client.25747 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb634c00700 1 -- 172.21.15.46:0/3316676003 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639b27e5810 con 0x5639b2830830 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb634c00700 10 client.25747 handle_command_reply: tid=0 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 client.25747 shutdown 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 msgr2=0x5639b281fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098c0 tx=0x7fb638009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 shutdown_connections 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b282c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2816890 0x5639b281fda0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.363+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2830830 0x5639b2832c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3316676003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb634c00700 0 client.25747 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 >> 172.21.15.46:0/3316676003 conn(0x5639b2774210 msgr2=0x5639b28105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 shutdown_connections 2024-05-08T23:04:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb665a81b80 1 -- 172.21.15.46:0/3316676003 wait complete. 2024-05-08T23:04:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.364+0000 7fb665a81b80 20 client.25747 trim_cache size 0 max 16384 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.386+0000 7fb665a81b80 1 Processor -- start 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.386+0000 7fb665a81b80 1 -- start start 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b281ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b28228f0 con 0x5639b2816890 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2837e20 con 0x5639b2820820 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb665a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2822eb0 con 0x5639b281be00 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35156/0 (socket says 172.21.15.46:35156) 2024-05-08T23:04:01.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 -- 172.21.15.46:0/3428530718 learned_addr learned my addr 172.21.15.46:0/3428530718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b281ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 -- 172.21.15.46:0/3428530718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 msgr2=0x5639b281ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b281ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 -- 172.21.15.46:0/3428530718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 msgr2=0x5639b2816350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.387+0000 7fb64b400700 1 -- 172.21.15.46:0/3428530718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b272c560 con 0x5639b281be00 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.388+0000 7fb64aa00700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b281ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:04:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.388+0000 7fb64b400700 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb63c011f30 tx=0x7fb63c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe9d555c1c4056e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.388+0000 7fb641200700 1 -- 172.21.15.46:0/3428530718 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c029040 con 0x5639b281be00 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.388+0000 7fb641200700 1 -- 172.21.15.46:0/3428530718 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c02b7a0 con 0x5639b281be00 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.388+0000 7fb641200700 1 -- 172.21.15.46:0/3428530718 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0112b0 con 0x5639b281be00 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 -- 172.21.15.46:0/3428530718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 msgr2=0x5639b2810d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb63c011f30 tx=0x7fb63c009310 comp rx=0 tx=0).stop 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 -- 172.21.15.46:0/3428530718 shutdown_connections 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2820820 0x5639b281ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b2816890 0x5639b2816350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 --2- 172.21.15.46:0/3428530718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b281be00 0x5639b2810d20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 -- 172.21.15.46:0/3428530718 >> 172.21.15.46:0/3428530718 conn(0x5639b2774210 msgr2=0x5639b281e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 -- 172.21.15.46:0/3428530718 shutdown_connections 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.389+0000 7fb665a81b80 1 -- 172.21.15.46:0/3428530718 wait complete. 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.390+0000 7fb665a81b80 1 -- start start 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.390+0000 7fb665a81b80 1 Processor -- start 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2816890 0x5639b2825470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2880270 con 0x5639b281be00 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b28803b0 con 0x5639b2816890 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b2880570 con 0x5639b2820820 2024-05-08T23:04:01.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb665a81b80 10 client.0 _ll_register_callbacks cb 0x5639b270bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:01.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb64b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35160/0 (socket says 172.21.15.46:35160) 2024-05-08T23:04:01.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb64aa00700 1 -- 172.21.15.46:0/2735690983 learned_addr learned my addr 172.21.15.46:0/2735690983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.391+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2816890 0x5639b2825470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 -- 172.21.15.46:0/2735690983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 msgr2=0x5639b287fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 -- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2816890 msgr2=0x5639b2825470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2816890 0x5639b2825470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 -- 172.21.15.46:0/2735690983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b25e73b0 con 0x5639b281be00 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb64b400700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28259b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5639b2811d40 tx=0x7fb63c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb0af2c08e554b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb634c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c029070 con 0x5639b281be00 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb63c011250 con 0x5639b281be00 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.392+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb63c004cf0 con 0x5639b281be00 2024-05-08T23:04:01.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb63c02b2e0 con 0x5639b281be00 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 10 client.15741 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5639b25ebcd0 con 0x5639b281be00 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c03f080 con 0x5639b281be00 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb63c011900 con 0x5639b281be00 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 20 client.15741 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2887940 0x5639b2889d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 4 client.15741 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.393+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639b275eec0 con 0x5639b2887940 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2887940 0x5639b2889d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb64aa00700 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2887940 0x5639b2889d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0201d4db849d257 server_cookie=16e011a6c5cde864 in_seq=0 out_seq=0 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb634c00700 10 client.15741 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb634c00700 1 -- 172.21.15.46:0/2735690983 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5639b275eec0 con 0x5639b2887940 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb634c00700 10 client.15741 handle_command_reply: tid=0 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.394+0000 7fb665a81b80 1 client.15741 shutdown 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 msgr2=0x5639b28259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28259b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5639b2811d40 tx=0x7fb63c004ef0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 shutdown_connections 2024-05-08T23:04:01.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639b2820820 0x5639b287fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639b2816890 0x5639b2825470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5639b2887940 0x5639b2889d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 --2- 172.21.15.46:0/2735690983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639b281be00 0x5639b28259b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 >> 172.21.15.46:0/2735690983 conn(0x5639b2774210 msgr2=0x5639b281e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb634c00700 0 client.15741 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 shutdown_connections 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 1 -- 172.21.15.46:0/2735690983 wait complete. 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.395+0000 7fb665a81b80 20 client.15741 trim_cache size 0 max 16384 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:01.398 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: "1a5c2e5f-54de-4303-b368-85ccd813bd8a": { 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "1a5c2e5f-54de-4303-b368-85ccd813bd8a", 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:04:01.399 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 msgr2=0x7fb6300a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 0x7fb6300a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ca0d0 tx=0x7fb6440c3ea0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 msgr2=0x7fb64c19e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 0x7fb64c19e2b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb64400efc0 tx=0x7fb64400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 shutdown_connections 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb64c19de30 0x7fb64c19e2b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6300a13b0 0x7fb6300a3870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb64c101080 0x7fb64c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 --2- 172.21.15.46:0/2911947281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb64c1061a0 0x7fb64c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.396+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 >> 172.21.15.46:0/2911947281 conn(0x7fb64c0fc8e0 msgr2=0x7fb64c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.397+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 shutdown_connections 2024-05-08T23:04:01.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.397+0000 7fb652000700 1 -- 172.21.15.46:0/2911947281 wait complete. 2024-05-08T23:04:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.417+0000 7f8da1600700 1 -- 172.21.15.46:0/3552197801 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8da40747f0 con 0x7f8d8c0a13a0 2024-05-08T23:04:01.417 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path / 2024-05-08T23:04:01.418 INFO:tasks.cephfs.filesystem:scrub status for tag:1a5c2e5f-54de-4303-b368-85ccd813bd8a - {'path': '/', 'tag': '1a5c2e5f-54de-4303-b368-85ccd813bd8a', 'options': 'recursive,force'} 2024-05-08T23:04:01.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 msgr2=0x7f8d8c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 0x7f8d8c0a3860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8da40749d0 tx=0x7f8d90009260 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 msgr2=0x7f8da40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005910 tx=0x7f8d98004730 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 shutdown_connections 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da4073d20 0x7f8da41a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8da4101090 0x7f8da40732a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d8c0a13a0 0x7f8d8c0a3860 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 --2- 172.21.15.46:0/3552197801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da41061b0 0x7f8da40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 >> 172.21.15.46:0/3552197801 conn(0x7f8da40fc970 msgr2=0x7f8da40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 shutdown_connections 2024-05-08T23:04:01.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.419+0000 7f8dab600700 1 -- 172.21.15.46:0/3552197801 wait complete. 2024-05-08T23:04:01.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 -- 172.21.15.46:0/635276027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c101a40 msgr2=0x7fa94c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 --2- 172.21.15.46:0/635276027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c101a40 0x7fa94c105a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa938009990 tx=0x7fa938009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 -- 172.21.15.46:0/635276027 shutdown_connections 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 --2- 172.21.15.46:0/635276027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c1061b0 0x7fa94c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 --2- 172.21.15.46:0/635276027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c101090 0x7fa94c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 --2- 172.21.15.46:0/635276027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c101a40 0x7fa94c105a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 -- 172.21.15.46:0/635276027 >> 172.21.15.46:0/635276027 conn(0x7fa94c0fc970 msgr2=0x7fa94c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 -- 172.21.15.46:0/635276027 shutdown_connections 2024-05-08T23:04:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.677+0000 7fa952a00700 1 -- 172.21.15.46:0/635276027 wait complete. 2024-05-08T23:04:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.678+0000 7fa952a00700 1 Processor -- start 2024-05-08T23:04:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.678+0000 7fa952a00700 1 -- start start 2024-05-08T23:04:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c101090 0x7fa94c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c1061b0 0x7fa94c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19c670 con 0x7fa94c19bbf0 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19c7e0 con 0x7fa94c1061b0 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa952a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19c9a0 con 0x7fa94c101090 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa950a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa94b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c1061b0 0x7fa94c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa950a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34046/0 (socket says 172.21.15.46:34046) 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa950a00700 1 -- 172.21.15.46:0/3374333008 learned_addr learned my addr 172.21.15.46:0/3374333008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.679+0000 7fa94be00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c101090 0x7fa94c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 -- 172.21.15.46:0/3374333008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c101090 msgr2=0x7fa94c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c101090 0x7fa94c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 -- 172.21.15.46:0/3374333008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c1061b0 msgr2=0x7fa94c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c1061b0 0x7fa94c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 -- 172.21.15.46:0/3374333008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380095f0 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa950a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa944007ee0 tx=0x7fa94400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017070 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c073230 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa94400f040 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c073790 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.680+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944005510 con 0x7fa94c19bbf0 2024-05-08T23:04:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.681+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c109480 con 0x7fa94c19bbf0 2024-05-08T23:04:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.683+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fa94400db20 con 0x7fa94c19bbf0 2024-05-08T23:04:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.684+0000 7fa948c00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 0x7fa9300a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.685+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa9440cd0e0 con 0x7fa94c19bbf0 2024-05-08T23:04:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.685+0000 7fa94be00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 0x7fa9300a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.685+0000 7fa94be00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 0x7fa9300a3860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c60 tx=0x7fa93c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:01.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.687+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa944095ed0 con 0x7fa94c19bbf0 2024-05-08T23:04:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:01.942+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fa94c101a40 con 0x7fa9300a13a0 2024-05-08T23:04:01.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:01 smithi086 ceph-mon[78610]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.0 KiB/s wr, 21 op/s 2024-05-08T23:04:01.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:01 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2959733429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:04:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.009+0000 7fa948c00700 1 -- 172.21.15.46:0/3374333008 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa94c101a40 con 0x7fa9300a13a0 2024-05-08T23:04:02.010 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path / 2024-05-08T23:04:02.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:01 smithi053 ceph-mon[77587]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.0 KiB/s wr, 21 op/s 2024-05-08T23:04:02.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:01 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2959733429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:04:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 msgr2=0x7fa9300a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 0x7fa9300a3860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c60 tx=0x7fa93c009260 comp rx=0 tx=0).stop 2024-05-08T23:04:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 msgr2=0x7fa94c19c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa944007ee0 tx=0x7fa94400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 shutdown_connections 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9300a13a0 0x7fa9300a3860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94c19bbf0 0x7fa94c19c070 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa94c101090 0x7fa94c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 --2- 172.21.15.46:0/3374333008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c1061b0 0x7fa94c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 >> 172.21.15.46:0/3374333008 conn(0x7fa94c0fc970 msgr2=0x7fa94c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 shutdown_connections 2024-05-08T23:04:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.012+0000 7fa952a00700 1 -- 172.21.15.46:0/3374333008 wait complete. 2024-05-08T23:04:02.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:01 smithi046 ceph-mon[36898]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.0 KiB/s wr, 21 op/s 2024-05-08T23:04:02.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:01 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2959733429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:04:02.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T23:04:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 -- 172.21.15.46:0/1694507516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45601061b0 msgr2=0x7f456010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 --2- 172.21.15.46:0/1694507516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45601061b0 0x7f456010dd30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f455800b0a0 tx=0x7f455800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 -- 172.21.15.46:0/1694507516 shutdown_connections 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 --2- 172.21.15.46:0/1694507516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45601061b0 0x7f456010dd30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 --2- 172.21.15.46:0/1694507516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4560101090 0x7f4560101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 --2- 172.21.15.46:0/1694507516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101a40 0x7f4560105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 -- 172.21.15.46:0/1694507516 >> 172.21.15.46:0/1694507516 conn(0x7f45600fc970 msgr2=0x7f45600fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 -- 172.21.15.46:0/1694507516 shutdown_connections 2024-05-08T23:04:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.227+0000 7f4566c00700 1 -- 172.21.15.46:0/1694507516 wait complete. 2024-05-08T23:04:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.228+0000 7f4566c00700 1 Processor -- start 2024-05-08T23:04:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.228+0000 7f4566c00700 1 -- start start 2024-05-08T23:04:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45601061b0 0x7f45600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 0x7f45601a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601a5bf0 con 0x7f4560073d20 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601a5d60 con 0x7f45601061b0 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4566c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601a5f20 con 0x7f4560101090 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f455fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 0x7f45601a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f455fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48630/0 (socket says 172.21.15.46:48630) 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f455fe00700 1 -- 172.21.15.46:0/547620341 learned_addr learned my addr 172.21.15.46:0/547620341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f455f400700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45601061b0 0x7f45600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 -- 172.21.15.46:0/547620341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 msgr2=0x7f45600732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 -- 172.21.15.46:0/547620341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45601061b0 msgr2=0x7f45600737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45601061b0 0x7f45600737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f4564c00700 1 -- 172.21.15.46:0/547620341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009d00 con 0x7f4560073d20 2024-05-08T23:04:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.229+0000 7f455fe00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f4564c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 0x7f45601a5620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f455800b070 tx=0x7f455800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800e040 con 0x7f4560073d20 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f455801d480 con 0x7f4560073d20 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580052a0 con 0x7f4560073d20 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45601a61a0 con 0x7f4560073d20 2024-05-08T23:04:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.230+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45601a6570 con 0x7f4560073d20 2024-05-08T23:04:02.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.231+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45601093d0 con 0x7f4560073d20 2024-05-08T23:04:02.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.233+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f455801d5f0 con 0x7f4560073d20 2024-05-08T23:04:02.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.234+0000 7f455cc00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 0x7f45440a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.234+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f45580cdff0 con 0x7f4560073d20 2024-05-08T23:04:02.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.235+0000 7f455fe00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 0x7f45440a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.235+0000 7f455fe00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 0x7f45440a38e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4550011550 tx=0x7f4550009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:02.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.238+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4558101d60 con 0x7f4560073d20 2024-05-08T23:04:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.484+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4560101a40 con 0x7f45440a1420 2024-05-08T23:04:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.493+0000 7f455cc00700 1 -- 172.21.15.46:0/547620341 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f4560101a40 con 0x7f45440a1420 2024-05-08T23:04:02.493 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:04:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.495+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 msgr2=0x7f45440a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.495+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 0x7f45440a38e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4550011550 tx=0x7f4550009260 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.495+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 msgr2=0x7f45601a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.495+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 0x7f45601a5620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f455800b070 tx=0x7f455800bcf0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 shutdown_connections 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45440a1420 0x7f45440a38e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4560073d20 0x7f45601a5620 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4560101090 0x7f45600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 --2- 172.21.15.46:0/547620341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45601061b0 0x7f45600737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 >> 172.21.15.46:0/547620341 conn(0x7f45600fc970 msgr2=0x7f45600fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 shutdown_connections 2024-05-08T23:04:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.496+0000 7f4566c00700 1 -- 172.21.15.46:0/547620341 wait complete. 2024-05-08T23:04:02.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T23:04:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 -- 172.21.15.46:0/3347197934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101a40 msgr2=0x7fe2fc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 --2- 172.21.15.46:0/3347197934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101a40 0x7fe2fc105a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009990 tx=0x7fe2e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 -- 172.21.15.46:0/3347197934 shutdown_connections 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 --2- 172.21.15.46:0/3347197934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 --2- 172.21.15.46:0/3347197934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc101090 0x7fe2fc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 --2- 172.21.15.46:0/3347197934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101a40 0x7fe2fc105a80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 -- 172.21.15.46:0/3347197934 >> 172.21.15.46:0/3347197934 conn(0x7fe2fc0fc970 msgr2=0x7fe2fc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 -- 172.21.15.46:0/3347197934 shutdown_connections 2024-05-08T23:04:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.712+0000 7fe303000700 1 -- 172.21.15.46:0/3347197934 wait complete. 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.713+0000 7fe303000700 1 Processor -- start 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 -- start start 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 0x7fe2fc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1a9f50 con 0x7fe2fc101090 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1aa0c0 con 0x7fe2fc074f10 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe303000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1aa280 con 0x7fe2fc1061b0 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe2fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe301000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe2fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 0x7fe2fc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe301000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49770/0 (socket says 172.21.15.46:49770) 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe2fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48644/0 (socket says 172.21.15.46:48644) 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.714+0000 7fe301000700 1 -- 172.21.15.46:0/603302049 learned_addr learned my addr 172.21.15.46:0/603302049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 -- 172.21.15.46:0/603302049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 msgr2=0x7fe2fc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 -- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 msgr2=0x7fe2fc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 0x7fe2fc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 -- 172.21.15.46:0/603302049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80095f0 con 0x7fe2fc074f10 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe2fbe00700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 0x7fe2fc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe2fb400700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.715+0000 7fe301000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400ef30 tx=0x7fe2f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.716+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4018070 con 0x7fe2fc074f10 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.716+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2f400f040 con 0x7fe2fc074f10 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.716+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc1aa500 con 0x7fe2fc074f10 2024-05-08T23:04:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.716+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400cd00 con 0x7fe2fc074f10 2024-05-08T23:04:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.716+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc1aa9e0 con 0x7fe2fc074f10 2024-05-08T23:04:02.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.717+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc109480 con 0x7fe2fc074f10 2024-05-08T23:04:02.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.719+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fe2f4003a00 con 0x7fe2fc074f10 2024-05-08T23:04:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.721+0000 7fe2f8c00700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 0x7fe2e00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.721+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe2f40ccdc0 con 0x7fe2fc074f10 2024-05-08T23:04:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.721+0000 7fe2fbe00700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 0x7fe2e00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.722+0000 7fe2fbe00700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 0x7fe2e00a38e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec011490 tx=0x7fe2ec009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.726+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2f4095c80 con 0x7fe2fc074f10 2024-05-08T23:04:02.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:02 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:04:02.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:02 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:04:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:02 smithi046 ceph-mon[36898]: from='client.25717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:02.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.975+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe2fc074730 con 0x7fe2e00a1420 2024-05-08T23:04:02.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.982+0000 7fe2f8c00700 1 -- 172.21.15.46:0/603302049 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe2fc074730 con 0x7fe2e00a1420 2024-05-08T23:04:02.982 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 msgr2=0x7fe2e00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 0x7fe2e00a38e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec011490 tx=0x7fe2ec009260 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 msgr2=0x7fe2fc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400ef30 tx=0x7fe2f400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 shutdown_connections 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2fc074f10 0x7fe2fc073d20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2e00a1420 0x7fe2e00a38e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2fc101090 0x7fe2fc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 --2- 172.21.15.46:0/603302049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2fc1061b0 0x7fe2fc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.984+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 >> 172.21.15.46:0/603302049 conn(0x7fe2fc0fc970 msgr2=0x7fe2fc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.985+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 shutdown_connections 2024-05-08T23:04:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:02.985+0000 7fe303000700 1 -- 172.21.15.46:0/603302049 wait complete. 2024-05-08T23:04:03.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T23:04:03.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:02 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:04:03.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:03.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:02 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:04:03.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:03.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:02 smithi053 ceph-mon[77587]: from='client.25717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:03.073 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T23:04:03.084 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T23:04:03.107 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:04:03.108 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T23:04:03.108 INFO:tasks.workunit:timeout=3h 2024-05-08T23:04:03.108 INFO:tasks.workunit:cleanup=True 2024-05-08T23:04:03.108 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 7bh/123d Inode: 1099511628283 Links: 2 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 23:02:18.703752642 +0000 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 23:02:18.703752642 +0000 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 23:03:48.104727976 +0000 2024-05-08T23:04:03.152 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 23:02:18.703752642 +0000 2024-05-08T23:04:03.152 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T23:04:03.153 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T23:04:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:02 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:04:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:02 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:04:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:02 smithi086 ceph-mon[78610]: from='client.25717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:03.272 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:04:03.316 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T23:04:04.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:03 smithi053 ceph-mon[77587]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:04.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:03 smithi053 ceph-mon[77587]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:04.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:03 smithi053 ceph-mon[77587]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:04.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:03 smithi046 ceph-mon[36898]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:03 smithi046 ceph-mon[36898]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:03 smithi046 ceph-mon[36898]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:03 smithi086 ceph-mon[78610]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:03 smithi086 ceph-mon[78610]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:03 smithi086 ceph-mon[78610]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:05.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:04 smithi053 ceph-mon[77587]: from='client.25739 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:05.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:04 smithi046 ceph-mon[36898]: from='client.25739 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:05.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:04 smithi086 ceph-mon[78610]: from='client.25739 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:04:06.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:05 smithi053 ceph-mon[77587]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:06.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:05 smithi046 ceph-mon[36898]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:05 smithi086 ceph-mon[78610]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 19 op/s 2024-05-08T23:04:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:07 smithi053 ceph-mon[77587]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-05-08T23:04:08.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:07 smithi046 ceph-mon[36898]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-05-08T23:04:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:07 smithi086 ceph-mon[78610]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-05-08T23:04:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:09 smithi053 ceph-mon[77587]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-08T23:04:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:09 smithi046 ceph-mon[36898]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-08T23:04:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:09 smithi086 ceph-mon[78610]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-08T23:04:12.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:11 smithi053 ceph-mon[77587]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-08T23:04:12.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:11 smithi046 ceph-mon[36898]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-08T23:04:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:11 smithi086 ceph-mon[78610]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-08T23:04:14.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:13 smithi053 ceph-mon[77587]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:14.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:13 smithi046 ceph-mon[36898]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:13 smithi086 ceph-mon[78610]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:16.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:15 smithi053 ceph-mon[77587]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:16.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:04:16.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:15 smithi046 ceph-mon[36898]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:04:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:15 smithi086 ceph-mon[78610]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-08T23:04:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:04:18.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:17 smithi053 ceph-mon[77587]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-08T23:04:18.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:17 smithi046 ceph-mon[36898]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-08T23:04:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:17 smithi086 ceph-mon[78610]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-08T23:04:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:19 smithi053 ceph-mon[77587]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:04:20.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:19 smithi046 ceph-mon[36898]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:04:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:19 smithi086 ceph-mon[78610]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:04:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:21 smithi053 ceph-mon[77587]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-08T23:04:22.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:21 smithi046 ceph-mon[36898]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-08T23:04:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:21 smithi086 ceph-mon[78610]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-08T23:04:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:23 smithi053 ceph-mon[77587]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:24.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:23 smithi046 ceph-mon[36898]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:23 smithi086 ceph-mon[78610]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:26.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:25 smithi046 ceph-mon[36898]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:25 smithi086 ceph-mon[78610]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:25 smithi053 ceph-mon[77587]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:28.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:27 smithi046 ceph-mon[36898]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:27 smithi086 ceph-mon[78610]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:27 smithi053 ceph-mon[77587]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:29 smithi046 ceph-mon[36898]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:29 smithi086 ceph-mon[78610]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:29 smithi053 ceph-mon[77587]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:31.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:04:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 -- 172.21.15.46:0/3136182599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 msgr2=0x7fd18c10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 --2- 172.21.15.46:0/3136182599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c10d110 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd178009990 tx=0x7fd178009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 -- 172.21.15.46:0/3136182599 shutdown_connections 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 --2- 172.21.15.46:0/3136182599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10d650 0x7fd18c10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 --2- 172.21.15.46:0/3136182599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c0733d0 0x7fd18c0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 --2- 172.21.15.46:0/3136182599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c10d110 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 -- 172.21.15.46:0/3136182599 >> 172.21.15.46:0/3136182599 conn(0x7fd18c0fc8e0 msgr2=0x7fd18c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 -- 172.21.15.46:0/3136182599 shutdown_connections 2024-05-08T23:04:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.639+0000 7fd193e00700 1 -- 172.21.15.46:0/3136182599 wait complete. 2024-05-08T23:04:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 Processor -- start 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 -- start start 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c0733d0 0x7fd18c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 0x7fd18c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c19e410 con 0x7fd18c073cf0 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c19e580 con 0x7fd18c10d650 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.640+0000 7fd193e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c19e740 con 0x7fd18c0733d0 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd190a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c0733d0 0x7fd18c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd190a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50394/0 (socket says 172.21.15.46:50394) 2024-05-08T23:04:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd190a00700 1 -- 172.21.15.46:0/1339597018 learned_addr learned my addr 172.21.15.46:0/1339597018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 0x7fd18c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 -- 172.21.15.46:0/1339597018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c0733d0 msgr2=0x7fd18c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c0733d0 0x7fd18c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 -- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 msgr2=0x7fd18c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 -- 172.21.15.46:0/1339597018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780095f0 con 0x7fd18c10d650 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd190a00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.641+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 0x7fd18c19ddd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd18400efc0 tx=0x7fd18400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.642+0000 7fd18a000700 1 -- 172.21.15.46:0/1339597018 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184018070 con 0x7fd18c10d650 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.642+0000 7fd18a000700 1 -- 172.21.15.46:0/1339597018 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18400f040 con 0x7fd18c10d650 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.642+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c19e9c0 con 0x7fd18c10d650 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.642+0000 7fd18a000700 1 -- 172.21.15.46:0/1339597018 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400cd40 con 0x7fd18c10d650 2024-05-08T23:04:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.642+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c1a6260 con 0x7fd18c10d650 2024-05-08T23:04:31.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.645+0000 7fd18a000700 1 -- 172.21.15.46:0/1339597018 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fd184003a00 con 0x7fd18c10d650 2024-05-08T23:04:31.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.646+0000 7fd1a7932b80 1 Processor -- start 2024-05-08T23:04:31.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.646+0000 7fd1a7932b80 1 -- start start 2024-05-08T23:04:31.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.646+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.646+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.646+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 0x5584949e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849493f150 con 0x5584949e8890 2024-05-08T23:04:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849493cb10 con 0x5584949ede00 2024-05-08T23:04:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494938400 con 0x5584949ee720 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd18a000700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 0x7fd1740a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd190a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd190a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43376/0 (socket says 172.21.15.46:43376) 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd190a00700 1 -- 172.21.15.46:0/2649305981 learned_addr learned my addr 172.21.15.46:0/2649305981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd190a00700 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 0x5584949e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd18a000700 1 -- 172.21.15.46:0/1339597018 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd1840ccf30 con 0x7fd18c10d650 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 0x7fd1740a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 -- 172.21.15.46:0/2649305981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 msgr2=0x5584949ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 -- 172.21.15.46:0/2649305981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 msgr2=0x5584949e8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 0x5584949e8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd191400700 1 -- 172.21.15.46:0/2649305981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584947b93b0 con 0x5584949ee720 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.647+0000 7fd190a00700 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 0x5584949e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd191400700 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949e8350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd180009cb0 tx=0x7fd180011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82e52b091b79658 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd191e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 0x7fd1740a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1840ca150 tx=0x7fd184009f10 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd17e600700 1 -- 172.21.15.46:0/2649305981 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011d20 con 0x5584949ee720 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd17e600700 1 -- 172.21.15.46:0/2649305981 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd180011e80 con 0x5584949ee720 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd17e600700 1 -- 172.21.15.46:0/2649305981 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180005280 con 0x5584949ee720 2024-05-08T23:04:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd1a7932b80 1 -- 172.21.15.46:0/2649305981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 msgr2=0x5584949e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.648+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949e8350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd180009cb0 tx=0x7fd180011440 comp rx=0 tx=0).stop 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 -- 172.21.15.46:0/2649305981 shutdown_connections 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949e8890 0x5584949e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/2649305981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949e8350 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 -- 172.21.15.46:0/2649305981 >> 172.21.15.46:0/2649305981 conn(0x558494946210 msgr2=0x5584949e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 -- 172.21.15.46:0/2649305981 shutdown_connections 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 -- 172.21.15.46:0/2649305981 wait complete. 2024-05-08T23:04:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 -- start start 2024-05-08T23:04:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.649+0000 7fd1a7932b80 1 Processor -- start 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x5584949f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584948d4560 con 0x5584949ee720 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584948d4200 con 0x5584949ede00 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584947bb6a0 con 0x5584949e8890 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd1a7932b80 10 client.0 _ll_register_callbacks cb 0x5584948ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd190a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.650+0000 7fd190a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x5584949f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46632/0 (socket says 172.21.15.46:46632) 2024-05-08T23:04:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 -- 172.21.15.46:0/4206072576 learned_addr learned my addr 172.21.15.46:0/4206072576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd191400700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 -- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 msgr2=0x5584949f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 -- 172.21.15.46:0/4206072576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 msgr2=0x5584949f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x5584949f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 -- 172.21.15.46:0/4206072576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1800098e0 con 0x5584949e8890 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd190a00700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd178005950 tx=0x7fd178009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf36851cabe322b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd172a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.651+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17801d070 con 0x5584949e8890 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584949b7810 con 0x5584949e8890 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd17802a480 con 0x5584949e8890 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd17802a640 con 0x5584949e8890 2024-05-08T23:04:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 10 client.25783 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558494765b60 con 0x5584949e8890 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584949b7810 con 0x5584949e8890 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd1780177f0 con 0x5584949e8890 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 20 client.25783 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a021b0 0x558494a045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 4 client.25783 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.652+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558494930ec0 con 0x558494a021b0 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.653+0000 7fd191400700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a021b0 0x558494a045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.653+0000 7fd172a00700 10 client.25783 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.653+0000 7fd191400700 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a021b0 0x558494a045d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1f43a6e0a8c43e server_cookie=2fd0e17535c0bf2a in_seq=0 out_seq=0 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd172a00700 1 -- 172.21.15.46:0/4206072576 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558494930ec0 con 0x558494a021b0 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd172a00700 10 client.25783 handle_command_reply: tid=0 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 client.25783 shutdown 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 msgr2=0x5584949f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd178005950 tx=0x7fd178009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 shutdown_connections 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a021b0 0x558494a045d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x5584949f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f47d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/4206072576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd172a00700 0 client.25783 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 >> 172.21.15.46:0/4206072576 conn(0x558494946210 msgr2=0x5584949e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 shutdown_connections 2024-05-08T23:04:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.654+0000 7fd1a7932b80 1 -- 172.21.15.46:0/4206072576 wait complete. 2024-05-08T23:04:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.655+0000 7fd1a7932b80 20 client.25783 trim_cache size 0 max 16384 2024-05-08T23:04:31.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 Processor -- start 2024-05-08T23:04:31.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 -- start start 2024-05-08T23:04:31.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949e8890 0x5584949e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494930ec0 con 0x5584949ede00 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584949b7810 con 0x5584949e8890 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584949ff910 con 0x5584949ee720 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd191400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd191400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50402/0 (socket says 172.21.15.46:50402) 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd191400700 1 -- 172.21.15.46:0/3548748597 learned_addr learned my addr 172.21.15.46:0/3548748597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd190a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.673+0000 7fd190a00700 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949e8890 0x5584949e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 -- 172.21.15.46:0/3548748597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 msgr2=0x5584949f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 -- 172.21.15.46:0/3548748597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949e8890 msgr2=0x5584949e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949e8890 0x5584949e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 -- 172.21.15.46:0/3548748597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584948fe560 con 0x5584949ede00 2024-05-08T23:04:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd191400700 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5584949eeb80 tx=0x7fd180009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b23fa444178c963 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd17f000700 1 -- 172.21.15.46:0/3548748597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011b20 con 0x5584949ede00 2024-05-08T23:04:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd17f000700 1 -- 172.21.15.46:0/3548748597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd180011c80 con 0x5584949ede00 2024-05-08T23:04:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.674+0000 7fd17f000700 1 -- 172.21.15.46:0/3548748597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180019650 con 0x5584949ede00 2024-05-08T23:04:31.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3548748597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 msgr2=0x5584949e2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5584949eeb80 tx=0x7fd180009260 comp rx=0 tx=0).stop 2024-05-08T23:04:31.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3548748597 shutdown_connections 2024-05-08T23:04:31.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949ee720 0x5584949f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949e8890 0x5584949e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3548748597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ede00 0x5584949e2d20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3548748597 >> 172.21.15.46:0/3548748597 conn(0x558494946210 msgr2=0x5584949fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3548748597 shutdown_connections 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3548748597 wait complete. 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.675+0000 7fd1a7932b80 1 -- start start 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 Processor -- start 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x558494a57670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494a57bb0 con 0x5584949ee720 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494a57cf0 con 0x5584949ede00 2024-05-08T23:04:31.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494a57e60 con 0x5584949e8890 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd1a7932b80 10 client.0 _ll_register_callbacks cb 0x5584948ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd191400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd190a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x558494a57670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd191400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43404/0 (socket says 172.21.15.46:43404) 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.676+0000 7fd191400700 1 -- 172.21.15.46:0/3155306119 learned_addr learned my addr 172.21.15.46:0/3155306119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd190a00700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 -- 172.21.15.46:0/3155306119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 msgr2=0x5584949f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 -- 172.21.15.46:0/3155306119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 msgr2=0x558494a57670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x558494a57670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 -- 172.21.15.46:0/3155306119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18001d040 con 0x5584949ede00 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd190a00700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x558494a57670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:31.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd191400700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd18002d040 tx=0x7fd1800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed7504758539585 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd172a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011040 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558494891270 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.677+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd1800111a0 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd180024420 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 10 client.25757 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5584947bdcd0 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558494891270 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd18003b070 con 0x5584949ede00 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 20 client.25757 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.678+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a59260 0x558494a5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd1a7932b80 4 client.25757 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558494786cf0 con 0x558494a59260 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd190a00700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a59260 0x558494a5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd190a00700 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a59260 0x558494a5b680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b984838271eb2c server_cookie=b24d623123a0393f in_seq=0 out_seq=0 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd172a00700 10 client.25757 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd172a00700 1 -- 172.21.15.46:0/3155306119 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558494786cf0 con 0x558494a59260 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd172a00700 10 client.25757 handle_command_reply: tid=0 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.679+0000 7fd1a7932b80 1 client.25757 shutdown 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 msgr2=0x5584949f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd18002d040 tx=0x7fd1800047c0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 shutdown_connections 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558494a59260 0x558494a5b680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584949ee720 0x558494a57670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd172a00700 0 client.25757 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584949e8890 0x5584949f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 --2- 172.21.15.46:0/3155306119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584949ede00 0x5584949f7a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 >> 172.21.15.46:0/3155306119 conn(0x558494946210 msgr2=0x5584949f02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 shutdown_connections 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 1 -- 172.21.15.46:0/3155306119 wait complete. 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.680+0000 7fd1a7932b80 20 client.25757 trim_cache size 0 max 16384 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:04:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:04:31.685 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:31.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 msgr2=0x7fd1740a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 0x7fd1740a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1840ca150 tx=0x7fd184009f10 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 msgr2=0x7fd18c19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 0x7fd18c19ddd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd18400efc0 tx=0x7fd18400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 shutdown_connections 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd18c10d650 0x7fd18c19ddd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c0733d0 0x7fd18c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd1740a13b0 0x7fd1740a3870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 --2- 172.21.15.46:0/1339597018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c073cf0 0x7fd18c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 >> 172.21.15.46:0/1339597018 conn(0x7fd18c0fc8e0 msgr2=0x7fd18c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 shutdown_connections 2024-05-08T23:04:31.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.681+0000 7fd193e00700 1 -- 172.21.15.46:0/1339597018 wait complete. 2024-05-08T23:04:31.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:04:31.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:04:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 -- 172.21.15.46:0/801002804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a0100e90 msgr2=0x7f32a0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 --2- 172.21.15.46:0/801002804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a0100e90 0x7f32a0101270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3290009990 tx=0x7f3290009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 -- 172.21.15.46:0/801002804 shutdown_connections 2024-05-08T23:04:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 --2- 172.21.15.46:0/801002804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0105fb0 0x7f32a010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 --2- 172.21.15.46:0/801002804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a0100e90 0x7f32a0101270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 --2- 172.21.15.46:0/801002804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0101840 0x7f32a0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 -- 172.21.15.46:0/801002804 >> 172.21.15.46:0/801002804 conn(0x7f32a00fc730 msgr2=0x7f32a00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.901+0000 7f32a5c00700 1 -- 172.21.15.46:0/801002804 shutdown_connections 2024-05-08T23:04:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.902+0000 7f32a5c00700 1 -- 172.21.15.46:0/801002804 wait complete. 2024-05-08T23:04:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.902+0000 7f32a5c00700 1 Processor -- start 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 -- start start 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 0x7f32a0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a019dc40 0x7f32a019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a019e6c0 con 0x7f32a0105fb0 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a019e830 con 0x7f32a019dc40 2024-05-08T23:04:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a019e9f0 con 0x7f32a0100e90 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f329f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f329f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45962/0 (socket says 172.21.15.46:45962) 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f329f400700 1 -- 172.21.15.46:0/562454137 learned_addr learned my addr 172.21.15.46:0/562454137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f32a5200700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a019dc40 0x7f32a019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.903+0000 7f329fe00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 0x7f32a0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 -- 172.21.15.46:0/562454137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 msgr2=0x7f32a0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 0x7f32a0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 -- 172.21.15.46:0/562454137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a019dc40 msgr2=0x7f32a019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a019dc40 0x7f32a019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 -- 172.21.15.46:0/562454137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900095f0 con 0x7f32a0105fb0 2024-05-08T23:04:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329fe00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 0x7f32a0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329f400700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e970 tx=0x7f328c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c009790 con 0x7f32a0105fb0 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f328c00eec0 con 0x7f32a0105fb0 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a01a5f50 con 0x7f32a0105fb0 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.904+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c005590 con 0x7f32a0105fb0 2024-05-08T23:04:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.905+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a01a6420 con 0x7f32a0105fb0 2024-05-08T23:04:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.906+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0102360 con 0x7f32a0105fb0 2024-05-08T23:04:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.907+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f328c00cbd0 con 0x7f32a0105fb0 2024-05-08T23:04:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.909+0000 7f329cc00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 0x7f32840a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.909+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f328c014070 con 0x7f32a0105fb0 2024-05-08T23:04:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.909+0000 7f329fe00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 0x7f32840a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.909+0000 7f329fe00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 0x7f32840a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3290006010 tx=0x7f329000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:31.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:31.913+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f328c096360 con 0x7f32a0105fb0 2024-05-08T23:04:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:31 smithi046 ceph-mon[36898]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:32.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.179+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32a01018e0 con 0x7f32a0105fb0 2024-05-08T23:04:32.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.180+0000 7f329cc00700 1 -- 172.21.15.46:0/562454137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f328c004ef0 con 0x7f32a0105fb0 2024-05-08T23:04:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:04:32.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:03:50.080154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:04:32.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 msgr2=0x7f32840a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 0x7f32840a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3290006010 tx=0x7f329000b260 comp rx=0 tx=0).stop 2024-05-08T23:04:32.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 msgr2=0x7f32a01998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e970 tx=0x7f328c00ec80 comp rx=0 tx=0).stop 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 shutdown_connections 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a019dc40 0x7f32a019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a0100e90 0x7f32a0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32840a1430 0x7f32840a38f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 --2- 172.21.15.46:0/562454137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32a0105fb0 0x7f32a01998a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 >> 172.21.15.46:0/562454137 conn(0x7f32a00fc730 msgr2=0x7f32a00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 shutdown_connections 2024-05-08T23:04:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.183+0000 7f32a5c00700 1 -- 172.21.15.46:0/562454137 wait complete. 2024-05-08T23:04:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-05-08T23:04:32.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:04:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:31 smithi086 ceph-mon[78610]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:31 smithi053 ceph-mon[77587]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 -- 172.21.15.46:0/3443087820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 msgr2=0x7f90480737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 --2- 172.21.15.46:0/3443087820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90480737b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f90380098f0 tx=0x7f9038009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 -- 172.21.15.46:0/3443087820 shutdown_connections 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 --2- 172.21.15.46:0/3443087820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904810d490 0x7f904810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 --2- 172.21.15.46:0/3443087820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90480737b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 --2- 172.21.15.46:0/3443087820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9048073cf0 0x7f904810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 -- 172.21.15.46:0/3443087820 >> 172.21.15.46:0/3443087820 conn(0x7f90480fc740 msgr2=0x7f90480feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.404+0000 7f9047200700 1 -- 172.21.15.46:0/3443087820 shutdown_connections 2024-05-08T23:04:32.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.405+0000 7f9047200700 1 -- 172.21.15.46:0/3443087820 wait complete. 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.405+0000 7f9047200700 1 Processor -- start 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 -- start start 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9048073cf0 0x7f9048199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904810d490 0x7f904819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904819e4b0 con 0x7f904810d490 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904819e620 con 0x7f9048073cf0 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9047200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904819e7e0 con 0x7f90480733d0 2024-05-08T23:04:32.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9045e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9045e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52416/0 (socket says 172.21.15.46:52416) 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9045e00700 1 -- 172.21.15.46:0/2668924332 learned_addr learned my addr 172.21.15.46:0/2668924332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9046800700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904810d490 0x7f904819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.406+0000 7f9045400700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9048073cf0 0x7f9048199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 -- 172.21.15.46:0/2668924332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9048073cf0 msgr2=0x7f9048199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9048073cf0 0x7f9048199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 -- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904810d490 msgr2=0x7f904819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904810d490 0x7f904819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 -- 172.21.15.46:0/2668924332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340095f0 con 0x7f90480733d0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9045e00700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9038005950 tx=0x7f903800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f903ea00700 1 -- 172.21.15.46:0/2668924332 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903801c070 con 0x7f90480733d0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f903ea00700 1 -- 172.21.15.46:0/2668924332 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903800bb20 con 0x7f90480733d0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f903ea00700 1 -- 172.21.15.46:0/2668924332 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380038e0 con 0x7f90480733d0 2024-05-08T23:04:32.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.407+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9040009c40 con 0x7f90480733d0 2024-05-08T23:04:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.408+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038009520 con 0x7f90480733d0 2024-05-08T23:04:32.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.411+0000 7f903ea00700 1 -- 172.21.15.46:0/2668924332 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f9038020030 con 0x7f90480733d0 2024-05-08T23:04:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 Processor -- start 2024-05-08T23:04:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f903ea00700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 0x7f902c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 -- start start 2024-05-08T23:04:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f903ea00700 1 -- 172.21.15.46:0/2668924332 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90380ce100 con 0x7f90480733d0 2024-05-08T23:04:32.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9046800700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 0x7f902c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf52d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf52dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf484150 con 0x564fcf533970 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf481b10 con 0x564fcf52d890 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.412+0000 7f9060abab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf47d400 con 0x564fcf533050 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf52d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52432/0 (socket says 172.21.15.46:52432) 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 -- 172.21.15.46:0/517460430 learned_addr learned my addr 172.21.15.46:0/517460430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9046800700 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf52dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 -- 172.21.15.46:0/517460430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 msgr2=0x564fcf52dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9046800700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 0x7f902c0a3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9040009fd0 tx=0x7f9040009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf52dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 -- 172.21.15.46:0/517460430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 msgr2=0x564fcf52d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf52d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.413+0000 7f9045e00700 1 -- 172.21.15.46:0/517460430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcf2fe3b0 con 0x564fcf533050 2024-05-08T23:04:32.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9045e00700 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90380098c0 tx=0x7f90380054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e3edb70a87b60e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9033400700 1 -- 172.21.15.46:0/517460430 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903809ac40 con 0x564fcf533050 2024-05-08T23:04:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9033400700 1 -- 172.21.15.46:0/517460430 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903809ea70 con 0x564fcf533050 2024-05-08T23:04:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9033400700 1 -- 172.21.15.46:0/517460430 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038005340 con 0x564fcf533050 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 -- 172.21.15.46:0/517460430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 msgr2=0x564fcf533430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90380098c0 tx=0x7f90380054e0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 -- 172.21.15.46:0/517460430 shutdown_connections 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf52dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf533430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 --2- 172.21.15.46:0/517460430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf52d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.414+0000 7f9060abab80 1 -- 172.21.15.46:0/517460430 >> 172.21.15.46:0/517460430 conn(0x564fcf48b210 msgr2=0x564fcf5282d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.415+0000 7f9060abab80 1 -- 172.21.15.46:0/517460430 shutdown_connections 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.415+0000 7f9060abab80 1 -- 172.21.15.46:0/517460430 wait complete. 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.415+0000 7f9060abab80 1 -- start start 2024-05-08T23:04:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.415+0000 7f9060abab80 1 Processor -- start 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf5398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 0x564fcf536cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf419560 con 0x564fcf52d890 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf419200 con 0x564fcf533050 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf3006a0 con 0x564fcf533970 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9060abab80 10 client.0 _ll_register_callbacks cb 0x564fcf422b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9045e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf5398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9045400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 0x564fcf536cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9046800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9045400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 0x564fcf536cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41746/0 (socket says 172.21.15.46:41746) 2024-05-08T23:04:32.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9046800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52448/0 (socket says 172.21.15.46:52448) 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.416+0000 7f9045400700 1 -- 172.21.15.46:0/333512433 learned_addr learned my addr 172.21.15.46:0/333512433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 -- 172.21.15.46:0/333512433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 msgr2=0x564fcf536cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 0x564fcf536cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 -- 172.21.15.46:0/333512433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 msgr2=0x564fcf5398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf5398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 -- 172.21.15.46:0/333512433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038009520 con 0x564fcf533970 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9046800700 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f904000fe30 tx=0x7f9040004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98efecf2fbc4ff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9027e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000e040 con 0x564fcf533970 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9040017a60 con 0x564fcf533970 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.417+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fcf3d6270 con 0x564fcf533970 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9040017c20 con 0x564fcf533970 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 10 client.25807 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564fcf2aab60 con 0x564fcf533970 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fcf3d6270 con 0x564fcf533970 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f904002a3c0 con 0x564fcf533970 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 10 client.25807 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 10 client.25807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 20 client.25807 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf547050 0x564fcf549470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 4 client.25807 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.418+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fcf4fc810 con 0x564fcf547050 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.419+0000 7f9045e00700 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf547050 0x564fcf549470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.419+0000 7f9045e00700 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf547050 0x564fcf549470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac081ddb61a2b19a server_cookie=1e46cac87a3b982a in_seq=0 out_seq=0 2024-05-08T23:04:32.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.419+0000 7f9027e00700 10 client.25807 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9027e00700 1 -- 172.21.15.46:0/333512433 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fcf4fc810 con 0x564fcf547050 2024-05-08T23:04:32.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9027e00700 10 client.25807 handle_command_reply: tid=0 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 client.25807 shutdown 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 msgr2=0x564fcf5373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f904000fe30 tx=0x7f9040004b40 comp rx=0 tx=0).stop 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 shutdown_connections 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf547050 0x564fcf549470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533970 0x564fcf5373c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf5398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9027e00700 0 client.25807 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 --2- 172.21.15.46:0/333512433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533050 0x564fcf536cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 >> 172.21.15.46:0/333512433 conn(0x564fcf48b210 msgr2=0x564fcf5280b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 shutdown_connections 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.420+0000 7f9060abab80 1 -- 172.21.15.46:0/333512433 wait complete. 2024-05-08T23:04:32.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.421+0000 7f9060abab80 20 client.25807 trim_cache size 0 max 16384 2024-05-08T23:04:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.434+0000 7f9060abab80 1 Processor -- start 2024-05-08T23:04:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 -- start start 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf5372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf536460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf52d650 con 0x564fcf533970 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf520040 con 0x564fcf52d890 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9060abab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf52d350 con 0x564fcf533050 2024-05-08T23:04:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9046800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9046800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45998/0 (socket says 172.21.15.46:45998) 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9046800700 1 -- 172.21.15.46:0/1320676299 learned_addr learned my addr 172.21.15.46:0/1320676299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.435+0000 7f9045e00700 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf5372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9045400700 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf536460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 -- 172.21.15.46:0/1320676299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 msgr2=0x564fcf536460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf536460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 -- 172.21.15.46:0/1320676299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 msgr2=0x564fcf5372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf5372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 -- 172.21.15.46:0/1320676299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcf443560 con 0x564fcf533970 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9046800700 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f904002a720 tx=0x7f90400177a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab0ba9e86966087 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9033e00700 1 -- 172.21.15.46:0/1320676299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000e040 con 0x564fcf533970 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9033e00700 1 -- 172.21.15.46:0/1320676299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904000f040 con 0x564fcf533970 2024-05-08T23:04:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.436+0000 7f9033e00700 1 -- 172.21.15.46:0/1320676299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040037690 con 0x564fcf533970 2024-05-08T23:04:32.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- 172.21.15.46:0/1320676299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 msgr2=0x564fcf539490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f904002a720 tx=0x7f90400177a0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- 172.21.15.46:0/1320676299 shutdown_connections 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf533970 0x564fcf539490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf52d890 0x564fcf5372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 --2- 172.21.15.46:0/1320676299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf536460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- 172.21.15.46:0/1320676299 >> 172.21.15.46:0/1320676299 conn(0x564fcf48b210 msgr2=0x564fcf526d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- 172.21.15.46:0/1320676299 shutdown_connections 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- 172.21.15.46:0/1320676299 wait complete. 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 -- start start 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.437+0000 7f9060abab80 1 Processor -- start 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf53c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 0x564fcf59c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf53cc70 con 0x564fcf52d890 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf53cde0 con 0x564fcf533970 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcf59ca90 con 0x564fcf533050 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9060abab80 10 client.0 _ll_register_callbacks cb 0x564fcf422b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9045e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9045e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46000/0 (socket says 172.21.15.46:46000) 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9045e00700 1 -- 172.21.15.46:0/4005456043 learned_addr learned my addr 172.21.15.46:0/4005456043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9045400700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf53c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.438+0000 7f9046800700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 0x564fcf59c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 -- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 msgr2=0x564fcf53c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf53c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 -- 172.21.15.46:0/4005456043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 msgr2=0x564fcf59c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 0x564fcf59c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 -- 172.21.15.46:0/4005456043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fcf2aab60 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9046800700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 0x564fcf59c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9045e00700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9038009fd0 tx=0x7f9038009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113cea9ea7ab40a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9027e00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038005140 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90380052a0 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f903809ea70 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9038021380 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9060abab80 10 client.15801 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564fcf302cd0 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.439+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903809ea70 con 0x564fcf52d890 2024-05-08T23:04:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f903802a060 con 0x564fcf52d890 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 10 client.15801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 20 client.15801 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf59dea0 0x564fcf5a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 4 client.15801 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564fcf475ec0 con 0x564fcf59dea0 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.440+0000 7f9045400700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf59dea0 0x564fcf5a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9045400700 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf59dea0 0x564fcf5a02c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32dbf5ec426dd355 server_cookie=a6d07a8ae6fb457b in_seq=0 out_seq=0 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9027e00700 10 client.15801 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9027e00700 1 -- 172.21.15.46:0/4005456043 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564fcf475ec0 con 0x564fcf59dea0 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9027e00700 10 client.15801 handle_command_reply: tid=0 2024-05-08T23:04:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 client.15801 shutdown 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 msgr2=0x564fcf53bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9038009fd0 tx=0x7f9038009260 comp rx=0 tx=0).stop 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 shutdown_connections 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564fcf59dea0 0x564fcf5a02c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fcf533970 0x564fcf59c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fcf52d890 0x564fcf53bfc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 --2- 172.21.15.46:0/4005456043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564fcf533050 0x564fcf53c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9027e00700 0 client.15801 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 >> 172.21.15.46:0/4005456043 conn(0x564fcf48b210 msgr2=0x564fcf54ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 shutdown_connections 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 1 -- 172.21.15.46:0/4005456043 wait complete. 2024-05-08T23:04:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.441+0000 7f9060abab80 20 client.15801 trim_cache size 0 max 16384 2024-05-08T23:04:32.446 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:32.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 msgr2=0x7f902c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 0x7f902c0a3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9040009fd0 tx=0x7f9040009920 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 msgr2=0x7f90481995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9038005950 tx=0x7f903800fa00 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 shutdown_connections 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f902c0a14d0 0x7f902c0a3990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904810d490 0x7f904819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90480733d0 0x7f90481995e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 --2- 172.21.15.46:0/2668924332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9048073cf0 0x7f9048199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 >> 172.21.15.46:0/2668924332 conn(0x7f90480fc740 msgr2=0x7f90480fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 shutdown_connections 2024-05-08T23:04:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.442+0000 7f9047200700 1 -- 172.21.15.46:0/2668924332 wait complete. 2024-05-08T23:04:32.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 -- 172.21.15.46:0/3140203895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d28101a30 msgr2=0x7f7d28105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3140203895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d28101a30 0x7f7d28105a70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009990 tx=0x7f7d14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 -- 172.21.15.46:0/3140203895 shutdown_connections 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3140203895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d281061a0 0x7f7d2810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3140203895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d28101080 0x7f7d28101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3140203895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d28101a30 0x7f7d28105a70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 -- 172.21.15.46:0/3140203895 >> 172.21.15.46:0/3140203895 conn(0x7f7d280fc8e0 msgr2=0x7f7d280fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 -- 172.21.15.46:0/3140203895 shutdown_connections 2024-05-08T23:04:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.663+0000 7f7d2f600700 1 -- 172.21.15.46:0/3140203895 wait complete. 2024-05-08T23:04:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.664+0000 7f7d2f600700 1 Processor -- start 2024-05-08T23:04:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.664+0000 7f7d2f600700 1 -- start start 2024-05-08T23:04:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d28101080 0x7f7d28199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d2819de10 0x7f7d2819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819e890 con 0x7f7d281061a0 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819ea00 con 0x7f7d2819de10 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819ebc0 con 0x7f7d28101080 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46020/0 (socket says 172.21.15.46:46020) 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d27e00700 1 -- 172.21.15.46:0/3042737080 learned_addr learned my addr 172.21.15.46:0/3042737080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2d600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d2819de10 0x7f7d2819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.665+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d28101080 0x7f7d28199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 -- 172.21.15.46:0/3042737080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d28101080 msgr2=0x7f7d28199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d28101080 0x7f7d28199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 -- 172.21.15.46:0/3042737080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d2819de10 msgr2=0x7f7d2819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d2819de10 0x7f7d2819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 -- 172.21.15.46:0/3042737080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140095f0 con 0x7f7d281061a0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d27e00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d140059d0 tx=0x7f7d14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d25600700 1 -- 172.21.15.46:0/3042737080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1401d070 con 0x7f7d281061a0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d25600700 1 -- 172.21.15.46:0/3042737080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d14022470 con 0x7f7d281061a0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d281a60c0 con 0x7f7d281061a0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d25600700 1 -- 172.21.15.46:0/3042737080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14017650 con 0x7f7d281061a0 2024-05-08T23:04:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.666+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281a64e0 con 0x7f7d281061a0 2024-05-08T23:04:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.669+0000 7f7d25600700 1 -- 172.21.15.46:0/3042737080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f7d140225e0 con 0x7f7d281061a0 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.670+0000 7f7d25600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 0x7f7d100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d25600700 1 -- 172.21.15.46:0/3042737080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7d140ce5c0 con 0x7f7d281061a0 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 Processor -- start 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 0x7f7d100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 -- start start 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b5720 0x563ccd2af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2af890 0x563ccd2afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd206150 con 0x563ccd2b4e00 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd203b10 con 0x563ccd2af890 2024-05-08T23:04:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.671+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd1ff400 con 0x563ccd2b5720 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46022/0 (socket says 172.21.15.46:46022) 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 -- 172.21.15.46:0/182729337 learned_addr learned my addr 172.21.15.46:0/182729337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 0x7f7d100a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009c60 tx=0x7f7d1c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2d600700 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2af890 0x563ccd2afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d27e00700 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b5720 0x563ccd2af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 -- 172.21.15.46:0/182729337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b5720 msgr2=0x563ccd2af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b5720 0x563ccd2af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 -- 172.21.15.46:0/182729337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2af890 msgr2=0x563ccd2afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2af890 0x563ccd2afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 -- 172.21.15.46:0/182729337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ccd0803b0 con 0x563ccd2b4e00 2024-05-08T23:04:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.672+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e430 tx=0x7f7d1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70180950b3b89dfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.673+0000 7f7d19c00700 1 -- 172.21.15.46:0/182729337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00edd0 con 0x563ccd2b4e00 2024-05-08T23:04:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.673+0000 7f7d19c00700 1 -- 172.21.15.46:0/182729337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c0044f0 con 0x563ccd2b4e00 2024-05-08T23:04:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.673+0000 7f7d19c00700 1 -- 172.21.15.46:0/182729337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0266d0 con 0x563ccd2b4e00 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.673+0000 7f7d4310eb80 1 -- 172.21.15.46:0/182729337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 msgr2=0x563ccd2b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.673+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e430 tx=0x7f7d1c014000 comp rx=0 tx=0).stop 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 -- 172.21.15.46:0/182729337 shutdown_connections 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2af890 0x563ccd2afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b51e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/182729337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b5720 0x563ccd2af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 -- 172.21.15.46:0/182729337 >> 172.21.15.46:0/182729337 conn(0x563ccd20d210 msgr2=0x563ccd2a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 -- 172.21.15.46:0/182729337 shutdown_connections 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 -- 172.21.15.46:0/182729337 wait complete. 2024-05-08T23:04:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.674+0000 7f7d4310eb80 1 -- start start 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 Processor -- start 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd19b560 con 0x563ccd2af890 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd19b200 con 0x563ccd2b5720 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.675+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd0826a0 con 0x563ccd2b4e00 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d4310eb80 10 client.0 _ll_register_callbacks cb 0x563ccd1a4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46032/0 (socket says 172.21.15.46:46032) 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 -- 172.21.15.46:0/2248757179 learned_addr learned my addr 172.21.15.46:0/2248757179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d27e00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 -- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 msgr2=0x563ccd2b89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 -- 172.21.15.46:0/2248757179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 msgr2=0x563ccd2b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2cc00700 1 -- 172.21.15.46:0/2248757179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c017070 con 0x563ccd2af890 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.676+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c011ab0 tx=0x7f7d1c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef35399361dce6a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d0e000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004080 con 0x563ccd2af890 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7d1c0041e0 con 0x563ccd2af890 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ccd158270 con 0x563ccd2af890 2024-05-08T23:04:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1c026470 con 0x563ccd2af890 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d4310eb80 10 client.15813 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ccd02cb60 con 0x563ccd2af890 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.677+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ccd158270 con 0x563ccd2af890 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d1c01d610 con 0x563ccd2af890 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 10 client.15813 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 20 client.15813 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd2c8ff0 0x563ccd2cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 4 client.15813 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ccd27e810 con 0x563ccd2c8ff0 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.678+0000 7f7d27e00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd2c8ff0 0x563ccd2cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.679+0000 7f7d27e00700 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd2c8ff0 0x563ccd2cb410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25dd7a31c78d50f2 server_cookie=eacd2512d949337a in_seq=0 out_seq=0 2024-05-08T23:04:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.679+0000 7f7d0e000700 10 client.15813 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.679+0000 7f7d0e000700 1 -- 172.21.15.46:0/2248757179 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ccd27e810 con 0x563ccd2c8ff0 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.679+0000 7f7d0e000700 10 client.15813 handle_command_reply: tid=0 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.679+0000 7f7d4310eb80 1 client.15813 shutdown 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 msgr2=0x563ccd2bb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c011ab0 tx=0x7f7d1c0057d0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 shutdown_connections 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd2c8ff0 0x563ccd2cb410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2bb6c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d0e000700 0 client.15813 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2248757179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 >> 172.21.15.46:0/2248757179 conn(0x563ccd20d210 msgr2=0x563ccd2a95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 shutdown_connections 2024-05-08T23:04:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2248757179 wait complete. 2024-05-08T23:04:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.680+0000 7f7d4310eb80 20 client.15813 trim_cache size 0 max 16384 2024-05-08T23:04:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 Processor -- start 2024-05-08T23:04:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 -- start start 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2af890 0x563ccd2af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2b9930 con 0x563ccd2b4e00 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2a4800 con 0x563ccd2b5720 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.694+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2aa260 con 0x563ccd2af890 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46040/0 (socket says 172.21.15.46:46040) 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2af890 0x563ccd2af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d27e00700 1 -- 172.21.15.46:0/2723677574 learned_addr learned my addr 172.21.15.46:0/2723677574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 -- 172.21.15.46:0/2723677574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2af890 msgr2=0x563ccd2af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2af890 0x563ccd2af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 -- 172.21.15.46:0/2723677574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 msgr2=0x563ccd2b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 -- 172.21.15.46:0/2723677574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ccd1c5560 con 0x563ccd2b5720 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d27e00700 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.695+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2a9d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d20011b30 tx=0x7f7d20009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505503a53971d5b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d1a600700 1 -- 172.21.15.46:0/2723677574 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000e040 con 0x563ccd2b5720 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d1a600700 1 -- 172.21.15.46:0/2723677574 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2000f040 con 0x563ccd2b5720 2024-05-08T23:04:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d1a600700 1 -- 172.21.15.46:0/2723677574 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20017d60 con 0x563ccd2b5720 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2723677574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 msgr2=0x563ccd2a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2a9d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d20011b30 tx=0x7f7d20009920 comp rx=0 tx=0).stop 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2723677574 shutdown_connections 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2a9d20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2af890 0x563ccd2af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2723677574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2b4e00 0x563ccd2b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2723677574 >> 172.21.15.46:0/2723677574 conn(0x563ccd20d210 msgr2=0x563ccd2afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2723677574 shutdown_connections 2024-05-08T23:04:32.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.696+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2723677574 wait complete. 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.697+0000 7f7d4310eb80 1 -- start start 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.697+0000 7f7d4310eb80 1 Processor -- start 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.697+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2c0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.697+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.697+0000 7f7d4310eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2b1650 con 0x563ccd2af890 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2b17c0 con 0x563ccd2b5720 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d4310eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ccd2b1980 con 0x563ccd2b4e00 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d4310eb80 10 client.0 _ll_register_callbacks cb 0x563ccd1a4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41838/0 (socket says 172.21.15.46:41838) 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 -- 172.21.15.46:0/2369012462 learned_addr learned my addr 172.21.15.46:0/2369012462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52508/0 (socket says 172.21.15.46:52508) 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 -- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 msgr2=0x563ccd2b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 -- 172.21.15.46:0/2369012462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 msgr2=0x563ccd2c0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2c0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 -- 172.21.15.46:0/2369012462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281a64e0 con 0x563ccd2b5720 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d2d600700 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d20017860 tx=0x7f7d20009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1307f68e937885dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.698+0000 7f7d0e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000e050 con 0x563ccd2b5720 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d1c0044f0 con 0x563ccd2b5720 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7d20003c00 con 0x563ccd2b5720 2024-05-08T23:04:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d20017e50 con 0x563ccd2b5720 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 10 client.25799 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7d200117e0 con 0x563ccd2b5720 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c0044f0 con 0x563ccd2b5720 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d20005260 con 0x563ccd2b5720 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 10 client.25799 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 10 client.25799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.699+0000 7f7d4310eb80 20 client.25799 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd31fd80 0x563ccd3221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d4310eb80 4 client.25799 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ccd1f7ec0 con 0x563ccd31fd80 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd31fd80 0x563ccd3221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d2cc00700 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd31fd80 0x563ccd3221a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=153a641c841bdd2a server_cookie=c2bf31eea1d4d2b1 in_seq=0 out_seq=0 2024-05-08T23:04:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.700+0000 7f7d0e000700 10 client.25799 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d0e000700 1 -- 172.21.15.46:0/2369012462 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ccd1f7ec0 con 0x563ccd31fd80 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d0e000700 10 client.25799 handle_command_reply: tid=0 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 client.25799 shutdown 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 msgr2=0x563ccd2b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d20017860 tx=0x7f7d20009650 comp rx=0 tx=0).stop 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 shutdown_connections 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x563ccd31fd80 0x563ccd3221a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ccd2b5720 0x563ccd2b0f40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ccd2af890 0x563ccd2c0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d0e000700 0 client.25799 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 --2- 172.21.15.46:0/2369012462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ccd2b4e00 0x563ccd2b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 >> 172.21.15.46:0/2369012462 conn(0x563ccd20d210 msgr2=0x563ccd2cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 shutdown_connections 2024-05-08T23:04:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 1 -- 172.21.15.46:0/2369012462 wait complete. 2024-05-08T23:04:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.701+0000 7f7d4310eb80 20 client.25799 trim_cache size 0 max 16384 2024-05-08T23:04:32.706 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 msgr2=0x7f7d100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 0x7f7d100a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009c60 tx=0x7f7d1c009260 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 msgr2=0x7f7d28199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d140059d0 tx=0x7f7d14004730 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 shutdown_connections 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d2819de10 0x7f7d2819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d28101080 0x7f7d28199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7d100a13b0 0x7f7d100a3870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 --2- 172.21.15.46:0/3042737080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d281061a0 0x7f7d28199a70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 >> 172.21.15.46:0/3042737080 conn(0x7f7d280fc8e0 msgr2=0x7f7d280fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 shutdown_connections 2024-05-08T23:04:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.702+0000 7f7d2f600700 1 -- 172.21.15.46:0/3042737080 wait complete. 2024-05-08T23:04:32.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:04:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 -- 172.21.15.46:0/1738106762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 msgr2=0x7efd2810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 --2- 172.21.15.46:0/1738106762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd2810db60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd2000b0a0 tx=0x7efd2000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 -- 172.21.15.46:0/1738106762 shutdown_connections 2024-05-08T23:04:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 --2- 172.21.15.46:0/1738106762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd2810db60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 --2- 172.21.15.46:0/1738106762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd281012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 --2- 172.21.15.46:0/1738106762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28101870 0x7efd281058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.923+0000 7efd2e800700 1 -- 172.21.15.46:0/1738106762 >> 172.21.15.46:0/1738106762 conn(0x7efd280fc740 msgr2=0x7efd280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.924+0000 7efd2e800700 1 -- 172.21.15.46:0/1738106762 shutdown_connections 2024-05-08T23:04:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.924+0000 7efd2e800700 1 -- 172.21.15.46:0/1738106762 wait complete. 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.924+0000 7efd2e800700 1 Processor -- start 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 -- start start 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 0x7efd281a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281a5a40 con 0x7efd28073d20 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281a5b80 con 0x7efd28105fe0 2024-05-08T23:04:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281a5d40 con 0x7efd28100ec0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 0x7efd281a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52526/0 (socket says 172.21.15.46:52526) 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2d400700 1 -- 172.21.15.46:0/3529518161 learned_addr learned my addr 172.21.15.46:0/3529518161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.925+0000 7efd2ca00700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 -- 172.21.15.46:0/3529518161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 msgr2=0x7efd280737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd280737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 -- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 msgr2=0x7efd281a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 0x7efd281a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 -- 172.21.15.46:0/3529518161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20009d00 con 0x7efd28100ec0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2de00700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 0x7efd281a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2d400700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00e9c0 tx=0x7efd1c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd26000700 1 -- 172.21.15.46:0/3529518161 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0097b0 con 0x7efd28100ec0 2024-05-08T23:04:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd26000700 1 -- 172.21.15.46:0/3529518161 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c00ee60 con 0x7efd28100ec0 2024-05-08T23:04:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd281a5fc0 con 0x7efd28100ec0 2024-05-08T23:04:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.926+0000 7efd26000700 1 -- 172.21.15.46:0/3529518161 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0054f0 con 0x7efd28100ec0 2024-05-08T23:04:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.927+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281a6450 con 0x7efd28100ec0 2024-05-08T23:04:32.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.931+0000 7efd26000700 1 -- 172.21.15.46:0/3529518161 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7efd1c004750 con 0x7efd28100ec0 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.932+0000 7efd43994b80 1 Processor -- start 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.932+0000 7efd43994b80 1 -- start start 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.932+0000 7efd26000700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 0x7efd100a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd26000700 1 -- 172.21.15.46:0/3529518161 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efd1c014070 con 0x7efd28100ec0 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac5f890 0x55e3dac5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dabb6150 con 0x55e3dac65970 2024-05-08T23:04:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dabb3b10 con 0x55e3dac5f890 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd43994b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dabaf400 con 0x55e3dac65050 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2de00700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 0x7efd100a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52530/0 (socket says 172.21.15.46:52530) 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 -- 172.21.15.46:0/3475965066 learned_addr learned my addr 172.21.15.46:0/3475965066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2de00700 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac5f890 0x55e3dac5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 -- 172.21.15.46:0/3475965066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac5f890 msgr2=0x55e3dac5fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac5f890 0x55e3dac5fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 -- 172.21.15.46:0/3475965066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 msgr2=0x55e3dac5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac5f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.933+0000 7efd2d400700 1 -- 172.21.15.46:0/3475965066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3daa303b0 con 0x55e3dac65050 2024-05-08T23:04:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.934+0000 7efd2de00700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 0x7efd100a3910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd20009610 tx=0x7efd2000bb30 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.934+0000 7efd2d400700 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efd1c099d30 tx=0x7efd1c09a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af204c2028502 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.934+0000 7efd1a600700 1 -- 172.21.15.46:0/3475965066 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0a24b0 con 0x55e3dac65050 2024-05-08T23:04:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.934+0000 7efd1a600700 1 -- 172.21.15.46:0/3475965066 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c098170 con 0x55e3dac65050 2024-05-08T23:04:32.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.934+0000 7efd1a600700 1 -- 172.21.15.46:0/3475965066 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0cd320 con 0x55e3dac65050 2024-05-08T23:04:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- 172.21.15.46:0/3475965066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 msgr2=0x55e3dac65430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efd1c099d30 tx=0x7efd1c09a7b0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- 172.21.15.46:0/3475965066 shutdown_connections 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac5f890 0x55e3dac5fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac65050 0x55e3dac65430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 --2- 172.21.15.46:0/3475965066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- 172.21.15.46:0/3475965066 >> 172.21.15.46:0/3475965066 conn(0x55e3dabbd210 msgr2=0x55e3dac5a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- 172.21.15.46:0/3475965066 shutdown_connections 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- 172.21.15.46:0/3475965066 wait complete. 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.935+0000 7efd43994b80 1 -- start start 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 Processor -- start 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac69310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dab4b560 con 0x55e3dac65970 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dab4b200 con 0x55e3dac65050 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3daa326a0 con 0x55e3dac5f890 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.936+0000 7efd43994b80 10 client.0 _ll_register_callbacks cb 0x55e3dab54b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac69310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52542/0 (socket says 172.21.15.46:52542) 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 -- 172.21.15.46:0/739534706 learned_addr learned my addr 172.21.15.46:0/739534706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2ca00700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 -- 172.21.15.46:0/739534706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 msgr2=0x55e3dac68bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac68bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 -- 172.21.15.46:0/739534706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 msgr2=0x55e3dac69310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac69310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 -- 172.21.15.46:0/739534706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281a6450 con 0x55e3dac5f890 2024-05-08T23:04:32.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd2d400700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0964e0 tx=0x7efd1c09dfe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e1c24863adfe0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0b8430 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efd1c0a24b0 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.937+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3dab08270 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd1c0cd120 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 10 client.25843 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e3da9dcb60 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c07f020 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd1c09a3b0 con 0x55e3dac5f890 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 10 client.25843 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 10 client.25843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 20 client.25843 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3dac78ef0 0x55e3dac7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 4 client.25843 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3dac2e810 con 0x55e3dac78ef0 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.938+0000 7efd2ca00700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3dac78ef0 0x55e3dac7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.939+0000 7efd2ca00700 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3dac78ef0 0x55e3dac7b310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25091763db99568c server_cookie=96d80fb5aa8112e2 in_seq=0 out_seq=0 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.939+0000 7efd0ea00700 10 client.25843 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd0ea00700 1 -- 172.21.15.46:0/739534706 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3dac2e810 con 0x55e3dac78ef0 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd0ea00700 10 client.25843 handle_command_reply: tid=0 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 client.25843 shutdown 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 msgr2=0x55e3dac6b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0964e0 tx=0x7efd1c09dfe0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 shutdown_connections 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3dac78ef0 0x55e3dac7b310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac69310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac6b890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd0ea00700 0 client.25843 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 --2- 172.21.15.46:0/739534706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac68bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 >> 172.21.15.46:0/739534706 conn(0x55e3dabbd210 msgr2=0x55e3dac5a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 shutdown_connections 2024-05-08T23:04:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 1 -- 172.21.15.46:0/739534706 wait complete. 2024-05-08T23:04:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.940+0000 7efd43994b80 20 client.25843 trim_cache size 0 max 16384 2024-05-08T23:04:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 Processor -- start 2024-05-08T23:04:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 -- start start 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac75760 con 0x55e3dac65970 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac5f350 con 0x55e3dac65050 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.954+0000 7efd43994b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac52040 con 0x55e3dac5f890 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46100/0 (socket says 172.21.15.46:46100) 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2de00700 1 -- 172.21.15.46:0/50492844 learned_addr learned my addr 172.21.15.46:0/50492844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 -- 172.21.15.46:0/50492844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 msgr2=0x55e3dac7b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac7b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 -- 172.21.15.46:0/50492844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 msgr2=0x55e3dac62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 -- 172.21.15.46:0/50492844 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3dab75560 con 0x55e3dac65050 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2de00700 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.955+0000 7efd2ca00700 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac5b820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55e3dac24d00 tx=0x7efd14009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7cd458bac0b7be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd1b000700 1 -- 172.21.15.46:0/50492844 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x55e3dac65050 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd1b000700 1 -- 172.21.15.46:0/50492844 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd14018040 con 0x55e3dac65050 2024-05-08T23:04:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd1b000700 1 -- 172.21.15.46:0/50492844 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14010430 con 0x55e3dac65050 2024-05-08T23:04:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 -- 172.21.15.46:0/50492844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 msgr2=0x55e3dac5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac5b820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55e3dac24d00 tx=0x7efd14009260 comp rx=0 tx=0).stop 2024-05-08T23:04:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 -- 172.21.15.46:0/50492844 shutdown_connections 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65970 0x55e3dac62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac7b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 --2- 172.21.15.46:0/50492844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65050 0x55e3dac5b820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 -- 172.21.15.46:0/50492844 >> 172.21.15.46:0/50492844 conn(0x55e3dabbd210 msgr2=0x55e3dac5fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 -- 172.21.15.46:0/50492844 shutdown_connections 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.956+0000 7efd43994b80 1 -- 172.21.15.46:0/50492844 wait complete. 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 -- start start 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 Processor -- start 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac61c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 0x55e3dac60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac708e0 con 0x55e3dac65050 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.957+0000 7efd43994b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac70a50 con 0x55e3dac65970 2024-05-08T23:04:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd43994b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dac70c10 con 0x55e3dac5f890 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd43994b80 10 client.0 _ll_register_callbacks cb 0x55e3dab54b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 0x55e3dac60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46108/0 (socket says 172.21.15.46:46108) 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 -- 172.21.15.46:0/2990043673 learned_addr learned my addr 172.21.15.46:0/2990043673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2d400700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac61c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 -- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 msgr2=0x55e3dac61c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac61c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 -- 172.21.15.46:0/2990043673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 msgr2=0x55e3dac60ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 0x55e3dac60ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2ca00700 1 -- 172.21.15.46:0/2990043673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2810e3d0 con 0x55e3dac65050 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2de00700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 0x55e3dac60ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.958+0000 7efd2d400700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac61c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd2ca00700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd14009c70 tx=0x7efd1401f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6af4aa4621cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14017040 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efd1401f260 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd1c098170 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd1401f460 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd43994b80 10 client.15837 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efd14009ac0 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.959+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c098170 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd14026070 con 0x55e3dac65050 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 10 client.15837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 20 client.15837 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3daccfe50 0x55e3dacd2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 4 client.15837 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3daba7ec0 con 0x55e3daccfe50 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd2d400700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3daccfe50 0x55e3dacd2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd0ea00700 10 client.15837 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.960+0000 7efd2d400700 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3daccfe50 0x55e3dacd2270 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11cdcb1ff46e6b2a server_cookie=74af6a177e03851 in_seq=0 out_seq=0 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd0ea00700 1 -- 172.21.15.46:0/2990043673 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3daba7ec0 con 0x55e3daccfe50 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd0ea00700 10 client.15837 handle_command_reply: tid=0 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 client.15837 shutdown 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 msgr2=0x55e3dac62180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd14009c70 tx=0x7efd1401f6a0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 shutdown_connections 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3dac65970 0x55e3dac60ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3dac5f890 0x55e3dac61c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e3daccfe50 0x55e3dacd2270 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 --2- 172.21.15.46:0/2990043673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dac65050 0x55e3dac62180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 >> 172.21.15.46:0/2990043673 conn(0x55e3dabbd210 msgr2=0x55e3dac62cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd0ea00700 0 client.15837 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 shutdown_connections 2024-05-08T23:04:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 1 -- 172.21.15.46:0/2990043673 wait complete. 2024-05-08T23:04:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.961+0000 7efd43994b80 20 client.15837 trim_cache size 0 max 16384 2024-05-08T23:04:32.965 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 msgr2=0x7efd100a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 0x7efd100a3910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd20009610 tx=0x7efd2000bb30 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 msgr2=0x7efd280732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00e9c0 tx=0x7efd1c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 shutdown_connections 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efd100a1450 0x7efd100a3910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd28073d20 0x7efd281a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd28100ec0 0x7efd280732a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 --2- 172.21.15.46:0/3529518161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28105fe0 0x7efd280737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 >> 172.21.15.46:0/3529518161 conn(0x7efd280fc740 msgr2=0x7efd280fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 shutdown_connections 2024-05-08T23:04:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:32.962+0000 7efd2e800700 1 -- 172.21.15.46:0/3529518161 wait complete. 2024-05-08T23:04:32.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/562454137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:32.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 -- 172.21.15.46:0/3878615441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 msgr2=0x7f8bd810dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 --2- 172.21.15.46:0/3878615441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd810dad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009f90 tx=0x7f8bd000b330 comp rx=0 tx=0).stop 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 -- 172.21.15.46:0/3878615441 shutdown_connections 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 --2- 172.21.15.46:0/3878615441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd810dad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 --2- 172.21.15.46:0/3878615441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8106d30 0x7f8bd8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.202+0000 7f8bde000700 1 --2- 172.21.15.46:0/3878615441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8068160 0x7f8bd80685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.203+0000 7f8bde000700 1 -- 172.21.15.46:0/3878615441 >> 172.21.15.46:0/3878615441 conn(0x7f8bd8075710 msgr2=0x7f8bd8075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.203+0000 7f8bde000700 1 -- 172.21.15.46:0/3878615441 shutdown_connections 2024-05-08T23:04:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.203+0000 7f8bde000700 1 -- 172.21.15.46:0/3878615441 wait complete. 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 Processor -- start 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 -- start start 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 0x7f8bd819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819e220 con 0x7f8bd8068160 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819e390 con 0x7f8bd8106d30 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bde000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819e550 con 0x7f8bd8068b20 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 0x7f8bd819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46114/0 (socket says 172.21.15.46:46114) 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 0x7f8bd819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41910/0 (socket says 172.21.15.46:41910) 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bdcc00700 1 -- 172.21.15.46:0/3030921198 learned_addr learned my addr 172.21.15.46:0/3030921198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.204+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 -- 172.21.15.46:0/3030921198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 msgr2=0x7f8bd8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 -- 172.21.15.46:0/3030921198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 msgr2=0x7f8bd819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 0x7f8bd819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 -- 172.21.15.46:0/3030921198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009c40 con 0x7f8bd8068160 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e9a0 tx=0x7f8bcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bd5600700 1 -- 172.21.15.46:0/3030921198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc009790 con 0x7f8bd8068160 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bd5600700 1 -- 172.21.15.46:0/3030921198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bcc00ce80 con 0x7f8bd8068160 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd819e830 con 0x7f8bd8068160 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bd5600700 1 -- 172.21.15.46:0/3030921198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc018680 con 0x7f8bd8068160 2024-05-08T23:04:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.205+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81a6070 con 0x7f8bd8068160 2024-05-08T23:04:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.208+0000 7f8bd5600700 1 -- 172.21.15.46:0/3030921198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f8bcc010430 con 0x7f8bd8068160 2024-05-08T23:04:33.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.210+0000 7f8bf3204b80 1 Processor -- start 2024-05-08T23:04:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 -- start start 2024-05-08T23:04:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bd5600700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 0x7f8bc00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bd5600700 1 -- 172.21.15.46:0/3030921198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8bcc014070 con 0x7f8bd8068160 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e62c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bdd600700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 0x7f8bc00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e626350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e57d150 con 0x562b3e62c050 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e57ab10 con 0x562b3e626890 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e576400 con 0x562b3e62c970 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e62c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41916/0 (socket says 172.21.15.46:41916) 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e626350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.211+0000 7f8bdd600700 1 -- 172.21.15.46:0/102959395 learned_addr learned my addr 172.21.15.46:0/102959395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 -- 172.21.15.46:0/102959395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 msgr2=0x562b3e626350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e626350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 -- 172.21.15.46:0/102959395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 msgr2=0x562b3e626cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 -- 172.21.15.46:0/102959395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b3e3f73b0 con 0x562b3e62c050 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdd600700 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdd600700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 0x7f8bc00a3990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007790 tx=0x7f8bd00077c0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e62c430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0ca690 tx=0x7f8bcc09a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b28f4384ed1ec88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bc9c00700 1 -- 172.21.15.46:0/102959395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc09ddb0 con 0x562b3e62c050 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bc9c00700 1 -- 172.21.15.46:0/102959395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bcc0a2400 con 0x562b3e62c050 2024-05-08T23:04:33.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.212+0000 7f8bc9c00700 1 -- 172.21.15.46:0/102959395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0996d0 con 0x562b3e62c050 2024-05-08T23:04:33.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/562454137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 -- 172.21.15.46:0/102959395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 msgr2=0x562b3e62c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e62c430 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0ca690 tx=0x7f8bcc09a680 comp rx=0 tx=0).stop 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 -- 172.21.15.46:0/102959395 shutdown_connections 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e62c430 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/102959395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e626350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 -- 172.21.15.46:0/102959395 >> 172.21.15.46:0/102959395 conn(0x562b3e584210 msgr2=0x562b3e6212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 -- 172.21.15.46:0/102959395 shutdown_connections 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.213+0000 7f8bf3204b80 1 -- 172.21.15.46:0/102959395 wait complete. 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.214+0000 7f8bf3204b80 1 -- start start 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.214+0000 7f8bf3204b80 1 Processor -- start 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e626890 0x562b3e62fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e62c050 0x562b3e6301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e512560 con 0x562b3e62c970 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e512200 con 0x562b3e62c050 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e3f96a0 con 0x562b3e626890 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bf3204b80 10 client.0 _ll_register_callbacks cb 0x562b3e51bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46126/0 (socket says 172.21.15.46:46126) 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bdd600700 1 -- 172.21.15.46:0/2788182733 learned_addr learned my addr 172.21.15.46:0/2788182733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e626890 0x562b3e62fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.215+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e62c050 0x562b3e6301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 -- 172.21.15.46:0/2788182733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e626890 msgr2=0x562b3e62fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e626890 0x562b3e62fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 -- 172.21.15.46:0/2788182733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e62c050 msgr2=0x562b3e6301a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e62c050 0x562b3e6301a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 -- 172.21.15.46:0/2788182733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81a6070 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bdd600700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bca0 tx=0x7f8bd0022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744b00a07f020f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bbe000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000e040 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8bd00054c0 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b3e4cf270 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.216+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bd0007e60 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 10 client.15849 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562b3e3a3b60 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b3e4cf270 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bd002bb00 con 0x562b3e62c970 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 10 client.15849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:04:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 20 client.15849 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6403f0 0x562b3e642810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 4 client.15849 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b3e5f5810 con 0x562b3e6403f0 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.217+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6403f0 0x562b3e642810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.218+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6403f0 0x562b3e642810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=435b2b6a9f9357f9 server_cookie=e326f13816b51d53 in_seq=0 out_seq=0 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.218+0000 7f8bbe000700 10 client.15849 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.218+0000 7f8bbe000700 1 -- 172.21.15.46:0/2788182733 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b3e5f5810 con 0x562b3e6403f0 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.218+0000 7f8bbe000700 10 client.15849 handle_command_reply: tid=0 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.218+0000 7f8bf3204b80 1 client.15849 shutdown 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 msgr2=0x562b3e6306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bca0 tx=0x7f8bd0022040 comp rx=0 tx=0).stop 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 shutdown_connections 2024-05-08T23:04:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6403f0 0x562b3e642810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e6306e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e626890 0x562b3e62fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2788182733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e62c050 0x562b3e6301a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bbe000700 0 client.15849 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 >> 172.21.15.46:0/2788182733 conn(0x562b3e584210 msgr2=0x562b3e6210b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 shutdown_connections 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2788182733 wait complete. 2024-05-08T23:04:33.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.219+0000 7f8bf3204b80 20 client.15849 trim_cache size 0 max 16384 2024-05-08T23:04:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.233+0000 7f8bf3204b80 1 Processor -- start 2024-05-08T23:04:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.233+0000 7f8bf3204b80 1 -- start start 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e647130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e621dc0 con 0x562b3e62c970 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e622820 con 0x562b3e626890 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e622a50 con 0x562b3e62c050 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e647130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52632/0 (socket says 172.21.15.46:52632) 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bd7e00700 1 -- 172.21.15.46:0/2233978577 learned_addr learned my addr 172.21.15.46:0/2233978577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e647130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46130/0 (socket says 172.21.15.46:46130) 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 -- 172.21.15.46:0/2233978577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 msgr2=0x562b3e630500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 -- 172.21.15.46:0/2233978577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 msgr2=0x562b3e647130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e647130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bdcc00700 1 -- 172.21.15.46:0/2233978577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b3e53c560 con 0x562b3e626890 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.234+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x562b3e632e80 tx=0x7f8bcc09a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37edea4ce9d9001 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bca600700 1 -- 172.21.15.46:0/2233978577 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010710 con 0x562b3e626890 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bca600700 1 -- 172.21.15.46:0/2233978577 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bcc010870 con 0x562b3e626890 2024-05-08T23:04:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bca600700 1 -- 172.21.15.46:0/2233978577 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0a21d0 con 0x562b3e626890 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2233978577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 msgr2=0x562b3e626350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x562b3e632e80 tx=0x7f8bcc09a100 comp rx=0 tx=0).stop 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2233978577 shutdown_connections 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c970 0x562b3e647130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e626350 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/2233978577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c050 0x562b3e630500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.235+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2233978577 >> 172.21.15.46:0/2233978577 conn(0x562b3e584210 msgr2=0x562b3e6210b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.236+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2233978577 shutdown_connections 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.236+0000 7f8bf3204b80 1 -- 172.21.15.46:0/2233978577 wait complete. 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.236+0000 7f8bf3204b80 1 -- start start 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.236+0000 7f8bf3204b80 1 Processor -- start 2024-05-08T23:04:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e633300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e633d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e695960 con 0x562b3e62c050 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e695ad0 con 0x562b3e626890 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3e695c90 con 0x562b3e62c970 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bf3204b80 10 client.0 _ll_register_callbacks cb 0x562b3e51bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46132/0 (socket says 172.21.15.46:46132) 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 -- 172.21.15.46:0/1242505229 learned_addr learned my addr 172.21.15.46:0/1242505229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e633300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bdd600700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e633d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 -- 172.21.15.46:0/1242505229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 msgr2=0x562b3e633d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e633d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 -- 172.21.15.46:0/1242505229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 msgr2=0x562b3e633300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e633300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.237+0000 7f8bd7e00700 1 -- 172.21.15.46:0/1242505229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc4009be0 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bd7e00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4012c80 tx=0x7f8bc4012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c847c4ad9e670094 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bbe000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4022070 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8bc4004020 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bcc0a2400 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc4017040 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bf3204b80 10 client.15855 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562b3e3f73b0 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.238+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bcc0a2400 con 0x562b3e62c050 2024-05-08T23:04:33.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bc4025020 con 0x562b3e62c050 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 20 client.15855 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6974e0 0x562b3e699900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 4 client.15855 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b3e56eec0 con 0x562b3e6974e0 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6974e0 0x562b3e699900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bdcc00700 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6974e0 0x562b3e699900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fabc493e53ebfae5 server_cookie=24b1787a6bf3ea33 in_seq=0 out_seq=0 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.239+0000 7f8bbe000700 10 client.15855 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bbe000700 1 -- 172.21.15.46:0/1242505229 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b3e56eec0 con 0x562b3e6974e0 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bbe000700 10 client.15855 handle_command_reply: tid=0 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 client.15855 shutdown 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 msgr2=0x562b3e633840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4012c80 tx=0x7f8bc4012f90 comp rx=0 tx=0).stop 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 shutdown_connections 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b3e62c970 0x562b3e633d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562b3e626890 0x562b3e633300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562b3e6974e0 0x562b3e699900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 --2- 172.21.15.46:0/1242505229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b3e62c050 0x562b3e633840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 >> 172.21.15.46:0/1242505229 conn(0x562b3e584210 msgr2=0x562b3e630540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bbe000700 0 client.15855 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 shutdown_connections 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 1 -- 172.21.15.46:0/1242505229 wait complete. 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.240+0000 7f8bf3204b80 20 client.15855 trim_cache size 0 max 16384 2024-05-08T23:04:33.244 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 msgr2=0x7f8bc00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 0x7f8bc00a3990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007790 tx=0x7f8bd00077c0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 msgr2=0x7f8bd8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e9a0 tx=0x7f8bcc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 shutdown_connections 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd8106d30 0x7f8bd819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8bc00a14d0 0x7f8bc00a3990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8068160 0x7f8bd8199300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 --2- 172.21.15.46:0/3030921198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd8068b20 0x7f8bd8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 >> 172.21.15.46:0/3030921198 conn(0x7f8bd8075710 msgr2=0x7f8bd810be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 shutdown_connections 2024-05-08T23:04:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.241+0000 7f8bde000700 1 -- 172.21.15.46:0/3030921198 wait complete. 2024-05-08T23:04:33.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:04:33.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/562454137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.460+0000 7f10e5e00700 1 -- 172.21.15.46:0/1053908115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 msgr2=0x7f10e010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.460+0000 7f10e5e00700 1 --2- 172.21.15.46:0/1053908115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e010dd20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b0a0 tx=0x7f10d800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 -- 172.21.15.46:0/1053908115 shutdown_connections 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 --2- 172.21.15.46:0/1053908115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e010dd20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 --2- 172.21.15.46:0/1053908115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0101080 0x7f10e0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 --2- 172.21.15.46:0/1053908115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101a30 0x7f10e0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 -- 172.21.15.46:0/1053908115 >> 172.21.15.46:0/1053908115 conn(0x7f10e00fc8e0 msgr2=0x7f10e00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 -- 172.21.15.46:0/1053908115 shutdown_connections 2024-05-08T23:04:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.461+0000 7f10e5e00700 1 -- 172.21.15.46:0/1053908115 wait complete. 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 Processor -- start 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 -- start start 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0073d20 0x7f10e01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e01a5c00 con 0x7f10e01061a0 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e01a5d70 con 0x7f10e0073d20 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.462+0000 7f10e5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e01a5f30 con 0x7f10e0101080 2024-05-08T23:04:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52672/0 (socket says 172.21.15.46:52672) 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 -- 172.21.15.46:0/2620060233 learned_addr learned my addr 172.21.15.46:0/2620060233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10df400700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0073d20 0x7f10e01a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 -- 172.21.15.46:0/2620060233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0073d20 msgr2=0x7f10e01a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0073d20 0x7f10e01a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 -- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 msgr2=0x7f10e00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 -- 172.21.15.46:0/2620060233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8009d00 con 0x7f10e0101080 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10de000700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.463+0000 7f10dea00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9a0 tx=0x7f10d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.464+0000 7f10e4a00700 1 -- 172.21.15.46:0/2620060233 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0009790 con 0x7f10e0101080 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.464+0000 7f10e4a00700 1 -- 172.21.15.46:0/2620060233 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10d0010430 con 0x7f10e0101080 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.464+0000 7f10e4a00700 1 -- 172.21.15.46:0/2620060233 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0018680 con 0x7f10e0101080 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.464+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e01a6210 con 0x7f10e0101080 2024-05-08T23:04:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.464+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e01a6720 con 0x7f10e0101080 2024-05-08T23:04:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.467+0000 7f10e4a00700 1 -- 172.21.15.46:0/2620060233 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f10d00105a0 con 0x7f10e0101080 2024-05-08T23:04:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.468+0000 7f10e4a00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 0x7f10c40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.468+0000 7f10f9851b80 1 Processor -- start 2024-05-08T23:04:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.468+0000 7f10e4a00700 1 -- 172.21.15.46:0/2620060233 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f10d0014070 con 0x7f10e0101080 2024-05-08T23:04:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.468+0000 7f10f9851b80 1 -- start start 2024-05-08T23:04:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.468+0000 7f10de000700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 0x7f10c40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e243150 con 0x55b83e2f1e00 2024-05-08T23:04:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e240b10 con 0x55b83e2ec890 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10f9851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e23c400 con 0x55b83e2f2720 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10de000700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 0x7f10c40a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10e00749d0 tx=0x7f10cc009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10de000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46164/0 (socket says 172.21.15.46:46164) 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10dea00700 1 -- 172.21.15.46:0/1369671864 learned_addr learned my addr 172.21.15.46:0/1369671864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.469+0000 7f10df400700 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 -- 172.21.15.46:0/1369671864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 msgr2=0x55b83e2eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 -- 172.21.15.46:0/1369671864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 msgr2=0x55b83e2f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 -- 172.21.15.46:0/1369671864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b83e0bd3b0 con 0x55b83e2f2720 2024-05-08T23:04:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10de000700 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2ec350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f10cc004290 tx=0x7f10cc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97557d9ffe019706 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10cbe00700 1 -- 172.21.15.46:0/1369671864 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc003d10 con 0x55b83e2f2720 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10cbe00700 1 -- 172.21.15.46:0/1369671864 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10cc003e70 con 0x55b83e2f2720 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.470+0000 7f10cbe00700 1 -- 172.21.15.46:0/1369671864 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01e680 con 0x55b83e2f2720 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 -- 172.21.15.46:0/1369671864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 msgr2=0x55b83e2ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2ec350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f10cc004290 tx=0x7f10cc014000 comp rx=0 tx=0).stop 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 -- 172.21.15.46:0/1369671864 shutdown_connections 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1369671864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2ec350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 -- 172.21.15.46:0/1369671864 >> 172.21.15.46:0/1369671864 conn(0x55b83e24a210 msgr2=0x55b83e2e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 -- 172.21.15.46:0/1369671864 shutdown_connections 2024-05-08T23:04:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.471+0000 7f10f9851b80 1 -- 172.21.15.46:0/1369671864 wait complete. 2024-05-08T23:04:33.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.472+0000 7f10f9851b80 1 -- start start 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.472+0000 7f10f9851b80 1 Processor -- start 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e1d8560 con 0x55b83e2f1e00 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e1d8200 con 0x55b83e2f2720 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e0bf6a0 con 0x55b83e2ec890 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10f9851b80 10 client.0 _ll_register_callbacks cb 0x55b83e1e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52680/0 (socket says 172.21.15.46:52680) 2024-05-08T23:04:33.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10dea00700 1 -- 172.21.15.46:0/3886847656 learned_addr learned my addr 172.21.15.46:0/3886847656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.473+0000 7f10df400700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 -- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 msgr2=0x55b83e2f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 -- 172.21.15.46:0/3886847656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 msgr2=0x55b83e2f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 -- 172.21.15.46:0/3886847656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc017070 con 0x55b83e2ec890 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10de000700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10dea00700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c2d0 tx=0x7f10d009a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6b7c8d2bd18bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d009ddd0 con 0x55b83e2ec890 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f10d00a2420 con 0x55b83e2ec890 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b83e195270 con 0x55b83e2ec890 2024-05-08T23:04:33.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10d00a1f90 con 0x55b83e2ec890 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10f9851b80 10 client.25879 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.474+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b83e069b60 con 0x55b83e2ec890 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b83e195270 con 0x55b83e2ec890 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10d00ca990 con 0x55b83e2ec890 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 10 client.25879 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 20 client.25879 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e3062a0 0x55b83e308690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 4 client.25879 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b83e2bb810 con 0x55b83e3062a0 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.475+0000 7f10de000700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e3062a0 0x55b83e308690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.476+0000 7f10de000700 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e3062a0 0x55b83e308690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbf30e26548ca95 server_cookie=bc30bff8d856b734 in_seq=0 out_seq=0 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.476+0000 7f10c8c00700 10 client.25879 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10c8c00700 1 -- 172.21.15.46:0/3886847656 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b83e2bb810 con 0x55b83e3062a0 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10c8c00700 10 client.25879 handle_command_reply: tid=0 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 client.25879 shutdown 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 msgr2=0x55b83e2f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c2d0 tx=0x7f10d009a4e0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 shutdown_connections 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2f5bd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e3062a0 0x55b83e308690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 --2- 172.21.15.46:0/3886847656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10c8c00700 0 client.25879 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 >> 172.21.15.46:0/3886847656 conn(0x55b83e24a210 msgr2=0x55b83e2e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 shutdown_connections 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 1 -- 172.21.15.46:0/3886847656 wait complete. 2024-05-08T23:04:33.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.477+0000 7f10f9851b80 20 client.25879 trim_cache size 0 max 16384 2024-05-08T23:04:33.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.491+0000 7f10f9851b80 1 Processor -- start 2024-05-08T23:04:33.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.491+0000 7f10f9851b80 1 -- start start 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e30d8c0 con 0x55b83e2f1e00 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e2f3bc0 con 0x55b83e2ec890 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10f9851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e2e8a00 con 0x55b83e2f2720 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46188/0 (socket says 172.21.15.46:46188) 2024-05-08T23:04:33.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 -- 172.21.15.46:0/2888804709 learned_addr learned my addr 172.21.15.46:0/2888804709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10dea00700 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10df400700 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 -- 172.21.15.46:0/2888804709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 msgr2=0x55b83e2e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 -- 172.21.15.46:0/2888804709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 msgr2=0x55b83e2ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.492+0000 7f10de000700 1 -- 172.21.15.46:0/2888804709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b83e202560 con 0x55b83e2f1e00 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10de000700 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0097a0 tx=0x7f10cc003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a20031cef1142e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10d5200700 1 -- 172.21.15.46:0/2888804709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc02b040 con 0x55b83e2f1e00 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10d5200700 1 -- 172.21.15.46:0/2888804709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10cc005660 con 0x55b83e2f1e00 2024-05-08T23:04:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10d5200700 1 -- 172.21.15.46:0/2888804709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc026930 con 0x55b83e2f1e00 2024-05-08T23:04:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10f9851b80 1 -- 172.21.15.46:0/2888804709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 msgr2=0x55b83e302370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.493+0000 7f10f9851b80 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0097a0 tx=0x7f10cc003a10 comp rx=0 tx=0).stop 2024-05-08T23:04:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 -- 172.21.15.46:0/2888804709 shutdown_connections 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2f2720 0x55b83e2e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2ec890 0x55b83e2ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 --2- 172.21.15.46:0/2888804709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e302370 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 -- 172.21.15.46:0/2888804709 >> 172.21.15.46:0/2888804709 conn(0x55b83e24a210 msgr2=0x55b83e2ecc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 -- 172.21.15.46:0/2888804709 shutdown_connections 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 -- 172.21.15.46:0/2888804709 wait complete. 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 -- start start 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.494+0000 7f10f9851b80 1 Processor -- start 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e35b790 con 0x55b83e2f1e00 2024-05-08T23:04:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e35b8d0 con 0x55b83e2f2720 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b83e35ba10 con 0x55b83e2ec890 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10f9851b80 10 client.0 _ll_register_callbacks cb 0x55b83e1e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52696/0 (socket says 172.21.15.46:52696) 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10dea00700 1 -- 172.21.15.46:0/1365065671 learned_addr learned my addr 172.21.15.46:0/1365065671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10df400700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 -- 172.21.15.46:0/1365065671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 msgr2=0x55b83e2fb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 -- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 msgr2=0x55b83e2fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.495+0000 7f10de000700 1 -- 172.21.15.46:0/1365065671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b83e0bd3b0 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10dea00700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10df400700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10de000700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2fb690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b83e2f6dc0 tx=0x7f10cc005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb01890e57d14e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc02b050 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f10cc0269c0 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10cc003e70 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10cc026b80 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10f9851b80 10 client.15873 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.496+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b83e0c1cd0 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10cc003e70 con 0x55b83e2f1e00 2024-05-08T23:04:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10cc03b070 con 0x55b83e2f1e00 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 20 client.15873 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e35cc80 0x55b83e35f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 4 client.15873 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b83e234ec0 con 0x55b83e35cc80 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.497+0000 7f10dea00700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e35cc80 0x55b83e35f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10dea00700 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e35cc80 0x55b83e35f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77e3ce7e3abc098 server_cookie=b25e554154e7ed96 in_seq=0 out_seq=0 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10c8c00700 10 client.15873 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10c8c00700 1 -- 172.21.15.46:0/1365065671 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b83e234ec0 con 0x55b83e35cc80 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10c8c00700 10 client.15873 handle_command_reply: tid=0 2024-05-08T23:04:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 client.15873 shutdown 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 msgr2=0x55b83e2fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2fb690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b83e2f6dc0 tx=0x7f10cc005c40 comp rx=0 tx=0).stop 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 shutdown_connections 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83e2f2720 0x55b83e2fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83e2ec890 0x55b83e2fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b83e35cc80 0x55b83e35f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 --2- 172.21.15.46:0/1365065671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83e2f1e00 0x55b83e2fb690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 >> 172.21.15.46:0/1365065671 conn(0x55b83e24a210 msgr2=0x55b83e2f4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10c8c00700 0 client.15873 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 shutdown_connections 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.498+0000 7f10f9851b80 1 -- 172.21.15.46:0/1365065671 wait complete. 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10f9851b80 20 client.15873 trim_cache size 0 max 16384 2024-05-08T23:04:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 msgr2=0x7f10c40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 0x7f10c40a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f10e00749d0 tx=0x7f10cc009260 comp rx=0 tx=0).stop 2024-05-08T23:04:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 msgr2=0x7f10e00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9a0 tx=0x7f10d000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 shutdown_connections 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10e0073d20 0x7f10e01a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e0101080 0x7f10e00732a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10c40a1430 0x7f10c40a38f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 --2- 172.21.15.46:0/2620060233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10e01061a0 0x7f10e00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 >> 172.21.15.46:0/2620060233 conn(0x7f10e00fc8e0 msgr2=0x7f10e00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 shutdown_connections 2024-05-08T23:04:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:33.499+0000 7f10e5e00700 1 -- 172.21.15.46:0/2620060233 wait complete. 2024-05-08T23:04:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:33 smithi086 ceph-mon[78610]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:33 smithi053 ceph-mon[77587]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:33 smithi046 ceph-mon[36898]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:34.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:04:34.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:04:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 -- 172.21.15.46:0/620521671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4100ec0 msgr2=0x7f22c41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 --2- 172.21.15.46:0/620521671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4100ec0 0x7f22c41012a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009990 tx=0x7f22b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 -- 172.21.15.46:0/620521671 shutdown_connections 2024-05-08T23:04:34.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 --2- 172.21.15.46:0/620521671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4105fe0 0x7f22c410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 --2- 172.21.15.46:0/620521671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4100ec0 0x7f22c41012a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 --2- 172.21.15.46:0/620521671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4101870 0x7f22c41058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 -- 172.21.15.46:0/620521671 >> 172.21.15.46:0/620521671 conn(0x7f22c40fc740 msgr2=0x7f22c40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 -- 172.21.15.46:0/620521671 shutdown_connections 2024-05-08T23:04:34.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.739+0000 7f22c9800700 1 -- 172.21.15.46:0/620521671 wait complete. 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.740+0000 7f22c9800700 1 Processor -- start 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 -- start start 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 0x7f22c40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4073d20 0x7f22c41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a5b40 con 0x7f22c4105fe0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a5c80 con 0x7f22c4100ec0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a5e40 con 0x7f22c4073d20 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46228/0 (socket says 172.21.15.46:46228) 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c3400700 1 -- 172.21.15.46:0/1360956513 learned_addr learned my addr 172.21.15.46:0/1360956513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c3e00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 0x7f22c40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.741+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4073d20 0x7f22c41a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 -- 172.21.15.46:0/1360956513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4073d20 msgr2=0x7f22c41a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4073d20 0x7f22c41a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 -- 172.21.15.46:0/1360956513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 msgr2=0x7f22c40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 0x7f22c40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 -- 172.21.15.46:0/1360956513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40095f0 con 0x7f22c4105fe0 2024-05-08T23:04:34.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3e00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 0x7f22c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c3400700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d950 tx=0x7f22b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c0c00700 1 -- 172.21.15.46:0/1360956513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004220 con 0x7f22c4105fe0 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c0c00700 1 -- 172.21.15.46:0/1360956513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b0007620 con 0x7f22c4105fe0 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c0c00700 1 -- 172.21.15.46:0/1360956513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00056c0 con 0x7f22c4105fe0 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c41a6120 con 0x7f22c4105fe0 2024-05-08T23:04:34.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.742+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c41a65e0 con 0x7f22c4105fe0 2024-05-08T23:04:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.746+0000 7f22c0c00700 1 -- 172.21.15.46:0/1360956513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f22b0004380 con 0x7f22c4105fe0 2024-05-08T23:04:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 Processor -- start 2024-05-08T23:04:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22c0c00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 0x7f22a80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 -- start start 2024-05-08T23:04:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22c0c00700 1 -- 172.21.15.46:0/1360956513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f22b00ccfd0 con 0x7f22c4105fe0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 0x7f22a80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 0x562f1893f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 0x562f189393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f18939d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f188901d0 con 0x562f1893f0d0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1888db90 con 0x562f18939910 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.748+0000 7f22de912b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18889480 con 0x562f1893f9f0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 0x562f1893f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 0x562f1893f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46242/0 (socket says 172.21.15.46:46242) 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3e00700 1 -- 172.21.15.46:0/3338071402 learned_addr learned my addr 172.21.15.46:0/3338071402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 0x562f189393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c8e00700 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f18939d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 0x7f22a80a3990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00f590 tx=0x7f22bc009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 -- 172.21.15.46:0/3338071402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 msgr2=0x562f18939d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f18939d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 -- 172.21.15.46:0/3338071402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 msgr2=0x562f1893f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 0x562f1893f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 -- 172.21.15.46:0/3338071402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f1870a430 con 0x562f1893f9f0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.749+0000 7f22c3400700 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 0x562f189393d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ca1f0 tx=0x7f22b0099570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35332fd612470fba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22b9600700 1 -- 172.21.15.46:0/3338071402 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0099ae0 con 0x562f1893f9f0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22b9600700 1 -- 172.21.15.46:0/3338071402 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b009d910 con 0x562f1893f9f0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22b9600700 1 -- 172.21.15.46:0/3338071402 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00a1d30 con 0x562f1893f9f0 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 -- 172.21.15.46:0/3338071402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 msgr2=0x562f189393d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 0x562f189393d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ca1f0 tx=0x7f22b0099570 comp rx=0 tx=0).stop 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 -- 172.21.15.46:0/3338071402 shutdown_connections 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f18939d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f0d0 0x562f1893f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 --2- 172.21.15.46:0/3338071402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f9f0 0x562f189393d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.750+0000 7f22de912b80 1 -- 172.21.15.46:0/3338071402 >> 172.21.15.46:0/3338071402 conn(0x562f18897290 msgr2=0x562f18934350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.751+0000 7f22de912b80 1 -- 172.21.15.46:0/3338071402 shutdown_connections 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.751+0000 7f22de912b80 1 -- 172.21.15.46:0/3338071402 wait complete. 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.751+0000 7f22de912b80 1 -- start start 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.751+0000 7f22de912b80 1 Processor -- start 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f18939910 0x562f18945940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f189433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f188255e0 con 0x562f1893f9f0 2024-05-08T23:04:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18825280 con 0x562f1893f0d0 2024-05-08T23:04:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1870c720 con 0x562f18939910 2024-05-08T23:04:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22de912b80 10 client.0 _ll_register_callbacks cb 0x562f1882ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f189433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f18939910 0x562f18945940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42064/0 (socket says 172.21.15.46:42064) 2024-05-08T23:04:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c3400700 1 -- 172.21.15.46:0/4046195171 learned_addr learned my addr 172.21.15.46:0/4046195171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 -- 172.21.15.46:0/4046195171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f18939910 msgr2=0x562f18945940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f18939910 0x562f18945940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 -- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 msgr2=0x562f18942cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c8e00700 1 -- 172.21.15.46:0/4046195171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c41a65e0 con 0x562f1893f9f0 2024-05-08T23:04:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.752+0000 7f22c3400700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:34.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22c8e00700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f189433c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f22bc01b000 tx=0x7f22bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b1240eaccfbad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22ae000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00e040 con 0x562f1893f9f0 2024-05-08T23:04:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f187e22f0 con 0x562f1893f9f0 2024-05-08T23:04:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f22bc019e30 con 0x562f1893f9f0 2024-05-08T23:04:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22bc004bb0 con 0x562f1893f9f0 2024-05-08T23:04:34.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22de912b80 10 client.15885 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:34.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.753+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562f186b6be0 con 0x562f1893f9f0 2024-05-08T23:04:34.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f187e22f0 con 0x562f1893f9f0 2024-05-08T23:04:34.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22bc004020 con 0x562f1893f9f0 2024-05-08T23:04:34.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:34.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:34.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 20 client.15885 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:34.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f18953390 0x562f189557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 4 client.15885 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:34.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f18908890 con 0x562f18953390 2024-05-08T23:04:34.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.754+0000 7f22c3e00700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f18953390 0x562f189557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.755+0000 7f22c3e00700 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f18953390 0x562f189557b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c1924703d269b1 server_cookie=1eb62d9899feda62 in_seq=0 out_seq=0 2024-05-08T23:04:34.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.755+0000 7f22ae000700 10 client.15885 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:34.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22ae000700 1 -- 172.21.15.46:0/4046195171 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f18908890 con 0x562f18953390 2024-05-08T23:04:34.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22ae000700 10 client.15885 handle_command_reply: tid=0 2024-05-08T23:04:34.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 client.15885 shutdown 2024-05-08T23:04:34.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 msgr2=0x562f189433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f189433c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f22bc01b000 tx=0x7f22bc004b40 comp rx=0 tx=0).stop 2024-05-08T23:04:34.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 shutdown_connections 2024-05-08T23:04:34.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f18953390 0x562f189557b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f189433c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f18939910 0x562f18945940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22ae000700 0 client.15885 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 --2- 172.21.15.46:0/4046195171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f1893f0d0 0x562f18942cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 >> 172.21.15.46:0/4046195171 conn(0x562f18897290 msgr2=0x562f18934130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 shutdown_connections 2024-05-08T23:04:34.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.756+0000 7f22de912b80 1 -- 172.21.15.46:0/4046195171 wait complete. 2024-05-08T23:04:34.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.757+0000 7f22de912b80 20 client.15885 trim_cache size 0 max 16384 2024-05-08T23:04:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 Processor -- start 2024-05-08T23:04:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 -- start start 2024-05-08T23:04:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f189432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18945470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1895a9c0 con 0x562f1893f9f0 2024-05-08T23:04:34.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18950aa0 con 0x562f18939910 2024-05-08T23:04:34.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22de912b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18935e70 con 0x562f1893f0d0 2024-05-08T23:04:34.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46264/0 (socket says 172.21.15.46:46264) 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22c8e00700 1 -- 172.21.15.46:0/3871053996 learned_addr learned my addr 172.21.15.46:0/3871053996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22c3e00700 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f189432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.775+0000 7f22c3400700 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18945470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 -- 172.21.15.46:0/3871053996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 msgr2=0x562f18945470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18945470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 -- 172.21.15.46:0/3871053996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 msgr2=0x562f189432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f189432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 -- 172.21.15.46:0/3871053996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f18942830 con 0x562f1893f9f0 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22c8e00700 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00fd60 tx=0x7f22bc00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee51b5afb39b19d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22ba000700 1 -- 172.21.15.46:0/3871053996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00e030 con 0x562f1893f9f0 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22ba000700 1 -- 172.21.15.46:0/3871053996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22bc00f040 con 0x562f1893f9f0 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.776+0000 7f22ba000700 1 -- 172.21.15.46:0/3871053996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc003a00 con 0x562f1893f9f0 2024-05-08T23:04:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 -- 172.21.15.46:0/3871053996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 msgr2=0x562f1894ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00fd60 tx=0x7f22bc00fee0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 -- 172.21.15.46:0/3871053996 shutdown_connections 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f1894ee60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f189432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 --2- 172.21.15.46:0/3871053996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18945470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 -- 172.21.15.46:0/3871053996 >> 172.21.15.46:0/3871053996 conn(0x562f18897290 msgr2=0x562f18932d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 -- 172.21.15.46:0/3871053996 shutdown_connections 2024-05-08T23:04:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.777+0000 7f22de912b80 1 -- 172.21.15.46:0/3871053996 wait complete. 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.778+0000 7f22de912b80 1 -- start start 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.778+0000 7f22de912b80 1 Processor -- start 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f1894a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18946980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18947400 con 0x562f1893f9f0 2024-05-08T23:04:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18947570 con 0x562f18939910 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f18947730 con 0x562f1893f0d0 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22de912b80 10 client.0 _ll_register_callbacks cb 0x562f1882ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46270/0 (socket says 172.21.15.46:46270) 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 -- 172.21.15.46:0/1554685078 learned_addr learned my addr 172.21.15.46:0/1554685078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c3e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f1894a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c3400700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18946980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 -- 172.21.15.46:0/1554685078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 msgr2=0x562f18946980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18946980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 -- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 msgr2=0x562f1894a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f1894a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.779+0000 7f22c8e00700 1 -- 172.21.15.46:0/1554685078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f186b6be0 con 0x562f1893f9f0 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22c8e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00f450 tx=0x7f22bc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feba53defadf76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22ae000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00e040 con 0x562f1893f9f0 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b009d910 con 0x562f1893f9f0 2024-05-08T23:04:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f22bc003a00 con 0x562f1893f9f0 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22bc004a90 con 0x562f1893f9f0 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22de912b80 10 client.15891 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562f1884f5e0 con 0x562f1893f9f0 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.780+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b009d910 con 0x562f1893f9f0 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22bc03e980 con 0x562f1893f9f0 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 20 client.15891 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f189aa8e0 0x562f189acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 4 client.15891 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562f18881f40 con 0x562f189aa8e0 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22c3e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f189aa8e0 0x562f189acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.781+0000 7f22c3e00700 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f189aa8e0 0x562f189acd00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877d6df0d9862f8a server_cookie=eefefd37914c5fd4 in_seq=0 out_seq=0 2024-05-08T23:04:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.782+0000 7f22ae000700 10 client.15891 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22ae000700 1 -- 172.21.15.46:0/1554685078 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562f18881f40 con 0x562f189aa8e0 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22ae000700 10 client.15891 handle_command_reply: tid=0 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 client.15891 shutdown 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 msgr2=0x562f18946ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00f450 tx=0x7f22bc0047d0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 shutdown_connections 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562f189aa8e0 0x562f189acd00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f1893f9f0 0x562f18946ec0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f18939910 0x562f1894a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 --2- 172.21.15.46:0/1554685078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f1893f0d0 0x562f18946980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 >> 172.21.15.46:0/1554685078 conn(0x562f18897290 msgr2=0x562f1894ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22ae000700 0 client.15891 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 shutdown_connections 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.783+0000 7f22de912b80 1 -- 172.21.15.46:0/1554685078 wait complete. 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22de912b80 20 client.15891 trim_cache size 0 max 16384 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:04:34.796 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "71d9cec0-3bf5-439a-bf0e-20e633af88dc", 2024-05-08T23:04:34.797 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:04:34.797 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 msgr2=0x7f22a80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 0x7f22a80a3990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00f590 tx=0x7f22bc009920 comp rx=0 tx=0).stop 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 msgr2=0x7f22c40737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d950 tx=0x7f22b000dc60 comp rx=0 tx=0).stop 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 shutdown_connections 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c4073d20 0x7f22c41a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22c4100ec0 0x7f22c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22a80a14d0 0x7f22a80a3990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 --2- 172.21.15.46:0/1360956513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22c4105fe0 0x7f22c40737e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.784+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 >> 172.21.15.46:0/1360956513 conn(0x7f22c40fc740 msgr2=0x7f22c40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.785+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 shutdown_connections 2024-05-08T23:04:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:34.785+0000 7f22c9800700 1 -- 172.21.15.46:0/1360956513 wait complete. 2024-05-08T23:04:34.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 -- 172.21.15.46:0/3055037659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3010b3c0 msgr2=0x7fbf3010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 --2- 172.21.15.46:0/3055037659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3010b3c0 0x7fbf3010d880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800b0a0 tx=0x7fbf2800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 -- 172.21.15.46:0/3055037659 shutdown_connections 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 --2- 172.21.15.46:0/3055037659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3010b3c0 0x7fbf3010d880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 --2- 172.21.15.46:0/3055037659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf30101fa0 0x7fbf30102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 --2- 172.21.15.46:0/3055037659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf301028c0 0x7fbf3010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 -- 172.21.15.46:0/3055037659 >> 172.21.15.46:0/3055037659 conn(0x7fbf300fb760 msgr2=0x7fbf300fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 -- 172.21.15.46:0/3055037659 shutdown_connections 2024-05-08T23:04:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.005+0000 7fbf35000700 1 -- 172.21.15.46:0/3055037659 wait complete. 2024-05-08T23:04:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.007+0000 7fbf35000700 1 Processor -- start 2024-05-08T23:04:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.007+0000 7fbf35000700 1 -- start start 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf301028c0 0x7fbf30100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3010b3c0 0x7fbf30100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301a5790 con 0x7fbf30101fa0 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301a5900 con 0x7fbf3010b3c0 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf35000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301a5ac0 con 0x7fbf301028c0 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46290/0 (socket says 172.21.15.46:46290) 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 -- 172.21.15.46:0/4112712649 learned_addr learned my addr 172.21.15.46:0/4112712649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2e800700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3010b3c0 0x7fbf30100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2d400700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf301028c0 0x7fbf30100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 -- 172.21.15.46:0/4112712649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf301028c0 msgr2=0x7fbf30100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf301028c0 0x7fbf30100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 -- 172.21.15.46:0/4112712649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3010b3c0 msgr2=0x7fbf30100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3010b3c0 0x7fbf30100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.008+0000 7fbf2de00700 1 -- 172.21.15.46:0/4112712649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c009520 con 0x7fbf30101fa0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf2de00700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000e9c0 tx=0x7fbf2000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf26a00700 1 -- 172.21.15.46:0/4112712649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200097b0 con 0x7fbf30101fa0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf28009d00 con 0x7fbf30101fa0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf26a00700 1 -- 172.21.15.46:0/4112712649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2000ee60 con 0x7fbf30101fa0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf26a00700 1 -- 172.21.15.46:0/4112712649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200054f0 con 0x7fbf30101fa0 2024-05-08T23:04:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.009+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf301a6120 con 0x7fbf30101fa0 2024-05-08T23:04:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.012+0000 7fbf26a00700 1 -- 172.21.15.46:0/4112712649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fbf20004750 con 0x7fbf30101fa0 2024-05-08T23:04:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.013+0000 7fbf26a00700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 0x7fbf140a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.013+0000 7fbf48b21b80 1 Processor -- start 2024-05-08T23:04:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.013+0000 7fbf26a00700 1 -- 172.21.15.46:0/4112712649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fbf20014070 con 0x7fbf30101fa0 2024-05-08T23:04:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf2d400700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 0x7fbf140a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 -- start start 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e854720 0x55630e84e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 0x55630e84ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e7a5150 con 0x55630e854720 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e7a2b10 con 0x55630e853e00 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e79e400 con 0x55630e84e890 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.014+0000 7fbf2d400700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 0x7fbf140a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf30101320 tx=0x7fbf1c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e854720 0x55630e84e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42124/0 (socket says 172.21.15.46:42124) 2024-05-08T23:04:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 -- 172.21.15.46:0/1230187662 learned_addr learned my addr 172.21.15.46:0/1230187662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2e800700 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 0x55630e84ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 -- 172.21.15.46:0/1230187662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 msgr2=0x55630e84ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 0x55630e84ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 -- 172.21.15.46:0/1230187662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e854720 msgr2=0x55630e84e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e854720 0x55630e84e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 -- 172.21.15.46:0/1230187662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55630e61f3b0 con 0x55630e853e00 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2e800700 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 0x55630e84ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf2de00700 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c5b0 tx=0x7fbf2009a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb9e1b20a3d6d8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.015+0000 7fbf1b400700 1 -- 172.21.15.46:0/1230187662 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2009dea0 con 0x55630e853e00 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf1b400700 1 -- 172.21.15.46:0/1230187662 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf200a24f0 con 0x55630e853e00 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf1b400700 1 -- 172.21.15.46:0/1230187662 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200cd3c0 con 0x55630e853e00 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 -- 172.21.15.46:0/1230187662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 msgr2=0x55630e8541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c5b0 tx=0x7fbf2009a7f0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 -- 172.21.15.46:0/1230187662 shutdown_connections 2024-05-08T23:04:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e84e890 0x55630e84ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e8541e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/1230187662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e854720 0x55630e84e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 -- 172.21.15.46:0/1230187662 >> 172.21.15.46:0/1230187662 conn(0x55630e7ac210 msgr2=0x55630e848df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 -- 172.21.15.46:0/1230187662 shutdown_connections 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.016+0000 7fbf48b21b80 1 -- 172.21.15.46:0/1230187662 wait complete. 2024-05-08T23:04:35.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.017+0000 7fbf48b21b80 1 -- start start 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.017+0000 7fbf48b21b80 1 Processor -- start 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e853e00 0x55630e858010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e73a560 con 0x55630e853e00 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e73a200 con 0x55630e84e890 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e6216a0 con 0x55630e854720 2024-05-08T23:04:35.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf48b21b80 10 client.0 _ll_register_callbacks cb 0x55630e743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42140/0 (socket says 172.21.15.46:42140) 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf2d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e853e00 0x55630e858010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.018+0000 7fbf2de00700 1 -- 172.21.15.46:0/2994034781 learned_addr learned my addr 172.21.15.46:0/2994034781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2e800700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 -- 172.21.15.46:0/2994034781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 msgr2=0x55630e858550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 -- 172.21.15.46:0/2994034781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e853e00 msgr2=0x55630e858010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e853e00 0x55630e858010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 -- 172.21.15.46:0/2994034781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf301a6120 con 0x55630e84e890 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2e800700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf200a1fc0 tx=0x7fbf2009a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75bd62ebf25f1d3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2009dea0 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.019+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55630e6f7270 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fbf20065030 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf200b82e0 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 10 client.25883 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55630e5cbb60 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55630e6f7270 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf20099560 con 0x55630e84e890 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 10 client.25883 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 20 client.25883 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e867e60 0x55630e86a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.020+0000 7fbf48b21b80 4 client.25883 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.021+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55630e81d810 con 0x55630e867e60 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.021+0000 7fbf2d400700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e867e60 0x55630e86a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.021+0000 7fbf2d400700 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e867e60 0x55630e86a280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b283a0d2f6e081 server_cookie=837ad5e9dcfaa467 in_seq=0 out_seq=0 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.021+0000 7fbf0fe00700 10 client.25883 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.022+0000 7fbf0fe00700 1 -- 172.21.15.46:0/2994034781 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55630e81d810 con 0x55630e867e60 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.022+0000 7fbf0fe00700 10 client.25883 handle_command_reply: tid=0 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.022+0000 7fbf48b21b80 1 client.25883 shutdown 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 msgr2=0x55630e857ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf200a1fc0 tx=0x7fbf2009a1f0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 shutdown_connections 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e84e890 0x55630e857ad0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e867e60 0x55630e86a280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2994034781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e853e00 0x55630e858010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf0fe00700 0 client.25883 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 >> 172.21.15.46:0/2994034781 conn(0x55630e7ac210 msgr2=0x55630e8485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 shutdown_connections 2024-05-08T23:04:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2994034781 wait complete. 2024-05-08T23:04:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.023+0000 7fbf48b21b80 20 client.25883 trim_cache size 0 max 16384 2024-05-08T23:04:35.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.041+0000 7fbf48b21b80 1 Processor -- start 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.041+0000 7fbf48b21b80 1 -- start start 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e848d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e845e00 con 0x55630e84e890 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e84ae50 con 0x55630e853e00 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e84a670 con 0x55630e854720 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46314/0 (socket says 172.21.15.46:46314) 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 -- 172.21.15.46:0/2348970845 learned_addr learned my addr 172.21.15.46:0/2348970845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2d400700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e848d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2e800700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 -- 172.21.15.46:0/2348970845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 msgr2=0x55630e858020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 -- 172.21.15.46:0/2348970845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 msgr2=0x55630e848d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e848d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2de00700 1 -- 172.21.15.46:0/2348970845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55630e764560 con 0x55630e84e890 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2e800700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.042+0000 7fbf2d400700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e848d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf2de00700 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c5b0 tx=0x7fbf20098620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8f798b78e5c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf1be00700 1 -- 172.21.15.46:0/2348970845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2009a2d0 con 0x55630e84e890 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf1be00700 1 -- 172.21.15.46:0/2348970845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2009dea0 con 0x55630e84e890 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf1be00700 1 -- 172.21.15.46:0/2348970845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20099560 con 0x55630e84e890 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2348970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 msgr2=0x55630e863c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.043+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c5b0 tx=0x7fbf20098620 comp rx=0 tx=0).stop 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2348970845 shutdown_connections 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e854720 0x55630e858020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e863c20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/2348970845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e853e00 0x55630e848d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2348970845 >> 172.21.15.46:0/2348970845 conn(0x55630e7ac210 msgr2=0x55630e84ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2348970845 shutdown_connections 2024-05-08T23:04:35.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 -- 172.21.15.46:0/2348970845 wait complete. 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 -- start start 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.044+0000 7fbf48b21b80 1 Processor -- start 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e853e00 0x55630e85d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 0x55630e85d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e85dd90 con 0x55630e84e890 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e85df00 con 0x55630e854720 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630e8bd360 con 0x55630e853e00 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf48b21b80 10 client.0 _ll_register_callbacks cb 0x55630e743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 0x55630e85d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46324/0 (socket says 172.21.15.46:46324) 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 -- 172.21.15.46:0/3927499833 learned_addr learned my addr 172.21.15.46:0/3927499833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2d400700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e853e00 0x55630e85d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 -- 172.21.15.46:0/3927499833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e853e00 msgr2=0x55630e85d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e853e00 0x55630e85d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 -- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 msgr2=0x55630e85d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 0x55630e85d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2de00700 1 -- 172.21.15.46:0/3927499833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c017070 con 0x55630e84e890 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.045+0000 7fbf2e800700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 0x55630e85d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf2de00700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf200058b0 tx=0x7fbf20004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f836b307bd7818e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200a2330 con 0x55630e84e890 2024-05-08T23:04:35.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fbf20065040 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf200a24f0 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf2009a2d0 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf48b21b80 10 client.15909 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55630e5cbb60 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.046+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf200a24f0 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf200cd160 con 0x55630e84e890 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 20 client.15909 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e8be5a0 0x55630e8c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 4 client.15909 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:04:35.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55630e796ec0 con 0x55630e8be5a0 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.047+0000 7fbf2d400700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e8be5a0 0x55630e8c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf2d400700 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e8be5a0 0x55630e8c0990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2943aee0900e1c05 server_cookie=6a9e5d7ddba03608 in_seq=0 out_seq=0 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf0fe00700 10 client.15909 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf0fe00700 1 -- 172.21.15.46:0/3927499833 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55630e796ec0 con 0x55630e8be5a0 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf0fe00700 10 client.15909 handle_command_reply: tid=0 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 client.15909 shutdown 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 msgr2=0x55630e85f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf200058b0 tx=0x7fbf20004f60 comp rx=0 tx=0).stop 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 shutdown_connections 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55630e854720 0x55630e85d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55630e8be5a0 0x55630e8c0990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55630e84e890 0x55630e85f520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 --2- 172.21.15.46:0/3927499833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630e853e00 0x55630e85d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf0fe00700 0 client.15909 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 >> 172.21.15.46:0/3927499833 conn(0x55630e7ac210 msgr2=0x55630e856360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 shutdown_connections 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 1 -- 172.21.15.46:0/3927499833 wait complete. 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.048+0000 7fbf48b21b80 20 client.15909 trim_cache size 0 max 16384 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:04:35.053 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 msgr2=0x7fbf140a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 0x7fbf140a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf30101320 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 msgr2=0x7fbf300ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000e9c0 tx=0x7fbf2000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 shutdown_connections 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3010b3c0 0x7fbf30100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf140a13b0 0x7fbf140a3870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf30101fa0 0x7fbf300ffbf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 --2- 172.21.15.46:0/4112712649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf301028c0 0x7fbf30100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 >> 172.21.15.46:0/4112712649 conn(0x7fbf300fb760 msgr2=0x7fbf300fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 shutdown_connections 2024-05-08T23:04:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.049+0000 7fbf35000700 1 -- 172.21.15.46:0/4112712649 wait complete. 2024-05-08T23:04:35.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:04:35.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:04:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 -- 172.21.15.46:0/1570159344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 msgr2=0x7f226410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 --2- 172.21.15.46:0/1570159344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f226410dd10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f225c009f90 tx=0x7f225c00b330 comp rx=0 tx=0).stop 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 -- 172.21.15.46:0/1570159344 shutdown_connections 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 --2- 172.21.15.46:0/1570159344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f226410dd10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 --2- 172.21.15.46:0/1570159344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2264101070 0x7f2264101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 --2- 172.21.15.46:0/1570159344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101a20 0x7f2264105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 -- 172.21.15.46:0/1570159344 >> 172.21.15.46:0/1570159344 conn(0x7f22640fc930 msgr2=0x7f22640fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 -- 172.21.15.46:0/1570159344 shutdown_connections 2024-05-08T23:04:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.267+0000 7f226ba00700 1 -- 172.21.15.46:0/1570159344 wait complete. 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 Processor -- start 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 -- start start 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 0x7f2264199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f2264199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 0x7f226419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226419e980 con 0x7f2264101070 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226419eaf0 con 0x7f226419df00 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.268+0000 7f226ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226419ecb0 con 0x7f2264106190 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 0x7f2264199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 0x7f2264199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46338/0 (socket says 172.21.15.46:46338) 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2263e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f2264199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269000700 1 -- 172.21.15.46:0/3097074082 learned_addr learned my addr 172.21.15.46:0/3097074082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 0x7f226419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 -- 172.21.15.46:0/3097074082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 msgr2=0x7f2264199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f2264199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 -- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 msgr2=0x7f2264199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 0x7f2264199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 -- 172.21.15.46:0/3097074082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c009c40 con 0x7f226419df00 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.269+0000 7f2269a00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 0x7f226419e380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b6c0 tx=0x7f225c008a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.270+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00e040 con 0x7f226419df00 2024-05-08T23:04:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.270+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f225c00be50 con 0x7f226419df00 2024-05-08T23:04:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.270+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c005040 con 0x7f226419df00 2024-05-08T23:04:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.270+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22641a61b0 con 0x7f226419df00 2024-05-08T23:04:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.270+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22641a6630 con 0x7f226419df00 2024-05-08T23:04:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.271+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2264102540 con 0x7f226419df00 2024-05-08T23:04:35.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.273+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f225c01d6e0 con 0x7f226419df00 2024-05-08T23:04:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.274+0000 7f2261600700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 0x7f224c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.274+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f225c0ce940 con 0x7f226419df00 2024-05-08T23:04:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.274+0000 7f2269000700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 0x7f224c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.275+0000 7f2269000700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 0x7f224c0a38f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2258009d30 tx=0x7f2258009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.277+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f225c0977c0 con 0x7f226419df00 2024-05-08T23:04:35.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.542+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2264067d10 con 0x7f226419df00 2024-05-08T23:04:35.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.543+0000 7f2261600700 1 -- 172.21.15.46:0/3097074082 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f225c09ade0 con 0x7f226419df00 2024-05-08T23:04:35.542 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:04:35.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:03:50.080154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:04:35.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 msgr2=0x7f224c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 0x7f224c0a38f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2258009d30 tx=0x7f2258009310 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 msgr2=0x7f226419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 0x7f226419e380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b6c0 tx=0x7f225c008a10 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 shutdown_connections 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f226419df00 0x7f226419e380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f224c0a1430 0x7f224c0a38f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2264101070 0x7f2264199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 --2- 172.21.15.46:0/3097074082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2264106190 0x7f2264199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 >> 172.21.15.46:0/3097074082 conn(0x7f22640fc930 msgr2=0x7f22640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 shutdown_connections 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.545+0000 7f226ba00700 1 -- 172.21.15.46:0/3097074082 wait complete. 2024-05-08T23:04:35.548 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-05-08T23:04:35.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 -- 172.21.15.46:0/3660006824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101a30 msgr2=0x7f7794105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 --2- 172.21.15.46:0/3660006824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101a30 0x7f7794105a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7780009990 tx=0x7f7780009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 -- 172.21.15.46:0/3660006824 shutdown_connections 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 --2- 172.21.15.46:0/3660006824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77941061a0 0x7f779410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 --2- 172.21.15.46:0/3660006824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794101080 0x7f7794101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 --2- 172.21.15.46:0/3660006824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101a30 0x7f7794105a70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.767+0000 7f779a400700 1 -- 172.21.15.46:0/3660006824 >> 172.21.15.46:0/3660006824 conn(0x7f77940fc8e0 msgr2=0x7f77940fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.768+0000 7f779a400700 1 -- 172.21.15.46:0/3660006824 shutdown_connections 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.768+0000 7f779a400700 1 -- 172.21.15.46:0/3660006824 wait complete. 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 Processor -- start 2024-05-08T23:04:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 -- start start 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77941061a0 0x7f7794199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779419de20 0x7f779419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779419e8a0 con 0x7f7794101080 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779419ea10 con 0x7f77941061a0 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f779a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779419ebd0 con 0x7f779419de20 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f7793400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f7793400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46348/0 (socket says 172.21.15.46:46348) 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.769+0000 7f7793400700 1 -- 172.21.15.46:0/4141394403 learned_addr learned my addr 172.21.15.46:0/4141394403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7792a00700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77941061a0 0x7f7794199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793e00700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779419de20 0x7f779419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 -- 172.21.15.46:0/4141394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779419de20 msgr2=0x7f779419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779419de20 0x7f779419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 -- 172.21.15.46:0/4141394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77941061a0 msgr2=0x7f7794199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77941061a0 0x7f7794199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 -- 172.21.15.46:0/4141394403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77800095f0 con 0x7f7794101080 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f7793400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9a0 tx=0x7f778400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f778be00700 1 -- 172.21.15.46:0/4141394403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784009790 con 0x7f7794101080 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f778be00700 1 -- 172.21.15.46:0/4141394403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778400cbd0 con 0x7f7794101080 2024-05-08T23:04:35.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f778be00700 1 -- 172.21.15.46:0/4141394403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784018680 con 0x7f7794101080 2024-05-08T23:04:35.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.770+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77941a6130 con 0x7f7794101080 2024-05-08T23:04:35.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.771+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a65c0 con 0x7f7794101080 2024-05-08T23:04:35.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.774+0000 7f778be00700 1 -- 172.21.15.46:0/4141394403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f778400cd40 con 0x7f7794101080 2024-05-08T23:04:35.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 Processor -- start 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 -- start start 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f778be00700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 0x7f77780a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f778be00700 1 -- 172.21.15.46:0/4141394403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7784014070 con 0x7f7794101080 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07bc890 0x55fac07bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac0713150 con 0x55fac07bc890 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac0710b10 con 0x55fac07c1e00 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.775+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac070c400 con 0x55fac07c2720 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42198/0 (socket says 172.21.15.46:42198) 2024-05-08T23:04:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 -- 172.21.15.46:0/454276008 learned_addr learned my addr 172.21.15.46:0/454276008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793e00700 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07bc890 0x55fac07bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7792a00700 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 -- 172.21.15.46:0/454276008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 msgr2=0x55fac07bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 -- 172.21.15.46:0/454276008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07bc890 msgr2=0x55fac07bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793e00700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 0x7f77780a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07bc890 0x55fac07bccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 -- 172.21.15.46:0/454276008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fac058d3b0 con 0x55fac07c1e00 2024-05-08T23:04:35.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7793400700 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7784099fa0 tx=0x7f77840999f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75eaa82d2a0ca17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.776+0000 7f7788c00700 1 -- 172.21.15.46:0/454276008 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778409de40 con 0x55fac07c1e00 2024-05-08T23:04:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f7788c00700 1 -- 172.21.15.46:0/454276008 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77840a2490 con 0x55fac07c1e00 2024-05-08T23:04:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f7788c00700 1 -- 172.21.15.46:0/454276008 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840a2000 con 0x55fac07c1e00 2024-05-08T23:04:35.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 -- 172.21.15.46:0/454276008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 msgr2=0x55fac07c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f7793e00700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 0x7f77780a38f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f778c009fd0 tx=0x7f778c00b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7784099fa0 tx=0x7f77840999f0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 -- 172.21.15.46:0/454276008 shutdown_connections 2024-05-08T23:04:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07bc890 0x55fac07bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07c21e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/454276008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 -- 172.21.15.46:0/454276008 >> 172.21.15.46:0/454276008 conn(0x55fac071a210 msgr2=0x55fac07b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.777+0000 7f77adeb3b80 1 -- 172.21.15.46:0/454276008 shutdown_connections 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.778+0000 7f77adeb3b80 1 -- 172.21.15.46:0/454276008 wait complete. 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.778+0000 7f77adeb3b80 1 -- start start 2024-05-08T23:04:35.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.778+0000 7f77adeb3b80 1 Processor -- start 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c2720 0x55fac07c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac06a8560 con 0x55fac07c1e00 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac06a8200 con 0x55fac07c2720 2024-05-08T23:04:35.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac058f6a0 con 0x55fac07bc890 2024-05-08T23:04:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f77adeb3b80 10 client.0 _ll_register_callbacks cb 0x55fac06b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7792a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52830/0 (socket says 172.21.15.46:52830) 2024-05-08T23:04:35.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 -- 172.21.15.46:0/3987193292 learned_addr learned my addr 172.21.15.46:0/3987193292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793e00700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c2720 0x55fac07c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 -- 172.21.15.46:0/3987193292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c2720 msgr2=0x55fac07c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c2720 0x55fac07c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 -- 172.21.15.46:0/3987193292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 msgr2=0x55fac07c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.779+0000 7f7793400700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f7793400700 1 -- 172.21.15.46:0/3987193292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a65c0 con 0x55fac07bc890 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f7793400700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7784009230 tx=0x7f778400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f86ddf180baa62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f777d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778409de40 con 0x55fac07bc890 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fac0665270 con 0x55fac07bc890 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f77840a2210 con 0x55fac07bc890 2024-05-08T23:04:35.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77840b8270 con 0x55fac07bc890 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f77adeb3b80 10 client.25939 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.780+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fac0539b60 con 0x55fac07bc890 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fac0665270 con 0x55fac07bc890 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77840caa00 con 0x55fac07bc890 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:35.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 10 client.25939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 20 client.25939 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac07d5e30 0x55fac07d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 4 client.25939 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fac078b810 con 0x55fac07d5e30 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.781+0000 7f7792a00700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac07d5e30 0x55fac07d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.782+0000 7f7792a00700 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac07d5e30 0x55fac07d8220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef67e796df8efe5b server_cookie=2a86beb33e8967a9 in_seq=0 out_seq=0 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.782+0000 7f777d600700 10 client.25939 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.782+0000 7f777d600700 1 -- 172.21.15.46:0/3987193292 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fac078b810 con 0x55fac07d5e30 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f777d600700 10 client.25939 handle_command_reply: tid=0 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 client.25939 shutdown 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 msgr2=0x55fac07c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7784009230 tx=0x7f778400ef60 comp rx=0 tx=0).stop 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 shutdown_connections 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac07d5e30 0x55fac07d8220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c2720 0x55fac07c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07c5be0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f777d600700 0 client.25939 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:35.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/3987193292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 >> 172.21.15.46:0/3987193292 conn(0x55fac071a210 msgr2=0x55fac07b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 shutdown_connections 2024-05-08T23:04:35.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 1 -- 172.21.15.46:0/3987193292 wait complete. 2024-05-08T23:04:35.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.783+0000 7f77adeb3b80 20 client.25939 trim_cache size 0 max 16384 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 Processor -- start 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 -- start start 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 0x55fac07b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07b1890 con 0x55fac07c2720 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07b8810 con 0x55fac07c1e00 2024-05-08T23:04:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07c4160 con 0x55fac07bc890 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7793e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 0x55fac07b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7792a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7793400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7793e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 0x55fac07b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46364/0 (socket says 172.21.15.46:46364) 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7792a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42204/0 (socket says 172.21.15.46:42204) 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.798+0000 7f7793e00700 1 -- 172.21.15.46:0/1645612318 learned_addr learned my addr 172.21.15.46:0/1645612318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 -- 172.21.15.46:0/1645612318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 msgr2=0x55fac07bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 -- 172.21.15.46:0/1645612318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 msgr2=0x55fac07b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 0x55fac07b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 -- 172.21.15.46:0/1645612318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fac06d2560 con 0x55fac07c1e00 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7792a00700 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77800098e0 tx=0x7f778000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c0fff542c70198 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7789600700 1 -- 172.21.15.46:0/1645612318 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778001d040 con 0x55fac07c1e00 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7789600700 1 -- 172.21.15.46:0/1645612318 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778001e040 con 0x55fac07c1e00 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.799+0000 7f7789600700 1 -- 172.21.15.46:0/1645612318 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000b950 con 0x55fac07c1e00 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 -- 172.21.15.46:0/1645612318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 msgr2=0x55fac07d8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77800098e0 tx=0x7f778000b260 comp rx=0 tx=0).stop 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 -- 172.21.15.46:0/1645612318 shutdown_connections 2024-05-08T23:04:35.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c2720 0x55fac07b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07bc890 0x55fac07bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/1645612318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07c1e00 0x55fac07d8220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 -- 172.21.15.46:0/1645612318 >> 172.21.15.46:0/1645612318 conn(0x55fac071a210 msgr2=0x55fac07bcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 -- 172.21.15.46:0/1645612318 shutdown_connections 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.800+0000 7f77adeb3b80 1 -- 172.21.15.46:0/1645612318 wait complete. 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 -- start start 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 Processor -- start 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 0x55fac07cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07cb230 con 0x55fac07c1e00 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07cb3a0 con 0x55fac07bc890 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac07cb560 con 0x55fac07c2720 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.801+0000 7f77adeb3b80 10 client.0 _ll_register_callbacks cb 0x55fac06b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7793400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 0x55fac07cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7793e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46370/0 (socket says 172.21.15.46:46370) 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7793400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 0x55fac07cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42212/0 (socket says 172.21.15.46:42212) 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 -- 172.21.15.46:0/894267275 learned_addr learned my addr 172.21.15.46:0/894267275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 -- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 msgr2=0x55fac07caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 -- 172.21.15.46:0/894267275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 msgr2=0x55fac07cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 0x55fac07cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 -- 172.21.15.46:0/894267275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779410e590 con 0x55fac07c1e00 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f7792a00700 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7780009990 tx=0x7f7780005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2a33cca6b86405 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.802+0000 7f777d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778001d040 con 0x55fac07c1e00 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7780029350 con 0x55fac07c1e00 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77840a2490 con 0x55fac07c1e00 2024-05-08T23:04:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7780005440 con 0x55fac07c1e00 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 10 client.15933 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fac0591cd0 con 0x55fac07c1e00 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77840a2490 con 0x55fac07c1e00 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f778003b470 con 0x55fac07c1e00 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 10 client.15933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:04:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 20 client.15933 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac082c400 0x55fac082e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 4 client.15933 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.803+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fac0704ec0 con 0x55fac082c400 2024-05-08T23:04:35.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.804+0000 7f7793400700 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac082c400 0x55fac082e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.804+0000 7f7793400700 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac082c400 0x55fac082e820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8031c80a87c38193 server_cookie=726abacdb072e59 in_seq=0 out_seq=0 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.804+0000 7f777d600700 10 client.15933 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f777d600700 1 -- 172.21.15.46:0/894267275 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fac0704ec0 con 0x55fac082c400 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f777d600700 10 client.15933 handle_command_reply: tid=0 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 client.15933 shutdown 2024-05-08T23:04:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 msgr2=0x55fac07ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7780009990 tx=0x7f7780005a20 comp rx=0 tx=0).stop 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 shutdown_connections 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55fac082c400 0x55fac082e820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fac07c2720 0x55fac07caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac07bc890 0x55fac07cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 --2- 172.21.15.46:0/894267275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fac07c1e00 0x55fac07ca760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f777d600700 0 client.15933 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:04:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 >> 172.21.15.46:0/894267275 conn(0x55fac071a210 msgr2=0x55fac07c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 shutdown_connections 2024-05-08T23:04:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 1 -- 172.21.15.46:0/894267275 wait complete. 2024-05-08T23:04:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.805+0000 7f77adeb3b80 20 client.15933 trim_cache size 0 max 16384 2024-05-08T23:04:35.822 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:35.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 msgr2=0x7f77780a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 0x7f77780a38f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f778c009fd0 tx=0x7f778c00b040 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 msgr2=0x7f7794199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9a0 tx=0x7f778400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 shutdown_connections 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779419de20 0x7f779419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f77780a1430 0x7f77780a38f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794101080 0x7f7794199540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 --2- 172.21.15.46:0/4141394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77941061a0 0x7f7794199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 >> 172.21.15.46:0/4141394403 conn(0x7f77940fc8e0 msgr2=0x7f77940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 shutdown_connections 2024-05-08T23:04:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:35.806+0000 7f779a400700 1 -- 172.21.15.46:0/4141394403 wait complete. 2024-05-08T23:04:35.825 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:35.825 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:04:35.825 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:35.825 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:04:35.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:35.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:04:35.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:35 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3097074082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:35.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:04:36.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 -- 172.21.15.46:0/357106065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101a10 msgr2=0x7fca90105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 --2- 172.21.15.46:0/357106065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101a10 0x7fca90105a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0098f0 tx=0x7fca7c009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 -- 172.21.15.46:0/357106065 shutdown_connections 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 --2- 172.21.15.46:0/357106065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca9010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 --2- 172.21.15.46:0/357106065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90101060 0x7fca90101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 --2- 172.21.15.46:0/357106065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101a10 0x7fca90105a50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 -- 172.21.15.46:0/357106065 >> 172.21.15.46:0/357106065 conn(0x7fca900fc900 msgr2=0x7fca900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 -- 172.21.15.46:0/357106065 shutdown_connections 2024-05-08T23:04:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.032+0000 7fca96a00700 1 -- 172.21.15.46:0/357106065 wait complete. 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.033+0000 7fca96a00700 1 Processor -- start 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.033+0000 7fca96a00700 1 -- start start 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca900737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90073d20 0x7fca901a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901a5b00 con 0x7fca90073d20 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901a5c40 con 0x7fca90101060 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca96a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901a5e00 con 0x7fca90106180 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca94a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90073d20 0x7fca901a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42242/0 (socket says 172.21.15.46:42242) 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 -- 172.21.15.46:0/2253449834 learned_addr learned my addr 172.21.15.46:0/2253449834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8f400700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca900737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 -- 172.21.15.46:0/2253449834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 msgr2=0x7fca900737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca900737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 -- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90073d20 msgr2=0x7fca901a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90073d20 0x7fca901a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8fe00700 1 -- 172.21.15.46:0/2253449834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800095f0 con 0x7fca90101060 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.034+0000 7fca8f400700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca900737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca8fe00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca8000ec10 tx=0x7fca8000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca8cc00700 1 -- 172.21.15.46:0/2253449834 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000cc90 con 0x7fca90101060 2024-05-08T23:04:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c009520 con 0x7fca90101060 2024-05-08T23:04:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca8cc00700 1 -- 172.21.15.46:0/2253449834 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca800044e0 con 0x7fca90101060 2024-05-08T23:04:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca8cc00700 1 -- 172.21.15.46:0/2253449834 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80010430 con 0x7fca90101060 2024-05-08T23:04:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.035+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca901a6480 con 0x7fca90101060 2024-05-08T23:04:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.038+0000 7fca8cc00700 1 -- 172.21.15.46:0/2253449834 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fca80003fb0 con 0x7fca90101060 2024-05-08T23:04:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.039+0000 7fcaaa47cb80 1 Processor -- start 2024-05-08T23:04:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 -- start start 2024-05-08T23:04:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a053720 0x55c94a04d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca8cc00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 0x7fca740a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a04d890 0x55c94a04dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949fa4150 con 0x55c94a053720 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949fa1b10 con 0x55c94a04d890 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949f9d400 con 0x55c94a052e00 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca8cc00700 1 -- 172.21.15.46:0/2253449834 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca80014070 con 0x7fca90101060 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca94a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca94a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52880/0 (socket says 172.21.15.46:52880) 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca94a00700 1 -- 172.21.15.46:0/812043403 learned_addr learned my addr 172.21.15.46:0/812043403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca94a00700 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a04d890 0x55c94a04dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca8fe00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 0x7fca740a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.040+0000 7fca8f400700 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a053720 0x55c94a04d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 -- 172.21.15.46:0/812043403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a04d890 msgr2=0x55c94a04dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a04d890 0x55c94a04dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 -- 172.21.15.46:0/812043403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a053720 msgr2=0x55c94a04d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a053720 0x55c94a04d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 -- 172.21.15.46:0/812043403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c949e1e3b0 con 0x55c94a052e00 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca8fe00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 0x7fca740a3870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca800cab50 tx=0x7fca800bb400 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca94a00700 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca88008cd0 tx=0x7fca88009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237b03c4b47d4bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca85200700 1 -- 172.21.15.46:0/812043403 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800e050 con 0x55c94a052e00 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca85200700 1 -- 172.21.15.46:0/812043403 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8800f040 con 0x55c94a052e00 2024-05-08T23:04:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.041+0000 7fca85200700 1 -- 172.21.15.46:0/812043403 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88004280 con 0x55c94a052e00 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/812043403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 msgr2=0x55c94a0531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca88008cd0 tx=0x7fca88009920 comp rx=0 tx=0).stop 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/812043403 shutdown_connections 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a04d890 0x55c94a04dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a0531e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/812043403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a053720 0x55c94a04d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/812043403 >> 172.21.15.46:0/812043403 conn(0x55c949fab210 msgr2=0x55c94a047df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/812043403 shutdown_connections 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.042+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/812043403 wait complete. 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.043+0000 7fcaaa47cb80 1 -- start start 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.043+0000 7fcaaa47cb80 1 Processor -- start 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a056ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a057540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949f39560 con 0x55c94a04d890 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949f39200 con 0x55c94a053720 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c949e206a0 con 0x55c94a052e00 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fcaaa47cb80 10 client.0 _ll_register_callbacks cb 0x55c949f42b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fca8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fca8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52884/0 (socket says 172.21.15.46:52884) 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fca8f400700 1 -- 172.21.15.46:0/2871823491 learned_addr learned my addr 172.21.15.46:0/2871823491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.044+0000 7fca94a00700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a056ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca94a00700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a057540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 -- 172.21.15.46:0/2871823491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 msgr2=0x55c94a057540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a057540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 -- 172.21.15.46:0/2871823491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 msgr2=0x55c94a056ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a056ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 -- 172.21.15.46:0/2871823491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880089b0 con 0x55c94a052e00 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca94a00700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a056ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca8f400700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca9006a480 tx=0x7fca7c010fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685ae4762d301f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.045+0000 7fca79600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c01c040 con 0x55c94a052e00 2024-05-08T23:04:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c949ef6270 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca7c005040 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c00bbe0 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 10 client.25957 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c949dcab60 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c949ef6270 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca7c015020 con 0x55c94a052e00 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 10 client.25957 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 10 client.25957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.046+0000 7fcaaa47cb80 20 client.25957 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0671d0 0x55c94a0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fcaaa47cb80 4 client.25957 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c94a01c810 con 0x55c94a0671d0 2024-05-08T23:04:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fca94a00700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0671d0 0x55c94a0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fca94a00700 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0671d0 0x55c94a0695f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16cc9105c55856c6 server_cookie=d6f22191892e9587 in_seq=0 out_seq=0 2024-05-08T23:04:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.047+0000 7fca79600700 10 client.25957 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fca79600700 1 -- 172.21.15.46:0/2871823491 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c94a01c810 con 0x55c94a0671d0 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fca79600700 10 client.25957 handle_command_reply: tid=0 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 client.25957 shutdown 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 msgr2=0x55c94a057000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca9006a480 tx=0x7fca7c010fa0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 shutdown_connections 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0671d0 0x55c94a0695f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a057540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a056ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2871823491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a057000 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fca79600700 0 client.25957 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.048+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 >> 172.21.15.46:0/2871823491 conn(0x55c949fab210 msgr2=0x55c94a0475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.049+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 shutdown_connections 2024-05-08T23:04:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.049+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2871823491 wait complete. 2024-05-08T23:04:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.049+0000 7fcaaa47cb80 20 client.25957 trim_cache size 0 max 16384 2024-05-08T23:04:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 Processor -- start 2024-05-08T23:04:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 -- start start 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a062c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a047d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a063d80 con 0x55c94a04d890 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a058130 con 0x55c94a053720 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a06e7e0 con 0x55c94a052e00 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fca94a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fca94a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46412/0 (socket says 172.21.15.46:46412) 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fca94a00700 1 -- 172.21.15.46:0/2013274817 learned_addr learned my addr 172.21.15.46:0/2013274817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.063+0000 7fca8f400700 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a062c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a047d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 -- 172.21.15.46:0/2013274817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 msgr2=0x55c94a062c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a062c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 -- 172.21.15.46:0/2013274817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 msgr2=0x55c94a047d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a047d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 -- 172.21.15.46:0/2013274817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c949f63560 con 0x55c94a04d890 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca94a00700 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca88008ca0 tx=0x7fca880093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95bdc2036356048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca85c00700 1 -- 172.21.15.46:0/2013274817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8802f070 con 0x55c94a04d890 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca85c00700 1 -- 172.21.15.46:0/2013274817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca88018880 con 0x55c94a04d890 2024-05-08T23:04:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.064+0000 7fca85c00700 1 -- 172.21.15.46:0/2013274817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800e070 con 0x55c94a04d890 2024-05-08T23:04:36.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2013274817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 msgr2=0x55c94a04d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca88008ca0 tx=0x7fca880093f0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2013274817 shutdown_connections 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a047d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a04d890 0x55c94a04d350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/2013274817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a052e00 0x55c94a062c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2013274817 >> 172.21.15.46:0/2013274817 conn(0x55c949fab210 msgr2=0x55c94a04dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2013274817 shutdown_connections 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.065+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/2013274817 wait complete. 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 -- start start 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 Processor -- start 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 0x55c94a05c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a05cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a05ea60 con 0x55c94a052e00 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a0bc6b0 con 0x55c94a053720 2024-05-08T23:04:36.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94a0bc7f0 con 0x55c94a04d890 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fcaaa47cb80 10 client.0 _ll_register_callbacks cb 0x55c949f42b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.066+0000 7fca8f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca8f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46416/0 (socket says 172.21.15.46:46416) 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 0x55c94a05c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca8f400700 1 -- 172.21.15.46:0/451784044 learned_addr learned my addr 172.21.15.46:0/451784044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a05cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 -- 172.21.15.46:0/451784044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 msgr2=0x55c94a05cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a05cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 -- 172.21.15.46:0/451784044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 msgr2=0x55c94a05c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 -- 172.21.15.46:0/451784044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c949e1e3b0 con 0x55c94a04d890 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca8f400700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca94a00700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 0x55c94a05c150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca88008ca0 tx=0x7fca88005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5caf4e7483af3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.067+0000 7fca79600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800e070 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca880083b0 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca8800f040 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8802f040 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 10 client.25963 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c949e22cd0 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8800f040 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca88008d00 con 0x55c94a04d890 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 10 client.25963 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 10 client.25963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 20 client.25963 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0bdbf0 0x55c94a0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 4 client.25963 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.068+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c949f95ec0 con 0x55c94a0bdbf0 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fca8f400700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0bdbf0 0x55c94a0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fca8f400700 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0bdbf0 0x55c94a0c0010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a000bf9ba246a6c3 server_cookie=76cf202ed5591cb3 in_seq=0 out_seq=0 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fca79600700 10 client.25963 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fca79600700 1 -- 172.21.15.46:0/451784044 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c949f95ec0 con 0x55c94a0bdbf0 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fca79600700 10 client.25963 handle_command_reply: tid=0 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fcaaa47cb80 1 client.25963 shutdown 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 msgr2=0x55c94a05c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.069+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 0x55c94a05c150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca88008ca0 tx=0x7fca88005040 comp rx=0 tx=0).stop 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 shutdown_connections 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c94a0bdbf0 0x55c94a0c0010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c94a053720 0x55c94a05cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94a04d890 0x55c94a05c150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 --2- 172.21.15.46:0/451784044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c94a052e00 0x55c94a05c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca79600700 0 client.25963 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 >> 172.21.15.46:0/451784044 conn(0x55c949fab210 msgr2=0x55c94a0552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 shutdown_connections 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 1 -- 172.21.15.46:0/451784044 wait complete. 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fcaaa47cb80 20 client.25963 trim_cache size 0 max 16384 2024-05-08T23:04:36.074 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 msgr2=0x7fca740a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 0x7fca740a3870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca800cab50 tx=0x7fca800bb400 comp rx=0 tx=0).stop 2024-05-08T23:04:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 msgr2=0x7fca900732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca8000ec10 tx=0x7fca8000ef20 comp rx=0 tx=0).stop 2024-05-08T23:04:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 shutdown_connections 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca740a13b0 0x7fca740a3870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90073d20 0x7fca901a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90101060 0x7fca900732a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.070+0000 7fca96a00700 1 --2- 172.21.15.46:0/2253449834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca90106180 0x7fca900737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.071+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 >> 172.21.15.46:0/2253449834 conn(0x7fca900fc900 msgr2=0x7fca900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.071+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 shutdown_connections 2024-05-08T23:04:36.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.071+0000 7fca96a00700 1 -- 172.21.15.46:0/2253449834 wait complete. 2024-05-08T23:04:36.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:04:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:04:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:04:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:04:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:35 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3097074082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:04:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:35 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3097074082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:04:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 -- 172.21.15.46:0/2209671694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101a30 msgr2=0x7fae20105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 --2- 172.21.15.46:0/2209671694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101a30 0x7fae20105a70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009990 tx=0x7fae0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 -- 172.21.15.46:0/2209671694 shutdown_connections 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 --2- 172.21.15.46:0/2209671694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae2010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 --2- 172.21.15.46:0/2209671694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 --2- 172.21.15.46:0/2209671694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101a30 0x7fae20105a70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 -- 172.21.15.46:0/2209671694 >> 172.21.15.46:0/2209671694 conn(0x7fae200fc8e0 msgr2=0x7fae200fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 -- 172.21.15.46:0/2209671694 shutdown_connections 2024-05-08T23:04:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.290+0000 7fae25a00700 1 -- 172.21.15.46:0/2209671694 wait complete. 2024-05-08T23:04:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.291+0000 7fae25a00700 1 Processor -- start 2024-05-08T23:04:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 -- start start 2024-05-08T23:04:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae20199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2019de00 0x7fae2019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2019e880 con 0x7fae2019de00 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2019e9f0 con 0x7fae201061a0 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae25a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2019ebb0 con 0x7fae20101080 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae1f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2019de00 0x7fae2019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae1e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae20199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.292+0000 7fae1ea00700 1 -- 172.21.15.46:0/1672684093 learned_addr learned my addr 172.21.15.46:0/1672684093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 -- 172.21.15.46:0/1672684093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 msgr2=0x7fae20199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 -- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2019de00 msgr2=0x7fae2019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2019de00 0x7fae2019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 -- 172.21.15.46:0/1672684093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0095f0 con 0x7fae201061a0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae1e000700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae20199a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae0c000c00 tx=0x7fae0c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae17400700 1 -- 172.21.15.46:0/1672684093 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01d070 con 0x7fae201061a0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae17400700 1 -- 172.21.15.46:0/1672684093 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c021410 con 0x7fae201061a0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae201a60b0 con 0x7fae201061a0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.293+0000 7fae17400700 1 -- 172.21.15.46:0/1672684093 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c017650 con 0x7fae201061a0 2024-05-08T23:04:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.294+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae201a64d0 con 0x7fae201061a0 2024-05-08T23:04:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.297+0000 7fae17400700 1 -- 172.21.15.46:0/1672684093 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fae0c021580 con 0x7fae201061a0 2024-05-08T23:04:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.298+0000 7fae39512b80 1 Processor -- start 2024-05-08T23:04:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.298+0000 7fae39512b80 1 -- start start 2024-05-08T23:04:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae17400700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 0x7fae040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb41cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea98150 con 0x564a8eb46e00 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea95b10 con 0x564a8eb47720 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae39512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea91400 con 0x564a8eb41890 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae17400700 1 -- 172.21.15.46:0/1672684093 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fae0c0ce570 con 0x7fae201061a0 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae1f400700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 0x7fae040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46452/0 (socket says 172.21.15.46:46452) 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae1ea00700 1 -- 172.21.15.46:0/628767213 learned_addr learned my addr 172.21.15.46:0/628767213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.299+0000 7fae1e000700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1f400700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb41cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 -- 172.21.15.46:0/628767213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 msgr2=0x564a8eb41cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb41cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 -- 172.21.15.46:0/628767213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 msgr2=0x564a8eb471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 -- 172.21.15.46:0/628767213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8e9123b0 con 0x564a8eb47720 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1ea00700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1e000700 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cb730 tx=0x7fae0c00b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76b3891e2f4e021 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae1f400700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 0x7fae040a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae18009ec0 tx=0x7fae18009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae0be00700 1 -- 172.21.15.46:0/628767213 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c09aa10 con 0x564a8eb47720 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae0be00700 1 -- 172.21.15.46:0/628767213 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c07f050 con 0x564a8eb47720 2024-05-08T23:04:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.300+0000 7fae0be00700 1 -- 172.21.15.46:0/628767213 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c099450 con 0x564a8eb47720 2024-05-08T23:04:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 -- 172.21.15.46:0/628767213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 msgr2=0x564a8eb41350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cb730 tx=0x7fae0c00b650 comp rx=0 tx=0).stop 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 -- 172.21.15.46:0/628767213 shutdown_connections 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb46e00 0x564a8eb471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 --2- 172.21.15.46:0/628767213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41350 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 -- 172.21.15.46:0/628767213 >> 172.21.15.46:0/628767213 conn(0x564a8ea9f210 msgr2=0x564a8eb3bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 -- 172.21.15.46:0/628767213 shutdown_connections 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.301+0000 7fae39512b80 1 -- 172.21.15.46:0/628767213 wait complete. 2024-05-08T23:04:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 -- start start 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 Processor -- start 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb4d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb4b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea2d560 con 0x564a8eb47720 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ea2d200 con 0x564a8eb46e00 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8e9146a0 con 0x564a8eb41890 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.302+0000 7fae39512b80 10 client.0 _ll_register_callbacks cb 0x564a8ea36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb4b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42310/0 (socket says 172.21.15.46:42310) 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 -- 172.21.15.46:0/2690538332 learned_addr learned my addr 172.21.15.46:0/2690538332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb4d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 -- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 msgr2=0x564a8eb4d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb4d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 -- 172.21.15.46:0/2690538332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 msgr2=0x564a8eb4b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb4b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 -- 172.21.15.46:0/2690538332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae10009a80 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae1e000700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cb790 tx=0x7fae0c09aa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5aeeda4082e64f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae08c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.303+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c09b0b0 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fae0c07b060 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a8e9ea270 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0c09eee0 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 10 client.25943 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fae201a64d0 con 0x564a8eb46e00 2024-05-08T23:04:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c0b2030 con 0x564a8eb46e00 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae0c09a5a0 con 0x564a8eb46e00 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 10 client.25943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 20 client.25943 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8eb5aa00 0x564a8eb5ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 4 client.25943 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.304+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a8eb10810 con 0x564a8eb5aa00 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.305+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8eb5aa00 0x564a8eb5ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.305+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8eb5aa00 0x564a8eb5ce20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=534dc348df6a2a26 server_cookie=9f90974c1d3a50d5 in_seq=0 out_seq=0 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.305+0000 7fae08c00700 10 client.25943 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae08c00700 1 -- 172.21.15.46:0/2690538332 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a8eb10810 con 0x564a8eb5aa00 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae08c00700 10 client.25943 handle_command_reply: tid=0 2024-05-08T23:04:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 client.25943 shutdown 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 msgr2=0x564a8eb4aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cb790 tx=0x7fae0c09aa80 comp rx=0 tx=0).stop 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 shutdown_connections 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8eb5aa00 0x564a8eb5ce20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb4b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb41890 0x564a8eb4d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 --2- 172.21.15.46:0/2690538332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb46e00 0x564a8eb4aaf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 >> 172.21.15.46:0/2690538332 conn(0x564a8ea9f210 msgr2=0x564a8eb3b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae08c00700 0 client.25943 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 shutdown_connections 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 1 -- 172.21.15.46:0/2690538332 wait complete. 2024-05-08T23:04:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.306+0000 7fae39512b80 20 client.25943 trim_cache size 0 max 16384 2024-05-08T23:04:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 Processor -- start 2024-05-08T23:04:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 -- start start 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb4c160 con 0x564a8eb41890 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb61ff0 con 0x564a8eb47720 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.320+0000 7fae39512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb36800 con 0x564a8eb46e00 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46462/0 (socket says 172.21.15.46:46462) 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 -- 172.21.15.46:0/2018011920 learned_addr learned my addr 172.21.15.46:0/2018011920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1e000700 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 -- 172.21.15.46:0/2018011920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 msgr2=0x564a8eb4aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb4aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 -- 172.21.15.46:0/2018011920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 msgr2=0x564a8eb41c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 -- 172.21.15.46:0/2018011920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8e8beb60 con 0x564a8eb41890 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.321+0000 7fae1ea00700 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x564a8eb5d6c0 tx=0x7fae10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70de952e86da38a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae14c00700 1 -- 172.21.15.46:0/2018011920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10017030 con 0x564a8eb41890 2024-05-08T23:04:36.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae14c00700 1 -- 172.21.15.46:0/2018011920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae10018040 con 0x564a8eb41890 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae14c00700 1 -- 172.21.15.46:0/2018011920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10005280 con 0x564a8eb41890 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 -- 172.21.15.46:0/2018011920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 msgr2=0x564a8eb41350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x564a8eb5d6c0 tx=0x7fae10009260 comp rx=0 tx=0).stop 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 -- 172.21.15.46:0/2018011920 shutdown_connections 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb47720 0x564a8eb41c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb41890 0x564a8eb41350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 --2- 172.21.15.46:0/2018011920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 -- 172.21.15.46:0/2018011920 >> 172.21.15.46:0/2018011920 conn(0x564a8ea9f210 msgr2=0x564a8eb4a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 -- 172.21.15.46:0/2018011920 shutdown_connections 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.322+0000 7fae39512b80 1 -- 172.21.15.46:0/2018011920 wait complete. 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 -- start start 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 Processor -- start 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 0x564a8eb51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb52160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb50910 con 0x564a8eb47720 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb50a80 con 0x564a8eb41890 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8eb50c40 con 0x564a8eb46e00 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.323+0000 7fae39512b80 10 client.0 _ll_register_callbacks cb 0x564a8ea36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46474/0 (socket says 172.21.15.46:46474) 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 0x564a8eb51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 -- 172.21.15.46:0/915994351 learned_addr learned my addr 172.21.15.46:0/915994351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1e000700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb52160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 -- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 msgr2=0x564a8eb52160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb52160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 -- 172.21.15.46:0/915994351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 msgr2=0x564a8eb51c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 0x564a8eb51c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 -- 172.21.15.46:0/915994351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae10012040 con 0x564a8eb47720 2024-05-08T23:04:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1e000700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb52160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1ea00700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 0x564a8eb51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae1f400700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae18018970 tx=0x7fae1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fbdfdcc05dd68b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.324+0000 7fae08c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1800e040 con 0x564a8eb47720 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae0c07f050 con 0x564a8eb47720 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fae1801f040 con 0x564a8eb47720 2024-05-08T23:04:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae180077d0 con 0x564a8eb47720 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae39512b80 10 client.15969 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a8ea57560 con 0x564a8eb47720 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.325+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c07f050 con 0x564a8eb47720 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae1802e030 con 0x564a8eb47720 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 10 client.15969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 20 client.15969 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8ebb22a0 0x564a8ebb46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 4 client.15969 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a8ea89ec0 con 0x564a8ebb22a0 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae1ea00700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8ebb22a0 0x564a8ebb46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae1ea00700 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8ebb22a0 0x564a8ebb46c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=233cd80652222b2d server_cookie=9fb6c7d8b31e0ab in_seq=0 out_seq=0 2024-05-08T23:04:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.326+0000 7fae08c00700 10 client.15969 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae08c00700 1 -- 172.21.15.46:0/915994351 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a8ea89ec0 con 0x564a8ebb22a0 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae08c00700 10 client.15969 handle_command_reply: tid=0 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 client.15969 shutdown 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 msgr2=0x564a8eb502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae18018970 tx=0x7fae1800b040 comp rx=0 tx=0).stop 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 shutdown_connections 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564a8ebb22a0 0x564a8ebb46c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8eb47720 0x564a8eb502f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8eb41890 0x564a8eb51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 --2- 172.21.15.46:0/915994351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8eb46e00 0x564a8eb52160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 >> 172.21.15.46:0/915994351 conn(0x564a8ea9f210 msgr2=0x564a8eb45bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae08c00700 0 client.15969 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:04:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 shutdown_connections 2024-05-08T23:04:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 1 -- 172.21.15.46:0/915994351 wait complete. 2024-05-08T23:04:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.327+0000 7fae39512b80 20 client.15969 trim_cache size 0 max 16384 2024-05-08T23:04:36.331 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 msgr2=0x7fae040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 0x7fae040a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae18009ec0 tx=0x7fae18009920 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 msgr2=0x7fae20199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae20199a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae0c000c00 tx=0x7fae0c004730 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 shutdown_connections 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fae040a13b0 0x7fae040a3870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae2019de00 0x7fae2019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae20101080 0x7fae20199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 --2- 172.21.15.46:0/1672684093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae201061a0 0x7fae20199a60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 >> 172.21.15.46:0/1672684093 conn(0x7fae200fc8e0 msgr2=0x7fae200fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 shutdown_connections 2024-05-08T23:04:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.328+0000 7fae25a00700 1 -- 172.21.15.46:0/1672684093 wait complete. 2024-05-08T23:04:36.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:04:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 -- 172.21.15.46:0/616283909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 msgr2=0x7f2244101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 --2- 172.21.15.46:0/616283909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f2244101440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f22340098f0 tx=0x7f2234009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 -- 172.21.15.46:0/616283909 shutdown_connections 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 --2- 172.21.15.46:0/616283909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 0x7f224410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 --2- 172.21.15.46:0/616283909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f2244101440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 --2- 172.21.15.46:0/616283909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244101a10 0x7f2244105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 -- 172.21.15.46:0/616283909 >> 172.21.15.46:0/616283909 conn(0x7f22440fc900 msgr2=0x7f22440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 -- 172.21.15.46:0/616283909 shutdown_connections 2024-05-08T23:04:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.548+0000 7f224aa00700 1 -- 172.21.15.46:0/616283909 wait complete. 2024-05-08T23:04:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.549+0000 7f224aa00700 1 Processor -- start 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 -- start start 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 0x7f2244199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419e910 con 0x7f224419de90 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419ea80 con 0x7f2244101060 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f224aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419ec40 con 0x7f2244106180 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2248a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2243e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2248a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46490/0 (socket says 172.21.15.46:46490) 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2243e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42348/0 (socket says 172.21.15.46:42348) 2024-05-08T23:04:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2248a00700 1 -- 172.21.15.46:0/193118065 learned_addr learned my addr 172.21.15.46:0/193118065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.550+0000 7f2243400700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 0x7f2244199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 -- 172.21.15.46:0/193118065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 msgr2=0x7f2244199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 0x7f2244199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 -- 172.21.15.46:0/193118065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 msgr2=0x7f22441995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 -- 172.21.15.46:0/193118065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300095f0 con 0x7f224419de90 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2243e00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2248a00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f223c00ef30 tx=0x7f223c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2240c00700 1 -- 172.21.15.46:0/193118065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c018070 con 0x7f224419de90 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2240c00700 1 -- 172.21.15.46:0/193118065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223c00f040 con 0x7f224419de90 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f2240c00700 1 -- 172.21.15.46:0/193118065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00cd00 con 0x7f224419de90 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2234009520 con 0x7f224419de90 2024-05-08T23:04:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.551+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22441a6440 con 0x7f224419de90 2024-05-08T23:04:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.554+0000 7f2240c00700 1 -- 172.21.15.46:0/193118065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f223c003a00 con 0x7f224419de90 2024-05-08T23:04:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f2240c00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 0x7f22280a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f2240c00700 1 -- 172.21.15.46:0/193118065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f223c0ccea0 con 0x7f224419de90 2024-05-08T23:04:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f225e41db80 1 Processor -- start 2024-05-08T23:04:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f2243e00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 0x7f22280a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f225e41db80 1 -- start start 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.556+0000 7f225e41db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f225e41db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f972720 0x56063f96c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f225e41db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f96c890 0x56063f96ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f225e41db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f8c3150 con 0x56063f971e00 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f225e41db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f8c0b10 con 0x56063f972720 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f225e41db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f8bc400 con 0x56063f96c890 2024-05-08T23:04:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 0x7f22280a3870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22340098c0 tx=0x7f2234019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46498/0 (socket says 172.21.15.46:46498) 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 -- 172.21.15.46:0/2847378928 learned_addr learned my addr 172.21.15.46:0/2847378928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243400700 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f972720 0x56063f96c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2248a00700 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f96c890 0x56063f96ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 -- 172.21.15.46:0/2847378928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f96c890 msgr2=0x56063f96ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f96c890 0x56063f96ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 -- 172.21.15.46:0/2847378928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f972720 msgr2=0x56063f96c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f972720 0x56063f96c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.557+0000 7f2243e00700 1 -- 172.21.15.46:0/2847378928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56063f73d3b0 con 0x56063f971e00 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.558+0000 7f2243e00700 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f22340098f0 tx=0x7f223401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3132230260691bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.558+0000 7f2239200700 1 -- 172.21.15.46:0/2847378928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400b7e0 con 0x56063f971e00 2024-05-08T23:04:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.558+0000 7f2239200700 1 -- 172.21.15.46:0/2847378928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223401d040 con 0x56063f971e00 2024-05-08T23:04:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.558+0000 7f2239200700 1 -- 172.21.15.46:0/2847378928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340176f0 con 0x56063f971e00 2024-05-08T23:04:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- 172.21.15.46:0/2847378928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 msgr2=0x56063f9721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f22340098f0 tx=0x7f223401f000 comp rx=0 tx=0).stop 2024-05-08T23:04:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- 172.21.15.46:0/2847378928 shutdown_connections 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f96c890 0x56063f96ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f9721e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 --2- 172.21.15.46:0/2847378928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f972720 0x56063f96c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- 172.21.15.46:0/2847378928 >> 172.21.15.46:0/2847378928 conn(0x56063f8ca210 msgr2=0x56063f966df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- 172.21.15.46:0/2847378928 shutdown_connections 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- 172.21.15.46:0/2847378928 wait complete. 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.559+0000 7f225e41db80 1 -- start start 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 Processor -- start 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 0x56063f975b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f976070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f858560 con 0x56063f971e00 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f858200 con 0x56063f96c890 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f73f6a0 con 0x56063f972720 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f225e41db80 10 client.0 _ll_register_callbacks cb 0x56063f861b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f2243e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 0x56063f975b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f2243400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f976070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f2243e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 0x56063f975b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42368/0 (socket says 172.21.15.46:42368) 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f2243e00700 1 -- 172.21.15.46:0/505784922 learned_addr learned my addr 172.21.15.46:0/505784922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.560+0000 7f2248a00700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 -- 172.21.15.46:0/505784922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 msgr2=0x56063f9765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 -- 172.21.15.46:0/505784922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 msgr2=0x56063f975b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 0x56063f975b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 -- 172.21.15.46:0/505784922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224410e570 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f2243400700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f976070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x56063f972b80 tx=0x7f2230012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f191ff4d8b4d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f222d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230017070 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f223000f560 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.561+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56063f815270 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223000f720 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 10 client.15981 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56063f6e9b60 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56063f815270 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22300108f0 con 0x56063f971e00 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 10 client.15981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:04:36.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 20 client.15981 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f986230 0x56063f988650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 4 client.15981 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56063f93b810 con 0x56063f986230 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.562+0000 7f2243e00700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f986230 0x56063f988650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.563+0000 7f2243e00700 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f986230 0x56063f988650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fd6efd1b0db319c server_cookie=f30885df1cfb9ae3 in_seq=0 out_seq=0 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.563+0000 7f222d600700 10 client.15981 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f222d600700 1 -- 172.21.15.46:0/505784922 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56063f93b810 con 0x56063f986230 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f222d600700 10 client.15981 handle_command_reply: tid=0 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 client.15981 shutdown 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 msgr2=0x56063f976070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f976070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x56063f972b80 tx=0x7f2230012c30 comp rx=0 tx=0).stop 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 shutdown_connections 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f96c890 0x56063f975b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f986230 0x56063f988650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 --2- 172.21.15.46:0/505784922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f971e00 0x56063f976070 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 >> 172.21.15.46:0/505784922 conn(0x56063f8ca210 msgr2=0x56063f9665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f222d600700 0 client.15981 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 shutdown_connections 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 1 -- 172.21.15.46:0/505784922 wait complete. 2024-05-08T23:04:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.564+0000 7f225e41db80 20 client.15981 trim_cache size 0 max 16384 2024-05-08T23:04:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.578+0000 7f225e41db80 1 Processor -- start 2024-05-08T23:04:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.578+0000 7f225e41db80 1 -- start start 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.578+0000 7f225e41db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f225e41db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f966d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f225e41db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f225e41db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f977130 con 0x56063f96c890 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f225e41db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f98d860 con 0x56063f971e00 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f225e41db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f974080 con 0x56063f972720 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46516/0 (socket says 172.21.15.46:46516) 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 -- 172.21.15.46:0/1346817856 learned_addr learned my addr 172.21.15.46:0/1346817856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243400700 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f966d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2248a00700 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 -- 172.21.15.46:0/1346817856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 msgr2=0x56063f9763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9763d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 -- 172.21.15.46:0/1346817856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 msgr2=0x56063f966d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f966d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.579+0000 7f2243e00700 1 -- 172.21.15.46:0/1346817856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56063f882560 con 0x56063f96c890 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f2243e00700 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2234011040 tx=0x7f2234009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27644f7e37ad2020 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f2239c00700 1 -- 172.21.15.46:0/1346817856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400b6a0 con 0x56063f96c890 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f2239c00700 1 -- 172.21.15.46:0/1346817856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223403f830 con 0x56063f96c890 2024-05-08T23:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f2239c00700 1 -- 172.21.15.46:0/1346817856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234003680 con 0x56063f96c890 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 -- 172.21.15.46:0/1346817856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 msgr2=0x56063f981bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2234011040 tx=0x7f2234009440 comp rx=0 tx=0).stop 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 -- 172.21.15.46:0/1346817856 shutdown_connections 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f9763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f981bc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 --2- 172.21.15.46:0/1346817856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f966d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.580+0000 7f225e41db80 1 -- 172.21.15.46:0/1346817856 >> 172.21.15.46:0/1346817856 conn(0x56063f8ca210 msgr2=0x56063f974400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.581+0000 7f225e41db80 1 -- 172.21.15.46:0/1346817856 shutdown_connections 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.581+0000 7f225e41db80 1 -- 172.21.15.46:0/1346817856 wait complete. 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.581+0000 7f225e41db80 1 -- start start 2024-05-08T23:04:36.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.581+0000 7f225e41db80 1 Processor -- start 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f97b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f97bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f9db730 con 0x56063f96c890 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f9db870 con 0x56063f971e00 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f9db9b0 con 0x56063f972720 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f225e41db80 10 client.0 _ll_register_callbacks cb 0x56063f861b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f2243e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46530/0 (socket says 172.21.15.46:46530) 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f2243e00700 1 -- 172.21.15.46:0/3107392045 learned_addr learned my addr 172.21.15.46:0/3107392045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f2243400700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f97b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.582+0000 7f2248a00700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f97bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 -- 172.21.15.46:0/3107392045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 msgr2=0x56063f97bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f97bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 -- 172.21.15.46:0/3107392045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 msgr2=0x56063f97b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f97b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 -- 172.21.15.46:0/3107392045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56063f73d3b0 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f2243e00700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2234005e00 tx=0x7f223400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8075e14377eb61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f222d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234005460 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f223401d040 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f223403d070 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.583+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22340055c0 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 10 client.15987 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56063f741cd0 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223401d040 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2234003710 con 0x56063f96c890 2024-05-08T23:04:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 20 client.15987 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f9dcbb0 0x56063f9defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 4 client.15987 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56063f8b4ec0 con 0x56063f9dcbb0 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.584+0000 7f2243400700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f9dcbb0 0x56063f9defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f2243400700 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f9dcbb0 0x56063f9defa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c1df9d5f785258a server_cookie=5a29a77960bef8b4 in_seq=0 out_seq=0 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f222d600700 10 client.15987 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f222d600700 1 -- 172.21.15.46:0/3107392045 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56063f8b4ec0 con 0x56063f9dcbb0 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f222d600700 10 client.15987 handle_command_reply: tid=0 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 client.15987 shutdown 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 msgr2=0x56063f97b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2234005e00 tx=0x7f223400b730 comp rx=0 tx=0).stop 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 shutdown_connections 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063f972720 0x56063f97bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56063f9dcbb0 0x56063f9defa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56063f96c890 0x56063f97b0f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 --2- 172.21.15.46:0/3107392045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56063f971e00 0x56063f97b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f222d600700 0 client.15987 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.585+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 >> 172.21.15.46:0/3107392045 conn(0x56063f8ca210 msgr2=0x56063f970c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 shutdown_connections 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f225e41db80 1 -- 172.21.15.46:0/3107392045 wait complete. 2024-05-08T23:04:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f225e41db80 20 client.15987 trim_cache size 0 max 16384 2024-05-08T23:04:36.591 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:36.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 msgr2=0x7f22280a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 0x7f22280a3870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22340098c0 tx=0x7f2234019040 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 msgr2=0x7f224419e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f223c00ef30 tx=0x7f223c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 shutdown_connections 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22280a13b0 0x7f22280a3870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224419de90 0x7f224419e310 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2244101060 0x7f22441995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 --2- 172.21.15.46:0/193118065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2244106180 0x7f2244199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 >> 172.21.15.46:0/193118065 conn(0x7f22440fc900 msgr2=0x7f22440fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.586+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 shutdown_connections 2024-05-08T23:04:36.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.587+0000 7f224aa00700 1 -- 172.21.15.46:0/193118065 wait complete. 2024-05-08T23:04:36.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 -- 172.21.15.46:0/977921658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc101080 msgr2=0x7f26bc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 --2- 172.21.15.46:0/977921658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc101080 0x7f26bc101460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009990 tx=0x7f26ac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 -- 172.21.15.46:0/977921658 shutdown_connections 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 --2- 172.21.15.46:0/977921658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc1061a0 0x7f26bc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 --2- 172.21.15.46:0/977921658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc101080 0x7f26bc101460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 --2- 172.21.15.46:0/977921658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc101a30 0x7f26bc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 -- 172.21.15.46:0/977921658 >> 172.21.15.46:0/977921658 conn(0x7f26bc0fc8e0 msgr2=0x7f26bc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 -- 172.21.15.46:0/977921658 shutdown_connections 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.806+0000 7f26c2e00700 1 -- 172.21.15.46:0/977921658 wait complete. 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.807+0000 7f26c2e00700 1 Processor -- start 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.807+0000 7f26c2e00700 1 -- start start 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc101080 0x7f26bc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 0x7f26bc1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc1a5b30 con 0x7f26bc1061a0 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc1a5ca0 con 0x7f26bc073d20 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc1a5e60 con 0x7f26bc101080 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 0x7f26bc1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46540/0 (socket says 172.21.15.46:46540) 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 -- 172.21.15.46:0/2068843762 learned_addr learned my addr 172.21.15.46:0/2068843762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26bbe00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc101080 0x7f26bc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 -- 172.21.15.46:0/2068843762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc101080 msgr2=0x7f26bc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc101080 0x7f26bc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 -- 172.21.15.46:0/2068843762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 msgr2=0x7f26bc1a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 0x7f26bc1a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26b3e00700 1 -- 172.21.15.46:0/2068843762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0095f0 con 0x7f26bc1061a0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.808+0000 7f26c0e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 0x7f26bc1a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26b3e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e970 tx=0x7f26a400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26b9600700 1 -- 172.21.15.46:0/2068843762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4009790 con 0x7f26bc1061a0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26b9600700 1 -- 172.21.15.46:0/2068843762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a400c7e0 con 0x7f26bc1061a0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc1a6140 con 0x7f26bc1061a0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26b9600700 1 -- 172.21.15.46:0/2068843762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40054e0 con 0x7f26bc1061a0 2024-05-08T23:04:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.809+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc1a6650 con 0x7f26bc1061a0 2024-05-08T23:04:36.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.814+0000 7f26b9600700 1 -- 172.21.15.46:0/2068843762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f26a400c950 con 0x7f26bc1061a0 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26d6927b80 1 Processor -- start 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26b9600700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 0x7f26a00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26b9600700 1 -- 172.21.15.46:0/2068843762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f26a4014070 con 0x7f26bc1061a0 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26d6927b80 1 -- start start 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26c0e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 0x7f26a00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.815+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d129350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d129890 0x559d6d129cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26d6927b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d080150 con 0x559d6d12ee00 2024-05-08T23:04:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26d6927b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d07db10 con 0x559d6d12f720 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26d6927b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d079400 con 0x559d6d129890 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46554/0 (socket says 172.21.15.46:46554) 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 -- 172.21.15.46:0/328394717 learned_addr learned my addr 172.21.15.46:0/328394717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26b3e00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d129350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26c0e00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d129890 0x559d6d129cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 -- 172.21.15.46:0/328394717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d129890 msgr2=0x559d6d129cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d129890 0x559d6d129cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 -- 172.21.15.46:0/328394717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 msgr2=0x559d6d129350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d129350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26bbe00700 1 -- 172.21.15.46:0/328394717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6cefa3b0 con 0x559d6d12ee00 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26c0e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 0x7f26a00a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26b4003e80 tx=0x7f26b400b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.816+0000 7f26b3e00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26bbe00700 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f26ac005ec0 tx=0x7f26ac005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c564dd262c55b9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26b1600700 1 -- 172.21.15.46:0/328394717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01d070 con 0x559d6d12ee00 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26b1600700 1 -- 172.21.15.46:0/328394717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac01e040 con 0x559d6d12ee00 2024-05-08T23:04:36.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26b1600700 1 -- 172.21.15.46:0/328394717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac004880 con 0x559d6d12ee00 2024-05-08T23:04:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26d6927b80 1 -- 172.21.15.46:0/328394717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 msgr2=0x559d6d12f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.817+0000 7f26d6927b80 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f26ac005ec0 tx=0x7f26ac005ef0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 -- 172.21.15.46:0/328394717 shutdown_connections 2024-05-08T23:04:36.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d129890 0x559d6d129cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d12f1e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 --2- 172.21.15.46:0/328394717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 -- 172.21.15.46:0/328394717 >> 172.21.15.46:0/328394717 conn(0x559d6d087210 msgr2=0x559d6d123df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 -- 172.21.15.46:0/328394717 shutdown_connections 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 -- 172.21.15.46:0/328394717 wait complete. 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 -- start start 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.818+0000 7f26d6927b80 1 Processor -- start 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12ee00 0x559d6d133020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d133560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d015560 con 0x559d6d129890 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d015200 con 0x559d6d12f720 2024-05-08T23:04:36.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6cefc6a0 con 0x559d6d12ee00 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26d6927b80 10 client.0 _ll_register_callbacks cb 0x559d6d01eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46566/0 (socket says 172.21.15.46:46566) 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26c0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d133560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26bbe00700 1 -- 172.21.15.46:0/933556487 learned_addr learned my addr 172.21.15.46:0/933556487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.819+0000 7f26b3e00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12ee00 0x559d6d133020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 -- 172.21.15.46:0/933556487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12ee00 msgr2=0x559d6d133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12ee00 0x559d6d133020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 -- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 msgr2=0x559d6d133560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d133560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 -- 172.21.15.46:0/933556487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc1a6650 con 0x559d6d129890 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26bbe00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00bee0 tx=0x7f26ac00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295e0a1714595d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26aa000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01d030 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f26ac0058e0 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d6cfd2270 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.820+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26ac00bce0 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 10 client.15999 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f26bc10e590 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d6cfd2270 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26ac00f840 con 0x559d6d129890 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 20 client.15999 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d142c60 0x559d6d145050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 4 client.15999 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d6d0f8810 con 0x559d6d142c60 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.821+0000 7f26b3e00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d142c60 0x559d6d145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.822+0000 7f26b3e00700 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d142c60 0x559d6d145050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba66279c27f60e1 server_cookie=59a861e594c61f67 in_seq=0 out_seq=0 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.822+0000 7f26aa000700 10 client.15999 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26aa000700 1 -- 172.21.15.46:0/933556487 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d6d0f8810 con 0x559d6d142c60 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26aa000700 10 client.15999 handle_command_reply: tid=0 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 client.15999 shutdown 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 msgr2=0x559d6d132ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00bee0 tx=0x7f26ac00b920 comp rx=0 tx=0).stop 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 shutdown_connections 2024-05-08T23:04:36.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d12f720 0x559d6d133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d142c60 0x559d6d145050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d129890 0x559d6d132ae0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 --2- 172.21.15.46:0/933556487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12ee00 0x559d6d133020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26aa000700 0 client.15999 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:36.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 >> 172.21.15.46:0/933556487 conn(0x559d6d087210 msgr2=0x559d6d1235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 shutdown_connections 2024-05-08T23:04:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.823+0000 7f26d6927b80 1 -- 172.21.15.46:0/933556487 wait complete. 2024-05-08T23:04:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.824+0000 7f26d6927b80 20 client.15999 trim_cache size 0 max 16384 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.837+0000 7f26d6927b80 1 Processor -- start 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.837+0000 7f26d6927b80 1 -- start start 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d129c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d123d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d134150 con 0x559d6d12ee00 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d133e30 con 0x559d6d129890 2024-05-08T23:04:36.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26d6927b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d120dc0 con 0x559d6d12f720 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d129c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42432/0 (socket says 172.21.15.46:42432) 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26bbe00700 1 -- 172.21.15.46:0/1268830248 learned_addr learned my addr 172.21.15.46:0/1268830248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.838+0000 7f26c0e00700 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d123d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 -- 172.21.15.46:0/1268830248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 msgr2=0x559d6d123d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d123d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 -- 172.21.15.46:0/1268830248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 msgr2=0x559d6d12f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 -- 172.21.15.46:0/1268830248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6cea6b60 con 0x559d6d12ee00 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26bbe00700 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:36.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b3e00700 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d129c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26a40ca7c0 tx=0x7f26a40aef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86267c2278d500c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b2000700 1 -- 172.21.15.46:0/1268830248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a409d040 con 0x559d6d12ee00 2024-05-08T23:04:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b2000700 1 -- 172.21.15.46:0/1268830248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a409a7b0 con 0x559d6d12ee00 2024-05-08T23:04:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.839+0000 7f26b2000700 1 -- 172.21.15.46:0/1268830248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4098450 con 0x559d6d12ee00 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- 172.21.15.46:0/1268830248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 msgr2=0x559d6d129c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d129c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26a40ca7c0 tx=0x7f26a40aef90 comp rx=0 tx=0).stop 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- 172.21.15.46:0/1268830248 shutdown_connections 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d12f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 --2- 172.21.15.46:0/1268830248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d129c70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- 172.21.15.46:0/1268830248 >> 172.21.15.46:0/1268830248 conn(0x559d6d087210 msgr2=0x559d6d12cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- 172.21.15.46:0/1268830248 shutdown_connections 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- 172.21.15.46:0/1268830248 wait complete. 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.840+0000 7f26d6927b80 1 -- start start 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 Processor -- start 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d1360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d136b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d1980e0 con 0x559d6d12ee00 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d198220 con 0x559d6d129890 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6d198360 con 0x559d6d12f720 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.841+0000 7f26d6927b80 10 client.0 _ll_register_callbacks cb 0x559d6d01eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46588/0 (socket says 172.21.15.46:46588) 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 -- 172.21.15.46:0/3425748792 learned_addr learned my addr 172.21.15.46:0/3425748792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26bbe00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d1360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26c0e00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 -- 172.21.15.46:0/3425748792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 msgr2=0x559d6d136b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d136b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 -- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 msgr2=0x559d6d1360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:36.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d1360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 -- 172.21.15.46:0/3425748792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc10e590 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26b3e00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26a400efd0 tx=0x7f26a4005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8057d9a60df17b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26aa000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a409d050 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f26a407a030 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.842+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26ac01e040 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26a409dee0 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 10 client.16005 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f26bc1a6140 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a40a6080 con 0x559d6d12ee00 2024-05-08T23:04:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26a40995a0 con 0x559d6d12ee00 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 20 client.16005 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d199880 0x559d6d19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 4 client.16005 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d6d071ec0 con 0x559d6d199880 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.843+0000 7f26bbe00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d199880 0x559d6d19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26bbe00700 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d199880 0x559d6d19bca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc18b90b0610d3b server_cookie=f1bdf870970f2c1f in_seq=0 out_seq=0 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26aa000700 10 client.16005 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26aa000700 1 -- 172.21.15.46:0/3425748792 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d6d071ec0 con 0x559d6d199880 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26aa000700 10 client.16005 handle_command_reply: tid=0 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 client.16005 shutdown 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 msgr2=0x559d6d136620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26a400efd0 tx=0x7f26a4005d00 comp rx=0 tx=0).stop 2024-05-08T23:04:36.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 shutdown_connections 2024-05-08T23:04:36.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d6d12f720 0x559d6d136b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d6d129890 0x559d6d1360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559d6d199880 0x559d6d19bca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 --2- 172.21.15.46:0/3425748792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d6d12ee00 0x559d6d136620 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 >> 172.21.15.46:0/3425748792 conn(0x559d6d087210 msgr2=0x559d6d131260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.844+0000 7f26aa000700 0 client.16005 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 shutdown_connections 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26d6927b80 1 -- 172.21.15.46:0/3425748792 wait complete. 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26d6927b80 20 client.16005 trim_cache size 0 max 16384 2024-05-08T23:04:36.852 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:04:36.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 msgr2=0x7f26a00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 0x7f26a00a3870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f26b4003e80 tx=0x7f26b400b040 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 msgr2=0x7f26bc0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26a400e970 tx=0x7f26a400ec80 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 shutdown_connections 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc073d20 0x7f26bc1a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc101080 0x7f26bc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26a00a13b0 0x7f26a00a3870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 --2- 172.21.15.46:0/2068843762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc1061a0 0x7f26bc0737e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.845+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 >> 172.21.15.46:0/2068843762 conn(0x7f26bc0fc8e0 msgr2=0x7f26bc0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.846+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 shutdown_connections 2024-05-08T23:04:36.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:36.846+0000 7f26c2e00700 1 -- 172.21.15.46:0/2068843762 wait complete. 2024-05-08T23:04:37.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:04:37.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:04:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 -- 172.21.15.46:0/1237883065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a981061a0 msgr2=0x7f4a9810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/1237883065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a981061a0 0x7f4a9810dd20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009f90 tx=0x7f4a9000b330 comp rx=0 tx=0).stop 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 -- 172.21.15.46:0/1237883065 shutdown_connections 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/1237883065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a981061a0 0x7f4a9810dd20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/1237883065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a98101080 0x7f4a98101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/1237883065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101a30 0x7f4a98105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 -- 172.21.15.46:0/1237883065 >> 172.21.15.46:0/1237883065 conn(0x7f4a980fc8e0 msgr2=0x7f4a980fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.080+0000 7f4a9ce00700 1 -- 172.21.15.46:0/1237883065 shutdown_connections 2024-05-08T23:04:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.081+0000 7f4a9ce00700 1 -- 172.21.15.46:0/1237883065 wait complete. 2024-05-08T23:04:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.081+0000 7f4a9ce00700 1 Processor -- start 2024-05-08T23:04:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 -- start start 2024-05-08T23:04:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101080 0x7f4a981971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981061a0 0x7f4a98197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819c530 con 0x7f4a9819bab0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819c6a0 con 0x7f4a98101080 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a9ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9819c860 con 0x7f4a981061a0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101080 0x7f4a981971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46606/0 (socket says 172.21.15.46:46606) 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a96800700 1 -- 172.21.15.46:0/3415842799 learned_addr learned my addr 172.21.15.46:0/3415842799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.082+0000 7f4a95400700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981061a0 0x7f4a98197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 -- 172.21.15.46:0/3415842799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981061a0 msgr2=0x7f4a98197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981061a0 0x7f4a98197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 -- 172.21.15.46:0/3415842799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101080 msgr2=0x7f4a981971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101080 0x7f4a981971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 -- 172.21.15.46:0/3415842799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880095f0 con 0x7f4a9819bab0 2024-05-08T23:04:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a96800700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000b6c0 tx=0x7f4a90004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a8ea00700 1 -- 172.21.15.46:0/3415842799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000e040 con 0x7f4a9819bab0 2024-05-08T23:04:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a8ea00700 1 -- 172.21.15.46:0/3415842799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90004e50 con 0x7f4a9819bab0 2024-05-08T23:04:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a90009c40 con 0x7f4a9819bab0 2024-05-08T23:04:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a8ea00700 1 -- 172.21.15.46:0/3415842799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9001b6b0 con 0x7f4a9819bab0 2024-05-08T23:04:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.083+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a981a40a0 con 0x7f4a9819bab0 2024-05-08T23:04:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.087+0000 7f4a8ea00700 1 -- 172.21.15.46:0/3415842799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7f4a9000bbf0 con 0x7f4a9819bab0 2024-05-08T23:04:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.088+0000 7f4a8ea00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 0x7f4a7c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.088+0000 7f4ab08d1b80 1 Processor -- start 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.088+0000 7f4a8ea00700 1 -- 172.21.15.46:0/3415842799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a900ce9b0 con 0x7f4a9819bab0 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.088+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 0x7f4a7c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.088+0000 7f4ab08d1b80 1 -- start start 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e53c0 0x559dd42e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd42381d0 con 0x559dd42e1e50 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4235b90 con 0x559dd42e4fe0 2024-05-08T23:04:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4231480 con 0x559dd42e53c0 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46616/0 (socket says 172.21.15.46:46616) 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a96800700 1 -- 172.21.15.46:0/4024280949 learned_addr learned my addr 172.21.15.46:0/4024280949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 0x7f4a7c0a38f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005fd0 tx=0x7f4a88009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a95400700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e53c0 0x559dd42e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.089+0000 7f4a95e00700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 -- 172.21.15.46:0/4024280949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e53c0 msgr2=0x559dd42e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e53c0 0x559dd42e1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 -- 172.21.15.46:0/4024280949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 msgr2=0x559dd42e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 -- 172.21.15.46:0/4024280949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dd40b2430 con 0x559dd42e1e50 2024-05-08T23:04:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a95e00700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a96800700 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000bea0 tx=0x7f4a9001dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd27c1f985c38dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a83400700 1 -- 172.21.15.46:0/4024280949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9009f120 con 0x559dd42e1e50 2024-05-08T23:04:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a83400700 1 -- 172.21.15.46:0/4024280949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90011390 con 0x559dd42e1e50 2024-05-08T23:04:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.090+0000 7f4a83400700 1 -- 172.21.15.46:0/4024280949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9009b750 con 0x559dd42e1e50 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/4024280949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 msgr2=0x559dd42f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000bea0 tx=0x7f4a9001dca0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/4024280949 shutdown_connections 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e1e50 0x559dd42f6040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/4024280949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e53c0 0x559dd42e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/4024280949 >> 172.21.15.46:0/4024280949 conn(0x559dd423f290 msgr2=0x559dd42dbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/4024280949 shutdown_connections 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.091+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/4024280949 wait complete. 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 -- start start 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 Processor -- start 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd41cd5e0 con 0x559dd42e53c0 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd41cd280 con 0x559dd42e4fe0 2024-05-08T23:04:38.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd40b4720 con 0x559dd42e1e50 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.092+0000 7f4ab08d1b80 10 client.0 _ll_register_callbacks cb 0x559dd41d6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42504/0 (socket says 172.21.15.46:42504) 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 -- 172.21.15.46:0/3829482666 learned_addr learned my addr 172.21.15.46:0/3829482666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a96800700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 -- 172.21.15.46:0/3829482666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 msgr2=0x559dd42f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 -- 172.21.15.46:0/3829482666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 msgr2=0x559dd42f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 -- 172.21.15.46:0/3829482666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a981a40a0 con 0x559dd42e4fe0 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:38.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a96800700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a95400700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012680 tx=0x7f4a84012990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4c905271ff505f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a77e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.093+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84017070 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dd418a2f0 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a84004020 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a840041e0 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4ab08d1b80 10 client.25997 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559dd405ebe0 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559dd418a2f0 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.094+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a8400f510 con 0x559dd42e4fe0 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:38.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 20 client.25997 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd43037a0 0x559dd4305bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 4 client.25997 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dd42b0890 con 0x559dd43037a0 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd43037a0 0x559dd4305bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd43037a0 0x559dd4305bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef8b21dff0913299 server_cookie=5ffa42674e539177 in_seq=0 out_seq=0 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.095+0000 7f4a77e00700 10 client.25997 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4a77e00700 1 -- 172.21.15.46:0/3829482666 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dd42b0890 con 0x559dd43037a0 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4a77e00700 10 client.25997 handle_command_reply: tid=0 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 client.25997 shutdown 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 msgr2=0x559dd42f38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012680 tx=0x7f4a84012990 comp rx=0 tx=0).stop 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 shutdown_connections 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd43037a0 0x559dd4305bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.096+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3829482666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f38a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4a77e00700 0 client.25997 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 >> 172.21.15.46:0/3829482666 conn(0x559dd423f290 msgr2=0x559dd42db620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 shutdown_connections 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3829482666 wait complete. 2024-05-08T23:04:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.097+0000 7f4ab08d1b80 20 client.25997 trim_cache size 0 max 16384 2024-05-08T23:04:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.115+0000 7f4ab08d1b80 1 Processor -- start 2024-05-08T23:04:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 -- start start 2024-05-08T23:04:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4300310 con 0x559dd42e53c0 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd42f4960 con 0x559dd42e4fe0 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd42ddaa0 con 0x559dd42e1e50 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a95400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46632/0 (socket says 172.21.15.46:46632) 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53092/0 (socket says 172.21.15.46:53092) 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.116+0000 7f4a96800700 1 -- 172.21.15.46:0/3470249719 learned_addr learned my addr 172.21.15.46:0/3470249719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 -- 172.21.15.46:0/3470249719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 msgr2=0x559dd42fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 -- 172.21.15.46:0/3470249719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 msgr2=0x559dd42fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 -- 172.21.15.46:0/3470249719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dd42d8560 con 0x559dd42e1e50 2024-05-08T23:04:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a95e00700 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559dd4306470 tx=0x7f4a88014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39642da9da53ef4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a83e00700 1 -- 172.21.15.46:0/3470249719 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88023040 con 0x559dd42e1e50 2024-05-08T23:04:38.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a83e00700 1 -- 172.21.15.46:0/3470249719 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a88029410 con 0x559dd42e1e50 2024-05-08T23:04:38.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.117+0000 7f4a83e00700 1 -- 172.21.15.46:0/3470249719 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003c50 con 0x559dd42e1e50 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3470249719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 msgr2=0x559dd42f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559dd4306470 tx=0x7f4a88014000 comp rx=0 tx=0).stop 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3470249719 shutdown_connections 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f22b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/3470249719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3470249719 >> 172.21.15.46:0/3470249719 conn(0x559dd423f290 msgr2=0x559dd42e0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3470249719 shutdown_connections 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/3470249719 wait complete. 2024-05-08T23:04:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.118+0000 7f4ab08d1b80 1 -- start start 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 Processor -- start 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4359cb0 con 0x559dd42e53c0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4359df0 con 0x559dd42e4fe0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dd4359f30 con 0x559dd42e1e50 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4ab08d1b80 10 client.0 _ll_register_callbacks cb 0x559dd41d6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53106/0 (socket says 172.21.15.46:53106) 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4a95e00700 1 -- 172.21.15.46:0/2098898868 learned_addr learned my addr 172.21.15.46:0/2098898868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4a96800700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.119+0000 7f4a95400700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 -- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 msgr2=0x559dd42f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 -- 172.21.15.46:0/2098898868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 msgr2=0x559dd42f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 -- 172.21.15.46:0/2098898868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a88017070 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a95e00700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a8801eb90 tx=0x7f4a8801ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ba485e6ee20347 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a77e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88023070 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a90011390 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.120+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a88029790 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8801ee20 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 10 client.26035 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559dd41f75e0 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90011390 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a88039020 con 0x559dd42e1e50 2024-05-08T23:04:38.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 10 client.26035 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 20 client.26035 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd435b6f0 0x559dd435db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 4 client.26035 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.121+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559dd4229f40 con 0x559dd435b6f0 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.122+0000 7f4a95400700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd435b6f0 0x559dd435db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.122+0000 7f4a95400700 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd435b6f0 0x559dd435db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63bdd176770fde4f server_cookie=11e2dec14391bc6f in_seq=0 out_seq=0 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.122+0000 7f4a77e00700 10 client.26035 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4a77e00700 1 -- 172.21.15.46:0/2098898868 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559dd4229f40 con 0x559dd435b6f0 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4a77e00700 10 client.26035 handle_command_reply: tid=0 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 client.26035 shutdown 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 msgr2=0x559dd42f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a8801eb90 tx=0x7f4a8801ebc0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 shutdown_connections 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559dd435b6f0 0x559dd435db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dd42e53c0 0x559dd42f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dd42e1e50 0x559dd42f9360 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 --2- 172.21.15.46:0/2098898868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559dd42e4fe0 0x559dd42f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 >> 172.21.15.46:0/2098898868 conn(0x559dd423f290 msgr2=0x559dd42e03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4a77e00700 0 client.26035 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.123+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 shutdown_connections 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4ab08d1b80 1 -- 172.21.15.46:0/2098898868 wait complete. 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4ab08d1b80 20 client.26035 trim_cache size 0 max 16384 2024-05-08T23:04:38.127 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "e47ddceb-4424-4f0f-89d5-97e878c00146", 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 msgr2=0x7f4a7c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 0x7f4a7c0a38f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005fd0 tx=0x7f4a88009310 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 msgr2=0x7f4a9819bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000b6c0 tx=0x7f4a90004a50 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 shutdown_connections 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a7c0a1430 0x7f4a7c0a38f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9819bab0 0x7f4a9819bf30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a98101080 0x7f4a981971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 --2- 172.21.15.46:0/3415842799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981061a0 0x7f4a98197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.124+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 >> 172.21.15.46:0/3415842799 conn(0x7f4a980fc8e0 msgr2=0x7f4a980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.125+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 shutdown_connections 2024-05-08T23:04:38.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.125+0000 7f4a9ce00700 1 -- 172.21.15.46:0/3415842799 wait complete. 2024-05-08T23:04:38.139 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:37 smithi046 ceph-mon[36898]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:38.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:04:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:37 smithi086 ceph-mon[78610]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:37 smithi053 ceph-mon[77587]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 -- 172.21.15.46:0/2988150183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb934101080 msgr2=0x7fb934101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 --2- 172.21.15.46:0/2988150183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb934101080 0x7fb934101460 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098f0 tx=0x7fb924009c00 comp rx=0 tx=0).stop 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 -- 172.21.15.46:0/2988150183 shutdown_connections 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 --2- 172.21.15.46:0/2988150183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 0x7fb93410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 --2- 172.21.15.46:0/2988150183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb934101080 0x7fb934101460 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 --2- 172.21.15.46:0/2988150183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101a30 0x7fb934105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 -- 172.21.15.46:0/2988150183 >> 172.21.15.46:0/2988150183 conn(0x7fb9340fc8e0 msgr2=0x7fb9340fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 -- 172.21.15.46:0/2988150183 shutdown_connections 2024-05-08T23:04:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.344+0000 7fb938c00700 1 -- 172.21.15.46:0/2988150183 wait complete. 2024-05-08T23:04:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.345+0000 7fb938c00700 1 Processor -- start 2024-05-08T23:04:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 -- start start 2024-05-08T23:04:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 0x7fb934199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93419dfa0 0x7fb93419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93419ea20 con 0x7fb934101080 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93419eb90 con 0x7fb9341061a0 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb938c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93419ed50 con 0x7fb93419dfa0 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb931e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb931e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46652/0 (socket says 172.21.15.46:46652) 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb931e00700 1 -- 172.21.15.46:0/3229010630 learned_addr learned my addr 172.21.15.46:0/3229010630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb931400700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 0x7fb934199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.346+0000 7fb932800700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93419dfa0 0x7fb93419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 -- 172.21.15.46:0/3229010630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93419dfa0 msgr2=0x7fb93419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93419dfa0 0x7fb93419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 -- 172.21.15.46:0/3229010630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 msgr2=0x7fb934199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 0x7fb934199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 -- 172.21.15.46:0/3229010630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924009520 con 0x7fb934101080 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb931e00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb924005950 tx=0x7fb92400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb92aa00700 1 -- 172.21.15.46:0/3229010630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92401c070 con 0x7fb934101080 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb92aa00700 1 -- 172.21.15.46:0/3229010630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92400ba40 con 0x7fb934101080 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9341a6250 con 0x7fb934101080 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb92aa00700 1 -- 172.21.15.46:0/3229010630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240038e0 con 0x7fb934101080 2024-05-08T23:04:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.347+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9341a6670 con 0x7fb934101080 2024-05-08T23:04:38.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.350+0000 7fb92aa00700 1 -- 172.21.15.46:0/3229010630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136100+0+0 (secure 0 0 0) 0x7fb924020030 con 0x7fb934101080 2024-05-08T23:04:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.352+0000 7fb94c6d2b80 1 Processor -- start 2024-05-08T23:04:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 -- start start 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb92aa00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 0x7fb9180a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb92aa00700 1 -- 172.21.15.46:0/3229010630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb9240ce2b0 con 0x7fb934101080 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb932800700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 0x7fb9180a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 0x55a24a1ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a13f150 con 0x55a24a1ee720 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a13cb10 con 0x55a24a1e8890 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a138400 con 0x55a24a1ede00 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb932800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 0x55a24a1ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46662/0 (socket says 172.21.15.46:46662) 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.353+0000 7fb931400700 1 -- 172.21.15.46:0/605611648 learned_addr learned my addr 172.21.15.46:0/605611648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb932800700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 0x7fb9180a3870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00f5a0 tx=0x7fb92c00f960 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 -- 172.21.15.46:0/605611648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 msgr2=0x55a24a1e8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1e8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 -- 172.21.15.46:0/605611648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 msgr2=0x55a24a1e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 -- 172.21.15.46:0/605611648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a249fb93b0 con 0x55a24a1ede00 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931400700 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb931e00700 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 0x55a24a1ee1e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cb470 tx=0x7fb9240054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cba679fd0ea8c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb91f000700 1 -- 172.21.15.46:0/605611648 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ce070 con 0x55a24a1ede00 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb91f000700 1 -- 172.21.15.46:0/605611648 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92409adf0 con 0x55a24a1ede00 2024-05-08T23:04:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.354+0000 7fb91f000700 1 -- 172.21.15.46:0/605611648 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924005140 con 0x55a24a1ede00 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/605611648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 msgr2=0x55a24a1ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 0x55a24a1ee1e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cb470 tx=0x7fb9240054e0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/605611648 shutdown_connections 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ede00 0x55a24a1ee1e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/605611648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ee720 0x55a24a1e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/605611648 >> 172.21.15.46:0/605611648 conn(0x55a24a146210 msgr2=0x55a24a1e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/605611648 shutdown_connections 2024-05-08T23:04:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.355+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/605611648 wait complete. 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.356+0000 7fb94c6d2b80 1 -- start start 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.356+0000 7fb94c6d2b80 1 Processor -- start 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 0x55a24a1f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a0d4560 con 0x55a24a1ede00 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a0d4200 con 0x55a24a1e8890 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a249fbb6a0 con 0x55a24a1ee720 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb94c6d2b80 10 client.0 _ll_register_callbacks cb 0x55a24a0ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46666/0 (socket says 172.21.15.46:46666) 2024-05-08T23:04:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 0x55a24a1f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb931400700 1 -- 172.21.15.46:0/972783543 learned_addr learned my addr 172.21.15.46:0/972783543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb931e00700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 -- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 msgr2=0x55a24a1f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 -- 172.21.15.46:0/972783543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 msgr2=0x55a24a1f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb932800700 1 -- 172.21.15.46:0/972783543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb920009920 con 0x55a24a1ee720 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.357+0000 7fb931400700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb932800700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 0x55a24a1f20f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00fd60 tx=0x7fb92c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd1528a47c23dea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb913400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c028070 con 0x55a24a1ee720 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a24a091270 con 0x55a24a1ee720 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb92c020040 con 0x55a24a1ee720 2024-05-08T23:04:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c00e040 con 0x55a24a1ee720 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb94c6d2b80 10 client.26047 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb9341a6670 con 0x55a24a1ee720 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.358+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a24a091270 con 0x55a24a1ee720 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb92c02b030 con 0x55a24a1ee720 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 20 client.26047 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a201f40 0x55a24a204360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 4 client.26047 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a24a1b7810 con 0x55a24a201f40 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.359+0000 7fb931e00700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a201f40 0x55a24a204360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.360+0000 7fb931e00700 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a201f40 0x55a24a204360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97ef12c941f4e6b server_cookie=612afc0215638a7e in_seq=0 out_seq=0 2024-05-08T23:04:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.360+0000 7fb913400700 10 client.26047 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.360+0000 7fb913400700 1 -- 172.21.15.46:0/972783543 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a24a1b7810 con 0x55a24a201f40 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.360+0000 7fb913400700 10 client.26047 handle_command_reply: tid=0 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 client.26047 shutdown 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 msgr2=0x55a24a1f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 0x55a24a1f20f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00fd60 tx=0x7fb92c004b40 comp rx=0 tx=0).stop 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 shutdown_connections 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1ee720 0x55a24a1f20f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1e8890 0x55a24a1f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a201f40 0x55a24a204360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/972783543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 >> 172.21.15.46:0/972783543 conn(0x55a24a146210 msgr2=0x55a24a1e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb913400700 0 client.26047 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 shutdown_connections 2024-05-08T23:04:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/972783543 wait complete. 2024-05-08T23:04:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.361+0000 7fb94c6d2b80 20 client.26047 trim_cache size 0 max 16384 2024-05-08T23:04:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 Processor -- start 2024-05-08T23:04:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 -- start start 2024-05-08T23:04:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a1f2e70 con 0x55a24a1ede00 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a1dd800 con 0x55a24a1ee720 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a2095a0 con 0x55a24a1e8890 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.379+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53134/0 (socket says 172.21.15.46:53134) 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 -- 172.21.15.46:0/2082537094 learned_addr learned my addr 172.21.15.46:0/2082537094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb932800700 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 -- 172.21.15.46:0/2082537094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 msgr2=0x55a24a1e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 -- 172.21.15.46:0/2082537094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 msgr2=0x55a24a1df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 -- 172.21.15.46:0/2082537094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a249f65b60 con 0x55a24a1e8890 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb932800700 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb931e00700 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098c0 tx=0x7fb924009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5e2431cbbf7cef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb91fa00700 1 -- 172.21.15.46:0/2082537094 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400bbb0 con 0x55a24a1e8890 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb91fa00700 1 -- 172.21.15.46:0/2082537094 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92409ec20 con 0x55a24a1e8890 2024-05-08T23:04:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.380+0000 7fb91fa00700 1 -- 172.21.15.46:0/2082537094 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92409b050 con 0x55a24a1e8890 2024-05-08T23:04:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/2082537094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 msgr2=0x55a24a1fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098c0 tx=0x7fb924009260 comp rx=0 tx=0).stop 2024-05-08T23:04:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/2082537094 shutdown_connections 2024-05-08T23:04:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1fd990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/2082537094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/2082537094 >> 172.21.15.46:0/2082537094 conn(0x55a24a146210 msgr2=0x55a24a1f1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/2082537094 shutdown_connections 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/2082537094 wait complete. 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.381+0000 7fb94c6d2b80 1 -- start start 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 Processor -- start 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a1ea4b0 con 0x55a24a1ede00 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a1ea620 con 0x55a24a1ee720 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24a1ea7e0 con 0x55a24a1e8890 2024-05-08T23:04:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb94c6d2b80 10 client.0 _ll_register_callbacks cb 0x55a24a0ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53146/0 (socket says 172.21.15.46:53146) 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb931e00700 1 -- 172.21.15.46:0/1252669433 learned_addr learned my addr 172.21.15.46:0/1252669433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb931400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46678/0 (socket says 172.21.15.46:46678) 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.382+0000 7fb932800700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 -- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 msgr2=0x55a24a1e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 -- 172.21.15.46:0/1252669433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 msgr2=0x55a24a1e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 -- 172.21.15.46:0/1252669433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9341a6670 con 0x55a24a1e8890 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931400700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb931e00700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9240a2cb0 tx=0x7fb924009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c1edb5f8b2df8a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:04:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb913400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240b8070 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb9240a31e0 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb92409adf0 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.383+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92400bbb0 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 10 client.26053 fetch_fsmap learned FSMap version 27 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a249fb93b0 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92409adf0 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb92409b050 con 0x55a24a1e8890 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 10 client.26053 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 20 client.26053 populate_metadata read hostname 'smithi046' 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a2586e0 0x55a24a25aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 4 client.26053 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:04:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a24a130ec0 con 0x55a24a2586e0 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.384+0000 7fb931400700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a2586e0 0x55a24a25aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb931400700 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a2586e0 0x55a24a25aad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698cc98a0dd59467 server_cookie=749c456b2cb391c9 in_seq=0 out_seq=0 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb913400700 10 client.26053 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb913400700 1 -- 172.21.15.46:0/1252669433 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a24a130ec0 con 0x55a24a2586e0 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb913400700 10 client.26053 handle_command_reply: tid=0 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 client.26053 shutdown 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 msgr2=0x55a24a1f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9240a2cb0 tx=0x7fb924009f90 comp rx=0 tx=0).stop 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 shutdown_connections 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24a1ee720 0x55a24a1e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24a1e8890 0x55a24a1f92a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24a2586e0 0x55a24a25aad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 --2- 172.21.15.46:0/1252669433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24a1ede00 0x55a24a1e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 >> 172.21.15.46:0/1252669433 conn(0x55a24a146210 msgr2=0x55a24a1df210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb913400700 0 client.26053 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 shutdown_connections 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.385+0000 7fb94c6d2b80 1 -- 172.21.15.46:0/1252669433 wait complete. 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb94c6d2b80 20 client.26053 trim_cache size 0 max 16384 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout: "e47ddceb-4424-4f0f-89d5-97e878c00146": { 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:04:38.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "e47ddceb-4424-4f0f-89d5-97e878c00146", 2024-05-08T23:04:38.391 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:04:38.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:04:38.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:04:38.391 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:04:38.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 msgr2=0x7fb9180a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 0x7fb9180a3870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00f5a0 tx=0x7fb92c00f960 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 msgr2=0x7fb9341996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb924005950 tx=0x7fb92400fa00 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 shutdown_connections 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93419dfa0 0x7fb93419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb9180a13b0 0x7fb9180a3870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb934101080 0x7fb9341996c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 --2- 172.21.15.46:0/3229010630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9341061a0 0x7fb934199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 >> 172.21.15.46:0/3229010630 conn(0x7fb9340fc8e0 msgr2=0x7fb9340fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 shutdown_connections 2024-05-08T23:04:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:04:38.386+0000 7fb938c00700 1 -- 172.21.15.46:0/3229010630 wait complete. 2024-05-08T23:04:38.403 INFO:tasks.cephfs.filesystem:scrub status for tag:e47ddceb-4424-4f0f-89d5-97e878c00146 - {'path': '/', 'tag': 'e47ddceb-4424-4f0f-89d5-97e878c00146', 'options': 'recursive,force'} 2024-05-08T23:04:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:39 smithi086 ceph-mon[78610]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:39 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:04:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:39 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:04:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:39 smithi053 ceph-mon[77587]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:39 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:04:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:39 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:04:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:40.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:39 smithi046 ceph-mon[36898]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:40.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:39 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:04:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:39 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:04:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:04:41.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:04:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:04:41.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:04:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:41 smithi086 ceph-mon[78610]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:41 smithi053 ceph-mon[77587]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:41 smithi046 ceph-mon[36898]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:43 smithi086 ceph-mon[78610]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:43 smithi053 ceph-mon[77587]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:43 smithi046 ceph-mon[36898]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:45 smithi086 ceph-mon[78610]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:45 smithi053 ceph-mon[77587]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:46.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:45 smithi046 ceph-mon[36898]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:47 smithi086 ceph-mon[78610]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-08T23:04:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:47 smithi053 ceph-mon[77587]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-08T23:04:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:47 smithi046 ceph-mon[36898]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-08T23:04:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:49 smithi086 ceph-mon[78610]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:49 smithi053 ceph-mon[77587]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:49 smithi046 ceph-mon[36898]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T23:04:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:51 smithi086 ceph-mon[78610]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:51 smithi053 ceph-mon[77587]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:52.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:51 smithi046 ceph-mon[36898]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T23:04:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:53 smithi086 ceph-mon[78610]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:53 smithi053 ceph-mon[77587]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:54.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:53 smithi046 ceph-mon[36898]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:55.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:54 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:04:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:54 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:04:55.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:54 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:04:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:55 smithi086 ceph-mon[78610]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:55 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:04:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:55 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:04:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:55 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:04:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:55 smithi053 ceph-mon[77587]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:55 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:04:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:55 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:04:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:55 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:04:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:55 smithi046 ceph-mon[36898]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:04:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:55 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:04:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:55 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:04:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:55 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:04:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:57 smithi086 ceph-mon[78610]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:57 smithi053 ceph-mon[77587]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:04:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:57 smithi046 ceph-mon[36898]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:05:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:04:59 smithi086 ceph-mon[78610]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:04:59 smithi053 ceph-mon[77587]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:00.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:04:59 smithi046 ceph-mon[36898]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:00.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:05:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:05:00.021+0000 7fcf1d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:05:00.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:05:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:05:00.021+0000 7fcf1d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:05:00.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:05:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:05:00.021+0000 7fcf1d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:05:00.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:05:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:05:00.021+0000 7fcf1d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:05:00.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:05:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:05:00.021+0000 7fcf1d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:05:02.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:01 smithi086 ceph-mon[78610]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:05:02.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:01 smithi086 ceph-mon[78610]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-08T23:05:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:01 smithi053 ceph-mon[77587]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:05:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:01 smithi053 ceph-mon[77587]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-08T23:05:02.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:01 smithi046 ceph-mon[36898]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T23:05:02.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:01 smithi046 ceph-mon[36898]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-08T23:05:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:03 smithi086 ceph-mon[78610]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:03 smithi053 ceph-mon[77587]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:03 smithi046 ceph-mon[36898]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:05 smithi086 ceph-mon[78610]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-05-08T23:05:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:05 smithi053 ceph-mon[77587]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-05-08T23:05:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:05 smithi046 ceph-mon[36898]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-05-08T23:05:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:07 smithi086 ceph-mon[78610]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:07 smithi053 ceph-mon[77587]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:08.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:07 smithi046 ceph-mon[36898]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:08.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:05:08.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 -- 172.21.15.46:0/4269560690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac80686a0 msgr2=0x7f2ac8068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 --2- 172.21.15.46:0/4269560690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac80686a0 0x7f2ac8068b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009990 tx=0x7f2ab4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 -- 172.21.15.46:0/4269560690 shutdown_connections 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 --2- 172.21.15.46:0/4269560690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac8103680 0x7f2ac8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 --2- 172.21.15.46:0/4269560690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 0x7f2ac8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 --2- 172.21.15.46:0/4269560690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac80686a0 0x7f2ac8068b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.677+0000 7f2acd200700 1 -- 172.21.15.46:0/4269560690 >> 172.21.15.46:0/4269560690 conn(0x7f2ac8075710 msgr2=0x7f2ac8075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.678+0000 7f2acd200700 1 -- 172.21.15.46:0/4269560690 shutdown_connections 2024-05-08T23:05:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.678+0000 7f2acd200700 1 -- 172.21.15.46:0/4269560690 wait complete. 2024-05-08T23:05:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.679+0000 7f2acd200700 1 Processor -- start 2024-05-08T23:05:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.679+0000 7f2acd200700 1 -- start start 2024-05-08T23:05:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.679+0000 7f2acd200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.679+0000 7f2acd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac8103680 0x7f2ac819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2acd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 0x7f2ac81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2acd200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac81a2490 con 0x7f2ac80686a0 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2acd200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac81a2600 con 0x7f2ac8105bf0 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2acd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac81a27c0 con 0x7f2ac8103680 2024-05-08T23:05:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56452/0 (socket says 172.21.15.46:56452) 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 -- 172.21.15.46:0/3145244393 learned_addr learned my addr 172.21.15.46:0/3145244393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 0x7f2ac81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac8103680 0x7f2ac819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 -- 172.21.15.46:0/3145244393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac8103680 msgr2=0x7f2ac819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac8103680 0x7f2ac819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 -- 172.21.15.46:0/3145244393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 msgr2=0x7f2ac81a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 0x7f2ac81a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.680+0000 7f2ac7400700 1 -- 172.21.15.46:0/3145244393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8009520 con 0x7f2ac80686a0 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800eb30 tx=0x7f2ab800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2abfe00700 1 -- 172.21.15.46:0/3145244393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8009880 con 0x7f2ac80686a0 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2abfe00700 1 -- 172.21.15.46:0/3145244393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab80044e0 con 0x7f2ac80686a0 2024-05-08T23:05:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2abfe00700 1 -- 172.21.15.46:0/3145244393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8003d80 con 0x7f2ac80686a0 2024-05-08T23:05:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab40095f0 con 0x7f2ac80686a0 2024-05-08T23:05:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.681+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac81aa100 con 0x7f2ac80686a0 2024-05-08T23:05:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.686+0000 7f2abfe00700 1 -- 172.21.15.46:0/3145244393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f2ab8003f20 con 0x7f2ac80686a0 2024-05-08T23:05:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.687+0000 7f2abfe00700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 0x7f2aac0a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 Processor -- start 2024-05-08T23:05:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2abfe00700 1 -- 172.21.15.46:0/3145244393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2ab8014070 con 0x7f2ac80686a0 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 0x7f2aac0a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 -- start start 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969414350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a96936b150 con 0x55a969414890 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969368b10 con 0x55a96941a970 2024-05-08T23:05:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.688+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969364400 con 0x55a96941a050 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43252/0 (socket says 172.21.15.46:43252) 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 -- 172.21.15.46:0/2457930560 learned_addr learned my addr 172.21.15.46:0/2457930560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 -- 172.21.15.46:0/2457930560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 msgr2=0x55a969414350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969414350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 -- 172.21.15.46:0/2457930560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 msgr2=0x55a969414cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 -- 172.21.15.46:0/2457930560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9691e53b0 con 0x55a96941a050 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 0x7f2aac0a39a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ac819eac0 tx=0x7f2ab400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.689+0000 7f2ac7400700 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80a22a0 tx=0x7f2ab8099d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a1b103765466d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.690+0000 7f2abcc00700 1 -- 172.21.15.46:0/2457930560 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab809a350 con 0x55a96941a050 2024-05-08T23:05:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.690+0000 7f2abcc00700 1 -- 172.21.15.46:0/2457930560 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab809e180 con 0x55a96941a050 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.690+0000 7f2abcc00700 1 -- 172.21.15.46:0/2457930560 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80cd440 con 0x55a96941a050 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 -- 172.21.15.46:0/2457930560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 msgr2=0x55a96941a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80a22a0 tx=0x7f2ab8099d80 comp rx=0 tx=0).stop 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 -- 172.21.15.46:0/2457930560 shutdown_connections 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a050 0x55a96941a430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/2457930560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969414350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 -- 172.21.15.46:0/2457930560 >> 172.21.15.46:0/2457930560 conn(0x55a969372210 msgr2=0x55a96940f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 -- 172.21.15.46:0/2457930560 shutdown_connections 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.691+0000 7f2ae23afb80 1 -- 172.21.15.46:0/2457930560 wait complete. 2024-05-08T23:05:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.692+0000 7f2ae23afb80 1 -- start start 2024-05-08T23:05:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.692+0000 7f2ae23afb80 1 Processor -- start 2024-05-08T23:05:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 0x55a96941dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a96941e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969300560 con 0x55a96941a970 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969300200 con 0x55a96941a050 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9691e76a0 con 0x55a969414890 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ae23afb80 10 client.0 _ll_register_callbacks cb 0x55a969309b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56474/0 (socket says 172.21.15.46:56474) 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ac7e00700 1 -- 172.21.15.46:0/3076444690 learned_addr learned my addr 172.21.15.46:0/3076444690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 0x55a96941dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.693+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a96941e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 -- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 msgr2=0x55a96941e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a96941e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 -- 172.21.15.46:0/3076444690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 msgr2=0x55a96941e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 -- 172.21.15.46:0/3076444690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac81aa100 con 0x55a969414890 2024-05-08T23:05:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ac7400700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 0x55a96941dd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8096dc0 tx=0x7f2ab80ca980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f5ca4675903baf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ab1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80cd640 con 0x55a969414890 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2ab80a27d0 con 0x55a969414890 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9692bd270 con 0x55a969414890 2024-05-08T23:05:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.694+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ab809a350 con 0x55a969414890 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 10 client.26071 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a969191b60 con 0x55a969414890 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9692bd270 con 0x55a969414890 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ab8063070 con 0x55a969414890 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 10 client.26071 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 20 client.26071 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a96942dfe0 0x55a9694303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 4 client.26071 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.695+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9693e3810 con 0x55a96942dfe0 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.696+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a96942dfe0 0x55a9694303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.696+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a96942dfe0 0x55a9694303d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e68ddf054083bd server_cookie=e31dc02adff306e2 in_seq=0 out_seq=0 2024-05-08T23:05:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.696+0000 7f2ab1600700 10 client.26071 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ab1600700 1 -- 172.21.15.46:0/3076444690 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9693e3810 con 0x55a96942dfe0 2024-05-08T23:05:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ab1600700 10 client.26071 handle_command_reply: tid=0 2024-05-08T23:05:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 client.26071 shutdown 2024-05-08T23:05:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 msgr2=0x55a96941dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 0x55a96941dd70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8096dc0 tx=0x7f2ab80ca980 comp rx=0 tx=0).stop 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 shutdown_connections 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a96942dfe0 0x55a9694303d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a970 0x55a96941e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a969414890 0x55a96941dd70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/3076444690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a96941e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ab1600700 0 client.26071 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.697+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 >> 172.21.15.46:0/3076444690 conn(0x55a969372210 msgr2=0x55a96940f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.698+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 shutdown_connections 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.698+0000 7f2ae23afb80 1 -- 172.21.15.46:0/3076444690 wait complete. 2024-05-08T23:05:08.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.698+0000 7f2ae23afb80 20 client.26071 trim_cache size 0 max 16384 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.731+0000 7f2ae23afb80 1 Processor -- start 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.731+0000 7f2ae23afb80 1 -- start start 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a969429de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a970 0x55a96941e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a96942af60 con 0x55a969414890 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969410820 con 0x55a96941a050 2024-05-08T23:05:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969435570 con 0x55a96941a970 2024-05-08T23:05:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56484/0 (socket says 172.21.15.46:56484) 2024-05-08T23:05:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ac7400700 1 -- 172.21.15.46:0/4157170040 learned_addr learned my addr 172.21.15.46:0/4157170040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a969429de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.732+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a970 0x55a96941e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 -- 172.21.15.46:0/4157170040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a970 msgr2=0x55a96941e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a970 0x55a96941e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 -- 172.21.15.46:0/4157170040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 msgr2=0x55a969429de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a969429de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 -- 172.21.15.46:0/4157170040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a96932a560 con 0x55a969414890 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a969429de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2ac7400700 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80986a0 tx=0x7f2ab8099d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e195f5c6f22a93c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2abd600700 1 -- 172.21.15.46:0/4157170040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab807d070 con 0x55a969414890 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.733+0000 7f2abd600700 1 -- 172.21.15.46:0/4157170040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab8096690 con 0x55a969414890 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.734+0000 7f2abd600700 1 -- 172.21.15.46:0/4157170040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80cd680 con 0x55a969414890 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.734+0000 7f2ae23afb80 1 -- 172.21.15.46:0/4157170040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 msgr2=0x55a969414350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.734+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80986a0 tx=0x7f2ab8099d80 comp rx=0 tx=0).stop 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 -- 172.21.15.46:0/4157170040 shutdown_connections 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a96941a970 0x55a96941e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a969414890 0x55a969414350 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/4157170040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a050 0x55a969429de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 -- 172.21.15.46:0/4157170040 >> 172.21.15.46:0/4157170040 conn(0x55a969372210 msgr2=0x55a969417e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 -- 172.21.15.46:0/4157170040 shutdown_connections 2024-05-08T23:05:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 -- 172.21.15.46:0/4157170040 wait complete. 2024-05-08T23:05:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.735+0000 7f2ae23afb80 1 -- start start 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 Processor -- start 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9694239b0 0x55a969483440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969423fd0 con 0x55a96941a050 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969424140 con 0x55a96941a970 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a969483980 con 0x55a9694239b0 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.736+0000 7f2ae23afb80 10 client.0 _ll_register_callbacks cb 0x55a969309b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42588/0 (socket says 172.21.15.46:42588) 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac6a00700 1 -- 172.21.15.46:0/861783292 learned_addr learned my addr 172.21.15.46:0/861783292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56488/0 (socket says 172.21.15.46:56488) 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7e00700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9694239b0 0x55a969483440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 -- 172.21.15.46:0/861783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9694239b0 msgr2=0x55a969483440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9694239b0 0x55a969483440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 -- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 msgr2=0x55a969423310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac7400700 1 -- 172.21.15.46:0/861783292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac8104500 con 0x55a96941a050 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.737+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ac7400700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a9694210b0 tx=0x7f2ab80cd6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d2003806d36c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ab1600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab807d070 con 0x55a96941a050 2024-05-08T23:05:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2ab80a8060 con 0x55a96941a050 2024-05-08T23:05:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ab809e180 con 0x55a96941a050 2024-05-08T23:05:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ab80a27d0 con 0x55a96941a050 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ae23afb80 10 client.16062 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.738+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a969191b60 con 0x55a96941a050 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab809e180 con 0x55a96941a050 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ab8063070 con 0x55a96941a050 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 20 client.16062 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a969484b90 0x55a969486f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 4 client.16062 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a96935cec0 con 0x55a969484b90 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.739+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a969484b90 0x55a969486f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ab1600700 10 client.16062 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ac6a00700 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a969484b90 0x55a969486f80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923445eb82e61a16 server_cookie=4f44600c66eb8105 in_seq=0 out_seq=0 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ab1600700 1 -- 172.21.15.46:0/861783292 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a96935cec0 con 0x55a969484b90 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ab1600700 10 client.16062 handle_command_reply: tid=0 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ae23afb80 1 client.16062 shutdown 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 msgr2=0x55a969425830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.740+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a9694210b0 tx=0x7f2ab80cd6d0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 shutdown_connections 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9694239b0 0x55a969483440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a969484b90 0x55a969486f80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a96941a050 0x55a969425830 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 --2- 172.21.15.46:0/861783292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a96941a970 0x55a969423310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ab1600700 0 client.16062 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 >> 172.21.15.46:0/861783292 conn(0x55a969372210 msgr2=0x55a96941c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 shutdown_connections 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 1 -- 172.21.15.46:0/861783292 wait complete. 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.741+0000 7f2ae23afb80 20 client.16062 trim_cache size 0 max 16384 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:05:08.744 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 msgr2=0x7f2aac0a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 0x7f2aac0a39a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ac819eac0 tx=0x7f2ab400b260 comp rx=0 tx=0).stop 2024-05-08T23:05:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 msgr2=0x7f2ac819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800eb30 tx=0x7f2ab800ee40 comp rx=0 tx=0).stop 2024-05-08T23:05:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 shutdown_connections 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac8105bf0 0x7f2ac81a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2aac0a14e0 0x7f2aac0a39a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ac80686a0 0x7f2ac819d5c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 --2- 172.21.15.46:0/3145244393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac8103680 0x7f2ac819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 >> 172.21.15.46:0/3145244393 conn(0x7f2ac8075710 msgr2=0x7f2ac81024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 shutdown_connections 2024-05-08T23:05:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:08.742+0000 7f2acd200700 1 -- 172.21.15.46:0/3145244393 wait complete. 2024-05-08T23:05:08.765 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:05:08.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.018+0000 7f5041800700 1 -- 172.21.15.46:0/1288859805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c105bf0 msgr2=0x7f503c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.018+0000 7f5041800700 1 --2- 172.21.15.46:0/1288859805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c105bf0 0x7f503c105fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f502c00dc80 tx=0x7f502c00df90 comp rx=0 tx=0).stop 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 -- 172.21.15.46:0/1288859805 shutdown_connections 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 --2- 172.21.15.46:0/1288859805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c1007a0 0x7f503c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 --2- 172.21.15.46:0/1288859805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c105bf0 0x7f503c105fd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 --2- 172.21.15.46:0/1288859805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 -- 172.21.15.46:0/1288859805 >> 172.21.15.46:0/1288859805 conn(0x7f503c075690 msgr2=0x7f503c075aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 -- 172.21.15.46:0/1288859805 shutdown_connections 2024-05-08T23:05:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.019+0000 7f5041800700 1 -- 172.21.15.46:0/1288859805 wait complete. 2024-05-08T23:05:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.020+0000 7f5041800700 1 Processor -- start 2024-05-08T23:05:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.020+0000 7f5041800700 1 -- start start 2024-05-08T23:05:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 0x7f503c195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c105bf0 0x7f503c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c199e20 con 0x7f503c105bf0 2024-05-08T23:05:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c199f90 con 0x7f503c1007a0 2024-05-08T23:05:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f5041800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c19a150 con 0x7f503c0ffde0 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f503b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 0x7f503c195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f503be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f503b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 0x7f503c195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42602/0 (socket says 172.21.15.46:42602) 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.021+0000 7f503b400700 1 -- 172.21.15.46:0/4100822220 learned_addr learned my addr 172.21.15.46:0/4100822220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f5040e00700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c105bf0 0x7f503c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 -- 172.21.15.46:0/4100822220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 msgr2=0x7f503c195440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 0x7f503c195440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 -- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c105bf0 msgr2=0x7f503c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c105bf0 0x7f503c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 -- 172.21.15.46:0/4100822220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280095f0 con 0x7f503c0ffde0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f503be00700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c194f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f502c019000 tx=0x7f502c00ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.022+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c012040 con 0x7f503c0ffde0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.023+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c00d450 con 0x7f503c0ffde0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.023+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c003be0 con 0x7f503c0ffde0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.023+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c00d930 con 0x7f503c0ffde0 2024-05-08T23:05:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.023+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c1a1a90 con 0x7f503c0ffde0 2024-05-08T23:05:09.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.024+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c108070 con 0x7f503c0ffde0 2024-05-08T23:05:09.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.028+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f502c00f8e0 con 0x7f503c0ffde0 2024-05-08T23:05:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.030+0000 7f5038c00700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 0x7f50200a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.030+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f502c0d2110 con 0x7f503c0ffde0 2024-05-08T23:05:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.030+0000 7f503b400700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 0x7f50200a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.031+0000 7f503b400700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 0x7f50200a3980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f503c196450 tx=0x7f5028009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.033+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f502c09b000 con 0x7f503c0ffde0 2024-05-08T23:05:09.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.496+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f503c196200 con 0x7f503c0ffde0 2024-05-08T23:05:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.497+0000 7f5038c00700 1 -- 172.21.15.46:0/4100822220 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f502c09e620 con 0x7f503c0ffde0 2024-05-08T23:05:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:05:09.497 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:03:50.080154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:05:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 msgr2=0x7f50200a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 0x7f50200a3980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f503c196450 tx=0x7f5028009310 comp rx=0 tx=0).stop 2024-05-08T23:05:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 msgr2=0x7f503c194f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c194f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f502c019000 tx=0x7f502c00ca70 comp rx=0 tx=0).stop 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 shutdown_connections 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f50200a14c0 0x7f50200a3980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f503c105bf0 0x7f503c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f503c0ffde0 0x7f503c194f00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 --2- 172.21.15.46:0/4100822220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f503c1007a0 0x7f503c195440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 >> 172.21.15.46:0/4100822220 conn(0x7f503c075690 msgr2=0x7f503c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 shutdown_connections 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.501+0000 7f5041800700 1 -- 172.21.15.46:0/4100822220 wait complete. 2024-05-08T23:05:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-05-08T23:05:09.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:05:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 -- 172.21.15.46:0/3653954278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1800ffc00 msgr2=0x7fa1800fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 --2- 172.21.15.46:0/3653954278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1800ffc00 0x7fa1800fffe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa170009990 tx=0x7fa170009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 -- 172.21.15.46:0/3653954278 shutdown_connections 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 --2- 172.21.15.46:0/3653954278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa180105000 0x7fa1801054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 --2- 172.21.15.46:0/3653954278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1800ffc00 0x7fa1800fffe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 --2- 172.21.15.46:0/3653954278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa1801049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 -- 172.21.15.46:0/3653954278 >> 172.21.15.46:0/3653954278 conn(0x7fa1800fb5f0 msgr2=0x7fa1800fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 -- 172.21.15.46:0/3653954278 shutdown_connections 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.735+0000 7fa17fe00700 1 -- 172.21.15.46:0/3653954278 wait complete. 2024-05-08T23:05:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.736+0000 7fa17fe00700 1 Processor -- start 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 -- start start 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1800ffc00 0x7fa18019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 0x7fa1801a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a2520 con 0x7fa1800ffc00 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a2690 con 0x7fa180105000 2024-05-08T23:05:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801a2850 con 0x7fa180100520 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43292/0 (socket says 172.21.15.46:43292) 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.737+0000 7fa17e000700 1 -- 172.21.15.46:0/2153709992 learned_addr learned my addr 172.21.15.46:0/2153709992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 0x7fa1801a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1800ffc00 0x7fa18019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 -- 172.21.15.46:0/2153709992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 msgr2=0x7fa18019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 -- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1800ffc00 msgr2=0x7fa18019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1800ffc00 0x7fa18019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 -- 172.21.15.46:0/2153709992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700095f0 con 0x7fa180105000 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17e000700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa17f400700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 0x7fa1801a1ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa17800efc0 tx=0x7fa17800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.738+0000 7fa184a00700 1 -- 172.21.15.46:0/2153709992 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178018070 con 0x7fa180105000 2024-05-08T23:05:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.739+0000 7fa184a00700 1 -- 172.21.15.46:0/2153709992 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17800f040 con 0x7fa180105000 2024-05-08T23:05:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.739+0000 7fa184a00700 1 -- 172.21.15.46:0/2153709992 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800cd40 con 0x7fa180105000 2024-05-08T23:05:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.739+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1801a2ad0 con 0x7fa180105000 2024-05-08T23:05:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.739+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801aa370 con 0x7fa180105000 2024-05-08T23:05:09.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.742+0000 7fa184a00700 1 -- 172.21.15.46:0/2153709992 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fa178003a00 con 0x7fa180105000 2024-05-08T23:05:09.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.744+0000 7fa184a00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 0x7fa1640a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.744+0000 7fa184a00700 1 -- 172.21.15.46:0/2153709992 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa1780ccdd0 con 0x7fa180105000 2024-05-08T23:05:09.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.744+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 0x7fa1640a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.745+0000 7fa1998aab80 1 Processor -- start 2024-05-08T23:05:09.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.745+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 0x7fa1640a3a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b2e0 tx=0x7fa170005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.745+0000 7fa1998aab80 1 -- start start 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ea49150 con 0x55cc1eaf8970 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ea46b10 con 0x55cc1eaf2890 2024-05-08T23:05:09.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa1998aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ea42400 con 0x55cc1eaf8050 2024-05-08T23:05:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa17ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa17f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa17e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa17f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42622/0 (socket says 172.21.15.46:42622) 2024-05-08T23:05:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.746+0000 7fa17f400700 1 -- 172.21.15.46:0/1077652083 learned_addr learned my addr 172.21.15.46:0/1077652083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 -- 172.21.15.46:0/1077652083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 msgr2=0x55cc1eaf8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 -- 172.21.15.46:0/1077652083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8970 msgr2=0x55cc1eaf2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 -- 172.21.15.46:0/1077652083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1e8c33b0 con 0x55cc1eaf2890 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17ea00700 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa17f400700 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1780a1840 tx=0x7fa178099750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25c5e1812dce334 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa16be00700 1 -- 172.21.15.46:0/1077652083 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17801b320 con 0x55cc1eaf2890 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa16be00700 1 -- 172.21.15.46:0/1077652083 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa178004ad0 con 0x55cc1eaf2890 2024-05-08T23:05:09.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.747+0000 7fa16be00700 1 -- 172.21.15.46:0/1077652083 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178098e70 con 0x55cc1eaf2890 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 -- 172.21.15.46:0/1077652083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 msgr2=0x55cc1eaf2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1780a1840 tx=0x7fa178099750 comp rx=0 tx=0).stop 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 -- 172.21.15.46:0/1077652083 shutdown_connections 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2cd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 --2- 172.21.15.46:0/1077652083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.748+0000 7fa1998aab80 1 -- 172.21.15.46:0/1077652083 >> 172.21.15.46:0/1077652083 conn(0x55cc1ea50210 msgr2=0x55cc1eaed2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.749+0000 7fa1998aab80 1 -- 172.21.15.46:0/1077652083 shutdown_connections 2024-05-08T23:05:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.749+0000 7fa1998aab80 1 -- 172.21.15.46:0/1077652083 wait complete. 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.749+0000 7fa1998aab80 1 -- start start 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 Processor -- start 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf2890 0x55cc1eafe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8050 0x55cc1eafbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1e9de560 con 0x55cc1eaf8050 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1e9de200 con 0x55cc1eaf8970 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1e8c56a0 con 0x55cc1eaf2890 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa1998aab80 10 client.0 _ll_register_callbacks cb 0x55cc1e9e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.750+0000 7fa17f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42636/0 (socket says 172.21.15.46:42636) 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 -- 172.21.15.46:0/3460320030 learned_addr learned my addr 172.21.15.46:0/3460320030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17ea00700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf2890 0x55cc1eafe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 -- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf2890 msgr2=0x55cc1eafe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf2890 0x55cc1eafe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 -- 172.21.15.46:0/3460320030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8050 msgr2=0x55cc1eafbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8050 0x55cc1eafbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.751+0000 7fa17f400700 1 -- 172.21.15.46:0/3460320030 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1801149b0 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa17f400700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa178004c60 tx=0x7fa178099e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0bb120550525fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa168c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17801b320 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa178099980 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc1e99b270 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17809a080 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa1998aab80 10 client.26060 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.752+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa1801aa370 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc1e99b270 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1780988e0 con 0x55cc1eaf8970 2024-05-08T23:05:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 10 client.26060 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 10 client.26060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 20 client.26060 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb0bec0 0x55cc1eb0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 4 client.26060 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc1eac1810 con 0x55cc1eb0bec0 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.753+0000 7fa17ea00700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb0bec0 0x55cc1eb0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.754+0000 7fa17ea00700 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb0bec0 0x55cc1eb0e2b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=210b4839d338f250 server_cookie=693205550131bdb8 in_seq=0 out_seq=0 2024-05-08T23:05:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.754+0000 7fa168c00700 10 client.26060 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa168c00700 1 -- 172.21.15.46:0/3460320030 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc1eac1810 con 0x55cc1eb0bec0 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa168c00700 10 client.26060 handle_command_reply: tid=0 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 client.26060 shutdown 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 msgr2=0x55cc1eafc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa178004c60 tx=0x7fa178099e90 comp rx=0 tx=0).stop 2024-05-08T23:05:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 shutdown_connections 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb0bec0 0x55cc1eb0e2b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8970 0x55cc1eafc2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf2890 0x55cc1eafe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa168c00700 0 client.26060 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 --2- 172.21.15.46:0/3460320030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf8050 0x55cc1eafbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 >> 172.21.15.46:0/3460320030 conn(0x55cc1ea50210 msgr2=0x55cc1eaed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 shutdown_connections 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.755+0000 7fa1998aab80 1 -- 172.21.15.46:0/3460320030 wait complete. 2024-05-08T23:05:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.756+0000 7fa1998aab80 20 client.26060 trim_cache size 0 max 16384 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.770+0000 7fa1998aab80 1 Processor -- start 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 -- start start 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaeddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eae7800 con 0x55cc1eaf2890 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eafed50 con 0x55cc1eaf8050 2024-05-08T23:05:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa1998aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eae5040 con 0x55cc1eaf8970 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa17ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.771+0000 7fa17ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56564/0 (socket says 172.21.15.46:56564) 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 -- 172.21.15.46:0/308498397 learned_addr learned my addr 172.21.15.46:0/308498397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17e000700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaeddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17f400700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 -- 172.21.15.46:0/308498397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 msgr2=0x55cc1eaee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 -- 172.21.15.46:0/308498397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 msgr2=0x55cc1eaeddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaeddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 -- 172.21.15.46:0/308498397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1e86fb60 con 0x55cc1eaf2890 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17e000700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaeddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.772+0000 7fa17ea00700 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa170009990 tx=0x7fa170023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87eedcf68654aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.773+0000 7fa175200700 1 -- 172.21.15.46:0/308498397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000f840 con 0x55cc1eaf2890 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.773+0000 7fa175200700 1 -- 172.21.15.46:0/308498397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17001e040 con 0x55cc1eaf2890 2024-05-08T23:05:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.773+0000 7fa175200700 1 -- 172.21.15.46:0/308498397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170036880 con 0x55cc1eaf2890 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 -- 172.21.15.46:0/308498397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 msgr2=0x55cc1eaf2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa170009990 tx=0x7fa170023000 comp rx=0 tx=0).stop 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 -- 172.21.15.46:0/308498397 shutdown_connections 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eaf2350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 --2- 172.21.15.46:0/308498397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaeddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 -- 172.21.15.46:0/308498397 >> 172.21.15.46:0/308498397 conn(0x55cc1ea50210 msgr2=0x55cc1eaebd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 -- 172.21.15.46:0/308498397 shutdown_connections 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.774+0000 7fa1998aab80 1 -- 172.21.15.46:0/308498397 wait complete. 2024-05-08T23:05:09.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.775+0000 7fa1998aab80 1 -- start start 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.775+0000 7fa1998aab80 1 Processor -- start 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eaf44c0 con 0x55cc1eaf2890 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eaf4630 con 0x55cc1eaf8050 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1eaf47f0 con 0x55cc1eaf8970 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa1998aab80 10 client.0 _ll_register_callbacks cb 0x55cc1e9e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:09.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa17ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa17ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56580/0 (socket says 172.21.15.46:56580) 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa17ea00700 1 -- 172.21.15.46:0/2851335254 learned_addr learned my addr 172.21.15.46:0/2851335254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa17e000700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.776+0000 7fa17f400700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 -- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 msgr2=0x55cc1eaf3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 -- 172.21.15.46:0/2851335254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 msgr2=0x55cc1eaf3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 -- 172.21.15.46:0/2851335254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c007d70 con 0x55cc1eaf2890 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17f400700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17e000700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa17ea00700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b6f0 tx=0x7fa1700363c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8edc55d80bda697 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.777+0000 7fa168c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000f840 con 0x55cc1eaf2890 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa17000f9a0 con 0x55cc1eaf2890 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa178004ad0 con 0x55cc1eaf2890 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa17000fb60 con 0x55cc1eaf2890 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa1998aab80 10 client.16083 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.778+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa1700056d0 con 0x55cc1eaf2890 2024-05-08T23:05:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa178004ad0 con 0x55cc1eaf2890 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa17003f030 con 0x55cc1eaf2890 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 10 client.16083 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 20 client.16083 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb62570 0x55cc1eb64960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 4 client.16083 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.779+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cc1ea3aec0 con 0x55cc1eb62570 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa17e000700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb62570 0x55cc1eb64960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa17e000700 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb62570 0x55cc1eb64960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=363f546e16836cbd server_cookie=b80648b9690abc13 in_seq=0 out_seq=0 2024-05-08T23:05:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa168c00700 10 client.16083 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa168c00700 1 -- 172.21.15.46:0/2851335254 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cc1ea3aec0 con 0x55cc1eb62570 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa168c00700 10 client.16083 handle_command_reply: tid=0 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.780+0000 7fa1998aab80 1 client.16083 shutdown 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 msgr2=0x55cc1eb03520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b6f0 tx=0x7fa1700363c0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 shutdown_connections 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55cc1eb62570 0x55cc1eb64960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1eaf8970 0x55cc1eaf3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc1eaf2890 0x55cc1eb03520 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 --2- 172.21.15.46:0/2851335254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1eaf8050 0x55cc1eaf3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 >> 172.21.15.46:0/2851335254 conn(0x55cc1ea50210 msgr2=0x55cc1eaf5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa168c00700 0 client.16083 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 shutdown_connections 2024-05-08T23:05:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.781+0000 7fa1998aab80 1 -- 172.21.15.46:0/2851335254 wait complete. 2024-05-08T23:05:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.782+0000 7fa1998aab80 20 client.16083 trim_cache size 0 max 16384 2024-05-08T23:05:09.784 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.782+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 msgr2=0x7fa1640a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.782+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 0x7fa1640a3a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b2e0 tx=0x7fa170005fd0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.782+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 msgr2=0x7fa1801a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.782+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 0x7fa1801a1ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa17800efc0 tx=0x7fa17800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 shutdown_connections 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa180105000 0x7fa1801a1ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa1640a1560 0x7fa1640a3a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1800ffc00 0x7fa18019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 --2- 172.21.15.46:0/2153709992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa180100520 0x7fa18019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 >> 172.21.15.46:0/2153709992 conn(0x7fa1800fb5f0 msgr2=0x7fa1801029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 shutdown_connections 2024-05-08T23:05:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:09.783+0000 7fa17fe00700 1 -- 172.21.15.46:0/2153709992 wait complete. 2024-05-08T23:05:09.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:05:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.064+0000 7fceeac00700 1 -- 172.21.15.46:0/4233320431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee4101e10 msgr2=0x7fcee41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.064+0000 7fceeac00700 1 --2- 172.21.15.46:0/4233320431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee4101e10 0x7fcee41021f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fced8009990 tx=0x7fced8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 -- 172.21.15.46:0/4233320431 shutdown_connections 2024-05-08T23:05:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 --2- 172.21.15.46:0/4233320431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee410b230 0x7fcee410d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 --2- 172.21.15.46:0/4233320431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee4101e10 0x7fcee41021f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 --2- 172.21.15.46:0/4233320431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4102730 0x7fcee410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 -- 172.21.15.46:0/4233320431 >> 172.21.15.46:0/4233320431 conn(0x7fcee40fb5f0 msgr2=0x7fcee40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 -- 172.21.15.46:0/4233320431 shutdown_connections 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.065+0000 7fceeac00700 1 -- 172.21.15.46:0/4233320431 wait complete. 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.066+0000 7fceeac00700 1 Processor -- start 2024-05-08T23:05:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 -- start start 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 0x7fcee4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 0x7fcee419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee419e240 con 0x7fcee4101e10 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee419e3b0 con 0x7fcee410b230 2024-05-08T23:05:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.067+0000 7fceeac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee419e570 con 0x7fcee4102730 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 0x7fcee4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 0x7fcee419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56598/0 (socket says 172.21.15.46:56598) 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee9800700 1 -- 172.21.15.46:0/1951876642 learned_addr learned my addr 172.21.15.46:0/1951876642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 -- 172.21.15.46:0/1951876642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 msgr2=0x7fcee4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 0x7fcee4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 -- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 msgr2=0x7fcee4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 -- 172.21.15.46:0/1951876642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80095f0 con 0x7fcee410b230 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 0x7fcee4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fcee9800700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.068+0000 7fceea200700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 0x7fcee419dc00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00efc0 tx=0x7fcedc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.069+0000 7fcee2000700 1 -- 172.21.15.46:0/1951876642 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc018070 con 0x7fcee410b230 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.069+0000 7fcee2000700 1 -- 172.21.15.46:0/1951876642 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcedc00f040 con 0x7fcee410b230 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.069+0000 7fcee2000700 1 -- 172.21.15.46:0/1951876642 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00cd40 con 0x7fcee410b230 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.069+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee419e7f0 con 0x7fcee410b230 2024-05-08T23:05:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.069+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee41a6090 con 0x7fcee410b230 2024-05-08T23:05:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.073+0000 7fceffc40b80 1 Processor -- start 2024-05-08T23:05:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.074+0000 7fcee2000700 1 -- 172.21.15.46:0/1951876642 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fcedc003a00 con 0x7fcee410b230 2024-05-08T23:05:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.074+0000 7fceffc40b80 1 -- start start 2024-05-08T23:05:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.074+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48050 0x55872ac48430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.074+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.074+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac42890 0x55872ac42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fceffc40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ab99150 con 0x55872ac48970 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fceffc40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ab96b10 con 0x55872ac42890 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fceffc40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ab92400 con 0x55872ac48050 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fcee9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fcee9800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56608/0 (socket says 172.21.15.46:56608) 2024-05-08T23:05:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fcee9800700 1 -- 172.21.15.46:0/1710025489 learned_addr learned my addr 172.21.15.46:0/1710025489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48050 0x55872ac48430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.075+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac42890 0x55872ac42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee2000700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 0x7fcecc0a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee9800700 1 -- 172.21.15.46:0/1710025489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48050 msgr2=0x55872ac48430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee9800700 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48050 0x55872ac48430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee9800700 1 -- 172.21.15.46:0/1710025489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac42890 msgr2=0x55872ac42cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee9800700 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac42890 0x55872ac42cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee9800700 1 -- 172.21.15.46:0/1710025489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55872aa133b0 con 0x55872ac48970 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fcee2000700 1 -- 172.21.15.46:0/1951876642 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fcedc0cce70 con 0x7fcee410b230 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.076+0000 7fceea200700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 0x7fcecc0a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.077+0000 7fcee9800700 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fced800b6f0 tx=0x7fced8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a8bb8622bf2613 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.077+0000 7fced6a00700 1 -- 172.21.15.46:0/1710025489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801d050 con 0x55872ac48970 2024-05-08T23:05:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.077+0000 7fced6a00700 1 -- 172.21.15.46:0/1710025489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced801e040 con 0x55872ac48970 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.077+0000 7fced6a00700 1 -- 172.21.15.46:0/1710025489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80054f0 con 0x55872ac48970 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 -- 172.21.15.46:0/1710025489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 msgr2=0x55872ac42350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fced800b6f0 tx=0x7fced8005bc0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceea200700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 0x7fcecc0a39a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ca030 tx=0x7fcedc0092e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 -- 172.21.15.46:0/1710025489 shutdown_connections 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac42890 0x55872ac42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48050 0x55872ac48430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1710025489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac42350 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 -- 172.21.15.46:0/1710025489 >> 172.21.15.46:0/1710025489 conn(0x55872aba0210 msgr2=0x55872ac3d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 -- 172.21.15.46:0/1710025489 shutdown_connections 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.078+0000 7fceffc40b80 1 -- 172.21.15.46:0/1710025489 wait complete. 2024-05-08T23:05:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.079+0000 7fceffc40b80 1 -- start start 2024-05-08T23:05:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.079+0000 7fceffc40b80 1 Processor -- start 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48050 0x55872ac4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48970 0x55872ac4c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ab2e560 con 0x55872ac48050 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ab2e200 con 0x55872ac48970 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872aa156a0 con 0x55872ac42890 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fceffc40b80 10 client.0 _ll_register_callbacks cb 0x55872ab37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43364/0 (socket says 172.21.15.46:43364) 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 -- 172.21.15.46:0/3768178952 learned_addr learned my addr 172.21.15.46:0/3768178952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48970 0x55872ac4c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 -- 172.21.15.46:0/3768178952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48970 msgr2=0x55872ac4c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48970 0x55872ac4c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 -- 172.21.15.46:0/3768178952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48050 msgr2=0x55872ac4bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48050 0x55872ac4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.080+0000 7fcee8e00700 1 -- 172.21.15.46:0/3768178952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee410e3b0 con 0x55872ac42890 2024-05-08T23:05:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fcee8e00700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fced0012b30 tx=0x7fced0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8505c408e6fd57f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fcecb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000ea70 con 0x55872ac42890 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55872aaeb270 con 0x55872ac42890 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fced0029030 con 0x55872ac42890 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.081+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced000ebd0 con 0x55872ac42890 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 10 client.26113 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55872a9bfb60 con 0x55872ac42890 2024-05-08T23:05:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55872aaeb270 con 0x55872ac42890 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fced00051b0 con 0x55872ac42890 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 10 client.26113 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 10 client.26113 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 20 client.26113 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872ac5c390 0x55872ac5e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 4 client.26113 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.082+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55872ac11810 con 0x55872ac5c390 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.083+0000 7fcee9800700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872ac5c390 0x55872ac5e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.083+0000 7fcee9800700 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872ac5c390 0x55872ac5e7b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96c549048966ceff server_cookie=de2a6aa16cd6eb8f in_seq=0 out_seq=0 2024-05-08T23:05:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.083+0000 7fcecb400700 10 client.26113 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.084+0000 7fcecb400700 1 -- 172.21.15.46:0/3768178952 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55872ac11810 con 0x55872ac5c390 2024-05-08T23:05:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.084+0000 7fcecb400700 10 client.26113 handle_command_reply: tid=0 2024-05-08T23:05:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.084+0000 7fceffc40b80 1 client.26113 shutdown 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.084+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 msgr2=0x55872ac4e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.084+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fced0012b30 tx=0x7fced0004750 comp rx=0 tx=0).stop 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 shutdown_connections 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872ac5c390 0x55872ac5e7b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48970 0x55872ac4c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac4e880 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 --2- 172.21.15.46:0/3768178952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48050 0x55872ac4bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 >> 172.21.15.46:0/3768178952 conn(0x55872aba0210 msgr2=0x55872ac3d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fcecb400700 0 client.26113 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 shutdown_connections 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 1 -- 172.21.15.46:0/3768178952 wait complete. 2024-05-08T23:05:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.085+0000 7fceffc40b80 20 client.26113 trim_cache size 0 max 16384 2024-05-08T23:05:10.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.111+0000 7fceffc40b80 1 Processor -- start 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 -- start start 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac42890 0x55872ac42c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48970 0x55872ac63110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac4a100 con 0x55872ac42890 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac35040 con 0x55872ac48050 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fceffc40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac4c2b0 con 0x55872ac48970 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fcee9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fcee9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42692/0 (socket says 172.21.15.46:42692) 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.112+0000 7fcee9800700 1 -- 172.21.15.46:0/1772243941 learned_addr learned my addr 172.21.15.46:0/1772243941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48970 0x55872ac63110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac42890 0x55872ac42c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 -- 172.21.15.46:0/1772243941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48970 msgr2=0x55872ac63110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48970 0x55872ac63110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 -- 172.21.15.46:0/1772243941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac42890 msgr2=0x55872ac42c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac42890 0x55872ac42c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 -- 172.21.15.46:0/1772243941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55872ab58560 con 0x55872ac48050 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fcee9800700 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fced8026040 tx=0x7fced801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185192c8c5b4f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fced7400700 1 -- 172.21.15.46:0/1772243941 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801d070 con 0x55872ac48050 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fced7400700 1 -- 172.21.15.46:0/1772243941 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced8004ca0 con 0x55872ac48050 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.113+0000 7fced7400700 1 -- 172.21.15.46:0/1772243941 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8031030 con 0x55872ac48050 2024-05-08T23:05:10.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.114+0000 7fceffc40b80 1 -- 172.21.15.46:0/1772243941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 msgr2=0x55872ac3d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.114+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fced8026040 tx=0x7fced801a040 comp rx=0 tx=0).stop 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 -- 172.21.15.46:0/1772243941 shutdown_connections 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac48970 0x55872ac63110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac42890 0x55872ac42c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1772243941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac3d060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 -- 172.21.15.46:0/1772243941 >> 172.21.15.46:0/1772243941 conn(0x55872aba0210 msgr2=0x55872ac3bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 -- 172.21.15.46:0/1772243941 shutdown_connections 2024-05-08T23:05:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 -- 172.21.15.46:0/1772243941 wait complete. 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.115+0000 7fceffc40b80 1 -- start start 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 Processor -- start 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac53540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac43a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac43f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac44480 con 0x55872ac48970 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac445f0 con 0x55872ac48050 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872ac447b0 con 0x55872ac42890 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.116+0000 7fceffc40b80 10 client.0 _ll_register_callbacks cb 0x55872ab37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac53540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac43a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac43a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42702/0 (socket says 172.21.15.46:42702) 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee9800700 1 -- 172.21.15.46:0/1580208480 learned_addr learned my addr 172.21.15.46:0/1580208480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac43f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 -- 172.21.15.46:0/1580208480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 msgr2=0x55872ac43a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac43a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 -- 172.21.15.46:0/1580208480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 msgr2=0x55872ac43f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac43f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.117+0000 7fcee8e00700 1 -- 172.21.15.46:0/1580208480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55872aa133b0 con 0x55872ac42890 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fcee8e00700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac53540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fced0007ad0 tx=0x7fced0024040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fdf37bdb00d1bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fcecb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0004460 con 0x55872ac42890 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fced0029070 con 0x55872ac42890 2024-05-08T23:05:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fced801e040 con 0x55872ac42890 2024-05-08T23:05:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced00045c0 con 0x55872ac42890 2024-05-08T23:05:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fceffc40b80 10 client.26119 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.118+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55872aa17cd0 con 0x55872ac42890 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced801e040 con 0x55872ac42890 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fced0003950 con 0x55872ac42890 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 10 client.26119 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 10 client.26119 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 20 client.26119 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872acb2c20 0x55872acb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 4 client.26119 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55872ab8aec0 con 0x55872acb2c20 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.119+0000 7fcee9800700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872acb2c20 0x55872acb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fcee9800700 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872acb2c20 0x55872acb5010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cacf2f79a5324624 server_cookie=e68061bfd324fe6d in_seq=0 out_seq=0 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fcecb400700 10 client.26119 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fcecb400700 1 -- 172.21.15.46:0/1580208480 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55872ab8aec0 con 0x55872acb2c20 2024-05-08T23:05:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fcecb400700 10 client.26119 handle_command_reply: tid=0 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fceffc40b80 1 client.26119 shutdown 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 msgr2=0x55872ac53540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.120+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac53540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fced0007ad0 tx=0x7fced0024040 comp rx=0 tx=0).stop 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 shutdown_connections 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55872acb2c20 0x55872acb5010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55872ac48970 0x55872ac43f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55872ac42890 0x55872ac53540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fcecb400700 0 client.26119 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 --2- 172.21.15.46:0/1580208480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55872ac48050 0x55872ac43a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 >> 172.21.15.46:0/1580208480 conn(0x55872aba0210 msgr2=0x55872ac42c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 shutdown_connections 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 1 -- 172.21.15.46:0/1580208480 wait complete. 2024-05-08T23:05:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.121+0000 7fceffc40b80 20 client.26119 trim_cache size 0 max 16384 2024-05-08T23:05:10.124 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 msgr2=0x7fcecc0a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 0x7fcecc0a39a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ca030 tx=0x7fcedc0092e0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 msgr2=0x7fcee419dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 0x7fcee419dc00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00efc0 tx=0x7fcedc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 shutdown_connections 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee410b230 0x7fcee419dc00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcecc0a14e0 0x7fcecc0a39a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee4101e10 0x7fcee4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 --2- 172.21.15.46:0/1951876642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee4102730 0x7fcee4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 >> 172.21.15.46:0/1951876642 conn(0x7fcee40fb5f0 msgr2=0x7fcee410d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 shutdown_connections 2024-05-08T23:05:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.122+0000 7fceeac00700 1 -- 172.21.15.46:0/1951876642 wait complete. 2024-05-08T23:05:10.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:09 smithi046 ceph-mon[36898]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:10.139 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:09 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4100822220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:10.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:05:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:09 smithi086 ceph-mon[78610]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:09 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4100822220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:09 smithi053 ceph-mon[77587]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:09 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4100822220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 -- 172.21.15.46:0/1895174613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 msgr2=0x7fc08c104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 --2- 172.21.15.46:0/1895174613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c104b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc078009990 tx=0x7fc078009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 -- 172.21.15.46:0/1895174613 shutdown_connections 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 --2- 172.21.15.46:0/1895174613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 --2- 172.21.15.46:0/1895174613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 --2- 172.21.15.46:0/1895174613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c104b90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 -- 172.21.15.46:0/1895174613 >> 172.21.15.46:0/1895174613 conn(0x7fc08c0fb760 msgr2=0x7fc08c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 -- 172.21.15.46:0/1895174613 shutdown_connections 2024-05-08T23:05:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.411+0000 7fc093e00700 1 -- 172.21.15.46:0/1895174613 wait complete. 2024-05-08T23:05:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.413+0000 7fc093e00700 1 Processor -- start 2024-05-08T23:05:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.413+0000 7fc093e00700 1 -- start start 2024-05-08T23:05:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19ddd0 con 0x7fc08c1052c0 2024-05-08T23:05:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19df40 con 0x7fc08c0ffdb0 2024-05-08T23:05:10.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc093e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19e100 con 0x7fc08c1006d0 2024-05-08T23:05:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc091400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc091e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc091400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c1a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42728/0 (socket says 172.21.15.46:42728) 2024-05-08T23:05:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc091400700 1 -- 172.21.15.46:0/1728501958 learned_addr learned my addr 172.21.15.46:0/1728501958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.414+0000 7fc090a00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 -- 172.21.15.46:0/1728501958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 msgr2=0x7fc08c1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 -- 172.21.15.46:0/1728501958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 msgr2=0x7fc08c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 -- 172.21.15.46:0/1728501958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009520 con 0x7fc08c1052c0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc091e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c19d770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc084007f10 tx=0x7fc08400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc08a000700 1 -- 172.21.15.46:0/1728501958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084009ea0 con 0x7fc08c1052c0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0780095f0 con 0x7fc08c1052c0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc08a000700 1 -- 172.21.15.46:0/1728501958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08400f040 con 0x7fc08c1052c0 2024-05-08T23:05:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.415+0000 7fc08a000700 1 -- 172.21.15.46:0/1728501958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084004350 con 0x7fc08c1052c0 2024-05-08T23:05:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.416+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c19e5d0 con 0x7fc08c1052c0 2024-05-08T23:05:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.419+0000 7fc08a000700 1 -- 172.21.15.46:0/1728501958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fc084003a00 con 0x7fc08c1052c0 2024-05-08T23:05:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.420+0000 7fc08a000700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 0x7fc0740a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.420+0000 7fc0a79bbb80 1 Processor -- start 2024-05-08T23:05:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc08a000700 1 -- 172.21.15.46:0/1728501958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc0840cc9e0 con 0x7fc08c1052c0 2024-05-08T23:05:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 -- start start 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c3fe00 0x564143c401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143b91150 con 0x564143c40720 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143b8eb10 con 0x564143c3a890 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.421+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143b8a400 con 0x564143c3fe00 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc091400700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 0x7fc0740a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56672/0 (socket says 172.21.15.46:56672) 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 -- 172.21.15.46:0/641393442 learned_addr learned my addr 172.21.15.46:0/641393442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc091e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc091400700 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c3fe00 0x564143c401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 -- 172.21.15.46:0/641393442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c3fe00 msgr2=0x564143c401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c3fe00 0x564143c401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 -- 172.21.15.46:0/641393442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 msgr2=0x564143c3acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.422+0000 7fc090a00700 1 -- 172.21.15.46:0/641393442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564143a0b3b0 con 0x564143c40720 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.423+0000 7fc091400700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 0x7fc0740a3980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc080011b80 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.423+0000 7fc090a00700 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc078009990 tx=0x7fc07800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64123388bda36dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.423+0000 7fc07e600700 1 -- 172.21.15.46:0/641393442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07801d070 con 0x564143c40720 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.423+0000 7fc07e600700 1 -- 172.21.15.46:0/641393442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07801e040 con 0x564143c40720 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.423+0000 7fc07e600700 1 -- 172.21.15.46:0/641393442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078004760 con 0x564143c40720 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/641393442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 msgr2=0x564143c3a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc078009990 tx=0x7fc07800b610 comp rx=0 tx=0).stop 2024-05-08T23:05:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/641393442 shutdown_connections 2024-05-08T23:05:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c3fe00 0x564143c401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/641393442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c40720 0x564143c3a350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/641393442 >> 172.21.15.46:0/641393442 conn(0x564143b98210 msgr2=0x564143c34df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/641393442 shutdown_connections 2024-05-08T23:05:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.424+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/641393442 wait complete. 2024-05-08T23:05:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.425+0000 7fc0a79bbb80 1 -- start start 2024-05-08T23:05:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.425+0000 7fc0a79bbb80 1 Processor -- start 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 0x564143c43ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 0x564143c44000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143b26560 con 0x564143c3a890 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143b26200 con 0x564143c3fe00 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143a0d6a0 con 0x564143c40720 2024-05-08T23:05:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc0a79bbb80 10 client.0 _ll_register_callbacks cb 0x564143b2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc091400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 0x564143c43ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc091e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc090a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 0x564143c44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc091e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c44540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43424/0 (socket says 172.21.15.46:43424) 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.426+0000 7fc091e00700 1 -- 172.21.15.46:0/3999815906 learned_addr learned my addr 172.21.15.46:0/3999815906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 -- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 msgr2=0x564143c44540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c44540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 -- 172.21.15.46:0/3999815906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 msgr2=0x564143c43ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 0x564143c43ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 -- 172.21.15.46:0/3999815906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c114c20 con 0x564143c3fe00 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc091400700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 0x564143c43ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc090a00700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 0x564143c44000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc078000c00 tx=0x7fc07800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d47f9c0eb9dad21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc072a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.427+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07801d030 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564143ae3270 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc078005980 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc078004910 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc0a79bbb80 10 client.26096 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641439b7b60 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07803c080 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc07800faf0 con 0x564143c3fe00 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.428+0000 7fc0a79bbb80 10 client.26096 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc0a79bbb80 10 client.26096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc0a79bbb80 20 client.26096 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143c53da0 0x564143c561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc0a79bbb80 4 client.26096 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564143c09810 con 0x564143c53da0 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.429+0000 7fc091400700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143c53da0 0x564143c561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.430+0000 7fc091400700 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143c53da0 0x564143c561c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45c09d9eba56f976 server_cookie=f8a08315017d743c in_seq=0 out_seq=0 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.430+0000 7fc072a00700 10 client.26096 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.430+0000 7fc072a00700 1 -- 172.21.15.46:0/3999815906 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564143c09810 con 0x564143c53da0 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.430+0000 7fc072a00700 10 client.26096 handle_command_reply: tid=0 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.430+0000 7fc0a79bbb80 1 client.26096 shutdown 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 msgr2=0x564143c44000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 0x564143c44000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc078000c00 tx=0x7fc07800b9e0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 shutdown_connections 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c44540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143c53da0 0x564143c561c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3a890 0x564143c43ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/3999815906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3fe00 0x564143c44000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc072a00700 0 client.26096 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 >> 172.21.15.46:0/3999815906 conn(0x564143b98210 msgr2=0x564143c345a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 shutdown_connections 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/3999815906 wait complete. 2024-05-08T23:05:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.431+0000 7fc0a79bbb80 20 client.26096 trim_cache size 0 max 16384 2024-05-08T23:05:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.445+0000 7fc0a79bbb80 1 Processor -- start 2024-05-08T23:05:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 -- start start 2024-05-08T23:05:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143c09810 con 0x564143c3fe00 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143c35e80 con 0x564143c3a890 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143c438e0 con 0x564143c40720 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56688/0 (socket says 172.21.15.46:56688) 2024-05-08T23:05:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.446+0000 7fc090a00700 1 -- 172.21.15.46:0/2859027126 learned_addr learned my addr 172.21.15.46:0/2859027126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 -- 172.21.15.46:0/2859027126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 msgr2=0x564143c34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc091400700 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 -- 172.21.15.46:0/2859027126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 msgr2=0x564143c3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 -- 172.21.15.46:0/2859027126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564143b50560 con 0x564143c3fe00 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc091400700 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.447+0000 7fc090a00700 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc07800fe50 tx=0x7fc07800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98431690d7f320a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.448+0000 7fc07f000700 1 -- 172.21.15.46:0/2859027126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07801d070 con 0x564143c3fe00 2024-05-08T23:05:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.448+0000 7fc07f000700 1 -- 172.21.15.46:0/2859027126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564143ae3270 con 0x564143c3fe00 2024-05-08T23:05:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.448+0000 7fc07f000700 1 -- 172.21.15.46:0/2859027126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f6c0 con 0x564143c3fe00 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/2859027126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 msgr2=0x564143c433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc07800fe50 tx=0x7fc07800b260 comp rx=0 tx=0).stop 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/2859027126 shutdown_connections 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143c34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/2859027126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143c433a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/2859027126 >> 172.21.15.46:0/2859027126 conn(0x564143b98210 msgr2=0x564143c3eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/2859027126 shutdown_connections 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.449+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/2859027126 wait complete. 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.450+0000 7fc0a79bbb80 1 -- start start 2024-05-08T23:05:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.450+0000 7fc0a79bbb80 1 Processor -- start 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143ca2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143ca3f60 con 0x564143c3fe00 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143ca40d0 con 0x564143c3a890 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564143ca2790 con 0x564143c40720 2024-05-08T23:05:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.451+0000 7fc0a79bbb80 10 client.0 _ll_register_callbacks cb 0x564143b2fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc091e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143ca2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc090a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56696/0 (socket says 172.21.15.46:56696) 2024-05-08T23:05:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc090a00700 1 -- 172.21.15.46:0/417383241 learned_addr learned my addr 172.21.15.46:0/417383241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc091e00700 1 -- 172.21.15.46:0/417383241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 msgr2=0x564143c4b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.452+0000 7fc091400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091e00700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c4b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091e00700 1 -- 172.21.15.46:0/417383241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 msgr2=0x564143ca3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091e00700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091e00700 1 -- 172.21.15.46:0/417383241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641439b7b60 con 0x564143c40720 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091400700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc090a00700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc091e00700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143ca2250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc084099490 tx=0x7fc084097ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0685f8c672995b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc072a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084099960 con 0x564143c40720 2024-05-08T23:05:10.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.453+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07801e040 con 0x564143c40720 2024-05-08T23:05:10.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc084099c00 con 0x564143c40720 2024-05-08T23:05:10.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08409d160 con 0x564143c40720 2024-05-08T23:05:10.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 10 client.26134 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564143a0fcd0 con 0x564143c40720 2024-05-08T23:05:10.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07801e040 con 0x564143c40720 2024-05-08T23:05:10.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0840a14f0 con 0x564143c40720 2024-05-08T23:05:10.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 10 client.26134 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 10 client.26134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 20 client.26134 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143cbbef0 0x564143cbe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 4 client.26134 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.454+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564143b82ec0 con 0x564143cbbef0 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.455+0000 7fc091400700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143cbbef0 0x564143cbe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.455+0000 7fc091400700 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143cbbef0 0x564143cbe310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9d80d2de75a22f server_cookie=7bf39700fab0f0ad in_seq=0 out_seq=0 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.455+0000 7fc072a00700 10 client.26134 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.455+0000 7fc072a00700 1 -- 172.21.15.46:0/417383241 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564143b82ec0 con 0x564143cbbef0 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.455+0000 7fc072a00700 10 client.26134 handle_command_reply: tid=0 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 client.26134 shutdown 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 msgr2=0x564143ca2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143ca2250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc084099490 tx=0x7fc084097ab0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 shutdown_connections 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564143c40720 0x564143ca2250 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564143c3a890 0x564143c4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x564143cbbef0 0x564143cbe310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 --2- 172.21.15.46:0/417383241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564143c3fe00 0x564143ca3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc072a00700 0 client.26134 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 >> 172.21.15.46:0/417383241 conn(0x564143b98210 msgr2=0x564143c3f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 shutdown_connections 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.456+0000 7fc0a79bbb80 1 -- 172.21.15.46:0/417383241 wait complete. 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc0a79bbb80 20 client.26134 trim_cache size 0 max 16384 2024-05-08T23:05:10.458 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 msgr2=0x7fc0740a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 0x7fc0740a3980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc080011b80 comp rx=0 tx=0).stop 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 msgr2=0x7fc08c19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c19d770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc084007f10 tx=0x7fc08400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 shutdown_connections 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0740a14c0 0x7fc0740a3980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c1052c0 0x7fc08c19d770 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c0ffdb0 0x7fc08c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 --2- 172.21.15.46:0/1728501958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1006d0 0x7fc08c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.457+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 >> 172.21.15.46:0/1728501958 conn(0x7fc08c0fb760 msgr2=0x7fc08c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.458+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 shutdown_connections 2024-05-08T23:05:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.458+0000 7fc093e00700 1 -- 172.21.15.46:0/1728501958 wait complete. 2024-05-08T23:05:10.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 -- 172.21.15.46:0/4146262567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a08100560 msgr2=0x7f2a0810d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/4146262567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a08100560 0x7f2a0810d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009990 tx=0x7f29f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 -- 172.21.15.46:0/4146262567 shutdown_connections 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/4146262567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0810d760 0x7f2a0810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/4146262567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a080fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/4146262567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a08100560 0x7f2a0810d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.704+0000 7f2a0cc00700 1 -- 172.21.15.46:0/4146262567 >> 172.21.15.46:0/4146262567 conn(0x7f2a080fb760 msgr2=0x7f2a080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.705+0000 7f2a0cc00700 1 -- 172.21.15.46:0/4146262567 shutdown_connections 2024-05-08T23:05:10.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.705+0000 7f2a0cc00700 1 -- 172.21.15.46:0/4146262567 wait complete. 2024-05-08T23:05:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 Processor -- start 2024-05-08T23:05:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 -- start start 2024-05-08T23:05:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a08199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 0x7f2a08199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e460 con 0x7f2a080ffbb0 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e5d0 con 0x7f2a08100560 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.706+0000 7f2a0cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e790 con 0x7f2a0810d760 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a08199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43450/0 (socket says 172.21.15.46:43450) 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a08199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56706/0 (socket says 172.21.15.46:56706) 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 -- 172.21.15.46:0/3240897949 learned_addr learned my addr 172.21.15.46:0/3240897949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a05400700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 0x7f2a08199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 -- 172.21.15.46:0/3240897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 msgr2=0x7f2a08199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 0x7f2a08199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 -- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 msgr2=0x7f2a08199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a08199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a06800700 1 -- 172.21.15.46:0/3240897949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40095f0 con 0x7f2a0810d760 2024-05-08T23:05:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.707+0000 7f2a05400700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 0x7f2a08199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f2a06800700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000ada0 tx=0x7f2a0000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f29fea00700 1 -- 172.21.15.46:0/3240897949 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000f070 con 0x7f2a0810d760 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f29fea00700 1 -- 172.21.15.46:0/3240897949 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a00007af0 con 0x7f2a0810d760 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0819ea70 con 0x7f2a0810d760 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f29fea00700 1 -- 172.21.15.46:0/3240897949 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00004290 con 0x7f2a0810d760 2024-05-08T23:05:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.708+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081a62b0 con 0x7f2a0810d760 2024-05-08T23:05:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.711+0000 7f29fea00700 1 -- 172.21.15.46:0/3240897949 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f2a00003a20 con 0x7f2a0810d760 2024-05-08T23:05:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.714+0000 7f29fea00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 0x7f29ec0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.714+0000 7f29fea00700 1 -- 172.21.15.46:0/3240897949 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2a000cc3c0 con 0x7f2a0810d760 2024-05-08T23:05:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.714+0000 7f2a2073cb80 1 Processor -- start 2024-05-08T23:05:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.714+0000 7f2a05e00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 0x7f29ec0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.714+0000 7f2a2073cb80 1 -- start start 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fb9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fb4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86f02150 con 0x559a86fb9f80 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86effb10 con 0x559a86fb9660 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86efb400 con 0x559a86fb40f0 2024-05-08T23:05:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a05400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a05400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56714/0 (socket says 172.21.15.46:56714) 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a05400700 1 -- 172.21.15.46:0/208784323 learned_addr learned my addr 172.21.15.46:0/208784323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a05e00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 0x7f29ec0a3900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009c60 tx=0x7f29f8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a06800700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fb4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.715+0000 7f2a05e00700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fb9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 -- 172.21.15.46:0/208784323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 msgr2=0x559a86fb9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fb9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 -- 172.21.15.46:0/208784323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 msgr2=0x559a86fb3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 -- 172.21.15.46:0/208784323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a86d7c3b0 con 0x559a86fb40f0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a05400700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f2a06800700 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fb4570 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a000c9520 tx=0x7f2a00098aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bb1e075e77ffc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f29f3000700 1 -- 172.21.15.46:0/208784323 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00098ee0 con 0x559a86fb40f0 2024-05-08T23:05:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f29f3000700 1 -- 172.21.15.46:0/208784323 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a0009cd10 con 0x559a86fb40f0 2024-05-08T23:05:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.716+0000 7f29f3000700 1 -- 172.21.15.46:0/208784323 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00099360 con 0x559a86fb40f0 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 -- 172.21.15.46:0/208784323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 msgr2=0x559a86fb4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fb4570 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a000c9520 tx=0x7f2a00098aa0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 -- 172.21.15.46:0/208784323 shutdown_connections 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fb4570 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fb9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/208784323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fb3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 -- 172.21.15.46:0/208784323 >> 172.21.15.46:0/208784323 conn(0x559a86f09210 msgr2=0x559a86fa5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 -- 172.21.15.46:0/208784323 shutdown_connections 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.717+0000 7f2a2073cb80 1 -- 172.21.15.46:0/208784323 wait complete. 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.718+0000 7f2a2073cb80 1 -- start start 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.718+0000 7f2a2073cb80 1 Processor -- start 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86e97560 con 0x559a86fb9f80 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86e97200 con 0x559a86fb9660 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86d7e6a0 con 0x559a86fb40f0 2024-05-08T23:05:10.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a2073cb80 10 client.0 _ll_register_callbacks cb 0x559a86ea0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.719+0000 7f2a05e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43478/0 (socket says 172.21.15.46:43478) 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 -- 172.21.15.46:0/152927426 learned_addr learned my addr 172.21.15.46:0/152927426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05400700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a06800700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 -- 172.21.15.46:0/152927426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 msgr2=0x559a86fbd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fbd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 -- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 msgr2=0x559a86fbdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fbdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 -- 172.21.15.46:0/152927426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081107c0 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a06800700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fbdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05400700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fbd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f2a05e00700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29f8003900 tx=0x7f29f80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cdc56563806612 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.720+0000 7f29e7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8022070 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f29f801d820 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a86e54270 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f80045a0 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f2a2073cb80 10 client.26146 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2a081a62b0 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.721+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f8030020 con 0x559a86fb40f0 2024-05-08T23:05:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29f8005020 con 0x559a86fb40f0 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 10 client.26146 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 10 client.26146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 20 client.26146 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a86fcda60 0x559a86fcfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 4 client.26146 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a86f7a810 con 0x559a86fcda60 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.722+0000 7f2a05400700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a86fcda60 0x559a86fcfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.723+0000 7f2a05400700 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a86fcda60 0x559a86fcfe80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81d2bac75c80233b server_cookie=773f10973dba14db in_seq=0 out_seq=0 2024-05-08T23:05:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.723+0000 7f29e7400700 10 client.26146 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f29e7400700 1 -- 172.21.15.46:0/152927426 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a86f7a810 con 0x559a86fcda60 2024-05-08T23:05:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f29e7400700 10 client.26146 handle_command_reply: tid=0 2024-05-08T23:05:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 client.26146 shutdown 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 msgr2=0x559a86fbd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29f8003900 tx=0x7f29f80047c0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 shutdown_connections 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a86fcda60 0x559a86fcfe80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9f80 0x559a86fbdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fbd2d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/152927426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9660 0x559a86fbd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 >> 172.21.15.46:0/152927426 conn(0x559a86f09210 msgr2=0x559a86fa55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.724+0000 7f29e7400700 0 client.26146 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.725+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 shutdown_connections 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.725+0000 7f2a2073cb80 1 -- 172.21.15.46:0/152927426 wait complete. 2024-05-08T23:05:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.725+0000 7f2a2073cb80 20 client.26146 trim_cache size 0 max 16384 2024-05-08T23:05:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 Processor -- start 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 -- start start 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 0x559a86fa5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86fa7e40 con 0x559a86fb40f0 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.752+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86fa0800 con 0x559a86fb9f80 2024-05-08T23:05:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a86fc0310 con 0x559a86fb9660 2024-05-08T23:05:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 0x559a86fa5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43490/0 (socket says 172.21.15.46:43490) 2024-05-08T23:05:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a05400700 1 -- 172.21.15.46:0/3042367127 learned_addr learned my addr 172.21.15.46:0/3042367127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.753+0000 7f2a06800700 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 -- 172.21.15.46:0/3042367127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 msgr2=0x559a86fb81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 -- 172.21.15.46:0/3042367127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 msgr2=0x559a86fb8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fb8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05400700 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 -- 172.21.15.46:0/3042367127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a86d28b60 con 0x559a86fb40f0 2024-05-08T23:05:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f2a05e00700 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 0x559a86fa5d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005950 tx=0x7f29f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c858ba98512515ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.754+0000 7f29f3a00700 1 -- 172.21.15.46:0/3042367127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8022070 con 0x559a86fb40f0 2024-05-08T23:05:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.755+0000 7f29f3a00700 1 -- 172.21.15.46:0/3042367127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f8030070 con 0x559a86fb40f0 2024-05-08T23:05:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.755+0000 7f29f3a00700 1 -- 172.21.15.46:0/3042367127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8004b60 con 0x559a86fb40f0 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.755+0000 7f2a2073cb80 1 -- 172.21.15.46:0/3042367127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 msgr2=0x559a86fa5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 0x559a86fa5d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005950 tx=0x7f29f8004600 comp rx=0 tx=0).stop 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 -- 172.21.15.46:0/3042367127 shutdown_connections 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fb8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb40f0 0x559a86fa5d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/3042367127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb9660 0x559a86fb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 -- 172.21.15.46:0/3042367127 >> 172.21.15.46:0/3042367127 conn(0x559a86f09210 msgr2=0x559a86fb7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 -- 172.21.15.46:0/3042367127 shutdown_connections 2024-05-08T23:05:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.756+0000 7f2a2073cb80 1 -- 172.21.15.46:0/3042367127 wait complete. 2024-05-08T23:05:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.757+0000 7f2a2073cb80 1 -- start start 2024-05-08T23:05:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.757+0000 7f2a2073cb80 1 Processor -- start 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fc3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 0x559a86fc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8701e0c0 con 0x559a86fb9660 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8701e230 con 0x559a86fb9f80 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8701e3f0 con 0x559a86fb40f0 2024-05-08T23:05:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a2073cb80 10 client.0 _ll_register_callbacks cb 0x559a86ea0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a05400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 0x559a86fc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a06800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a06800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42818/0 (socket says 172.21.15.46:42818) 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a06800700 1 -- 172.21.15.46:0/1500620011 learned_addr learned my addr 172.21.15.46:0/1500620011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.758+0000 7f2a05e00700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fc3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 -- 172.21.15.46:0/1500620011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 msgr2=0x559a86fc3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fc3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 -- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 msgr2=0x559a86fc3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 0x559a86fc3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 -- 172.21.15.46:0/1500620011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081a62b0 con 0x559a86fb9f80 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a05400700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 0x559a86fc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f2a06800700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a00004000 tx=0x7f2a00095240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140d15d17df2d947 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.759+0000 7f29e7400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000a10d0 con 0x559a86fb9f80 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2a00097020 con 0x559a86fb9f80 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a0009cd10 con 0x559a86fb9f80 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a00098ee0 con 0x559a86fb9f80 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f2a2073cb80 10 client.26111 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559a86ec1560 con 0x559a86fb9f80 2024-05-08T23:05:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.760+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a000a1360 con 0x559a86fb9f80 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a00094df0 con 0x559a86fb9f80 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 10 client.26111 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 10 client.26111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 20 client.26111 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a8701f8a0 0x559a86fb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 4 client.26111 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f2a0009cd10 con 0x559a8701f8a0 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.761+0000 7f2a05e00700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a8701f8a0 0x559a86fb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.762+0000 7f29e7400700 10 client.26111 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.762+0000 7f2a05e00700 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a8701f8a0 0x559a86fb2380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc7a0130e9e4c3a server_cookie=abccf4298a328de9 in_seq=0 out_seq=0 2024-05-08T23:05:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f29e7400700 1 -- 172.21.15.46:0/1500620011 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f2a0009cd10 con 0x559a8701f8a0 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f29e7400700 10 client.26111 handle_command_reply: tid=0 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f2a2073cb80 1 client.26111 shutdown 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 msgr2=0x559a86fc3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a00004000 tx=0x7f2a00095240 comp rx=0 tx=0).stop 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 shutdown_connections 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.763+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a86fb9f80 0x559a86fc3fb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a86fb40f0 0x559a86fc3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x559a8701f8a0 0x559a86fb2380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 --2- 172.21.15.46:0/1500620011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a86fb9660 0x559a86fc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f29e7400700 0 client.26111 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 >> 172.21.15.46:0/1500620011 conn(0x559a86f09210 msgr2=0x559a86fb81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 shutdown_connections 2024-05-08T23:05:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 1 -- 172.21.15.46:0/1500620011 wait complete. 2024-05-08T23:05:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.764+0000 7f2a2073cb80 20 client.26111 trim_cache size 0 max 16384 2024-05-08T23:05:10.767 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 msgr2=0x7f29ec0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 0x7f29ec0a3900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009c60 tx=0x7f29f8009260 comp rx=0 tx=0).stop 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 msgr2=0x7f2a0819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000ada0 tx=0x7f2a0000c330 comp rx=0 tx=0).stop 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 shutdown_connections 2024-05-08T23:05:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a0810d760 0x7f2a0819de20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29ec0a1440 0x7f29ec0a3900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a080ffbb0 0x7f2a08199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 --2- 172.21.15.46:0/3240897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a08100560 0x7f2a08199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 >> 172.21.15.46:0/3240897949 conn(0x7f2a080fb760 msgr2=0x7f2a080fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 shutdown_connections 2024-05-08T23:05:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:10.765+0000 7f2a0cc00700 1 -- 172.21.15.46:0/3240897949 wait complete. 2024-05-08T23:05:10.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:05:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 -- 172.21.15.46:0/67023930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4748101fa0 msgr2=0x7f4748102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 --2- 172.21.15.46:0/67023930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4748101fa0 0x7f4748102380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f473c009990 tx=0x7f473c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 -- 172.21.15.46:0/67023930 shutdown_connections 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 --2- 172.21.15.46:0/67023930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f474810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 --2- 172.21.15.46:0/67023930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4748101fa0 0x7f4748102380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 --2- 172.21.15.46:0/67023930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47481028c0 0x7f474810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.021+0000 7f474f600700 1 -- 172.21.15.46:0/67023930 >> 172.21.15.46:0/67023930 conn(0x7f47480fb760 msgr2=0x7f47480fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.022+0000 7f474f600700 1 -- 172.21.15.46:0/67023930 shutdown_connections 2024-05-08T23:05:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.022+0000 7f474f600700 1 -- 172.21.15.46:0/67023930 wait complete. 2024-05-08T23:05:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.023+0000 7f474f600700 1 Processor -- start 2024-05-08T23:05:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.023+0000 7f474f600700 1 -- start start 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.023+0000 7f474f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 0x7f474810a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.023+0000 7f474f600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 0x7f474810a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.023+0000 7f474f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f47481a3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f474f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a3940 con 0x7f4748101fa0 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f474f600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a3ab0 con 0x7f47481028c0 2024-05-08T23:05:11.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f474f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481a3c70 con 0x7f474810b3c0 2024-05-08T23:05:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 0x7f474810a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 0x7f474810a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42826/0 (socket says 172.21.15.46:42826) 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f4747e00700 1 -- 172.21.15.46:0/1527326259 learned_addr learned my addr 172.21.15.46:0/1527326259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f474cc00700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 0x7f474810a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.024+0000 7f474d600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f47481a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 -- 172.21.15.46:0/1527326259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 msgr2=0x7f47481a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f47481a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 -- 172.21.15.46:0/1527326259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 msgr2=0x7f474810a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 0x7f474810a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 -- 172.21.15.46:0/1527326259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0095f0 con 0x7f4748101fa0 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474d600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f47481a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474cc00700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 0x7f474810a160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f473c005950 tx=0x7f473c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f4745600700 1 -- 172.21.15.46:0/1527326259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c01d070 con 0x7f4748101fa0 2024-05-08T23:05:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47481a3ef0 con 0x7f4748101fa0 2024-05-08T23:05:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.025+0000 7f4745600700 1 -- 172.21.15.46:0/1527326259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f473c005da0 con 0x7f4748101fa0 2024-05-08T23:05:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.026+0000 7f4745600700 1 -- 172.21.15.46:0/1527326259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0173f0 con 0x7f4748101fa0 2024-05-08T23:05:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.026+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481aa590 con 0x7f4748101fa0 2024-05-08T23:05:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.030+0000 7f4763084b80 1 Processor -- start 2024-05-08T23:05:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.030+0000 7f4763084b80 1 -- start start 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4745600700 1 -- 172.21.15.46:0/1527326259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f473c0216a0 con 0x7f4748101fa0 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb940b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9405350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb935c150 con 0x55fbb940ae00 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb9359b10 con 0x55fbb9405890 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4763084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb9355400 con 0x55fbb940b720 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42832/0 (socket says 172.21.15.46:42832) 2024-05-08T23:05:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.031+0000 7f4747e00700 1 -- 172.21.15.46:0/792642386 learned_addr learned my addr 172.21.15.46:0/792642386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f474cc00700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9405350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f474d600700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb940b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 -- 172.21.15.46:0/792642386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 msgr2=0x55fbb9405350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9405350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 -- 172.21.15.46:0/792642386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 msgr2=0x55fbb940b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb940b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 -- 172.21.15.46:0/792642386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb91d63b0 con 0x55fbb9405890 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f474cc00700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9405350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.032+0000 7f4747e00700 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4734003a50 tx=0x7f4734003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ec52fb86c85cea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.033+0000 7f4739c00700 1 -- 172.21.15.46:0/792642386 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400bab0 con 0x55fbb9405890 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.033+0000 7f4739c00700 1 -- 172.21.15.46:0/792642386 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4734003e90 con 0x55fbb9405890 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.033+0000 7f4739c00700 1 -- 172.21.15.46:0/792642386 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734019660 con 0x55fbb9405890 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.033+0000 7f4745600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 0x7f47300a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.033+0000 7f4745600700 1 -- 172.21.15.46:0/1527326259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f473c0ce360 con 0x7f4748101fa0 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f474d600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 0x7f47300a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 -- 172.21.15.46:0/792642386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 msgr2=0x55fbb9405cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4734003a50 tx=0x7f4734003d60 comp rx=0 tx=0).stop 2024-05-08T23:05:11.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 -- 172.21.15.46:0/792642386 shutdown_connections 2024-05-08T23:05:11.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb9405890 0x55fbb9405cd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb940b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 --2- 172.21.15.46:0/792642386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9405350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 -- 172.21.15.46:0/792642386 >> 172.21.15.46:0/792642386 conn(0x55fbb9363210 msgr2=0x55fbb93ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 -- 172.21.15.46:0/792642386 shutdown_connections 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.034+0000 7f4763084b80 1 -- 172.21.15.46:0/792642386 wait complete. 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.035+0000 7f4763084b80 1 -- start start 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.035+0000 7f4763084b80 1 Processor -- start 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb940eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb92f1560 con 0x55fbb9405890 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb92f1200 con 0x55fbb940b720 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb91d86a0 con 0x55fbb940ae00 2024-05-08T23:05:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4763084b80 10 client.0 _ll_register_callbacks cb 0x55fbb92fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f474d600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 0x7f47300a3980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4740003fd0 tx=0x7f474000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f474cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb940eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4747e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.036+0000 7f4747e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43544/0 (socket says 172.21.15.46:43544) 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f474d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 -- 172.21.15.46:0/1890849453 learned_addr learned my addr 172.21.15.46:0/1890849453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 -- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 msgr2=0x55fbb940f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 -- 172.21.15.46:0/1890849453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 msgr2=0x55fbb940eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb940eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 -- 172.21.15.46:0/1890849453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481aa590 con 0x55fbb940ae00 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f4747e00700 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f47340114f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8f65a7b87a739d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.037+0000 7f472e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473401d070 con 0x55fbb940ae00 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbb92ae270 con 0x55fbb940ae00 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f473400bab0 con 0x55fbb940ae00 2024-05-08T23:05:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473400bc70 con 0x55fbb940ae00 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f4763084b80 10 client.26164 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.038+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4734003680 con 0x55fbb940ae00 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbb92ae270 con 0x55fbb940ae00 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4734017020 con 0x55fbb940ae00 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f4763084b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f4763084b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f4763084b80 20 client.26164 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.039+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb941f150 0x55fbb9421570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.040+0000 7f4763084b80 4 client.26164 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.040+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbb93d4810 con 0x55fbb941f150 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.040+0000 7f474cc00700 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb941f150 0x55fbb9421570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.041+0000 7f472e000700 10 client.26164 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.041+0000 7f474cc00700 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb941f150 0x55fbb9421570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32854c9fd0bebd5f server_cookie=b7c1b11bbba14fc6 in_seq=0 out_seq=0 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.041+0000 7f472e000700 1 -- 172.21.15.46:0/1890849453 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fbb93d4810 con 0x55fbb941f150 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.041+0000 7f472e000700 10 client.26164 handle_command_reply: tid=0 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 client.26164 shutdown 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 msgr2=0x55fbb940f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f47340114f0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 shutdown_connections 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb941f150 0x55fbb9421570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb940eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 --2- 172.21.15.46:0/1890849453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940ae00 0x55fbb940f030 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 >> 172.21.15.46:0/1890849453 conn(0x55fbb9363210 msgr2=0x55fbb93ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f472e000700 0 client.26164 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.042+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 shutdown_connections 2024-05-08T23:05:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.043+0000 7f4763084b80 1 -- 172.21.15.46:0/1890849453 wait complete. 2024-05-08T23:05:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.043+0000 7f4763084b80 20 client.26164 trim_cache size 0 max 16384 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.070+0000 7f4763084b80 1 Processor -- start 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.070+0000 7f4763084b80 1 -- start start 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.070+0000 7f4763084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4763084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb93ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4763084b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4763084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb940cb10 con 0x55fbb940ae00 2024-05-08T23:05:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4763084b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb94100f0 con 0x55fbb940b720 2024-05-08T23:05:11.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4763084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb93fcdc0 con 0x55fbb9405890 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43546/0 (socket says 172.21.15.46:43546) 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474cc00700 1 -- 172.21.15.46:0/1765848982 learned_addr learned my addr 172.21.15.46:0/1765848982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474d600700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f4747e00700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb93ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474cc00700 1 -- 172.21.15.46:0/1765848982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 msgr2=0x55fbb940a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.071+0000 7f474cc00700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f474cc00700 1 -- 172.21.15.46:0/1765848982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 msgr2=0x55fbb93ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f474cc00700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb93ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f474cc00700 1 -- 172.21.15.46:0/1765848982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb9182b60 con 0x55fbb9405890 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f4747e00700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb93ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f474cc00700 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f473c0974a0 tx=0x7f473c0d1df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5444a6a47d1fb616 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f473a600700 1 -- 172.21.15.46:0/1765848982 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0d1920 con 0x55fbb9405890 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f473a600700 1 -- 172.21.15.46:0/1765848982 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f473c09ad70 con 0x55fbb9405890 2024-05-08T23:05:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.072+0000 7f473a600700 1 -- 172.21.15.46:0/1765848982 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0d1ae0 con 0x55fbb9405890 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 -- 172.21.15.46:0/1765848982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 msgr2=0x55fbb9405350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f473c0974a0 tx=0x7f473c0d1df0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 -- 172.21.15.46:0/1765848982 shutdown_connections 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940b720 0x55fbb940a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb9405890 0x55fbb9405350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 --2- 172.21.15.46:0/1765848982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb940ae00 0x55fbb93ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.073+0000 7f4763084b80 1 -- 172.21.15.46:0/1765848982 >> 172.21.15.46:0/1765848982 conn(0x55fbb9363210 msgr2=0x55fbb9409bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.074+0000 7f4763084b80 1 -- 172.21.15.46:0/1765848982 shutdown_connections 2024-05-08T23:05:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.074+0000 7f4763084b80 1 -- 172.21.15.46:0/1765848982 wait complete. 2024-05-08T23:05:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.074+0000 7f4763084b80 1 -- start start 2024-05-08T23:05:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 Processor -- start 2024-05-08T23:05:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb9416650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9414890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb9414f70 con 0x55fbb9405890 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb9474630 con 0x55fbb940ae00 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.075+0000 7f4763084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb9474770 con 0x55fbb940b720 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f4763084b80 10 client.0 _ll_register_callbacks cb 0x55fbb92fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f4747e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42858/0 (socket says 172.21.15.46:42858) 2024-05-08T23:05:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f4747e00700 1 -- 172.21.15.46:0/3035268251 learned_addr learned my addr 172.21.15.46:0/3035268251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f474d600700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9414890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.076+0000 7f474cc00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb9416650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 -- 172.21.15.46:0/3035268251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 msgr2=0x55fbb9414890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9414890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 -- 172.21.15.46:0/3035268251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 msgr2=0x55fbb9416650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb9416650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 -- 172.21.15.46:0/3035268251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481a3ef0 con 0x55fbb940ae00 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f474cc00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb9416650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f4747e00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55fbb9400990 tx=0x7f473400d6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192be317bbe54869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f472e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.077+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473401d030 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4734011610 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4734003e90 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4734036080 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f4763084b80 10 client.26129 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fbb931b560 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.078+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f473401e040 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4734017070 con 0x55fbb940ae00 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 10 client.26129 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 20 client.26129 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb9486a10 0x55fbb9488e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 4 client.26129 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f4734003e90 con 0x55fbb9486a10 2024-05-08T23:05:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.079+0000 7f474cc00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb9486a10 0x55fbb9488e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f474cc00700 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb9486a10 0x55fbb9488e30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edb0a0c8177bb9a server_cookie=ee998075852199e in_seq=0 out_seq=0 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f472e000700 10 client.26129 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f472e000700 1 -- 172.21.15.46:0/3035268251 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f4734003e90 con 0x55fbb9486a10 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f472e000700 10 client.26129 handle_command_reply: tid=0 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f4763084b80 1 client.26129 shutdown 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.080+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 msgr2=0x55fbb9414150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55fbb9400990 tx=0x7f473400d6d0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 shutdown_connections 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbb940b720 0x55fbb9414890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fbb9486a10 0x55fbb9488e30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbb9405890 0x55fbb9416650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 --2- 172.21.15.46:0/3035268251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbb940ae00 0x55fbb9414150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 >> 172.21.15.46:0/3035268251 conn(0x55fbb9363210 msgr2=0x55fbb9409bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f472e000700 0 client.26129 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 shutdown_connections 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 1 -- 172.21.15.46:0/3035268251 wait complete. 2024-05-08T23:05:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.081+0000 7f4763084b80 20 client.26129 trim_cache size 0 max 16384 2024-05-08T23:05:11.087 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.082+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 msgr2=0x7f47300a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.082+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 0x7f47300a3980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4740003fd0 tx=0x7f474000b040 comp rx=0 tx=0).stop 2024-05-08T23:05:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.082+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 msgr2=0x7f474810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.082+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 0x7f474810a160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f473c005950 tx=0x7f473c005740 comp rx=0 tx=0).stop 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 shutdown_connections 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474810b3c0 0x7f47481a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47300a14c0 0x7f47300a3980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4748101fa0 0x7f474810a160 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 --2- 172.21.15.46:0/1527326259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47481028c0 0x7f474810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 >> 172.21.15.46:0/1527326259 conn(0x7f47480fb760 msgr2=0x7f47481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 shutdown_connections 2024-05-08T23:05:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:11.083+0000 7f474f600700 1 -- 172.21.15.46:0/1527326259 wait complete. 2024-05-08T23:05:12.115 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:05:12.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:05:12.196 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:11 smithi046 ceph-mon[36898]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:11 smithi086 ceph-mon[78610]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:11 smithi053 ceph-mon[77587]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 -- 172.21.15.46:0/3822172994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 msgr2=0x7fdf64068540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 --2- 172.21.15.46:0/3822172994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf64068540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009990 tx=0x7fdf54009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 -- 172.21.15.46:0/3822172994 shutdown_connections 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 --2- 172.21.15.46:0/3822172994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf6410d940 0x7fdf6410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 --2- 172.21.15.46:0/3822172994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf64068540 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 --2- 172.21.15.46:0/3822172994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64068a80 0x7fdf6410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 -- 172.21.15.46:0/3822172994 >> 172.21.15.46:0/3822172994 conn(0x7fdf64075710 msgr2=0x7fdf64075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 -- 172.21.15.46:0/3822172994 shutdown_connections 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.348+0000 7fdf63200700 1 -- 172.21.15.46:0/3822172994 wait complete. 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.349+0000 7fdf63200700 1 Processor -- start 2024-05-08T23:05:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 -- start start 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 0x7fdf6410a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6410d940 0x7fdf6410a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6410ac10 con 0x7fdf6410d940 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6410ad80 con 0x7fdf64068160 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf63200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6410af40 con 0x7fdf64068a80 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.350+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 0x7fdf6410a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48648/0 (socket says 172.21.15.46:48648) 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61e00700 1 -- 172.21.15.46:0/1275827593 learned_addr learned my addr 172.21.15.46:0/1275827593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf62800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6410d940 0x7fdf6410a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 -- 172.21.15.46:0/1275827593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 msgr2=0x7fdf6410d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 -- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6410d940 msgr2=0x7fdf6410a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6410d940 0x7fdf6410a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61400700 1 -- 172.21.15.46:0/1275827593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540095f0 con 0x7fdf64068a80 2024-05-08T23:05:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.351+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf61400700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 0x7fdf6410a0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b450 tx=0x7fdf5000b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf5aa00700 1 -- 172.21.15.46:0/1275827593 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50011830 con 0x7fdf64068a80 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf5aa00700 1 -- 172.21.15.46:0/1275827593 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf5000be60 con 0x7fdf64068a80 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf641a9db0 con 0x7fdf64068a80 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf5aa00700 1 -- 172.21.15.46:0/1275827593 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50010d20 con 0x7fdf64068a80 2024-05-08T23:05:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.352+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf641aa2e0 con 0x7fdf64068a80 2024-05-08T23:05:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.357+0000 7fdf7c7c4b80 1 Processor -- start 2024-05-08T23:05:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.357+0000 7fdf7c7c4b80 1 -- start start 2024-05-08T23:05:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf5aa00700 1 -- 172.21.15.46:0/1275827593 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7fdf5001a400 con 0x7fdf64068a80 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c2b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 0x564b17c25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17b7c1d0 con 0x564b17c2b9f0 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17b79b90 con 0x564b17c2b0d0 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17b75480 con 0x564b17c25910 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 0x564b17c25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 0x564b17c25d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45346/0 (socket says 172.21.15.46:45346) 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf61400700 1 -- 172.21.15.46:0/1702186610 learned_addr learned my addr 172.21.15.46:0/1702186610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.358+0000 7fdf62800700 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c2b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 -- 172.21.15.46:0/1702186610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 msgr2=0x564b17c25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 0x564b17c25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 -- 172.21.15.46:0/1702186610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 msgr2=0x564b17c253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 -- 172.21.15.46:0/1702186610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b179f6430 con 0x564b17c2b0d0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.359+0000 7fdf62800700 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c2b4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c011550 tx=0x7fdf5c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4f4bbb80b03bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.360+0000 7fdf4f000700 1 -- 172.21.15.46:0/1702186610 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00e040 con 0x564b17c2b0d0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.360+0000 7fdf4f000700 1 -- 172.21.15.46:0/1702186610 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf5c00f040 con 0x564b17c2b0d0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.360+0000 7fdf4f000700 1 -- 172.21.15.46:0/1702186610 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004490 con 0x564b17c2b0d0 2024-05-08T23:05:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.360+0000 7fdf5aa00700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 0x7fdf480a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.360+0000 7fdf5aa00700 1 -- 172.21.15.46:0/1275827593 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf500cc840 con 0x7fdf64068a80 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 0x7fdf480a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1702186610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 msgr2=0x564b17c2b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c2b4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c011550 tx=0x7fdf5c009920 comp rx=0 tx=0).stop 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1702186610 shutdown_connections 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c25910 0x564b17c25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c2b4b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1702186610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1702186610 >> 172.21.15.46:0/1702186610 conn(0x564b17b83290 msgr2=0x564b17c20350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.361+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1702186610 shutdown_connections 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.362+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 0x7fdf480a3a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x564b17c26630 tx=0x7fdf5400b280 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.362+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1702186610 wait complete. 2024-05-08T23:05:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.363+0000 7fdf7c7c4b80 1 -- start start 2024-05-08T23:05:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.363+0000 7fdf7c7c4b80 1 Processor -- start 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c31940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17b115e0 con 0x564b17c2b9f0 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17b11280 con 0x564b17c25910 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b179f8720 con 0x564b17c2b0d0 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.364+0000 7fdf7c7c4b80 10 client.0 _ll_register_callbacks cb 0x564b17b1abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf62800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45358/0 (socket says 172.21.15.46:45358) 2024-05-08T23:05:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 -- 172.21.15.46:0/3648140808 learned_addr learned my addr 172.21.15.46:0/3648140808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61e00700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c31940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 -- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 msgr2=0x564b17c31940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c31940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.365+0000 7fdf61400700 1 -- 172.21.15.46:0/3648140808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 msgr2=0x564b17c2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf61400700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf61400700 1 -- 172.21.15.46:0/3648140808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c011230 con 0x564b17c2b0d0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf62800700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf61400700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf50099170 tx=0x7fdf50011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fe2d6beeb9757b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf43400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50099240 con 0x564b17c2b0d0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf50099b70 con 0x564b17c2b0d0 2024-05-08T23:05:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.366+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b17ace2f0 con 0x564b17c2b0d0 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf50098fb0 con 0x564b17c2b0d0 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 10 client.26182 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b179a2be0 con 0x564b17c2b0d0 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b17ace2f0 con 0x564b17c2b0d0 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf50095b80 con 0x564b17c2b0d0 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 20 client.26182 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17c50320 0x564b17c52740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 4 client.26182 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.367+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b17bf4890 con 0x564b17c50320 2024-05-08T23:05:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.368+0000 7fdf61e00700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17c50320 0x564b17c52740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.368+0000 7fdf61e00700 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17c50320 0x564b17c52740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75898cd35cfcb1a server_cookie=4f23d6e637552842 in_seq=0 out_seq=0 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.369+0000 7fdf43400700 10 client.26182 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.369+0000 7fdf43400700 1 -- 172.21.15.46:0/3648140808 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b17bf4890 con 0x564b17c50320 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.369+0000 7fdf43400700 10 client.26182 handle_command_reply: tid=0 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.369+0000 7fdf7c7c4b80 1 client.26182 shutdown 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 msgr2=0x564b17c2ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf50099170 tx=0x7fdf50011000 comp rx=0 tx=0).stop 2024-05-08T23:05:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 shutdown_connections 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17c50320 0x564b17c52740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c31940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/3648140808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c2ec30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 >> 172.21.15.46:0/3648140808 conn(0x564b17b83290 msgr2=0x564b17c20130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.370+0000 7fdf43400700 0 client.26182 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.371+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 shutdown_connections 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.371+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/3648140808 wait complete. 2024-05-08T23:05:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.371+0000 7fdf7c7c4b80 20 client.26182 trim_cache size 0 max 16384 2024-05-08T23:05:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.406+0000 7fdf7c7c4b80 1 Processor -- start 2024-05-08T23:05:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 -- start start 2024-05-08T23:05:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c1a880 con 0x564b17c2b9f0 2024-05-08T23:05:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c1f410 con 0x564b17c25910 2024-05-08T23:05:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.407+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c20840 con 0x564b17c2b0d0 2024-05-08T23:05:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c52330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48682/0 (socket says 172.21.15.46:48682) 2024-05-08T23:05:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61e00700 1 -- 172.21.15.46:0/135751763 learned_addr learned my addr 172.21.15.46:0/135751763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf62800700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 -- 172.21.15.46:0/135751763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 msgr2=0x564b17c52330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c52330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 -- 172.21.15.46:0/135751763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 msgr2=0x564b17c363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf61400700 1 -- 172.21.15.46:0/135751763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b17b3b5e0 con 0x564b17c2b0d0 2024-05-08T23:05:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.408+0000 7fdf62800700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.409+0000 7fdf61400700 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c570a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf50010770 tx=0x7fdf50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8ca27123ca79fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.409+0000 7fdf4fa00700 1 -- 172.21.15.46:0/135751763 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50098dd0 con 0x564b17c2b0d0 2024-05-08T23:05:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.409+0000 7fdf4fa00700 1 -- 172.21.15.46:0/135751763 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf50098fb0 con 0x564b17c2b0d0 2024-05-08T23:05:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.409+0000 7fdf4fa00700 1 -- 172.21.15.46:0/135751763 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50099b70 con 0x564b17c2b0d0 2024-05-08T23:05:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/135751763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 msgr2=0x564b17c570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c570a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf50010770 tx=0x7fdf50004750 comp rx=0 tx=0).stop 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/135751763 shutdown_connections 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c2b9f0 0x564b17c363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c25910 0x564b17c52330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/135751763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b0d0 0x564b17c570a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/135751763 >> 172.21.15.46:0/135751763 conn(0x564b17b83290 msgr2=0x564b17c20130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/135751763 shutdown_connections 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.410+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/135751763 wait complete. 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.411+0000 7fdf7c7c4b80 1 -- start start 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.411+0000 7fdf7c7c4b80 1 Processor -- start 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b9f0 0x564b17c275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c27b20 con 0x564b17c25910 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c27c60 con 0x564b17c2b0d0 2024-05-08T23:05:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b17c27e20 con 0x564b17c2b9f0 2024-05-08T23:05:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf7c7c4b80 10 client.0 _ll_register_callbacks cb 0x564b17b1abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf61e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45144/0 (socket says 172.21.15.46:45144) 2024-05-08T23:05:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf61e00700 1 -- 172.21.15.46:0/1856614446 learned_addr learned my addr 172.21.15.46:0/1856614446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.412+0000 7fdf61400700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 -- 172.21.15.46:0/1856614446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b9f0 msgr2=0x564b17c275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b9f0 0x564b17c275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 -- 172.21.15.46:0/1856614446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 msgr2=0x564b17c365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 -- 172.21.15.46:0/1856614446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b179a2be0 con 0x564b17c2b0d0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf61400700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c270a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x564b17c1ced0 tx=0x7fdf500cc660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d94d41795992f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf43400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50098b60 con 0x564b17c2b0d0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf50018520 con 0x564b17c2b0d0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.413+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf5c00f040 con 0x564b17c2b0d0 2024-05-08T23:05:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf500d2290 con 0x564b17c2b0d0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 10 client.26144 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b179fad50 con 0x564b17c2b0d0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf5c00f040 con 0x564b17c2b0d0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf500a1460 con 0x564b17c2b0d0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 20 client.26144 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17ca7840 0x564b17ca9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 4 client.26144 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.414+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564b17b6df40 con 0x564b17ca7840 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.415+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17ca7840 0x564b17ca9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.415+0000 7fdf61e00700 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17ca7840 0x564b17ca9c30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ed45c2974beb7 server_cookie=45697bef8aeaaf87 in_seq=0 out_seq=0 2024-05-08T23:05:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.415+0000 7fdf43400700 10 client.26144 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.416+0000 7fdf43400700 1 -- 172.21.15.46:0/1856614446 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564b17b6df40 con 0x564b17ca7840 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.417+0000 7fdf43400700 10 client.26144 handle_command_reply: tid=0 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.417+0000 7fdf7c7c4b80 1 client.26144 shutdown 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 msgr2=0x564b17c270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c270a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x564b17c1ced0 tx=0x7fdf500cc660 comp rx=0 tx=0).stop 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 shutdown_connections 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b17c2b9f0 0x564b17c275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564b17ca7840 0x564b17ca9c30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b17c25910 0x564b17c365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 --2- 172.21.15.46:0/1856614446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b17c2b0d0 0x564b17c270a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 >> 172.21.15.46:0/1856614446 conn(0x564b17b83290 msgr2=0x564b17c570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf43400700 0 client.26144 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 shutdown_connections 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.418+0000 7fdf7c7c4b80 1 -- 172.21.15.46:0/1856614446 wait complete. 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.419+0000 7fdf7c7c4b80 20 client.26144 trim_cache size 0 max 16384 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "80432962-eb1b-4f69-adc6-dc18d2a28a6c", 2024-05-08T23:05:12.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:05:12.421 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.419+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 msgr2=0x7fdf480a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.419+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 0x7fdf480a3a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x564b17c26630 tx=0x7fdf5400b280 comp rx=0 tx=0).stop 2024-05-08T23:05:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.419+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 msgr2=0x7fdf6410a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.419+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 0x7fdf6410a0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b450 tx=0x7fdf5000b760 comp rx=0 tx=0).stop 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 shutdown_connections 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf480a1560 0x7fdf480a3a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf6410d940 0x7fdf6410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf64068160 0x7fdf6410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 --2- 172.21.15.46:0/1275827593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf64068a80 0x7fdf6410a0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 >> 172.21.15.46:0/1275827593 conn(0x7fdf64075710 msgr2=0x7fdf6410edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 shutdown_connections 2024-05-08T23:05:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.420+0000 7fdf63200700 1 -- 172.21.15.46:0/1275827593 wait complete. 2024-05-08T23:05:12.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:05:12.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 -- 172.21.15.46:0/3977684308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84100ec0 msgr2=0x7f8e841012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 --2- 172.21.15.46:0/3977684308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84100ec0 0x7f8e841012a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009990 tx=0x7f8e74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 -- 172.21.15.46:0/3977684308 shutdown_connections 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 --2- 172.21.15.46:0/3977684308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84105fe0 0x7f8e8410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 --2- 172.21.15.46:0/3977684308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84100ec0 0x7f8e841012a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 --2- 172.21.15.46:0/3977684308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84101870 0x7f8e841058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 -- 172.21.15.46:0/3977684308 >> 172.21.15.46:0/3977684308 conn(0x7f8e840fc740 msgr2=0x7f8e840feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 -- 172.21.15.46:0/3977684308 shutdown_connections 2024-05-08T23:05:12.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.783+0000 7f8e83200700 1 -- 172.21.15.46:0/3977684308 wait complete. 2024-05-08T23:05:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.785+0000 7f8e83200700 1 Processor -- start 2024-05-08T23:05:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.785+0000 7f8e83200700 1 -- start start 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84105fe0 0x7f8e84195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 0x7f8e84199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419a490 con 0x7f8e84199a10 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419a600 con 0x7f8e84105fe0 2024-05-08T23:05:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e83200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419a7c0 con 0x7f8e84100ec0 2024-05-08T23:05:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45406/0 (socket says 172.21.15.46:45406) 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e81e00700 1 -- 172.21.15.46:0/196763636 learned_addr learned my addr 172.21.15.46:0/196763636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e81400700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84105fe0 0x7f8e84195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.786+0000 7f8e82800700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 0x7f8e84199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 -- 172.21.15.46:0/196763636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84105fe0 msgr2=0x7f8e84195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84105fe0 0x7f8e84195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 -- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 msgr2=0x7f8e84199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 0x7f8e84199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 -- 172.21.15.46:0/196763636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740095f0 con 0x7f8e84100ec0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e82800700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 0x7f8e84199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e81e00700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e740095c0 tx=0x7f8e7400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e7aa00700 1 -- 172.21.15.46:0/196763636 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7401d070 con 0x7f8e84100ec0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e841a1cc0 con 0x7f8e84100ec0 2024-05-08T23:05:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.787+0000 7f8e7aa00700 1 -- 172.21.15.46:0/196763636 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e74005050 con 0x7f8e84100ec0 2024-05-08T23:05:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.788+0000 7f8e7aa00700 1 -- 172.21.15.46:0/196763636 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74004b10 con 0x7f8e84100ec0 2024-05-08T23:05:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.788+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a2190 con 0x7f8e84100ec0 2024-05-08T23:05:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.792+0000 7f8e9caf0b80 1 Processor -- start 2024-05-08T23:05:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 -- start start 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6499d9150 con 0x55c649a88050 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6499d6b10 con 0x55c649a88970 2024-05-08T23:05:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.793+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6499d2400 con 0x55c649a82890 2024-05-08T23:05:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e7aa00700 1 -- 172.21.15.46:0/196763636 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136157+0+0 (secure 0 0 0) 0x7f8e740051c0 con 0x7f8e84100ec0 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45422/0 (socket says 172.21.15.46:45422) 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 -- 172.21.15.46:0/1385464229 learned_addr learned my addr 172.21.15.46:0/1385464229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e82800700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 -- 172.21.15.46:0/1385464229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 msgr2=0x55c649a82350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a82350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 -- 172.21.15.46:0/1385464229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 msgr2=0x55c649a88430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a88430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81400700 1 -- 172.21.15.46:0/1385464229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6498533b0 con 0x55c649a82890 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e82800700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.794+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a82350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:05:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.795+0000 7f8e81400700 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e700141c0 tx=0x7f8e700144d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde0ca8b8aa3527e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.795+0000 7f8e6f400700 1 -- 172.21.15.46:0/1385464229 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004230 con 0x55c649a82890 2024-05-08T23:05:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.795+0000 7f8e6f400700 1 -- 172.21.15.46:0/1385464229 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e70013040 con 0x55c649a82890 2024-05-08T23:05:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.795+0000 7f8e6f400700 1 -- 172.21.15.46:0/1385464229 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700053c0 con 0x55c649a82890 2024-05-08T23:05:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e7aa00700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 0x7f8e680a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1385464229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 msgr2=0x55c649a82cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e700141c0 tx=0x7f8e700144d0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e7aa00700 1 -- 172.21.15.46:0/196763636 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8e740ce130 con 0x7f8e84100ec0 2024-05-08T23:05:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1385464229 shutdown_connections 2024-05-08T23:05:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a82cd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1385464229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a82350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1385464229 >> 172.21.15.46:0/1385464229 conn(0x55c6499e0210 msgr2=0x55c649a7d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.796+0000 7f8e82800700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 0x7f8e680a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.797+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1385464229 shutdown_connections 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.797+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1385464229 wait complete. 2024-05-08T23:05:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.797+0000 7f8e82800700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 0x7f8e680a39a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009db0 tx=0x7f8e7c009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.798+0000 7f8e9caf0b80 1 -- start start 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.798+0000 7f8e9caf0b80 1 Processor -- start 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a8e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64996e560 con 0x55c649a88050 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64996e200 con 0x55c649a88970 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6498556a0 con 0x55c649a82890 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e9caf0b80 10 client.0 _ll_register_callbacks cb 0x55c649977b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.799+0000 7f8e82800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e82800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a8c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48728/0 (socket says 172.21.15.46:48728) 2024-05-08T23:05:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e82800700 1 -- 172.21.15.46:0/1212821007 learned_addr learned my addr 172.21.15.46:0/1212821007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a8e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 -- 172.21.15.46:0/1212821007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 msgr2=0x55c649a8e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a8e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 -- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 msgr2=0x55c649a8c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a8c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.800+0000 7f8e81400700 1 -- 172.21.15.46:0/1212821007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7000bc10 con 0x55c649a88050 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e81400700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a8bb80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000bfd0 tx=0x7f8e7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362dfc34315a7f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e63e00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004430 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c64992b270 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8e7000b410 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e70004960 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e9caf0b80 10 client.16176 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.801+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c6497ffb60 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.802+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7002c080 con 0x55c649a88050 2024-05-08T23:05:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.802+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e70019b20 con 0x55c649a88050 2024-05-08T23:05:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 20 client.16176 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649a9c120 0x55c649a9e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 4 client.16176 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c649a51810 con 0x55c649a9c120 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649a9c120 0x55c649a9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.803+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649a9c120 0x55c649a9e540 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9761e0fed86fed89 server_cookie=6b5f12dd30244143 in_seq=0 out_seq=0 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.804+0000 7f8e63e00700 10 client.16176 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.804+0000 7f8e63e00700 1 -- 172.21.15.46:0/1212821007 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c649a51810 con 0x55c649a9c120 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.804+0000 7f8e63e00700 10 client.16176 handle_command_reply: tid=0 2024-05-08T23:05:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.804+0000 7f8e9caf0b80 1 client.16176 shutdown 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 msgr2=0x55c649a8bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a8bb80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000bfd0 tx=0x7f8e7000b960 comp rx=0 tx=0).stop 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 shutdown_connections 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a8c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a82890 0x55c649a8e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649a9c120 0x55c649a9e540 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1212821007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a88050 0x55c649a8bb80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 >> 172.21.15.46:0/1212821007 conn(0x55c6499e0210 msgr2=0x55c649a7d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e63e00700 0 client.16176 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 shutdown_connections 2024-05-08T23:05:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.805+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1212821007 wait complete. 2024-05-08T23:05:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.806+0000 7f8e9caf0b80 20 client.16176 trim_cache size 0 max 16384 2024-05-08T23:05:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 Processor -- start 2024-05-08T23:05:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 -- start start 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649a7ec10 con 0x55c649a82890 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649a77a90 con 0x55c649a88970 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.824+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649a79dc0 con 0x55c649a88050 2024-05-08T23:05:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45444/0 (socket says 172.21.15.46:45444) 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 -- 172.21.15.46:0/436124786 learned_addr learned my addr 172.21.15.46:0/436124786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e82800700 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 -- 172.21.15.46:0/436124786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 msgr2=0x55c649a93860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a93860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 -- 172.21.15.46:0/436124786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 msgr2=0x55c649a82350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 -- 172.21.15.46:0/436124786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c649a83190 con 0x55c649a88050 2024-05-08T23:05:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.825+0000 7f8e81400700 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001eb0 tx=0x7f8e7000b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92176826354786f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e6fe00700 1 -- 172.21.15.46:0/436124786 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700148d0 con 0x55c649a88050 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e6fe00700 1 -- 172.21.15.46:0/436124786 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c64992b270 con 0x55c649a88050 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e6fe00700 1 -- 172.21.15.46:0/436124786 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004450 con 0x55c649a88050 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/436124786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 msgr2=0x55c649a93320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001eb0 tx=0x7f8e7000b000 comp rx=0 tx=0).stop 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/436124786 shutdown_connections 2024-05-08T23:05:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a82350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/436124786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a93320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.826+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/436124786 >> 172.21.15.46:0/436124786 conn(0x55c6499e0210 msgr2=0x55c649a97de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.827+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/436124786 shutdown_connections 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.827+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/436124786 wait complete. 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.827+0000 7f8e9caf0b80 1 -- start start 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.827+0000 7f8e9caf0b80 1 Processor -- start 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649a84d70 con 0x55c649a82890 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649a84ee0 con 0x55c649a88970 2024-05-08T23:05:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c649af25e0 con 0x55c649a88050 2024-05-08T23:05:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e9caf0b80 10 client.0 _ll_register_callbacks cb 0x55c649977b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45454/0 (socket says 172.21.15.46:45454) 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 -- 172.21.15.46:0/1427525380 learned_addr learned my addr 172.21.15.46:0/1427525380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e82800700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 -- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 msgr2=0x55c649a846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 -- 172.21.15.46:0/1427525380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 msgr2=0x55c649a93640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.828+0000 7f8e81400700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a93640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e81400700 1 -- 172.21.15.46:0/1427525380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6497ffb60 con 0x55c649a88050 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e81400700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e70019fd0 tx=0x7f8e70005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69cbef78707f9719 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e63e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70038070 con 0x55c649a88050 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8e70000e40 con 0x55c649a88050 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.829+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e70013040 con 0x55c649a88050 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e700233f0 con 0x55c649a88050 2024-05-08T23:05:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 10 client.26203 fetch_fsmap learned FSMap version 27 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c649998560 con 0x55c649a88050 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e70013040 con 0x55c649a88050 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e70023c00 con 0x55c649a88050 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 10 client.26203 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 20 client.26203 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649af3890 0x55c649af5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 4 client.26203 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.830+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6499caec0 con 0x55c649af3890 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649af3890 0x55c649af5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e81e00700 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649af3890 0x55c649af5cb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10518c3c7255d161 server_cookie=c32b0a8070d3e1ae in_seq=0 out_seq=0 2024-05-08T23:05:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e63e00700 10 client.26203 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e63e00700 1 -- 172.21.15.46:0/1427525380 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6499caec0 con 0x55c649af3890 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e63e00700 10 client.26203 handle_command_reply: tid=0 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.831+0000 7f8e9caf0b80 1 client.26203 shutdown 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 msgr2=0x55c649a84160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e70019fd0 tx=0x7f8e70005400 comp rx=0 tx=0).stop 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 shutdown_connections 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c649a88970 0x55c649a846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c649af3890 0x55c649af5cb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c649a82890 0x55c649a93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e63e00700 0 client.26203 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 --2- 172.21.15.46:0/1427525380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c649a88050 0x55c649a84160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 >> 172.21.15.46:0/1427525380 conn(0x55c6499e0210 msgr2=0x55c649a8a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 shutdown_connections 2024-05-08T23:05:12.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.832+0000 7f8e9caf0b80 1 -- 172.21.15.46:0/1427525380 wait complete. 2024-05-08T23:05:12.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e9caf0b80 20 client.26203 trim_cache size 0 max 16384 2024-05-08T23:05:12.836 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:12.836 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:05:12.836 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: "80432962-eb1b-4f69-adc6-dc18d2a28a6c": { 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "80432962-eb1b-4f69-adc6-dc18d2a28a6c", 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:05:12.837 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 msgr2=0x7f8e680a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 0x7f8e680a39a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009db0 tx=0x7f8e7c009040 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 msgr2=0x7f8e84195130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e740095c0 tx=0x7f8e7400f690 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 shutdown_connections 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e680a14e0 0x7f8e680a39a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e84199a10 0x7f8e84199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84100ec0 0x7f8e84195130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 --2- 172.21.15.46:0/196763636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84105fe0 0x7f8e84195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.833+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 >> 172.21.15.46:0/196763636 conn(0x7f8e840fc740 msgr2=0x7f8e840fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.834+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 shutdown_connections 2024-05-08T23:05:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:12.834+0000 7f8e83200700 1 -- 172.21.15.46:0/196763636 wait complete. 2024-05-08T23:05:12.852 INFO:tasks.cephfs.filesystem:scrub status for tag:80432962-eb1b-4f69-adc6-dc18d2a28a6c - {'path': '/', 'tag': '80432962-eb1b-4f69-adc6-dc18d2a28a6c', 'options': 'recursive,force'} 2024-05-08T23:05:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:13 smithi086 ceph-mon[78610]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:13 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:05:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:13 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:13 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:05:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:13 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:13 smithi053 ceph-mon[77587]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:13 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:05:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:13 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:13 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:05:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:13 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:13 smithi046 ceph-mon[36898]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-08T23:05:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:13 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:05:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:13 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:13 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:05:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:13 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:15.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:14 smithi086 ceph-mon[78610]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:15.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:14 smithi053 ceph-mon[77587]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:15.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:14 smithi046 ceph-mon[36898]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-08T23:05:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:05:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:05:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:05:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:16 smithi086 ceph-mon[78610]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-08T23:05:17.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:16 smithi053 ceph-mon[77587]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-08T23:05:17.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:16 smithi046 ceph-mon[36898]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-08T23:05:19.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:19 smithi086 ceph-mon[78610]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:05:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:19 smithi053 ceph-mon[77587]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:05:19.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:19 smithi046 ceph-mon[36898]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-08T23:05:21.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:21 smithi086 ceph-mon[78610]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:21.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:21 smithi086 ceph-mon[78610]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T23:05:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:21 smithi053 ceph-mon[77587]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:21 smithi053 ceph-mon[77587]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T23:05:21.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:21 smithi046 ceph-mon[36898]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:21 smithi046 ceph-mon[36898]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T23:05:23.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:23 smithi086 ceph-mon[78610]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:23 smithi053 ceph-mon[77587]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:23.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:23 smithi046 ceph-mon[36898]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:25 smithi086 ceph-mon[78610]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:25 smithi053 ceph-mon[77587]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:26.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:25 smithi046 ceph-mon[36898]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-08T23:05:27.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:27 smithi086 ceph-mon[78610]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T23:05:28.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:27 smithi053 ceph-mon[77587]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T23:05:28.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:27 smithi046 ceph-mon[36898]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T23:05:29.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:29 smithi086 ceph-mon[78610]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:30.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:29 smithi053 ceph-mon[77587]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:29 smithi046 ceph-mon[36898]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T23:05:31.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:31 smithi086 ceph-mon[78610]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-08T23:05:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:31 smithi053 ceph-mon[77587]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-08T23:05:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:31 smithi046 ceph-mon[36898]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-08T23:05:33.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:33 smithi086 ceph-mon[78610]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T23:05:34.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:33 smithi053 ceph-mon[77587]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T23:05:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:33 smithi046 ceph-mon[36898]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T23:05:35.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:35 smithi086 ceph-mon[78610]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T23:05:36.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:35 smithi053 ceph-mon[77587]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T23:05:36.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:35 smithi046 ceph-mon[36898]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T23:05:36.500 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 60% (7440/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.543 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.544 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T23:05:37.545 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-08T23:05:37.545 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:05:37.559 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T23:05:37.615 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T23:05:37.619 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:05:37.619 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T23:05:37.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:37 smithi086 ceph-mon[78610]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:05:38.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:37 smithi053 ceph-mon[77587]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:05:38.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:37 smithi046 ceph-mon[36898]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T23:05:38.365 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T23:05:38.456 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T23:05:38.520 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:05:38.524 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:05:38.524 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T23:05:38.587 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:05:38.594 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:05:38.594 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T23:05:38.636 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-08T23:05:38.637 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-08T23:05:38.637 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-08T23:05:38.757 INFO:tasks.workunit.client.0.smithi046.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-08T23:05:38.757 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-08T23:05:38.757 INFO:tasks.workunit.client.0.smithi046.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-08T23:05:38.768 INFO:tasks.workunit.client.0.smithi046.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-08T23:05:38.868 INFO:tasks.workunit.client.0.smithi046.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-08T23:05:38.868 INFO:tasks.workunit.client.0.smithi046.stdout:releasing clients 2024-05-08T23:05:39.868 INFO:tasks.workunit.client.0.smithi046.stdout: 1 2917 33.46 MB/sec warmup 1 sec latency 8.170 ms 2024-05-08T23:05:39.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:39 smithi086 ceph-mon[78610]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:40.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:39 smithi053 ceph-mon[77587]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:40.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:39 smithi046 ceph-mon[36898]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T23:05:40.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 4823 21.10 MB/sec warmup 2 sec latency 28.801 ms 2024-05-08T23:05:41.868 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6317 17.02 MB/sec warmup 3 sec latency 605.537 ms 2024-05-08T23:05:41.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:41 smithi086 ceph-mon[78610]: pgmap v59: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-08T23:05:41.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:41 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:05:42.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:41 smithi053 ceph-mon[77587]: pgmap v59: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-08T23:05:42.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:41 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:05:42.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:41 smithi046 ceph-mon[36898]: pgmap v59: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-08T23:05:42.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:41 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:05:42.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:05:42.868 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6317 12.77 MB/sec warmup 4 sec latency 1605.586 ms 2024-05-08T23:05:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.069+0000 7f1566800700 1 -- 172.21.15.46:0/2902031616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101a30 msgr2=0x7f1560105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.069+0000 7f1566800700 1 --2- 172.21.15.46:0/2902031616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101a30 0x7f1560105a70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f154c009990 tx=0x7f154c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 -- 172.21.15.46:0/2902031616 shutdown_connections 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 --2- 172.21.15.46:0/2902031616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f156010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 --2- 172.21.15.46:0/2902031616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560101080 0x7f1560101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 --2- 172.21.15.46:0/2902031616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101a30 0x7f1560105a70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 -- 172.21.15.46:0/2902031616 >> 172.21.15.46:0/2902031616 conn(0x7f15600fc8e0 msgr2=0x7f15600fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 -- 172.21.15.46:0/2902031616 shutdown_connections 2024-05-08T23:05:43.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.070+0000 7f1566800700 1 -- 172.21.15.46:0/2902031616 wait complete. 2024-05-08T23:05:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 Processor -- start 2024-05-08T23:05:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 -- start start 2024-05-08T23:05:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f15600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560073d20 0x7f15601a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a5b50 con 0x7f1560101080 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a5cc0 con 0x7f1560073d20 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.071+0000 7f1566800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601a5e80 con 0x7f15601061a0 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48000/0 (socket says 172.21.15.46:48000) 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155f400700 1 -- 172.21.15.46:0/1919363146 learned_addr learned my addr 172.21.15.46:0/1919363146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155fe00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560073d20 0x7f15601a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f15600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 -- 172.21.15.46:0/1919363146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560073d20 msgr2=0x7f15601a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560073d20 0x7f15601a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 -- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 msgr2=0x7f15600732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 -- 172.21.15.46:0/1919363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550009520 con 0x7f15601061a0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155f400700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.072+0000 7f155ea00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f15600737e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.073+0000 7f1557e00700 1 -- 172.21.15.46:0/1919363146 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c01d070 con 0x7f15601061a0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.073+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c0095f0 con 0x7f15601061a0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.073+0000 7f1557e00700 1 -- 172.21.15.46:0/1919363146 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154c021410 con 0x7f15601061a0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.073+0000 7f1557e00700 1 -- 172.21.15.46:0/1919363146 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c017650 con 0x7f15601061a0 2024-05-08T23:05:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.073+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15601a63f0 con 0x7f15601061a0 2024-05-08T23:05:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.076+0000 7f1557e00700 1 -- 172.21.15.46:0/1919363146 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f154c021580 con 0x7f15601061a0 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f157a351b80 1 Processor -- start 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f157a351b80 1 -- start start 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f1557e00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 0x7f15440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f157a351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f157a351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f1557e00700 1 -- 172.21.15.46:0/1919363146 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f154c0ce2c0 con 0x7f15601061a0 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.077+0000 7f157a351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f157a351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950b9f150 con 0x557950c4e720 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f157a351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950b9cb10 con 0x557950c48890 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f157a351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950b98400 con 0x557950c4de00 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155fe00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 0x7f15440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42758/0 (socket says 172.21.15.46:42758) 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 -- 172.21.15.46:0/294093091 learned_addr learned my addr 172.21.15.46:0/294093091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155ea00700 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155fe00700 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 -- 172.21.15.46:0/294093091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 msgr2=0x557950c48cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 -- 172.21.15.46:0/294093091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 msgr2=0x557950c48350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c48350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.078+0000 7f155f400700 1 -- 172.21.15.46:0/294093091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557950a193b0 con 0x557950c4de00 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f155f400700 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1550011840 tx=0x7f1550011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909b118b26cf930d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f155fe00700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 0x7f15440a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ec0 tx=0x7f1558016450 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f1554c00700 1 -- 172.21.15.46:0/294093091 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550011030 con 0x557950c4de00 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f1554c00700 1 -- 172.21.15.46:0/294093091 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550011190 con 0x557950c4de00 2024-05-08T23:05:43.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f1554c00700 1 -- 172.21.15.46:0/294093091 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500107b0 con 0x557950c4de00 2024-05-08T23:05:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f157a351b80 1 -- 172.21.15.46:0/294093091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 msgr2=0x557950c4e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.079+0000 7f157a351b80 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1550011840 tx=0x7f1550011b50 comp rx=0 tx=0).stop 2024-05-08T23:05:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 -- 172.21.15.46:0/294093091 shutdown_connections 2024-05-08T23:05:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c4e1e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 --2- 172.21.15.46:0/294093091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c48350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 -- 172.21.15.46:0/294093091 >> 172.21.15.46:0/294093091 conn(0x557950ba6210 msgr2=0x557950c42df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 -- 172.21.15.46:0/294093091 shutdown_connections 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 -- 172.21.15.46:0/294093091 wait complete. 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 -- start start 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.080+0000 7f157a351b80 1 Processor -- start 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 0x557950c51a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 0x557950c51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950b34560 con 0x557950c4e720 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950b34200 con 0x557950c4de00 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950a1b6a0 con 0x557950c48890 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f157a351b80 10 client.0 _ll_register_callbacks cb 0x557950b3db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 0x557950c51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 0x557950c51a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 0x557950c51fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46200/0 (socket says 172.21.15.46:46200) 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155ea00700 1 -- 172.21.15.46:0/3105923638 learned_addr learned my addr 172.21.15.46:0/3105923638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.081+0000 7f155fe00700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 -- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 msgr2=0x557950c51fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 0x557950c51fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 -- 172.21.15.46:0/3105923638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 msgr2=0x557950c52510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 -- 172.21.15.46:0/3105923638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500114a0 con 0x557950c48890 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155fe00700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f155f400700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 0x557950c51a90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1550011fd0 tx=0x7f1550005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2519ba9963ece494 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f1549600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550005ce0 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557950af1270 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1550004260 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.082+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1550004420 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 10 client.26215 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5579509c5b60 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557950af1270 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f155000f780 con 0x557950c48890 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 20 client.26215 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950c620d0 0x557950c644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 4 client.26215 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557950c17810 con 0x557950c620d0 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.083+0000 7f155ea00700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950c620d0 0x557950c644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.084+0000 7f155ea00700 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950c620d0 0x557950c644f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6165db4946e423c server_cookie=adab0f22bc33c6cd in_seq=0 out_seq=0 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.084+0000 7f1549600700 10 client.26215 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f1549600700 1 -- 172.21.15.46:0/3105923638 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557950c17810 con 0x557950c620d0 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f1549600700 10 client.26215 handle_command_reply: tid=0 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 client.26215 shutdown 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 msgr2=0x557950c51a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 0x557950c51a90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1550011fd0 tx=0x7f1550005700 comp rx=0 tx=0).stop 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 shutdown_connections 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950c620d0 0x557950c644f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c52510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c48890 0x557950c51a90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 --2- 172.21.15.46:0/3105923638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c4de00 0x557950c51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f1549600700 0 client.26215 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 >> 172.21.15.46:0/3105923638 conn(0x557950ba6210 msgr2=0x557950c425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 shutdown_connections 2024-05-08T23:05:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 1 -- 172.21.15.46:0/3105923638 wait complete. 2024-05-08T23:05:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.085+0000 7f157a351b80 20 client.26215 trim_cache size 0 max 16384 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.103+0000 7f157a351b80 1 Processor -- start 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 -- start start 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c5dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950c4ffe0 con 0x557950c4e720 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950c43260 con 0x557950c48890 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f157a351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950c4c450 con 0x557950c4de00 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f155f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f155fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f155f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46208/0 (socket says 172.21.15.46:46208) 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f155f400700 1 -- 172.21.15.46:0/1343648822 learned_addr learned my addr 172.21.15.46:0/1343648822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.104+0000 7f155ea00700 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c5dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 -- 172.21.15.46:0/1343648822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 msgr2=0x557950c5dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c5dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 -- 172.21.15.46:0/1343648822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 msgr2=0x557950c42d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c42d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 -- 172.21.15.46:0/1343648822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557950b5e560 con 0x557950c48890 2024-05-08T23:05:43.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f155f400700 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1550004020 tx=0x7f155000fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4bc89b918fd373d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f1555600700 1 -- 172.21.15.46:0/1343648822 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550011300 con 0x557950c48890 2024-05-08T23:05:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f1555600700 1 -- 172.21.15.46:0/1343648822 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f155000fe00 con 0x557950c48890 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.105+0000 7f1555600700 1 -- 172.21.15.46:0/1343648822 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550003cb0 con 0x557950c48890 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- 172.21.15.46:0/1343648822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 msgr2=0x557950c48350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1550004020 tx=0x7f155000fcb0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- 172.21.15.46:0/1343648822 shutdown_connections 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4e720 0x557950c42d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c48350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 --2- 172.21.15.46:0/1343648822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4de00 0x557950c5dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- 172.21.15.46:0/1343648822 >> 172.21.15.46:0/1343648822 conn(0x557950ba6210 msgr2=0x557950c48c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- 172.21.15.46:0/1343648822 shutdown_connections 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- 172.21.15.46:0/1343648822 wait complete. 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.106+0000 7f157a351b80 1 -- start start 2024-05-08T23:05:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 Processor -- start 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 0x557950c57690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 0x557950c57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950cb75d0 con 0x557950c4de00 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950cb7710 con 0x557950c48890 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557950cb7850 con 0x557950c4e720 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f157a351b80 10 client.0 _ll_register_callbacks cb 0x557950b3db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f155ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 0x557950c57690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f155f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f155f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46210/0 (socket says 172.21.15.46:46210) 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f155f400700 1 -- 172.21.15.46:0/3238907063 learned_addr learned my addr 172.21.15.46:0/3238907063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.107+0000 7f155fe00700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 0x557950c57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 -- 172.21.15.46:0/3238907063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 msgr2=0x557950c57bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 0x557950c57bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 -- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 msgr2=0x557950c57150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 -- 172.21.15.46:0/3238907063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579509c5b60 con 0x557950c4de00 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155fe00700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 0x557950c57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155f400700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f155ea00700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 0x557950c57690 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f154c0cb660 tx=0x7f154c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc83df413997a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f1549600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c09b750 con 0x557950c4de00 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f154c09b050 con 0x557950c4de00 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1550011190 con 0x557950c4de00 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c09ee80 con 0x557950c4de00 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f157a351b80 10 client.16197 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.108+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557950a1dcd0 con 0x557950c4de00 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1550011190 con 0x557950c4de00 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f154c021860 con 0x557950c4de00 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 20 client.16197 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950cb8d50 0x557950cbb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 4 client.16197 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557950b90ec0 con 0x557950cb8d50 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.109+0000 7f155f400700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950cb8d50 0x557950cbb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f155f400700 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950cb8d50 0x557950cbb170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2598d538934a35b2 server_cookie=ebecb60cd3990ae5 in_seq=0 out_seq=0 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f1549600700 10 client.16197 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f1549600700 1 -- 172.21.15.46:0/3238907063 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557950b90ec0 con 0x557950cb8d50 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f1549600700 10 client.16197 handle_command_reply: tid=0 2024-05-08T23:05:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 client.16197 shutdown 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 msgr2=0x557950c57690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 0x557950c57690 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f154c0cb660 tx=0x7f154c0047a0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 shutdown_connections 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557950c4e720 0x557950c57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557950c48890 0x557950c57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557950cb8d50 0x557950cbb170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 --2- 172.21.15.46:0/3238907063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557950c4de00 0x557950c57690 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 >> 172.21.15.46:0/3238907063 conn(0x557950ba6210 msgr2=0x557950c4bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f1549600700 0 client.16197 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.110+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 shutdown_connections 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f157a351b80 1 -- 172.21.15.46:0/3238907063 wait complete. 2024-05-08T23:05:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f157a351b80 20 client.16197 trim_cache size 0 max 16384 2024-05-08T23:05:43.117 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:43.117 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:05:43.117 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:05:43.117 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 msgr2=0x7f15440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 0x7f15440a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ec0 tx=0x7f1558016450 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 msgr2=0x7f15600737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f15600737e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004730 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 shutdown_connections 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1560073d20 0x7f15601a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f15440a1290 0x7f15440a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1560101080 0x7f15600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 --2- 172.21.15.46:0/1919363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15601061a0 0x7f15600737e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 >> 172.21.15.46:0/1919363146 conn(0x7f15600fc8e0 msgr2=0x7f15600fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.111+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 shutdown_connections 2024-05-08T23:05:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.112+0000 7f1566800700 1 -- 172.21.15.46:0/1919363146 wait complete. 2024-05-08T23:05:43.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:05:43.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 -- 172.21.15.46:0/7064007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101a20 msgr2=0x7fa9e0105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/7064007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101a20 0x7fa9e0105a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009990 tx=0x7fa9cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 -- 172.21.15.46:0/7064007 shutdown_connections 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/7064007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 0x7fa9e010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/7064007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e0101070 0x7fa9e0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/7064007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101a20 0x7fa9e0105a60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 -- 172.21.15.46:0/7064007 >> 172.21.15.46:0/7064007 conn(0x7fa9e00fc930 msgr2=0x7fa9e00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 -- 172.21.15.46:0/7064007 shutdown_connections 2024-05-08T23:05:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.369+0000 7fa9e7a00700 1 -- 172.21.15.46:0/7064007 wait complete. 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.370+0000 7fa9e7a00700 1 Processor -- start 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 -- start start 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 0x7fa9e0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 0x7fa9e019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e019c6b0 con 0x7fa9e0101070 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e019c820 con 0x7fa9e019bc30 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.371+0000 7fa9e7a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e019c9e0 con 0x7fa9e0106190 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48050/0 (socket says 172.21.15.46:48050) 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 -- 172.21.15.46:0/3284828897 learned_addr learned my addr 172.21.15.46:0/3284828897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 0x7fa9e019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9dfe00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 0x7fa9e0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 -- 172.21.15.46:0/3284828897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 msgr2=0x7fa9e0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 0x7fa9e0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 -- 172.21.15.46:0/3284828897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 msgr2=0x7fa9e019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 0x7fa9e019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 -- 172.21.15.46:0/3284828897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0095f0 con 0x7fa9e0101070 2024-05-08T23:05:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 0x7fa9e019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.372+0000 7fa9e5000700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e9a0 tx=0x7fa9d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.373+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4009790 con 0x7fa9e0101070 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.373+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9d4010430 con 0x7fa9e0101070 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.373+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e01a3f40 con 0x7fa9e0101070 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.373+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4018680 con 0x7fa9e0101070 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.373+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e01a43c0 con 0x7fa9e0101070 2024-05-08T23:05:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.374+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0109460 con 0x7fa9e0101070 2024-05-08T23:05:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.376+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa9d40105a0 con 0x7fa9e0101070 2024-05-08T23:05:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.377+0000 7fa9dd600700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 0x7fa9c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.377+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa9d4014070 con 0x7fa9e0101070 2024-05-08T23:05:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.377+0000 7fa9dfe00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 0x7fa9c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.378+0000 7fa9dfe00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 0x7fa9c80a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9e01988a0 tx=0x7fa9cc00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.382+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9d4052d60 con 0x7fa9e0101070 2024-05-08T23:05:43.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.652+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9e01986b0 con 0x7fa9e0101070 2024-05-08T23:05:43.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.653+0000 7fa9dd600700 1 -- 172.21.15.46:0/3284828897 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10907 (secure 0 0 0) 0x7fa9d4096300 con 0x7fa9e0101070 2024-05-08T23:05:43.653 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:05:43.654 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:05:40.700994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 msgr2=0x7fa9c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 0x7fa9c80a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9e01988a0 tx=0x7fa9cc00b260 comp rx=0 tx=0).stop 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 msgr2=0x7fa9e0197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e9a0 tx=0x7fa9d400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 shutdown_connections 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e019bc30 0x7fa9e019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9c80a1290 0x7fa9c80a3750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e0101070 0x7fa9e0197350 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 --2- 172.21.15.46:0/3284828897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e0106190 0x7fa9e0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 >> 172.21.15.46:0/3284828897 conn(0x7fa9e00fc930 msgr2=0x7fa9e00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 shutdown_connections 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.656+0000 7fa9e7a00700 1 -- 172.21.15.46:0/3284828897 wait complete. 2024-05-08T23:05:43.657 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-05-08T23:05:43.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:05:43.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6317 10.21 MB/sec warmup 5 sec latency 2605.637 ms 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.877+0000 7f88df800700 1 -- 172.21.15.46:0/3080812542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101a30 msgr2=0x7f88d8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.877+0000 7f88df800700 1 --2- 172.21.15.46:0/3080812542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101a30 0x7f88d8105a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009990 tx=0x7f88c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 -- 172.21.15.46:0/3080812542 shutdown_connections 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 --2- 172.21.15.46:0/3080812542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d81061a0 0x7f88d810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 --2- 172.21.15.46:0/3080812542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d8101080 0x7f88d8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 --2- 172.21.15.46:0/3080812542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101a30 0x7f88d8105a70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 -- 172.21.15.46:0/3080812542 >> 172.21.15.46:0/3080812542 conn(0x7f88d80fc8e0 msgr2=0x7f88d80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 -- 172.21.15.46:0/3080812542 shutdown_connections 2024-05-08T23:05:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.878+0000 7f88df800700 1 -- 172.21.15.46:0/3080812542 wait complete. 2024-05-08T23:05:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 Processor -- start 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 -- start start 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81061a0 0x7f88d80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 0x7f88d81a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5b50 con 0x7f88d8073d20 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5c90 con 0x7f88d8101080 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88df800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81a5e50 con 0x7f88d81061a0 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.879+0000 7f88dce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81061a0 0x7f88d80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dd800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 0x7f88d81a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46242/0 (socket says 172.21.15.46:46242) 2024-05-08T23:05:43.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 -- 172.21.15.46:0/1952489333 learned_addr learned my addr 172.21.15.46:0/1952489333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 -- 172.21.15.46:0/1952489333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81061a0 msgr2=0x7f88d80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81061a0 0x7f88d80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 -- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 msgr2=0x7f88d81a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 0x7f88d81a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 -- 172.21.15.46:0/1952489333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009c40 con 0x7f88d8101080 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dd800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 0x7f88d81a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88dce00700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00e9c0 tx=0x7f88cc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88d5600700 1 -- 172.21.15.46:0/1952489333 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0097b0 con 0x7f88d8101080 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.880+0000 7f88d5600700 1 -- 172.21.15.46:0/1952489333 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88cc00ee50 con 0x7f88d8101080 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.881+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c40095f0 con 0x7f88d8101080 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.881+0000 7f88d5600700 1 -- 172.21.15.46:0/1952489333 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0054f0 con 0x7f88d8101080 2024-05-08T23:05:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.881+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d81a64b0 con 0x7f88d8101080 2024-05-08T23:05:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.884+0000 7f88d5600700 1 -- 172.21.15.46:0/1952489333 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f88cc004750 con 0x7f88d8101080 2024-05-08T23:05:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88f3370b80 1 Processor -- start 2024-05-08T23:05:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88d5600700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 0x7f88c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88f3370b80 1 -- start start 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88d5600700 1 -- 172.21.15.46:0/1952489333 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f88cc014070 con 0x7f88d8101080 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88dd800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 0x7f88c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 0x557576d541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.885+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d4e890 0x557576d4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88f3370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576ca5150 con 0x557576d4e890 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88f3370b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576ca2b10 con 0x557576d54720 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88f3370b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576c9e400 con 0x557576d53e00 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46246/0 (socket says 172.21.15.46:46246) 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 -- 172.21.15.46:0/2798247248 learned_addr learned my addr 172.21.15.46:0/2798247248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88dce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 0x557576d541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88dd800700 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d4e890 0x557576d4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 -- 172.21.15.46:0/2798247248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 msgr2=0x557576d541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88dd800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 0x7f88c00a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009fd0 tx=0x7f88d0009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 0x557576d541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 -- 172.21.15.46:0/2798247248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d4e890 msgr2=0x557576d4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d4e890 0x557576d4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88d7e00700 1 -- 172.21.15.46:0/2798247248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557576b1f3b0 con 0x557576d54720 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.886+0000 7f88dce00700 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 0x557576d541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.887+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009990 tx=0x7f88c400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6217924aeaae15e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.887+0000 7f88ca000700 1 -- 172.21.15.46:0/2798247248 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c401d070 con 0x557576d54720 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.887+0000 7f88ca000700 1 -- 172.21.15.46:0/2798247248 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c401e040 con 0x557576d54720 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.887+0000 7f88ca000700 1 -- 172.21.15.46:0/2798247248 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40048c0 con 0x557576d54720 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- 172.21.15.46:0/2798247248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 msgr2=0x557576d4e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009990 tx=0x7f88c400b690 comp rx=0 tx=0).stop 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- 172.21.15.46:0/2798247248 shutdown_connections 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d4e890 0x557576d4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d53e00 0x557576d541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2798247248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4e350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- 172.21.15.46:0/2798247248 >> 172.21.15.46:0/2798247248 conn(0x557576cac210 msgr2=0x557576d48df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- 172.21.15.46:0/2798247248 shutdown_connections 2024-05-08T23:05:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- 172.21.15.46:0/2798247248 wait complete. 2024-05-08T23:05:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.888+0000 7f88f3370b80 1 -- start start 2024-05-08T23:05:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 Processor -- start 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d57b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576c3a560 con 0x557576d53e00 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576c3a200 con 0x557576d54720 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576b216a0 con 0x557576d4e890 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88f3370b80 10 client.0 _ll_register_callbacks cb 0x557576c43b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48070/0 (socket says 172.21.15.46:48070) 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88d7e00700 1 -- 172.21.15.46:0/2302800876 learned_addr learned my addr 172.21.15.46:0/2302800876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88dce00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d57b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.889+0000 7f88dd800700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 -- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 msgr2=0x557576d57b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d57b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 -- 172.21.15.46:0/2302800876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 msgr2=0x557576d58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 -- 172.21.15.46:0/2302800876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d81a64b0 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009ce0 tx=0x7f88c400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644846e9f92fecc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88bea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c401d070 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f88c4004910 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557576bf7270 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.890+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c40059d0 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 10 client.16215 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f88d810e590 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557576bf7270 con 0x557576d53e00 2024-05-08T23:05:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f88c400f910 con 0x557576d53e00 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 10 client.16215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 20 client.16215 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576d67d80 0x557576d6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 4 client.16215 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.891+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557576d1d810 con 0x557576d67d80 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.892+0000 7f88dce00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576d67d80 0x557576d6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.892+0000 7f88dce00700 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576d67d80 0x557576d6a1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43060e00f8ebdaa3 server_cookie=d0b642a698cdde34 in_seq=0 out_seq=0 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.892+0000 7f88bea00700 10 client.16215 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88bea00700 1 -- 172.21.15.46:0/2302800876 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557576d1d810 con 0x557576d67d80 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88bea00700 10 client.16215 handle_command_reply: tid=0 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 client.16215 shutdown 2024-05-08T23:05:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 msgr2=0x557576d580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009ce0 tx=0x7f88c400bd20 comp rx=0 tx=0).stop 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 shutdown_connections 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576d67d80 0x557576d6a1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d57b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2302800876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d580d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88bea00700 0 client.16215 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 >> 172.21.15.46:0/2302800876 conn(0x557576cac210 msgr2=0x557576d485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.893+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 shutdown_connections 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.894+0000 7f88f3370b80 1 -- 172.21.15.46:0/2302800876 wait complete. 2024-05-08T23:05:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.894+0000 7f88f3370b80 20 client.16215 trim_cache size 0 max 16384 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 Processor -- start 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 -- start start 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d63bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d4a870 con 0x557576d53e00 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d64d10 con 0x557576d54720 2024-05-08T23:05:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88f3370b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d59230 con 0x557576d4e890 2024-05-08T23:05:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d63bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88dd800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88dce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88dd800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46268/0 (socket says 172.21.15.46:46268) 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.908+0000 7f88dd800700 1 -- 172.21.15.46:0/2754169799 learned_addr learned my addr 172.21.15.46:0/2754169799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 -- 172.21.15.46:0/2754169799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 msgr2=0x557576d541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 -- 172.21.15.46:0/2754169799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 msgr2=0x557576d4ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 -- 172.21.15.46:0/2754169799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557576acbb60 con 0x557576d53e00 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d63bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f88c400fd70 tx=0x7f88c400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25cfc0cc3de98f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88caa00700 1 -- 172.21.15.46:0/2754169799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4045070 con 0x557576d53e00 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88caa00700 1 -- 172.21.15.46:0/2754169799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c4005320 con 0x557576d53e00 2024-05-08T23:05:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.909+0000 7f88caa00700 1 -- 172.21.15.46:0/2754169799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c401d070 con 0x557576d53e00 2024-05-08T23:05:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- 172.21.15.46:0/2754169799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 msgr2=0x557576d63bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d63bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f88c400fd70 tx=0x7f88c400ff90 comp rx=0 tx=0).stop 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- 172.21.15.46:0/2754169799 shutdown_connections 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2754169799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d63bc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- 172.21.15.46:0/2754169799 >> 172.21.15.46:0/2754169799 conn(0x557576cac210 msgr2=0x557576d45210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- 172.21.15.46:0/2754169799 shutdown_connections 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- 172.21.15.46:0/2754169799 wait complete. 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.910+0000 7f88f3370b80 1 -- start start 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 Processor -- start 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d5f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d5d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d5d600 con 0x557576d53e00 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d5d770 con 0x557576d54720 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557576d5d930 con 0x557576d4e890 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88f3370b80 10 client.0 _ll_register_callbacks cb 0x557576c43b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.911+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88dd800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d5d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48098/0 (socket says 172.21.15.46:48098) 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 -- 172.21.15.46:0/2188597380 learned_addr learned my addr 172.21.15.46:0/2188597380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88dce00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d5f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 -- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 msgr2=0x557576d5f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d5f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 -- 172.21.15.46:0/2188597380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 msgr2=0x557576d5d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d5d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 -- 172.21.15.46:0/2188597380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d810e590 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88d7e00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005ac0 tx=0x7f88c400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb879b93da0f283 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88bea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4045040 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f88c4000e40 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.912+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557576c96ec0 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c401d040 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 10 client.16221 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557576c64560 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557576c96ec0 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f88c4040400 con 0x557576d53e00 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 10 client.16221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:05:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 20 client.16221 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576dbdd10 0x557576dc0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 4 client.16221 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557576aeccf0 con 0x557576dbdd10 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.913+0000 7f88dce00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576dbdd10 0x557576dc0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88dce00700 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576dbdd10 0x557576dc0130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84e04012299e0f0b server_cookie=75966dc7f7aad4ac in_seq=0 out_seq=0 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88bea00700 10 client.16221 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88bea00700 1 -- 172.21.15.46:0/2188597380 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557576aeccf0 con 0x557576dbdd10 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88bea00700 10 client.16221 handle_command_reply: tid=0 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 client.16221 shutdown 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 msgr2=0x557576d5f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005ac0 tx=0x7f88c400bcc0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 shutdown_connections 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557576dbdd10 0x557576dc0130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557576d54720 0x557576d5d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557576d4e890 0x557576d5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 --2- 172.21.15.46:0/2188597380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557576d53e00 0x557576d5f620 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 >> 172.21.15.46:0/2188597380 conn(0x557576cac210 msgr2=0x557576d4ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88bea00700 0 client.16221 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.914+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 shutdown_connections 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88f3370b80 1 -- 172.21.15.46:0/2188597380 wait complete. 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88f3370b80 20 client.16221 trim_cache size 0 max 16384 2024-05-08T23:05:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 msgr2=0x7f88c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 0x7f88c00a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009fd0 tx=0x7f88d0009920 comp rx=0 tx=0).stop 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 msgr2=0x7f88d80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00e9c0 tx=0x7f88cc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 shutdown_connections 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88c00a1310 0x7f88c00a37d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88d8073d20 0x7f88d81a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88d8101080 0x7f88d80732a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 --2- 172.21.15.46:0/1952489333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81061a0 0x7f88d80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 >> 172.21.15.46:0/1952489333 conn(0x7f88d80fc8e0 msgr2=0x7f88d80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 shutdown_connections 2024-05-08T23:05:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:43.915+0000 7f88df800700 1 -- 172.21.15.46:0/1952489333 wait complete. 2024-05-08T23:05:43.931 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:43 smithi046 ceph-mon[36898]: pgmap v60: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-08T23:05:43.931 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:43 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3284828897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:43.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:05:43.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:43 smithi086 ceph-mon[78610]: pgmap v60: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-08T23:05:43.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:43 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3284828897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:44.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:43 smithi053 ceph-mon[77587]: pgmap v60: 129 pgs: 129 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-08T23:05:44.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:43 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3284828897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:05:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.136+0000 7faacf000700 1 -- 172.21.15.46:0/2271382596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 msgr2=0x7faac810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.136+0000 7faacf000700 1 --2- 172.21.15.46:0/2271382596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac810d880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faac000b0a0 tx=0x7faac000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 -- 172.21.15.46:0/2271382596 shutdown_connections 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 --2- 172.21.15.46:0/2271382596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac810d880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 --2- 172.21.15.46:0/2271382596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8101fa0 0x7faac8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 --2- 172.21.15.46:0/2271382596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac81028c0 0x7faac810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 -- 172.21.15.46:0/2271382596 >> 172.21.15.46:0/2271382596 conn(0x7faac80fb760 msgr2=0x7faac80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 -- 172.21.15.46:0/2271382596 shutdown_connections 2024-05-08T23:05:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.137+0000 7faacf000700 1 -- 172.21.15.46:0/2271382596 wait complete. 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 Processor -- start 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 -- start start 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac81028c0 0x7faac8100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac8100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81a5780 con 0x7faac810b3c0 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81a58f0 con 0x7faac8101fa0 2024-05-08T23:05:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacf000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81a5ab0 con 0x7faac81028c0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faac7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faacd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac8100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faac7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46292/0 (socket says 172.21.15.46:46292) 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.138+0000 7faac7e00700 1 -- 172.21.15.46:0/4186883711 learned_addr learned my addr 172.21.15.46:0/4186883711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faac7400700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac81028c0 0x7faac8100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 -- 172.21.15.46:0/4186883711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac81028c0 msgr2=0x7faac8100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac81028c0 0x7faac8100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 -- 172.21.15.46:0/4186883711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 msgr2=0x7faac80ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 -- 172.21.15.46:0/4186883711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009d00 con 0x7faac810b3c0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faac7e00700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacd000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac8100670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faac000b070 tx=0x7faac0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faac4c00700 1 -- 172.21.15.46:0/4186883711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000e040 con 0x7faac810b3c0 2024-05-08T23:05:44.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac81a5d30 con 0x7faac810b3c0 2024-05-08T23:05:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faac4c00700 1 -- 172.21.15.46:0/4186883711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faac001d470 con 0x7faac810b3c0 2024-05-08T23:05:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.139+0000 7faac4c00700 1 -- 172.21.15.46:0/4186883711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0005280 con 0x7faac810b3c0 2024-05-08T23:05:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.140+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac81a61f0 con 0x7faac810b3c0 2024-05-08T23:05:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.143+0000 7faac4c00700 1 -- 172.21.15.46:0/4186883711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faac001d5e0 con 0x7faac810b3c0 2024-05-08T23:05:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.144+0000 7faae2a86b80 1 Processor -- start 2024-05-08T23:05:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.144+0000 7faac4c00700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 0x7faaac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.144+0000 7faae2a86b80 1 -- start start 2024-05-08T23:05:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.144+0000 7faac4c00700 1 -- 172.21.15.46:0/4186883711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faac00ce3e0 con 0x7faac810b3c0 2024-05-08T23:05:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.144+0000 7faac7400700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 0x7faaac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db084350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db084890 0x55f2db084cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2dafdb150 con 0x55f2db089e00 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2dafd8b10 con 0x55f2db08a720 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faae2a86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2dafd4400 con 0x55f2db084890 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48118/0 (socket says 172.21.15.46:48118) 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 -- 172.21.15.46:0/1952402150 learned_addr learned my addr 172.21.15.46:0/1952402150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7400700 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db084350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faacd000700 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db084890 0x55f2db084cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7400700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 0x7faaac0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faac81012d0 tx=0x7faab4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 -- 172.21.15.46:0/1952402150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db084890 msgr2=0x55f2db084cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db084890 0x55f2db084cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 -- 172.21.15.46:0/1952402150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 msgr2=0x55f2db084350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db084350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.145+0000 7faac7e00700 1 -- 172.21.15.46:0/1952402150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2dae553b0 con 0x55f2db089e00 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.146+0000 7faac7e00700 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faab80117e0 tx=0x7faab8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1615f1b21d064bc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.146+0000 7faabd600700 1 -- 172.21.15.46:0/1952402150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800f480 con 0x55f2db089e00 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.146+0000 7faabd600700 1 -- 172.21.15.46:0/1952402150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab800faa0 con 0x55f2db089e00 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.146+0000 7faabd600700 1 -- 172.21.15.46:0/1952402150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8019a00 con 0x55f2db089e00 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- 172.21.15.46:0/1952402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 msgr2=0x55f2db08a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faab80117e0 tx=0x7faab8011af0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- 172.21.15.46:0/1952402150 shutdown_connections 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db084890 0x55f2db084cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db08a1e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1952402150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db084350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- 172.21.15.46:0/1952402150 >> 172.21.15.46:0/1952402150 conn(0x55f2dafe2210 msgr2=0x55f2db07edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- 172.21.15.46:0/1952402150 shutdown_connections 2024-05-08T23:05:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- 172.21.15.46:0/1952402150 wait complete. 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 -- start start 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.147+0000 7faae2a86b80 1 Processor -- start 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db08da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db08df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2daf70560 con 0x55f2db08a720 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2daf70200 con 0x55f2db084890 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2dae576a0 con 0x55f2db089e00 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faae2a86b80 10 client.0 _ll_register_callbacks cb 0x55f2daf79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faacd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faac7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db08da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db08df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faacd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48132/0 (socket says 172.21.15.46:48132) 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.148+0000 7faacd000700 1 -- 172.21.15.46:0/726299654 learned_addr learned my addr 172.21.15.46:0/726299654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 -- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 msgr2=0x55f2db08df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db08df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 -- 172.21.15.46:0/726299654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 msgr2=0x55f2db08da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db08da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 -- 172.21.15.46:0/726299654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab8011440 con 0x55f2db08a720 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faacd000700 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faac000b0a0 tx=0x7faac009ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d61929a8c3a2202 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faab2000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac001dac0 con 0x55f2db08a720 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faac009efa0 con 0x55f2db08a720 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2daf2d270 con 0x55f2db08a720 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac00a3560 con 0x55f2db08a720 2024-05-08T23:05:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faae2a86b80 10 client.16233 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.149+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f2dae01b60 con 0x55f2db08a720 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2daf2d270 con 0x55f2db08a720 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faac009b460 con 0x55f2db08a720 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 10 client.16233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 20 client.16233 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db09de90 0x55f2db0a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 4 client.16233 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2db053810 con 0x55f2db09de90 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.150+0000 7faac7e00700 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db09de90 0x55f2db0a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.151+0000 7faac7e00700 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db09de90 0x55f2db0a02b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20df303f3634d43a server_cookie=18983495fee8503d in_seq=0 out_seq=0 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.151+0000 7faab2000700 10 client.16233 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.151+0000 7faab2000700 1 -- 172.21.15.46:0/726299654 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2db053810 con 0x55f2db09de90 2024-05-08T23:05:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.151+0000 7faab2000700 10 client.16233 handle_command_reply: tid=0 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 client.16233 shutdown 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 msgr2=0x55f2db08e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faac000b0a0 tx=0x7faac009ab50 comp rx=0 tx=0).stop 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 shutdown_connections 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db09de90 0x55f2db0a02b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db08a720 0x55f2db08e4d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db08da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 --2- 172.21.15.46:0/726299654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db08df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faab2000700 0 client.16233 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 >> 172.21.15.46:0/726299654 conn(0x55f2dafe2210 msgr2=0x55f2db07e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 shutdown_connections 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 1 -- 172.21.15.46:0/726299654 wait complete. 2024-05-08T23:05:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.152+0000 7faae2a86b80 20 client.16233 trim_cache size 0 max 16384 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 Processor -- start 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 -- start start 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db07ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db090690 con 0x55f2db084890 2024-05-08T23:05:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db08f0f0 con 0x55f2db08a720 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.166+0000 7faae2a86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db080e50 con 0x55f2db089e00 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42918/0 (socket says 172.21.15.46:42918) 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 -- 172.21.15.46:0/1057284898 learned_addr learned my addr 172.21.15.46:0/1057284898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48148/0 (socket says 172.21.15.46:48148) 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faacd000700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db07ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 -- 172.21.15.46:0/1057284898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 msgr2=0x55f2db07ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db07ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 -- 172.21.15.46:0/1057284898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 msgr2=0x55f2db080640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 -- 172.21.15.46:0/1057284898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2daf9a560 con 0x55f2db089e00 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faacd000700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db07ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:44.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7e00700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.167+0000 7faac7400700 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faab4015000 tx=0x7faab4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f11743389a5439 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faabe000700 1 -- 172.21.15.46:0/1057284898 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab402a070 con 0x55f2db089e00 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faabe000700 1 -- 172.21.15.46:0/1057284898 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab4022040 con 0x55f2db089e00 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faabe000700 1 -- 172.21.15.46:0/1057284898 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4021070 con 0x55f2db089e00 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 -- 172.21.15.46:0/1057284898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 msgr2=0x55f2db084c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faab4015000 tx=0x7faab4014000 comp rx=0 tx=0).stop 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 -- 172.21.15.46:0/1057284898 shutdown_connections 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db08a720 0x55f2db07ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db084890 0x55f2db080640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 --2- 172.21.15.46:0/1057284898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db089e00 0x55f2db084c70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 -- 172.21.15.46:0/1057284898 >> 172.21.15.46:0/1057284898 conn(0x55f2dafe2210 msgr2=0x55f2db07b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 -- 172.21.15.46:0/1057284898 shutdown_connections 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.168+0000 7faae2a86b80 1 -- 172.21.15.46:0/1057284898 wait complete. 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.169+0000 7faae2a86b80 1 -- start start 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.169+0000 7faae2a86b80 1 Processor -- start 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.169+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db095610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.169+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db086c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.169+0000 7faae2a86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faae2a86b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db0859f0 con 0x55f2db089e00 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faae2a86b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db085b60 con 0x55f2db084890 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faae2a86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2db085d20 con 0x55f2db08a720 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faae2a86b80 10 client.0 _ll_register_callbacks cb 0x55f2daf79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faac7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db086c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42926/0 (socket says 172.21.15.46:42926) 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 -- 172.21.15.46:0/2587255972 learned_addr learned my addr 172.21.15.46:0/2587255972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faac7e00700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db095610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 -- 172.21.15.46:0/2587255972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 msgr2=0x55f2db095610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db095610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 -- 172.21.15.46:0/2587255972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 msgr2=0x55f2db086c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db086c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 -- 172.21.15.46:0/2587255972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2dae553b0 con 0x55f2db08a720 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faac7400700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db086c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faacd000700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faac009a680 tx=0x7faac001db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7bab1c1051449 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.170+0000 7faab2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac009b5d0 con 0x55f2db08a720 2024-05-08T23:05:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faac009efa0 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab800faa0 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac00a12c0 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 10 client.26263 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7faab4017050 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab800faa0 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faac009a0d0 con 0x55f2db08a720 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 10 client.26263 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 10 client.26263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 20 client.26263 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db0f4a50 0x55f2db0f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 4 client.26263 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.171+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2dafccec0 con 0x55f2db0f4a50 2024-05-08T23:05:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.172+0000 7faac7e00700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db0f4a50 0x55f2db0f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.172+0000 7faac7e00700 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db0f4a50 0x55f2db0f6e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb45e3fd65e273a8 server_cookie=3684a3d3a22d50db in_seq=0 out_seq=0 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.172+0000 7faab2000700 10 client.26263 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faab2000700 1 -- 172.21.15.46:0/2587255972 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2dafccec0 con 0x55f2db0f4a50 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faab2000700 10 client.26263 handle_command_reply: tid=0 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 client.26263 shutdown 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 msgr2=0x55f2db087270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faac009a680 tx=0x7faac001db70 comp rx=0 tx=0).stop 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 shutdown_connections 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f2db0f4a50 0x55f2db0f6e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2db08a720 0x55f2db087270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2db084890 0x55f2db095610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 --2- 172.21.15.46:0/2587255972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2db089e00 0x55f2db086c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faab2000700 0 client.26263 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 >> 172.21.15.46:0/2587255972 conn(0x55f2dafe2210 msgr2=0x55f2db08c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 shutdown_connections 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 1 -- 172.21.15.46:0/2587255972 wait complete. 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.173+0000 7faae2a86b80 20 client.26263 trim_cache size 0 max 16384 2024-05-08T23:05:44.178 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 msgr2=0x7faaac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 0x7faaac0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faac81012d0 tx=0x7faab4009260 comp rx=0 tx=0).stop 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 msgr2=0x7faac8100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac8100670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faac000b070 tx=0x7faac0007860 comp rx=0 tx=0).stop 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 shutdown_connections 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faaac0a1290 0x7faaac0a3750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810b3c0 0x7faac8100670 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac8101fa0 0x7faac80ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 --2- 172.21.15.46:0/4186883711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac81028c0 0x7faac8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 >> 172.21.15.46:0/4186883711 conn(0x7faac80fb760 msgr2=0x7faac80fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 shutdown_connections 2024-05-08T23:05:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.174+0000 7faacf000700 1 -- 172.21.15.46:0/4186883711 wait complete. 2024-05-08T23:05:44.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.393+0000 7f90da400700 1 -- 172.21.15.46:0/952165368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 msgr2=0x7f90d4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.393+0000 7f90da400700 1 --2- 172.21.15.46:0/952165368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4101460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009990 tx=0x7f90c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 -- 172.21.15.46:0/952165368 shutdown_connections 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 --2- 172.21.15.46:0/952165368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 --2- 172.21.15.46:0/952165368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4101460 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 --2- 172.21.15.46:0/952165368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d4101a30 0x7f90d4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 -- 172.21.15.46:0/952165368 >> 172.21.15.46:0/952165368 conn(0x7f90d40fc8e0 msgr2=0x7f90d40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 -- 172.21.15.46:0/952165368 shutdown_connections 2024-05-08T23:05:44.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.394+0000 7f90da400700 1 -- 172.21.15.46:0/952165368 wait complete. 2024-05-08T23:05:44.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.395+0000 7f90da400700 1 Processor -- start 2024-05-08T23:05:44.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.395+0000 7f90da400700 1 -- start start 2024-05-08T23:05:44.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d419de30 0x7f90d419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419e8b0 con 0x7f90d4101080 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419ea20 con 0x7f90d41061a0 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90da400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419ebe0 con 0x7f90d419de30 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48162/0 (socket says 172.21.15.46:48162) 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90d3400700 1 -- 172.21.15.46:0/722655207 learned_addr learned my addr 172.21.15.46:0/722655207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90d2a00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.396+0000 7f90d3e00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d419de30 0x7f90d419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 -- 172.21.15.46:0/722655207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d419de30 msgr2=0x7f90d419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d419de30 0x7f90d419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 -- 172.21.15.46:0/722655207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 msgr2=0x7f90d4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d4199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 -- 172.21.15.46:0/722655207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40095f0 con 0x7f90d4101080 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d2a00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90d3400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90c4000c00 tx=0x7f90c4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90cbe00700 1 -- 172.21.15.46:0/722655207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c401d070 con 0x7f90d4101080 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90cbe00700 1 -- 172.21.15.46:0/722655207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c4022470 con 0x7f90d4101080 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90cbe00700 1 -- 172.21.15.46:0/722655207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4017650 con 0x7f90d4101080 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d41a60e0 con 0x7f90d4101080 2024-05-08T23:05:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.397+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d41a64b0 con 0x7f90d4101080 2024-05-08T23:05:44.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.400+0000 7f90cbe00700 1 -- 172.21.15.46:0/722655207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f90c40225e0 con 0x7f90d4101080 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.401+0000 7f90ede40b80 1 Processor -- start 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90cbe00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 0x7f90b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90cbe00700 1 -- 172.21.15.46:0/722655207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90c40ce2e0 con 0x7f90d4101080 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 -- start start 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90d3e00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 0x7f90b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 0x563d9b5d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b527150 con 0x563d9b5de660 2024-05-08T23:05:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b524b10 con 0x563d9b5d90f0 2024-05-08T23:05:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90ede40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b520400 con 0x563d9b5def80 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.402+0000 7f90d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42952/0 (socket says 172.21.15.46:42952) 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 -- 172.21.15.46:0/683117350 learned_addr learned my addr 172.21.15.46:0/683117350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d3e00700 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 0x563d9b5d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 -- 172.21.15.46:0/683117350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 msgr2=0x563d9b5d9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 0x563d9b5d9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 -- 172.21.15.46:0/683117350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 msgr2=0x563d9b5dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 -- 172.21.15.46:0/683117350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9b3a13b0 con 0x563d9b5def80 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d3e00700 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 0x563d9b5d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d3e00700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 0x7f90b80a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00f590 tx=0x7f90cc00b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90d2a00700 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90c0011540 tx=0x7f90c0011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3223afccc1476efd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90c8c00700 1 -- 172.21.15.46:0/683117350 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0004a30 con 0x563d9b5def80 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90c8c00700 1 -- 172.21.15.46:0/683117350 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c00054e0 con 0x563d9b5def80 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.403+0000 7f90c8c00700 1 -- 172.21.15.46:0/683117350 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c000e830 con 0x563d9b5def80 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 -- 172.21.15.46:0/683117350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 msgr2=0x563d9b5d8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90c0011540 tx=0x7f90c0011850 comp rx=0 tx=0).stop 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 -- 172.21.15.46:0/683117350 shutdown_connections 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5d90f0 0x563d9b5d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 --2- 172.21.15.46:0/683117350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5d8b60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 -- 172.21.15.46:0/683117350 >> 172.21.15.46:0/683117350 conn(0x563d9b52e210 msgr2=0x563d9b5cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 -- 172.21.15.46:0/683117350 shutdown_connections 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.404+0000 7f90ede40b80 1 -- 172.21.15.46:0/683117350 wait complete. 2024-05-08T23:05:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.405+0000 7f90ede40b80 1 -- start start 2024-05-08T23:05:44.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.405+0000 7f90ede40b80 1 Processor -- start 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.405+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.405+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5e2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90ede40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b4bc560 con 0x563d9b5de660 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90ede40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b4bc200 con 0x563d9b5def80 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90ede40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b3a36a0 con 0x563d9b5d90f0 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90ede40b80 10 client.0 _ll_register_callbacks cb 0x563d9b4c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42954/0 (socket says 172.21.15.46:42954) 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5e2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48180/0 (socket says 172.21.15.46:48180) 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 -- 172.21.15.46:0/1166769488 learned_addr learned my addr 172.21.15.46:0/1166769488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3e00700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 -- 172.21.15.46:0/1166769488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 msgr2=0x563d9b5e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 -- 172.21.15.46:0/1166769488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 msgr2=0x563d9b5e2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5e2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 -- 172.21.15.46:0/1166769488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c0009d10 con 0x563d9b5d90f0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90d3400700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90c409ae90 tx=0x7f90c409aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854ba22a3bdab269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.406+0000 7f90bd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c409b5f0 con 0x563d9b5d90f0 2024-05-08T23:05:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90c409b890 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d9b479270 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c409b190 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 10 client.26275 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563d9b34db60 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d9b479270 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f90c409a780 con 0x563d9b5d90f0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 10 client.26275 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 10 client.26275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 20 client.26275 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b5f25d0 0x563d9b5f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 4 client.26275 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.407+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d9b59f810 con 0x563d9b5f25d0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.408+0000 7f90d2a00700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b5f25d0 0x563d9b5f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.408+0000 7f90d2a00700 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b5f25d0 0x563d9b5f49c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c16542be386fdbc9 server_cookie=cc1fe205d1e3c99b in_seq=0 out_seq=0 2024-05-08T23:05:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.408+0000 7f90bd600700 10 client.26275 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90bd600700 1 -- 172.21.15.46:0/1166769488 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d9b59f810 con 0x563d9b5f25d0 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90bd600700 10 client.26275 handle_command_reply: tid=0 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 client.26275 shutdown 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 msgr2=0x563d9b5e22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90c409ae90 tx=0x7f90c409aff0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 shutdown_connections 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5d90f0 0x563d9b5e22f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b5f25d0 0x563d9b5f49c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1166769488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5de660 0x563d9b5e2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90bd600700 0 client.26275 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 >> 172.21.15.46:0/1166769488 conn(0x563d9b52e210 msgr2=0x563d9b5ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 shutdown_connections 2024-05-08T23:05:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.409+0000 7f90ede40b80 1 -- 172.21.15.46:0/1166769488 wait complete. 2024-05-08T23:05:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.410+0000 7f90ede40b80 20 client.26275 trim_cache size 0 max 16384 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.423+0000 7f90ede40b80 1 Processor -- start 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.423+0000 7f90ede40b80 1 -- start start 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 0x563d9b5dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b5d9910 con 0x563d9b5d90f0 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b5cb8e0 con 0x563d9b5de660 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90ede40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b5c7dc0 con 0x563d9b5def80 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48188/0 (socket says 172.21.15.46:48188) 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 -- 172.21.15.46:0/326350155 learned_addr learned my addr 172.21.15.46:0/326350155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d2a00700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 0x563d9b5dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3e00700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 -- 172.21.15.46:0/326350155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 msgr2=0x563d9b5cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 -- 172.21.15.46:0/326350155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 msgr2=0x563d9b5dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 0x563d9b5dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d3400700 1 -- 172.21.15.46:0/326350155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9b4e6560 con 0x563d9b5d90f0 2024-05-08T23:05:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.424+0000 7f90d2a00700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 0x563d9b5dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90d3400700 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90c4097c00 tx=0x7f90c40ce210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef72215f9e154bda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90c9600700 1 -- 172.21.15.46:0/326350155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c407f070 con 0x563d9b5d90f0 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90c9600700 1 -- 172.21.15.46:0/326350155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c409b890 con 0x563d9b5d90f0 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90c9600700 1 -- 172.21.15.46:0/326350155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c409a8e0 con 0x563d9b5d90f0 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 -- 172.21.15.46:0/326350155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 msgr2=0x563d9b5eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90c4097c00 tx=0x7f90c40ce210 comp rx=0 tx=0).stop 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 -- 172.21.15.46:0/326350155 shutdown_connections 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5def80 0x563d9b5cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5eeac0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 --2- 172.21.15.46:0/326350155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5de660 0x563d9b5dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.425+0000 7f90ede40b80 1 -- 172.21.15.46:0/326350155 >> 172.21.15.46:0/326350155 conn(0x563d9b52e210 msgr2=0x563d9b5dd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.426+0000 7f90ede40b80 1 -- 172.21.15.46:0/326350155 shutdown_connections 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.426+0000 7f90ede40b80 1 -- 172.21.15.46:0/326350155 wait complete. 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.426+0000 7f90ede40b80 1 -- start start 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.426+0000 7f90ede40b80 1 Processor -- start 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5de660 0x563d9b5e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b5e6590 con 0x563d9b5d90f0 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b5e6700 con 0x563d9b5def80 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9b647a60 con 0x563d9b5de660 2024-05-08T23:05:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90ede40b80 10 client.0 _ll_register_callbacks cb 0x563d9b4c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90d3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48196/0 (socket says 172.21.15.46:48196) 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90d3400700 1 -- 172.21.15.46:0/1086548120 learned_addr learned my addr 172.21.15.46:0/1086548120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90d3e00700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.427+0000 7f90d2a00700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5de660 0x563d9b5e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 -- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5de660 msgr2=0x563d9b5e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5de660 0x563d9b5e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 -- 172.21.15.46:0/1086548120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 msgr2=0x563d9b5e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 -- 172.21.15.46:0/1086548120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d9b34db60 con 0x563d9b5d90f0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90d3400700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90c40098e0 tx=0x7f90c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a65e2d0248659a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90bd600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c407f070 con 0x563d9b5d90f0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f90c40a3580 con 0x563d9b5d90f0 2024-05-08T23:05:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90c00054e0 con 0x563d9b5d90f0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c409b190 con 0x563d9b5d90f0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90ede40b80 10 client.16257 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.428+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563d9b3a5cd0 con 0x563d9b5d90f0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c00054e0 con 0x563d9b5d90f0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f90c40b9180 con 0x563d9b5d90f0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 10 client.16257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 20 client.16257 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b648bd0 0x563d9b64afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 4 client.16257 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d9b518ec0 con 0x563d9b648bd0 2024-05-08T23:05:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.429+0000 7f90d2a00700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b648bd0 0x563d9b64afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90d2a00700 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b648bd0 0x563d9b64afc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60e0b83d7f3e65e5 server_cookie=82ffd2124674411e in_seq=0 out_seq=0 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90bd600700 10 client.16257 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90bd600700 1 -- 172.21.15.46:0/1086548120 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d9b518ec0 con 0x563d9b648bd0 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90bd600700 10 client.16257 handle_command_reply: tid=0 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 client.16257 shutdown 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 msgr2=0x563d9b5e9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90c40098e0 tx=0x7f90c4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 shutdown_connections 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d9b5def80 0x563d9b5e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563d9b648bd0 0x563d9b64afc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d9b5d90f0 0x563d9b5e9c40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 --2- 172.21.15.46:0/1086548120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9b5de660 0x563d9b5e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 >> 172.21.15.46:0/1086548120 conn(0x563d9b52e210 msgr2=0x563d9b5e2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90bd600700 0 client.16257 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 shutdown_connections 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.430+0000 7f90ede40b80 1 -- 172.21.15.46:0/1086548120 wait complete. 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90ede40b80 20 client.16257 trim_cache size 0 max 16384 2024-05-08T23:05:44.435 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 msgr2=0x7f90b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 0x7f90b80a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00f590 tx=0x7f90cc00b040 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 msgr2=0x7f90d4199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90c4000c00 tx=0x7f90c4005740 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 shutdown_connections 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d419de30 0x7f90d419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f90b80a1310 0x7f90b80a37d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d4101080 0x7f90d4199550 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 --2- 172.21.15.46:0/722655207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90d41061a0 0x7f90d4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 >> 172.21.15.46:0/722655207 conn(0x7f90d40fc8e0 msgr2=0x7f90d40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 shutdown_connections 2024-05-08T23:05:44.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.431+0000 7f90da400700 1 -- 172.21.15.46:0/722655207 wait complete. 2024-05-08T23:05:44.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:05:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 -- 172.21.15.46:0/2787722131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 msgr2=0x7f5ccc10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 --2- 172.21.15.46:0/2787722131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc10d110 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 -- 172.21.15.46:0/2787722131 shutdown_connections 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 --2- 172.21.15.46:0/2787722131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc10d650 0x7f5ccc10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 --2- 172.21.15.46:0/2787722131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 --2- 172.21.15.46:0/2787722131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc10d110 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 -- 172.21.15.46:0/2787722131 >> 172.21.15.46:0/2787722131 conn(0x7f5ccc0fc8e0 msgr2=0x7f5ccc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 -- 172.21.15.46:0/2787722131 shutdown_connections 2024-05-08T23:05:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.652+0000 7f5cd3200700 1 -- 172.21.15.46:0/2787722131 wait complete. 2024-05-08T23:05:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.653+0000 7f5cd3200700 1 Processor -- start 2024-05-08T23:05:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.653+0000 7f5cd3200700 1 -- start start 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.653+0000 7f5cd3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.653+0000 7f5cd3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.653+0000 7f5cd3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc10d650 0x7f5ccc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5cd3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19e440 con 0x7f5ccc073cf0 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5cd3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19e5b0 con 0x7f5ccc10d650 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5cd3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc19e770 con 0x7f5ccc0733d0 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48220/0 (socket says 172.21.15.46:48220) 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 -- 172.21.15.46:0/1396722926 learned_addr learned my addr 172.21.15.46:0/1396722926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5cd1200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc10d650 0x7f5ccc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 -- 172.21.15.46:0/1396722926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 msgr2=0x7f5ccc199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 -- 172.21.15.46:0/1396722926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc10d650 msgr2=0x7f5ccc19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc10d650 0x7f5ccc19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 -- 172.21.15.46:0/1396722926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80095f0 con 0x7f5ccc073cf0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.654+0000 7f5ccb400700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.655+0000 7f5cc8c00700 1 -- 172.21.15.46:0/1396722926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801d070 con 0x7f5ccc073cf0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.655+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc19e9f0 con 0x7f5ccc073cf0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.655+0000 7f5cc8c00700 1 -- 172.21.15.46:0/1396722926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb8022470 con 0x7f5ccc073cf0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.655+0000 7f5cc8c00700 1 -- 172.21.15.46:0/1396722926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8017650 con 0x7f5ccc073cf0 2024-05-08T23:05:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.655+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc1a6290 con 0x7f5ccc073cf0 2024-05-08T23:05:44.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.658+0000 7f5cc8c00700 1 -- 172.21.15.46:0/1396722926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5cb80225e0 con 0x7f5ccc073cf0 2024-05-08T23:05:44.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.659+0000 7f5ce6cafb80 1 Processor -- start 2024-05-08T23:05:44.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 -- start start 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d0623d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5cc8c00700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0618e150 con 0x555d0623d720 2024-05-08T23:05:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0618bb10 con 0x555d0623ce00 2024-05-08T23:05:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06187400 con 0x555d06237890 2024-05-08T23:05:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5cc8c00700 1 -- 172.21.15.46:0/1396722926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5cb80ce390 con 0x7f5ccc073cf0 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ccb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5cd1200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ccb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48228/0 (socket says 172.21.15.46:48228) 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ccb400700 1 -- 172.21.15.46:0/3874849413 learned_addr learned my addr 172.21.15.46:0/3874849413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5ccbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d0623d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.660+0000 7f5cd1200700 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 -- 172.21.15.46:0/3874849413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 msgr2=0x555d06237cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 -- 172.21.15.46:0/3874849413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 msgr2=0x555d0623d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d0623d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 -- 172.21.15.46:0/3874849413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d060083b0 con 0x555d0623d720 2024-05-08T23:05:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d0623d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5ccb400700 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80a2fe0 tx=0x7f5cb8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42493ad62247ce73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5cd1200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 0x7f5cb00a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003fd0 tx=0x7f5cc4003a00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.661+0000 7f5cc1600700 1 -- 172.21.15.46:0/3874849413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb809b820 con 0x555d0623d720 2024-05-08T23:05:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5cc1600700 1 -- 172.21.15.46:0/3874849413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb809b120 con 0x555d0623d720 2024-05-08T23:05:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5cc1600700 1 -- 172.21.15.46:0/3874849413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8099590 con 0x555d0623d720 2024-05-08T23:05:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/3874849413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 msgr2=0x555d06237350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80a2fe0 tx=0x7f5cb8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/3874849413 shutdown_connections 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d0623d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/3874849413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06237350 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/3874849413 >> 172.21.15.46:0/3874849413 conn(0x555d06195210 msgr2=0x555d06231df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.662+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/3874849413 shutdown_connections 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.663+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/3874849413 wait complete. 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.663+0000 7f5ce6cafb80 1 -- start start 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.663+0000 7f5ce6cafb80 1 Processor -- start 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06243720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06240ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06123560 con 0x555d0623d720 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06123200 con 0x555d0623ce00 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0600a6a0 con 0x555d06237890 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ce6cafb80 10 client.0 _ll_register_callbacks cb 0x555d0612cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48236/0 (socket says 172.21.15.46:48236) 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 -- 172.21.15.46:0/214397278 learned_addr learned my addr 172.21.15.46:0/214397278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06243720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5ccb400700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06240ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 -- 172.21.15.46:0/214397278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 msgr2=0x555d06243720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06243720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 -- 172.21.15.46:0/214397278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 msgr2=0x555d06240ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06240ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.664+0000 7f5cd1200700 1 -- 172.21.15.46:0/214397278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc0098f0 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cd1200700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4008740 tx=0x7f5cc4009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65033dfddbdb903 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cb6000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4019810 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5cc4019970 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d060e0270 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc4008ee0 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5ce6cafb80 10 client.16269 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5cb809fe70 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.665+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d060e0270 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5cc4024c50 con 0x555d0623d720 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 20 client.16269 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d06250f30 0x555d06253350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 4 client.16269 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d06206810 con 0x555d06250f30 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d06250f30 0x555d06253350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5cb6000700 10 client.16269 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.666+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d06250f30 0x555d06253350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1994e6eb17355cd2 server_cookie=cc9c4a621f4108b2 in_seq=0 out_seq=0 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5cb6000700 1 -- 172.21.15.46:0/214397278 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d06206810 con 0x555d06250f30 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5cb6000700 10 client.16269 handle_command_reply: tid=0 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 client.16269 shutdown 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 msgr2=0x555d06241210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4008740 tx=0x7f5cc4009130 comp rx=0 tx=0).stop 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 shutdown_connections 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d06250f30 0x555d06253350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623d720 0x555d06241210 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06243720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/214397278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06240ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5cb6000700 0 client.16269 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.667+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 >> 172.21.15.46:0/214397278 conn(0x555d06195210 msgr2=0x555d062315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.668+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 shutdown_connections 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.668+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/214397278 wait complete. 2024-05-08T23:05:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.668+0000 7f5ce6cafb80 20 client.16269 trim_cache size 0 max 16384 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.681+0000 7f5ce6cafb80 1 Processor -- start 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 -- start start 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 0x555d06231d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 0x555d06247ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0622ee00 con 0x555d0623ce00 2024-05-08T23:05:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06233e50 con 0x555d0623d720 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0622c830 con 0x555d06237890 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ccb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 0x555d06231d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ccbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5cd1200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 0x555d06247ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ccbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43022/0 (socket says 172.21.15.46:43022) 2024-05-08T23:05:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.682+0000 7f5ccbe00700 1 -- 172.21.15.46:0/4266724186 learned_addr learned my addr 172.21.15.46:0/4266724186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 -- 172.21.15.46:0/4266724186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 msgr2=0x555d06247ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 0x555d06247ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 -- 172.21.15.46:0/4266724186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 msgr2=0x555d06231d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 0x555d06231d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 -- 172.21.15.46:0/4266724186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d05fb4b60 con 0x555d06237890 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5cd1200700 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 0x555d06247ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccb400700 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 0x555d06231d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x555d06253c00 tx=0x7f5cbc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ac59220b71840e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5cc2000700 1 -- 172.21.15.46:0/4266724186 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc017070 con 0x555d06237890 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5cc2000700 1 -- 172.21.15.46:0/4266724186 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc018040 con 0x555d06237890 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.683+0000 7f5cc2000700 1 -- 172.21.15.46:0/4266724186 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0128b0 con 0x555d06237890 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/4266724186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 msgr2=0x555d06237350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x555d06253c00 tx=0x7f5cbc009260 comp rx=0 tx=0).stop 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/4266724186 shutdown_connections 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623d720 0x555d06247ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d06237890 0x555d06237350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/4266724186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d0623ce00 0x555d06231d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/4266724186 >> 172.21.15.46:0/4266724186 conn(0x555d06195210 msgr2=0x555d0623abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/4266724186 shutdown_connections 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/4266724186 wait complete. 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.684+0000 7f5ce6cafb80 1 -- start start 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 Processor -- start 2024-05-08T23:05:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06246500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d0623d720 0x555d06246a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06246f80 con 0x555d06237890 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d062470c0 con 0x555d0623ce00 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d06247280 con 0x555d0623d720 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ce6cafb80 10 client.0 _ll_register_callbacks cb 0x555d0612cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ccbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ccbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48256/0 (socket says 172.21.15.46:48256) 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ccbe00700 1 -- 172.21.15.46:0/552868799 learned_addr learned my addr 172.21.15.46:0/552868799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5ccb400700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06246500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.685+0000 7f5cd1200700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d0623d720 0x555d06246a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 -- 172.21.15.46:0/552868799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d0623d720 msgr2=0x555d06246a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d0623d720 0x555d06246a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 -- 172.21.15.46:0/552868799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 msgr2=0x555d06246500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06246500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 -- 172.21.15.46:0/552868799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d060083b0 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ccbe00700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000c00 tx=0x7f5cbc003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08b182eb59891d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5cb6000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc017070 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cb809b120 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.686+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5cbc01e840 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cbc005040 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 10 client.16275 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5cb809fe70 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb809b120 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5cbc004d20 con 0x555d06237890 2024-05-08T23:05:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 10 client.16275 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 10 client.16275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 20 client.16275 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d062a88e0 0x555d062aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 4 client.16275 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d0617fec0 con 0x555d062a88e0 2024-05-08T23:05:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.687+0000 7f5ccb400700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d062a88e0 0x555d062aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ccb400700 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d062a88e0 0x555d062aad00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca0d29bda42d6905 server_cookie=1cb418dbf4c822b0 in_seq=0 out_seq=0 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5cb6000700 10 client.16275 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5cb6000700 1 -- 172.21.15.46:0/552868799 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d0617fec0 con 0x555d062a88e0 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5cb6000700 10 client.16275 handle_command_reply: tid=0 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 client.16275 shutdown 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 msgr2=0x555d06248060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000c00 tx=0x7f5cbc003c40 comp rx=0 tx=0).stop 2024-05-08T23:05:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 shutdown_connections 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d0623d720 0x555d06246a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x555d062a88e0 0x555d062aad00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d06237890 0x555d06248060 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 --2- 172.21.15.46:0/552868799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555d0623ce00 0x555d06246500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5cb6000700 0 client.16275 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 >> 172.21.15.46:0/552868799 conn(0x555d06195210 msgr2=0x555d0623f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 shutdown_connections 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.688+0000 7f5ce6cafb80 1 -- 172.21.15.46:0/552868799 wait complete. 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5ce6cafb80 20 client.16275 trim_cache size 0 max 16384 2024-05-08T23:05:44.693 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 msgr2=0x7f5cb00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 0x7f5cb00a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003fd0 tx=0x7f5cc4003a00 comp rx=0 tx=0).stop 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 msgr2=0x7f5ccc199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb8005740 comp rx=0 tx=0).stop 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 shutdown_connections 2024-05-08T23:05:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ccc10d650 0x7f5ccc19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ccc0733d0 0x7f5ccc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 --2- 172.21.15.46:0/1396722926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc073cf0 0x7f5ccc199a60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 >> 172.21.15.46:0/1396722926 conn(0x7f5ccc0fc8e0 msgr2=0x7f5ccc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 shutdown_connections 2024-05-08T23:05:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.689+0000 7f5cd3200700 1 -- 172.21.15.46:0/1396722926 wait complete. 2024-05-08T23:05:44.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:05:44.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6317 8.51 MB/sec warmup 6 sec latency 3605.679 ms 2024-05-08T23:05:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 -- 172.21.15.46:0/538492668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 msgr2=0x7f1d58073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 --2- 172.21.15.46:0/538492668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d58073750 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009990 tx=0x7f1d48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 -- 172.21.15.46:0/538492668 shutdown_connections 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 --2- 172.21.15.46:0/538492668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5810d6d0 0x7f1d5810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 --2- 172.21.15.46:0/538492668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d58073750 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 --2- 172.21.15.46:0/538492668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d58073c90 0x7f1d5810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 -- 172.21.15.46:0/538492668 >> 172.21.15.46:0/538492668 conn(0x7f1d580fc960 msgr2=0x7f1d580fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 -- 172.21.15.46:0/538492668 shutdown_connections 2024-05-08T23:05:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.910+0000 7f1d5d200700 1 -- 172.21.15.46:0/538492668 wait complete. 2024-05-08T23:05:44.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 Processor -- start 2024-05-08T23:05:44.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 -- start start 2024-05-08T23:05:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d5810d6d0 0x7f1d5819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5819e4c0 con 0x7f1d58073c90 2024-05-08T23:05:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5819e630 con 0x7f1d5810d6d0 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.912+0000 7f1d5d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5819e7f0 con 0x7f1d58073370 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43048/0 (socket says 172.21.15.46:43048) 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48274/0 (socket says 172.21.15.46:48274) 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 -- 172.21.15.46:0/221953520 learned_addr learned my addr 172.21.15.46:0/221953520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d56800700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d5810d6d0 0x7f1d5819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 -- 172.21.15.46:0/221953520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d5810d6d0 msgr2=0x7f1d5819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d5810d6d0 0x7f1d5819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 -- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 msgr2=0x7f1d58199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 -- 172.21.15.46:0/221953520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d50009c40 con 0x7f1d58073370 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55400700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.913+0000 7f1d55e00700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d48005910 tx=0x7f1d48004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.914+0000 7f1d4ea00700 1 -- 172.21.15.46:0/221953520 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4801d070 con 0x7f1d58073370 2024-05-08T23:05:44.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.914+0000 7f1d4ea00700 1 -- 172.21.15.46:0/221953520 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d48022470 con 0x7f1d58073370 2024-05-08T23:05:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.914+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d480095f0 con 0x7f1d58073370 2024-05-08T23:05:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.914+0000 7f1d4ea00700 1 -- 172.21.15.46:0/221953520 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48017650 con 0x7f1d58073370 2024-05-08T23:05:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.914+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d581a6130 con 0x7f1d58073370 2024-05-08T23:05:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.917+0000 7f1d4ea00700 1 -- 172.21.15.46:0/221953520 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1d480225e0 con 0x7f1d58073370 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.918+0000 7f1d70c49b80 1 Processor -- start 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.918+0000 7f1d4ea00700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 0x7f1d3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.918+0000 7f1d4ea00700 1 -- 172.21.15.46:0/221953520 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1d480ce200 con 0x7f1d58073370 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.918+0000 7f1d70c49b80 1 -- start start 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.918+0000 7f1d56800700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 0x7f1d3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 0x556aced80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced80cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acecd7150 con 0x556aced86720 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acecd4b10 con 0x556aced85e00 2024-05-08T23:05:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acecd0400 con 0x556aced80890 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 0x556aced80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 0x556aced80350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48282/0 (socket says 172.21.15.46:48282) 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55400700 1 -- 172.21.15.46:0/885418899 learned_addr learned my addr 172.21.15.46:0/885418899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d56800700 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced80cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d56800700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 0x7f1d3c0a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009fd0 tx=0x7f1d5000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 -- 172.21.15.46:0/885418899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 msgr2=0x556aced80cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced80cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 -- 172.21.15.46:0/885418899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 msgr2=0x556aced80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 0x556aced80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.919+0000 7f1d55e00700 1 -- 172.21.15.46:0/885418899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aceb513b0 con 0x556aced85e00 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d55e00700 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced861e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d480cb540 tx=0x7f1d4809ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724d4dc619268ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d43400700 1 -- 172.21.15.46:0/885418899 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480ce020 con 0x556aced85e00 2024-05-08T23:05:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d43400700 1 -- 172.21.15.46:0/885418899 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4809b690 con 0x556aced85e00 2024-05-08T23:05:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d43400700 1 -- 172.21.15.46:0/885418899 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4809b1f0 con 0x556aced85e00 2024-05-08T23:05:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d70c49b80 1 -- 172.21.15.46:0/885418899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 msgr2=0x556aced861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.920+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced861e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d480cb540 tx=0x7f1d4809ad60 comp rx=0 tx=0).stop 2024-05-08T23:05:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 -- 172.21.15.46:0/885418899 shutdown_connections 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced80cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced861e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/885418899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced86720 0x556aced80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 -- 172.21.15.46:0/885418899 >> 172.21.15.46:0/885418899 conn(0x556acecde210 msgr2=0x556aced7adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 -- 172.21.15.46:0/885418899 shutdown_connections 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 -- 172.21.15.46:0/885418899 wait complete. 2024-05-08T23:05:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.921+0000 7f1d70c49b80 1 -- start start 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 Processor -- start 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced89b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acec6c560 con 0x556aced85e00 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acec6c200 con 0x556aced86720 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.922+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aceb536a0 con 0x556aced80890 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d70c49b80 10 client.0 _ll_register_callbacks cb 0x556acec75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48290/0 (socket says 172.21.15.46:48290) 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced89b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55400700 1 -- 172.21.15.46:0/3103290207 learned_addr learned my addr 172.21.15.46:0/3103290207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d56800700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 -- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 msgr2=0x556aced8a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 -- 172.21.15.46:0/3103290207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 msgr2=0x556aced8a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 -- 172.21.15.46:0/3103290207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d581a6130 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55400700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d55e00700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced89b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d480992e0 tx=0x7f1d48022cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6230e8bd6ac4bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.923+0000 7f1d37e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480ce040 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1d4809af90 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556acec29270 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d480a10e0 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 10 client.26311 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556aceafdb60 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556acec29270 con 0x556aced80890 2024-05-08T23:05:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d48095ba0 con 0x556aced80890 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 10 client.26311 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 10 client.26311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 20 client.26311 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aced99b00 0x556aced9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 4 client.26311 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.924+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aced4f810 con 0x556aced99b00 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.925+0000 7f1d55400700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aced99b00 0x556aced9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.925+0000 7f1d55400700 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aced99b00 0x556aced9bf20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d055eb383048ffd server_cookie=859792752dc48965 in_seq=0 out_seq=0 2024-05-08T23:05:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.925+0000 7f1d37e00700 10 client.26311 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d37e00700 1 -- 172.21.15.46:0/3103290207 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556aced4f810 con 0x556aced99b00 2024-05-08T23:05:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d37e00700 10 client.26311 handle_command_reply: tid=0 2024-05-08T23:05:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 client.26311 shutdown 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 msgr2=0x556aced89b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced89b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d480992e0 tx=0x7f1d48022cb0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 shutdown_connections 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced89b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aced99b00 0x556aced9bf20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.926+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/3103290207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.927+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 >> 172.21.15.46:0/3103290207 conn(0x556acecde210 msgr2=0x556aced7a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.927+0000 7f1d37e00700 0 client.26311 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.927+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 shutdown_connections 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.927+0000 7f1d70c49b80 1 -- 172.21.15.46:0/3103290207 wait complete. 2024-05-08T23:05:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.927+0000 7f1d70c49b80 20 client.26311 trim_cache size 0 max 16384 2024-05-08T23:05:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 Processor -- start 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 -- start start 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced86720 0x556aced8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aced77dc0 con 0x556aced80890 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aced85830 con 0x556aced85e00 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.941+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aceda10f0 con 0x556aced86720 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48300/0 (socket says 172.21.15.46:48300) 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 -- 172.21.15.46:0/839336534 learned_addr learned my addr 172.21.15.46:0/839336534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d56800700 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced86720 0x556aced8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55400700 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 -- 172.21.15.46:0/839336534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced86720 msgr2=0x556aced8a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced86720 0x556aced8a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 -- 172.21.15.46:0/839336534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 msgr2=0x556aced7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 -- 172.21.15.46:0/839336534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556acec96560 con 0x556aced80890 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.942+0000 7f1d55e00700 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000c00 tx=0x7f1d4800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4052d080f8a34f38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d43e00700 1 -- 172.21.15.46:0/839336534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480ce040 con 0x556aced80890 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d43e00700 1 -- 172.21.15.46:0/839336534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4809ad60 con 0x556aced80890 2024-05-08T23:05:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d43e00700 1 -- 172.21.15.46:0/839336534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480a1c70 con 0x556aced80890 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 -- 172.21.15.46:0/839336534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 msgr2=0x556aced852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000c00 tx=0x7f1d4800b490 comp rx=0 tx=0).stop 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 -- 172.21.15.46:0/839336534 shutdown_connections 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced86720 0x556aced8a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced80890 0x556aced852f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/839336534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced85e00 0x556aced7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 -- 172.21.15.46:0/839336534 >> 172.21.15.46:0/839336534 conn(0x556acecde210 msgr2=0x556aced84be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 -- 172.21.15.46:0/839336534 shutdown_connections 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.943+0000 7f1d70c49b80 1 -- 172.21.15.46:0/839336534 wait complete. 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.944+0000 7f1d70c49b80 1 -- start start 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.944+0000 7f1d70c49b80 1 Processor -- start 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced91070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aced8f950 con 0x556aced85e00 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aced8fac0 con 0x556aced86720 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aced8fc80 con 0x556aced80890 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d70c49b80 10 client.0 _ll_register_callbacks cb 0x556acec75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48304/0 (socket says 172.21.15.46:48304) 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 -- 172.21.15.46:0/816609168 learned_addr learned my addr 172.21.15.46:0/816609168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55e00700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced91070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d56800700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 -- 172.21.15.46:0/816609168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 msgr2=0x556aced91070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced91070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 -- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 msgr2=0x556aced8f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 -- 172.21.15.46:0/816609168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aceafdb60 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d56800700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d55400700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556aced9c7c0 tx=0x7f1d44012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0ea6c0a7926a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.945+0000 7f1d37e00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44017040 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1d44004230 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d4809b690 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d440043f0 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d70c49b80 10 client.16293 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556aceb55cd0 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4809b690 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.946+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d44003b40 con 0x556aced85e00 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 20 client.16293 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556acedf0640 0x556acedf2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 4 client.16293 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556acecc8ec0 con 0x556acedf0640 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.947+0000 7f1d55e00700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556acedf0640 0x556acedf2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d55e00700 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556acedf0640 0x556acedf2a60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6696603e9ca245 server_cookie=9cf8ff144a83a5fc in_seq=0 out_seq=0 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d37e00700 10 client.16293 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d37e00700 1 -- 172.21.15.46:0/816609168 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556acecc8ec0 con 0x556acedf0640 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d37e00700 10 client.16293 handle_command_reply: tid=0 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 client.16293 shutdown 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 msgr2=0x556aced8ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556aced9c7c0 tx=0x7f1d44012f20 comp rx=0 tx=0).stop 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 shutdown_connections 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aced86720 0x556aced8f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aced80890 0x556aced91070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556acedf0640 0x556acedf2a60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 --2- 172.21.15.46:0/816609168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aced85e00 0x556aced8ee60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 >> 172.21.15.46:0/816609168 conn(0x556acecde210 msgr2=0x556aced86b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d37e00700 0 client.16293 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 shutdown_connections 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.948+0000 7f1d70c49b80 1 -- 172.21.15.46:0/816609168 wait complete. 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d70c49b80 20 client.16293 trim_cache size 0 max 16384 2024-05-08T23:05:44.953 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 msgr2=0x7f1d3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 0x7f1d3c0a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009fd0 tx=0x7f1d5000b040 comp rx=0 tx=0).stop 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 msgr2=0x7f1d581995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d48005910 tx=0x7f1d48004730 comp rx=0 tx=0).stop 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 shutdown_connections 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d5810d6d0 0x7f1d5819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58073370 0x7f1d581995f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1d3c0a1290 0x7f1d3c0a3750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 --2- 172.21.15.46:0/221953520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58073c90 0x7f1d58199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 >> 172.21.15.46:0/221953520 conn(0x7f1d580fc960 msgr2=0x7f1d580fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 shutdown_connections 2024-05-08T23:05:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:44.949+0000 7f1d5d200700 1 -- 172.21.15.46:0/221953520 wait complete. 2024-05-08T23:05:45.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6457 7.42 MB/sec warmup 7 sec latency 4085.801 ms 2024-05-08T23:05:45.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:45 smithi086 ceph-mon[78610]: pgmap v61: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 351 KiB/s wr, 52 op/s 2024-05-08T23:05:45.969 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:05:45.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:05:46.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:45 smithi053 ceph-mon[77587]: pgmap v61: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 351 KiB/s wr, 52 op/s 2024-05-08T23:05:46.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:45 smithi046 ceph-mon[36898]: pgmap v61: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 351 KiB/s wr, 52 op/s 2024-05-08T23:05:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2102827333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b84073cf0 msgr2=0x7f0b8410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2102827333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b84073cf0 0x7f0b8410d110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009990 tx=0x7f0b6c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2102827333 shutdown_connections 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2102827333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8410d650 0x7f0b8410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2102827333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b840737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2102827333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b84073cf0 0x7f0b8410d110 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2102827333 >> 172.21.15.46:0/2102827333 conn(0x7f0b840fc8e0 msgr2=0x7f0b840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2102827333 shutdown_connections 2024-05-08T23:05:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.184+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2102827333 wait complete. 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.185+0000 7f0b8ac00700 1 Processor -- start 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.185+0000 7f0b8ac00700 1 -- start start 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 0x7f0b84199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8410d650 0x7f0b8419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8419e420 con 0x7f0b84073cf0 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8419e590 con 0x7f0b840733d0 2024-05-08T23:05:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b8ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8419e750 con 0x7f0b8410d650 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b7be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 0x7f0b84199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46502/0 (socket says 172.21.15.46:46502) 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 -- 172.21.15.46:0/2636764100 learned_addr learned my addr 172.21.15.46:0/2636764100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b88c00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8410d650 0x7f0b8419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 -- 172.21.15.46:0/2636764100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8410d650 msgr2=0x7f0b8419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8410d650 0x7f0b8419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 -- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 msgr2=0x7f0b84199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 0x7f0b84199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b83e00700 1 -- 172.21.15.46:0/2636764100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0095f0 con 0x7f0b840733d0 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.186+0000 7f0b7be00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 0x7f0b84199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b83e00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400e9c0 tx=0x7f0b7400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b81600700 1 -- 172.21.15.46:0/2636764100 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b740097b0 con 0x7f0b840733d0 2024-05-08T23:05:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8419ea30 con 0x7f0b840733d0 2024-05-08T23:05:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b841a6270 con 0x7f0b840733d0 2024-05-08T23:05:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b81600700 1 -- 172.21.15.46:0/2636764100 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7400ee60 con 0x7f0b840733d0 2024-05-08T23:05:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.187+0000 7f0b81600700 1 -- 172.21.15.46:0/2636764100 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74005480 con 0x7f0b840733d0 2024-05-08T23:05:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.192+0000 7f0b81600700 1 -- 172.21.15.46:0/2636764100 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0b7400cbd0 con 0x7f0b840733d0 2024-05-08T23:05:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 Processor -- start 2024-05-08T23:05:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 -- start start 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f90910 0x55f884f90d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884ee71d0 con 0x55f884f95e80 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884ee4b90 con 0x55f884f967a0 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.193+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884ee0480 con 0x55f884f90910 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b88c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b81600700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 0x7f0b680a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46518/0 (socket says 172.21.15.46:46518) 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b88c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f96260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48338/0 (socket says 172.21.15.46:48338) 2024-05-08T23:05:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 -- 172.21.15.46:0/742968759 learned_addr learned my addr 172.21.15.46:0/742968759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b7be00700 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f90910 0x55f884f90d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b81600700 1 -- 172.21.15.46:0/2636764100 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0b74014070 con 0x7f0b840733d0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b88c00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 0x7f0b680a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 -- 172.21.15.46:0/742968759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f90910 msgr2=0x55f884f90d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f90910 0x55f884f90d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 -- 172.21.15.46:0/742968759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 msgr2=0x55f884f96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 -- 172.21.15.46:0/742968759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f884d61430 con 0x55f884f967a0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.194+0000 7f0b83e00700 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b740ca520 tx=0x7f0b7400efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbbe6040b07676 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b79200700 1 -- 172.21.15.46:0/742968759 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74099f70 con 0x55f884f967a0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b88c00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 0x7f0b680a37d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ec0 tx=0x7f0b7c004b40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b79200700 1 -- 172.21.15.46:0/742968759 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7409dda0 con 0x55f884f967a0 2024-05-08T23:05:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b79200700 1 -- 172.21.15.46:0/742968759 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74016590 con 0x55f884f967a0 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/742968759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 msgr2=0x55f884f903d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b740ca520 tx=0x7f0b7400efd0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/742968759 shutdown_connections 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f90910 0x55f884f90d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/742968759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f967a0 0x55f884f903d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/742968759 >> 172.21.15.46:0/742968759 conn(0x55f884eee290 msgr2=0x55f884f8ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.195+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/742968759 shutdown_connections 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.196+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/742968759 wait complete. 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.196+0000 7f0b9e69bb80 1 -- start start 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.196+0000 7f0b9e69bb80 1 Processor -- start 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f99ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 0x55f884f9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884e7c5e0 con 0x55f884f95e80 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884e7c280 con 0x55f884f90910 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884d63720 con 0x55f884f967a0 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b9e69bb80 10 client.0 _ll_register_callbacks cb 0x55f884e85bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b7be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b83e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f99ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b7be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48354/0 (socket says 172.21.15.46:48354) 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b7be00700 1 -- 172.21.15.46:0/4228364747 learned_addr learned my addr 172.21.15.46:0/4228364747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.197+0000 7f0b88c00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 0x55f884f9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 -- 172.21.15.46:0/4228364747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 msgr2=0x55f884f9a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 0x55f884f9a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 -- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 msgr2=0x55f884f99ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f99ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 -- 172.21.15.46:0/4228364747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b841a6270 con 0x55f884f95e80 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b83e00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f99ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b88c00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 0x55f884f9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b7be00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c015000 tx=0x7f0b6c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f451730fcd703f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b71600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c01d070 con 0x55f884f95e80 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f884e392f0 con 0x55f884f95e80 2024-05-08T23:05:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0b6c004910 con 0x55f884f95e80 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.198+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b6c004ad0 con 0x55f884f95e80 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 10 client.16305 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f884d0dbe0 con 0x55f884f95e80 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f884e392f0 con 0x55f884f95e80 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0b6c00fb80 con 0x55f884f95e80 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 20 client.16305 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f884faa310 0x55f884fac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 4 client.16305 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f884f5f890 con 0x55f884faa310 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.199+0000 7f0b83e00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f884faa310 0x55f884fac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.200+0000 7f0b83e00700 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f884faa310 0x55f884fac730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbefb5037cedeb7a server_cookie=2b385a9ed4aa3e98 in_seq=0 out_seq=0 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.200+0000 7f0b71600700 10 client.16305 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b71600700 1 -- 172.21.15.46:0/4228364747 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f884f5f890 con 0x55f884faa310 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b71600700 10 client.16305 handle_command_reply: tid=0 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 client.16305 shutdown 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 msgr2=0x55f884f9a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c015000 tx=0x7f0b6c005270 comp rx=0 tx=0).stop 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 shutdown_connections 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f967a0 0x55f884f9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f99ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f884faa310 0x55f884fac730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/4228364747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f95e80 0x55f884f9a020 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 >> 172.21.15.46:0/4228364747 conn(0x55f884eee290 msgr2=0x55f884f8a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b71600700 0 client.16305 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 shutdown_connections 2024-05-08T23:05:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/4228364747 wait complete. 2024-05-08T23:05:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.201+0000 7f0b9e69bb80 20 client.16305 trim_cache size 0 max 16384 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.219+0000 7f0b9e69bb80 1 Processor -- start 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.219+0000 7f0b9e69bb80 1 -- start start 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f98a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f95440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884f9b180 con 0x55f884f967a0 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884f87e70 con 0x55f884f90910 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884fb19e0 con 0x55f884f95e80 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48362/0 (socket says 172.21.15.46:48362) 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 -- 172.21.15.46:0/1606831433 learned_addr learned my addr 172.21.15.46:0/1606831433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f98a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b7be00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f95440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 -- 172.21.15.46:0/1606831433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 msgr2=0x55f884f95440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f95440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 -- 172.21.15.46:0/1606831433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 msgr2=0x55f884f98a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f98a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b88c00700 1 -- 172.21.15.46:0/1606831433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f884f90fc0 con 0x55f884f967a0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.220+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f98a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b88c00700 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0189b0 tx=0x7f0b7c009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660ad4a309f29108 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b79c00700 1 -- 172.21.15.46:0/1606831433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02a070 con 0x55f884f967a0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b79c00700 1 -- 172.21.15.46:0/1606831433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7c022040 con 0x55f884f967a0 2024-05-08T23:05:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b79c00700 1 -- 172.21.15.46:0/1606831433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c021050 con 0x55f884f967a0 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1606831433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 msgr2=0x55f884f8ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.221+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0189b0 tx=0x7f0b7c009070 comp rx=0 tx=0).stop 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1606831433 shutdown_connections 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f884f8ada0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f98a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1606831433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f95440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1606831433 >> 172.21.15.46:0/1606831433 conn(0x55f884eee290 msgr2=0x55f884f96c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1606831433 shutdown_connections 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1606831433 wait complete. 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 -- start start 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.222+0000 7f0b9e69bb80 1 Processor -- start 2024-05-08T23:05:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f885000830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884f93550 con 0x55f884f967a0 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f885000d70 con 0x55f884f90910 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f885000eb0 con 0x55f884f95e80 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b9e69bb80 10 client.0 _ll_register_callbacks cb 0x55f884e85bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b88c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f885000830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43140/0 (socket says 172.21.15.46:43140) 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 -- 172.21.15.46:0/1813816334 learned_addr learned my addr 172.21.15.46:0/1813816334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 -- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 msgr2=0x55f884f926d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f926d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 -- 172.21.15.46:0/1813816334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 msgr2=0x55f885000830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f885000830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b7be00700 1 -- 172.21.15.46:0/1813816334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c0186f0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.223+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b7be00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b670 tx=0x7f0b6c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d1774836979cec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b71600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c01d070 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0b6c0244a0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b7409dda0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b6c004710 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b9e69bb80 10 client.26335 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.224+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f884ea65e0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7409dda0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0b6c0099c0 con 0x55f884f95e80 2024-05-08T23:05:46.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 10 client.26335 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 20 client.26335 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f885002240 0x55f885004630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 4 client.26335 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f884ed8f40 con 0x55f885002240 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.225+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f885002240 0x55f885004630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.226+0000 7f0b83e00700 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f885002240 0x55f885004630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22f739aa9e22db6 server_cookie=ab0d1ee9289ee6ff in_seq=0 out_seq=0 2024-05-08T23:05:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.226+0000 7f0b71600700 10 client.26335 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b71600700 1 -- 172.21.15.46:0/1813816334 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f884ed8f40 con 0x55f885002240 2024-05-08T23:05:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b71600700 10 client.26335 handle_command_reply: tid=0 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 client.26335 shutdown 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 msgr2=0x55f884f92c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b670 tx=0x7f0b6c005e80 comp rx=0 tx=0).stop 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 shutdown_connections 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f885002240 0x55f885004630 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f884f967a0 0x55f885000830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f884f90910 0x55f884f926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 --2- 172.21.15.46:0/1813816334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f884f95e80 0x55f884f92c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b71600700 0 client.26335 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 >> 172.21.15.46:0/1813816334 conn(0x55f884eee290 msgr2=0x55f884f974c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 shutdown_connections 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 1 -- 172.21.15.46:0/1813816334 wait complete. 2024-05-08T23:05:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.227+0000 7f0b9e69bb80 20 client.26335 trim_cache size 0 max 16384 2024-05-08T23:05:46.232 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:05:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "344271c0-56dd-40f9-b349-bba5a6658182", 2024-05-08T23:05:46.232 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:05:46.232 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 msgr2=0x7f0b680a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 0x7f0b680a37d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ec0 tx=0x7f0b7c004b40 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 msgr2=0x7f0b84199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400e9c0 tx=0x7f0b7400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 shutdown_connections 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8410d650 0x7f0b8419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b840733d0 0x7f0b84199550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b680a1310 0x7f0b680a37d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 --2- 172.21.15.46:0/2636764100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b84073cf0 0x7f0b84199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 >> 172.21.15.46:0/2636764100 conn(0x7f0b840fc8e0 msgr2=0x7f0b840fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 shutdown_connections 2024-05-08T23:05:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.228+0000 7f0b8ac00700 1 -- 172.21.15.46:0/2636764100 wait complete. 2024-05-08T23:05:46.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:05:46.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 -- 172.21.15.46:0/443998686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e0101080 msgr2=0x7f99e0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 --2- 172.21.15.46:0/443998686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e0101080 0x7f99e0101460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009990 tx=0x7f99d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 -- 172.21.15.46:0/443998686 shutdown_connections 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 --2- 172.21.15.46:0/443998686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e01061a0 0x7f99e010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 --2- 172.21.15.46:0/443998686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e0101080 0x7f99e0101460 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 --2- 172.21.15.46:0/443998686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101a30 0x7f99e0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 -- 172.21.15.46:0/443998686 >> 172.21.15.46:0/443998686 conn(0x7f99e00fc8e0 msgr2=0x7f99e00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 -- 172.21.15.46:0/443998686 shutdown_connections 2024-05-08T23:05:46.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.453+0000 7f99e5800700 1 -- 172.21.15.46:0/443998686 wait complete. 2024-05-08T23:05:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.454+0000 7f99e5800700 1 Processor -- start 2024-05-08T23:05:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.454+0000 7f99e5800700 1 -- start start 2024-05-08T23:05:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101080 0x7f99e00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e0073d20 0x7f99e01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01a5b30 con 0x7f99e01061a0 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01a5ca0 con 0x7f99e0101080 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99e5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01a5e60 con 0x7f99e0073d20 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48398/0 (socket says 172.21.15.46:48398) 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 -- 172.21.15.46:0/1836359559 learned_addr learned my addr 172.21.15.46:0/1836359559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99dea00700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101080 0x7f99e00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99df400700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e0073d20 0x7f99e01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 -- 172.21.15.46:0/1836359559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e0073d20 msgr2=0x7f99e01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e0073d20 0x7f99e01a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 -- 172.21.15.46:0/1836359559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101080 msgr2=0x7f99e00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101080 0x7f99e00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.455+0000 7f99de000700 1 -- 172.21.15.46:0/1836359559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00095f0 con 0x7f99e01061a0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99de000700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e970 tx=0x7f99cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99d7400700 1 -- 172.21.15.46:0/1836359559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc009790 con 0x7f99e01061a0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99d7400700 1 -- 172.21.15.46:0/1836359559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99cc00c7e0 con 0x7f99e01061a0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e01a6140 con 0x7f99e01061a0 2024-05-08T23:05:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99d7400700 1 -- 172.21.15.46:0/1836359559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0054e0 con 0x7f99e01061a0 2024-05-08T23:05:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.456+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01a6650 con 0x7f99e01061a0 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.459+0000 7f99d7400700 1 -- 172.21.15.46:0/1836359559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f99cc00c950 con 0x7f99e01061a0 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.461+0000 7f99f9382b80 1 Processor -- start 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.461+0000 7f99f9382b80 1 -- start start 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.461+0000 7f99d7400700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 0x7f99c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99d7400700 1 -- 172.21.15.46:0/1836359559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99cc014070 con 0x7f99e01061a0 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99df400700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 0x7f99c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d965ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d95b3150 con 0x55d4d9661e00 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d95b0b10 con 0x55d4d9662720 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99f9382b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d95ac400 con 0x55d4d965c890 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48406/0 (socket says 172.21.15.46:48406) 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99dea00700 1 -- 172.21.15.46:0/2103596113 learned_addr learned my addr 172.21.15.46:0/2103596113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99de000700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.462+0000 7f99df400700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d965ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 -- 172.21.15.46:0/2103596113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 msgr2=0x55d4d965ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d965ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 -- 172.21.15.46:0/2103596113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 msgr2=0x55d4d965c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 -- 172.21.15.46:0/2103596113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d942d3b0 con 0x55d4d9661e00 2024-05-08T23:05:46.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99de000700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99df400700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 0x7f99c40a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99d8003ea0 tx=0x7f99d8009ba0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99dea00700 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005f30 tx=0x7f99d0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e84d95b4da107e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99cbe00700 1 -- 172.21.15.46:0/2103596113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d001d070 con 0x55d4d9661e00 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99cbe00700 1 -- 172.21.15.46:0/2103596113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99d001e040 con 0x55d4d9661e00 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.463+0000 7f99cbe00700 1 -- 172.21.15.46:0/2103596113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0004760 con 0x55d4d9661e00 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 -- 172.21.15.46:0/2103596113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 msgr2=0x55d4d96621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005f30 tx=0x7f99d0005c00 comp rx=0 tx=0).stop 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 -- 172.21.15.46:0/2103596113 shutdown_connections 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d965ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96621e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2103596113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 -- 172.21.15.46:0/2103596113 >> 172.21.15.46:0/2103596113 conn(0x55d4d95ba210 msgr2=0x55d4d9656df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 -- 172.21.15.46:0/2103596113 shutdown_connections 2024-05-08T23:05:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.464+0000 7f99f9382b80 1 -- 172.21.15.46:0/2103596113 wait complete. 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.465+0000 7f99f9382b80 1 -- start start 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.465+0000 7f99f9382b80 1 Processor -- start 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.465+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9668750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d9666110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d9548560 con 0x55d4d9661e00 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d9548200 con 0x55d4d9662720 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d942f6a0 con 0x55d4d965c890 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99f9382b80 10 client.0 _ll_register_callbacks cb 0x55d4d9551b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48412/0 (socket says 172.21.15.46:48412) 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 -- 172.21.15.46:0/998082888 learned_addr learned my addr 172.21.15.46:0/998082888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99dea00700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9668750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99df400700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d9666110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 -- 172.21.15.46:0/998082888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 msgr2=0x55d4d9668750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9668750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 -- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 msgr2=0x55d4d9666110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d9666110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.466+0000 7f99de000700 1 -- 172.21.15.46:0/998082888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01a6650 con 0x55d4d9661e00 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99de000700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f99cc099d80 tx=0x7f99cc00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab4bee7f5f3ba5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0a2500 con 0x55d4d9661e00 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99cc098420 con 0x55d4d9661e00 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4d9505270 con 0x55d4d9661e00 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99cc0b8250 con 0x55d4d9661e00 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99f9382b80 10 client.16323 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:46.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f99e010e590 con 0x55d4d9661e00 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4d9505270 con 0x55d4d9661e00 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99cc09a2e0 con 0x55d4d9661e00 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99f9382b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.467+0000 7f99f9382b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99f9382b80 20 client.16323 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d9676090 0x55d4d96784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99f9382b80 4 client.16323 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4d962b810 con 0x55d4d9676090 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99dea00700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d9676090 0x55d4d96784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99dea00700 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d9676090 0x55d4d96784b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b8d9cd12e48ee5 server_cookie=b90bdd4e8fb2c16 in_seq=0 out_seq=0 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.468+0000 7f99c8c00700 10 client.16323 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99c8c00700 1 -- 172.21.15.46:0/998082888 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d4d962b810 con 0x55d4d9676090 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99c8c00700 10 client.16323 handle_command_reply: tid=0 2024-05-08T23:05:46.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 client.16323 shutdown 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 msgr2=0x55d4d96659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f99cc099d80 tx=0x7f99cc00cd90 comp rx=0 tx=0).stop 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 shutdown_connections 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d9666110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9668750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d9676090 0x55d4d96784b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.470+0000 7f99c8c00700 0 client.16323 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.469+0000 7f99f9382b80 1 --2- 172.21.15.46:0/998082888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9661e00 0x55d4d96659d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.470+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 >> 172.21.15.46:0/998082888 conn(0x55d4d95ba210 msgr2=0x55d4d96565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.470+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 shutdown_connections 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.470+0000 7f99f9382b80 1 -- 172.21.15.46:0/998082888 wait complete. 2024-05-08T23:05:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.470+0000 7f99f9382b80 20 client.16323 trim_cache size 0 max 16384 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 Processor -- start 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 -- start start 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9665970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9661e00 0x55d4d9671bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 0x55d4d9656d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d9666e60 con 0x55d4d9662720 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d967d6c0 con 0x55d4d9661e00 2024-05-08T23:05:46.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99f9382b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d9663f20 con 0x55d4d965c890 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99df400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 0x55d4d9656d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99df400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 0x55d4d9656d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48422/0 (socket says 172.21.15.46:48422) 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99df400700 1 -- 172.21.15.46:0/1332400067 learned_addr learned my addr 172.21.15.46:0/1332400067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99dea00700 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9665970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.488+0000 7f99de000700 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9661e00 0x55d4d9671bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 -- 172.21.15.46:0/1332400067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9661e00 msgr2=0x55d4d9671bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9661e00 0x55d4d9671bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 -- 172.21.15.46:0/1332400067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 msgr2=0x55d4d9656d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 0x55d4d9656d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 -- 172.21.15.46:0/1332400067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d93d9b60 con 0x55d4d965c890 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99dea00700 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9665970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005320 tx=0x7f99d000bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5577dab2098fabb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99d4c00700 1 -- 172.21.15.46:0/1332400067 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d001d040 con 0x55d4d965c890 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99d4c00700 1 -- 172.21.15.46:0/1332400067 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99d0005a00 con 0x55d4d965c890 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.489+0000 7f99d4c00700 1 -- 172.21.15.46:0/1332400067 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0034610 con 0x55d4d965c890 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- 172.21.15.46:0/1332400067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 msgr2=0x55d4d9665970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9665970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99d0005320 tx=0x7f99d000bd50 comp rx=0 tx=0).stop 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- 172.21.15.46:0/1332400067 shutdown_connections 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d9662720 0x55d4d9656d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d965c890 0x55d4d9665970 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 --2- 172.21.15.46:0/1332400067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9661e00 0x55d4d9671bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- 172.21.15.46:0/1332400067 >> 172.21.15.46:0/1332400067 conn(0x55d4d95ba210 msgr2=0x55d4d9665260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- 172.21.15.46:0/1332400067 shutdown_connections 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- 172.21.15.46:0/1332400067 wait complete. 2024-05-08T23:05:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.490+0000 7f99f9382b80 1 -- start start 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 Processor -- start 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d9661e00 0x55d4d965d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d965e7d0 con 0x55d4d965c890 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d965e940 con 0x55d4d9662720 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d965eb00 con 0x55d4d9661e00 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99f9382b80 10 client.0 _ll_register_callbacks cb 0x55d4d9551b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99dea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48432/0 (socket says 172.21.15.46:48432) 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99dea00700 1 -- 172.21.15.46:0/2973377783 learned_addr learned my addr 172.21.15.46:0/2973377783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:05:46.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.491+0000 7f99de000700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d9661e00 0x55d4d965d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 -- 172.21.15.46:0/2973377783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d9661e00 msgr2=0x55d4d965d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d9661e00 0x55d4d965d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 -- 172.21.15.46:0/2973377783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 msgr2=0x55d4d966d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 -- 172.21.15.46:0/2973377783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4d942d3b0 con 0x55d4d9662720 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99dea00700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99df400700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965e0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55d4d9653f40 tx=0x7f99d800b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e018ce3cf8b84afd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8007930 con 0x55d4d9662720 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99d801f070 con 0x55d4d9662720 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.492+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d001e040 con 0x55d4d9662720 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d8007a90 con 0x55d4d9662720 2024-05-08T23:05:46.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 10 client.26309 fetch_fsmap learned FSMap version 28 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d4d9572560 con 0x55d4d9662720 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99d001e040 con 0x55d4d9662720 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99d8022070 con 0x55d4d9662720 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 10 client.26309 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 20 client.26309 populate_metadata read hostname 'smithi046' 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d96cd000 0x55d4d96cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 4 client.26309 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4d95a4ec0 con 0x55d4d96cd000 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.493+0000 7f99dea00700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d96cd000 0x55d4d96cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99dea00700 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d96cd000 0x55d4d96cf420 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba28eec6c211655 server_cookie=293ae7c08f9f2cbc in_seq=0 out_seq=0 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99c8c00700 10 client.26309 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99c8c00700 1 -- 172.21.15.46:0/2973377783 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d4d95a4ec0 con 0x55d4d96cd000 2024-05-08T23:05:46.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99c8c00700 10 client.26309 handle_command_reply: tid=0 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 client.26309 shutdown 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 msgr2=0x55d4d965e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965e0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55d4d9653f40 tx=0x7f99d800b050 comp rx=0 tx=0).stop 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 shutdown_connections 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d4d9662720 0x55d4d965e0c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d4d96cd000 0x55d4d96cf420 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4d965c890 0x55d4d966d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 --2- 172.21.15.46:0/2973377783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4d9661e00 0x55d4d965d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99c8c00700 0 client.26309 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 >> 172.21.15.46:0/2973377783 conn(0x55d4d95ba210 msgr2=0x55d4d9663390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.494+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 shutdown_connections 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99f9382b80 1 -- 172.21.15.46:0/2973377783 wait complete. 2024-05-08T23:05:46.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99f9382b80 20 client.26309 trim_cache size 0 max 16384 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "344271c0-56dd-40f9-b349-bba5a6658182": { 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "344271c0-56dd-40f9-b349-bba5a6658182", 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:05:46.503 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 msgr2=0x7f99c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 0x7f99c40a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99d8003ea0 tx=0x7f99d8009ba0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 msgr2=0x7f99e00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00e970 tx=0x7f99cc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 shutdown_connections 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e0073d20 0x7f99e01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99e0101080 0x7f99e00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99c40a1290 0x7f99c40a3750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 --2- 172.21.15.46:0/1836359559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e01061a0 0x7f99e00737e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:05:46.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 >> 172.21.15.46:0/1836359559 conn(0x7f99e00fc8e0 msgr2=0x7f99e00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:05:46.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 shutdown_connections 2024-05-08T23:05:46.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:05:46.495+0000 7f99e5800700 1 -- 172.21.15.46:0/1836359559 wait complete. 2024-05-08T23:05:46.514 INFO:tasks.cephfs.filesystem:scrub status for tag:344271c0-56dd-40f9-b349-bba5a6658182 - {'path': '/', 'tag': '344271c0-56dd-40f9-b349-bba5a6658182', 'options': 'recursive,force'} 2024-05-08T23:05:46.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6530 6.58 MB/sec warmup 8 sec latency 832.001 ms 2024-05-08T23:05:47.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:46 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:05:47.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:46 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:47.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:46 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:05:47.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:46 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:46 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:05:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:46 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:47.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6530 5.85 MB/sec warmup 9 sec latency 1576.125 ms 2024-05-08T23:05:48.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: pgmap v62: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:48.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:05:48.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:05:48.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:05:48.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: pgmap v62: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:05:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:05:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:05:48.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: pgmap v62: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:05:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:05:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:05:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:05:48.869 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6530 5.26 MB/sec warmup 10 sec latency 2576.260 ms 2024-05-08T23:05:49.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6530 4.78 MB/sec warmup 11 sec latency 3576.420 ms 2024-05-08T23:05:50.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:49 smithi053 ceph-mon[77587]: pgmap v63: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:50.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:49 smithi046 ceph-mon[36898]: pgmap v63: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:49 smithi086 ceph-mon[78610]: pgmap v63: 129 pgs: 129 active+clean; 4.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 350 KiB/s wr, 48 op/s 2024-05-08T23:05:50.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6567 4.39 MB/sec warmup 12 sec latency 4068.571 ms 2024-05-08T23:05:51.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6567 4.05 MB/sec warmup 13 sec latency 1466.043 ms 2024-05-08T23:05:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:51 smithi053 ceph-mon[77587]: pgmap v64: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 467 KiB/s wr, 57 op/s 2024-05-08T23:05:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:51 smithi046 ceph-mon[36898]: pgmap v64: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 467 KiB/s wr, 57 op/s 2024-05-08T23:05:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:51 smithi086 ceph-mon[78610]: pgmap v64: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 467 KiB/s wr, 57 op/s 2024-05-08T23:05:52.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6567 3.76 MB/sec warmup 14 sec latency 2466.202 ms 2024-05-08T23:05:53.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6567 3.51 MB/sec warmup 15 sec latency 3466.370 ms 2024-05-08T23:05:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:53 smithi053 ceph-mon[77587]: pgmap v65: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 455 KiB/s wr, 50 op/s 2024-05-08T23:05:54.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:53 smithi046 ceph-mon[36898]: pgmap v65: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 455 KiB/s wr, 50 op/s 2024-05-08T23:05:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:53 smithi086 ceph-mon[78610]: pgmap v65: 129 pgs: 129 active+clean; 6.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 455 KiB/s wr, 50 op/s 2024-05-08T23:05:54.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6579 3.29 MB/sec warmup 16 sec latency 3876.156 ms 2024-05-08T23:05:55.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6731 3.13 MB/sec warmup 17 sec latency 1055.009 ms 2024-05-08T23:05:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:55 smithi053 ceph-mon[77587]: pgmap v66: 129 pgs: 129 active+clean; 22 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2024-05-08T23:05:56.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:55 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:05:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:55 smithi046 ceph-mon[36898]: pgmap v66: 129 pgs: 129 active+clean; 22 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2024-05-08T23:05:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:55 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:05:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:55 smithi086 ceph-mon[78610]: pgmap v66: 129 pgs: 129 active+clean; 22 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2024-05-08T23:05:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:55 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:05:56.870 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6731 2.96 MB/sec warmup 18 sec latency 1504.030 ms 2024-05-08T23:05:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:56 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:05:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:56 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:05:57.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:56 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:05:57.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:56 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:05:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:56 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:05:57.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:56 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:05:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:56 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:05:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:56 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:05:57.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:56 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:05:57.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6731 2.80 MB/sec warmup 19 sec latency 2504.167 ms 2024-05-08T23:05:58.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:57 smithi053 ceph-mon[77587]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:05:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:57 smithi046 ceph-mon[36898]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:05:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:57 smithi086 ceph-mon[78610]: pgmap v67: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:05:58.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6731 2.66 MB/sec warmup 20 sec latency 3504.381 ms 2024-05-08T23:05:59.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 6731 2.54 MB/sec warmup 21 sec latency 4504.524 ms 2024-05-08T23:06:00.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:05:59 smithi053 ceph-mon[77587]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:06:00.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:05:59 smithi046 ceph-mon[36898]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:06:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:05:59 smithi086 ceph-mon[78610]: pgmap v68: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-05-08T23:06:00.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 7550 2.69 MB/sec warmup 22 sec latency 4761.012 ms 2024-05-08T23:06:01.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:01.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:01.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:01.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 9973 3.06 MB/sec warmup 23 sec latency 8.876 ms 2024-05-08T23:06:02.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:01 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:02.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:01 smithi053 ceph-mon[77587]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.1 MiB/s wr, 42 op/s 2024-05-08T23:06:02.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:01 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:02.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:01 smithi046 ceph-mon[36898]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.1 MiB/s wr, 42 op/s 2024-05-08T23:06:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:01 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:01 smithi086 ceph-mon[78610]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.1 MiB/s wr, 42 op/s 2024-05-08T23:06:02.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 11562 3.29 MB/sec warmup 24 sec latency 9.283 ms 2024-05-08T23:06:03.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 14285 3.84 MB/sec warmup 25 sec latency 9.971 ms 2024-05-08T23:06:04.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:03 smithi053 ceph-mon[77587]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-05-08T23:06:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:03 smithi046 ceph-mon[36898]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-05-08T23:06:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:03 smithi086 ceph-mon[78610]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-05-08T23:06:04.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 16056 3.96 MB/sec warmup 26 sec latency 10.118 ms 2024-05-08T23:06:05.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 18281 4.32 MB/sec warmup 27 sec latency 9.800 ms 2024-05-08T23:06:06.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:05 smithi053 ceph-mon[77587]: pgmap v71: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 110 op/s 2024-05-08T23:06:06.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:05 smithi046 ceph-mon[36898]: pgmap v71: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 110 op/s 2024-05-08T23:06:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:05 smithi086 ceph-mon[78610]: pgmap v71: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 110 op/s 2024-05-08T23:06:06.871 INFO:tasks.workunit.client.0.smithi046.stdout: 1 20949 4.63 MB/sec warmup 28 sec latency 19.858 ms 2024-05-08T23:06:07.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 22726 4.85 MB/sec warmup 29 sec latency 10.670 ms 2024-05-08T23:06:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:07 smithi053 ceph-mon[77587]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-05-08T23:06:08.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:07 smithi046 ceph-mon[36898]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-05-08T23:06:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:07 smithi086 ceph-mon[78610]: pgmap v72: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-05-08T23:06:08.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 25235 5.16 MB/sec warmup 30 sec latency 10.704 ms 2024-05-08T23:06:09.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 27592 5.35 MB/sec warmup 31 sec latency 14.452 ms 2024-05-08T23:06:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:09 smithi053 ceph-mon[77587]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 908 KiB/s wr, 100 op/s 2024-05-08T23:06:10.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:09 smithi046 ceph-mon[36898]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 908 KiB/s wr, 100 op/s 2024-05-08T23:06:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:09 smithi086 ceph-mon[78610]: pgmap v73: 129 pgs: 129 active+clean; 30 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 908 KiB/s wr, 100 op/s 2024-05-08T23:06:10.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 29217 5.45 MB/sec warmup 32 sec latency 10.467 ms 2024-05-08T23:06:11.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 31802 5.71 MB/sec warmup 33 sec latency 52.996 ms 2024-05-08T23:06:12.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:11 smithi046 ceph-mon[36898]: pgmap v74: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-08T23:06:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:11 smithi086 ceph-mon[78610]: pgmap v74: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-08T23:06:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:11 smithi053 ceph-mon[77587]: pgmap v74: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-08T23:06:12.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 33434 5.83 MB/sec warmup 34 sec latency 12.656 ms 2024-05-08T23:06:13.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 35799 6.05 MB/sec warmup 35 sec latency 13.351 ms 2024-05-08T23:06:14.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:13 smithi046 ceph-mon[36898]: pgmap v75: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-08T23:06:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:13 smithi086 ceph-mon[78610]: pgmap v75: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-08T23:06:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:13 smithi053 ceph-mon[77587]: pgmap v75: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-08T23:06:14.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 38066 6.12 MB/sec warmup 36 sec latency 14.409 ms 2024-05-08T23:06:15.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 39788 6.26 MB/sec warmup 37 sec latency 10.346 ms 2024-05-08T23:06:16.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:15 smithi046 ceph-mon[36898]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-05-08T23:06:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:15 smithi086 ceph-mon[78610]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-05-08T23:06:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:15 smithi053 ceph-mon[77587]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-05-08T23:06:16.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 -- 172.21.15.46:0/1859471161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c103aa0 msgr2=0x7fab8c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 --2- 172.21.15.46:0/1859471161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c103aa0 0x7fab8c107ae0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab78009990 tx=0x7fab78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 -- 172.21.15.46:0/1859471161 shutdown_connections 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 --2- 172.21.15.46:0/1859471161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 --2- 172.21.15.46:0/1859471161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c1030f0 0x7fab8c1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 --2- 172.21.15.46:0/1859471161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c103aa0 0x7fab8c107ae0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 -- 172.21.15.46:0/1859471161 >> 172.21.15.46:0/1859471161 conn(0x7fab8c0fe930 msgr2=0x7fab8c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 -- 172.21.15.46:0/1859471161 shutdown_connections 2024-05-08T23:06:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.737+0000 7fab8be00700 1 -- 172.21.15.46:0/1859471161 wait complete. 2024-05-08T23:06:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.739+0000 7fab8be00700 1 Processor -- start 2024-05-08T23:06:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.739+0000 7fab8be00700 1 -- start start 2024-05-08T23:06:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 0x7fab8c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19e610 con 0x7fab8c1030f0 2024-05-08T23:06:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19e780 con 0x7fab8c19db90 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19e940 con 0x7fab8c108210 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43400/0 (socket says 172.21.15.46:43400) 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8aa00700 1 -- 172.21.15.46:0/828279450 learned_addr learned my addr 172.21.15.46:0/828279450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38492/0 (socket says 172.21.15.46:38492) 2024-05-08T23:06:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.740+0000 7fab8b400700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 0x7fab8c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 -- 172.21.15.46:0/828279450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 msgr2=0x7fab8c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 -- 172.21.15.46:0/828279450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 msgr2=0x7fab8c19e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 0x7fab8c19e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 -- 172.21.15.46:0/828279450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c009520 con 0x7fab8c1030f0 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8b400700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 0x7fab8c19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8a000700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8aa00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00eb30 tx=0x7fab7c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab90a00700 1 -- 172.21.15.46:0/828279450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c009880 con 0x7fab8c1030f0 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab90a00700 1 -- 172.21.15.46:0/828279450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab7c0044e0 con 0x7fab8c1030f0 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab90a00700 1 -- 172.21.15.46:0/828279450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c003e90 con 0x7fab8c1030f0 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.741+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab780095f0 con 0x7fab8c1030f0 2024-05-08T23:06:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.742+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c075770 con 0x7fab8c1030f0 2024-05-08T23:06:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.745+0000 7fab90a00700 1 -- 172.21.15.46:0/828279450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab7c004000 con 0x7fab8c1030f0 2024-05-08T23:06:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.746+0000 7fab90a00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 0x7fab700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.746+0000 7fab90a00700 1 -- 172.21.15.46:0/828279450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab7c014070 con 0x7fab8c1030f0 2024-05-08T23:06:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.746+0000 7fab8a000700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 0x7fab700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.747+0000 7faba5962b80 1 Processor -- start 2024-05-08T23:06:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.747+0000 7fab8a000700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 0x7fab700a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab8c19a800 tx=0x7fab7800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.747+0000 7faba5962b80 1 -- start start 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1ac4970 0x5556d1abe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1abe890 0x5556d1abecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1a15150 con 0x5556d1abe890 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1a12b10 con 0x5556d1ac4970 2024-05-08T23:06:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7faba5962b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1a0e400 con 0x5556d1ac4050 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38508/0 (socket says 172.21.15.46:38508) 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 -- 172.21.15.46:0/381370638 learned_addr learned my addr 172.21.15.46:0/381370638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8a000700 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1ac4970 0x5556d1abe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8b400700 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1abe890 0x5556d1abecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 -- 172.21.15.46:0/381370638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1ac4970 msgr2=0x5556d1abe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1ac4970 0x5556d1abe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 -- 172.21.15.46:0/381370638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1abe890 msgr2=0x5556d1abecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1abe890 0x5556d1abecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.748+0000 7fab8aa00700 1 -- 172.21.15.46:0/381370638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d188f3b0 con 0x5556d1ac4050 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.749+0000 7fab8aa00700 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0a21e0 tx=0x7fab7c099e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6127ee16846201e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.749+0000 7fab77e00700 1 -- 172.21.15.46:0/381370638 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00cc90 con 0x5556d1ac4050 2024-05-08T23:06:16.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.749+0000 7fab77e00700 1 -- 172.21.15.46:0/381370638 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab7c09a290 con 0x5556d1ac4050 2024-05-08T23:06:16.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.749+0000 7fab77e00700 1 -- 172.21.15.46:0/381370638 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0cd140 con 0x5556d1ac4050 2024-05-08T23:06:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 -- 172.21.15.46:0/381370638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 msgr2=0x5556d1ac4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0a21e0 tx=0x7fab7c099e50 comp rx=0 tx=0).stop 2024-05-08T23:06:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 -- 172.21.15.46:0/381370638 shutdown_connections 2024-05-08T23:06:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1abe890 0x5556d1abecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac4430 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 --2- 172.21.15.46:0/381370638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1ac4970 0x5556d1abe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 -- 172.21.15.46:0/381370638 >> 172.21.15.46:0/381370638 conn(0x5556d1a1c210 msgr2=0x5556d1ab92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 -- 172.21.15.46:0/381370638 shutdown_connections 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.750+0000 7faba5962b80 1 -- 172.21.15.46:0/381370638 wait complete. 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.751+0000 7faba5962b80 1 -- start start 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.751+0000 7faba5962b80 1 Processor -- start 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 0x5556d1ac7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1ac8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d19aa560 con 0x5556d1ac4970 2024-05-08T23:06:16.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d19aa200 con 0x5556d1abe890 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d18916a0 con 0x5556d1ac4050 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7faba5962b80 10 client.0 _ll_register_callbacks cb 0x5556d19b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 0x5556d1ac7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 0x5556d1ac7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47704/0 (socket says 172.21.15.46:47704) 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7fab8aa00700 1 -- 172.21.15.46:0/50871254 learned_addr learned my addr 172.21.15.46:0/50871254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7fab8b400700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1ac8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.752+0000 7fab8a000700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 -- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 msgr2=0x5556d1ac7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 0x5556d1ac7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 -- 172.21.15.46:0/50871254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 msgr2=0x5556d1ac8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1ac8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 -- 172.21.15.46:0/50871254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c075770 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab8a000700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac8210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab780098e0 tx=0x7fab7801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafcf63e1f2448a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab74c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7802b050 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.753+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab78004e40 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.754+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556d1967270 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.754+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab7800f870 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.754+0000 7faba5962b80 10 client.26365 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.754+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5556d183bb60 con 0x5556d1ac4050 2024-05-08T23:06:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.754+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556d1967270 con 0x5556d1ac4050 2024-05-08T23:06:16.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fab780249b0 con 0x5556d1ac4050 2024-05-08T23:06:16.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:16.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:16.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 20 client.26365 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1ad8420 0x5556d1ada840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 4 client.26365 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556d1a8d810 con 0x5556d1ad8420 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7fab8aa00700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1ad8420 0x5556d1ada840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7fab8aa00700 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1ad8420 0x5556d1ada840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf3d8c0a66be49 server_cookie=211e4cba14060109 in_seq=0 out_seq=0 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.755+0000 7fab74c00700 10 client.26365 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7fab74c00700 1 -- 172.21.15.46:0/50871254 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fab7c065a90 con 0x5556d1ad8420 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7fab74c00700 10 client.26365 handle_command_reply: tid=0 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7faba5962b80 1 client.26365 shutdown 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 msgr2=0x5556d1ac8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac8210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab780098e0 tx=0x7fab7801a050 comp rx=0 tx=0).stop 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 shutdown_connections 2024-05-08T23:06:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.756+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1ad8420 0x5556d1ada840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1ac8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1abe890 0x5556d1ac7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7fab74c00700 0 client.26365 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 --2- 172.21.15.46:0/50871254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1ac4050 0x5556d1ac8210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 >> 172.21.15.46:0/50871254 conn(0x5556d1a1c210 msgr2=0x5556d1ab90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 shutdown_connections 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 1 -- 172.21.15.46:0/50871254 wait complete. 2024-05-08T23:06:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.757+0000 7faba5962b80 20 client.26365 trim_cache size 0 max 16384 2024-05-08T23:06:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.778+0000 7faba5962b80 1 Processor -- start 2024-05-08T23:06:16.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 -- start start 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1adf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1ac1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ab3a90 con 0x5556d1ac4970 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ad4f40 con 0x5556d1adf6e0 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7faba5962b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ad5ad0 con 0x5556d1abe890 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38524/0 (socket says 172.21.15.46:38524) 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7fab8b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1ac1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7fab8a000700 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1adf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.779+0000 7fab8aa00700 1 -- 172.21.15.46:0/1054648040 learned_addr learned my addr 172.21.15.46:0/1054648040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 -- 172.21.15.46:0/1054648040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 msgr2=0x5556d1ac1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1ac1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 -- 172.21.15.46:0/1054648040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 msgr2=0x5556d1adf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1adf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 -- 172.21.15.46:0/1054648040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556d19d4560 con 0x5556d1abe890 2024-05-08T23:06:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab8aa00700 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a2cd8b5a3e25b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab81200700 1 -- 172.21.15.46:0/1054648040 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0a2710 con 0x5556d1abe890 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab81200700 1 -- 172.21.15.46:0/1054648040 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab7c061070 con 0x5556d1abe890 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.780+0000 7fab81200700 1 -- 172.21.15.46:0/1054648040 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c09a4f0 con 0x5556d1abe890 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 -- 172.21.15.46:0/1054648040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 msgr2=0x5556d1abe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c003930 comp rx=0 tx=0).stop 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 -- 172.21.15.46:0/1054648040 shutdown_connections 2024-05-08T23:06:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1ac1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1abe350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 --2- 172.21.15.46:0/1054648040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1adf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 -- 172.21.15.46:0/1054648040 >> 172.21.15.46:0/1054648040 conn(0x5556d1a1c210 msgr2=0x5556d1ab90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.781+0000 7faba5962b80 1 -- 172.21.15.46:0/1054648040 shutdown_connections 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.782+0000 7faba5962b80 1 -- 172.21.15.46:0/1054648040 wait complete. 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.782+0000 7faba5962b80 1 -- start start 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.782+0000 7faba5962b80 1 Processor -- start 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1acf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1acda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ace170 con 0x5556d1ac4970 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ace2e0 con 0x5556d1adf6e0 2024-05-08T23:06:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556d1ace4a0 con 0x5556d1abe890 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7faba5962b80 10 client.0 _ll_register_callbacks cb 0x5556d19b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38536/0 (socket says 172.21.15.46:38536) 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1acda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 -- 172.21.15.46:0/619621364 learned_addr learned my addr 172.21.15.46:0/619621364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8a000700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1acf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 -- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 msgr2=0x5556d1acda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1acda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 -- 172.21.15.46:0/619621364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 msgr2=0x5556d1acf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1acf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.783+0000 7fab8aa00700 1 -- 172.21.15.46:0/619621364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556d188f3b0 con 0x5556d1abe890 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab8a000700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1acf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab8aa00700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0cae50 tx=0x7fab7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31065f8ceebd7b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab74c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0986d0 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab7c0b2e40 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab7c09a290 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab7c0a2710 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7faba5962b80 10 client.26371 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5556d1893cd0 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.784+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab7c09a290 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fab7c09a4f0 con 0x5556d1abe890 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 10 client.26371 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 10 client.26371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 20 client.26371 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1b2f7b0 0x5556d1b31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 4 client.26371 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:06:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556d1a06ec0 con 0x5556d1b2f7b0 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7fab8a000700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1b2f7b0 0x5556d1b31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7fab8a000700 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1b2f7b0 0x5556d1b31ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559a705ec62a0e0 server_cookie=80efee158c322284 in_seq=0 out_seq=0 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.785+0000 7fab74c00700 10 client.26371 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7fab74c00700 1 -- 172.21.15.46:0/619621364 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5556d1a06ec0 con 0x5556d1b2f7b0 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7fab74c00700 10 client.26371 handle_command_reply: tid=0 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 client.26371 shutdown 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 msgr2=0x5556d1acf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0cae50 tx=0x7fab7c004750 comp rx=0 tx=0).stop 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 shutdown_connections 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556d1adf6e0 0x5556d1acda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556d1abe890 0x5556d1acf2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5556d1b2f7b0 0x5556d1b31ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 --2- 172.21.15.46:0/619621364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556d1ac4970 0x5556d1acf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 >> 172.21.15.46:0/619621364 conn(0x5556d1a1c210 msgr2=0x5556d1ac1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7fab74c00700 0 client.26371 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 shutdown_connections 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 1 -- 172.21.15.46:0/619621364 wait complete. 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.786+0000 7faba5962b80 20 client.26371 trim_cache size 0 max 16384 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:06:16.792 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 msgr2=0x7fab700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 0x7fab700a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab8c19a800 tx=0x7fab7800b260 comp rx=0 tx=0).stop 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 msgr2=0x7fab8c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00eb30 tx=0x7fab7c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 shutdown_connections 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8c19db90 0x7fab8c19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab700a1330 0x7fab700a37f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c1030f0 0x7fab8c1992b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 --2- 172.21.15.46:0/828279450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab8c108210 0x7fab8c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 >> 172.21.15.46:0/828279450 conn(0x7fab8c0fe930 msgr2=0x7fab8c100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 shutdown_connections 2024-05-08T23:06:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:16.787+0000 7fab8be00700 1 -- 172.21.15.46:0/828279450 wait complete. 2024-05-08T23:06:16.806 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:06:16.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:06:16.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42607 6.55 MB/sec warmup 38 sec latency 9.943 ms 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.022+0000 7f257ac00700 1 -- 172.21.15.46:0/3609497843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2574103c30 msgr2=0x7f2574107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.022+0000 7f257ac00700 1 --2- 172.21.15.46:0/3609497843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2574103c30 0x7f2574107c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2560009990 tx=0x7f2560009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 -- 172.21.15.46:0/3609497843 shutdown_connections 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 --2- 172.21.15.46:0/3609497843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741083a0 0x7f257410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 --2- 172.21.15.46:0/3609497843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f2574103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 --2- 172.21.15.46:0/3609497843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2574103c30 0x7f2574107c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 -- 172.21.15.46:0/3609497843 >> 172.21.15.46:0/3609497843 conn(0x7f25740feaa0 msgr2=0x7f2574100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 -- 172.21.15.46:0/3609497843 shutdown_connections 2024-05-08T23:06:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.023+0000 7f257ac00700 1 -- 172.21.15.46:0/3609497843 wait complete. 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 Processor -- start 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 -- start start 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 0x7f2574199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 0x7f257419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257419e8c0 con 0x7f257419de40 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257419ea30 con 0x7f2574103280 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.024+0000 7f257ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257419ebf0 con 0x7f25741083a0 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2578c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 0x7f257419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 0x7f2574199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2578c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 0x7f257419e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43464/0 (socket says 172.21.15.46:43464) 2024-05-08T23:06:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47732/0 (socket says 172.21.15.46:47732) 2024-05-08T23:06:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2578c00700 1 -- 172.21.15.46:0/1143765485 learned_addr learned my addr 172.21.15.46:0/1143765485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 -- 172.21.15.46:0/1143765485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 msgr2=0x7f2574199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 0x7f2574199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 -- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 msgr2=0x7f257419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 0x7f257419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 -- 172.21.15.46:0/1143765485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600095f0 con 0x7f2574103280 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573400700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 0x7f2574199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.025+0000 7f2573e00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9f0 tx=0x7f256400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.026+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564004220 con 0x7f2574103280 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.026+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2564004cf0 con 0x7f2574103280 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.026+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400ce70 con 0x7f2574103280 2024-05-08T23:06:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.026+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25741a6150 con 0x7f2574103280 2024-05-08T23:06:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.026+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25741a65e0 con 0x7f2574103280 2024-05-08T23:06:17.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.027+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257410b670 con 0x7f2574103280 2024-05-08T23:06:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.029+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2564004750 con 0x7f2574103280 2024-05-08T23:06:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.030+0000 7f2570c00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 0x7f25580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.031+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2564014070 con 0x7f2574103280 2024-05-08T23:06:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.031+0000 7f2573400700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 0x7f25580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.031+0000 7f2573400700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 0x7f25580a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f257419aab0 tx=0x7f2560005960 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.033+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f25640960d0 con 0x7f2574103280 2024-05-08T23:06:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.326+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f257419a900 con 0x7f2574103280 2024-05-08T23:06:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.327+0000 7f2570c00700 1 -- 172.21.15.46:0/1143765485 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10907 (secure 0 0 0) 0x7f2564004e60 con 0x7f2574103280 2024-05-08T23:06:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:06:17.326 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:05:40.700994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:06:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.329+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 msgr2=0x7f25580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.329+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 0x7f25580a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f257419aab0 tx=0x7f2560005960 comp rx=0 tx=0).stop 2024-05-08T23:06:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.329+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 msgr2=0x7f25741995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.329+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9f0 tx=0x7f256400ed00 comp rx=0 tx=0).stop 2024-05-08T23:06:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 shutdown_connections 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f25580a1310 0x7f25580a37d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257419de40 0x7f257419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2574103280 0x7f25741995b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 --2- 172.21.15.46:0/1143765485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25741083a0 0x7f2574199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 >> 172.21.15.46:0/1143765485 conn(0x7f25740feaa0 msgr2=0x7f25740ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 shutdown_connections 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.330+0000 7f257ac00700 1 -- 172.21.15.46:0/1143765485 wait complete. 2024-05-08T23:06:17.330 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-05-08T23:06:17.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:06:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 -- 172.21.15.46:0/437797301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28010b3c0 msgr2=0x7fe28010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 --2- 172.21.15.46:0/437797301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28010b3c0 0x7fe28010d880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe27800b0a0 tx=0x7fe27800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 -- 172.21.15.46:0/437797301 shutdown_connections 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 --2- 172.21.15.46:0/437797301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28010b3c0 0x7fe28010d880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 --2- 172.21.15.46:0/437797301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe280101fa0 0x7fe280102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 --2- 172.21.15.46:0/437797301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2801028c0 0x7fe28010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 -- 172.21.15.46:0/437797301 >> 172.21.15.46:0/437797301 conn(0x7fe2800fb760 msgr2=0x7fe2800fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 -- 172.21.15.46:0/437797301 shutdown_connections 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.577+0000 7fe287800700 1 -- 172.21.15.46:0/437797301 wait complete. 2024-05-08T23:06:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.578+0000 7fe287800700 1 Processor -- start 2024-05-08T23:06:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 -- start start 2024-05-08T23:06:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 0x7fe280199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 0x7fe280199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 0x7fe28019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28019e520 con 0x7fe280101fa0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28019e690 con 0x7fe28010b3c0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.579+0000 7fe287800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28019e850 con 0x7fe2801028c0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe284e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 0x7fe280199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 0x7fe280199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 0x7fe28019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 0x7fe280199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38570/0 (socket says 172.21.15.46:38570) 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe27fe00700 1 -- 172.21.15.46:0/2451422566 learned_addr learned my addr 172.21.15.46:0/2451422566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 -- 172.21.15.46:0/2451422566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 msgr2=0x7fe280199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 0x7fe280199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 -- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 msgr2=0x7fe280199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 0x7fe280199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 -- 172.21.15.46:0/2451422566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c009520 con 0x7fe28010b3c0 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe284e00700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 0x7fe280199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.580+0000 7fe285800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 0x7fe28019dee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe27800b070 tx=0x7fe278007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.581+0000 7fe27d600700 1 -- 172.21.15.46:0/2451422566 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27800e040 con 0x7fe28010b3c0 2024-05-08T23:06:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.581+0000 7fe27d600700 1 -- 172.21.15.46:0/2451422566 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27801d470 con 0x7fe28010b3c0 2024-05-08T23:06:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.581+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278009d00 con 0x7fe28010b3c0 2024-05-08T23:06:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.581+0000 7fe27d600700 1 -- 172.21.15.46:0/2451422566 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278005280 con 0x7fe28010b3c0 2024-05-08T23:06:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.581+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2801a6190 con 0x7fe28010b3c0 2024-05-08T23:06:17.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.584+0000 7fe27d600700 1 -- 172.21.15.46:0/2451422566 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe27801d5e0 con 0x7fe28010b3c0 2024-05-08T23:06:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe27d600700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 0x7fe2680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 Processor -- start 2024-05-08T23:06:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe27d600700 1 -- 172.21.15.46:0/2451422566 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe2780ce3e0 con 0x7fe28010b3c0 2024-05-08T23:06:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 -- start start 2024-05-08T23:06:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe284e00700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 0x7fe2680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee5e00 0x55f848ee61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848ee0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848e37150 con 0x55f848ee5e00 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848e34b10 con 0x55f848ee0890 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848e30400 con 0x55f848ee6720 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38576/0 (socket says 172.21.15.46:38576) 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.586+0000 7fe27fe00700 1 -- 172.21.15.46:0/1484115286 learned_addr learned my addr 172.21.15.46:0/1484115286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe284e00700 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee5e00 0x55f848ee61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe285800700 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848ee0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 -- 172.21.15.46:0/1484115286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 msgr2=0x55f848ee0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848ee0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 -- 172.21.15.46:0/1484115286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee5e00 msgr2=0x55f848ee61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee5e00 0x55f848ee61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 -- 172.21.15.46:0/1484115286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f848cb13b0 con 0x55f848ee6720 2024-05-08T23:06:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe284e00700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 0x7fe2680a3750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe274011550 tx=0x7fe274009f90 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe27fe00700 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009f30 tx=0x7fe26c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d04caf677502b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe272000700 1 -- 172.21.15.46:0/1484115286 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c004a30 con 0x55f848ee6720 2024-05-08T23:06:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe272000700 1 -- 172.21.15.46:0/1484115286 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c0054e0 con 0x55f848ee6720 2024-05-08T23:06:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.587+0000 7fe272000700 1 -- 172.21.15.46:0/1484115286 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0198d0 con 0x55f848ee6720 2024-05-08T23:06:17.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 -- 172.21.15.46:0/1484115286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 msgr2=0x55f848ee0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009f30 tx=0x7fe26c011720 comp rx=0 tx=0).stop 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 -- 172.21.15.46:0/1484115286 shutdown_connections 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848ee0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee5e00 0x55f848ee61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/1484115286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee6720 0x55f848ee0350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 -- 172.21.15.46:0/1484115286 >> 172.21.15.46:0/1484115286 conn(0x55f848e3e210 msgr2=0x55f848edadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 -- 172.21.15.46:0/1484115286 shutdown_connections 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.588+0000 7fe29b39ab80 1 -- 172.21.15.46:0/1484115286 wait complete. 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.589+0000 7fe29b39ab80 1 -- start start 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.589+0000 7fe29b39ab80 1 Processor -- start 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 0x55f848ee9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 0x55f848eea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848dcc560 con 0x55f848ee0890 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848dcc200 con 0x55f848ee6720 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848cb36a0 con 0x55f848ee5e00 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe29b39ab80 10 client.0 _ll_register_callbacks cb 0x55f848dd5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe284e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 0x55f848ee9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38580/0 (socket says 172.21.15.46:38580) 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe284e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 0x55f848ee9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43484/0 (socket says 172.21.15.46:43484) 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.590+0000 7fe27fe00700 1 -- 172.21.15.46:0/2880620056 learned_addr learned my addr 172.21.15.46:0/2880620056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe285800700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 0x55f848eea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 -- 172.21.15.46:0/2880620056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 msgr2=0x55f848eea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 0x55f848eea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 -- 172.21.15.46:0/2880620056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 msgr2=0x55f848ee9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 0x55f848ee9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 -- 172.21.15.46:0/2880620056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c009b60 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe285800700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 0x55f848eea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe27fe00700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe26c01b040 tx=0x7fe26c004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad1c8fefa81f73e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe266a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c004040 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe26c0041a0 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.591+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f848d89270 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe26c004360 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 10 client.26389 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f848c5db60 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f848d89270 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe26c028020 con 0x55f848ee5e00 2024-05-08T23:06:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 10 client.26389 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 10 client.26389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 20 client.26389 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848efa250 0x55f848efc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 4 client.26389 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.592+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f848eaf810 con 0x55f848efa250 2024-05-08T23:06:17.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.593+0000 7fe284e00700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848efa250 0x55f848efc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.593+0000 7fe284e00700 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848efa250 0x55f848efc670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2677a07a55e7491 server_cookie=9826b40fc958f32 in_seq=0 out_seq=0 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.593+0000 7fe266a00700 10 client.26389 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe266a00700 1 -- 172.21.15.46:0/2880620056 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f848eaf810 con 0x55f848efa250 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe266a00700 10 client.26389 handle_command_reply: tid=0 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 client.26389 shutdown 2024-05-08T23:06:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 msgr2=0x55f848eea130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe26c01b040 tx=0x7fe26c004cd0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 shutdown_connections 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848efa250 0x55f848efc670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee6720 0x55f848eea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee0890 0x55f848ee9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/2880620056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eea130 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe266a00700 0 client.26389 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 >> 172.21.15.46:0/2880620056 conn(0x55f848e3e210 msgr2=0x55f848eda5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 shutdown_connections 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 1 -- 172.21.15.46:0/2880620056 wait complete. 2024-05-08T23:06:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.594+0000 7fe29b39ab80 20 client.26389 trim_cache size 0 max 16384 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 Processor -- start 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 -- start start 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee5e00 0x55f848eea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848edbca0 con 0x55f848ee6720 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848ee8140 con 0x55f848ee5e00 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848ee3870 con 0x55f848ee0890 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe285800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe284e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe285800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43494/0 (socket says 172.21.15.46:43494) 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe285800700 1 -- 172.21.15.46:0/3699536 learned_addr learned my addr 172.21.15.46:0/3699536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe284e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38584/0 (socket says 172.21.15.46:38584) 2024-05-08T23:06:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.609+0000 7fe27fe00700 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee5e00 0x55f848eea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 -- 172.21.15.46:0/3699536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee5e00 msgr2=0x55f848eea490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee5e00 0x55f848eea490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 -- 172.21.15.46:0/3699536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 msgr2=0x55f848ee4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 -- 172.21.15.46:0/3699536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f848df6560 con 0x55f848ee0890 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe285800700 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe284e00700 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f848efa630 tx=0x7fe27401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a20a97b8433e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe272a00700 1 -- 172.21.15.46:0/3699536 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274022040 con 0x55f848ee0890 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe272a00700 1 -- 172.21.15.46:0/3699536 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe274023040 con 0x55f848ee0890 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.610+0000 7fe272a00700 1 -- 172.21.15.46:0/3699536 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274029910 con 0x55f848ee0890 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3699536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 msgr2=0x55f848edad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f848efa630 tx=0x7fe27401c000 comp rx=0 tx=0).stop 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3699536 shutdown_connections 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848ee4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee0890 0x55f848edad20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3699536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee5e00 0x55f848eea490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3699536 >> 172.21.15.46:0/3699536 conn(0x55f848e3e210 msgr2=0x55f848ee84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3699536 shutdown_connections 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3699536 wait complete. 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.611+0000 7fe29b39ab80 1 -- start start 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 Processor -- start 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848eef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848f4f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848ef0090 con 0x55f848ee6720 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848f4fc80 con 0x55f848ee0890 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f848f4fe40 con 0x55f848ee5e00 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe29b39ab80 10 client.0 _ll_register_callbacks cb 0x55f848dd5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe285800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848f4f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.612+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38594/0 (socket says 172.21.15.46:38594) 2024-05-08T23:06:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 -- 172.21.15.46:0/3540025403 learned_addr learned my addr 172.21.15.46:0/3540025403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe284e00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848eef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 -- 172.21.15.46:0/3540025403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 msgr2=0x55f848eef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848eef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 -- 172.21.15.46:0/3540025403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 msgr2=0x55f848f4f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848f4f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 -- 172.21.15.46:0/3540025403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2801a6190 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe27fe00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c019840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4961a0402b3a645e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe266a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c019d70 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe26c019920 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.613+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe26c0054e0 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe26c004240 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 10 client.26395 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe274003680 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c0054e0 con 0x55f848ee5e00 2024-05-08T23:06:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe26c028070 con 0x55f848ee5e00 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 10 client.26395 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 10 client.26395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 20 client.26395 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848f512f0 0x55f848f53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 4 client.26395 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f848e28ec0 con 0x55f848f512f0 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.614+0000 7fe284e00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848f512f0 0x55f848f53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe266a00700 10 client.26395 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe284e00700 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848f512f0 0x55f848f53710 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fe7ef554885cb2 server_cookie=329ee241efb5e44d in_seq=0 out_seq=0 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe266a00700 1 -- 172.21.15.46:0/3540025403 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f848e28ec0 con 0x55f848f512f0 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe266a00700 10 client.26395 handle_command_reply: tid=0 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 client.26395 shutdown 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 msgr2=0x55f848eef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c019840 comp rx=0 tx=0).stop 2024-05-08T23:06:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 shutdown_connections 2024-05-08T23:06:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f848f512f0 0x55f848f53710 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f848ee6720 0x55f848f4f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f848ee0890 0x55f848eef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 --2- 172.21.15.46:0/3540025403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f848ee5e00 0x55f848eef900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe266a00700 0 client.26395 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.615+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 >> 172.21.15.46:0/3540025403 conn(0x55f848e3e210 msgr2=0x55f848ee4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 shutdown_connections 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe29b39ab80 1 -- 172.21.15.46:0/3540025403 wait complete. 2024-05-08T23:06:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe29b39ab80 20 client.26395 trim_cache size 0 max 16384 2024-05-08T23:06:17.623 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:17.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 msgr2=0x7fe2680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 0x7fe2680a3750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe274011550 tx=0x7fe274009f90 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 msgr2=0x7fe28019dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.616+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 0x7fe28019dee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe27800b070 tx=0x7fe278007860 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 shutdown_connections 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe28010b3c0 0x7fe28019dee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2680a1290 0x7fe2680a3750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe280101fa0 0x7fe280199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 --2- 172.21.15.46:0/2451422566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2801028c0 0x7fe280199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 >> 172.21.15.46:0/2451422566 conn(0x7fe2800fb760 msgr2=0x7fe2800fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 shutdown_connections 2024-05-08T23:06:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.617+0000 7fe287800700 1 -- 172.21.15.46:0/2451422566 wait complete. 2024-05-08T23:06:17.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:06:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.872+0000 7f724f600700 1 -- 172.21.15.46:0/3939112827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7248101a30 msgr2=0x7f7248105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.872+0000 7f724f600700 1 --2- 172.21.15.46:0/3939112827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7248101a30 0x7f7248105a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7234009990 tx=0x7f7234009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 -- 172.21.15.46:0/3939112827 shutdown_connections 2024-05-08T23:06:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 --2- 172.21.15.46:0/3939112827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72481061a0 0x7f724810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 --2- 172.21.15.46:0/3939112827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248101080 0x7f7248101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 --2- 172.21.15.46:0/3939112827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7248101a30 0x7f7248105a70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 -- 172.21.15.46:0/3939112827 >> 172.21.15.46:0/3939112827 conn(0x7f72480fc8e0 msgr2=0x7f72480fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 -- 172.21.15.46:0/3939112827 shutdown_connections 2024-05-08T23:06:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.873+0000 7f724f600700 1 -- 172.21.15.46:0/3939112827 wait complete. 2024-05-08T23:06:17.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 44660 6.57 MB/sec warmup 39 sec latency 13.809 ms 2024-05-08T23:06:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.874+0000 7f724f600700 1 Processor -- start 2024-05-08T23:06:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.874+0000 7f724f600700 1 -- start start 2024-05-08T23:06:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7248101080 0x7f72480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248073d20 0x7f72481a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481a5b00 con 0x7f7248073d20 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481a5c70 con 0x7f7248101080 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481a5e30 con 0x7f72481061a0 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f7247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f7247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38604/0 (socket says 172.21.15.46:38604) 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f7247e00700 1 -- 172.21.15.46:0/3261353206 learned_addr learned my addr 172.21.15.46:0/3261353206 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724d600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248073d20 0x7f72481a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f724cc00700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7248101080 0x7f72480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.875+0000 7f7247e00700 1 -- 172.21.15.46:0/3261353206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7248101080 msgr2=0x7f72480732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7247e00700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7248101080 0x7f72480732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7247e00700 1 -- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248073d20 msgr2=0x7f72481a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7247e00700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248073d20 0x7f72481a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7247e00700 1 -- 172.21.15.46:0/3261353206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340095f0 con 0x7f72481061a0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7247e00700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7234000c00 tx=0x7f72340036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7245600700 1 -- 172.21.15.46:0/3261353206 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723401d070 con 0x7f72481061a0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7245600700 1 -- 172.21.15.46:0/3261353206 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723400fb30 con 0x7f72481061a0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72481a60b0 con 0x7f72481061a0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f7245600700 1 -- 172.21.15.46:0/3261353206 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234022ad0 con 0x7f72481061a0 2024-05-08T23:06:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.876+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72481a6530 con 0x7f72481061a0 2024-05-08T23:06:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.880+0000 7f7245600700 1 -- 172.21.15.46:0/3261353206 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f723400fca0 con 0x7f72481061a0 2024-05-08T23:06:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.881+0000 7f7263101b80 1 Processor -- start 2024-05-08T23:06:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.881+0000 7f7263101b80 1 -- start start 2024-05-08T23:06:17.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a901f150 con 0x5650a90c8890 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a901cb10 con 0x5650a90ce720 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7263101b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a9018400 con 0x5650a90cde00 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38606/0 (socket says 172.21.15.46:38606) 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724d600700 1 -- 172.21.15.46:0/1793480843 learned_addr learned my addr 172.21.15.46:0/1793480843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724cc00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7247e00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7245600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 0x7f72300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f7245600700 1 -- 172.21.15.46:0/3261353206 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f72340ce6f0 con 0x7f72481061a0 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724cc00700 1 -- 172.21.15.46:0/1793480843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 msgr2=0x5650a90ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.882+0000 7f724cc00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f724cc00700 1 -- 172.21.15.46:0/1793480843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 msgr2=0x5650a90c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f724cc00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f724cc00700 1 -- 172.21.15.46:0/1793480843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650a8e993b0 con 0x5650a90ce720 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f7247e00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f724d600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 0x7f72300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f724cc00700 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90c8350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f723c007bf0 tx=0x7f723c007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9454d31048c34fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f7239c00700 1 -- 172.21.15.46:0/1793480843 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c004310 con 0x5650a90ce720 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f7239c00700 1 -- 172.21.15.46:0/1793480843 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723c010e40 con 0x5650a90ce720 2024-05-08T23:06:17.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.883+0000 7f7239c00700 1 -- 172.21.15.46:0/1793480843 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c005020 con 0x5650a90ce720 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 -- 172.21.15.46:0/1793480843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 msgr2=0x5650a90c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90c8350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f723c007bf0 tx=0x7f723c007f00 comp rx=0 tx=0).stop 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f724d600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 0x7f72300a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7240009a60 tx=0x7f7240004ad0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 -- 172.21.15.46:0/1793480843 shutdown_connections 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 --2- 172.21.15.46:0/1793480843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90c8350 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 -- 172.21.15.46:0/1793480843 >> 172.21.15.46:0/1793480843 conn(0x5650a9026210 msgr2=0x5650a90c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 -- 172.21.15.46:0/1793480843 shutdown_connections 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.884+0000 7f7263101b80 1 -- 172.21.15.46:0/1793480843 wait complete. 2024-05-08T23:06:17.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 -- start start 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 Processor -- start 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90cde00 0x5650a90d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a8fb4560 con 0x5650a90cde00 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.885+0000 7f7263101b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a8fb4200 con 0x5650a90ce720 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f7263101b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a8e9b6a0 con 0x5650a90c8890 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f7263101b80 10 client.0 _ll_register_callbacks cb 0x5650a8fbdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38608/0 (socket says 172.21.15.46:38608) 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 -- 172.21.15.46:0/4206531425 learned_addr learned my addr 172.21.15.46:0/4206531425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f7247e00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90cde00 0x5650a90d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724d600700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 -- 172.21.15.46:0/4206531425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 msgr2=0x5650a90d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 -- 172.21.15.46:0/4206531425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90cde00 msgr2=0x5650a90d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90cde00 0x5650a90d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.886+0000 7f724cc00700 1 -- 172.21.15.46:0/4206531425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c0078a0 con 0x5650a90c8890 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f724cc00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f723c007670 tx=0x7f723c0049f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc03dc1b92eefe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c01f070 con 0x5650a90c8890 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f723c004110 con 0x5650a90c8890 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650a8f71270 con 0x5650a90c8890 2024-05-08T23:06:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f723c0042f0 con 0x5650a90c8890 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f7263101b80 10 client.26407 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5650a8e45b60 con 0x5650a90c8890 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650a8f71270 con 0x5650a90c8890 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.887+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f723c011560 con 0x5650a90c8890 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 10 client.26407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 20 client.26407 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a90e2070 0x5650a90e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 4 client.26407 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650a9097810 con 0x5650a90e2070 2024-05-08T23:06:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7247e00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a90e2070 0x5650a90e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f7247e00700 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a90e2070 0x5650a90e4460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b24b664c9f277bb0 server_cookie=e792f00da1540c13 in_seq=0 out_seq=0 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.888+0000 7f722e000700 10 client.26407 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.889+0000 7f722e000700 1 -- 172.21.15.46:0/4206531425 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650a9097810 con 0x5650a90e2070 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.889+0000 7f722e000700 10 client.26407 handle_command_reply: tid=0 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.889+0000 7f7263101b80 1 client.26407 shutdown 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 msgr2=0x5650a90d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f723c007670 tx=0x7f723c0049f0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 shutdown_connections 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a90e2070 0x5650a90e4460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90c8890 0x5650a90d47b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 --2- 172.21.15.46:0/4206531425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90cde00 0x5650a90d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 >> 172.21.15.46:0/4206531425 conn(0x5650a9026210 msgr2=0x5650a90c25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f722e000700 0 client.26407 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 shutdown_connections 2024-05-08T23:06:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 1 -- 172.21.15.46:0/4206531425 wait complete. 2024-05-08T23:06:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.890+0000 7f7263101b80 20 client.26407 trim_cache size 0 max 16384 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.904+0000 7f7263101b80 1 Processor -- start 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.904+0000 7f7263101b80 1 -- start start 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90cffa0 con 0x5650a90c8890 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90e1e00 con 0x5650a90ce720 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7263101b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90deb80 con 0x5650a90cde00 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f724cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38620/0 (socket says 172.21.15.46:38620) 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 -- 172.21.15.46:0/3052476200 learned_addr learned my addr 172.21.15.46:0/3052476200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f724d600700 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 -- 172.21.15.46:0/3052476200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 msgr2=0x5650a90d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 -- 172.21.15.46:0/3052476200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 msgr2=0x5650a90d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f7247e00700 1 -- 172.21.15.46:0/3052476200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650a8fde560 con 0x5650a90cde00 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.905+0000 7f724cc00700 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f7247e00700 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f723409b410 tx=0x7f72340d0b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af5868ceed36910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f723a600700 1 -- 172.21.15.46:0/3052476200 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723409b250 con 0x5650a90cde00 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f723a600700 1 -- 172.21.15.46:0/3052476200 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723409bb80 con 0x5650a90cde00 2024-05-08T23:06:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f723a600700 1 -- 172.21.15.46:0/3052476200 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340995c0 con 0x5650a90cde00 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f7263101b80 1 -- 172.21.15.46:0/3052476200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 msgr2=0x5650a90c2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.906+0000 7f7263101b80 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f723409b410 tx=0x7f72340d0b70 comp rx=0 tx=0).stop 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 -- 172.21.15.46:0/3052476200 shutdown_connections 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90ce720 0x5650a90d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 --2- 172.21.15.46:0/3052476200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90cde00 0x5650a90c2d20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 -- 172.21.15.46:0/3052476200 >> 172.21.15.46:0/3052476200 conn(0x5650a9026210 msgr2=0x5650a90c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 -- 172.21.15.46:0/3052476200 shutdown_connections 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 -- 172.21.15.46:0/3052476200 wait complete. 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 -- start start 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.907+0000 7f7263101b80 1 Processor -- start 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 0x5650a90d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90d7890 con 0x5650a90c8890 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90d7a00 con 0x5650a90cde00 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a90d7bc0 con 0x5650a90ce720 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7263101b80 10 client.0 _ll_register_callbacks cb 0x5650a8fbdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7247e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f724d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 0x5650a90d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f724cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7247e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47838/0 (socket says 172.21.15.46:47838) 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f724d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 0x5650a90d7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38632/0 (socket says 172.21.15.46:38632) 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.908+0000 7f7247e00700 1 -- 172.21.15.46:0/3326083038 learned_addr learned my addr 172.21.15.46:0/3326083038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 -- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 msgr2=0x5650a90d7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 0x5650a90d7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 -- 172.21.15.46:0/3326083038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 msgr2=0x5650a90d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 -- 172.21.15.46:0/3326083038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650a8e45b60 con 0x5650a90cde00 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7247e00700 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f723400b5c0 tx=0x7f7234009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9dc48f1731527f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f722e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340a38a0 con 0x5650a90cde00 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f72340860c0 con 0x5650a90cde00 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f723c010e40 con 0x5650a90cde00 2024-05-08T23:06:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72340ce2a0 con 0x5650a90cde00 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7263101b80 10 client.26369 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.909+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f72340bc040 con 0x5650a90cde00 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723c010e40 con 0x5650a90cde00 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f723409b770 con 0x5650a90cde00 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 10 client.26369 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 10 client.26369 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 20 client.26369 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a9138330 0x5650a913a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 4 client.26369 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5650a9010ec0 con 0x5650a9138330 2024-05-08T23:06:17.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.910+0000 7f724cc00700 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a9138330 0x5650a913a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:17.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f724cc00700 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a9138330 0x5650a913a750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c7f2f5651fe57fb server_cookie=3169c4f2685bbb95 in_seq=0 out_seq=0 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f722e000700 10 client.26369 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f722e000700 1 -- 172.21.15.46:0/3326083038 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650a9010ec0 con 0x5650a9138330 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f722e000700 10 client.26369 handle_command_reply: tid=0 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 client.26369 shutdown 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 msgr2=0x5650a90d96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f723400b5c0 tx=0x7f7234009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 shutdown_connections 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5650a9138330 0x5650a913a750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650a90ce720 0x5650a90d7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650a90c8890 0x5650a90d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f722e000700 0 client.26369 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 --2- 172.21.15.46:0/3326083038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5650a90cde00 0x5650a90d96e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 >> 172.21.15.46:0/3326083038 conn(0x5650a9026210 msgr2=0x5650a90cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 shutdown_connections 2024-05-08T23:06:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 1 -- 172.21.15.46:0/3326083038 wait complete. 2024-05-08T23:06:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.911+0000 7f7263101b80 20 client.26369 trim_cache size 0 max 16384 2024-05-08T23:06:17.915 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 msgr2=0x7f72300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 0x7f72300a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7240009a60 tx=0x7f7240004ad0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 msgr2=0x7f72480737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7234000c00 tx=0x7f72340036f0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 shutdown_connections 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72300a1310 0x7f72300a37d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7248073d20 0x7f72481a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7248101080 0x7f72480732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 --2- 172.21.15.46:0/3261353206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72481061a0 0x7f72480737e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 >> 172.21.15.46:0/3261353206 conn(0x7f72480fc8e0 msgr2=0x7f72480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 shutdown_connections 2024-05-08T23:06:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:17.912+0000 7f724f600700 1 -- 172.21.15.46:0/3261353206 wait complete. 2024-05-08T23:06:17.928 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:17 smithi046 ceph-mon[36898]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:17.928 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:17 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1143765485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:17.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:06:18.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 -- 172.21.15.46:0/3147503689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81010a0 msgr2=0x7f9fb8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 --2- 172.21.15.46:0/3147503689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81010a0 0x7f9fb8101480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009990 tx=0x7f9fa8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 -- 172.21.15.46:0/3147503689 shutdown_connections 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 --2- 172.21.15.46:0/3147503689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81061c0 0x7f9fb810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 --2- 172.21.15.46:0/3147503689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81010a0 0x7f9fb8101480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 --2- 172.21.15.46:0/3147503689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8101a50 0x7f9fb8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 -- 172.21.15.46:0/3147503689 >> 172.21.15.46:0/3147503689 conn(0x7f9fb80fc960 msgr2=0x7f9fb80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 -- 172.21.15.46:0/3147503689 shutdown_connections 2024-05-08T23:06:18.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.154+0000 7f9fbe000700 1 -- 172.21.15.46:0/3147503689 wait complete. 2024-05-08T23:06:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.155+0000 7f9fbe000700 1 Processor -- start 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 -- start start 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 0x7f9fb80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8073d20 0x7f9fb81a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81a5bb0 con 0x7f9fb8073d20 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81a5cf0 con 0x7f9fb81010a0 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fbe000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81a5eb0 con 0x7f9fb81061c0 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 0x7f9fb80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.156+0000 7f9fb7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8073d20 0x7f9fb81a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 0x7f9fb80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47852/0 (socket says 172.21.15.46:47852) 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb7400700 1 -- 172.21.15.46:0/4229762808 learned_addr learned my addr 172.21.15.46:0/4229762808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38652/0 (socket says 172.21.15.46:38652) 2024-05-08T23:06:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 -- 172.21.15.46:0/4229762808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 msgr2=0x7f9fb80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 0x7f9fb80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 -- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8073d20 msgr2=0x7f9fb81a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8073d20 0x7f9fb81a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 -- 172.21.15.46:0/4229762808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80095f0 con 0x7f9fb81061c0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e9c0 tx=0x7f9fa400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fafe00700 1 -- 172.21.15.46:0/4229762808 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400c9c0 con 0x7f9fb81061c0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fafe00700 1 -- 172.21.15.46:0/4229762808 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa4010430 con 0x7f9fb81061c0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fafe00700 1 -- 172.21.15.46:0/4229762808 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4018680 con 0x7f9fb81061c0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.157+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb81a6190 con 0x7f9fb81061c0 2024-05-08T23:06:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.158+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb81a6650 con 0x7f9fb81061c0 2024-05-08T23:06:18.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.161+0000 7f9fafe00700 1 -- 172.21.15.46:0/4229762808 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9fa400cb20 con 0x7f9fb81061c0 2024-05-08T23:06:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.162+0000 7f9fafe00700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 0x7f9f9c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.162+0000 7f9fd1a6eb80 1 Processor -- start 2024-05-08T23:06:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.162+0000 7f9fafe00700 1 -- 172.21.15.46:0/4229762808 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fa4014070 con 0x7f9fb81061c0 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.162+0000 7f9fd1a6eb80 1 -- start start 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.162+0000 7f9fb7400700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 0x7f9f9c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa934271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa93421350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa93421cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93378150 con 0x55aa93421890 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93375b10 con 0x55aa93426e00 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93371400 con 0x55aa93427720 2024-05-08T23:06:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa934271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa934271e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47854/0 (socket says 172.21.15.46:47854) 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fb7400700 1 -- 172.21.15.46:0/205867867 learned_addr learned my addr 172.21.15.46:0/205867867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa93421350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.163+0000 7f9fb7400700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 0x7f9f9c0a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa800b280 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 -- 172.21.15.46:0/205867867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 msgr2=0x55aa934271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa934271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 -- 172.21.15.46:0/205867867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 msgr2=0x55aa93421cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa93421cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 -- 172.21.15.46:0/205867867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa931f23b0 con 0x55aa93427720 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa93421350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ca570 tx=0x7f9fa40963a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6bd78fb27c10ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fa3e00700 1 -- 172.21.15.46:0/205867867 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4099f30 con 0x55aa93427720 2024-05-08T23:06:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fa3e00700 1 -- 172.21.15.46:0/205867867 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa409dd60 con 0x55aa93427720 2024-05-08T23:06:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.164+0000 7f9fa3e00700 1 -- 172.21.15.46:0/205867867 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4010860 con 0x55aa93427720 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/205867867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 msgr2=0x55aa93421350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa93421350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ca570 tx=0x7f9fa40963a0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/205867867 shutdown_connections 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa93421cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa934271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/205867867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa93421350 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/205867867 >> 172.21.15.46:0/205867867 conn(0x55aa9337f210 msgr2=0x55aa9341bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/205867867 shutdown_connections 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.165+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/205867867 wait complete. 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.166+0000 7f9fd1a6eb80 1 -- start start 2024-05-08T23:06:18.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.166+0000 7f9fd1a6eb80 1 Processor -- start 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa9342b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9330d560 con 0x55aa93421890 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9330d200 con 0x55aa93426e00 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa931f46a0 con 0x55aa93427720 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fd1a6eb80 10 client.0 _ll_register_callbacks cb 0x55aa93316b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43588/0 (socket says 172.21.15.46:43588) 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fb7400700 1 -- 172.21.15.46:0/469298046 learned_addr learned my addr 172.21.15.46:0/469298046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fb7e00700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa9342b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.167+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 -- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 msgr2=0x55aa9342b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa9342b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 -- 172.21.15.46:0/469298046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 msgr2=0x55aa9342aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 -- 172.21.15.46:0/469298046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb81a6650 con 0x55aa93421890 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7e00700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa9342b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fb7400700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800be50 tx=0x7f9fa8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59214fc24f72f7bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fa0c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800be80 con 0x55aa93421890 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fa8025470 con 0x55aa93421890 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.168+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa932ca270 con 0x55aa93421890 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fa8025630 con 0x55aa93421890 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fd1a6eb80 10 client.16398 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa9319eb60 con 0x55aa93421890 2024-05-08T23:06:18.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa932ca270 con 0x55aa93421890 2024-05-08T23:06:18.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9fa800f870 con 0x55aa93421890 2024-05-08T23:06:18.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fd1a6eb80 10 client.16398 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fd1a6eb80 10 client.16398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.169+0000 7f9fd1a6eb80 20 client.16398 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa9343b150 0x55aa9343d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fd1a6eb80 4 client.16398 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa933f0810 con 0x55aa9343b150 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa9343b150 0x55aa9343d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fa0c00700 10 client.16398 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.170+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa9343b150 0x55aa9343d570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c186e40ff7b531fa server_cookie=b73acfc38f53e226 in_seq=0 out_seq=0 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fa0c00700 1 -- 172.21.15.46:0/469298046 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa933f0810 con 0x55aa9343b150 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fa0c00700 10 client.16398 handle_command_reply: tid=0 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 client.16398 shutdown 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 msgr2=0x55aa9342aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800be50 tx=0x7f9fa8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 shutdown_connections 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93427720 0x55aa9342b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa9343b150 0x55aa9343d570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93421890 0x55aa9342aab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/469298046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fa0c00700 0 client.16398 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.171+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 >> 172.21.15.46:0/469298046 conn(0x55aa9337f210 msgr2=0x55aa9341b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.172+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 shutdown_connections 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.172+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/469298046 wait complete. 2024-05-08T23:06:18.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.172+0000 7f9fd1a6eb80 20 client.16398 trim_cache size 0 max 16384 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 Processor -- start 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 -- start start 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 0x55aa9341bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93418dc0 con 0x55aa93427720 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9342c120 con 0x55aa93421890 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.188+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93416800 con 0x55aa93426e00 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 0x55aa9341bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38678/0 (socket says 172.21.15.46:38678) 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 -- 172.21.15.46:0/924406538 learned_addr learned my addr 172.21.15.46:0/924406538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb7e00700 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 -- 172.21.15.46:0/924406538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 msgr2=0x55aa9341bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 0x55aa9341bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 -- 172.21.15.46:0/924406538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 msgr2=0x55aa9342b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 -- 172.21.15.46:0/924406538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa93337560 con 0x55aa93426e00 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb7400700 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 0x55aa9341bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.189+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c5b0 tx=0x7f9fa4010c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e927081e5e8c53e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fad200700 1 -- 172.21.15.46:0/924406538 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4099890 con 0x55aa93426e00 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fad200700 1 -- 172.21.15.46:0/924406538 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa4099f30 con 0x55aa93426e00 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fad200700 1 -- 172.21.15.46:0/924406538 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa409a190 con 0x55aa93426e00 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/924406538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 msgr2=0x55aa934262b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c5b0 tx=0x7f9fa4010c30 comp rx=0 tx=0).stop 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/924406538 shutdown_connections 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93421890 0x55aa9341bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/924406538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93426e00 0x55aa934262b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.190+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/924406538 >> 172.21.15.46:0/924406538 conn(0x55aa9337f210 msgr2=0x55aa93425ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.191+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/924406538 shutdown_connections 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.191+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/924406538 wait complete. 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.191+0000 7f9fd1a6eb80 1 -- start start 2024-05-08T23:06:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.191+0000 7f9fd1a6eb80 1 Processor -- start 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9342ef10 con 0x55aa93427720 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93490600 con 0x55aa93426e00 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa93490740 con 0x55aa93421890 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fd1a6eb80 10 client.0 _ll_register_callbacks cb 0x55aa93316b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38694/0 (socket says 172.21.15.46:38694) 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 -- 172.21.15.46:0/3116113229 learned_addr learned my addr 172.21.15.46:0/3116113229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7e00700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 -- 172.21.15.46:0/3116113229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 msgr2=0x55aa9342e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 -- 172.21.15.46:0/3116113229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 msgr2=0x55aa9342e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.192+0000 7f9fb7400700 1 -- 172.21.15.46:0/3116113229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa9319eb60 con 0x55aa93421890 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fb7400700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800b690 tx=0x7f9fa8010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb5852e0fa05e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fa0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8005960 con 0x55aa93421890 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fa409dd60 con 0x55aa93421890 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fa800be80 con 0x55aa93421890 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fa800b2f0 con 0x55aa93421890 2024-05-08T23:06:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fd1a6eb80 10 client.26431 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.193+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa931f23b0 con 0x55aa93421890 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa409dd60 con 0x55aa93421890 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9fa8038070 con 0x55aa93421890 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 10 client.26431 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 10 client.26431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 20 client.26431 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa934919e0 0x55aa93493e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 4 client.26431 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa93369ec0 con 0x55aa934919e0 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.194+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa934919e0 0x55aa93493e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fb6a00700 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa934919e0 0x55aa93493e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f89b0fbf35e6e6f server_cookie=7683b37126e6b51f in_seq=0 out_seq=0 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fa0c00700 10 client.26431 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fa0c00700 1 -- 172.21.15.46:0/3116113229 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa93369ec0 con 0x55aa934919e0 2024-05-08T23:06:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fa0c00700 10 client.26431 handle_command_reply: tid=0 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 client.26431 shutdown 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 msgr2=0x55aa93432600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800b690 tx=0x7f9fa8010ff0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 shutdown_connections 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55aa934919e0 0x55aa93493e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa93427720 0x55aa9342e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa93421890 0x55aa93432600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fa0c00700 0 client.26431 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 --2- 172.21.15.46:0/3116113229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa93426e00 0x55aa9342e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 >> 172.21.15.46:0/3116113229 conn(0x55aa9337f210 msgr2=0x55aa93427b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 shutdown_connections 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.195+0000 7f9fd1a6eb80 1 -- 172.21.15.46:0/3116113229 wait complete. 2024-05-08T23:06:18.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fd1a6eb80 20 client.26431 trim_cache size 0 max 16384 2024-05-08T23:06:18.199 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:18.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 msgr2=0x7f9f9c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 0x7f9f9c0a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa800b280 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 msgr2=0x7f9fb80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e9c0 tx=0x7f9fa400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 shutdown_connections 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9f9c0a1290 0x7f9f9c0a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb8073d20 0x7f9fb81a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81010a0 0x7f9fb80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 --2- 172.21.15.46:0/4229762808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb81061c0 0x7f9fb80737e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.196+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 >> 172.21.15.46:0/4229762808 conn(0x7f9fb80fc960 msgr2=0x7f9fb80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.197+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 shutdown_connections 2024-05-08T23:06:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.197+0000 7f9fbe000700 1 -- 172.21.15.46:0/4229762808 wait complete. 2024-05-08T23:06:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:17 smithi086 ceph-mon[78610]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:17 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1143765485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:18.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:06:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:17 smithi053 ceph-mon[77587]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:17 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1143765485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 -- 172.21.15.46:0/1725125134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88106d30 msgr2=0x7fdd88107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 --2- 172.21.15.46:0/1725125134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88106d30 0x7fdd88107110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009990 tx=0x7fdd7c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 -- 172.21.15.46:0/1725125134 shutdown_connections 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 --2- 172.21.15.46:0/1725125134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068b20 0x7fdd8810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 --2- 172.21.15.46:0/1725125134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88106d30 0x7fdd88107110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 --2- 172.21.15.46:0/1725125134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88068160 0x7fdd880685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.436+0000 7fdd8f200700 1 -- 172.21.15.46:0/1725125134 >> 172.21.15.46:0/1725125134 conn(0x7fdd88075710 msgr2=0x7fdd88075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.437+0000 7fdd8f200700 1 -- 172.21.15.46:0/1725125134 shutdown_connections 2024-05-08T23:06:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.437+0000 7fdd8f200700 1 -- 172.21.15.46:0/1725125134 wait complete. 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 Processor -- start 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 -- start start 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 0x7fdd881003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88106d30 0x7fdd881008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881a53f0 con 0x7fdd88106d30 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881a5560 con 0x7fdd88068b20 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881a5720 con 0x7fdd88068160 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88106d30 0x7fdd881008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38706/0 (socket says 172.21.15.46:38706) 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.438+0000 7fdd8de00700 1 -- 172.21.15.46:0/2510811978 learned_addr learned my addr 172.21.15.46:0/2510811978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8d400700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 0x7fdd881003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 -- 172.21.15.46:0/2510811978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 msgr2=0x7fdd881003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 0x7fdd881003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 -- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88106d30 msgr2=0x7fdd881008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88106d30 0x7fdd881008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 -- 172.21.15.46:0/2510811978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0095f0 con 0x7fdd88068160 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8d400700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 0x7fdd881003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8de00700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c005910 tx=0x7fdd7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd82a00700 1 -- 172.21.15.46:0/2510811978 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c01d070 con 0x7fdd88068160 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd881a59a0 con 0x7fdd88068160 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd82a00700 1 -- 172.21.15.46:0/2510811978 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c022470 con 0x7fdd88068160 2024-05-08T23:06:18.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd82a00700 1 -- 172.21.15.46:0/2510811978 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c017650 con 0x7fdd88068160 2024-05-08T23:06:18.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.439+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd881a5e20 con 0x7fdd88068160 2024-05-08T23:06:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.443+0000 7fdd82a00700 1 -- 172.21.15.46:0/2510811978 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdd7c0225e0 con 0x7fdd88068160 2024-05-08T23:06:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.444+0000 7fdda432ab80 1 Processor -- start 2024-05-08T23:06:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.444+0000 7fdd82a00700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 0x7fdd700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.444+0000 7fdda432ab80 1 -- start start 2024-05-08T23:06:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.444+0000 7fdd82a00700 1 -- 172.21.15.46:0/2510811978 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdd7c0ce370 con 0x7fdd88068160 2024-05-08T23:06:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8e800700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 0x7fdd700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb92c150 con 0x5578eb9d5890 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb929b10 con 0x5578eb9db970 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdda432ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb925400 con 0x5578eb9db050 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38712/0 (socket says 172.21.15.46:38712) 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8de00700 1 -- 172.21.15.46:0/3527065270 learned_addr learned my addr 172.21.15.46:0/3527065270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.445+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 -- 172.21.15.46:0/3527065270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 msgr2=0x5578eb9d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9d5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 -- 172.21.15.46:0/3527065270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 msgr2=0x5578eb9d5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9d5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 -- 172.21.15.46:0/3527065270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578eb7a63b0 con 0x5578eb9db050 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9d5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8e800700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 0x7fdd700a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400f590 tx=0x7fdd84009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c09ffd0 tx=0x7fdd7c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bdd31c7be6c273c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd77000700 1 -- 172.21.15.46:0/3527065270 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c09b980 con 0x5578eb9db050 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd77000700 1 -- 172.21.15.46:0/3527065270 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c09b280 con 0x5578eb9db050 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.446+0000 7fdd77000700 1 -- 172.21.15.46:0/3527065270 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c09b740 con 0x5578eb9db050 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 -- 172.21.15.46:0/3527065270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 msgr2=0x5578eb9db430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c09ffd0 tx=0x7fdd7c022c90 comp rx=0 tx=0).stop 2024-05-08T23:06:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 -- 172.21.15.46:0/3527065270 shutdown_connections 2024-05-08T23:06:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9d5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9db430 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3527065270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 -- 172.21.15.46:0/3527065270 >> 172.21.15.46:0/3527065270 conn(0x5578eb933210 msgr2=0x5578eb9d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 -- 172.21.15.46:0/3527065270 shutdown_connections 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.447+0000 7fdda432ab80 1 -- 172.21.15.46:0/3527065270 wait complete. 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.448+0000 7fdda432ab80 1 -- start start 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.448+0000 7fdda432ab80 1 Processor -- start 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9df220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb8c1560 con 0x5578eb9d5890 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb8c1200 con 0x5578eb9db970 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb7a86a0 con 0x5578eb9db050 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdda432ab80 10 client.0 _ll_register_callbacks cb 0x5578eb8cab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43642/0 (socket says 172.21.15.46:43642) 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8de00700 1 -- 172.21.15.46:0/3021900406 learned_addr learned my addr 172.21.15.46:0/3021900406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.449+0000 7fdd8e800700 1 -- 172.21.15.46:0/3021900406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 msgr2=0x5578eb9df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8e800700 1 -- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 msgr2=0x5578eb9dece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8e800700 1 -- 172.21.15.46:0/3021900406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd881a5e20 con 0x5578eb9db970 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9df760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd84019e30 tx=0x7fdd84004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ae822e3d102541 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd6b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:06:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e040 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdd8400fe20 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578eb87e270 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.450+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd84004bb0 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 10 client.26399 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578eb752b60 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578eb87e270 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdd84004280 con 0x5578eb9db970 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 10 client.26399 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 10 client.26399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:06:18.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 20 client.26399 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eb9ef100 0x5578eb9f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 4 client.26399 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.451+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578eb9a4810 con 0x5578eb9ef100 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.452+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eb9ef100 0x5578eb9f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.452+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eb9ef100 0x5578eb9f1520 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f01beda032af58ad server_cookie=e0acfbc576da4e8a in_seq=0 out_seq=0 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.452+0000 7fdd6b400700 10 client.26399 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdd6b400700 1 -- 172.21.15.46:0/3021900406 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578eb9a4810 con 0x5578eb9ef100 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdd6b400700 10 client.26399 handle_command_reply: tid=0 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 client.26399 shutdown 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 msgr2=0x5578eb9df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9df760 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd84019e30 tx=0x7fdd84004b40 comp rx=0 tx=0).stop 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 shutdown_connections 2024-05-08T23:06:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db970 0x5578eb9df760 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eb9ef100 0x5578eb9f1520 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3021900406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db050 0x5578eb9df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 >> 172.21.15.46:0/3021900406 conn(0x5578eb933210 msgr2=0x5578eb9d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdd6b400700 0 client.26399 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 shutdown_connections 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.453+0000 7fdda432ab80 1 -- 172.21.15.46:0/3021900406 wait complete. 2024-05-08T23:06:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.454+0000 7fdda432ab80 20 client.26399 trim_cache size 0 max 16384 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.470+0000 7fdda432ab80 1 Processor -- start 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 -- start start 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db050 0x5578eb9f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb9d6330 con 0x5578eb9d5890 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb9dd250 con 0x5578eb9db050 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdda432ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eb9d07c0 con 0x5578eb9db970 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdd8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdd8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38726/0 (socket says 172.21.15.46:38726) 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdd8e800700 1 -- 172.21.15.46:0/3675762255 learned_addr learned my addr 172.21.15.46:0/3675762255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.471+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db050 0x5578eb9f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 -- 172.21.15.46:0/3675762255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db050 msgr2=0x5578eb9f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db050 0x5578eb9f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 -- 172.21.15.46:0/3675762255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 msgr2=0x5578eb9df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 -- 172.21.15.46:0/3675762255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578eb8eb560 con 0x5578eb9db970 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd84000f80 tx=0x7fdd84017720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a019160d1fd6f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd77a00700 1 -- 172.21.15.46:0/3675762255 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e030 con 0x5578eb9db970 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd77a00700 1 -- 172.21.15.46:0/3675762255 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8400f040 con 0x5578eb9db970 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.472+0000 7fdd77a00700 1 -- 172.21.15.46:0/3675762255 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84022850 con 0x5578eb9db970 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 -- 172.21.15.46:0/3675762255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 msgr2=0x5578eb9d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd84000f80 tx=0x7fdd84017720 comp rx=0 tx=0).stop 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 -- 172.21.15.46:0/3675762255 shutdown_connections 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eb9d8d60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9d5890 0x5578eb9df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3675762255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9db050 0x5578eb9f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 -- 172.21.15.46:0/3675762255 >> 172.21.15.46:0/3675762255 conn(0x5578eb933210 msgr2=0x5578eb9d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 -- 172.21.15.46:0/3675762255 shutdown_connections 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.473+0000 7fdda432ab80 1 -- 172.21.15.46:0/3675762255 wait complete. 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 -- start start 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 Processor -- start 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 0x5578eb9e2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 0x5578eb9e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eba445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eba44af0 con 0x5578eb9db050 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eba44c30 con 0x5578eb9d5890 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eba44d70 con 0x5578eb9db970 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.474+0000 7fdda432ab80 10 client.0 _ll_register_callbacks cb 0x5578eb8cab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 0x5578eb9e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 0x5578eb9e2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47924/0 (socket says 172.21.15.46:47924) 2024-05-08T23:06:18.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8de00700 1 -- 172.21.15.46:0/3725456614 learned_addr learned my addr 172.21.15.46:0/3725456614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 0x5578eb9e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eba445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 -- 172.21.15.46:0/3725456614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 msgr2=0x5578eb9e2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 0x5578eb9e2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 -- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 msgr2=0x5578eb9e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 0x5578eb9e2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 -- 172.21.15.46:0/3725456614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578eb7a63b0 con 0x5578eb9db970 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8d400700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 0x5578eb9e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd8e800700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eba445b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd840046e0 tx=0x7fdd84004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d540f87f63f47e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.475+0000 7fdd6b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e070 con 0x5578eb9db970 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd7c09b280 con 0x5578eb9db970 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdd84004c80 con 0x5578eb9db970 2024-05-08T23:06:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd84017cf0 con 0x5578eb9db970 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 10 client.26449 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578eb752b60 con 0x5578eb9db970 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c09b280 con 0x5578eb9db970 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdd8403a020 con 0x5578eb9db970 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 10 client.26449 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 10 client.26449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.476+0000 7fdda432ab80 20 client.26449 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eba46040 0x5578eba48430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdda432ab80 4 client.26449 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578eb91dec0 con 0x5578eba46040 2024-05-08T23:06:18.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eba46040 0x5578eba48430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdd8de00700 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eba46040 0x5578eba48430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29a0d164d3a3a28c server_cookie=334b486bd9d2eb22 in_seq=0 out_seq=0 2024-05-08T23:06:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.477+0000 7fdd6b400700 10 client.26449 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdd6b400700 1 -- 172.21.15.46:0/3725456614 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578eb91dec0 con 0x5578eba46040 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdd6b400700 10 client.26449 handle_command_reply: tid=0 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 client.26449 shutdown 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 msgr2=0x5578eba445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eba445b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd840046e0 tx=0x7fdd84004a90 comp rx=0 tx=0).stop 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 shutdown_connections 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578eb9db970 0x5578eba445b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578eb9d5890 0x5578eb9e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5578eba46040 0x5578eba48430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 --2- 172.21.15.46:0/3725456614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578eb9db050 0x5578eb9e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdd6b400700 0 client.26449 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 >> 172.21.15.46:0/3725456614 conn(0x5578eb933210 msgr2=0x5578eb9f5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 shutdown_connections 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 1 -- 172.21.15.46:0/3725456614 wait complete. 2024-05-08T23:06:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.478+0000 7fdda432ab80 20 client.26449 trim_cache size 0 max 16384 2024-05-08T23:06:18.482 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 msgr2=0x7fdd700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 0x7fdd700a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400f590 tx=0x7fdd84009920 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 msgr2=0x7fdd880ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c005910 tx=0x7fdd7c004730 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 shutdown_connections 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdd700a1330 0x7fdd700a37f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd88106d30 0x7fdd881008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd88068160 0x7fdd880ffe60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 --2- 172.21.15.46:0/2510811978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd88068b20 0x7fdd881003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 >> 172.21.15.46:0/2510811978 conn(0x7fdd88075710 msgr2=0x7fdd880fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 shutdown_connections 2024-05-08T23:06:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.479+0000 7fdd8f200700 1 -- 172.21.15.46:0/2510811978 wait complete. 2024-05-08T23:06:18.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:06:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 -- 172.21.15.46:0/1266619700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb241028c0 msgr2=0x7fcb2410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 --2- 172.21.15.46:0/1266619700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb241028c0 0x7fcb2410ada0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009990 tx=0x7fcb10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 -- 172.21.15.46:0/1266619700 shutdown_connections 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 --2- 172.21.15.46:0/1266619700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 0x7fcb2410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 --2- 172.21.15.46:0/1266619700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb24101fa0 0x7fcb24102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 --2- 172.21.15.46:0/1266619700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb241028c0 0x7fcb2410ada0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 -- 172.21.15.46:0/1266619700 >> 172.21.15.46:0/1266619700 conn(0x7fcb240fb760 msgr2=0x7fcb240fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 -- 172.21.15.46:0/1266619700 shutdown_connections 2024-05-08T23:06:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.724+0000 7fcb29e00700 1 -- 172.21.15.46:0/1266619700 wait complete. 2024-05-08T23:06:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.725+0000 7fcb29e00700 1 Processor -- start 2024-05-08T23:06:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.725+0000 7fcb29e00700 1 -- start start 2024-05-08T23:06:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb241028c0 0x7fcb241977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 0x7fcb2419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c1d0 con 0x7fcb241028c0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c340 con 0x7fcb2410b3c0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb29e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c500 con 0x7fcb24101fa0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38756/0 (socket says 172.21.15.46:38756) 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 -- 172.21.15.46:0/2588614981 learned_addr learned my addr 172.21.15.46:0/2588614981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22000700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb241028c0 0x7fcb241977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb23400700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 0x7fcb2419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 -- 172.21.15.46:0/2588614981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 msgr2=0x7fcb2419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 0x7fcb2419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 -- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb241028c0 msgr2=0x7fcb241977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb241028c0 0x7fcb241977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.726+0000 7fcb22a00700 1 -- 172.21.15.46:0/2588614981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100095f0 con 0x7fcb24101fa0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb22a00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb28a00700 1 -- 172.21.15.46:0/2588614981 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14009790 con 0x7fcb24101fa0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb28a00700 1 -- 172.21.15.46:0/2588614981 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb14010430 con 0x7fcb24101fa0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb28a00700 1 -- 172.21.15.46:0/2588614981 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018680 con 0x7fcb24101fa0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2419c7e0 con 0x7fcb24101fa0 2024-05-08T23:06:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.727+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb241a4020 con 0x7fcb24101fa0 2024-05-08T23:06:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.730+0000 7fcb28a00700 1 -- 172.21.15.46:0/2588614981 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb140105a0 con 0x7fcb24101fa0 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 Processor -- start 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 -- start start 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb28a00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 0x7fcb080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb28a00700 1 -- 172.21.15.46:0/2588614981 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fcb14014070 con 0x7fcb24101fa0 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a316980760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 0x55a3169810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3168d3150 con 0x55a316980ca0 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb23400700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 0x7fcb080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3168d0b10 con 0x55a316986210 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3168cc400 con 0x55a316986b30 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.732+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47962/0 (socket says 172.21.15.46:47962) 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 -- 172.21.15.46:0/1385114863 learned_addr learned my addr 172.21.15.46:0/1385114863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb23400700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 0x55a3169810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22000700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a316980760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 -- 172.21.15.46:0/1385114863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 msgr2=0x55a316980760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a316980760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 -- 172.21.15.46:0/1385114863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 msgr2=0x55a3169810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 0x55a3169810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 -- 172.21.15.46:0/1385114863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a31674d3b0 con 0x55a316986210 2024-05-08T23:06:18.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb23400700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 0x55a3169810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb23400700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 0x7fcb080a37d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00f870 tx=0x7fcb1c00fc30 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb22a00700 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb140969b0 tx=0x7fcb14099bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f5522819f68869 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.733+0000 7fcb0fe00700 1 -- 172.21.15.46:0/1385114863 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb140a23c0 con 0x55a316986210 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb0fe00700 1 -- 172.21.15.46:0/1385114863 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb14010880 con 0x55a316986210 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb0fe00700 1 -- 172.21.15.46:0/1385114863 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14099630 con 0x55a316986210 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/1385114863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 msgr2=0x55a3169865f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb140969b0 tx=0x7fcb14099bb0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/1385114863 shutdown_connections 2024-05-08T23:06:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316980ca0 0x55a3169810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316986210 0x55a3169865f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/1385114863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a316980760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.734+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/1385114863 >> 172.21.15.46:0/1385114863 conn(0x55a3168da210 msgr2=0x55a316976df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.735+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/1385114863 shutdown_connections 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.735+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/1385114863 wait complete. 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.735+0000 7fcb3d8d2b80 1 -- start start 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.735+0000 7fcb3d8d2b80 1 Processor -- start 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316989f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986210 0x55a31698a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316868560 con 0x55a316986210 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316868200 con 0x55a316980ca0 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31674f6a0 con 0x55a316986b30 2024-05-08T23:06:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb3d8d2b80 10 client.0 _ll_register_callbacks cb 0x55a316871b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb22000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986210 0x55a31698a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316989f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38776/0 (socket says 172.21.15.46:38776) 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316989f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47974/0 (socket says 172.21.15.46:47974) 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb23400700 1 -- 172.21.15.46:0/3770486941 learned_addr learned my addr 172.21.15.46:0/3770486941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.736+0000 7fcb23400700 1 -- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 msgr2=0x55a316989f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb23400700 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316989f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb23400700 1 -- 172.21.15.46:0/3770486941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986210 msgr2=0x55a31698a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb23400700 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986210 0x55a31698a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb23400700 1 -- 172.21.15.46:0/3770486941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb241a4020 con 0x55a316986b30 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb23400700 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c018b00 tx=0x7fcb1c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6dafad429c077f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00e040 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a316825270 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fcb1c003a00 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb1c003bc0 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb3d8d2b80 10 client.26461 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.737+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a3166f9b60 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a316825270 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb1c0042f0 con 0x55a316986b30 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 10 client.26461 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 20 client.26461 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a31699a6a0 0x55a31699cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 4 client.26461 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a31694b810 con 0x55a31699a6a0 2024-05-08T23:06:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.738+0000 7fcb22a00700 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a31699a6a0 0x55a31699cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.739+0000 7fcb22a00700 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a31699a6a0 0x55a31699cac0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1adc8ed4a4c20b server_cookie=f4a596e074164b7a in_seq=0 out_seq=0 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.739+0000 7fcb0cc00700 10 client.26461 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.739+0000 7fcb0cc00700 1 -- 172.21.15.46:0/3770486941 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a31694b810 con 0x55a31699a6a0 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.739+0000 7fcb0cc00700 10 client.26461 handle_command_reply: tid=0 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 client.26461 shutdown 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 msgr2=0x55a31698a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c018b00 tx=0x7fcb1c004b40 comp rx=0 tx=0).stop 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 shutdown_connections 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986b30 0x55a31698a990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316989f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a31699a6a0 0x55a31699cac0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/3770486941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986210 0x55a31698a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 >> 172.21.15.46:0/3770486941 conn(0x55a3168da210 msgr2=0x55a3169765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb0cc00700 0 client.26461 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 shutdown_connections 2024-05-08T23:06:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/3770486941 wait complete. 2024-05-08T23:06:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.740+0000 7fcb3d8d2b80 20 client.26461 trim_cache size 0 max 16384 2024-05-08T23:06:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 Processor -- start 2024-05-08T23:06:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 -- start start 2024-05-08T23:06:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a316976d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316997180 con 0x55a316986b30 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316989d20 con 0x55a316980ca0 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a316973dc0 con 0x55a316986210 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb23400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a316976d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38782/0 (socket says 172.21.15.46:38782) 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47976/0 (socket says 172.21.15.46:47976) 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.756+0000 7fcb22000700 1 -- 172.21.15.46:0/907209128 learned_addr learned my addr 172.21.15.46:0/907209128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 -- 172.21.15.46:0/907209128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 msgr2=0x55a316980760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 -- 172.21.15.46:0/907209128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 msgr2=0x55a316976d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a316976d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 -- 172.21.15.46:0/907209128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a316892560 con 0x55a316986210 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb23400700 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a316976d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22a00700 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb22000700 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009990 tx=0x7fcb1000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14b20f1f196c320 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb19200700 1 -- 172.21.15.46:0/907209128 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1001d030 con 0x55a316986210 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb19200700 1 -- 172.21.15.46:0/907209128 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1001e040 con 0x55a316986210 2024-05-08T23:06:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.757+0000 7fcb19200700 1 -- 172.21.15.46:0/907209128 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10004880 con 0x55a316986210 2024-05-08T23:06:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/907209128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 msgr2=0x55a3169966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009990 tx=0x7fcb1000b260 comp rx=0 tx=0).stop 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/907209128 shutdown_connections 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a316976d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a316980760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/907209128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a3169966d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/907209128 >> 172.21.15.46:0/907209128 conn(0x55a3168da210 msgr2=0x55a316981080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/907209128 shutdown_connections 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.758+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/907209128 wait complete. 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 -- start start 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 Processor -- start 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a31698d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a31698dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3169efb90 con 0x55a316986b30 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3169efcd0 con 0x55a316980ca0 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3169efe40 con 0x55a316986210 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.759+0000 7fcb3d8d2b80 10 client.0 _ll_register_callbacks cb 0x55a316871b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38798/0 (socket says 172.21.15.46:38798) 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 -- 172.21.15.46:0/4090230329 learned_addr learned my addr 172.21.15.46:0/4090230329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22a00700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a31698d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb23400700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a31698dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 -- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 msgr2=0x55a31698d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a31698d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 -- 172.21.15.46:0/4090230329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 msgr2=0x55a31698dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a31698dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 -- 172.21.15.46:0/4090230329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a31674d3b0 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb22000700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098e0 tx=0x7fcb1000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7461bb6ba729ef32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.760+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1001d070 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb14010880 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fcb10004910 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb10009290 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 10 client.26467 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a3166f9b60 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb14010880 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb10023040 con 0x55a316986210 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 20 client.26467 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a3169f1290 0x55a3169f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 4 client.26467 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.761+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3168c4ec0 con 0x55a3169f1290 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb22a00700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a3169f1290 0x55a3169f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb22a00700 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a3169f1290 0x55a3169f36b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ce66c60c42b7e8 server_cookie=97520913385e00e6 in_seq=0 out_seq=0 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb0cc00700 10 client.26467 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb0cc00700 1 -- 172.21.15.46:0/4090230329 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3168c4ec0 con 0x55a3169f1290 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb0cc00700 10 client.26467 handle_command_reply: tid=0 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.762+0000 7fcb3d8d2b80 1 client.26467 shutdown 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 msgr2=0x55a31698da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098e0 tx=0x7fcb1000b9e0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 shutdown_connections 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a3169f1290 0x55a3169f36b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a316986b30 0x55a31698dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a316980ca0 0x55a31698d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb0cc00700 0 client.26467 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 --2- 172.21.15.46:0/4090230329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a316986210 0x55a31698da90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 >> 172.21.15.46:0/4090230329 conn(0x55a3168da210 msgr2=0x55a3169966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 shutdown_connections 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 1 -- 172.21.15.46:0/4090230329 wait complete. 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.763+0000 7fcb3d8d2b80 20 client.26467 trim_cache size 0 max 16384 2024-05-08T23:06:18.767 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 msgr2=0x7fcb080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 0x7fcb080a37d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00f870 tx=0x7fcb1c00fc30 comp rx=0 tx=0).stop 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 msgr2=0x7fcb241972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 shutdown_connections 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb2410b3c0 0x7fcb2419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb24101fa0 0x7fcb241972b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcb080a1310 0x7fcb080a37d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 --2- 172.21.15.46:0/2588614981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb241028c0 0x7fcb241977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 >> 172.21.15.46:0/2588614981 conn(0x7fcb240fb760 msgr2=0x7fcb240fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 shutdown_connections 2024-05-08T23:06:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:18.764+0000 7fcb29e00700 1 -- 172.21.15.46:0/2588614981 wait complete. 2024-05-08T23:06:18.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 46647 6.74 MB/sec warmup 40 sec latency 13.054 ms 2024-05-08T23:06:19.784 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:06:19.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:06:19.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 49312 6.90 MB/sec warmup 41 sec latency 19.392 ms 2024-05-08T23:06:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:19 smithi046 ceph-mon[36898]: pgmap v78: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 -- 172.21.15.46:0/789400237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241032b0 msgr2=0x7f1924103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 --2- 172.21.15.46:0/789400237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241032b0 0x7f1924103690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1918009990 tx=0x7f1918009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 -- 172.21.15.46:0/789400237 shutdown_connections 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 --2- 172.21.15.46:0/789400237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19241083d0 0x7f192410ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 --2- 172.21.15.46:0/789400237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241032b0 0x7f1924103690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 --2- 172.21.15.46:0/789400237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1924103c60 0x7f1924107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.043+0000 7f192c400700 1 -- 172.21.15.46:0/789400237 >> 172.21.15.46:0/789400237 conn(0x7f19240feb10 msgr2=0x7f1924100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.044+0000 7f192c400700 1 -- 172.21.15.46:0/789400237 shutdown_connections 2024-05-08T23:06:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.044+0000 7f192c400700 1 -- 172.21.15.46:0/789400237 wait complete. 2024-05-08T23:06:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 Processor -- start 2024-05-08T23:06:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 -- start start 2024-05-08T23:06:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 0x7f1924199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 0x7f1924199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 0x7f192419e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192419e8d0 con 0x7f192419de50 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192419ea40 con 0x7f19241032b0 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.045+0000 7f192c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192419ec00 con 0x7f19241083d0 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 0x7f1924199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 0x7f1924199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 0x7f1924199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38820/0 (socket says 172.21.15.46:38820) 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929000700 1 -- 172.21.15.46:0/204440784 learned_addr learned my addr 172.21.15.46:0/204440784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f192a400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 0x7f192419e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 -- 172.21.15.46:0/204440784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 msgr2=0x7f1924199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 0x7f1924199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 -- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 msgr2=0x7f192419e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 0x7f192419e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 -- 172.21.15.46:0/204440784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180095f0 con 0x7f19241032b0 2024-05-08T23:06:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f192a400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 0x7f192419e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.046+0000 7f1929a00700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 0x7f1924199570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19180098e0 tx=0x7f1918005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.047+0000 7f1922000700 1 -- 172.21.15.46:0/204440784 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191801d070 con 0x7f19241032b0 2024-05-08T23:06:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.047+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19241a6100 con 0x7f19241032b0 2024-05-08T23:06:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.047+0000 7f1922000700 1 -- 172.21.15.46:0/204440784 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1918022470 con 0x7f19241032b0 2024-05-08T23:06:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.047+0000 7f1922000700 1 -- 172.21.15.46:0/204440784 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918017650 con 0x7f19241032b0 2024-05-08T23:06:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.047+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19241a6530 con 0x7f19241032b0 2024-05-08T23:06:20.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.051+0000 7f1922000700 1 -- 172.21.15.46:0/204440784 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f19180225e0 con 0x7f19241032b0 2024-05-08T23:06:20.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.052+0000 7f193ff82b80 1 Processor -- start 2024-05-08T23:06:20.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.052+0000 7f1922000700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 0x7f190c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.052+0000 7f1922000700 1 -- 172.21.15.46:0/204440784 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f19180ce350 con 0x7f19241032b0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 -- start start 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f192a400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 0x7f190c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed27a0 0x563462ecc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462eccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462e231d0 con 0x563462ed1e80 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462e20b90 con 0x563462ed27a0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f193ff82b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462e1c480 con 0x563462ecc910 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f1929a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f1929a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43736/0 (socket says 172.21.15.46:43736) 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f192a400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 0x7f190c0a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f191c003e30 tx=0x7f191c00b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.053+0000 7f1929a00700 1 -- 172.21.15.46:0/1732109850 learned_addr learned my addr 172.21.15.46:0/1732109850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f192a400700 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462eccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929000700 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed27a0 0x563462ecc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 -- 172.21.15.46:0/1732109850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 msgr2=0x563462eccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462eccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 -- 172.21.15.46:0/1732109850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed27a0 msgr2=0x563462ecc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed27a0 0x563462ecc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 -- 172.21.15.46:0/1732109850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563462c9d430 con 0x563462ed1e80 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1929a00700 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1918097c70 tx=0x7f191809afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9d34cfc1797d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1916a00700 1 -- 172.21.15.46:0/1732109850 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191809b900 con 0x563462ed1e80 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1916a00700 1 -- 172.21.15.46:0/1732109850 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f191809b200 con 0x563462ed1e80 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.054+0000 7f1916a00700 1 -- 172.21.15.46:0/1732109850 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180a10c0 con 0x563462ed1e80 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 -- 172.21.15.46:0/1732109850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 msgr2=0x563462ed2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1918097c70 tx=0x7f191809afd0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 -- 172.21.15.46:0/1732109850 shutdown_connections 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462eccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed1e80 0x563462ed2260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 --2- 172.21.15.46:0/1732109850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed27a0 0x563462ecc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 -- 172.21.15.46:0/1732109850 >> 172.21.15.46:0/1732109850 conn(0x563462e2a290 msgr2=0x563462ec6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 -- 172.21.15.46:0/1732109850 shutdown_connections 2024-05-08T23:06:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.055+0000 7f193ff82b80 1 -- 172.21.15.46:0/1732109850 wait complete. 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.056+0000 7f193ff82b80 1 -- start start 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.056+0000 7f193ff82b80 1 Processor -- start 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462ed5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462ed6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462db85e0 con 0x563462ed27a0 2024-05-08T23:06:20.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462db8280 con 0x563462ecc910 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462c9f720 con 0x563462ed1e80 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f193ff82b80 10 client.0 _ll_register_callbacks cb 0x563462dc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f1929a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f1929a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f1929a00700 1 -- 172.21.15.46:0/4240812338 learned_addr learned my addr 172.21.15.46:0/4240812338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.057+0000 7f192a400700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462ed6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929000700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462ed5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 -- 172.21.15.46:0/4240812338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 msgr2=0x563462ed5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462ed5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 -- 172.21.15.46:0/4240812338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 msgr2=0x563462ed6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462ed6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 -- 172.21.15.46:0/4240812338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19241a6530 con 0x563462ecc910 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f192a400700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462ed6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f1929a00700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1918009350 tx=0x7f191809b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2df4a0a57723fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.058+0000 7f190b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180995a0 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f191809b670 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563462d752f0 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191809a6f0 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f193ff82b80 10 client.26435 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563462c49be0 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1918080020 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.059+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f19180b3d20 con 0x563462ecc910 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 10 client.26435 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 20 client.26435 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462ee5a80 0x563462ee7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 4 client.26435 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563462e9b890 con 0x563462ee5a80 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f1929000700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462ee5a80 0x563462ee7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f1929000700 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462ee5a80 0x563462ee7ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238d8b43b5303f6 server_cookie=f46cd2e9576640da in_seq=0 out_seq=0 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.060+0000 7f190b400700 10 client.26435 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.061+0000 7f190b400700 1 -- 172.21.15.46:0/4240812338 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563462e9b890 con 0x563462ee5a80 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.061+0000 7f190b400700 10 client.26435 handle_command_reply: tid=0 2024-05-08T23:06:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.061+0000 7f193ff82b80 1 client.26435 shutdown 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.061+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 msgr2=0x563462ed8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1918009350 tx=0x7f191809b970 comp rx=0 tx=0).stop 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 shutdown_connections 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462ee5a80 0x563462ee7ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462ed6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462ed8790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4240812338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462ed5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f190b400700 0 client.26435 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 >> 172.21.15.46:0/4240812338 conn(0x563462e2a290 msgr2=0x563462ec6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 shutdown_connections 2024-05-08T23:06:20.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 1 -- 172.21.15.46:0/4240812338 wait complete. 2024-05-08T23:06:20.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.062+0000 7f193ff82b80 20 client.26435 trim_cache size 0 max 16384 2024-05-08T23:06:20.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 Processor -- start 2024-05-08T23:06:20.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 -- start start 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed1e80 0x563462edcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462ec8810 con 0x563462ed27a0 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462ec3e40 con 0x563462ed1e80 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.082+0000 7f193ff82b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462ed6a70 con 0x563462ecc910 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f192a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38858/0 (socket says 172.21.15.46:38858) 2024-05-08T23:06:20.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 -- 172.21.15.46:0/4248408947 learned_addr learned my addr 172.21.15.46:0/4248408947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929000700 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed1e80 0x563462edcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 -- 172.21.15.46:0/4248408947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed1e80 msgr2=0x563462edcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed1e80 0x563462edcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 -- 172.21.15.46:0/4248408947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 msgr2=0x563462edd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 -- 172.21.15.46:0/4248408947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563462ec3560 con 0x563462ecc910 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f192a400700 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.083+0000 7f1929a00700 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1918097c70 tx=0x7f19180b31d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=438d0d8bb4b598b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f1917400700 1 -- 172.21.15.46:0/4248408947 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180a11d0 con 0x563462ecc910 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f1917400700 1 -- 172.21.15.46:0/4248408947 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1918080070 con 0x563462ecc910 2024-05-08T23:06:20.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f1917400700 1 -- 172.21.15.46:0/4248408947 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180a3160 con 0x563462ecc910 2024-05-08T23:06:20.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 -- 172.21.15.46:0/4248408947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 msgr2=0x563462ec6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1918097c70 tx=0x7f19180b31d0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 -- 172.21.15.46:0/4248408947 shutdown_connections 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ecc910 0x563462ec6da0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 --2- 172.21.15.46:0/4248408947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ed1e80 0x563462edcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.084+0000 7f193ff82b80 1 -- 172.21.15.46:0/4248408947 >> 172.21.15.46:0/4248408947 conn(0x563462e2a290 msgr2=0x563462ed43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.085+0000 7f193ff82b80 1 -- 172.21.15.46:0/4248408947 shutdown_connections 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.085+0000 7f193ff82b80 1 -- 172.21.15.46:0/4248408947 wait complete. 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.085+0000 7f193ff82b80 1 -- start start 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.085+0000 7f193ff82b80 1 Processor -- start 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462edd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462edb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462edc350 con 0x563462ed27a0 2024-05-08T23:06:20.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462edc4c0 con 0x563462ecc910 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563462f3bf90 con 0x563462ed1e80 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f193ff82b80 10 client.0 _ll_register_callbacks cb 0x563462dc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f192a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f1929a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462edd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f192a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43762/0 (socket says 172.21.15.46:43762) 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f192a400700 1 -- 172.21.15.46:0/315536175 learned_addr learned my addr 172.21.15.46:0/315536175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.086+0000 7f1929000700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462edb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 -- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 msgr2=0x563462edd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462edd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 -- 172.21.15.46:0/315536175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 msgr2=0x563462edbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 -- 172.21.15.46:0/315536175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c00e800 con 0x563462ed1e80 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f192a400700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f1929000700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462edb730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563462ee8750 tx=0x7f191001f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc81c50d255c5114 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f190b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910010aa0 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f191809b200 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1910010c00 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1910010dc0 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f193ff82b80 10 client.26485 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.087+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563462c49be0 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f191809b200 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f19100263f0 con 0x563462ed1e80 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 10 client.26485 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 20 client.26485 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:20.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462f3d6d0 0x563462f3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 4 client.26485 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563462e14f40 con 0x563462f3d6d0 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.088+0000 7f1929a00700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462f3d6d0 0x563462f3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.089+0000 7f1929a00700 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462f3d6d0 0x563462f3faf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dc909e07c4023d server_cookie=af80bc6c9ba4be13 in_seq=0 out_seq=0 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.089+0000 7f190b400700 10 client.26485 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f190b400700 1 -- 172.21.15.46:0/315536175 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563462e14f40 con 0x563462f3d6d0 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f190b400700 10 client.26485 handle_command_reply: tid=0 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 client.26485 shutdown 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 msgr2=0x563462edb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462edb730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563462ee8750 tx=0x7f191001f680 comp rx=0 tx=0).stop 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 shutdown_connections 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563462f3d6d0 0x563462f3faf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563462ed27a0 0x563462edbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563462ecc910 0x563462edd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 --2- 172.21.15.46:0/315536175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563462ed1e80 0x563462edb730 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f190b400700 0 client.26485 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.090+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 >> 172.21.15.46:0/315536175 conn(0x563462e2a290 msgr2=0x563462ed5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 shutdown_connections 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f193ff82b80 1 -- 172.21.15.46:0/315536175 wait complete. 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f193ff82b80 20 client.26485 trim_cache size 0 max 16384 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:06:20.096 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "2381fd82-73ba-4a1d-a168-5abfc987a00c", 2024-05-08T23:06:20.097 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:06:20.097 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:20.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 msgr2=0x7f190c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 0x7f190c0a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f191c003e30 tx=0x7f191c00b040 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 msgr2=0x7f1924199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.091+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 0x7f1924199570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19180098e0 tx=0x7f1918005740 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 shutdown_connections 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f190c0a1290 0x7f190c0a3750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192419de50 0x7f192419e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19241032b0 0x7f1924199570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 --2- 172.21.15.46:0/204440784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19241083d0 0x7f1924199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 >> 172.21.15.46:0/204440784 conn(0x7f19240feb10 msgr2=0x7f19240ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 shutdown_connections 2024-05-08T23:06:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.092+0000 7f192c400700 1 -- 172.21.15.46:0/204440784 wait complete. 2024-05-08T23:06:20.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:06:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:19 smithi086 ceph-mon[78610]: pgmap v78: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T23:06:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:19 smithi053 ceph-mon[77587]: pgmap v78: 129 pgs: 129 active+clean; 42 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T23:06:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 -- 172.21.15.46:0/2969225151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724101a30 msgr2=0x7fe724105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 --2- 172.21.15.46:0/2969225151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724101a30 0x7fe724105a70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe710009990 tx=0x7fe710009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 -- 172.21.15.46:0/2969225151 shutdown_connections 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 --2- 172.21.15.46:0/2969225151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7241061a0 0x7fe72410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 --2- 172.21.15.46:0/2969225151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 0x7fe724101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 --2- 172.21.15.46:0/2969225151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724101a30 0x7fe724105a70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 -- 172.21.15.46:0/2969225151 >> 172.21.15.46:0/2969225151 conn(0x7fe7240fc8e0 msgr2=0x7fe7240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 -- 172.21.15.46:0/2969225151 shutdown_connections 2024-05-08T23:06:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.330+0000 7fe72be00700 1 -- 172.21.15.46:0/2969225151 wait complete. 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.331+0000 7fe72be00700 1 Processor -- start 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 -- start start 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 0x7fe7240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 0x7fe7241a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241a5ac0 con 0x7fe724101080 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241a5c30 con 0x7fe724073d20 2024-05-08T23:06:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe72be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241a5df0 con 0x7fe7241061a0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe728a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe728a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38906/0 (socket says 172.21.15.46:38906) 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe728a00700 1 -- 172.21.15.46:0/2200247311 learned_addr learned my addr 172.21.15.46:0/2200247311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe729400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 0x7fe7240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.332+0000 7fe729e00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 0x7fe7241a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 -- 172.21.15.46:0/2200247311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 msgr2=0x7fe7241a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 0x7fe7241a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 -- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 msgr2=0x7fe7240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 0x7fe7240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 -- 172.21.15.46:0/2200247311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100095f0 con 0x7fe7241061a0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe729e00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 0x7fe7241a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe728a00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe710000c00 tx=0x7fe710004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe722000700 1 -- 172.21.15.46:0/2200247311 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71001d070 con 0x7fe7241061a0 2024-05-08T23:06:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe722000700 1 -- 172.21.15.46:0/2200247311 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe710021410 con 0x7fe7241061a0 2024-05-08T23:06:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7241a6070 con 0x7fe7241061a0 2024-05-08T23:06:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe722000700 1 -- 172.21.15.46:0/2200247311 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710017650 con 0x7fe7241061a0 2024-05-08T23:06:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.333+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7241a6490 con 0x7fe7241061a0 2024-05-08T23:06:20.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.336+0000 7fe722000700 1 -- 172.21.15.46:0/2200247311 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe710021580 con 0x7fe7241061a0 2024-05-08T23:06:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 Processor -- start 2024-05-08T23:06:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe722000700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 0x7fe70c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 -- start start 2024-05-08T23:06:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe722000700 1 -- 172.21.15.46:0/2200247311 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe7100ce3a0 con 0x7fe7241061a0 2024-05-08T23:06:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe729e00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 0x7fe70c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b00b150 con 0x55582b0b4890 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b008b10 con 0x55582b0ba720 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.338+0000 7fe73f967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b004400 con 0x55582b0b9e00 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38910/0 (socket says 172.21.15.46:38910) 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 -- 172.21.15.46:0/3845001380 learned_addr learned my addr 172.21.15.46:0/3845001380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe728a00700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729e00700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 -- 172.21.15.46:0/3845001380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 msgr2=0x55582b0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 -- 172.21.15.46:0/3845001380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 msgr2=0x55582b0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 -- 172.21.15.46:0/3845001380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55582ae853b0 con 0x55582b0b9e00 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe728a00700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729e00700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729e00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 0x7fe70c0a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009410 tx=0x7fe71c009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.339+0000 7fe729400700 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe718009d80 tx=0x7fe718011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c46ac61f6a5b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.340+0000 7fe716600700 1 -- 172.21.15.46:0/3845001380 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718004230 con 0x55582b0b9e00 2024-05-08T23:06:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.340+0000 7fe716600700 1 -- 172.21.15.46:0/3845001380 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe718004390 con 0x55582b0b9e00 2024-05-08T23:06:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.340+0000 7fe716600700 1 -- 172.21.15.46:0/3845001380 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718019860 con 0x55582b0b9e00 2024-05-08T23:06:20.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 -- 172.21.15.46:0/3845001380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 msgr2=0x55582b0ba1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe718009d80 tx=0x7fe718011720 comp rx=0 tx=0).stop 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 -- 172.21.15.46:0/3845001380 shutdown_connections 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0ba1e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 --2- 172.21.15.46:0/3845001380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 -- 172.21.15.46:0/3845001380 >> 172.21.15.46:0/3845001380 conn(0x55582b012210 msgr2=0x55582b0aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 -- 172.21.15.46:0/3845001380 shutdown_connections 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.341+0000 7fe73f967b80 1 -- 172.21.15.46:0/3845001380 wait complete. 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 -- start start 2024-05-08T23:06:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 Processor -- start 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582afa0560 con 0x55582b0b4890 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582afa0200 con 0x55582b0ba720 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582ae876a0 con 0x55582b0b9e00 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.342+0000 7fe73f967b80 10 client.0 _ll_register_callbacks cb 0x55582afa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe729e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38922/0 (socket says 172.21.15.46:38922) 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 -- 172.21.15.46:0/2713536389 learned_addr learned my addr 172.21.15.46:0/2713536389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe729400700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 -- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 msgr2=0x55582b0be610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0be610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 -- 172.21.15.46:0/2713536389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 msgr2=0x55582b0bdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0bdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 -- 172.21.15.46:0/2713536389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7180099e0 con 0x55582b0b9e00 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe729e00700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0be610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe728a00700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe710012040 tx=0x7fe71009ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9502fdf79e3bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.343+0000 7fe70aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71009b130 con 0x55582b0b9e00 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe71009ef60 con 0x55582b0b9e00 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55582af5d270 con 0x55582b0b9e00 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7100a3520 con 0x55582b0b9e00 2024-05-08T23:06:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 10 client.26497 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55582ae31b60 con 0x55582b0b9e00 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55582af5d270 con 0x55582b0b9e00 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe71009a620 con 0x55582b0b9e00 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 10 client.26497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 20 client.26497 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b0cde20 0x55582b0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 4 client.26497 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.344+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55582b083810 con 0x55582b0cde20 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.345+0000 7fe729400700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b0cde20 0x55582b0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.345+0000 7fe729400700 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b0cde20 0x55582b0d0240 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6148594824d5891 server_cookie=2bc335bc8aedae2d in_seq=0 out_seq=0 2024-05-08T23:06:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.345+0000 7fe70aa00700 10 client.26497 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe70aa00700 1 -- 172.21.15.46:0/2713536389 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55582b083810 con 0x55582b0cde20 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe70aa00700 10 client.26497 handle_command_reply: tid=0 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 client.26497 shutdown 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 msgr2=0x55582b0be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe710012040 tx=0x7fe71009ad60 comp rx=0 tx=0).stop 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 shutdown_connections 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0ba720 0x55582b0be610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b0cde20 0x55582b0d0240 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b4890 0x55582b0bdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2713536389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b9e00 0x55582b0be0d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe70aa00700 0 client.26497 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.346+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 >> 172.21.15.46:0/2713536389 conn(0x55582b012210 msgr2=0x55582b0ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.347+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 shutdown_connections 2024-05-08T23:06:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.347+0000 7fe73f967b80 1 -- 172.21.15.46:0/2713536389 wait complete. 2024-05-08T23:06:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.347+0000 7fe73f967b80 20 client.26497 trim_cache size 0 max 16384 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 Processor -- start 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 -- start start 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 0x55582b0bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0ba720 0x55582b0be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b0a9800 con 0x55582b0b9e00 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b0b7870 con 0x55582b0b4890 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe73f967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b0bc0e0 con 0x55582b0ba720 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe729400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe729400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48106/0 (socket says 172.21.15.46:48106) 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.366+0000 7fe729400700 1 -- 172.21.15.46:0/400792345 learned_addr learned my addr 172.21.15.46:0/400792345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729e00700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0ba720 0x55582b0be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe728a00700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 0x55582b0bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 -- 172.21.15.46:0/400792345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0ba720 msgr2=0x55582b0be430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0ba720 0x55582b0be430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 -- 172.21.15.46:0/400792345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 msgr2=0x55582b0bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 0x55582b0bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 -- 172.21.15.46:0/400792345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55582afca560 con 0x55582b0b4890 2024-05-08T23:06:20.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe728a00700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 0x55582b0bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:20.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe729400700 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe71801b040 tx=0x7fe718005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c02753cdab47904 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.367+0000 7fe717000700 1 -- 172.21.15.46:0/400792345 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718029070 con 0x55582b0b4890 2024-05-08T23:06:20.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe717000700 1 -- 172.21.15.46:0/400792345 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe718009ed0 con 0x55582b0b4890 2024-05-08T23:06:20.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe717000700 1 -- 172.21.15.46:0/400792345 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71802a7b0 con 0x55582b0b4890 2024-05-08T23:06:20.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 -- 172.21.15.46:0/400792345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 msgr2=0x55582b0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe71801b040 tx=0x7fe718005fa0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 -- 172.21.15.46:0/400792345 shutdown_connections 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0ba720 0x55582b0be430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b4890 0x55582b0aed20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 --2- 172.21.15.46:0/400792345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0b9e00 0x55582b0bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 -- 172.21.15.46:0/400792345 >> 172.21.15.46:0/400792345 conn(0x55582b012210 msgr2=0x55582b0bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 -- 172.21.15.46:0/400792345 shutdown_connections 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.368+0000 7fe73f967b80 1 -- 172.21.15.46:0/400792345 wait complete. 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 -- start start 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 Processor -- start 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 0x55582b0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0ba720 0x55582b123320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.369+0000 7fe73f967b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b0c4050 con 0x55582b0ba720 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe73f967b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b123860 con 0x55582b0b9e00 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe73f967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582b1239f0 con 0x55582b0b4890 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe73f967b80 10 client.0 _ll_register_callbacks cb 0x55582afa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38942/0 (socket says 172.21.15.46:38942) 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 -- 172.21.15.46:0/2067712602 learned_addr learned my addr 172.21.15.46:0/2067712602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe728a00700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 0x55582b0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729e00700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0ba720 0x55582b123320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 -- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 msgr2=0x55582b0c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 0x55582b0c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 -- 172.21.15.46:0/2067712602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0ba720 msgr2=0x55582b123320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0ba720 0x55582b123320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe729400700 1 -- 172.21.15.46:0/2067712602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55582ae853b0 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.370+0000 7fe728a00700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 0x55582b0c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe729400700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e90 tx=0x7fe7180040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795500ab91ebb8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718029040 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe7180054d0 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe718004390 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe718005690 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 10 client.26503 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55582ae89cd0 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe718004390 con 0x55582b0b4890 2024-05-08T23:06:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe718011440 con 0x55582b0b4890 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 10 client.26503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.371+0000 7fe73f967b80 20 client.26503 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b124f00 0x55582b127320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe73f967b80 4 client.26503 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55582affcec0 con 0x55582b124f00 2024-05-08T23:06:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe728a00700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b124f00 0x55582b127320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:20.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe728a00700 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b124f00 0x55582b127320 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e0dcb359f26d1d server_cookie=7b21f4bc780efa89 in_seq=0 out_seq=0 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.372+0000 7fe70aa00700 10 client.26503 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe70aa00700 1 -- 172.21.15.46:0/2067712602 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55582affcec0 con 0x55582b124f00 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe70aa00700 10 client.26503 handle_command_reply: tid=0 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 client.26503 shutdown 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 msgr2=0x55582b0c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e90 tx=0x7fe7180040c0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 shutdown_connections 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55582b124f00 0x55582b127320 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55582b0ba720 0x55582b123320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55582b0b4890 0x55582b0c33e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 --2- 172.21.15.46:0/2067712602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55582b0b9e00 0x55582b0c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe70aa00700 0 client.26503 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 >> 172.21.15.46:0/2067712602 conn(0x55582b012210 msgr2=0x55582b0bb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 shutdown_connections 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 1 -- 172.21.15.46:0/2067712602 wait complete. 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.373+0000 7fe73f967b80 20 client.26503 trim_cache size 0 max 16384 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:20.378 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: "2381fd82-73ba-4a1d-a168-5abfc987a00c": { 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "2381fd82-73ba-4a1d-a168-5abfc987a00c", 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:06:20.379 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 msgr2=0x7fe70c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 0x7fe70c0a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009410 tx=0x7fe71c009920 comp rx=0 tx=0).stop 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 msgr2=0x7fe7240737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe710000c00 tx=0x7fe710004730 comp rx=0 tx=0).stop 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 shutdown_connections 2024-05-08T23:06:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe724073d20 0x7fe7241a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe70c0a1290 0x7fe70c0a3750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe724101080 0x7fe7240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 --2- 172.21.15.46:0/2200247311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241061a0 0x7fe7240737e0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe710000c00 tx=0x7fe710004730 comp rx=0 tx=0).stop 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 >> 172.21.15.46:0/2200247311 conn(0x7fe7240fc8e0 msgr2=0x7fe7240fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 shutdown_connections 2024-05-08T23:06:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:20.374+0000 7fe72be00700 1 -- 172.21.15.46:0/2200247311 wait complete. 2024-05-08T23:06:20.392 INFO:tasks.cephfs.filesystem:scrub status for tag:2381fd82-73ba-4a1d-a168-5abfc987a00c - {'path': '/', 'tag': '2381fd82-73ba-4a1d-a168-5abfc987a00c', 'options': 'recursive,force'} 2024-05-08T23:06:20.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50989 6.93 MB/sec warmup 42 sec latency 13.096 ms 2024-05-08T23:06:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:20 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:20 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:20 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:20 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:20 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:20 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:20 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:20 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:20 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:20 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:20 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:20 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:21.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 53556 7.15 MB/sec warmup 43 sec latency 9.752 ms 2024-05-08T23:06:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:21 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:06:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:21 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:21 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:21 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:21 smithi086 ceph-mon[78610]: pgmap v79: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:06:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:21 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:06:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:21 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:21 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:21 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:21 smithi053 ceph-mon[77587]: pgmap v79: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:06:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:21 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:06:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:21 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:21 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:21 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:21 smithi046 ceph-mon[36898]: pgmap v79: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:06:22.872 INFO:tasks.workunit.client.0.smithi046.stdout: 1 55839 7.18 MB/sec warmup 44 sec latency 10.021 ms 2024-05-08T23:06:23.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57572 7.27 MB/sec warmup 45 sec latency 11.920 ms 2024-05-08T23:06:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:23 smithi086 ceph-mon[78610]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:23 smithi053 ceph-mon[77587]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:24.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:23 smithi046 ceph-mon[36898]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:24.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 60331 7.49 MB/sec warmup 46 sec latency 11.006 ms 2024-05-08T23:06:25.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 61923 7.47 MB/sec warmup 47 sec latency 10.211 ms 2024-05-08T23:06:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:25 smithi086 ceph-mon[78610]: pgmap v81: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-05-08T23:06:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:25 smithi053 ceph-mon[77587]: pgmap v81: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-05-08T23:06:26.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:25 smithi046 ceph-mon[36898]: pgmap v81: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-05-08T23:06:26.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64269 7.61 MB/sec warmup 48 sec latency 14.290 ms 2024-05-08T23:06:27.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 66770 7.69 MB/sec warmup 49 sec latency 8.052 ms 2024-05-08T23:06:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:27 smithi086 ceph-mon[78610]: pgmap v82: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-08T23:06:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:27 smithi053 ceph-mon[77587]: pgmap v82: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-08T23:06:28.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:27 smithi046 ceph-mon[36898]: pgmap v82: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-08T23:06:28.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 68400 7.70 MB/sec warmup 50 sec latency 10.114 ms 2024-05-08T23:06:29.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 71127 7.88 MB/sec warmup 51 sec latency 13.595 ms 2024-05-08T23:06:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:29 smithi086 ceph-mon[78610]: pgmap v83: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:29 smithi053 ceph-mon[77587]: pgmap v83: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:30.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:29 smithi046 ceph-mon[36898]: pgmap v83: 129 pgs: 129 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-08T23:06:30.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 73404 7.89 MB/sec warmup 52 sec latency 9.874 ms 2024-05-08T23:06:31.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75135 7.97 MB/sec warmup 53 sec latency 10.688 ms 2024-05-08T23:06:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:31 smithi086 ceph-mon[78610]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-05-08T23:06:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:31 smithi053 ceph-mon[77587]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-05-08T23:06:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:31 smithi046 ceph-mon[36898]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-05-08T23:06:32.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 77445 8.05 MB/sec warmup 54 sec latency 140.728 ms 2024-05-08T23:06:33.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 78889 8.04 MB/sec warmup 55 sec latency 195.624 ms 2024-05-08T23:06:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:33 smithi086 ceph-mon[78610]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:33 smithi053 ceph-mon[77587]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:34.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:33 smithi046 ceph-mon[36898]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:34.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 81638 8.20 MB/sec warmup 56 sec latency 10.169 ms 2024-05-08T23:06:35.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82191 8.08 MB/sec warmup 57 sec latency 236.570 ms 2024-05-08T23:06:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:35 smithi086 ceph-mon[78610]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-05-08T23:06:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:06:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:35 smithi053 ceph-mon[77587]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-05-08T23:06:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:06:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:35 smithi046 ceph-mon[36898]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-05-08T23:06:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:06:36.873 INFO:tasks.workunit.client.0.smithi046.stdout: 1 84065 8.07 MB/sec warmup 58 sec latency 50.316 ms 2024-05-08T23:06:37.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85785 8.13 MB/sec warmup 59 sec latency 12.780 ms 2024-05-08T23:06:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:37 smithi086 ceph-mon[78610]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:37 smithi053 ceph-mon[77587]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:38.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:37 smithi046 ceph-mon[36898]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-05-08T23:06:38.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 88387 8.22 MB/sec warmup 60 sec latency 10.376 ms 2024-05-08T23:06:39.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 90179 8.26 MB/sec warmup 61 sec latency 12.295 ms 2024-05-08T23:06:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:39 smithi086 ceph-mon[78610]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:06:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:39 smithi053 ceph-mon[77587]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:06:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:39 smithi046 ceph-mon[36898]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:06:40.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92276 8.33 MB/sec warmup 62 sec latency 92.147 ms 2024-05-08T23:06:41.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 93703 8.31 MB/sec warmup 63 sec latency 19.195 ms 2024-05-08T23:06:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:41 smithi086 ceph-mon[78610]: pgmap v89: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 357 op/s 2024-05-08T23:06:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:41 smithi053 ceph-mon[77587]: pgmap v89: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 357 op/s 2024-05-08T23:06:42.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:41 smithi046 ceph-mon[36898]: pgmap v89: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 357 op/s 2024-05-08T23:06:42.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96184 8.39 MB/sec warmup 64 sec latency 10.130 ms 2024-05-08T23:06:43.874 INFO:tasks.workunit.client.0.smithi046.stdout: 1 98579 8.44 MB/sec warmup 65 sec latency 11.454 ms 2024-05-08T23:06:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:43 smithi086 ceph-mon[78610]: pgmap v90: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-05-08T23:06:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:43 smithi053 ceph-mon[77587]: pgmap v90: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-05-08T23:06:44.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:43 smithi046 ceph-mon[36898]: pgmap v90: 129 pgs: 129 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-05-08T23:06:44.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100169 8.44 MB/sec warmup 66 sec latency 16.381 ms 2024-05-08T23:06:45.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 102910 8.57 MB/sec warmup 67 sec latency 9.303 ms 2024-05-08T23:06:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:45 smithi086 ceph-mon[78610]: pgmap v91: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-05-08T23:06:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:45 smithi053 ceph-mon[77587]: pgmap v91: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-05-08T23:06:46.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:45 smithi046 ceph-mon[36898]: pgmap v91: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-05-08T23:06:46.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 105203 8.56 MB/sec warmup 68 sec latency 9.299 ms 2024-05-08T23:06:47.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 107007 8.62 MB/sec warmup 69 sec latency 9.869 ms 2024-05-08T23:06:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:47 smithi086 ceph-mon[78610]: pgmap v92: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-05-08T23:06:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:47 smithi053 ceph-mon[77587]: pgmap v92: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-05-08T23:06:48.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:47 smithi046 ceph-mon[36898]: pgmap v92: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-05-08T23:06:48.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 109626 8.68 MB/sec warmup 70 sec latency 21.885 ms 2024-05-08T23:06:49.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 111410 8.72 MB/sec warmup 71 sec latency 18.613 ms 2024-05-08T23:06:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:49 smithi086 ceph-mon[78610]: pgmap v93: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-05-08T23:06:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:49 smithi053 ceph-mon[77587]: pgmap v93: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-05-08T23:06:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:49 smithi046 ceph-mon[36898]: pgmap v93: 129 pgs: 129 active+clean; 68 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-05-08T23:06:50.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.618+0000 7f9c16600700 1 -- 172.21.15.46:0/1503160231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101a30 msgr2=0x7f9c10105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.618+0000 7f9c16600700 1 --2- 172.21.15.46:0/1503160231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101a30 0x7f9c10105a70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009990 tx=0x7f9bfc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.618+0000 7f9c16600700 1 -- 172.21.15.46:0/1503160231 shutdown_connections 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.618+0000 7f9c16600700 1 --2- 172.21.15.46:0/1503160231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 0x7f9c1010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.618+0000 7f9c16600700 1 --2- 172.21.15.46:0/1503160231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c10101080 0x7f9c10101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.619+0000 7f9c16600700 1 --2- 172.21.15.46:0/1503160231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101a30 0x7f9c10105a70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.619+0000 7f9c16600700 1 -- 172.21.15.46:0/1503160231 >> 172.21.15.46:0/1503160231 conn(0x7f9c100fc8e0 msgr2=0x7f9c100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.619+0000 7f9c16600700 1 -- 172.21.15.46:0/1503160231 shutdown_connections 2024-05-08T23:06:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.619+0000 7f9c16600700 1 -- 172.21.15.46:0/1503160231 wait complete. 2024-05-08T23:06:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.620+0000 7f9c16600700 1 Processor -- start 2024-05-08T23:06:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.620+0000 7f9c16600700 1 -- start start 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.620+0000 7f9c16600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c16600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 0x7f9c10199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c16600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1019de00 0x7f9c1019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c16600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019e880 con 0x7f9c10101080 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c16600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019e9f0 con 0x7f9c101061a0 2024-05-08T23:06:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c16600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019ebb0 con 0x7f9c1019de00 2024-05-08T23:06:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51066/0 (socket says 172.21.15.46:51066) 2024-05-08T23:06:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 -- 172.21.15.46:0/165444432 learned_addr learned my addr 172.21.15.46:0/165444432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1019de00 0x7f9c1019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 0x7f9c10199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 -- 172.21.15.46:0/165444432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1019de00 msgr2=0x7f9c1019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1019de00 0x7f9c1019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 -- 172.21.15.46:0/165444432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 msgr2=0x7f9c10199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 0x7f9c10199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.621+0000 7f9c0f400700 1 -- 172.21.15.46:0/165444432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0095f0 con 0x7f9c10101080 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c0f400700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9a0 tx=0x7f9c0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c07e00700 1 -- 172.21.15.46:0/165444432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00009790 con 0x7f9c10101080 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c07e00700 1 -- 172.21.15.46:0/165444432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c00010430 con 0x7f9c10101080 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c101a6110 con 0x7f9c10101080 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c07e00700 1 -- 172.21.15.46:0/165444432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00018680 con 0x7f9c10101080 2024-05-08T23:06:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.622+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c101a65f0 con 0x7f9c10101080 2024-05-08T23:06:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.627+0000 7f9c07e00700 1 -- 172.21.15.46:0/165444432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9c000105a0 con 0x7f9c10101080 2024-05-08T23:06:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 Processor -- start 2024-05-08T23:06:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 -- start start 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c051b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c051bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0472150 con 0x55c0c0521720 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c046fb10 con 0x55c0c051b890 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.628+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c046b400 con 0x55c0c0520e00 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c051b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59862/0 (socket says 172.21.15.46:59862) 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 -- 172.21.15.46:0/3065324626 learned_addr learned my addr 172.21.15.46:0/3065324626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c051bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 -- 172.21.15.46:0/3065324626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 msgr2=0x55c0c051bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c051bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 -- 172.21.15.46:0/3065324626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 msgr2=0x55c0c051b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c051b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 -- 172.21.15.46:0/3065324626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c02ec3b0 con 0x55c0c0520e00 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0f400700 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c051b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c07e00700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 0x7f9bf40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.629+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c08011680 tx=0x7f9c08009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c05fff74bb812da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c04c00700 1 -- 172.21.15.46:0/3065324626 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800e040 con 0x55c0c0520e00 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c07e00700 1 -- 172.21.15.46:0/165444432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9c00014070 con 0x7f9c10101080 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c04c00700 1 -- 172.21.15.46:0/3065324626 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0800f040 con 0x55c0c0520e00 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c04c00700 1 -- 172.21.15.46:0/3065324626 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080048c0 con 0x55c0c0520e00 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c0f400700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 0x7f9bf40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c2a212b80 1 -- 172.21.15.46:0/3065324626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 msgr2=0x55c0c05211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c08011680 tx=0x7f9c08009920 comp rx=0 tx=0).stop 2024-05-08T23:06:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.630+0000 7f9c0f400700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 0x7f9bf40a3750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c0c0521b80 tx=0x7f9c00010c80 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 -- 172.21.15.46:0/3065324626 shutdown_connections 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c051bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0520e00 0x55c0c05211e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/3065324626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c051b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 -- 172.21.15.46:0/3065324626 >> 172.21.15.46:0/3065324626 conn(0x55c0c0479210 msgr2=0x55c0c0515df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 -- 172.21.15.46:0/3065324626 shutdown_connections 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 -- 172.21.15.46:0/3065324626 wait complete. 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.631+0000 7f9c2a212b80 1 -- start start 2024-05-08T23:06:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 Processor -- start 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c0524f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0521720 0x55c0c05254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0407560 con 0x55c0c0520e00 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0407200 con 0x55c0c051b890 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c02ee6a0 con 0x55c0c0521720 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c2a212b80 10 client.0 _ll_register_callbacks cb 0x55c0c0410b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46408/0 (socket says 172.21.15.46:46408) 2024-05-08T23:06:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c0fe00700 1 -- 172.21.15.46:0/1943625642 learned_addr learned my addr 172.21.15.46:0/1943625642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c0524f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.632+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0521720 0x55c0c05254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 -- 172.21.15.46:0/1943625642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0521720 msgr2=0x55c0c05254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0521720 0x55c0c05254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 -- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 msgr2=0x55c0c0524a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 -- 172.21.15.46:0/1943625642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08011330 con 0x55c0c0520e00 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c0524f90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0098e0 tx=0x7f9bfc0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fef2028d8950bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.633+0000 7f9bf9600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc01d070 con 0x55c0c0520e00 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0c03c4270 con 0x55c0c0520e00 2024-05-08T23:06:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9bfc004500 con 0x55c0c0520e00 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc00bee0 con 0x55c0c0520e00 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 10 client.16482 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c0c0298b60 con 0x55c0c0520e00 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0c03c4270 con 0x55c0c0520e00 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9bfc00f580 con 0x55c0c0520e00 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 20 client.16482 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c0535200 0x55c0c0537620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 4 client.16482 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.634+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0c04ea810 con 0x55c0c0535200 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.635+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c0535200 0x55c0c0537620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.635+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c0535200 0x55c0c0537620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e031876c26ef59 server_cookie=635baf8823550379 in_seq=0 out_seq=0 2024-05-08T23:06:50.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.635+0000 7f9bf9600700 10 client.16482 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9bf9600700 1 -- 172.21.15.46:0/1943625642 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0c04ea810 con 0x55c0c0535200 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9bf9600700 10 client.16482 handle_command_reply: tid=0 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 client.16482 shutdown 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 msgr2=0x55c0c0524f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c0524f90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0098e0 tx=0x7f9bfc0057e0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 shutdown_connections 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c0521720 0x55c0c05254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c051b890 0x55c0c0524a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c0535200 0x55c0c0537620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1943625642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c0524f90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.636+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 >> 172.21.15.46:0/1943625642 conn(0x55c0c0479210 msgr2=0x55c0c05155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.637+0000 7f9bf9600700 0 client.16482 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.637+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 shutdown_connections 2024-05-08T23:06:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.637+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1943625642 wait complete. 2024-05-08T23:06:50.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.637+0000 7f9c2a212b80 20 client.16482 trim_cache size 0 max 16384 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.659+0000 7f9c2a212b80 1 Processor -- start 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.659+0000 7f9c2a212b80 1 -- start start 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c052c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0531d60 con 0x55c0c0521720 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c05275f0 con 0x55c0c0520e00 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c0527c90 con 0x55c0c051b890 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59882/0 (socket says 172.21.15.46:59882) 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46412/0 (socket says 172.21.15.46:46412) 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0fe00700 1 -- 172.21.15.46:0/1871454254 learned_addr learned my addr 172.21.15.46:0/1871454254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.660+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c052c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 -- 172.21.15.46:0/1871454254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 msgr2=0x55c0c0515d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 -- 172.21.15.46:0/1871454254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 msgr2=0x55c0c052c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c052c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 -- 172.21.15.46:0/1871454254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c051bf40 con 0x55c0c051b890 2024-05-08T23:06:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c080043f0 tx=0x7f9c08022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd2fb5ed5a21cf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c05600700 1 -- 172.21.15.46:0/1871454254 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800e040 con 0x55c0c051b890 2024-05-08T23:06:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c05600700 1 -- 172.21.15.46:0/1871454254 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c08004d10 con 0x55c0c051b890 2024-05-08T23:06:50.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.661+0000 7f9c05600700 1 -- 172.21.15.46:0/1871454254 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0802d800 con 0x55c0c051b890 2024-05-08T23:06:50.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1871454254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 msgr2=0x55c0c051b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c080043f0 tx=0x7f9c08022040 comp rx=0 tx=0).stop 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1871454254 shutdown_connections 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0521720 0x55c0c052c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c051b350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/1871454254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0520e00 0x55c0c0515d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1871454254 >> 172.21.15.46:0/1871454254 conn(0x55c0c0479210 msgr2=0x55c0c051eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1871454254 shutdown_connections 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- 172.21.15.46:0/1871454254 wait complete. 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.662+0000 7f9c2a212b80 1 -- start start 2024-05-08T23:06:50.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.663+0000 7f9c2a212b80 1 Processor -- start 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.663+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.663+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c051db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c2a212b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0521720 0x55c0c051e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c051ca10 con 0x55c0c0520e00 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c051cb80 con 0x55c0c0521720 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c2a212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c051cd40 con 0x55c0c051b890 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c2a212b80 10 client.0 _ll_register_callbacks cb 0x55c0c0410b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59886/0 (socket says 172.21.15.46:59886) 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c051db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 -- 172.21.15.46:0/2219002636 learned_addr learned my addr 172.21.15.46:0/2219002636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0521720 0x55c0c051e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 -- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0521720 msgr2=0x55c0c051e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0521720 0x55c0c051e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 -- 172.21.15.46:0/2219002636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 msgr2=0x55c0c051db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c051db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0fe00700 1 -- 172.21.15.46:0/2219002636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c0298b60 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.664+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c051db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c0fe00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009790 tx=0x7f9c08009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a30bbf4424637c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800e070 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c0800f040 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9c0802d310 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c08034040 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 10 client.26521 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f9c08028040 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0800f040 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9c08007500 con 0x55c0c051b890 2024-05-08T23:06:50.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 10 client.26521 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 20 client.26521 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c059cff0 0x55c0c059f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 4 client.26521 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:06:50.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.665+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0c0463ec0 con 0x55c0c059cff0 2024-05-08T23:06:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.666+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c059cff0 0x55c0c059f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c0ea00700 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c059cff0 0x55c0c059f410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c94227610a1979 server_cookie=9bdb14ee290f97b in_seq=0 out_seq=0 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9bf9600700 10 client.26521 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9bf9600700 1 -- 172.21.15.46:0/2219002636 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c0c0463ec0 con 0x55c0c059cff0 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9bf9600700 10 client.26521 handle_command_reply: tid=0 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 client.26521 shutdown 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 msgr2=0x55c0c052c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009790 tx=0x7f9c08009ac0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 shutdown_connections 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0c0521720 0x55c0c051e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c051b890 0x55c0c052c600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c0c059cff0 0x55c0c059f410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 --2- 172.21.15.46:0/2219002636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0c0520e00 0x55c0c051db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 >> 172.21.15.46:0/2219002636 conn(0x55c0c0479210 msgr2=0x55c0c05232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9bf9600700 0 client.26521 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 shutdown_connections 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.667+0000 7f9c2a212b80 1 -- 172.21.15.46:0/2219002636 wait complete. 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.668+0000 7f9c2a212b80 20 client.26521 trim_cache size 0 max 16384 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:06:50.672 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.668+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 msgr2=0x7f9bf40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.668+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 0x7f9bf40a3750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c0c0521b80 tx=0x7f9c00010c80 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.668+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 msgr2=0x7f9c10199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.668+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9a0 tx=0x7f9c0000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 shutdown_connections 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1019de00 0x7f9c1019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9bf40a1290 0x7f9bf40a3750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c10101080 0x7f9c10199520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 --2- 172.21.15.46:0/165444432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c101061a0 0x7f9c10199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 >> 172.21.15.46:0/165444432 conn(0x7f9c100fc8e0 msgr2=0x7f9c100fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 shutdown_connections 2024-05-08T23:06:50.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.669+0000 7f9c16600700 1 -- 172.21.15.46:0/165444432 wait complete. 2024-05-08T23:06:50.688 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:06:50.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:06:50.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 113852 8.79 MB/sec warmup 72 sec latency 10.787 ms 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.927+0000 7fe3e5000700 1 -- 172.21.15.46:0/4024604378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e010b3e0 msgr2=0x7fe3e010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.927+0000 7fe3e5000700 1 --2- 172.21.15.46:0/4024604378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e010b3e0 0x7fe3e010d8a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009f90 tx=0x7fe3d800b330 comp rx=0 tx=0).stop 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 -- 172.21.15.46:0/4024604378 shutdown_connections 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 --2- 172.21.15.46:0/4024604378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e010b3e0 0x7fe3e010d8a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 --2- 172.21.15.46:0/4024604378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 --2- 172.21.15.46:0/4024604378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e01028e0 0x7fe3e010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 -- 172.21.15.46:0/4024604378 >> 172.21.15.46:0/4024604378 conn(0x7fe3e00fb7e0 msgr2=0x7fe3e00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 -- 172.21.15.46:0/4024604378 shutdown_connections 2024-05-08T23:06:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.928+0000 7fe3e5000700 1 -- 172.21.15.46:0/4024604378 wait complete. 2024-05-08T23:06:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.929+0000 7fe3e5000700 1 Processor -- start 2024-05-08T23:06:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.929+0000 7fe3e5000700 1 -- start start 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.929+0000 7fe3e5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.929+0000 7fe3e5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 0x7fe3e0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3e5000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e010b3e0 0x7fe3e019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3e5000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019e540 con 0x7fe3e010b3e0 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3e5000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019e6b0 con 0x7fe3e0101fc0 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3e5000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019e870 con 0x7fe3e01028e0 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46440/0 (socket says 172.21.15.46:46440) 2024-05-08T23:06:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dde00700 1 -- 172.21.15.46:0/3032572177 learned_addr learned my addr 172.21.15.46:0/3032572177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3de800700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e010b3e0 0x7fe3e019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 0x7fe3e0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 -- 172.21.15.46:0/3032572177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 msgr2=0x7fe3e0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 -- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e010b3e0 msgr2=0x7fe3e019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e010b3e0 0x7fe3e019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.930+0000 7fe3dd400700 1 -- 172.21.15.46:0/3032572177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0095f0 con 0x7fe3e01028e0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3dd400700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 0x7fe3e0199bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00eb00 tx=0x7fe3cc00ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00cb70 con 0x7fe3e01028e0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3cc0044e0 con 0x7fe3e01028e0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc003d90 con 0x7fe3e01028e0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d8009c40 con 0x7fe3e01028e0 2024-05-08T23:06:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.931+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e01a61b0 con 0x7fe3e01028e0 2024-05-08T23:06:50.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.932+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e0069d50 con 0x7fe3e01028e0 2024-05-08T23:06:50.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.934+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe3cc003f30 con 0x7fe3e01028e0 2024-05-08T23:06:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.935+0000 7fe3d6a00700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 0x7fe3c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.936+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe3cc014070 con 0x7fe3e01028e0 2024-05-08T23:06:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.936+0000 7fe3dde00700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 0x7fe3c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:50.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.937+0000 7fe3dde00700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 0x7fe3c40a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d20 tx=0x7fe3d0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:50.940+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3cc0964b0 con 0x7fe3e01028e0 2024-05-08T23:06:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.239+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e0068160 con 0x7fe3e01028e0 2024-05-08T23:06:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.239+0000 7fe3d6a00700 1 -- 172.21.15.46:0/3032572177 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10907 (secure 0 0 0) 0x7fe3cc099ad0 con 0x7fe3e01028e0 2024-05-08T23:06:51.240 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:06:51.240 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:05:40.700994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:06:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 msgr2=0x7fe3c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 0x7fe3c40a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d20 tx=0x7fe3d0009260 comp rx=0 tx=0).stop 2024-05-08T23:06:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 msgr2=0x7fe3e0199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 0x7fe3e0199bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00eb00 tx=0x7fe3cc00ee10 comp rx=0 tx=0).stop 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 shutdown_connections 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3c40a1290 0x7fe3c40a3750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e010b3e0 0x7fe3e019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e0101fc0 0x7fe3e0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 --2- 172.21.15.46:0/3032572177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e01028e0 0x7fe3e0199bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 >> 172.21.15.46:0/3032572177 conn(0x7fe3e00fb7e0 msgr2=0x7fe3e0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 shutdown_connections 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.242+0000 7fe3e5000700 1 -- 172.21.15.46:0/3032572177 wait complete. 2024-05-08T23:06:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-05-08T23:06:51.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:06:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 -- 172.21.15.46:0/3216438134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41028c0 msgr2=0x7ff0b410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 --2- 172.21.15.46:0/3216438134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41028c0 0x7ff0b410ada0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009990 tx=0x7ff0a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 -- 172.21.15.46:0/3216438134 shutdown_connections 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 --2- 172.21.15.46:0/3216438134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 0x7ff0b410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 --2- 172.21.15.46:0/3216438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b4101fa0 0x7ff0b4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 --2- 172.21.15.46:0/3216438134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41028c0 0x7ff0b410ada0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 -- 172.21.15.46:0/3216438134 >> 172.21.15.46:0/3216438134 conn(0x7ff0b40fb760 msgr2=0x7ff0b40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 -- 172.21.15.46:0/3216438134 shutdown_connections 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.492+0000 7ff0ba800700 1 -- 172.21.15.46:0/3216438134 wait complete. 2024-05-08T23:06:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 Processor -- start 2024-05-08T23:06:51.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 -- start start 2024-05-08T23:06:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b41028c0 0x7ff0b4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 0x7ff0b419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419c1e0 con 0x7ff0b41028c0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419c350 con 0x7ff0b410b3c0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0ba800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419c510 con 0x7ff0b4101fa0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59930/0 (socket says 172.21.15.46:59930) 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0b3400700 1 -- 172.21.15.46:0/1061281250 learned_addr learned my addr 172.21.15.46:0/1061281250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b41028c0 0x7ff0b4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.494+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 0x7ff0b419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 -- 172.21.15.46:0/1061281250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 msgr2=0x7ff0b419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 0x7ff0b419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 -- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b41028c0 msgr2=0x7ff0b4197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b41028c0 0x7ff0b4197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 -- 172.21.15.46:0/1061281250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4009520 con 0x7ff0b4101fa0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400eb30 tx=0x7ff0a400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0abe00700 1 -- 172.21.15.46:0/1061281250 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4009880 con 0x7ff0b4101fa0 2024-05-08T23:06:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0abe00700 1 -- 172.21.15.46:0/1061281250 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a40044e0 con 0x7ff0b4101fa0 2024-05-08T23:06:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0abe00700 1 -- 172.21.15.46:0/1061281250 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4003d80 con 0x7ff0b4101fa0 2024-05-08T23:06:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a00095f0 con 0x7ff0b4101fa0 2024-05-08T23:06:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.495+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41a3e50 con 0x7ff0b4101fa0 2024-05-08T23:06:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.499+0000 7ff0abe00700 1 -- 172.21.15.46:0/1061281250 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff0a4003f20 con 0x7ff0b4101fa0 2024-05-08T23:06:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.500+0000 7ff0ce33db80 1 Processor -- start 2024-05-08T23:06:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 -- start start 2024-05-08T23:06:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0abe00700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0abe00700 1 -- 172.21.15.46:0/1061281250 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0a4014070 con 0x7ff0b4101fa0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78fb10 0x5610ce789760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce78a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce6dc150 con 0x5610ce789ca0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce6d9b10 con 0x5610ce78fb10 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.501+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce6d5400 con 0x5610ce78f1f0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59934/0 (socket says 172.21.15.46:59934) 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 -- 172.21.15.46:0/4102783893 learned_addr learned my addr 172.21.15.46:0/4102783893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78fb10 0x5610ce789760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce78a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 -- 172.21.15.46:0/4102783893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78fb10 msgr2=0x5610ce789760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78fb10 0x5610ce789760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 -- 172.21.15.46:0/4102783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 msgr2=0x5610ce78a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce78a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 -- 172.21.15.46:0/4102783893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ce5563b0 con 0x5610ce78f1f0 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce78a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.502+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4096d30 tx=0x7ff0a4099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191c9f5867d0b47e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0a8c00700 1 -- 172.21.15.46:0/4102783893 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409a2c0 con 0x5610ce78f1f0 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009ec0 tx=0x7ff0ac00b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0a8c00700 1 -- 172.21.15.46:0/4102783893 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a409e0f0 con 0x5610ce78f1f0 2024-05-08T23:06:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0a8c00700 1 -- 172.21.15.46:0/4102783893 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40cd190 con 0x5610ce78f1f0 2024-05-08T23:06:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4102783893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 msgr2=0x5610ce78f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.503+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4096d30 tx=0x7ff0a4099c20 comp rx=0 tx=0).stop 2024-05-08T23:06:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4102783893 shutdown_connections 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce78a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78f1f0 0x5610ce78f5d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4102783893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78fb10 0x5610ce789760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4102783893 >> 172.21.15.46:0/4102783893 conn(0x5610ce6e3210 msgr2=0x5610ce77fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4102783893 shutdown_connections 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4102783893 wait complete. 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 -- start start 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.504+0000 7ff0ce33db80 1 Processor -- start 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce792f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce7934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce671560 con 0x5610ce789ca0 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce671200 con 0x5610ce78f1f0 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce5586a0 con 0x5610ce78fb10 2024-05-08T23:06:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0ce33db80 10 client.0 _ll_register_callbacks cb 0x5610ce67ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59940/0 (socket says 172.21.15.46:59940) 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0b3e00700 1 -- 172.21.15.46:0/214838058 learned_addr learned my addr 172.21.15.46:0/214838058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce7934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.505+0000 7ff0b3400700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce792f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 -- 172.21.15.46:0/214838058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 msgr2=0x5610ce7934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce7934b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 -- 172.21.15.46:0/214838058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 msgr2=0x5610ce792f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce792f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 -- 172.21.15.46:0/214838058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41a3e50 con 0x5610ce78fb10 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009530 tx=0x7ff0ac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0de81883cd13d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff09d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac016850 con 0x5610ce78fb10 2024-05-08T23:06:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610ce62e270 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.506+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0ac0169b0 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0ac016b70 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 10 client.26539 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5610ce502b60 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610ce62e270 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff0ac0213a0 con 0x5610ce78fb10 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 10 client.26539 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 10 client.26539 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 20 client.26539 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7a35f0 0x5610ce7a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 4 client.26539 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610ce754810 con 0x5610ce7a35f0 2024-05-08T23:06:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.507+0000 7ff0b3400700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7a35f0 0x5610ce7a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.508+0000 7ff09d600700 10 client.26539 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.508+0000 7ff0b3400700 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7a35f0 0x5610ce7a5a10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fba3e29f1a5af4dd server_cookie=f2a1ee9a7febd32d in_seq=0 out_seq=0 2024-05-08T23:06:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.508+0000 7ff09d600700 1 -- 172.21.15.46:0/214838058 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610ce754810 con 0x5610ce7a35f0 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.508+0000 7ff09d600700 10 client.26539 handle_command_reply: tid=0 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 client.26539 shutdown 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 msgr2=0x5610ce7939f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009530 tx=0x7ff0ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 shutdown_connections 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7a35f0 0x5610ce7a5a10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce78fb10 0x5610ce7939f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce789ca0 0x5610ce792f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/214838058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce7934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff09d600700 0 client.26539 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 >> 172.21.15.46:0/214838058 conn(0x5610ce6e3210 msgr2=0x5610ce77f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 shutdown_connections 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.509+0000 7ff0ce33db80 1 -- 172.21.15.46:0/214838058 wait complete. 2024-05-08T23:06:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.510+0000 7ff0ce33db80 20 client.26539 trim_cache size 0 max 16384 2024-05-08T23:06:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 Processor -- start 2024-05-08T23:06:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 -- start start 2024-05-08T23:06:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce78e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce77fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce62e270 con 0x5610ce78fb10 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce7a0cf0 con 0x5610ce78f1f0 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce7942c0 con 0x5610ce789ca0 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.525+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59942/0 (socket says 172.21.15.46:59942) 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 -- 172.21.15.46:0/1839857146 learned_addr learned my addr 172.21.15.46:0/1839857146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce78e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce77fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 -- 172.21.15.46:0/1839857146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 msgr2=0x5610ce78e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce78e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 -- 172.21.15.46:0/1839857146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 msgr2=0x5610ce77fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce77fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 -- 172.21.15.46:0/1839857146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ce69b560 con 0x5610ce789ca0 2024-05-08T23:06:51.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0b3400700 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40caf00 tx=0x7ff0a4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa00e63fba192114 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0a9600700 1 -- 172.21.15.46:0/1839857146 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40b7070 con 0x5610ce789ca0 2024-05-08T23:06:51.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0a9600700 1 -- 172.21.15.46:0/1839857146 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a40a27c0 con 0x5610ce789ca0 2024-05-08T23:06:51.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.526+0000 7ff0a9600700 1 -- 172.21.15.46:0/1839857146 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409a520 con 0x5610ce789ca0 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 -- 172.21.15.46:0/1839857146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 msgr2=0x5610ce781a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40caf00 tx=0x7ff0a4003820 comp rx=0 tx=0).stop 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 -- 172.21.15.46:0/1839857146 shutdown_connections 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce77fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce781a00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/1839857146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce78e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.527+0000 7ff0ce33db80 1 -- 172.21.15.46:0/1839857146 >> 172.21.15.46:0/1839857146 conn(0x5610ce6e3210 msgr2=0x5610ce78e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.528+0000 7ff0ce33db80 1 -- 172.21.15.46:0/1839857146 shutdown_connections 2024-05-08T23:06:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.528+0000 7ff0ce33db80 1 -- 172.21.15.46:0/1839857146 wait complete. 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.528+0000 7ff0ce33db80 1 -- start start 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.528+0000 7ff0ce33db80 1 Processor -- start 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce798630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce798b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce7f8af0 con 0x5610ce78fb10 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce7f8c60 con 0x5610ce78f1f0 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ce7f8e20 con 0x5610ce789ca0 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0ce33db80 10 client.0 _ll_register_callbacks cb 0x5610ce67ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59956/0 (socket says 172.21.15.46:59956) 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 -- 172.21.15.46:0/4281518590 learned_addr learned my addr 172.21.15.46:0/4281518590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce798630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3e00700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce798b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 -- 172.21.15.46:0/4281518590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 msgr2=0x5610ce798630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce798630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 -- 172.21.15.46:0/4281518590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 msgr2=0x5610ce798b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce798b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.529+0000 7ff0b3400700 1 -- 172.21.15.46:0/4281518590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610ce5563b0 con 0x5610ce789ca0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce798630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0b3400700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099ed0 tx=0x7ff0a4003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb29ab000c8fcee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40b7030 con 0x5610ce789ca0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0a40b8070 con 0x5610ce789ca0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0a409e0f0 con 0x5610ce789ca0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0a40997b0 con 0x5610ce789ca0 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 10 client.26545 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5610ce55acd0 con 0x5610ce789ca0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a409e0f0 con 0x5610ce789ca0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff0a40a23d0 con 0x5610ce789ca0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 10 client.26545 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 10 client.26545 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.530+0000 7ff0ce33db80 20 client.26545 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7fa0f0 0x5610ce7fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0ce33db80 4 client.26545 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610ce6cdec0 con 0x5610ce7fa0f0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7fa0f0 0x5610ce7fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0b2a00700 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7fa0f0 0x5610ce7fc4e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f06c159c4e511d7d server_cookie=25b3d1dbea5ee791 in_seq=0 out_seq=0 2024-05-08T23:06:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff09d600700 10 client.26545 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:51.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff09d600700 1 -- 172.21.15.46:0/4281518590 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610ce6cdec0 con 0x5610ce7fa0f0 2024-05-08T23:06:51.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff09d600700 10 client.26545 handle_command_reply: tid=0 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.531+0000 7ff0ce33db80 1 client.26545 shutdown 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 msgr2=0x5610ce7980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099ed0 tx=0x7ff0a4003680 comp rx=0 tx=0).stop 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 shutdown_connections 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5610ce7fa0f0 0x5610ce7fc4e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610ce78fb10 0x5610ce798b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ce789ca0 0x5610ce7980f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff09d600700 0 client.26545 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 --2- 172.21.15.46:0/4281518590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610ce78f1f0 0x5610ce798630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 >> 172.21.15.46:0/4281518590 conn(0x5610ce6e3210 msgr2=0x5610ce793720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 shutdown_connections 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 1 -- 172.21.15.46:0/4281518590 wait complete. 2024-05-08T23:06:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.532+0000 7ff0ce33db80 20 client.26545 trim_cache size 0 max 16384 2024-05-08T23:06:51.537 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 msgr2=0x7ff0980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009ec0 tx=0x7ff0ac00b040 comp rx=0 tx=0).stop 2024-05-08T23:06:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 msgr2=0x7ff0b41972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400eb30 tx=0x7ff0a400ee40 comp rx=0 tx=0).stop 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 shutdown_connections 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b410b3c0 0x7ff0b419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b4101fa0 0x7ff0b41972c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 --2- 172.21.15.46:0/1061281250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b41028c0 0x7ff0b4197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 >> 172.21.15.46:0/1061281250 conn(0x7ff0b40fb760 msgr2=0x7ff0b40fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 shutdown_connections 2024-05-08T23:06:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.533+0000 7ff0ba800700 1 -- 172.21.15.46:0/1061281250 wait complete. 2024-05-08T23:06:51.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:06:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 -- 172.21.15.46:0/2270947719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc1061a0 msgr2=0x7f9bfc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 --2- 172.21.15.46:0/2270947719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc10dd20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009f90 tx=0x7f9bf400b330 comp rx=0 tx=0).stop 2024-05-08T23:06:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 -- 172.21.15.46:0/2270947719 shutdown_connections 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 --2- 172.21.15.46:0/2270947719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc10dd20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 --2- 172.21.15.46:0/2270947719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc101080 0x7f9bfc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 --2- 172.21.15.46:0/2270947719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc101a30 0x7f9bfc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 -- 172.21.15.46:0/2270947719 >> 172.21.15.46:0/2270947719 conn(0x7f9bfc0fc8e0 msgr2=0x7f9bfc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 -- 172.21.15.46:0/2270947719 shutdown_connections 2024-05-08T23:06:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.777+0000 7f9c04200700 1 -- 172.21.15.46:0/2270947719 wait complete. 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.778+0000 7f9c04200700 1 Processor -- start 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.778+0000 7f9c04200700 1 -- start start 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.778+0000 7f9c04200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.778+0000 7f9c04200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c04200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc19bd30 0x7f9bfc19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c04200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19c7b0 con 0x7f9bfc1061a0 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c04200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19c920 con 0x7f9bfc19bd30 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c04200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19cae0 con 0x7f9bfc101080 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59978/0 (socket says 172.21.15.46:59978) 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c00e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 -- 172.21.15.46:0/841513236 learned_addr learned my addr 172.21.15.46:0/841513236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c02200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc19bd30 0x7f9bfc19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 -- 172.21.15.46:0/841513236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc19bd30 msgr2=0x7f9bfc19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc19bd30 0x7f9bfc19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 -- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc1061a0 msgr2=0x7f9bfc197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.779+0000 7f9c01800700 1 -- 172.21.15.46:0/841513236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009520 con 0x7f9bfc101080 2024-05-08T23:06:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9c01800700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000eb30 tx=0x7f9bf000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9bfa000700 1 -- 172.21.15.46:0/841513236 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0009880 con 0x7f9bfc101080 2024-05-08T23:06:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9bfa000700 1 -- 172.21.15.46:0/841513236 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf00044e0 con 0x7f9bfc101080 2024-05-08T23:06:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bfc101080 2024-05-08T23:06:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9bfa000700 1 -- 172.21.15.46:0/841513236 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003d80 con 0x7f9bfc101080 2024-05-08T23:06:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.780+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf4009c40 con 0x7f9bfc101080 2024-05-08T23:06:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.783+0000 7f9bfa000700 1 -- 172.21.15.46:0/841513236 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9bf0003f20 con 0x7f9bfc101080 2024-05-08T23:06:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.785+0000 7f9bfa000700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 0x7f9be40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.786+0000 7f9bfa000700 1 -- 172.21.15.46:0/841513236 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9bf0014070 con 0x7f9bfc101080 2024-05-08T23:06:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.786+0000 7f9c00e00700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 0x7f9be40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.786+0000 7f9c17c76b80 1 Processor -- start 2024-05-08T23:06:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.786+0000 7f9c17c76b80 1 -- start start 2024-05-08T23:06:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.786+0000 7f9c00e00700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 0x7f9be40a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc1989a0 tx=0x7f9be8009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b424150 con 0x55794b4d2e00 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b421b10 con 0x55794b4cd890 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b41d400 con 0x55794b4d3720 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c01800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59982/0 (socket says 172.21.15.46:59982) 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 -- 172.21.15.46:0/2727760963 learned_addr learned my addr 172.21.15.46:0/2727760963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c02200700 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 -- 172.21.15.46:0/2727760963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 msgr2=0x55794b4cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 -- 172.21.15.46:0/2727760963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 msgr2=0x55794b4d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.787+0000 7f9c00e00700 1 -- 172.21.15.46:0/2727760963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55794b29e3b0 con 0x55794b4d3720 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.788+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e620 tx=0x7f9be8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a941c76d987a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.788+0000 7f9beea00700 1 -- 172.21.15.46:0/2727760963 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8004020 con 0x55794b4d3720 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.788+0000 7f9beea00700 1 -- 172.21.15.46:0/2727760963 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be8021070 con 0x55794b4d3720 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.788+0000 7f9beea00700 1 -- 172.21.15.46:0/2727760963 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80266e0 con 0x55794b4d3720 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2727760963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 msgr2=0x55794b4cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e620 tx=0x7f9be8014000 comp rx=0 tx=0).stop 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2727760963 shutdown_connections 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2727760963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4cd350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2727760963 >> 172.21.15.46:0/2727760963 conn(0x55794b42b210 msgr2=0x55794b4c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2727760963 shutdown_connections 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.789+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2727760963 wait complete. 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.790+0000 7f9c17c76b80 1 -- start start 2024-05-08T23:06:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.790+0000 7f9c17c76b80 1 Processor -- start 2024-05-08T23:06:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 0x55794b4d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 0x55794b4d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b3b9560 con 0x55794b4d3720 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b3b9200 con 0x55794b4cd890 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b2a06a0 con 0x55794b4d2e00 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c17c76b80 10 client.0 _ll_register_callbacks cb 0x55794b3c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c01800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 0x55794b4d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c01800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46508/0 (socket says 172.21.15.46:46508) 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c01800700 1 -- 172.21.15.46:0/2162495787 learned_addr learned my addr 172.21.15.46:0/2162495787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c02200700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 0x55794b4d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 -- 172.21.15.46:0/2162495787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 msgr2=0x55794b4d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 -- 172.21.15.46:0/2162495787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 msgr2=0x55794b4d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 0x55794b4d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c00e00700 1 -- 172.21.15.46:0/2162495787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be8017070 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c02200700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 0x55794b4d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.791+0000 7f9c01800700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9c00e00700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 0x55794b4d70a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ec10 tx=0x7f9be800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b10a2d7362801e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9be3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be802a040 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9be8005270 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55794b376270 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be8005430 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9c17c76b80 10 client.26557 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55794b24ab60 con 0x55794b4d2e00 2024-05-08T23:06:51.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.792+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55794b376270 con 0x55794b4d2e00 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9be801d410 con 0x55794b4d2e00 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 10 client.26557 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 10 client.26557 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 20 client.26557 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b4e7100 0x55794b4e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 4 client.26557 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55794b49c810 con 0x55794b4e7100 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.793+0000 7f9c01800700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b4e7100 0x55794b4e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c01800700 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b4e7100 0x55794b4e94f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e130210bb8d628f2 server_cookie=ec0de239d6cad709 in_seq=0 out_seq=0 2024-05-08T23:06:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9be3400700 10 client.26557 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9be3400700 1 -- 172.21.15.46:0/2162495787 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55794b49c810 con 0x55794b4e7100 2024-05-08T23:06:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9be3400700 10 client.26557 handle_command_reply: tid=0 2024-05-08T23:06:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 client.26557 shutdown 2024-05-08T23:06:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 msgr2=0x55794b4d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 0x55794b4d70a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ec10 tx=0x7f9be800edc0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 shutdown_connections 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b4e7100 0x55794b4e94f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d3720 0x55794b4d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/2162495787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d2e00 0x55794b4d70a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9be3400700 0 client.26557 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.794+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 >> 172.21.15.46:0/2162495787 conn(0x55794b42b210 msgr2=0x55794b4c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.795+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 shutdown_connections 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.795+0000 7f9c17c76b80 1 -- 172.21.15.46:0/2162495787 wait complete. 2024-05-08T23:06:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.795+0000 7f9c17c76b80 20 client.26557 trim_cache size 0 max 16384 2024-05-08T23:06:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 Processor -- start 2024-05-08T23:06:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 -- start start 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4d35a0 con 0x55794b4d2e00 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4d8110 con 0x55794b4cd890 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4c4dc0 con 0x55794b4d3720 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c02200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.811+0000 7f9c00e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c02200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48262/0 (socket says 172.21.15.46:48262) 2024-05-08T23:06:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c02200700 1 -- 172.21.15.46:0/1175655199 learned_addr learned my addr 172.21.15.46:0/1175655199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c01800700 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 -- 172.21.15.46:0/1175655199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 msgr2=0x55794b4c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 -- 172.21.15.46:0/1175655199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 msgr2=0x55794b4cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 -- 172.21.15.46:0/1175655199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55794b3e3560 con 0x55794b4d2e00 2024-05-08T23:06:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c02200700 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.812+0000 7f9c00e00700 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d2310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009890 tx=0x7f9be8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3161a7613f0ccff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.813+0000 7f9bef400700 1 -- 172.21.15.46:0/1175655199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be802a070 con 0x55794b4d2e00 2024-05-08T23:06:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.813+0000 7f9bef400700 1 -- 172.21.15.46:0/1175655199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be800ec40 con 0x55794b4d2e00 2024-05-08T23:06:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.813+0000 7f9bef400700 1 -- 172.21.15.46:0/1175655199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be801d990 con 0x55794b4d2e00 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- 172.21.15.46:0/1175655199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 msgr2=0x55794b4d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d2310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009890 tx=0x7f9be8005fb0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- 172.21.15.46:0/1175655199 shutdown_connections 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/1175655199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4d2310 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- 172.21.15.46:0/1175655199 >> 172.21.15.46:0/1175655199 conn(0x55794b42b210 msgr2=0x55794b4d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- 172.21.15.46:0/1175655199 shutdown_connections 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- 172.21.15.46:0/1175655199 wait complete. 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 -- start start 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.814+0000 7f9c17c76b80 1 Processor -- start 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4ce9a0 con 0x55794b4d2e00 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4ceb10 con 0x55794b4cd890 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55794b4cecd0 con 0x55794b4d3720 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c17c76b80 10 client.0 _ll_register_callbacks cb 0x55794b3c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c01800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c02200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c01800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56158/0 (socket says 172.21.15.46:56158) 2024-05-08T23:06:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c01800700 1 -- 172.21.15.46:0/3597154484 learned_addr learned my addr 172.21.15.46:0/3597154484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c02200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48264/0 (socket says 172.21.15.46:48264) 2024-05-08T23:06:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.815+0000 7f9c00e00700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 -- 172.21.15.46:0/3597154484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 msgr2=0x55794b4de670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 -- 172.21.15.46:0/3597154484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 msgr2=0x55794b4cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 -- 172.21.15.46:0/3597154484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc10e590 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c01800700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c02200700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4007c30 tx=0x7f9bf4007c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e51634be956f8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9be3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4020070 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9bf4003a00 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55794b415ec0 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bf400e040 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c17c76b80 10 client.26563 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.816+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55794b24ab60 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55794b415ec0 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9bf400b050 con 0x55794b4d3720 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 10 client.26563 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 10 client.26563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 20 client.26563 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b542330 0x55794b544750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 4 client.26563 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55794b26bcf0 con 0x55794b542330 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c01800700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b542330 0x55794b544750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9be3400700 10 client.26563 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.817+0000 7f9c01800700 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b542330 0x55794b544750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b707e775f3bd130b server_cookie=6989161da7f33186 in_seq=0 out_seq=0 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9be3400700 1 -- 172.21.15.46:0/3597154484 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55794b26bcf0 con 0x55794b542330 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9be3400700 10 client.26563 handle_command_reply: tid=0 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 client.26563 shutdown 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 msgr2=0x55794b4d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4007c30 tx=0x7f9bf4007c60 comp rx=0 tx=0).stop 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 shutdown_connections 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55794b542330 0x55794b544750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4d3720 0x55794b4d0160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55794b4cd890 0x55794b4de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9be3400700 0 client.26563 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 --2- 172.21.15.46:0/3597154484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794b4d2e00 0x55794b4cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 >> 172.21.15.46:0/3597154484 conn(0x55794b42b210 msgr2=0x55794b4d6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 shutdown_connections 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 1 -- 172.21.15.46:0/3597154484 wait complete. 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.818+0000 7f9c17c76b80 20 client.26563 trim_cache size 0 max 16384 2024-05-08T23:06:51.822 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 msgr2=0x7f9be40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 0x7f9be40a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc1989a0 tx=0x7f9be8009310 comp rx=0 tx=0).stop 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 msgr2=0x7f9bfc197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000eb30 tx=0x7f9bf000ee40 comp rx=0 tx=0).stop 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 shutdown_connections 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bfc19bd30 0x7f9bfc19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bfc101080 0x7f9bfc197450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9be40a1310 0x7f9be40a37d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 --2- 172.21.15.46:0/841513236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc1061a0 0x7f9bfc197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 >> 172.21.15.46:0/841513236 conn(0x7f9bfc0fc8e0 msgr2=0x7f9bfc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 shutdown_connections 2024-05-08T23:06:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:51.819+0000 7f9c04200700 1 -- 172.21.15.46:0/841513236 wait complete. 2024-05-08T23:06:51.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:06:51.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 116125 8.78 MB/sec warmup 73 sec latency 12.222 ms 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.090+0000 7f526ca00700 1 -- 172.21.15.46:0/1783301300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5268101a50 msgr2=0x7f5268105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.090+0000 7f526ca00700 1 --2- 172.21.15.46:0/1783301300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5268101a50 0x7f5268105a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5254009990 tx=0x7f5254009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 -- 172.21.15.46:0/1783301300 shutdown_connections 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 --2- 172.21.15.46:0/1783301300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681061c0 0x7f526810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 --2- 172.21.15.46:0/1783301300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681010a0 0x7f5268101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 --2- 172.21.15.46:0/1783301300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5268101a50 0x7f5268105a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 -- 172.21.15.46:0/1783301300 >> 172.21.15.46:0/1783301300 conn(0x7f52680fc960 msgr2=0x7f52680fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 -- 172.21.15.46:0/1783301300 shutdown_connections 2024-05-08T23:06:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.091+0000 7f526ca00700 1 -- 172.21.15.46:0/1783301300 wait complete. 2024-05-08T23:06:52.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 Processor -- start 2024-05-08T23:06:52.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 -- start start 2024-05-08T23:06:52.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526819de60 0x7f526819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819e8e0 con 0x7f52681010a0 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819ea50 con 0x7f52681061c0 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.092+0000 7f526ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819ec10 con 0x7f526819de60 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59332/0 (socket says 172.21.15.46:59332) 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56188/0 (socket says 172.21.15.46:56188) 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 -- 172.21.15.46:0/3820045058 learned_addr learned my addr 172.21.15.46:0/3820045058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5266800700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526819de60 0x7f526819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 -- 172.21.15.46:0/3820045058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526819de60 msgr2=0x7f526819e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526819de60 0x7f526819e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 -- 172.21.15.46:0/3820045058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 msgr2=0x7f5268199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265e00700 1 -- 172.21.15.46:0/3820045058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258009520 con 0x7f52681010a0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.093+0000 7f5265400700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f5265e00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f525800eb30 tx=0x7f525800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f525ea00700 1 -- 172.21.15.46:0/3820045058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258009880 con 0x7f52681010a0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f525ea00700 1 -- 172.21.15.46:0/3820045058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52580044e0 con 0x7f52681010a0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5260009c40 con 0x7f52681010a0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f525ea00700 1 -- 172.21.15.46:0/3820045058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258003d80 con 0x7f52681010a0 2024-05-08T23:06:52.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.094+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540095f0 con 0x7f52681010a0 2024-05-08T23:06:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.099+0000 7f525ea00700 1 -- 172.21.15.46:0/3820045058 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5258003ee0 con 0x7f52681010a0 2024-05-08T23:06:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 Processor -- start 2024-05-08T23:06:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 -- start start 2024-05-08T23:06:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f525ea00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 0x7f524c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f525ea00700 1 -- 172.21.15.46:0/3820045058 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5258014070 con 0x7f52681010a0 2024-05-08T23:06:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x5627883221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5266800700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 0x7f524c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788273150 con 0x56278831c890 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788270b10 con 0x562788321e00 2024-05-08T23:06:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.100+0000 7f5280501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278826c400 con 0x562788322720 2024-05-08T23:06:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48296/0 (socket says 172.21.15.46:48296) 2024-05-08T23:06:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 -- 172.21.15.46:0/2307555897 learned_addr learned my addr 172.21.15.46:0/2307555897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5266800700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 -- 172.21.15.46:0/2307555897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 msgr2=0x5627883221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265e00700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x5627883221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x5627883221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 -- 172.21.15.46:0/2307555897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 msgr2=0x56278831ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265400700 1 -- 172.21.15.46:0/2307555897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627880ed3b0 con 0x562788322720 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5265e00700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x5627883221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.101+0000 7f5266800700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.102+0000 7f5265400700 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5254009990 tx=0x7f5254005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f320f4382494bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.102+0000 7f5266800700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 0x7f524c0a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260011de0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.102+0000 7f5253000700 1 -- 172.21.15.46:0/2307555897 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525401d070 con 0x562788322720 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.102+0000 7f5253000700 1 -- 172.21.15.46:0/2307555897 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525401e040 con 0x562788322720 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.102+0000 7f5253000700 1 -- 172.21.15.46:0/2307555897 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254004c40 con 0x562788322720 2024-05-08T23:06:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 -- 172.21.15.46:0/2307555897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 msgr2=0x56278831c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5254009990 tx=0x7f5254005f00 comp rx=0 tx=0).stop 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 -- 172.21.15.46:0/2307555897 shutdown_connections 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x5627883221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 --2- 172.21.15.46:0/2307555897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278831c350 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 -- 172.21.15.46:0/2307555897 >> 172.21.15.46:0/2307555897 conn(0x56278827a210 msgr2=0x562788316df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 -- 172.21.15.46:0/2307555897 shutdown_connections 2024-05-08T23:06:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.103+0000 7f5280501b80 1 -- 172.21.15.46:0/2307555897 wait complete. 2024-05-08T23:06:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.104+0000 7f5280501b80 1 -- start start 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.104+0000 7f5280501b80 1 Processor -- start 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.104+0000 7f5280501b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x562788325ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.104+0000 7f5280501b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788326000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.104+0000 7f5280501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5280501b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788208560 con 0x56278831c890 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5280501b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788208200 con 0x562788321e00 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5280501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627880ef6a0 con 0x562788322720 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5280501b80 10 client.0 _ll_register_callbacks cb 0x562788211b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788326000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788326000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56204/0 (socket says 172.21.15.46:56204) 2024-05-08T23:06:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5265400700 1 -- 172.21.15.46:0/1603089356 learned_addr learned my addr 172.21.15.46:0/1603089356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5265e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x562788325ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48306/0 (socket says 172.21.15.46:48306) 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 -- 172.21.15.46:0/1603089356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 msgr2=0x562788326000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788326000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 -- 172.21.15.46:0/1603089356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 msgr2=0x562788325ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x562788325ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.105+0000 7f5266800700 1 -- 172.21.15.46:0/1603089356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526810e5b0 con 0x562788322720 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5266800700 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5260018640 tx=0x7f5260004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3138763b1ed313d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5247400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000e040 con 0x562788322720 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627881c5270 con 0x562788322720 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f52600047a0 con 0x562788322720 2024-05-08T23:06:52.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5260009070 con 0x562788322720 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5280501b80 10 client.26575 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562788099b60 con 0x562788322720 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.106+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627881c5270 con 0x562788322720 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f526001a440 con 0x562788322720 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 10 client.26575 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 10 client.26575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 20 client.26575 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5627883361c0 0x5627883385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 4 client.26575 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627882eb810 con 0x5627883361c0 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5265e00700 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5627883361c0 0x5627883385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5247400700 10 client.26575 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.107+0000 7f5265e00700 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5627883361c0 0x5627883385e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=695380fead273959 server_cookie=1207e9b6c2e2880c in_seq=0 out_seq=0 2024-05-08T23:06:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.108+0000 7f5247400700 1 -- 172.21.15.46:0/1603089356 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627882eb810 con 0x5627883361c0 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.108+0000 7f5247400700 10 client.26575 handle_command_reply: tid=0 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.108+0000 7f5280501b80 1 client.26575 shutdown 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.108+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 msgr2=0x562788326540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.108+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5260018640 tx=0x7f5260004b40 comp rx=0 tx=0).stop 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 shutdown_connections 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788326540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5627883361c0 0x5627883385e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x562788325ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 --2- 172.21.15.46:0/1603089356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788326000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 >> 172.21.15.46:0/1603089356 conn(0x56278827a210 msgr2=0x5627883165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5247400700 0 client.26575 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 shutdown_connections 2024-05-08T23:06:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 1 -- 172.21.15.46:0/1603089356 wait complete. 2024-05-08T23:06:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.109+0000 7f5280501b80 20 client.26575 trim_cache size 0 max 16384 2024-05-08T23:06:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 Processor -- start 2024-05-08T23:06:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 -- start start 2024-05-08T23:06:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788316d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788318a70 con 0x56278831c890 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788313dc0 con 0x562788321e00 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5280501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562788322560 con 0x562788322720 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788316d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5265e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788316d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48312/0 (socket says 172.21.15.46:48312) 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5266800700 1 -- 172.21.15.46:0/2321413418 learned_addr learned my addr 172.21.15.46:0/2321413418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.125+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56216/0 (socket says 172.21.15.46:56216) 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 -- 172.21.15.46:0/2321413418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 msgr2=0x562788316d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788316d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 -- 172.21.15.46:0/2321413418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 msgr2=0x56278831c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 -- 172.21.15.46:0/2321413418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562788232560 con 0x562788321e00 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265e00700 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.126+0000 7f5265400700 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x562788327100 tx=0x7f525400b2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ac5a3713fdfcec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.127+0000 7f5253a00700 1 -- 172.21.15.46:0/2321413418 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525401d040 con 0x562788321e00 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.127+0000 7f5253a00700 1 -- 172.21.15.46:0/2321413418 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52540058e0 con 0x562788321e00 2024-05-08T23:06:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.127+0000 7f5253a00700 1 -- 172.21.15.46:0/2321413418 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525400f460 con 0x562788321e00 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.127+0000 7f5280501b80 1 -- 172.21.15.46:0/2321413418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 msgr2=0x562788324c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.127+0000 7f5280501b80 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x562788327100 tx=0x7f525400b2f0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 -- 172.21.15.46:0/2321413418 shutdown_connections 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x562788316d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278831c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 --2- 172.21.15.46:0/2321413418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x562788324c20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 -- 172.21.15.46:0/2321413418 >> 172.21.15.46:0/2321413418 conn(0x56278827a210 msgr2=0x562788320bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 -- 172.21.15.46:0/2321413418 shutdown_connections 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 -- 172.21.15.46:0/2321413418 wait complete. 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 -- start start 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.128+0000 7f5280501b80 1 Processor -- start 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278832b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x56278832b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278832be50 con 0x56278831c890 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278838bbc0 con 0x562788321e00 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56278838bd50 con 0x562788322720 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5280501b80 10 client.0 _ll_register_callbacks cb 0x562788211b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x56278832b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5265400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x56278832b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56230/0 (socket says 172.21.15.46:56230) 2024-05-08T23:06:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5266800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48316/0 (socket says 172.21.15.46:48316) 2024-05-08T23:06:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5265400700 1 -- 172.21.15.46:0/2886443085 learned_addr learned my addr 172.21.15.46:0/2886443085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.129+0000 7f5265e00700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278832b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 -- 172.21.15.46:0/2886443085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 msgr2=0x56278832b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x56278832b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 -- 172.21.15.46:0/2886443085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 msgr2=0x56278832b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278832b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 -- 172.21.15.46:0/2886443085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540095f0 con 0x562788322720 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5266800700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5627882e1d40 tx=0x7f526001a670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e36cdac874f63e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5247400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600117c0 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f526001aa90 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.130+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f525401e040 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5260009320 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 10 client.26581 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5627880ed3b0 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525401e040 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5260004030 con 0x562788322720 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 10 client.26581 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 10 client.26581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 20 client.26581 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56278838d510 0x56278838f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 4 client.26581 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562788264ec0 con 0x56278838d510 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.131+0000 7f5265e00700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56278838d510 0x56278838f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5247400700 10 client.26581 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5265e00700 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56278838d510 0x56278838f930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=179f5b4f5fca6d5d server_cookie=8988b57a36643be2 in_seq=0 out_seq=0 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5247400700 1 -- 172.21.15.46:0/2886443085 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562788264ec0 con 0x56278838d510 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5247400700 10 client.26581 handle_command_reply: tid=0 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 client.26581 shutdown 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 msgr2=0x56278838b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5627882e1d40 tx=0x7f526001a670 comp rx=0 tx=0).stop 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 shutdown_connections 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562788322720 0x56278838b680 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56278838d510 0x56278838f930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56278831c890 0x56278832b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 --2- 172.21.15.46:0/2886443085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562788321e00 0x56278832b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 >> 172.21.15.46:0/2886443085 conn(0x56278827a210 msgr2=0x5627883263a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.132+0000 7f5247400700 0 client.26581 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:06:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 shutdown_connections 2024-05-08T23:06:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f5280501b80 1 -- 172.21.15.46:0/2886443085 wait complete. 2024-05-08T23:06:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f5280501b80 20 client.26581 trim_cache size 0 max 16384 2024-05-08T23:06:52.136 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 msgr2=0x7f524c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 0x7f524c0a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260011de0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 msgr2=0x7f52681995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.133+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f525800eb30 tx=0x7f525800ee40 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 shutdown_connections 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526819de60 0x7f526819e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f524c0a1290 0x7f524c0a3750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52681010a0 0x7f52681995d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 --2- 172.21.15.46:0/3820045058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52681061c0 0x7f5268199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 >> 172.21.15.46:0/3820045058 conn(0x7f52680fc960 msgr2=0x7f52680fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 shutdown_connections 2024-05-08T23:06:52.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.134+0000 7f526ca00700 1 -- 172.21.15.46:0/3820045058 wait complete. 2024-05-08T23:06:52.151 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:51 smithi046 ceph-mon[36898]: pgmap v94: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.9 MiB/s wr, 414 op/s 2024-05-08T23:06:52.151 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:51 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3032572177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:52.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:06:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:51 smithi086 ceph-mon[78610]: pgmap v94: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.9 MiB/s wr, 414 op/s 2024-05-08T23:06:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:51 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3032572177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:51 smithi053 ceph-mon[77587]: pgmap v94: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.9 MiB/s wr, 414 op/s 2024-05-08T23:06:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:51 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3032572177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.418+0000 7f738dc00700 1 -- 172.21.15.46:0/1780417149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 msgr2=0x7f7388102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.418+0000 7f738dc00700 1 --2- 172.21.15.46:0/1780417149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f7388102380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7378009990 tx=0x7f7378009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.418+0000 7f738dc00700 1 -- 172.21.15.46:0/1780417149 shutdown_connections 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.418+0000 7f738dc00700 1 --2- 172.21.15.46:0/1780417149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 0x7f738810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.418+0000 7f738dc00700 1 --2- 172.21.15.46:0/1780417149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f7388102380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.419+0000 7f738dc00700 1 --2- 172.21.15.46:0/1780417149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 0x7f738810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.419+0000 7f738dc00700 1 -- 172.21.15.46:0/1780417149 >> 172.21.15.46:0/1780417149 conn(0x7f73880fb760 msgr2=0x7f73880fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.419+0000 7f738dc00700 1 -- 172.21.15.46:0/1780417149 shutdown_connections 2024-05-08T23:06:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.419+0000 7f738dc00700 1 -- 172.21.15.46:0/1780417149 wait complete. 2024-05-08T23:06:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.420+0000 7f738dc00700 1 Processor -- start 2024-05-08T23:06:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.420+0000 7f738dc00700 1 -- start start 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.420+0000 7f738dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f738dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 0x7f7388197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f738dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 0x7f738819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f738dc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819c1f0 con 0x7f738810b3c0 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f738dc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819c360 con 0x7f73881028c0 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f738dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819c520 con 0x7f7388101fa0 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 0x7f7388197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48334/0 (socket says 172.21.15.46:48334) 2024-05-08T23:06:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 -- 172.21.15.46:0/1066949 learned_addr learned my addr 172.21.15.46:0/1066949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7387400700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 0x7f738819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 -- 172.21.15.46:0/1066949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 msgr2=0x7f7388197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 0x7f7388197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 -- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 msgr2=0x7f738819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 0x7f738819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.421+0000 7f7386a00700 1 -- 172.21.15.46:0/1066949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780095f0 con 0x7f7388101fa0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f7386a00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7378005910 tx=0x7f7378004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f737f400700 1 -- 172.21.15.46:0/1066949 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737801d070 con 0x7f7388101fa0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f737f400700 1 -- 172.21.15.46:0/1066949 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7378022470 con 0x7f7388101fa0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f737f400700 1 -- 172.21.15.46:0/1066949 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378017650 con 0x7f7388101fa0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738819c7a0 con 0x7f7388101fa0 2024-05-08T23:06:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.422+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738819ca20 con 0x7f7388101fa0 2024-05-08T23:06:52.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.425+0000 7f737f400700 1 -- 172.21.15.46:0/1066949 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f73780225e0 con 0x7f7388101fa0 2024-05-08T23:06:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 Processor -- start 2024-05-08T23:06:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f737f400700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 0x7f736c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 -- start start 2024-05-08T23:06:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f737f400700 1 -- 172.21.15.46:0/1066949 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f73780ce320 con 0x7f7388101fa0 2024-05-08T23:06:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f7387400700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 0x7f736c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x5587747555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587746a2150 con 0x558774755210 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877469fb10 con 0x558774755b30 2024-05-08T23:06:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.427+0000 7f73a173cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877469b400 con 0x55877474fca0 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x5587747555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56264/0 (socket says 172.21.15.46:56264) 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 -- 172.21.15.46:0/3095346666 learned_addr learned my addr 172.21.15.46:0/3095346666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7387400700 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 -- 172.21.15.46:0/3095346666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 msgr2=0x5587747500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 -- 172.21.15.46:0/3095346666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 msgr2=0x5587747555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x5587747555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 -- 172.21.15.46:0/3095346666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877451c3b0 con 0x558774755b30 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386a00700 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x5587747555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7386000700 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f737400bd20 tx=0x7f73740106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65624c9e9ba488b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.428+0000 7f7387400700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 0x7f736c0a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7380013de0 tx=0x7f7380013460 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.429+0000 7f7373e00700 1 -- 172.21.15.46:0/3095346666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374010e10 con 0x558774755b30 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.429+0000 7f7373e00700 1 -- 172.21.15.46:0/3095346666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7374015070 con 0x558774755b30 2024-05-08T23:06:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.429+0000 7f7373e00700 1 -- 172.21.15.46:0/3095346666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374003c80 con 0x558774755b30 2024-05-08T23:06:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 -- 172.21.15.46:0/3095346666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 msgr2=0x55877474f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f737400bd20 tx=0x7f73740106b0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 -- 172.21.15.46:0/3095346666 shutdown_connections 2024-05-08T23:06:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x5587747555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3095346666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877474f760 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 -- 172.21.15.46:0/3095346666 >> 172.21.15.46:0/3095346666 conn(0x5587746a9210 msgr2=0x558774745df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 -- 172.21.15.46:0/3095346666 shutdown_connections 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.430+0000 7f73a173cb80 1 -- 172.21.15.46:0/3095346666 wait complete. 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.431+0000 7f73a173cb80 1 -- start start 2024-05-08T23:06:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.431+0000 7f73a173cb80 1 Processor -- start 2024-05-08T23:06:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x55877475bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 0x558774758e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 0x558774759580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774637560 con 0x558774755b30 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774637200 con 0x558774755210 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877451e6a0 con 0x55877474fca0 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f73a173cb80 10 client.0 _ll_register_callbacks cb 0x558774640b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 0x558774758e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x55877475bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 0x558774758e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56280/0 (socket says 172.21.15.46:56280) 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386000700 1 -- 172.21.15.46:0/1195275139 learned_addr learned my addr 172.21.15.46:0/1195275139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7387400700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 0x558774759580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 -- 172.21.15.46:0/1195275139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 msgr2=0x558774758e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 0x558774758e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 -- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 msgr2=0x558774759580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 0x558774759580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 -- 172.21.15.46:0/1195275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738819c7a0 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7387400700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 0x558774759580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7386a00700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x55877475bbc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f73780cb660 tx=0x7f7378022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a667ab7d26d66fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.432+0000 7f7370c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737809b8d0 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f737809b1d0 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587745f4270 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f737809f000 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 10 client.26593 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f737400b950 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737807f020 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f737809a6c0 con 0x55877474fca0 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 10 client.26593 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 10 client.26593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 20 client.26593 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747692d0 0x55877476b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 4 client.26593 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.433+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55877471a810 con 0x5587747692d0 2024-05-08T23:06:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.434+0000 7f7386000700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747692d0 0x55877476b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.434+0000 7f7370c00700 10 client.26593 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.434+0000 7f7386000700 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747692d0 0x55877476b6f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92c35e19e5a2e39a server_cookie=14447cc9e7a7c6d5 in_seq=0 out_seq=0 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.435+0000 7f7370c00700 1 -- 172.21.15.46:0/1195275139 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55877471a810 con 0x5587747692d0 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.435+0000 7f7370c00700 10 client.26593 handle_command_reply: tid=0 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.435+0000 7f73a173cb80 1 client.26593 shutdown 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.435+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 msgr2=0x55877475bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.435+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x55877475bbc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f73780cb660 tx=0x7f7378022c90 comp rx=0 tx=0).stop 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 shutdown_connections 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747692d0 0x55877476b6f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755b30 0x558774759580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f7370c00700 0 client.26593 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x55877475bbc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1195275139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755210 0x558774758e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 >> 172.21.15.46:0/1195275139 conn(0x5587746a9210 msgr2=0x5587747455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 shutdown_connections 2024-05-08T23:06:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 1 -- 172.21.15.46:0/1195275139 wait complete. 2024-05-08T23:06:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.436+0000 7f73a173cb80 20 client.26593 trim_cache size 0 max 16384 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.451+0000 7f73a173cb80 1 Processor -- start 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.451+0000 7f73a173cb80 1 -- start start 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.451+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.451+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774745d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877475b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f73a173cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774740890 con 0x558774755210 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f73a173cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774759b80 con 0x558774755b30 2024-05-08T23:06:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f73a173cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774740b20 con 0x55877474fca0 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774745d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48360/0 (socket says 172.21.15.46:48360) 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 -- 172.21.15.46:0/3341212130 learned_addr learned my addr 172.21.15.46:0/3341212130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7387400700 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877475b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 -- 172.21.15.46:0/3341212130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 msgr2=0x55877475b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877475b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 -- 172.21.15.46:0/3341212130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 msgr2=0x558774745d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774745d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 -- 172.21.15.46:0/3341212130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558774750350 con 0x55877474fca0 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.452+0000 7f7386a00700 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7378005cd0 tx=0x7f737809ae40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5beab8a3ae88f37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f737cc00700 1 -- 172.21.15.46:0/3341212130 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780a1e60 con 0x55877474fca0 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f737cc00700 1 -- 172.21.15.46:0/3341212130 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737807f030 con 0x55877474fca0 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f737cc00700 1 -- 172.21.15.46:0/3341212130 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780a35c0 con 0x55877474fca0 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 -- 172.21.15.46:0/3341212130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 msgr2=0x558774754660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7378005cd0 tx=0x7f737809ae40 comp rx=0 tx=0).stop 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 -- 172.21.15.46:0/3341212130 shutdown_connections 2024-05-08T23:06:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x55877475b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x558774754660 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 --2- 172.21.15.46:0/3341212130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774745d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 -- 172.21.15.46:0/3341212130 >> 172.21.15.46:0/3341212130 conn(0x5587746a9210 msgr2=0x558774753f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 -- 172.21.15.46:0/3341212130 shutdown_connections 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.453+0000 7f73a173cb80 1 -- 172.21.15.46:0/3341212130 wait complete. 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.454+0000 7f73a173cb80 1 -- start start 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.454+0000 7f73a173cb80 1 Processor -- start 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774760b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x5587747511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587747518c0 con 0x558774755210 2024-05-08T23:06:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774751a30 con 0x558774755b30 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774751bf0 con 0x55877474fca0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f73a173cb80 10 client.0 _ll_register_callbacks cb 0x558774640b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48376/0 (socket says 172.21.15.46:48376) 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 -- 172.21.15.46:0/1597271022 learned_addr learned my addr 172.21.15.46:0/1597271022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386000700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774760b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7387400700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x5587747511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 -- 172.21.15.46:0/1597271022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 msgr2=0x5587747511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x5587747511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 -- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 msgr2=0x558774760b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774760b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386a00700 1 -- 172.21.15.46:0/1597271022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737400b950 con 0x55877474fca0 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.455+0000 7f7386000700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774760b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7386a00700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f73780a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb41a25f374c8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780a33d0 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7378080080 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7374015070 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f737809b8d0 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 10 client.26599 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587744c8b60 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7378088080 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f737809b550 con 0x55877474fca0 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 10 client.26599 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 10 client.26599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 20 client.26599 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.456+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747c0580 0x5587747c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.457+0000 7f73a173cb80 4 client.26599 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:52.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.457+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558774693ec0 con 0x5587747c0580 2024-05-08T23:06:52.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.457+0000 7f7386000700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747c0580 0x5587747c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.457+0000 7f7386000700 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747c0580 0x5587747c2970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=559a3ea22ca4a0e1 server_cookie=65f7c884453d32fe in_seq=0 out_seq=0 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.457+0000 7f7370c00700 10 client.26599 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f7370c00700 1 -- 172.21.15.46:0/1597271022 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558774693ec0 con 0x5587747c0580 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f7370c00700 10 client.26599 handle_command_reply: tid=0 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 client.26599 shutdown 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 msgr2=0x5587747605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f73780a1040 comp rx=0 tx=0).stop 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 shutdown_connections 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558774755b30 0x5587747511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55877474fca0 0x5587747605e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5587747c0580 0x5587747c2970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 --2- 172.21.15.46:0/1597271022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558774755210 0x558774760b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f7370c00700 0 client.26599 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 >> 172.21.15.46:0/1597271022 conn(0x5587746a9210 msgr2=0x558774753f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 shutdown_connections 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 1 -- 172.21.15.46:0/1597271022 wait complete. 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.458+0000 7f73a173cb80 20 client.26599 trim_cache size 0 max 16384 2024-05-08T23:06:52.463 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 msgr2=0x7f736c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 0x7f736c0a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7380013de0 tx=0x7f7380013460 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 msgr2=0x7f73881972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7378005910 tx=0x7f7378004730 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 shutdown_connections 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f736c0a1310 0x7f736c0a37d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738810b3c0 0x7f738819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7388101fa0 0x7f73881972d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 --2- 172.21.15.46:0/1066949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73881028c0 0x7f7388197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 >> 172.21.15.46:0/1066949 conn(0x7f73880fb760 msgr2=0x7f73880fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 shutdown_connections 2024-05-08T23:06:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.459+0000 7f738dc00700 1 -- 172.21.15.46:0/1066949 wait complete. 2024-05-08T23:06:52.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:06:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 -- 172.21.15.46:0/1743525417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc101080 msgr2=0x7fd3dc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 --2- 172.21.15.46:0/1743525417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc101080 0x7fd3dc101460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009990 tx=0x7fd3cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 -- 172.21.15.46:0/1743525417 shutdown_connections 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 --2- 172.21.15.46:0/1743525417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 --2- 172.21.15.46:0/1743525417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc101080 0x7fd3dc101460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 --2- 172.21.15.46:0/1743525417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc101a30 0x7fd3dc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 -- 172.21.15.46:0/1743525417 >> 172.21.15.46:0/1743525417 conn(0x7fd3dc0fc8e0 msgr2=0x7fd3dc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 -- 172.21.15.46:0/1743525417 shutdown_connections 2024-05-08T23:06:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.695+0000 7fd3e1400700 1 -- 172.21.15.46:0/1743525417 wait complete. 2024-05-08T23:06:52.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 Processor -- start 2024-05-08T23:06:52.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 -- start start 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc101080 0x7fd3dc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc074f60 0x7fd3dc073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc1a9f20 con 0x7fd3dc101080 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc1aa090 con 0x7fd3dc074f60 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.696+0000 7fd3e1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc1aa250 con 0x7fd3dc1061a0 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48394/0 (socket says 172.21.15.46:48394) 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3da800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc101080 0x7fd3dc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 -- 172.21.15.46:0/2482182120 learned_addr learned my addr 172.21.15.46:0/2482182120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3db200700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc074f60 0x7fd3dc073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 -- 172.21.15.46:0/2482182120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc074f60 msgr2=0x7fd3dc073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc074f60 0x7fd3dc073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 -- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc101080 msgr2=0x7fd3dc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc101080 0x7fd3dc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 -- 172.21.15.46:0/2482182120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0095f0 con 0x7fd3dc1061a0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.697+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800e970 tx=0x7fd3c800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.698+0000 7fd3d3400700 1 -- 172.21.15.46:0/2482182120 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8009790 con 0x7fd3dc1061a0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.698+0000 7fd3d3400700 1 -- 172.21.15.46:0/2482182120 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3c800c7e0 con 0x7fd3dc1061a0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.698+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc1aa530 con 0x7fd3dc1061a0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.698+0000 7fd3d3400700 1 -- 172.21.15.46:0/2482182120 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80054e0 con 0x7fd3dc1061a0 2024-05-08T23:06:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.698+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc1aaa90 con 0x7fd3dc1061a0 2024-05-08T23:06:52.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.701+0000 7fd3d3400700 1 -- 172.21.15.46:0/2482182120 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd3c800c950 con 0x7fd3dc1061a0 2024-05-08T23:06:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.703+0000 7fd3f4e83b80 1 Processor -- start 2024-05-08T23:06:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.703+0000 7fd3f4e83b80 1 -- start start 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.703+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.703+0000 7fd3d3400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 0x7fd3c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3a86150 con 0x5586a3b2f890 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3a83b10 con 0x5586a3b34e00 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3a7f400 con 0x5586a3b35720 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3d3400700 1 -- 172.21.15.46:0/2482182120 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd3c8014070 con 0x7fd3dc1061a0 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3da800700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 0x7fd3c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59458/0 (socket says 172.21.15.46:59458) 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 -- 172.21.15.46:0/2880731567 learned_addr learned my addr 172.21.15.46:0/2880731567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 -- 172.21.15.46:0/2880731567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 msgr2=0x5586a3b2f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b2f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 -- 172.21.15.46:0/2880731567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 msgr2=0x5586a3b351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.704+0000 7fd3db200700 1 -- 172.21.15.46:0/2880731567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586a39003b0 con 0x5586a3b2f890 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.705+0000 7fd3db200700 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4007b40 tx=0x7fd3d4009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd799ba2ffcf3c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.705+0000 7fd3da800700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 0x7fd3c00a37d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b2e0 tx=0x7fd3cc005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.705+0000 7fd3c7400700 1 -- 172.21.15.46:0/2880731567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004db0 con 0x5586a3b2f890 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.705+0000 7fd3c7400700 1 -- 172.21.15.46:0/2880731567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d400f050 con 0x5586a3b2f890 2024-05-08T23:06:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.705+0000 7fd3c7400700 1 -- 172.21.15.46:0/2880731567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004650 con 0x5586a3b2f890 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2880731567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 msgr2=0x5586a3b2fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4007b40 tx=0x7fd3d4009130 comp rx=0 tx=0).stop 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2880731567 shutdown_connections 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b2f890 0x5586a3b2fcd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2880731567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2880731567 >> 172.21.15.46:0/2880731567 conn(0x5586a3a8d210 msgr2=0x5586a3b29df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2880731567 shutdown_connections 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2880731567 wait complete. 2024-05-08T23:06:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.706+0000 7fd3f4e83b80 1 -- start start 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 Processor -- start 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b38b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3a1b560 con 0x5586a3b34e00 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3a1b200 con 0x5586a3b2f890 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a39026a0 con 0x5586a3b35720 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3f4e83b80 10 client.0 _ll_register_callbacks cb 0x5586a3a24b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3da800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b38b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3db200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59472/0 (socket says 172.21.15.46:59472) 2024-05-08T23:06:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.707+0000 7fd3d9e00700 1 -- 172.21.15.46:0/2854493901 learned_addr learned my addr 172.21.15.46:0/2854493901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 -- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 msgr2=0x5586a3b38b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b38b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 -- 172.21.15.46:0/2854493901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 msgr2=0x5586a3b390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 -- 172.21.15.46:0/2854493901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d4007800 con 0x5586a3b35720 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3db200700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b395f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4004270 tx=0x7fd3d4006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc604eb3fc204e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.708+0000 7fd3bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004020 con 0x5586a3b35720 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd3d40094e0 con 0x5586a3b35720 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586a39d8270 con 0x5586a3b35720 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d40044e0 con 0x5586a3b35720 2024-05-08T23:06:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3f4e83b80 10 client.26611 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5586a38acb60 con 0x5586a3b35720 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586a39d8270 con 0x5586a3b35720 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.709+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd3d402d060 con 0x5586a3b35720 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 20 client.26611 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b49180 0x5586a3b4b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 4 client.26611 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586a3afe810 con 0x5586a3b49180 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3da800700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b49180 0x5586a3b4b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3da800700 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b49180 0x5586a3b4b5a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9774e4d898e756fb server_cookie=4c1d75b793c24237 in_seq=0 out_seq=0 2024-05-08T23:06:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.710+0000 7fd3bbe00700 10 client.26611 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.711+0000 7fd3bbe00700 1 -- 172.21.15.46:0/2854493901 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586a3afe810 con 0x5586a3b49180 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.711+0000 7fd3bbe00700 10 client.26611 handle_command_reply: tid=0 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 client.26611 shutdown 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 msgr2=0x5586a3b395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b395f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4004270 tx=0x7fd3d4006070 comp rx=0 tx=0).stop 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 shutdown_connections 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b395f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b38b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b49180 0x5586a3b4b5a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/2854493901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 >> 172.21.15.46:0/2854493901 conn(0x5586a3a8d210 msgr2=0x5586a3b295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3bbe00700 0 client.26611 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 shutdown_connections 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/2854493901 wait complete. 2024-05-08T23:06:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.712+0000 7fd3f4e83b80 20 client.26611 trim_cache size 0 max 16384 2024-05-08T23:06:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 Processor -- start 2024-05-08T23:06:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 -- start start 2024-05-08T23:06:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b29d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b48ff0 con 0x5586a3b34e00 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b24830 con 0x5586a3b2f890 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.729+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b32870 con 0x5586a3b35720 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3da800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b29d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48426/0 (socket says 172.21.15.46:48426) 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 -- 172.21.15.46:0/1078136253 learned_addr learned my addr 172.21.15.46:0/1078136253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3da800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b2f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56358/0 (socket says 172.21.15.46:56358) 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 -- 172.21.15.46:0/1078136253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 msgr2=0x5586a3b2f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b2f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 -- 172.21.15.46:0/1078136253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 msgr2=0x5586a3b29d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b29d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 -- 172.21.15.46:0/1078136253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586a3a45560 con 0x5586a3b35720 2024-05-08T23:06:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3db200700 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4002230 tx=0x7fd3d40091b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2dde8e942589b41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.730+0000 7fd3c7e00700 1 -- 172.21.15.46:0/1078136253 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40193d0 con 0x5586a3b35720 2024-05-08T23:06:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3c7e00700 1 -- 172.21.15.46:0/1078136253 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d4019530 con 0x5586a3b35720 2024-05-08T23:06:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3c7e00700 1 -- 172.21.15.46:0/1078136253 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40175d0 con 0x5586a3b35720 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/1078136253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 msgr2=0x5586a3b390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4002230 tx=0x7fd3d40091b0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/1078136253 shutdown_connections 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b35720 0x5586a3b390e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b2f890 0x5586a3b2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/1078136253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b34e00 0x5586a3b29d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/1078136253 >> 172.21.15.46:0/1078136253 conn(0x5586a3a8d210 msgr2=0x5586a3b37460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.731+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/1078136253 shutdown_connections 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.732+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/1078136253 wait complete. 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.732+0000 7fd3f4e83b80 1 -- start start 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.732+0000 7fd3f4e83b80 1 Processor -- start 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b40650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b3e9d0 con 0x5586a3b35720 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b3eb40 con 0x5586a3b34e00 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a3b3ed00 con 0x5586a3b2f890 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3f4e83b80 10 client.0 _ll_register_callbacks cb 0x5586a3a24b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3db200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b40650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48428/0 (socket says 172.21.15.46:48428) 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 -- 172.21.15.46:0/611128598 learned_addr learned my addr 172.21.15.46:0/611128598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3db200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59494/0 (socket says 172.21.15.46:59494) 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 -- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 msgr2=0x5586a3b40650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b40650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 -- 172.21.15.46:0/611128598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 msgr2=0x5586a3b3e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3da800700 1 -- 172.21.15.46:0/611128598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586a38acb60 con 0x5586a3b2f890 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.733+0000 7fd3db200700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3da800700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005640 tx=0x7fd3cc01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5647fedfc83dabda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc035340 con 0x5586a3b2f890 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3d400f050 con 0x5586a3b2f890 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fd3cc0354a0 con 0x5586a3b2f890 2024-05-08T23:06:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3cc035660 con 0x5586a3b2f890 2024-05-08T23:06:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3f4e83b80 10 client.26617 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5586a3904cd0 con 0x5586a3b2f890 2024-05-08T23:06:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.734+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d400f050 con 0x5586a3b2f890 2024-05-08T23:06:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd3cc00f460 con 0x5586a3b2f890 2024-05-08T23:06:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 10 client.26617 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 10 client.26617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 20 client.26617 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b9f9d0 0x5586a3ba1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 4 client.26617 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586a3a77ec0 con 0x5586a3b9f9d0 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.735+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b9f9d0 0x5586a3ba1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3d9e00700 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b9f9d0 0x5586a3ba1df0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45e0d1fa3e99f65 server_cookie=812f3d6b841813a1 in_seq=0 out_seq=0 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3bbe00700 10 client.26617 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3bbe00700 1 -- 172.21.15.46:0/611128598 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586a3a77ec0 con 0x5586a3b9f9d0 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3bbe00700 10 client.26617 handle_command_reply: tid=0 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 client.26617 shutdown 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 msgr2=0x5586a3b40110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc005640 tx=0x7fd3cc01a050 comp rx=0 tx=0).stop 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 shutdown_connections 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586a3b9f9d0 0x5586a3ba1df0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586a3b35720 0x5586a3b3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a3b2f890 0x5586a3b40110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3bbe00700 0 client.26617 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 --2- 172.21.15.46:0/611128598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5586a3b34e00 0x5586a3b40650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 >> 172.21.15.46:0/611128598 conn(0x5586a3a8d210 msgr2=0x5586a3b33c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 shutdown_connections 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 1 -- 172.21.15.46:0/611128598 wait complete. 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.736+0000 7fd3f4e83b80 20 client.26617 trim_cache size 0 max 16384 2024-05-08T23:06:52.740 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 msgr2=0x7fd3c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 0x7fd3c00a37d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b2e0 tx=0x7fd3cc005fd0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 msgr2=0x7fd3dc0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800e970 tx=0x7fd3c800ec80 comp rx=0 tx=0).stop 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 shutdown_connections 2024-05-08T23:06:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc074f60 0x7fd3dc073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3c00a1310 0x7fd3c00a37d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3dc101080 0x7fd3dc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 --2- 172.21.15.46:0/2482182120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc1061a0 0x7fd3dc0737e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 >> 172.21.15.46:0/2482182120 conn(0x7fd3dc0fc8e0 msgr2=0x7fd3dc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 shutdown_connections 2024-05-08T23:06:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:52.737+0000 7fd3e1400700 1 -- 172.21.15.46:0/2482182120 wait complete. 2024-05-08T23:06:52.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 117812 8.81 MB/sec warmup 74 sec latency 12.626 ms 2024-05-08T23:06:53.757 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:06:53.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:06:53.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 120417 8.89 MB/sec warmup 75 sec latency 14.037 ms 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 -- 172.21.15.46:0/1557587485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c981061a0 msgr2=0x7f5c9810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1557587485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c981061a0 0x7f5c9810dd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009f90 tx=0x7f5c9000b330 comp rx=0 tx=0).stop 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 -- 172.21.15.46:0/1557587485 shutdown_connections 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1557587485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c981061a0 0x7f5c9810dd20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1557587485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c98101080 0x7f5c98101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1557587485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101a30 0x7f5c98105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 -- 172.21.15.46:0/1557587485 >> 172.21.15.46:0/1557587485 conn(0x7f5c980fc8e0 msgr2=0x7f5c980fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.990+0000 7f5c9d800700 1 -- 172.21.15.46:0/1557587485 shutdown_connections 2024-05-08T23:06:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.991+0000 7f5c9d800700 1 -- 172.21.15.46:0/1557587485 wait complete. 2024-05-08T23:06:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 Processor -- start 2024-05-08T23:06:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 -- start start 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981061a0 0x7f5c98199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 0x7f5c9819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9819e9a0 con 0x7f5c9819df20 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9819eb10 con 0x7f5c981061a0 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.992+0000 7f5c9d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9819ecd0 con 0x7f5c98101080 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48448/0 (socket says 172.21.15.46:48448) 2024-05-08T23:06:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 -- 172.21.15.46:0/1872395242 learned_addr learned my addr 172.21.15.46:0/1872395242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96000700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981061a0 0x7f5c98199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c97400700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 0x7f5c9819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 -- 172.21.15.46:0/1872395242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981061a0 msgr2=0x7f5c98199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981061a0 0x7f5c98199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 -- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 msgr2=0x7f5c9819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 0x7f5c9819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 -- 172.21.15.46:0/1872395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840095f0 con 0x7f5c98101080 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c97400700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 0x7f5c9819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c96a00700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800e9a0 tx=0x7f5c8800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.993+0000 7f5c8f400700 1 -- 172.21.15.46:0/1872395242 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88009790 con 0x7f5c98101080 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.994+0000 7f5c8f400700 1 -- 172.21.15.46:0/1872395242 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8800cbd0 con 0x7f5c98101080 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.994+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c90009c40 con 0x7f5c98101080 2024-05-08T23:06:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.994+0000 7f5c8f400700 1 -- 172.21.15.46:0/1872395242 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88018680 con 0x7f5c98101080 2024-05-08T23:06:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.994+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c981a6580 con 0x7f5c98101080 2024-05-08T23:06:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.997+0000 7f5c8f400700 1 -- 172.21.15.46:0/1872395242 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c8800cd40 con 0x7f5c98101080 2024-05-08T23:06:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.999+0000 7f5c8f400700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 0x7f5c7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.999+0000 7f5c8f400700 1 -- 172.21.15.46:0/1872395242 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c88014070 con 0x7f5c98101080 2024-05-08T23:06:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.999+0000 7f5c96000700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 0x7f5c7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:53.999+0000 7f5cb12ffb80 1 Processor -- start 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 -- start start 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c96000700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 0x7f5c7c0a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c9819ab90 tx=0x7f5c84009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fae80 0x5571f60fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f604c1d0 con 0x5571f60fae80 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f6049b90 con 0x5571f60fb7a0 2024-05-08T23:06:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f6045480 con 0x5571f60f5910 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c96000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c97400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c96000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56382/0 (socket says 172.21.15.46:56382) 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c96000700 1 -- 172.21.15.46:0/1577554655 learned_addr learned my addr 172.21.15.46:0/1577554655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.000+0000 7f5c96a00700 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fae80 0x5571f60fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 -- 172.21.15.46:0/1577554655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 msgr2=0x5571f60f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 -- 172.21.15.46:0/1577554655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fae80 msgr2=0x5571f60fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fae80 0x5571f60fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 -- 172.21.15.46:0/1577554655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571f5ec6430 con 0x5571f60fb7a0 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c97400700 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c96000700 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400e620 tx=0x7f5c84014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437e12f60626a8f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.001+0000 7f5c83e00700 1 -- 172.21.15.46:0/1577554655 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84004020 con 0x5571f60fb7a0 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5c83e00700 1 -- 172.21.15.46:0/1577554655 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c84021070 con 0x5571f60fb7a0 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5c83e00700 1 -- 172.21.15.46:0/1577554655 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8401c650 con 0x5571f60fb7a0 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/1577554655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 msgr2=0x5571f60f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400e620 tx=0x7f5c84014000 comp rx=0 tx=0).stop 2024-05-08T23:06:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/1577554655 shutdown_connections 2024-05-08T23:06:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fae80 0x5571f60fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/1577554655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60f53d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.002+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/1577554655 >> 172.21.15.46:0/1577554655 conn(0x5571f6053290 msgr2=0x5571f60efe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.003+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/1577554655 shutdown_connections 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.003+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/1577554655 wait complete. 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.003+0000 7f5cb12ffb80 1 -- start start 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.003+0000 7f5cb12ffb80 1 Processor -- start 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fae80 0x5571f60ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f60ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f5fe15e0 con 0x5571f60fb7a0 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f5fe1280 con 0x5571f60fae80 2024-05-08T23:06:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f5ec8720 con 0x5571f60f5910 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5cb12ffb80 10 client.0 _ll_register_callbacks cb 0x5571f5feabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fae80 0x5571f60ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48474/0 (socket says 172.21.15.46:48474) 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 -- 172.21.15.46:0/224913532 learned_addr learned my addr 172.21.15.46:0/224913532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c97400700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f60ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 -- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fae80 msgr2=0x5571f60ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fae80 0x5571f60ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 -- 172.21.15.46:0/224913532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 msgr2=0x5571f60ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f60ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c96a00700 1 -- 172.21.15.46:0/224913532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9810e590 con 0x5571f60f5910 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.004+0000 7f5c97400700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f60ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c96a00700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c88096ad0 tx=0x7f5c8809a7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e774783d813990f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c80c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88099750 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c8809de90 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571f5f9e2f0 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c880a24e0 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5cb12ffb80 10 client.26629 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5c84017070 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.005+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571f5f9e2f0 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c88098fc0 con 0x5571f60f5910 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 10 client.26629 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 20 client.26629 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f610f380 0x5571f61117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 4 client.26629 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571f60c4890 con 0x5571f610f380 2024-05-08T23:06:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.006+0000 7f5c96000700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f610f380 0x5571f61117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.007+0000 7f5c96000700 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f610f380 0x5571f61117a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87cf00cc2f6d530 server_cookie=ea477520499afdb2 in_seq=0 out_seq=0 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.007+0000 7f5c80c00700 10 client.26629 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5c80c00700 1 -- 172.21.15.46:0/224913532 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571f60c4890 con 0x5571f610f380 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5c80c00700 10 client.26629 handle_command_reply: tid=0 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 client.26629 shutdown 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 msgr2=0x5571f60fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c88096ad0 tx=0x7f5c8809a7e0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 shutdown_connections 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f610f380 0x5571f61117a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f60ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60f5910 0x5571f60fed10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/224913532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fae80 0x5571f60ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 >> 172.21.15.46:0/224913532 conn(0x5571f6053290 msgr2=0x5571f60ef620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5c80c00700 0 client.26629 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 shutdown_connections 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.008+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/224913532 wait complete. 2024-05-08T23:06:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.009+0000 7f5cb12ffb80 20 client.26629 trim_cache size 0 max 16384 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.031+0000 7f5cb12ffb80 1 Processor -- start 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 -- start start 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f60efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f60ece40 con 0x5571f60f5910 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f60f1860 con 0x5571f60fb7a0 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f61169b0 con 0x5571f60fae80 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59522/0 (socket says 172.21.15.46:59522) 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5c96a00700 1 -- 172.21.15.46:0/4273521949 learned_addr learned my addr 172.21.15.46:0/4273521949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.032+0000 7f5c96000700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f60efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c97400700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 -- 172.21.15.46:0/4273521949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 msgr2=0x5571f60efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f60efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 -- 172.21.15.46:0/4273521949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 msgr2=0x5571f60ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 -- 172.21.15.46:0/4273521949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571f5e72be0 con 0x5571f60f5910 2024-05-08T23:06:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.033+0000 7f5c96a00700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5c88099d10 tx=0x7f5c8809a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dd6dcc7934ca5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.034+0000 7f5c8cc00700 1 -- 172.21.15.46:0/4273521949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8809de90 con 0x5571f60f5910 2024-05-08T23:06:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.034+0000 7f5c8cc00700 1 -- 172.21.15.46:0/4273521949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8807a020 con 0x5571f60f5910 2024-05-08T23:06:54.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.034+0000 7f5c8cc00700 1 -- 172.21.15.46:0/4273521949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880a2050 con 0x5571f60f5910 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.034+0000 7f5c96000700 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f60efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/4273521949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 msgr2=0x5571f60fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5c88099d10 tx=0x7f5c8809a1a0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/4273521949 shutdown_connections 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60fb7a0 0x5571f60ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60f5910 0x5571f60fdc80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/4273521949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f60efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/4273521949 >> 172.21.15.46:0/4273521949 conn(0x5571f6053290 msgr2=0x5571f60fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/4273521949 shutdown_connections 2024-05-08T23:06:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.035+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/4273521949 wait complete. 2024-05-08T23:06:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.036+0000 7f5cb12ffb80 1 -- start start 2024-05-08T23:06:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.036+0000 7f5cb12ffb80 1 Processor -- start 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 0x5571f6105000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f616e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f6105cb0 con 0x5571f60fb7a0 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f6105e20 con 0x5571f60f5910 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571f616e5e0 con 0x5571f60fae80 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5cb12ffb80 10 client.0 _ll_register_callbacks cb 0x7f5c88099c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c96000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 0x5571f6105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c96000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48490/0 (socket says 172.21.15.46:48490) 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c96000700 1 -- 172.21.15.46:0/245731167 learned_addr learned my addr 172.21.15.46:0/245731167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c96a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 0x5571f6105000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56416/0 (socket says 172.21.15.46:56416) 2024-05-08T23:06:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.037+0000 7f5c97400700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f616e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 -- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 msgr2=0x5571f6105000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 0x5571f6105000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 -- 172.21.15.46:0/245731167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 msgr2=0x5571f616e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f616e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 -- 172.21.15.46:0/245731167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571f5ec6430 con 0x5571f60fae80 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c96000700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006010 tx=0x7f5c8400ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3b6608ee6474fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.038+0000 7f5c80c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8402c040 con 0x5571f60fae80 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c84031630 con 0x5571f60fae80 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c84021070 con 0x5571f60fae80 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c840317f0 con 0x5571f60fae80 2024-05-08T23:06:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5cb12ffb80 10 client.26635 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:54.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.039+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5c84017070 con 0x5571f60fae80 2024-05-08T23:06:54.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8403e030 con 0x5571f60fae80 2024-05-08T23:06:54.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c84038060 con 0x5571f60fae80 2024-05-08T23:06:54.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 10 client.26635 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:54.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 20 client.26635 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f616fe40 0x5571f6172260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 4 client.26635 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.040+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5571f603df40 con 0x5571f616fe40 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.041+0000 7f5c96a00700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f616fe40 0x5571f6172260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.041+0000 7f5c96a00700 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f616fe40 0x5571f6172260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163014df04947ddf server_cookie=981bc528aded606c in_seq=0 out_seq=0 2024-05-08T23:06:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.041+0000 7f5c80c00700 10 client.26635 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.042+0000 7f5c80c00700 1 -- 172.21.15.46:0/245731167 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5571f603df40 con 0x5571f616fe40 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.042+0000 7f5c80c00700 10 client.26635 handle_command_reply: tid=0 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.042+0000 7f5cb12ffb80 1 client.26635 shutdown 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.042+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 msgr2=0x5571f6105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.042+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c84006010 tx=0x7f5c8400ee20 comp rx=0 tx=0).stop 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 shutdown_connections 2024-05-08T23:06:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5571f616fe40 0x5571f6172260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5571f60fb7a0 0x5571f616e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571f60f5910 0x5571f6105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 --2- 172.21.15.46:0/245731167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571f60fae80 0x5571f6105540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5c80c00700 0 client.26635 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 >> 172.21.15.46:0/245731167 conn(0x5571f6053290 msgr2=0x5571f60fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 shutdown_connections 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 1 -- 172.21.15.46:0/245731167 wait complete. 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.043+0000 7f5cb12ffb80 20 client.26635 trim_cache size 0 max 16384 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:06:54.044 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ae94d4fe-fd7f-4aaa-aced-86e5169d2111", 2024-05-08T23:06:54.045 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:06:54.045 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 msgr2=0x7f5c7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 0x7f5c7c0a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c9819ab90 tx=0x7f5c84009310 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 msgr2=0x7f5c98199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800e9a0 tx=0x7f5c8800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 shutdown_connections 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c7c0a1290 0x7f5c7c0a3750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9819df20 0x7f5c9819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c98101080 0x7f5c98199690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 --2- 172.21.15.46:0/1872395242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981061a0 0x7f5c98199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 >> 172.21.15.46:0/1872395242 conn(0x7f5c980fc8e0 msgr2=0x7f5c980fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 shutdown_connections 2024-05-08T23:06:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.044+0000 7f5c9d800700 1 -- 172.21.15.46:0/1872395242 wait complete. 2024-05-08T23:06:54.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:06:54.133 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:53 smithi046 ceph-mon[36898]: pgmap v95: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-05-08T23:06:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:53 smithi086 ceph-mon[78610]: pgmap v95: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-05-08T23:06:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:53 smithi053 ceph-mon[77587]: pgmap v95: 129 pgs: 129 active+clean; 74 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 219 op/s 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 -- 172.21.15.46:0/3634180029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 msgr2=0x7f99c810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 --2- 172.21.15.46:0/3634180029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c810ada0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009990 tx=0x7f99b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 -- 172.21.15.46:0/3634180029 shutdown_connections 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 --2- 172.21.15.46:0/3634180029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 0x7f99c810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 --2- 172.21.15.46:0/3634180029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 --2- 172.21.15.46:0/3634180029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c810ada0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 -- 172.21.15.46:0/3634180029 >> 172.21.15.46:0/3634180029 conn(0x7f99c80fb760 msgr2=0x7f99c80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 -- 172.21.15.46:0/3634180029 shutdown_connections 2024-05-08T23:06:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.293+0000 7f99ce200700 1 -- 172.21.15.46:0/3634180029 wait complete. 2024-05-08T23:06:54.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.294+0000 7f99ce200700 1 Processor -- start 2024-05-08T23:06:54.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.294+0000 7f99ce200700 1 -- start start 2024-05-08T23:06:54.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.294+0000 7f99ce200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.294+0000 7f99ce200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99ce200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 0x7f99c819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99ce200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819e410 con 0x7f99c810b3c0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99ce200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819e580 con 0x7f99c8101fa0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99ce200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819e740 con 0x7f99c81028c0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48498/0 (socket says 172.21.15.46:48498) 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 -- 172.21.15.46:0/3613050239 learned_addr learned my addr 172.21.15.46:0/3613050239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c7e00700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 0x7f99c819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c7400700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 -- 172.21.15.46:0/3613050239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 msgr2=0x7f99c81994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c81994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 -- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 msgr2=0x7f99c819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 0x7f99c819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 -- 172.21.15.46:0/3613050239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40095f0 con 0x7f99c81028c0 2024-05-08T23:06:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c7400700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c81994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:06:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.295+0000 7f99c6a00700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.296+0000 7f99bfe00700 1 -- 172.21.15.46:0/3613050239 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b401d070 con 0x7f99c81028c0 2024-05-08T23:06:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.296+0000 7f99bfe00700 1 -- 172.21.15.46:0/3613050239 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b4022470 con 0x7f99c81028c0 2024-05-08T23:06:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.296+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c819e9c0 con 0x7f99c81028c0 2024-05-08T23:06:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.296+0000 7f99bfe00700 1 -- 172.21.15.46:0/3613050239 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4017650 con 0x7f99c81028c0 2024-05-08T23:06:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.296+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81a6260 con 0x7f99c81028c0 2024-05-08T23:06:54.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.299+0000 7f99bfe00700 1 -- 172.21.15.46:0/3613050239 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f99b40225e0 con 0x7f99c81028c0 2024-05-08T23:06:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.300+0000 7f99bfe00700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 0x7f99ac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.301+0000 7f99bfe00700 1 -- 172.21.15.46:0/3613050239 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99b40ce200 con 0x7f99c81028c0 2024-05-08T23:06:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.301+0000 7f99e1c6fb80 1 Processor -- start 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.301+0000 7f99c7400700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 0x7f99ac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.301+0000 7f99e1c6fb80 1 -- start start 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4258720 0x5612c4252350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4252890 0x5612c4252cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c41a9150 con 0x5612c4252890 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c41a6b10 con 0x5612c4257e00 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c41a2400 con 0x5612c4258720 2024-05-08T23:06:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 0x7f99ac0a37d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009cb0 tx=0x7f99b8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56436/0 (socket says 172.21.15.46:56436) 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 -- 172.21.15.46:0/664034007 learned_addr learned my addr 172.21.15.46:0/664034007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7e00700 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4252890 0x5612c4252cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c6a00700 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4258720 0x5612c4252350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 -- 172.21.15.46:0/664034007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4258720 msgr2=0x5612c4252350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4258720 0x5612c4252350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 -- 172.21.15.46:0/664034007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4252890 msgr2=0x5612c4252cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4252890 0x5612c4252cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.302+0000 7f99c7400700 1 -- 172.21.15.46:0/664034007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c40233b0 con 0x5612c4257e00 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.303+0000 7f99c7400700 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99b8011fb0 tx=0x7f99b8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7feec66e2ac78d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.303+0000 7f99b3e00700 1 -- 172.21.15.46:0/664034007 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8012dc0 con 0x5612c4257e00 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.303+0000 7f99b3e00700 1 -- 172.21.15.46:0/664034007 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b8016070 con 0x5612c4257e00 2024-05-08T23:06:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.303+0000 7f99b3e00700 1 -- 172.21.15.46:0/664034007 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b801e5b0 con 0x5612c4257e00 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/664034007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 msgr2=0x5612c42581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f99b8011fb0 tx=0x7f99b8014000 comp rx=0 tx=0).stop 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/664034007 shutdown_connections 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4252890 0x5612c4252cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c42581e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/664034007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4258720 0x5612c4252350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/664034007 >> 172.21.15.46:0/664034007 conn(0x5612c41b0210 msgr2=0x5612c424cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/664034007 shutdown_connections 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/664034007 wait complete. 2024-05-08T23:06:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.304+0000 7f99e1c6fb80 1 -- start start 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 Processor -- start 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c425baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c425c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c413e560 con 0x5612c4258720 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c413e200 con 0x5612c4257e00 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c40256a0 con 0x5612c4252890 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.305+0000 7f99e1c6fb80 10 client.0 _ll_register_callbacks cb 0x5612c4147b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c425baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56444/0 (socket says 172.21.15.46:56444) 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c425c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 -- 172.21.15.46:0/636693063 learned_addr learned my addr 172.21.15.46:0/636693063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 -- 172.21.15.46:0/636693063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 msgr2=0x5612c425baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c425baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 -- 172.21.15.46:0/636693063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 msgr2=0x5612c425c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c425c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 -- 172.21.15.46:0/636693063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b8011c60 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c7e00700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c425c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99c6a00700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99b409ac90 tx=0x7f99b409a970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a73d3fb8b33f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.306+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b409af90 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99b409edc0 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612c40fb270 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b40a3380 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99e1c6fb80 10 client.26600 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612c3fcfb60 con 0x5612c4257e00 2024-05-08T23:06:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b40a7080 con 0x5612c4257e00 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.307+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99b40a2ef0 con 0x5612c4257e00 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 10 client.26600 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 10 client.26600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 20 client.26600 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c426bde0 0x5612c426e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 4 client.26600 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:06:54.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612c4221810 con 0x5612c426bde0 2024-05-08T23:06:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.308+0000 7f99c7400700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c426bde0 0x5612c426e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.309+0000 7f99b0c00700 10 client.26600 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.309+0000 7f99c7400700 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c426bde0 0x5612c426e200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e500137b3c16987 server_cookie=edc2a1ae5302cdf6 in_seq=0 out_seq=0 2024-05-08T23:06:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99b0c00700 1 -- 172.21.15.46:0/636693063 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612c4221810 con 0x5612c426bde0 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99b0c00700 10 client.26600 handle_command_reply: tid=0 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 client.26600 shutdown 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 msgr2=0x5612c425bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99b409ac90 tx=0x7f99b409a970 comp rx=0 tx=0).stop 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 shutdown_connections 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c426bde0 0x5612c426e200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c425c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99b0c00700 0 client.26600 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c425baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/636693063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c425bfe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 >> 172.21.15.46:0/636693063 conn(0x5612c41b0210 msgr2=0x5612c424c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 shutdown_connections 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.310+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/636693063 wait complete. 2024-05-08T23:06:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.311+0000 7f99e1c6fb80 20 client.26600 trim_cache size 0 max 16384 2024-05-08T23:06:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 Processor -- start 2024-05-08T23:06:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 -- start start 2024-05-08T23:06:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4252350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4257e00 0x5612c424cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c42450d0 con 0x5612c4257e00 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c425d110 con 0x5612c4258720 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.332+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c4249dc0 con 0x5612c4252890 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4252350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56460/0 (socket says 172.21.15.46:56460) 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4252350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48524/0 (socket says 172.21.15.46:48524) 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 -- 172.21.15.46:0/921234160 learned_addr learned my addr 172.21.15.46:0/921234160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c6a00700 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4257e00 0x5612c424cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 -- 172.21.15.46:0/921234160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 msgr2=0x5612c4252350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4252350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 -- 172.21.15.46:0/921234160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4257e00 msgr2=0x5612c424cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4257e00 0x5612c424cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 -- 172.21.15.46:0/921234160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c4168560 con 0x5612c4258720 2024-05-08T23:06:54.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.333+0000 7f99c7e00700 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f99c0011cf0 tx=0x7f99c0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22a785b1be00ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.334+0000 7f99bd200700 1 -- 172.21.15.46:0/921234160 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000e040 con 0x5612c4258720 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.334+0000 7f99bd200700 1 -- 172.21.15.46:0/921234160 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c000f040 con 0x5612c4258720 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.334+0000 7f99bd200700 1 -- 172.21.15.46:0/921234160 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00043d0 con 0x5612c4258720 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.334+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/921234160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 msgr2=0x5612c42572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.334+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f99c0011cf0 tx=0x7f99c0009920 comp rx=0 tx=0).stop 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/921234160 shutdown_connections 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4258720 0x5612c42572d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4252350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/921234160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4257e00 0x5612c424cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/921234160 >> 172.21.15.46:0/921234160 conn(0x5612c41b0210 msgr2=0x5612c4256bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/921234160 shutdown_connections 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/921234160 wait complete. 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 -- start start 2024-05-08T23:06:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.335+0000 7f99e1c6fb80 1 Processor -- start 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c4263430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c42612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c42619c0 con 0x5612c4258720 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c4261b30 con 0x5612c4257e00 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c4261cf0 con 0x5612c4252890 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99e1c6fb80 10 client.0 _ll_register_callbacks cb 0x5612c4147b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.336+0000 7f99c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48528/0 (socket says 172.21.15.46:48528) 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 -- 172.21.15.46:0/4187144202 learned_addr learned my addr 172.21.15.46:0/4187144202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7e00700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c42612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c6a00700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c4263430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 -- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 msgr2=0x5612c4263430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c4263430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 -- 172.21.15.46:0/4187144202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 msgr2=0x5612c42612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c42612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 -- 172.21.15.46:0/4187144202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00119a0 con 0x5612c4252890 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c6a00700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c4263430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99c7400700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b801fe30 tx=0x7f99b801fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7847948465b27f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.337+0000 7f99b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8013e30 con 0x5612c4252890 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99b8016070 con 0x5612c4252890 2024-05-08T23:06:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f99b8018070 con 0x5612c4252890 2024-05-08T23:06:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b8035430 con 0x5612c4252890 2024-05-08T23:06:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99e1c6fb80 10 client.26653 fetch_fsmap learned FSMap version 28 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612c4027cd0 con 0x5612c4252890 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b8016070 con 0x5612c4252890 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.338+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99b8012450 con 0x5612c4252890 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 20 client.26653 populate_metadata read hostname 'smithi046' 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c42c3420 0x5612c42c5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 4 client.26653 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5612c419aec0 con 0x5612c42c3420 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99c6a00700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c42c3420 0x5612c42c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99c6a00700 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c42c3420 0x5612c42c5840 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affc4bc079b428da server_cookie=e08d88a955dbb452 in_seq=0 out_seq=0 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.339+0000 7f99b0c00700 10 client.26653 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99b0c00700 1 -- 172.21.15.46:0/4187144202 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5612c419aec0 con 0x5612c42c3420 2024-05-08T23:06:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99b0c00700 10 client.26653 handle_command_reply: tid=0 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 client.26653 shutdown 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 msgr2=0x5612c4262ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b801fe30 tx=0x7f99b801fdc0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 shutdown_connections 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5612c42c3420 0x5612c42c5840 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5612c4258720 0x5612c42612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c4252890 0x5612c4262ef0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 --2- 172.21.15.46:0/4187144202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5612c4257e00 0x5612c4263430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99b0c00700 0 client.26653 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 >> 172.21.15.46:0/4187144202 conn(0x5612c41b0210 msgr2=0x5612c425ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 shutdown_connections 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 1 -- 172.21.15.46:0/4187144202 wait complete. 2024-05-08T23:06:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.340+0000 7f99e1c6fb80 20 client.26653 trim_cache size 0 max 16384 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "ae94d4fe-fd7f-4aaa-aced-86e5169d2111": { 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ae94d4fe-fd7f-4aaa-aced-86e5169d2111", 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:06:54.345 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 msgr2=0x7f99ac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 0x7f99ac0a37d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009cb0 tx=0x7f99b8009260 comp rx=0 tx=0).stop 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 msgr2=0x7f99c8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4004730 comp rx=0 tx=0).stop 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 shutdown_connections 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f99ac0a1310 0x7f99ac0a37d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c810b3c0 0x7f99c819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c8101fa0 0x7f99c81994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 --2- 172.21.15.46:0/3613050239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c81028c0 0x7f99c8199a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:06:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 >> 172.21.15.46:0/3613050239 conn(0x7f99c80fb760 msgr2=0x7f99c80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:06:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 shutdown_connections 2024-05-08T23:06:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:06:54.341+0000 7f99ce200700 1 -- 172.21.15.46:0/3613050239 wait complete. 2024-05-08T23:06:54.362 INFO:tasks.cephfs.filesystem:scrub status for tag:ae94d4fe-fd7f-4aaa-aced-86e5169d2111 - {'path': '/', 'tag': 'ae94d4fe-fd7f-4aaa-aced-86e5169d2111', 'options': 'recursive,force'} 2024-05-08T23:06:54.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 122036 8.90 MB/sec warmup 76 sec latency 8.025 ms 2024-05-08T23:06:55.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:06:55.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:55.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:55.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:06:55.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:54 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:06:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:55.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:06:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:54 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:54 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:06:55.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 124464 8.96 MB/sec warmup 77 sec latency 12.646 ms 2024-05-08T23:06:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:55 smithi086 ceph-mon[78610]: pgmap v96: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 301 op/s 2024-05-08T23:06:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:55 smithi053 ceph-mon[77587]: pgmap v96: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 301 op/s 2024-05-08T23:06:56.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:55 smithi046 ceph-mon[36898]: pgmap v96: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 301 op/s 2024-05-08T23:06:56.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 126677 8.95 MB/sec warmup 78 sec latency 10.305 ms 2024-05-08T23:06:57.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:56 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:06:57.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:56 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:06:57.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:56 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:06:57.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 128424 8.98 MB/sec warmup 79 sec latency 10.127 ms 2024-05-08T23:06:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: pgmap v97: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:06:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:57 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: pgmap v97: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:06:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:57 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: pgmap v97: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:57 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:58.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131091 9.05 MB/sec warmup 80 sec latency 13.185 ms 2024-05-08T23:06:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:58 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:06:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:58 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:06:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:58 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:58 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:06:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:58 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:06:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:58 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:58 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:06:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:58 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:06:59.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:58 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:06:59.875 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132885 9.06 MB/sec warmup 81 sec latency 15.724 ms 2024-05-08T23:07:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:06:59 smithi086 ceph-mon[78610]: pgmap v98: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:07:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:06:59 smithi053 ceph-mon[77587]: pgmap v98: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:07:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:06:59 smithi046 ceph-mon[36898]: pgmap v98: 129 pgs: 129 active+clean; 80 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-08T23:07:00.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 134529 9.05 MB/sec warmup 82 sec latency 125.096 ms 2024-05-08T23:07:01.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 135497 9.01 MB/sec warmup 83 sec latency 128.998 ms 2024-05-08T23:07:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:01 smithi086 ceph-mon[78610]: pgmap v99: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:07:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:01 smithi053 ceph-mon[77587]: pgmap v99: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:07:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:01 smithi046 ceph-mon[36898]: pgmap v99: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-05-08T23:07:02.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 137713 9.04 MB/sec warmup 84 sec latency 20.412 ms 2024-05-08T23:07:03.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 139175 9.03 MB/sec warmup 85 sec latency 22.851 ms 2024-05-08T23:07:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:03 smithi086 ceph-mon[78610]: pgmap v100: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:07:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:03 smithi053 ceph-mon[77587]: pgmap v100: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:07:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:03 smithi046 ceph-mon[36898]: pgmap v100: 129 pgs: 129 active+clean; 85 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-05-08T23:07:04.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 141919 9.12 MB/sec warmup 86 sec latency 12.045 ms 2024-05-08T23:07:05.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 143579 9.09 MB/sec warmup 87 sec latency 7.693 ms 2024-05-08T23:07:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:05 smithi086 ceph-mon[78610]: pgmap v101: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 336 op/s 2024-05-08T23:07:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:05 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:05 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:05 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:05 smithi053 ceph-mon[77587]: pgmap v101: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 336 op/s 2024-05-08T23:07:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:05 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:05 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:05 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:05 smithi046 ceph-mon[36898]: pgmap v101: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 336 op/s 2024-05-08T23:07:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:05 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:05 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:05 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:06.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 145595 9.13 MB/sec warmup 88 sec latency 94.345 ms 2024-05-08T23:07:07.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 147108 9.10 MB/sec warmup 89 sec latency 22.488 ms 2024-05-08T23:07:08.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:07 smithi086 ceph-mon[78610]: pgmap v102: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:07 smithi053 ceph-mon[77587]: pgmap v102: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:08.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:07 smithi046 ceph-mon[36898]: pgmap v102: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:08.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149384 9.16 MB/sec warmup 90 sec latency 10.802 ms 2024-05-08T23:07:09.876 INFO:tasks.workunit.client.0.smithi046.stdout: 1 151786 9.18 MB/sec warmup 91 sec latency 8.821 ms 2024-05-08T23:07:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:09 smithi086 ceph-mon[78610]: pgmap v103: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:09 smithi053 ceph-mon[77587]: pgmap v103: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:09 smithi046 ceph-mon[36898]: pgmap v103: 129 pgs: 129 active+clean; 87 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-05-08T23:07:10.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 153500 9.17 MB/sec warmup 92 sec latency 19.729 ms 2024-05-08T23:07:11.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 156302 9.27 MB/sec warmup 93 sec latency 10.912 ms 2024-05-08T23:07:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:11 smithi086 ceph-mon[78610]: pgmap v104: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-05-08T23:07:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:11 smithi053 ceph-mon[77587]: pgmap v104: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-05-08T23:07:12.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:11 smithi046 ceph-mon[36898]: pgmap v104: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-05-08T23:07:12.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 158523 9.25 MB/sec warmup 94 sec latency 12.990 ms 2024-05-08T23:07:13.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 160239 9.28 MB/sec warmup 95 sec latency 10.613 ms 2024-05-08T23:07:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:13 smithi086 ceph-mon[78610]: pgmap v105: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:07:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:13 smithi053 ceph-mon[77587]: pgmap v105: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:07:14.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:13 smithi046 ceph-mon[36898]: pgmap v105: 129 pgs: 129 active+clean; 92 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:07:14.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 162808 9.32 MB/sec warmup 96 sec latency 11.759 ms 2024-05-08T23:07:15.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 164291 9.30 MB/sec warmup 97 sec latency 11.514 ms 2024-05-08T23:07:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:15 smithi086 ceph-mon[78610]: pgmap v106: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 375 op/s 2024-05-08T23:07:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:15 smithi053 ceph-mon[77587]: pgmap v106: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 375 op/s 2024-05-08T23:07:16.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:15 smithi046 ceph-mon[36898]: pgmap v106: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 375 op/s 2024-05-08T23:07:16.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 166873 9.37 MB/sec warmup 98 sec latency 10.468 ms 2024-05-08T23:07:17.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 168727 9.35 MB/sec warmup 99 sec latency 59.019 ms 2024-05-08T23:07:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:17 smithi086 ceph-mon[78610]: pgmap v107: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:17 smithi053 ceph-mon[77587]: pgmap v107: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:18.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:17 smithi046 ceph-mon[36898]: pgmap v107: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:18.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 170751 9.39 MB/sec warmup 100 sec latency 16.348 ms 2024-05-08T23:07:19.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 173256 9.42 MB/sec warmup 101 sec latency 9.597 ms 2024-05-08T23:07:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:19 smithi086 ceph-mon[78610]: pgmap v108: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:19 smithi053 ceph-mon[77587]: pgmap v108: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:20.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:19 smithi046 ceph-mon[36898]: pgmap v108: 129 pgs: 129 active+clean; 100 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-05-08T23:07:20.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 174812 9.40 MB/sec warmup 102 sec latency 16.235 ms 2024-05-08T23:07:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:20 smithi086 ceph-mon[78610]: pgmap v109: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-05-08T23:07:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:20 smithi053 ceph-mon[77587]: pgmap v109: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-05-08T23:07:21.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:20 smithi046 ceph-mon[36898]: pgmap v109: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-05-08T23:07:21.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 177459 9.47 MB/sec warmup 103 sec latency 11.482 ms 2024-05-08T23:07:22.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 179506 9.45 MB/sec warmup 104 sec latency 11.250 ms 2024-05-08T23:07:23.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:23 smithi086 ceph-mon[78610]: pgmap v110: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-08T23:07:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:23 smithi053 ceph-mon[77587]: pgmap v110: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-08T23:07:23.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:23 smithi046 ceph-mon[36898]: pgmap v110: 129 pgs: 129 active+clean; 106 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-08T23:07:23.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 181450 9.49 MB/sec warmup 105 sec latency 15.765 ms 2024-05-08T23:07:24.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 -- 172.21.15.46:0/201223299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa860101a30 msgr2=0x7fa860105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 --2- 172.21.15.46:0/201223299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa860101a30 0x7fa860105a70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009990 tx=0x7fa84c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 -- 172.21.15.46:0/201223299 shutdown_connections 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 --2- 172.21.15.46:0/201223299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8601061a0 0x7fa86010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 --2- 172.21.15.46:0/201223299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa860101080 0x7fa860101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 --2- 172.21.15.46:0/201223299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa860101a30 0x7fa860105a70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 -- 172.21.15.46:0/201223299 >> 172.21.15.46:0/201223299 conn(0x7fa8600fc8e0 msgr2=0x7fa8600fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 -- 172.21.15.46:0/201223299 shutdown_connections 2024-05-08T23:07:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.597+0000 7fa867400700 1 -- 172.21.15.46:0/201223299 wait complete. 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.598+0000 7fa867400700 1 Processor -- start 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 -- start start 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 0x7fa860199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86019de40 0x7fa86019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019e8c0 con 0x7fa86019de40 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019ea30 con 0x7fa860101080 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa867400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019ebf0 con 0x7fa8601061a0 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa85fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa85fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50830/0 (socket says 172.21.15.46:50830) 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa85fe00700 1 -- 172.21.15.46:0/3975364445 learned_addr learned my addr 172.21.15.46:0/3975364445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa865400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86019de40 0x7fa86019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.599+0000 7fa864a00700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 0x7fa860199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 -- 172.21.15.46:0/3975364445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 msgr2=0x7fa860199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 0x7fa860199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 -- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86019de40 msgr2=0x7fa86019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86019de40 0x7fa86019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 -- 172.21.15.46:0/3975364445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0095f0 con 0x7fa8601061a0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa864a00700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 0x7fa860199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85fe00700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85d600700 1 -- 172.21.15.46:0/3975364445 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01d070 con 0x7fa8601061a0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85d600700 1 -- 172.21.15.46:0/3975364445 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c022470 con 0x7fa8601061a0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa85d600700 1 -- 172.21.15.46:0/3975364445 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c017650 con 0x7fa8601061a0 2024-05-08T23:07:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8601a60f0 con 0x7fa8601061a0 2024-05-08T23:07:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.600+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601a6510 con 0x7fa8601061a0 2024-05-08T23:07:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.604+0000 7fa85d600700 1 -- 172.21.15.46:0/3975364445 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa84c00fad0 con 0x7fa8601061a0 2024-05-08T23:07:24.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 Processor -- start 2024-05-08T23:07:24.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 -- start start 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa85d600700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 0x7fa8480a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa85d600700 1 -- 172.21.15.46:0/3975364445 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa84c0ce3c0 con 0x7fa8601061a0 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa865400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 0x7fa8480a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399df51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399defcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399d46150 con 0x55e399df4e00 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399d43b10 con 0x55e399def890 2024-05-08T23:07:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.606+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399d3f400 con 0x55e399df5720 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa864a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399df51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50832/0 (socket says 172.21.15.46:50832) 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 -- 172.21.15.46:0/1720473984 learned_addr learned my addr 172.21.15.46:0/1720473984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa865400700 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399defcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa865400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 0x7fa8480a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8580093c0 tx=0x7fa858009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 -- 172.21.15.46:0/1720473984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 msgr2=0x55e399defcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399defcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 -- 172.21.15.46:0/1720473984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 msgr2=0x55e399df51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399df51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.607+0000 7fa85fe00700 1 -- 172.21.15.46:0/1720473984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e399bc03b0 con 0x55e399df5720 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa85fe00700 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa84c097bc0 tx=0x7fa84c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a55139ac5a068e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa851c00700 1 -- 172.21.15.46:0/1720473984 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c09fea0 con 0x55e399df5720 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa851c00700 1 -- 172.21.15.46:0/1720473984 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c09b150 con 0x55e399df5720 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa851c00700 1 -- 172.21.15.46:0/1720473984 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0995c0 con 0x55e399df5720 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa87afd1b80 1 -- 172.21.15.46:0/1720473984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 msgr2=0x55e399def350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa84c097bc0 tx=0x7fa84c009310 comp rx=0 tx=0).stop 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.608+0000 7fa87afd1b80 1 -- 172.21.15.46:0/1720473984 shutdown_connections 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399defcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399df51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/1720473984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399def350 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 -- 172.21.15.46:0/1720473984 >> 172.21.15.46:0/1720473984 conn(0x55e399d4d210 msgr2=0x55e399de9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 -- 172.21.15.46:0/1720473984 shutdown_connections 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 -- 172.21.15.46:0/1720473984 wait complete. 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 -- start start 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.609+0000 7fa87afd1b80 1 Processor -- start 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 0x55e399df8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 0x55e399df9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399cdb560 con 0x55e399df5720 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399cdb200 con 0x55e399df4e00 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399bc26a0 con 0x55e399def890 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa87afd1b80 10 client.0 _ll_register_callbacks cb 0x55e399ce4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:24.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa864a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa865400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 0x55e399df9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa864a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50834/0 (socket says 172.21.15.46:50834) 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa864a00700 1 -- 172.21.15.46:0/2507222068 learned_addr learned my addr 172.21.15.46:0/2507222068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.610+0000 7fa85fe00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 0x55e399df8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 -- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 msgr2=0x55e399df8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 0x55e399df8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 -- 172.21.15.46:0/2507222068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 msgr2=0x55e399df9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 0x55e399df9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 -- 172.21.15.46:0/2507222068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601a6510 con 0x55e399def890 2024-05-08T23:07:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa85fe00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 0x55e399df8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa865400700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 0x55e399df9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa864a00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa854012880 tx=0x7fa854012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4b7a6e2b7b64fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa846000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854017070 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa8540117f0 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e399c98270 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8540119b0 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa87afd1b80 10 client.26665 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.611+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e399b6cb60 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e399c98270 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa85401c420 con 0x55e399def890 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 10 client.26665 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 20 client.26665 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e08f40 0x55e399e0b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 4 client.26665 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e399dbe810 con 0x55e399e08f40 2024-05-08T23:07:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.612+0000 7fa85fe00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e08f40 0x55e399e0b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.613+0000 7fa85fe00700 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e08f40 0x55e399e0b360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f05471cab23733 server_cookie=6f9d0ed421fe8003 in_seq=0 out_seq=0 2024-05-08T23:07:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.613+0000 7fa846000700 10 client.26665 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.613+0000 7fa846000700 1 -- 172.21.15.46:0/2507222068 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e399dbe810 con 0x55e399e08f40 2024-05-08T23:07:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.613+0000 7fa846000700 10 client.26665 handle_command_reply: tid=0 2024-05-08T23:07:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 client.26665 shutdown 2024-05-08T23:07:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 msgr2=0x55e399dfb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa854012880 tx=0x7fa854012b90 comp rx=0 tx=0).stop 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 shutdown_connections 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e08f40 0x55e399e0b360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df5720 0x55e399df9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399def890 0x55e399dfb730 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2507222068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399df4e00 0x55e399df8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa846000700 0 client.26665 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 >> 172.21.15.46:0/2507222068 conn(0x55e399d4d210 msgr2=0x55e399de95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 shutdown_connections 2024-05-08T23:07:24.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2507222068 wait complete. 2024-05-08T23:07:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.614+0000 7fa87afd1b80 20 client.26665 trim_cache size 0 max 16384 2024-05-08T23:07:24.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.635+0000 7fa87afd1b80 1 Processor -- start 2024-05-08T23:07:24.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.635+0000 7fa87afd1b80 1 -- start start 2024-05-08T23:07:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399de9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399e0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399debe50 con 0x55e399df4e00 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399e06530 con 0x55e399def890 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399df9ff0 con 0x55e399df5720 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa864a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa864a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57538/0 (socket says 172.21.15.46:57538) 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa864a00700 1 -- 172.21.15.46:0/512865366 learned_addr learned my addr 172.21.15.46:0/512865366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa85fe00700 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399de9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.636+0000 7fa865400700 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399e0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 -- 172.21.15.46:0/512865366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 msgr2=0x55e399e0fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399e0fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 -- 172.21.15.46:0/512865366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 msgr2=0x55e399de9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399de9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 -- 172.21.15.46:0/512865366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e399d05560 con 0x55e399def890 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa864a00700 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa85401c7d0 tx=0x7fa8540093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead77d31a75e47d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa852600700 1 -- 172.21.15.46:0/512865366 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854017040 con 0x55e399def890 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa852600700 1 -- 172.21.15.46:0/512865366 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa854005030 con 0x55e399def890 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.637+0000 7fa852600700 1 -- 172.21.15.46:0/512865366 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854011b00 con 0x55e399def890 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 -- 172.21.15.46:0/512865366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 msgr2=0x55e399df9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa85401c7d0 tx=0x7fa8540093d0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 -- 172.21.15.46:0/512865366 shutdown_connections 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399e0fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399df9100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/512865366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399de9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 -- 172.21.15.46:0/512865366 >> 172.21.15.46:0/512865366 conn(0x55e399d4d210 msgr2=0x55e399defc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 -- 172.21.15.46:0/512865366 shutdown_connections 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.638+0000 7fa87afd1b80 1 -- 172.21.15.46:0/512865366 wait complete. 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.639+0000 7fa87afd1b80 1 -- start start 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.639+0000 7fa87afd1b80 1 Processor -- start 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399e00280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399dfbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399dfca80 con 0x55e399df4e00 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399dfcbf0 con 0x55e399def890 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e399e5e3f0 con 0x55e399df5720 2024-05-08T23:07:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa87afd1b80 10 client.0 _ll_register_callbacks cb 0x55e399ce4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50856/0 (socket says 172.21.15.46:50856) 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 -- 172.21.15.46:0/2783642296 learned_addr learned my addr 172.21.15.46:0/2783642296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa864a00700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399e00280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa85fe00700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399dfbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 -- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 msgr2=0x55e399e00280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399e00280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 -- 172.21.15.46:0/2783642296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 msgr2=0x55e399dfbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399dfbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.640+0000 7fa865400700 1 -- 172.21.15.46:0/2783642296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e399b6cb60 con 0x55e399df5720 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa865400700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa858007b70 tx=0x7fa858004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ddc12b03f62ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800e050 con 0x55e399df5720 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa858017e10 con 0x55e399df5720 2024-05-08T23:07:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa84c09b150 con 0x55e399df5720 2024-05-08T23:07:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa858018d80 con 0x55e399df5720 2024-05-08T23:07:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa87afd1b80 10 client.26671 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e399bc4cd0 con 0x55e399df5720 2024-05-08T23:07:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c09b150 con 0x55e399df5720 2024-05-08T23:07:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.641+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa85802b020 con 0x55e399df5720 2024-05-08T23:07:24.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 20 client.26671 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e5fbd0 0x55e399e61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 4 client.26671 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e399d37ec0 con 0x55e399e5fbd0 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.642+0000 7fa864a00700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e5fbd0 0x55e399e61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa864a00700 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e5fbd0 0x55e399e61ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb05fd76a096a23 server_cookie=1c52637658b745d in_seq=0 out_seq=0 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa846000700 10 client.26671 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa846000700 1 -- 172.21.15.46:0/2783642296 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e399d37ec0 con 0x55e399e5fbd0 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa846000700 10 client.26671 handle_command_reply: tid=0 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 client.26671 shutdown 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 msgr2=0x55e399dfc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa858007b70 tx=0x7fa858004b40 comp rx=0 tx=0).stop 2024-05-08T23:07:24.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 shutdown_connections 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e399df5720 0x55e399dfc390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e399def890 0x55e399e00280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e399e5fbd0 0x55e399e61ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 --2- 172.21.15.46:0/2783642296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e399df4e00 0x55e399dfbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa846000700 0 client.26671 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.643+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 >> 172.21.15.46:0/2783642296 conn(0x55e399d4d210 msgr2=0x55e399df2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 shutdown_connections 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa87afd1b80 1 -- 172.21.15.46:0/2783642296 wait complete. 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa87afd1b80 20 client.26671 trim_cache size 0 max 16384 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:07:24.649 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:07:24.650 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:07:24.650 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:07:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 msgr2=0x7fa8480a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 0x7fa8480a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8580093c0 tx=0x7fa858009920 comp rx=0 tx=0).stop 2024-05-08T23:07:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 msgr2=0x7fa860199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.644+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c0057f0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 shutdown_connections 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8480a1310 0x7fa8480a37d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa86019de40 0x7fa86019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa860101080 0x7fa860199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 --2- 172.21.15.46:0/3975364445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8601061a0 0x7fa860199aa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 >> 172.21.15.46:0/3975364445 conn(0x7fa8600fc8e0 msgr2=0x7fa8600fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 shutdown_connections 2024-05-08T23:07:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.645+0000 7fa867400700 1 -- 172.21.15.46:0/3975364445 wait complete. 2024-05-08T23:07:24.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:07:24.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:07:24.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 183886 9.52 MB/sec warmup 106 sec latency 11.357 ms 2024-05-08T23:07:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 -- 172.21.15.46:0/1278519440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073370 msgr2=0x7fb3f4073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/1278519440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073370 0x7fb3f4073750 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009990 tx=0x7fb3e4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 -- 172.21.15.46:0/1278519440 shutdown_connections 2024-05-08T23:07:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/1278519440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f410d4d0 0x7fb3f410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/1278519440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073370 0x7fb3f4073750 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/1278519440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f4073c90 0x7fb3f410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 -- 172.21.15.46:0/1278519440 >> 172.21.15.46:0/1278519440 conn(0x7fb3f40fc720 msgr2=0x7fb3f40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 -- 172.21.15.46:0/1278519440 shutdown_connections 2024-05-08T23:07:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.890+0000 7fb3f8e00700 1 -- 172.21.15.46:0/1278519440 wait complete. 2024-05-08T23:07:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.891+0000 7fb3f8e00700 1 Processor -- start 2024-05-08T23:07:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 -- start start 2024-05-08T23:07:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 0x7fb3f41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 0x7fb3f4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f419e1f0 con 0x7fb3f4073c90 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f419e360 con 0x7fb3f410d4d0 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f419e520 con 0x7fb3f4073370 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.892+0000 7fb3f3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 0x7fb3f41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 0x7fb3f4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57562/0 (socket says 172.21.15.46:57562) 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 0x7fb3f41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50876/0 (socket says 172.21.15.46:50876) 2024-05-08T23:07:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f3e00700 1 -- 172.21.15.46:0/2420208673 learned_addr learned my addr 172.21.15.46:0/2420208673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 -- 172.21.15.46:0/2420208673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 msgr2=0x7fb3f41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 0x7fb3f41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 -- 172.21.15.46:0/2420208673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 msgr2=0x7fb3f419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 -- 172.21.15.46:0/2420208673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40095f0 con 0x7fb3f4073c90 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f3e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.893+0000 7fb3f2a00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 0x7fb3f4199810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e9f0 tx=0x7fb3e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.894+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000c9c0 con 0x7fb3f4073c90 2024-05-08T23:07:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.894+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0010430 con 0x7fb3f4073c90 2024-05-08T23:07:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.894+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0018680 con 0x7fb3f4073c90 2024-05-08T23:07:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.894+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f419e800 con 0x7fb3f4073c90 2024-05-08T23:07:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.894+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f41a6040 con 0x7fb3f4073c90 2024-05-08T23:07:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.895+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f419ea20 con 0x7fb3f4073c90 2024-05-08T23:07:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.898+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb3e0010860 con 0x7fb3f4073c90 2024-05-08T23:07:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.900+0000 7fb3ebe00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 0x7fb3d80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.900+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb3e0014070 con 0x7fb3f4073c90 2024-05-08T23:07:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.900+0000 7fb3f3400700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 0x7fb3d80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:24.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.901+0000 7fb3f3400700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 0x7fb3d80a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098e0 tx=0x7fb3e4005960 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:24.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:24.902+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3e0096200 con 0x7fb3f4073c90 2024-05-08T23:07:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.208+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3f419e9f0 con 0x7fb3f4073c90 2024-05-08T23:07:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.208+0000 7fb3ebe00700 1 -- 172.21.15.46:0/2420208673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10907 (secure 0 0 0) 0x7fb3e0026090 con 0x7fb3f4073c90 2024-05-08T23:07:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:07:25.209 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:05:40.700994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:07:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 msgr2=0x7fb3d80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 0x7fb3d80a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098e0 tx=0x7fb3e4005960 comp rx=0 tx=0).stop 2024-05-08T23:07:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 msgr2=0x7fb3f4199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 0x7fb3f4199810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e9f0 tx=0x7fb3e000ed00 comp rx=0 tx=0).stop 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 shutdown_connections 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f410d4d0 0x7fb3f419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3f4073370 0x7fb3f41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb3d80a1310 0x7fb3d80a37d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 --2- 172.21.15.46:0/2420208673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f4073c90 0x7fb3f4199810 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 >> 172.21.15.46:0/2420208673 conn(0x7fb3f40fc720 msgr2=0x7fb3f40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 shutdown_connections 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.211+0000 7fb3f8e00700 1 -- 172.21.15.46:0/2420208673 wait complete. 2024-05-08T23:07:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-05-08T23:07:25.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.454+0000 7f49aae00700 1 -- 172.21.15.46:0/1774856451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4101880 msgr2=0x7f49a41058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.454+0000 7f49aae00700 1 --2- 172.21.15.46:0/1774856451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4101880 0x7f49a41058c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4990009990 tx=0x7f4990009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 -- 172.21.15.46:0/1774856451 shutdown_connections 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 --2- 172.21.15.46:0/1774856451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 --2- 172.21.15.46:0/1774856451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 --2- 172.21.15.46:0/1774856451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4101880 0x7f49a41058c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 -- 172.21.15.46:0/1774856451 >> 172.21.15.46:0/1774856451 conn(0x7f49a40fc770 msgr2=0x7f49a40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 -- 172.21.15.46:0/1774856451 shutdown_connections 2024-05-08T23:07:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.455+0000 7f49aae00700 1 -- 172.21.15.46:0/1774856451 wait complete. 2024-05-08T23:07:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.456+0000 7f49aae00700 1 Processor -- start 2024-05-08T23:07:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 -- start start 2024-05-08T23:07:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4073d20 0x7f49a41a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41a58a0 con 0x7f49a4073d20 2024-05-08T23:07:25.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41a59e0 con 0x7f49a4105ff0 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41a5ba0 con 0x7f49a4100ed0 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49a9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50888/0 (socket says 172.21.15.46:50888) 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49a9a00700 1 -- 172.21.15.46:0/2600449222 learned_addr learned my addr 172.21.15.46:0/2600449222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.457+0000 7f49aa400700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4073d20 0x7f49a41a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 -- 172.21.15.46:0/2600449222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 msgr2=0x7f49a40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 -- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4073d20 msgr2=0x7f49a41a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4073d20 0x7f49a41a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 -- 172.21.15.46:0/2600449222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900095f0 con 0x7f49a4105ff0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9a00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a9000700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a40737e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f4990004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a2000700 1 -- 172.21.15.46:0/2600449222 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499001d070 con 0x7f49a4105ff0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49a2000700 1 -- 172.21.15.46:0/2600449222 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4990021410 con 0x7f49a4105ff0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.458+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a41a5e20 con 0x7f49a4105ff0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.459+0000 7f49a2000700 1 -- 172.21.15.46:0/2600449222 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990017650 con 0x7f49a4105ff0 2024-05-08T23:07:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.459+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a41a6240 con 0x7f49a4105ff0 2024-05-08T23:07:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.463+0000 7f49a2000700 1 -- 172.21.15.46:0/2600449222 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4990021580 con 0x7f49a4105ff0 2024-05-08T23:07:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.464+0000 7f49bff60b80 1 Processor -- start 2024-05-08T23:07:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.464+0000 7f49bff60b80 1 -- start start 2024-05-08T23:07:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.464+0000 7f49a2000700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 0x7f498c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4aa6150 con 0x55ecb4b55970 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4aa3b10 con 0x55ecb4b4f890 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49bff60b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4a9f400 con 0x55ecb4b55050 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a2000700 1 -- 172.21.15.46:0/2600449222 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f49900ce1b0 con 0x7f49a4105ff0 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49aa400700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 0x7f498c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50894/0 (socket says 172.21.15.46:50894) 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 -- 172.21.15.46:0/2973391278 learned_addr learned my addr 172.21.15.46:0/2973391278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9000700 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49aa400700 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 -- 172.21.15.46:0/2973391278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b4f890 msgr2=0x55ecb4b4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 -- 172.21.15.46:0/2973391278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 msgr2=0x55ecb4b4f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b4f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.465+0000 7f49a9a00700 1 -- 172.21.15.46:0/2973391278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb49203b0 con 0x55ecb4b55050 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.466+0000 7f49a9a00700 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4998011810 tx=0x7f4998011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14238f34048f5757 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.466+0000 7f49aa400700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 0x7f498c0a3870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ec0 tx=0x7f499c00fc40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.466+0000 7f4996a00700 1 -- 172.21.15.46:0/2973391278 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800f540 con 0x55ecb4b55050 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.466+0000 7f4996a00700 1 -- 172.21.15.46:0/2973391278 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f499800fb60 con 0x55ecb4b55050 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.466+0000 7f4996a00700 1 -- 172.21.15.46:0/2973391278 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998019b40 con 0x55ecb4b55050 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- 172.21.15.46:0/2973391278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 msgr2=0x55ecb4b55430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4998011810 tx=0x7f4998011b20 comp rx=0 tx=0).stop 2024-05-08T23:07:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- 172.21.15.46:0/2973391278 shutdown_connections 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b55430 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 --2- 172.21.15.46:0/2973391278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b4f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- 172.21.15.46:0/2973391278 >> 172.21.15.46:0/2973391278 conn(0x55ecb4aad210 msgr2=0x55ecb4b4a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- 172.21.15.46:0/2973391278 shutdown_connections 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- 172.21.15.46:0/2973391278 wait complete. 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.467+0000 7f49bff60b80 1 -- start start 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 Processor -- start 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4a3b560 con 0x55ecb4b55970 2024-05-08T23:07:25.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4a3b200 con 0x55ecb4b55050 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb49226a0 con 0x55ecb4b4f890 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49bff60b80 10 client.0 _ll_register_callbacks cb 0x55ecb4a44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49a9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49aa400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57610/0 (socket says 172.21.15.46:57610) 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 -- 172.21.15.46:0/721241746 learned_addr learned my addr 172.21.15.46:0/721241746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.468+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50900/0 (socket says 172.21.15.46:50900) 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 -- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 msgr2=0x55ecb4b5b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 -- 172.21.15.46:0/721241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 msgr2=0x55ecb4b59360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b59360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 -- 172.21.15.46:0/721241746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4998011440 con 0x55ecb4b55050 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9a00700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49a9000700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f499000b2e0 tx=0x7f499009ab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683ff95cdf1a5f10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f498b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990021a60 con 0x55ecb4b55050 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f499009afa0 con 0x55ecb4b55050 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.469+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecb49f8270 con 0x55ecb4b55050 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f499009edd0 con 0x55ecb4b55050 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 10 client.26639 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ecb48ccb60 con 0x55ecb4b55050 2024-05-08T23:07:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecb49f8270 con 0x55ecb4b55050 2024-05-08T23:07:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f499009a490 con 0x55ecb4b55050 2024-05-08T23:07:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 10 client.26639 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 10 client.26639 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:07:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 20 client.26639 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4b68f50 0x55ecb4b6b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 4 client.26639 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.470+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecb4b1e810 con 0x55ecb4b68f50 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.471+0000 7f49a9a00700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4b68f50 0x55ecb4b6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.471+0000 7f49a9a00700 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4b68f50 0x55ecb4b6b340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=215e716d79583608 server_cookie=a3b8b9965a601c7c in_seq=0 out_seq=0 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.471+0000 7f498b400700 10 client.26639 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f498b400700 1 -- 172.21.15.46:0/721241746 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecb4b1e810 con 0x55ecb4b68f50 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f498b400700 10 client.26639 handle_command_reply: tid=0 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 client.26639 shutdown 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 msgr2=0x55ecb4b58c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f499000b2e0 tx=0x7f499009ab60 comp rx=0 tx=0).stop 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 shutdown_connections 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4b68f50 0x55ecb4b6b340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b5b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 --2- 172.21.15.46:0/721241746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b58c70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f498b400700 0 client.26639 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 >> 172.21.15.46:0/721241746 conn(0x55ecb4aad210 msgr2=0x55ecb4b497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 shutdown_connections 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.472+0000 7f49bff60b80 1 -- 172.21.15.46:0/721241746 wait complete. 2024-05-08T23:07:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.473+0000 7f49bff60b80 20 client.26639 trim_cache size 0 max 16384 2024-05-08T23:07:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 Processor -- start 2024-05-08T23:07:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 -- start start 2024-05-08T23:07:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b6fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b52d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b44840 con 0x55ecb4b55970 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b4b230 con 0x55ecb4b55050 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.489+0000 7f49bff60b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b4be20 con 0x55ecb4b4f890 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50902/0 (socket says 172.21.15.46:50902) 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49aa400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b52d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 -- 172.21.15.46:0/715555275 learned_addr learned my addr 172.21.15.46:0/715555275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9000700 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b6fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 -- 172.21.15.46:0/715555275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 msgr2=0x55ecb4b6fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b6fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 -- 172.21.15.46:0/715555275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 msgr2=0x55ecb4b52d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b52d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 -- 172.21.15.46:0/715555275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb4a65560 con 0x55ecb4b4f890 2024-05-08T23:07:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.490+0000 7f49a9a00700 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4998009340 tx=0x7f4998005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbe92e852452b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.491+0000 7f4997400700 1 -- 172.21.15.46:0/715555275 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998009d70 con 0x55ecb4b4f890 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.491+0000 7f4997400700 1 -- 172.21.15.46:0/715555275 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4998029070 con 0x55ecb4b4f890 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.491+0000 7f4997400700 1 -- 172.21.15.46:0/715555275 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998004970 con 0x55ecb4b4f890 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- 172.21.15.46:0/715555275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 msgr2=0x55ecb4b4f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4998009340 tx=0x7f4998005ce0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- 172.21.15.46:0/715555275 shutdown_connections 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b55970 0x55ecb4b52d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b4f890 0x55ecb4b4f350 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 --2- 172.21.15.46:0/715555275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55050 0x55ecb4b6fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- 172.21.15.46:0/715555275 >> 172.21.15.46:0/715555275 conn(0x55ecb4aad210 msgr2=0x55ecb4b5b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- 172.21.15.46:0/715555275 shutdown_connections 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- 172.21.15.46:0/715555275 wait complete. 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.492+0000 7f49bff60b80 1 -- start start 2024-05-08T23:07:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 Processor -- start 2024-05-08T23:07:25.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b4f890 0x55ecb4b60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 0x55ecb4b522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b50ae0 con 0x55ecb4b4f890 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b50c50 con 0x55ecb4b55970 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4b50e10 con 0x55ecb4b55050 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.493+0000 7f49bff60b80 10 client.0 _ll_register_callbacks cb 0x7f49900973b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49aa400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 0x55ecb4b522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b4f890 0x55ecb4b60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49aa400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 0x55ecb4b522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57624/0 (socket says 172.21.15.46:57624) 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49aa400700 1 -- 172.21.15.46:0/3299084842 learned_addr learned my addr 172.21.15.46:0/3299084842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 -- 172.21.15.46:0/3299084842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 msgr2=0x55ecb4b522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 0x55ecb4b522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 -- 172.21.15.46:0/3299084842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b4f890 msgr2=0x55ecb4b60810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b4f890 0x55ecb4b60810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 -- 172.21.15.46:0/3299084842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb49203b0 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f49a9000700 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b51b60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49900cb600 tx=0x7f499009b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbca96bf73e0f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f498b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.494+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990021a60 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f499800fb60 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f499009afa0 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f499009a900 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 10 client.26695 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ecb4924cd0 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49900a7080 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f49900a2f00 con 0x55ecb4b55050 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 10 client.26695 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 10 client.26695 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 20 client.26695 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4bc80b0 0x55ecb4bca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 4 client.26695 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:25.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ecb4a97ec0 con 0x55ecb4bc80b0 2024-05-08T23:07:25.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.495+0000 7f49a9a00700 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4bc80b0 0x55ecb4bca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f498b400700 10 client.26695 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49a9a00700 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4bc80b0 0x55ecb4bca4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13f891a8f863ebfd server_cookie=4d1445eb27a4cc97 in_seq=0 out_seq=0 2024-05-08T23:07:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f498b400700 1 -- 172.21.15.46:0/3299084842 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ecb4a97ec0 con 0x55ecb4bc80b0 2024-05-08T23:07:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f498b400700 10 client.26695 handle_command_reply: tid=0 2024-05-08T23:07:25.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 client.26695 shutdown 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 msgr2=0x55ecb4b51b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b51b60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49900cb600 tx=0x7f499009b590 comp rx=0 tx=0).stop 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 shutdown_connections 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55ecb4bc80b0 0x55ecb4bca4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecb4b55970 0x55ecb4b522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecb4b4f890 0x55ecb4b60810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 --2- 172.21.15.46:0/3299084842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb4b55050 0x55ecb4b51b60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f498b400700 0 client.26695 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.496+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 >> 172.21.15.46:0/3299084842 conn(0x55ecb4aad210 msgr2=0x55ecb4b52d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 shutdown_connections 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49bff60b80 1 -- 172.21.15.46:0/3299084842 wait complete. 2024-05-08T23:07:25.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49bff60b80 20 client.26695 trim_cache size 0 max 16384 2024-05-08T23:07:25.501 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:25.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 msgr2=0x7f498c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 0x7f498c0a3870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ec0 tx=0x7f499c00fc40 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 msgr2=0x7f49a40737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.497+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a40737e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f4990004730 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 shutdown_connections 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f498c0a13b0 0x7f498c0a3870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a4073d20 0x7f49a41a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a4100ed0 0x7f49a40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 --2- 172.21.15.46:0/2600449222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49a4105ff0 0x7f49a40737e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 >> 172.21.15.46:0/2600449222 conn(0x7f49a40fc770 msgr2=0x7f49a40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 shutdown_connections 2024-05-08T23:07:25.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.498+0000 7f49aae00700 1 -- 172.21.15.46:0/2600449222 wait complete. 2024-05-08T23:07:25.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:07:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.749+0000 7f277a200700 1 -- 172.21.15.46:0/2373075499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 msgr2=0x7f277410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.749+0000 7f277a200700 1 --2- 172.21.15.46:0/2373075499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277410d8d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2764009990 tx=0x7f2764009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 -- 172.21.15.46:0/2373075499 shutdown_connections 2024-05-08T23:07:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 --2- 172.21.15.46:0/2373075499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 0x7f2774114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 --2- 172.21.15.46:0/2373075499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277410d8d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 --2- 172.21.15.46:0/2373075499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f2774108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 -- 172.21.15.46:0/2373075499 >> 172.21.15.46:0/2373075499 conn(0x7f2774076980 msgr2=0x7f2774078da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.750+0000 7f277a200700 1 -- 172.21.15.46:0/2373075499 shutdown_connections 2024-05-08T23:07:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.751+0000 7f277a200700 1 -- 172.21.15.46:0/2373075499 wait complete. 2024-05-08T23:07:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.753+0000 7f277a200700 1 Processor -- start 2024-05-08T23:07:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.753+0000 7f277a200700 1 -- start start 2024-05-08T23:07:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 0x7f27741a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419dcb0 con 0x7f27741087a0 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419de20 con 0x7f2774107de0 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f277a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419dfe0 con 0x7f277410d4f0 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f2773400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f2773e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f2773400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57630/0 (socket says 172.21.15.46:57630) 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.754+0000 7f2773400700 1 -- 172.21.15.46:0/1558637193 learned_addr learned my addr 172.21.15.46:0/1558637193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2772a00700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 0x7f27741a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 -- 172.21.15.46:0/1558637193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 msgr2=0x7f27741a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 -- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 msgr2=0x7f27741a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 0x7f27741a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 -- 172.21.15.46:0/1558637193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640095f0 con 0x7f277410d4f0 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773400700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.755+0000 7f2773e00700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277419d650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f276c00cc50 tx=0x7f276c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.756+0000 7f276be00700 1 -- 172.21.15.46:0/1558637193 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00aa90 con 0x7f277410d4f0 2024-05-08T23:07:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.756+0000 7f276be00700 1 -- 172.21.15.46:0/1558637193 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c00abf0 con 0x7f277410d4f0 2024-05-08T23:07:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.756+0000 7f276be00700 1 -- 172.21.15.46:0/1558637193 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0115e0 con 0x7f277410d4f0 2024-05-08T23:07:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.757+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277419e2c0 con 0x7f277410d4f0 2024-05-08T23:07:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.757+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741aa510 con 0x7f277410d4f0 2024-05-08T23:07:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.761+0000 7f276be00700 1 -- 172.21.15.46:0/1558637193 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f276c012420 con 0x7f277410d4f0 2024-05-08T23:07:25.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.763+0000 7f276be00700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 0x7f27580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.763+0000 7f278dcd6b80 1 Processor -- start 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.763+0000 7f276be00700 1 -- 172.21.15.46:0/1558637193 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f276c0cce00 con 0x7f277410d4f0 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.763+0000 7f278dcd6b80 1 -- start start 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.763+0000 7f2773400700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 0x7f27580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcce00 0x55c926fcd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcd720 0x55c926fc7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926f1e150 con 0x55c926fcce00 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926f1bb10 con 0x55c926fcd720 2024-05-08T23:07:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926f17400 con 0x55c926fc7890 2024-05-08T23:07:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2773400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcce00 0x55c926fcd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2773e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2773e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50932/0 (socket says 172.21.15.46:50932) 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2773e00700 1 -- 172.21.15.46:0/699756582 learned_addr learned my addr 172.21.15.46:0/699756582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2772a00700 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcd720 0x55c926fc7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.764+0000 7f2773400700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 0x7f27580a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f276400b5c0 tx=0x7f2764005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 -- 172.21.15.46:0/699756582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcd720 msgr2=0x55c926fc7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcd720 0x55c926fc7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 -- 172.21.15.46:0/699756582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcce00 msgr2=0x55c926fcd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcce00 0x55c926fcd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 -- 172.21.15.46:0/699756582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c926d983b0 con 0x55c926fc7890 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f2773e00700 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f276c0966e0 tx=0x7f276c099b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82158e88222d1c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f275fe00700 1 -- 172.21.15.46:0/699756582 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c099c70 con 0x55c926fc7890 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f275fe00700 1 -- 172.21.15.46:0/699756582 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c09daa0 con 0x55c926fc7890 2024-05-08T23:07:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.765+0000 7f275fe00700 1 -- 172.21.15.46:0/699756582 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c012930 con 0x55c926fc7890 2024-05-08T23:07:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 -- 172.21.15.46:0/699756582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 msgr2=0x55c926fc7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f276c0966e0 tx=0x7f276c099b70 comp rx=0 tx=0).stop 2024-05-08T23:07:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 -- 172.21.15.46:0/699756582 shutdown_connections 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7cd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcce00 0x55c926fcd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/699756582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcd720 0x55c926fc7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 -- 172.21.15.46:0/699756582 >> 172.21.15.46:0/699756582 conn(0x55c926f25210 msgr2=0x55c926fc1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 -- 172.21.15.46:0/699756582 shutdown_connections 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.766+0000 7f278dcd6b80 1 -- 172.21.15.46:0/699756582 wait complete. 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.767+0000 7f278dcd6b80 1 -- start start 2024-05-08T23:07:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.767+0000 7f278dcd6b80 1 Processor -- start 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926eb3560 con 0x55c926fcd720 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926eb3200 con 0x55c926fcce00 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926d9a6a0 con 0x55c926fc7890 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f278dcd6b80 10 client.0 _ll_register_callbacks cb 0x55c926ebcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50946/0 (socket says 172.21.15.46:50946) 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 -- 172.21.15.46:0/1669316363 learned_addr learned my addr 172.21.15.46:0/1669316363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773e00700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2772a00700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 -- 172.21.15.46:0/1669316363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 msgr2=0x55c926fd0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fd0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 -- 172.21.15.46:0/1669316363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 msgr2=0x55c926fd1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.768+0000 7f2773400700 1 -- 172.21.15.46:0/1669316363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741aa510 con 0x55c926fc7890 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f2773400700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2764005670 tx=0x7f2764021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8277a2deef6017 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f275cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276402b040 con 0x55c926fc7890 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f27640056a0 con 0x55c926fc7890 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c926e70270 con 0x55c926fc7890 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.769+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2764003e80 con 0x55c926fc7890 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 10 client.26707 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c926d44b60 con 0x55c926fc7890 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c926e70270 con 0x55c926fc7890 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2764030860 con 0x55c926fc7890 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 10 client.26707 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 10 client.26707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 20 client.26707 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c926fe11a0 0x55c926fe35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 4 client.26707 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c926f96810 con 0x55c926fe11a0 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.770+0000 7f2772a00700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c926fe11a0 0x55c926fe35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.771+0000 7f2772a00700 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c926fe11a0 0x55c926fe35c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b9c5034783a6aa6 server_cookie=4ac7edc66c78a22f in_seq=0 out_seq=0 2024-05-08T23:07:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.771+0000 7f275cc00700 10 client.26707 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f275cc00700 1 -- 172.21.15.46:0/1669316363 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c926f96810 con 0x55c926fe11a0 2024-05-08T23:07:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f275cc00700 10 client.26707 handle_command_reply: tid=0 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 client.26707 shutdown 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 msgr2=0x55c926fd37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2764005670 tx=0x7f2764021000 comp rx=0 tx=0).stop 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 shutdown_connections 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c926fe11a0 0x55c926fe35c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fd37b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f275cc00700 0 client.26707 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/1669316363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fd0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 >> 172.21.15.46:0/1669316363 conn(0x55c926f25210 msgr2=0x55c926fc15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 shutdown_connections 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 1 -- 172.21.15.46:0/1669316363 wait complete. 2024-05-08T23:07:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.772+0000 7f278dcd6b80 20 client.26707 trim_cache size 0 max 16384 2024-05-08T23:07:25.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:25 smithi046 ceph-mon[36898]: pgmap v111: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-08T23:07:25.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:25 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2420208673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:07:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 Processor -- start 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 -- start start 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fddc70 con 0x55c926fcd720 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.789+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fc8230 con 0x55c926fcce00 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fc3e80 con 0x55c926fc7890 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50950/0 (socket says 172.21.15.46:50950) 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 -- 172.21.15.46:0/4243376614 learned_addr learned my addr 172.21.15.46:0/4243376614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773e00700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2772a00700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 -- 172.21.15.46:0/4243376614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 msgr2=0x55c926fc1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fc1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 -- 172.21.15.46:0/4243376614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 msgr2=0x55c926fd8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773400700 1 -- 172.21.15.46:0/4243376614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c926edd560 con 0x55c926fc7890 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.790+0000 7f2773e00700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.791+0000 7f2773400700 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2764033030 tx=0x7f2764009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce359f7d0c673d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.791+0000 7f2769200700 1 -- 172.21.15.46:0/4243376614 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276402b040 con 0x55c926fc7890 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.791+0000 7f2769200700 1 -- 172.21.15.46:0/4243376614 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276401e030 con 0x55c926fc7890 2024-05-08T23:07:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.791+0000 7f2769200700 1 -- 172.21.15.46:0/4243376614 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400f530 con 0x55c926fc7890 2024-05-08T23:07:25.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4243376614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 msgr2=0x55c926fc7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2764033030 tx=0x7f2764009310 comp rx=0 tx=0).stop 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4243376614 shutdown_connections 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c926fd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fc7890 0x55c926fc7350 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4243376614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fcce00 0x55c926fc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4243376614 >> 172.21.15.46:0/4243376614 conn(0x55c926f25210 msgr2=0x55c926fdcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4243376614 shutdown_connections 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.792+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4243376614 wait complete. 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 -- start start 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 Processor -- start 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fc7890 0x55c927032640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c9270330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fd6260 con 0x55c926fcd720 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fd63a0 con 0x55c926fc7890 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.793+0000 7f278dcd6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c926fd6560 con 0x55c926fcce00 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f278dcd6b80 10 client.0 _ll_register_callbacks cb 0x7f2764005220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50960/0 (socket says 172.21.15.46:50960) 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 -- 172.21.15.46:0/4184750251 learned_addr learned my addr 172.21.15.46:0/4184750251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2773e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c9270330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2773400700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fc7890 0x55c927032640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 -- 172.21.15.46:0/4184750251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fc7890 msgr2=0x55c927032640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fc7890 0x55c927032640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 -- 172.21.15.46:0/4184750251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 msgr2=0x55c9270330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c9270330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 -- 172.21.15.46:0/4184750251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c926d44b60 con 0x55c926fcce00 2024-05-08T23:07:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f2772a00700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c926fcdb80 tx=0x7f2760003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4410df46cf22ea4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:25.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.794+0000 7f275cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:25.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760016040 con 0x55c926fcce00 2024-05-08T23:07:25.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f276c09daa0 con 0x55c926fcce00 2024-05-08T23:07:25.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2760004a30 con 0x55c926fcce00 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2760003fa0 con 0x55c926fcce00 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 10 client.26713 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c926d9ccd0 con 0x55c926fcce00 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c09daa0 con 0x55c926fcce00 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f27600117a0 con 0x55c926fcce00 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 10 client.26713 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 10 client.26713 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 20 client.26713 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c927049230 0x55c92704b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 4 client.26713 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.795+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c926f0fec0 con 0x55c927049230 2024-05-08T23:07:25.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f2773400700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c927049230 0x55c92704b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f275cc00700 10 client.26713 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f2773400700 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c927049230 0x55c92704b650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45b90f67da61b888 server_cookie=1a10061bbbc2747b in_seq=0 out_seq=0 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f275cc00700 1 -- 172.21.15.46:0/4184750251 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c926f0fec0 con 0x55c927049230 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f275cc00700 10 client.26713 handle_command_reply: tid=0 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.796+0000 7f278dcd6b80 1 client.26713 shutdown 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 msgr2=0x55c927032b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c926fcdb80 tx=0x7f2760003960 comp rx=0 tx=0).stop 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 shutdown_connections 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c927049230 0x55c92704b650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c926fcd720 0x55c9270330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c926fc7890 0x55c927032640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 --2- 172.21.15.46:0/4184750251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c926fcce00 0x55c927032b80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f275cc00700 0 client.26713 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 >> 172.21.15.46:0/4184750251 conn(0x55c926f25210 msgr2=0x55c926fdcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 shutdown_connections 2024-05-08T23:07:25.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 1 -- 172.21.15.46:0/4184750251 wait complete. 2024-05-08T23:07:25.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.797+0000 7f278dcd6b80 20 client.26713 trim_cache size 0 max 16384 2024-05-08T23:07:25.801 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 msgr2=0x7f27580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 0x7f27580a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f276400b5c0 tx=0x7f2764005fd0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 msgr2=0x7f277419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277419d650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f276c00cc50 tx=0x7f276c00cf60 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 shutdown_connections 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277410d4f0 0x7f277419d650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2774107de0 0x7f27741a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f27580a1290 0x7f27580a3750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 --2- 172.21.15.46:0/1558637193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27741087a0 0x7f27741a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.798+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 >> 172.21.15.46:0/1558637193 conn(0x7f2774076980 msgr2=0x7f2774078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.799+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 shutdown_connections 2024-05-08T23:07:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:25.799+0000 7f277a200700 1 -- 172.21.15.46:0/1558637193 wait complete. 2024-05-08T23:07:25.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:07:25.877 INFO:tasks.workunit.client.0.smithi046.stdout: 1 185501 9.51 MB/sec warmup 107 sec latency 10.034 ms 2024-05-08T23:07:25.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:25 smithi086 ceph-mon[78610]: pgmap v111: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-08T23:07:25.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:25 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2420208673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:07:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:25 smithi053 ceph-mon[77587]: pgmap v111: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-08T23:07:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:25 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2420208673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 -- 172.21.15.46:0/2878562633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33841028c0 msgr2=0x7f338410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 --2- 172.21.15.46:0/2878562633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33841028c0 0x7f338410ada0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f33700098f0 tx=0x7f3370009c00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 -- 172.21.15.46:0/2878562633 shutdown_connections 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 --2- 172.21.15.46:0/2878562633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338410b3c0 0x7f338410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 --2- 172.21.15.46:0/2878562633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3384101fa0 0x7f3384102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 --2- 172.21.15.46:0/2878562633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33841028c0 0x7f338410ada0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.086+0000 7f338bc00700 1 -- 172.21.15.46:0/2878562633 >> 172.21.15.46:0/2878562633 conn(0x7f33840fb760 msgr2=0x7f33840fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.087+0000 7f338bc00700 1 -- 172.21.15.46:0/2878562633 shutdown_connections 2024-05-08T23:07:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.087+0000 7f338bc00700 1 -- 172.21.15.46:0/2878562633 wait complete. 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 Processor -- start 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 -- start start 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33841028c0 0x7f3384199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338410b3c0 0x7f338419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419e470 con 0x7f338410b3c0 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419e5e0 con 0x7f33841028c0 2024-05-08T23:07:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.088+0000 7f338bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419e7a0 con 0x7f3384101fa0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50976/0 (socket says 172.21.15.46:50976) 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 -- 172.21.15.46:0/3172501072 learned_addr learned my addr 172.21.15.46:0/3172501072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389c00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338410b3c0 0x7f338419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3383e00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33841028c0 0x7f3384199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 -- 172.21.15.46:0/3172501072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33841028c0 msgr2=0x7f3384199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33841028c0 0x7f3384199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 -- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338410b3c0 msgr2=0x7f338419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338410b3c0 0x7f338419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 -- 172.21.15.46:0/3172501072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370009520 con 0x7f3384101fa0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.089+0000 7f3389200700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f337800e9c0 tx=0x7f337800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.090+0000 7f3381600700 1 -- 172.21.15.46:0/3172501072 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378004a30 con 0x7f3384101fa0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.090+0000 7f3381600700 1 -- 172.21.15.46:0/3172501072 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337800cc80 con 0x7f3384101fa0 2024-05-08T23:07:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.090+0000 7f3381600700 1 -- 172.21.15.46:0/3172501072 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780054e0 con 0x7f3384101fa0 2024-05-08T23:07:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.090+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338419ea80 con 0x7f3384101fa0 2024-05-08T23:07:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.090+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33841a62c0 con 0x7f3384101fa0 2024-05-08T23:07:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.094+0000 7f3381600700 1 -- 172.21.15.46:0/3172501072 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3378004b90 con 0x7f3384101fa0 2024-05-08T23:07:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.095+0000 7f339f68bb80 1 Processor -- start 2024-05-08T23:07:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.095+0000 7f339f68bb80 1 -- start start 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3381600700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 0x7f336c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9ba890 0x557a5e9bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e911150 con 0x557a5e9c0720 2024-05-08T23:07:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e90eb10 con 0x557a5e9ba890 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f339f68bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e90a400 con 0x557a5e9bfe00 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3381600700 1 -- 172.21.15.46:0/3172501072 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3378014070 con 0x7f3384101fa0 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3389200700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 0x7f336c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50986/0 (socket says 172.21.15.46:50986) 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3389c00700 1 -- 172.21.15.46:0/1756345095 learned_addr learned my addr 172.21.15.46:0/1756345095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3389c00700 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9ba890 0x557a5e9bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.096+0000 7f3383e00700 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 -- 172.21.15.46:0/1756345095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9ba890 msgr2=0x557a5e9bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9ba890 0x557a5e9bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 -- 172.21.15.46:0/1756345095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 msgr2=0x557a5e9ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 -- 172.21.15.46:0/1756345095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a5e78b3b0 con 0x557a5e9bfe00 2024-05-08T23:07:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389200700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 0x7f336c0a37d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33780ca500 tx=0x7f3378009f00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3389c00700 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f337c008cd0 tx=0x7f337c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea9605b2948323f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3376000700 1 -- 172.21.15.46:0/1756345095 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00e050 con 0x557a5e9bfe00 2024-05-08T23:07:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3376000700 1 -- 172.21.15.46:0/1756345095 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337c00f040 con 0x557a5e9bfe00 2024-05-08T23:07:26.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.097+0000 7f3376000700 1 -- 172.21.15.46:0/1756345095 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c018ee0 con 0x557a5e9bfe00 2024-05-08T23:07:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 -- 172.21.15.46:0/1756345095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 msgr2=0x557a5e9c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f337c008cd0 tx=0x7f337c009920 comp rx=0 tx=0).stop 2024-05-08T23:07:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 -- 172.21.15.46:0/1756345095 shutdown_connections 2024-05-08T23:07:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9ba890 0x557a5e9bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9bfe00 0x557a5e9c01e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1756345095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 -- 172.21.15.46:0/1756345095 >> 172.21.15.46:0/1756345095 conn(0x557a5e918210 msgr2=0x557a5e9b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 -- 172.21.15.46:0/1756345095 shutdown_connections 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.098+0000 7f339f68bb80 1 -- 172.21.15.46:0/1756345095 wait complete. 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.099+0000 7f339f68bb80 1 -- start start 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.099+0000 7f339f68bb80 1 Processor -- start 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5e9c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e8a6560 con 0x557a5e9bfe00 2024-05-08T23:07:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e8a6200 con 0x557a5e9c0720 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e78d6a0 con 0x557a5e9ba890 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f339f68bb80 10 client.0 _ll_register_callbacks cb 0x557a5e8afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f3383e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f3383e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57620/0 (socket says 172.21.15.46:57620) 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f3383e00700 1 -- 172.21.15.46:0/3945116485 learned_addr learned my addr 172.21.15.46:0/3945116485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.100+0000 7f3389c00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3389c00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5e9c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 -- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 msgr2=0x557a5e9c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 -- 172.21.15.46:0/3945116485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 msgr2=0x557a5e9c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5e9c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 -- 172.21.15.46:0/3945116485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c0089b0 con 0x557a5e9bfe00 2024-05-08T23:07:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3389c00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5e9c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f3383e00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3370016000 tx=0x7f337000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982fefde059c57d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f336aa00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.101+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337001c070 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f337000b770 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a5e863270 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f337000b930 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 10 client.16683 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a5e737b60 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a5e863270 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3370015020 con 0x557a5e9bfe00 2024-05-08T23:07:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 10 client.16683 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 10 client.16683 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 20 client.16683 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5e9d42c0 0x557a5e9d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 4 client.16683 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.102+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a5e989810 con 0x557a5e9d42c0 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.103+0000 7f3389c00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5e9d42c0 0x557a5e9d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.103+0000 7f3389c00700 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5e9d42c0 0x557a5e9d66e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74ea4d53b14e1674 server_cookie=63748ca9e3ad13c1 in_seq=0 out_seq=0 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.103+0000 7f336aa00700 10 client.16683 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:07:26.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.104+0000 7f336aa00700 1 -- 172.21.15.46:0/3945116485 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a5e989810 con 0x557a5e9d42c0 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.104+0000 7f336aa00700 10 client.16683 handle_command_reply: tid=0 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.104+0000 7f339f68bb80 1 client.16683 shutdown 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 msgr2=0x557a5e9c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3370016000 tx=0x7f337000f690 comp rx=0 tx=0).stop 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 shutdown_connections 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5e9c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5e9d42c0 0x557a5e9d66e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 --2- 172.21.15.46:0/3945116485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c3fe0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f336aa00700 0 client.16683 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:07:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 >> 172.21.15.46:0/3945116485 conn(0x557a5e918210 msgr2=0x557a5e9b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 shutdown_connections 2024-05-08T23:07:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 1 -- 172.21.15.46:0/3945116485 wait complete. 2024-05-08T23:07:26.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.105+0000 7f339f68bb80 20 client.16683 trim_cache size 0 max 16384 2024-05-08T23:07:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 Processor -- start 2024-05-08T23:07:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 -- start start 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9bfe00 0x557a5e9b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e9b5e80 con 0x557a5e9c0720 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e9d0d70 con 0x557a5e9bfe00 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.122+0000 7f339f68bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e9c5110 con 0x557a5e9ba890 2024-05-08T23:07:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9ba350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51004/0 (socket says 172.21.15.46:51004) 2024-05-08T23:07:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 -- 172.21.15.46:0/1552503301 learned_addr learned my addr 172.21.15.46:0/1552503301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3383e00700 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9bfe00 0x557a5e9b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 -- 172.21.15.46:0/1552503301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 msgr2=0x557a5e9ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 -- 172.21.15.46:0/1552503301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9bfe00 msgr2=0x557a5e9b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.123+0000 7f3389c00700 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9bfe00 0x557a5e9b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.124+0000 7f3389c00700 1 -- 172.21.15.46:0/1552503301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a5e8d0560 con 0x557a5e9c0720 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.124+0000 7f3389c00700 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9cb160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f337c009690 tx=0x7f337c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250c2b12ef8fd5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.124+0000 7f3376a00700 1 -- 172.21.15.46:0/1552503301 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00e040 con 0x557a5e9c0720 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.124+0000 7f3376a00700 1 -- 172.21.15.46:0/1552503301 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337c018880 con 0x557a5e9c0720 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.124+0000 7f3376a00700 1 -- 172.21.15.46:0/1552503301 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c003d20 con 0x557a5e9c0720 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 -- 172.21.15.46:0/1552503301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 msgr2=0x557a5e9cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9cb160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f337c009690 tx=0x7f337c0093f0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 -- 172.21.15.46:0/1552503301 shutdown_connections 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9c0720 0x557a5e9cb160 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1552503301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9bfe00 0x557a5e9b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 -- 172.21.15.46:0/1552503301 >> 172.21.15.46:0/1552503301 conn(0x557a5e918210 msgr2=0x557a5e9c2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 -- 172.21.15.46:0/1552503301 shutdown_connections 2024-05-08T23:07:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.125+0000 7f339f68bb80 1 -- 172.21.15.46:0/1552503301 wait complete. 2024-05-08T23:07:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 -- start start 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 Processor -- start 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5ea29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5e9c7fe0 con 0x557a5e9bfe00 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5ea29cb0 con 0x557a5e9c0720 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.126+0000 7f339f68bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5ea29e70 con 0x557a5e9ba890 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f339f68bb80 10 client.0 _ll_register_callbacks cb 0x557a5e8afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3389c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51020/0 (socket says 172.21.15.46:51020) 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57630/0 (socket says 172.21.15.46:57630) 2024-05-08T23:07:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3389c00700 1 -- 172.21.15.46:0/1903967175 learned_addr learned my addr 172.21.15.46:0/1903967175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3389c00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5ea29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 -- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 msgr2=0x557a5e9c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 -- 172.21.15.46:0/1903967175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 msgr2=0x557a5ea29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5ea29770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 -- 172.21.15.46:0/1903967175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c030040 con 0x557a5e9bfe00 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.127+0000 7f3383e00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3370003910 tx=0x7f3370009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0406dbb2b06543 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337001c040 con 0x557a5e9bfe00 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3370000e40 con 0x557a5e9bfe00 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f337c00f040 con 0x557a5e9bfe00 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f337000b2c0 con 0x557a5e9bfe00 2024-05-08T23:07:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 10 client.16689 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a5e78fcd0 con 0x557a5e9bfe00 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337c00f040 con 0x557a5e9bfe00 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3370009920 con 0x557a5e9bfe00 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 10 client.16689 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 10 client.16689 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.128+0000 7f339f68bb80 20 client.16689 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5ea2b1b0 0x557a5ea2d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f339f68bb80 4 client.16689 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a5e902ec0 con 0x557a5ea2b1b0 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f3389c00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5ea2b1b0 0x557a5ea2d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f3389c00700 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5ea2b1b0 0x557a5ea2d5d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=233e81f42c9213fb server_cookie=f668ee346c3d8791 in_seq=0 out_seq=0 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f336aa00700 10 client.16689 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f336aa00700 1 -- 172.21.15.46:0/1903967175 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a5e902ec0 con 0x557a5ea2b1b0 2024-05-08T23:07:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f336aa00700 10 client.16689 handle_command_reply: tid=0 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.129+0000 7f339f68bb80 1 client.16689 shutdown 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 msgr2=0x557a5e9c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3370003910 tx=0x7f3370009f50 comp rx=0 tx=0).stop 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 shutdown_connections 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a5e9c0720 0x557a5ea29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a5e9ba890 0x557a5e9c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557a5ea2b1b0 0x557a5ea2d5d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 --2- 172.21.15.46:0/1903967175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a5e9bfe00 0x557a5e9c7870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f336aa00700 0 client.16689 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 >> 172.21.15.46:0/1903967175 conn(0x557a5e918210 msgr2=0x557a5e9c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 shutdown_connections 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 1 -- 172.21.15.46:0/1903967175 wait complete. 2024-05-08T23:07:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.130+0000 7f339f68bb80 20 client.16689 trim_cache size 0 max 16384 2024-05-08T23:07:26.135 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 msgr2=0x7f336c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 0x7f336c0a37d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33780ca500 tx=0x7f3378009f00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 msgr2=0x7f3384199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f337800e9c0 tx=0x7f337800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 shutdown_connections 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f336c0a1310 0x7f336c0a37d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338410b3c0 0x7f338419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3384101fa0 0x7f3384199550 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 --2- 172.21.15.46:0/3172501072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33841028c0 0x7f3384199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 >> 172.21.15.46:0/3172501072 conn(0x7f33840fb760 msgr2=0x7f33840fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 shutdown_connections 2024-05-08T23:07:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.131+0000 7f338bc00700 1 -- 172.21.15.46:0/3172501072 wait complete. 2024-05-08T23:07:26.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 -- 172.21.15.46:0/1755234376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf9810d680 msgr2=0x7fdf9810fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 --2- 172.21.15.46:0/1755234376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf9810d680 0x7fdf9810fab0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000b0a0 tx=0x7fdf9000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 -- 172.21.15.46:0/1755234376 shutdown_connections 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 --2- 172.21.15.46:0/1755234376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf9810d680 0x7fdf9810fab0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 --2- 172.21.15.46:0/1755234376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 0x7fdf98073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 --2- 172.21.15.46:0/1755234376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf98073c90 0x7fdf9810d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 -- 172.21.15.46:0/1755234376 >> 172.21.15.46:0/1755234376 conn(0x7fdf980fc8f0 msgr2=0x7fdf980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 -- 172.21.15.46:0/1755234376 shutdown_connections 2024-05-08T23:07:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.385+0000 7fdf9d200700 1 -- 172.21.15.46:0/1755234376 wait complete. 2024-05-08T23:07:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.386+0000 7fdf9d200700 1 Processor -- start 2024-05-08T23:07:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.387+0000 7fdf9d200700 1 -- start start 2024-05-08T23:07:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 0x7fdf981996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf9810d680 0x7fdf9819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9819e5e0 con 0x7fdf98073370 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9819e750 con 0x7fdf9810d680 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf9d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9819e910 con 0x7fdf98073c90 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51040/0 (socket says 172.21.15.46:51040) 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 -- 172.21.15.46:0/4137703668 learned_addr learned my addr 172.21.15.46:0/4137703668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 0x7fdf981996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf96800700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf9810d680 0x7fdf9819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 -- 172.21.15.46:0/4137703668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf9810d680 msgr2=0x7fdf9819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf9810d680 0x7fdf9819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 -- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 msgr2=0x7fdf981996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 0x7fdf981996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.388+0000 7fdf95400700 1 -- 172.21.15.46:0/4137703668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90009d00 con 0x7fdf98073c90 2024-05-08T23:07:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf95400700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e9c0 tx=0x7fdf8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf8ea00700 1 -- 172.21.15.46:0/4137703668 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840097b0 con 0x7fdf98073c90 2024-05-08T23:07:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf8ea00700 1 -- 172.21.15.46:0/4137703668 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8400ee60 con 0x7fdf98073c90 2024-05-08T23:07:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9819ebf0 con 0x7fdf98073c90 2024-05-08T23:07:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf8ea00700 1 -- 172.21.15.46:0/4137703668 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840054f0 con 0x7fdf98073c90 2024-05-08T23:07:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.389+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf981a6430 con 0x7fdf98073c90 2024-05-08T23:07:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.392+0000 7fdf8ea00700 1 -- 172.21.15.46:0/4137703668 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdf84004750 con 0x7fdf98073c90 2024-05-08T23:07:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.394+0000 7fdf8ea00700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 0x7fdf7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.394+0000 7fdf8ea00700 1 -- 172.21.15.46:0/4137703668 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf84014070 con 0x7fdf98073c90 2024-05-08T23:07:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.394+0000 7fdfb0d51b80 1 Processor -- start 2024-05-08T23:07:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.394+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 0x7fdf7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.394+0000 7fdfb0d51b80 1 -- start start 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8456350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add83ad150 con 0x55add8456890 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add83aab10 con 0x55add845c720 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add83a6400 con 0x55add845be00 2024-05-08T23:07:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 0x7fdf7c0a37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009c60 tx=0x7fdf88009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf96800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57662/0 (socket says 172.21.15.46:57662) 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf96800700 1 -- 172.21.15.46:0/1069669678 learned_addr learned my addr 172.21.15.46:0/1069669678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95400700 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8456350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 -- 172.21.15.46:0/1069669678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 msgr2=0x55add8456350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8456350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 -- 172.21.15.46:0/1069669678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 msgr2=0x55add8456cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.395+0000 7fdf95e00700 1 -- 172.21.15.46:0/1069669678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55add82273b0 con 0x55add845be00 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.396+0000 7fdf95e00700 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845c1e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005950 tx=0x7fdf88014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e555cb3131e1e29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.396+0000 7fdf83400700 1 -- 172.21.15.46:0/1069669678 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800ecf0 con 0x55add845be00 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.396+0000 7fdf83400700 1 -- 172.21.15.46:0/1069669678 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88021070 con 0x55add845be00 2024-05-08T23:07:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.396+0000 7fdf83400700 1 -- 172.21.15.46:0/1069669678 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88026480 con 0x55add845be00 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/1069669678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 msgr2=0x55add845c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845c1e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005950 tx=0x7fdf88014000 comp rx=0 tx=0).stop 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/1069669678 shutdown_connections 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845c1e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/1069669678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8456350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/1069669678 >> 172.21.15.46:0/1069669678 conn(0x55add83b4210 msgr2=0x55add8450df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/1069669678 shutdown_connections 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.397+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/1069669678 wait complete. 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 -- start start 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 Processor -- start 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add84600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845c720 0x55add8460600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.398+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add8342560 con 0x55add845c720 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add8342200 con 0x55add8456890 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add82296a0 con 0x55add845be00 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdfb0d51b80 10 client.0 _ll_register_callbacks cb 0x55add834bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add84600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57774/0 (socket says 172.21.15.46:57774) 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 -- 172.21.15.46:0/2429650595 learned_addr learned my addr 172.21.15.46:0/2429650595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add84600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51048/0 (socket says 172.21.15.46:51048) 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf96800700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845c720 0x55add8460600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 -- 172.21.15.46:0/2429650595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 msgr2=0x55add84600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add84600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 -- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845c720 msgr2=0x55add8460600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845c720 0x55add8460600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 -- 172.21.15.46:0/2429650595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf88017070 con 0x55add8456890 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf95e00700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005550 tx=0x7fdf88005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b36a532f455895e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.399+0000 7fdf77e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802a040 con 0x55add8456890 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf88004750 con 0x55add8456890 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55add82ff270 con 0x55add8456890 2024-05-08T23:07:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf88004910 con 0x55add8456890 2024-05-08T23:07:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdfb0d51b80 10 client.26690 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55add81d3b60 con 0x55add8456890 2024-05-08T23:07:26.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.400+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55add82ff270 con 0x55add8456890 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf880269d0 con 0x55add8456890 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 10 client.26690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 20 client.26690 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84700d0 0x55add84724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 4 client.26690 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55add8425810 con 0x55add84700d0 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdf95400700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84700d0 0x55add84724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdf95400700 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84700d0 0x55add84724f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4faae14778ae0d7c server_cookie=3a3db75deaacda36 in_seq=0 out_seq=0 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.401+0000 7fdf77e00700 10 client.26690 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.402+0000 7fdf77e00700 1 -- 172.21.15.46:0/2429650595 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55add8425810 con 0x55add84700d0 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.402+0000 7fdf77e00700 10 client.26690 handle_command_reply: tid=0 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.402+0000 7fdfb0d51b80 1 client.26690 shutdown 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.402+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 msgr2=0x55add845fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.402+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005550 tx=0x7fdf88005580 comp rx=0 tx=0).stop 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 shutdown_connections 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84700d0 0x55add84724f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845c720 0x55add8460600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add8456890 0x55add845fb80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdf77e00700 0 client.26690 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:07:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2429650595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add84600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 >> 172.21.15.46:0/2429650595 conn(0x55add83b4210 msgr2=0x55add84505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 shutdown_connections 2024-05-08T23:07:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2429650595 wait complete. 2024-05-08T23:07:26.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.403+0000 7fdfb0d51b80 20 client.26690 trim_cache size 0 max 16384 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 Processor -- start 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 -- start start 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8450d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add846cda0 con 0x55add8456890 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add84611b0 con 0x55add845c720 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.420+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add8452640 con 0x55add845be00 2024-05-08T23:07:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57680/0 (socket says 172.21.15.46:57680) 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51050/0 (socket says 172.21.15.46:51050) 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95e00700 1 -- 172.21.15.46:0/2459123345 learned_addr learned my addr 172.21.15.46:0/2459123345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf96800700 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8450d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 -- 172.21.15.46:0/2459123345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 msgr2=0x55add8450d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8450d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 -- 172.21.15.46:0/2459123345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 msgr2=0x55add8456350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 -- 172.21.15.46:0/2459123345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55add836c560 con 0x55add845be00 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95e00700 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.421+0000 7fdf95400700 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdf840ca580 tx=0x7fdf8409c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4833ddb369aaed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.422+0000 7fdf83e00700 1 -- 172.21.15.46:0/2459123345 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84099fc0 con 0x55add845be00 2024-05-08T23:07:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.422+0000 7fdf83e00700 1 -- 172.21.15.46:0/2459123345 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8409dd70 con 0x55add845be00 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.422+0000 7fdf83e00700 1 -- 172.21.15.46:0/2459123345 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840ca800 con 0x55add845be00 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2459123345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 msgr2=0x55add845b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdf840ca580 tx=0x7fdf8409c040 comp rx=0 tx=0).stop 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2459123345 shutdown_connections 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add8450d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add8456890 0x55add8456350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/2459123345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add845be00 0x55add845b350 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2459123345 >> 172.21.15.46:0/2459123345 conn(0x55add83b4210 msgr2=0x55add845ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2459123345 shutdown_connections 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/2459123345 wait complete. 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.423+0000 7fdfb0d51b80 1 -- start start 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 Processor -- start 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 0x55add84657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add84c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add8465f50 con 0x55add845be00 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add84c5b40 con 0x55add845c720 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add84c5d00 con 0x55add8456890 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdfb0d51b80 10 client.0 _ll_register_callbacks cb 0x55add834bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdf95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51066/0 (socket says 172.21.15.46:51066) 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdf95e00700 1 -- 172.21.15.46:0/4090617502 learned_addr learned my addr 172.21.15.46:0/4090617502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdf95400700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 0x55add84657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.424+0000 7fdf96800700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add84c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 -- 172.21.15.46:0/4090617502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 msgr2=0x55add84c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add84c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 -- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 msgr2=0x55add84657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 0x55add84657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 -- 172.21.15.46:0/4090617502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55add81d3b60 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf96800700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add84c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95400700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 0x55add84657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf95e00700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005930 tx=0x7fdf88005180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd589945920ef63e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf77e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802a070 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf88021070 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdf88026680 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.425+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf88026840 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 10 client.26749 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fdf981a6430 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88021070 con 0x55add8456890 2024-05-08T23:07:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf8802f670 con 0x55add8456890 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 10 client.26749 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 10 client.26749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 20 client.26749 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84c7180 0x55add84c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 4 client.26749 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55add839eec0 con 0x55add84c7180 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.426+0000 7fdf95400700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84c7180 0x55add84c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdf95400700 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84c7180 0x55add84c95a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf97e2343aab6475 server_cookie=766f1fed1977433c in_seq=0 out_seq=0 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdf77e00700 10 client.26749 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdf77e00700 1 -- 172.21.15.46:0/4090617502 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55add839eec0 con 0x55add84c7180 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdf77e00700 10 client.26749 handle_command_reply: tid=0 2024-05-08T23:07:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 client.26749 shutdown 2024-05-08T23:07:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 msgr2=0x55add8465270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005930 tx=0x7fdf88005180 comp rx=0 tx=0).stop 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 shutdown_connections 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55add845c720 0x55add84c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55add8456890 0x55add8465270 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55add84c7180 0x55add84c95a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 --2- 172.21.15.46:0/4090617502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55add845be00 0x55add84657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 >> 172.21.15.46:0/4090617502 conn(0x55add83b4210 msgr2=0x55add8460460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.427+0000 7fdf77e00700 0 client.26749 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 shutdown_connections 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdfb0d51b80 1 -- 172.21.15.46:0/4090617502 wait complete. 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdfb0d51b80 20 client.26749 trim_cache size 0 max 16384 2024-05-08T23:07:26.434 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 msgr2=0x7fdf7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 0x7fdf7c0a37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009c60 tx=0x7fdf88009260 comp rx=0 tx=0).stop 2024-05-08T23:07:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 msgr2=0x7fdf98199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.428+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e9c0 tx=0x7fdf8400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 shutdown_connections 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf9810d680 0x7fdf9819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf7c0a1310 0x7fdf7c0a37d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf98073370 0x7fdf981996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 --2- 172.21.15.46:0/4137703668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98073c90 0x7fdf98199c00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 >> 172.21.15.46:0/4137703668 conn(0x7fdf980fc8f0 msgr2=0x7fdf980fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 shutdown_connections 2024-05-08T23:07:26.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.429+0000 7fdf9d200700 1 -- 172.21.15.46:0/4137703668 wait complete. 2024-05-08T23:07:26.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.686+0000 7fa931e00700 1 -- 172.21.15.46:0/1409563444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c101080 msgr2=0x7fa92c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.686+0000 7fa931e00700 1 --2- 172.21.15.46:0/1409563444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c101080 0x7fa92c101460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0098f0 tx=0x7fa91c009c00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 -- 172.21.15.46:0/1409563444 shutdown_connections 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 --2- 172.21.15.46:0/1409563444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 0x7fa92c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 --2- 172.21.15.46:0/1409563444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c101080 0x7fa92c101460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 --2- 172.21.15.46:0/1409563444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101a30 0x7fa92c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 -- 172.21.15.46:0/1409563444 >> 172.21.15.46:0/1409563444 conn(0x7fa92c0fc8e0 msgr2=0x7fa92c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 -- 172.21.15.46:0/1409563444 shutdown_connections 2024-05-08T23:07:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.687+0000 7fa931e00700 1 -- 172.21.15.46:0/1409563444 wait complete. 2024-05-08T23:07:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.688+0000 7fa931e00700 1 Processor -- start 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.688+0000 7fa931e00700 1 -- start start 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 0x7fa92c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 0x7fa92c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19e930 con 0x7fa92c101080 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19eaa0 con 0x7fa92c1061a0 2024-05-08T23:07:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa931e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19ec60 con 0x7fa92c19deb0 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57700/0 (socket says 172.21.15.46:57700) 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 -- 172.21.15.46:0/3941955979 learned_addr learned my addr 172.21.15.46:0/3941955979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92a000700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 0x7fa92c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92b400700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 0x7fa92c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 -- 172.21.15.46:0/3941955979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 msgr2=0x7fa92c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 0x7fa92c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 -- 172.21.15.46:0/3941955979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 msgr2=0x7fa92c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 0x7fa92c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92aa00700 1 -- 172.21.15.46:0/3941955979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c009520 con 0x7fa92c101080 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.689+0000 7fa92b400700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 0x7fa92c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005950 tx=0x7fa91c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa930a00700 1 -- 172.21.15.46:0/3941955979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c01c070 con 0x7fa92c101080 2024-05-08T23:07:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa930a00700 1 -- 172.21.15.46:0/3941955979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91c00bac0 con 0x7fa92c101080 2024-05-08T23:07:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c1a6160 con 0x7fa92c101080 2024-05-08T23:07:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa930a00700 1 -- 172.21.15.46:0/3941955979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0038e0 con 0x7fa92c101080 2024-05-08T23:07:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.690+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a6580 con 0x7fa92c101080 2024-05-08T23:07:26.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.693+0000 7fa930a00700 1 -- 172.21.15.46:0/3941955979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa91c020030 con 0x7fa92c101080 2024-05-08T23:07:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.695+0000 7fa930a00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 0x7fa9100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.695+0000 7fa930a00700 1 -- 172.21.15.46:0/3941955979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa91c0cdfc0 con 0x7fa92c101080 2024-05-08T23:07:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.695+0000 7fa9459fdb80 1 Processor -- start 2024-05-08T23:07:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.695+0000 7fa92a000700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 0x7fa9100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.695+0000 7fa9459fdb80 1 -- start start 2024-05-08T23:07:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c02549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c024f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c019d150 con 0x55f8c024f0a0 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c019ab10 con 0x55f8c02545f0 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c0196400 con 0x55f8c0254f10 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92a000700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 0x7fa9100a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa92c19ab20 tx=0x7fa918009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51096/0 (socket says 172.21.15.46:51096) 2024-05-08T23:07:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92a000700 1 -- 172.21.15.46:0/3452978625 learned_addr learned my addr 172.21.15.46:0/3452978625 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92b400700 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c024f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.696+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c02549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 -- 172.21.15.46:0/3452978625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 msgr2=0x55f8c02549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c02549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 -- 172.21.15.46:0/3452978625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 msgr2=0x55f8c024f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c024f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 -- 172.21.15.46:0/3452978625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8c00173b0 con 0x55f8c0254f10 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa92a000700 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa91800a000 tx=0x7fa918014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83078dbd3db90194 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa917e00700 1 -- 172.21.15.46:0/3452978625 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918004a30 con 0x55f8c0254f10 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa917e00700 1 -- 172.21.15.46:0/3452978625 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa918003cf0 con 0x55f8c0254f10 2024-05-08T23:07:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.697+0000 7fa917e00700 1 -- 172.21.15.46:0/3452978625 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180055d0 con 0x55f8c0254f10 2024-05-08T23:07:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3452978625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 msgr2=0x55f8c024eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa91800a000 tx=0x7fa918014000 comp rx=0 tx=0).stop 2024-05-08T23:07:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3452978625 shutdown_connections 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c024f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c02549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3452978625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c024eb60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3452978625 >> 172.21.15.46:0/3452978625 conn(0x55f8c01a4210 msgr2=0x55f8c0240df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.698+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3452978625 shutdown_connections 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.699+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3452978625 wait complete. 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.699+0000 7fa9459fdb80 1 -- start start 2024-05-08T23:07:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.699+0000 7fa9459fdb80 1 Processor -- start 2024-05-08T23:07:26.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c0258a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c02545f0 0x55f8c0258f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c0132560 con 0x55f8c024f0a0 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c0132200 con 0x55f8c0254f10 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c00196a0 con 0x55f8c02545f0 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa9459fdb80 10 client.0 _ll_register_callbacks cb 0x55f8c013bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa92b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa92a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c02545f0 0x55f8c0258f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa92b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57820/0 (socket says 172.21.15.46:57820) 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.700+0000 7fa92b400700 1 -- 172.21.15.46:0/614650468 learned_addr learned my addr 172.21.15.46:0/614650468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 -- 172.21.15.46:0/614650468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c02545f0 msgr2=0x55f8c0258f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c02545f0 0x55f8c0258f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 -- 172.21.15.46:0/614650468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 msgr2=0x55f8c0258a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c0258a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 -- 172.21.15.46:0/614650468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa918017070 con 0x55f8c0254f10 2024-05-08T23:07:26.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa92b400700 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa924007ed0 tx=0x7fa924007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4417072d778c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa914c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e070 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8c00ef270 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.701+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa924013b30 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.702+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa924009180 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.702+0000 7fa9459fdb80 10 client.26708 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.702+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f8bffc3b60 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.702+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8c00ef270 con 0x55f8c0254f10 2024-05-08T23:07:26.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.702+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa92401e3b0 con 0x55f8c0254f10 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 10 client.26708 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 10 client.26708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 20 client.26708 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c0260320 0x55f8c0262740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 4 client.26708 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8c0215810 con 0x55f8c0260320 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa92aa00700 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c0260320 0x55f8c0262740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa92aa00700 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c0260320 0x55f8c0262740 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff81daaf4e8dff0 server_cookie=a40000f9e882aa08 in_seq=0 out_seq=0 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.703+0000 7fa914c00700 10 client.26708 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa914c00700 1 -- 172.21.15.46:0/614650468 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8c0215810 con 0x55f8c0260320 2024-05-08T23:07:26.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa914c00700 10 client.26708 handle_command_reply: tid=0 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 client.26708 shutdown 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 msgr2=0x55f8c024d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa924007ed0 tx=0x7fa924007f00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 shutdown_connections 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c0254f10 0x55f8c024d6b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c0260320 0x55f8c0262740 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c024f0a0 0x55f8c0258a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/614650468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c02545f0 0x55f8c0258f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa914c00700 0 client.26708 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.704+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 >> 172.21.15.46:0/614650468 conn(0x55f8c01a4210 msgr2=0x55f8c02405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.705+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 shutdown_connections 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.705+0000 7fa9459fdb80 1 -- 172.21.15.46:0/614650468 wait complete. 2024-05-08T23:07:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.705+0000 7fa9459fdb80 20 client.26708 trim_cache size 0 max 16384 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.722+0000 7fa9459fdb80 1 Processor -- start 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.722+0000 7fa9459fdb80 1 -- start start 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.722+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.722+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c0240d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.722+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c0254f10 0x55f8c024d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c0254c20 con 0x55f8c0254f10 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c024e740 con 0x55f8c02545f0 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c0259ea0 con 0x55f8c024f0a0 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51116/0 (socket says 172.21.15.46:51116) 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 -- 172.21.15.46:0/3906161872 learned_addr learned my addr 172.21.15.46:0/3906161872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92a000700 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c0240d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92b400700 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c0254f10 0x55f8c024d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 -- 172.21.15.46:0/3906161872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 msgr2=0x55f8c0240d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c0240d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 -- 172.21.15.46:0/3906161872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c0254f10 msgr2=0x55f8c024d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c0254f10 0x55f8c024d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 -- 172.21.15.46:0/3906161872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8c015c560 con 0x55f8c024f0a0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.723+0000 7fa92aa00700 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0977c0 tx=0x7fa91c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b330f615da3b103 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.724+0000 7fa921200700 1 -- 172.21.15.46:0/3906161872 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c09eb80 con 0x55f8c024f0a0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.724+0000 7fa921200700 1 -- 172.21.15.46:0/3906161872 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91c0a3140 con 0x55f8c024f0a0 2024-05-08T23:07:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.724+0000 7fa921200700 1 -- 172.21.15.46:0/3906161872 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0051d0 con 0x55f8c024f0a0 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.724+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3906161872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 msgr2=0x55f8c02563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.724+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0977c0 tx=0x7fa91c009c00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3906161872 shutdown_connections 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c0254f10 0x55f8c024d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c024f0a0 0x55f8c02563b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/3906161872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c02545f0 0x55f8c0240d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3906161872 >> 172.21.15.46:0/3906161872 conn(0x55f8c01a4210 msgr2=0x55f8c02529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3906161872 shutdown_connections 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 -- 172.21.15.46:0/3906161872 wait complete. 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 -- start start 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.725+0000 7fa9459fdb80 1 Processor -- start 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c02545f0 0x55f8c0248b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c0249070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c02b5820 con 0x55f8c02545f0 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c02b5960 con 0x55f8c024f0a0 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c02b5af0 con 0x55f8c0254f10 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa9459fdb80 10 client.0 _ll_register_callbacks cb 0x55f8c013bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa92aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57836/0 (socket says 172.21.15.46:57836) 2024-05-08T23:07:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa92aa00700 1 -- 172.21.15.46:0/1608098014 learned_addr learned my addr 172.21.15.46:0/1608098014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa92b400700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c0249070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.726+0000 7fa92a000700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c02545f0 0x55f8c0248b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 -- 172.21.15.46:0/1608098014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 msgr2=0x55f8c0249070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c0249070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 -- 172.21.15.46:0/1608098014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c02545f0 msgr2=0x55f8c0248b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c02545f0 0x55f8c0248b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 -- 172.21.15.46:0/1608098014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a6580 con 0x55f8c024f0a0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa92aa00700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91c09aa50 tx=0x7fa91c09ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8a89766e64431 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa914c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0cb580 con 0x55f8c024f0a0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa91c0cb6e0 con 0x55f8c024f0a0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.727+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa918003cf0 con 0x55f8c024f0a0 2024-05-08T23:07:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91c0051d0 con 0x55f8c024f0a0 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 10 client.26714 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f8c001bcd0 con 0x55f8c024f0a0 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa918003cf0 con 0x55f8c024f0a0 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa91c015020 con 0x55f8c024f0a0 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 10 client.26714 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 20 client.26714 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c02b6dc0 0x55f8c02b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 4 client.26714 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.728+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8c018eec0 con 0x55f8c02b6dc0 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.729+0000 7fa92a000700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c02b6dc0 0x55f8c02b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa92a000700 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c02b6dc0 0x55f8c02b91b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ac7d5191feb008 server_cookie=85ccff1393cb0aed in_seq=0 out_seq=0 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa914c00700 10 client.26714 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa914c00700 1 -- 172.21.15.46:0/1608098014 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8c018eec0 con 0x55f8c02b6dc0 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa914c00700 10 client.26714 handle_command_reply: tid=0 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 client.26714 shutdown 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 msgr2=0x55f8c02485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91c09aa50 tx=0x7fa91c09ae10 comp rx=0 tx=0).stop 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 shutdown_connections 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c0254f10 0x55f8c0249070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8c024f0a0 0x55f8c02485f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f8c02b6dc0 0x55f8c02b91b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 --2- 172.21.15.46:0/1608098014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c02545f0 0x55f8c0248b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa914c00700 0 client.26714 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 >> 172.21.15.46:0/1608098014 conn(0x55f8c01a4210 msgr2=0x55f8c024d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 shutdown_connections 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.730+0000 7fa9459fdb80 1 -- 172.21.15.46:0/1608098014 wait complete. 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.731+0000 7fa9459fdb80 20 client.26714 trim_cache size 0 max 16384 2024-05-08T23:07:26.734 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.731+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 msgr2=0x7fa9100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.731+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 0x7fa9100a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa92c19ab20 tx=0x7fa918009310 comp rx=0 tx=0).stop 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.731+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 msgr2=0x7fa92c1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.731+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005950 tx=0x7fa91c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 shutdown_connections 2024-05-08T23:07:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92c19deb0 0x7fa92c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa9100a1290 0x7fa9100a3750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c101080 0x7fa92c1995d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 --2- 172.21.15.46:0/3941955979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa92c1061a0 0x7fa92c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 >> 172.21.15.46:0/3941955979 conn(0x7fa92c0fc8e0 msgr2=0x7fa92c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 shutdown_connections 2024-05-08T23:07:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:26.732+0000 7fa931e00700 1 -- 172.21.15.46:0/3941955979 wait complete. 2024-05-08T23:07:26.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 188085 9.56 MB/sec warmup 108 sec latency 12.032 ms 2024-05-08T23:07:27.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:07:27.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:07:27.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 190097 9.54 MB/sec warmup 109 sec latency 6.820 ms 2024-05-08T23:07:27.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:27 smithi086 ceph-mon[78610]: pgmap v112: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.974+0000 7efbfa000700 1 -- 172.21.15.46:0/508684875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf41084e0 msgr2=0x7efbf4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 --2- 172.21.15.46:0/508684875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf41084e0 0x7efbf4108960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efbe00099c0 tx=0x7efbe0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 -- 172.21.15.46:0/508684875 shutdown_connections 2024-05-08T23:07:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 --2- 172.21.15.46:0/508684875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf4115fa0 0x7efbf4118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 --2- 172.21.15.46:0/508684875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf410b390 0x7efbf4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 --2- 172.21.15.46:0/508684875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf41084e0 0x7efbf4108960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 -- 172.21.15.46:0/508684875 >> 172.21.15.46:0/508684875 conn(0x7efbf406bd20 msgr2=0x7efbf406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 -- 172.21.15.46:0/508684875 shutdown_connections 2024-05-08T23:07:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.975+0000 7efbfa000700 1 -- 172.21.15.46:0/508684875 wait complete. 2024-05-08T23:07:27.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.976+0000 7efbfa000700 1 Processor -- start 2024-05-08T23:07:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.978+0000 7efbfa000700 1 -- start start 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 0x7efbf41a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf4115fa0 0x7efbf41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41a1d90 con 0x7efbf41084e0 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41a1f00 con 0x7efbf410b390 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbfa000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41a20c0 con 0x7efbf4115fa0 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbf3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbf3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57846/0 (socket says 172.21.15.46:57846) 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbf3e00700 1 -- 172.21.15.46:0/770741679 learned_addr learned my addr 172.21.15.46:0/770741679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbf9600700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf4115fa0 0x7efbf41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.979+0000 7efbf8c00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 0x7efbf41a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 -- 172.21.15.46:0/770741679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf4115fa0 msgr2=0x7efbf41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf4115fa0 0x7efbf41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 -- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 msgr2=0x7efbf41a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 0x7efbf41a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 -- 172.21.15.46:0/770741679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe00095f0 con 0x7efbf410b390 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf8c00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 0x7efbf41a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.980+0000 7efbf3e00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efbe00098e0 tx=0x7efbe00057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.981+0000 7efbf1600700 1 -- 172.21.15.46:0/770741679 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe001d070 con 0x7efbf410b390 2024-05-08T23:07:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.981+0000 7efbf1600700 1 -- 172.21.15.46:0/770741679 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbe000fc60 con 0x7efbf410b390 2024-05-08T23:07:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.981+0000 7efbf1600700 1 -- 172.21.15.46:0/770741679 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe00041c0 con 0x7efbf410b390 2024-05-08T23:07:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.981+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf41a2340 con 0x7efbf410b390 2024-05-08T23:07:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.981+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf410bd30 con 0x7efbf410b390 2024-05-08T23:07:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.986+0000 7efbf1600700 1 -- 172.21.15.46:0/770741679 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efbe00038c0 con 0x7efbf410b390 2024-05-08T23:07:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.989+0000 7efbf1600700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 0x7efbdc0a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.990+0000 7efbf8c00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 0x7efbdc0a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.990+0000 7efbf1600700 1 -- 172.21.15.46:0/770741679 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efbe00cdde0 con 0x7efbf410b390 2024-05-08T23:07:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.991+0000 7efc0f07fb80 1 Processor -- start 2024-05-08T23:07:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.991+0000 7efbf8c00700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 0x7efbdc0a39a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005fd0 tx=0x7efbe8005e20 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.992+0000 7efc0f07fb80 1 -- start start 2024-05-08T23:07:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a3d71d0 con 0x56254a4869f0 2024-05-08T23:07:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a3d4b90 con 0x56254a4860d0 2024-05-08T23:07:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a3d0480 con 0x56254a480910 2024-05-08T23:07:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efbf9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efbf9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51160/0 (socket says 172.21.15.46:51160) 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efbf9600700 1 -- 172.21.15.46:0/1723160773 learned_addr learned my addr 172.21.15.46:0/1723160773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efbf8c00700 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.994+0000 7efbf3e00700 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.995+0000 7efbf9600700 1 -- 172.21.15.46:0/1723160773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 msgr2=0x56254a4864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.995+0000 7efbf9600700 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.995+0000 7efbf9600700 1 -- 172.21.15.46:0/1723160773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 msgr2=0x56254a4803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.995+0000 7efbf9600700 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.995+0000 7efbf9600700 1 -- 172.21.15.46:0/1723160773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254a251430 con 0x56254a480910 2024-05-08T23:07:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.996+0000 7efbf9600700 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efbec00f570 tx=0x7efbec00f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bbaa6f44796e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.996+0000 7efbe5c00700 1 -- 172.21.15.46:0/1723160773 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec004bb0 con 0x56254a480910 2024-05-08T23:07:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.996+0000 7efbe5c00700 1 -- 172.21.15.46:0/1723160773 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbec004d10 con 0x56254a480910 2024-05-08T23:07:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.996+0000 7efbe5c00700 1 -- 172.21.15.46:0/1723160773 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec005060 con 0x56254a480910 2024-05-08T23:07:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1723160773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 msgr2=0x56254a480d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efbec00f570 tx=0x7efbec00f880 comp rx=0 tx=0).stop 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1723160773 shutdown_connections 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a480d50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1723160773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.998+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1723160773 >> 172.21.15.46:0/1723160773 conn(0x56254a3de290 msgr2=0x56254a47b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.999+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1723160773 shutdown_connections 2024-05-08T23:07:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:27.999+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1723160773 wait complete. 2024-05-08T23:07:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.000+0000 7efc0f07fb80 1 -- start start 2024-05-08T23:07:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.001+0000 7efc0f07fb80 1 Processor -- start 2024-05-08T23:07:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 0x56254a48c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a48a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a36c5e0 con 0x56254a4869f0 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a36c280 con 0x56254a480910 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a253720 con 0x56254a4860d0 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efc0f07fb80 10 client.0 _ll_register_callbacks cb 0x56254a375bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.002+0000 7efbf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51168/0 (socket says 172.21.15.46:51168) 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 -- 172.21.15.46:0/1690650009 learned_addr learned my addr 172.21.15.46:0/1690650009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf9600700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a48a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 -- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 msgr2=0x56254a48c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf8c00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 0x56254a48c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 0x56254a48c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 -- 172.21.15.46:0/1690650009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 msgr2=0x56254a48a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a48a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.003+0000 7efbf3e00700 1 -- 172.21.15.46:0/1690650009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbec00f1d0 con 0x56254a4860d0 2024-05-08T23:07:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efbf8c00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 0x56254a48c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efbf3e00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe009b380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2866ccfac89c1ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efbda000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe00ba040 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efbe00a2fc0 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.004+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254a3292f0 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe009bde0 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 10 client.26776 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56254a1fdbe0 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbe00b7070 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efbe00b3200 con 0x56254a4860d0 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 10 client.26776 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 20 client.26776 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a499fe0 0x56254a49c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 4 client.26776 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.005+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56254a44f890 con 0x56254a499fe0 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.006+0000 7efbf8c00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a499fe0 0x56254a49c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.006+0000 7efbf8c00700 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a499fe0 0x56254a49c400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961e45ca62793190 server_cookie=c4b26d8da9a66da7 in_seq=0 out_seq=0 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.006+0000 7efbda000700 10 client.26776 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efbda000700 1 -- 172.21.15.46:0/1690650009 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56254a44f890 con 0x56254a499fe0 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efbda000700 10 client.26776 handle_command_reply: tid=0 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 client.26776 shutdown 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 msgr2=0x56254a489c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe009b380 comp rx=0 tx=0).stop 2024-05-08T23:07:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 shutdown_connections 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a499fe0 0x56254a49c400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a48a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a480910 0x56254a48c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/1690650009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a4860d0 0x56254a489c10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 >> 172.21.15.46:0/1690650009 conn(0x56254a3de290 msgr2=0x56254a47b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.007+0000 7efbda000700 0 client.26776 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.008+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 shutdown_connections 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.008+0000 7efc0f07fb80 1 -- 172.21.15.46:0/1690650009 wait complete. 2024-05-08T23:07:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.008+0000 7efc0f07fb80 20 client.26776 trim_cache size 0 max 16384 2024-05-08T23:07:28.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:27 smithi053 ceph-mon[77587]: pgmap v112: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:28.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:27 smithi046 ceph-mon[36898]: pgmap v112: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 Processor -- start 2024-05-08T23:07:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 -- start start 2024-05-08T23:07:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4860d0 0x56254a49bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4869f0 0x56254a4a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a496ef0 con 0x56254a4860d0 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a4864b0 con 0x56254a4869f0 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.027+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a47c240 con 0x56254a480910 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51184/0 (socket says 172.21.15.46:51184) 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4869f0 0x56254a4a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 -- 172.21.15.46:0/2653965293 learned_addr learned my addr 172.21.15.46:0/2653965293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf3e00700 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4860d0 0x56254a49bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 -- 172.21.15.46:0/2653965293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4869f0 msgr2=0x56254a4a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4869f0 0x56254a4a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 -- 172.21.15.46:0/2653965293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4860d0 msgr2=0x56254a49bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4860d0 0x56254a49bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 -- 172.21.15.46:0/2653965293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254a477560 con 0x56254a480910 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.028+0000 7efbf8c00700 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efbe800ddf0 tx=0x7efbe8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdac7525b3cf70cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efbe6600700 1 -- 172.21.15.46:0/2653965293 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8023040 con 0x56254a480910 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efbe6600700 1 -- 172.21.15.46:0/2653965293 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbe801e440 con 0x56254a480910 2024-05-08T23:07:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efbe6600700 1 -- 172.21.15.46:0/2653965293 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800dee0 con 0x56254a480910 2024-05-08T23:07:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 -- 172.21.15.46:0/2653965293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 msgr2=0x56254a4803d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efbe800ddf0 tx=0x7efbe8013000 comp rx=0 tx=0).stop 2024-05-08T23:07:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 -- 172.21.15.46:0/2653965293 shutdown_connections 2024-05-08T23:07:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4869f0 0x56254a4a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4803d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/2653965293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4860d0 0x56254a49bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.029+0000 7efc0f07fb80 1 -- 172.21.15.46:0/2653965293 >> 172.21.15.46:0/2653965293 conn(0x56254a3de290 msgr2=0x56254a47b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.030+0000 7efc0f07fb80 1 -- 172.21.15.46:0/2653965293 shutdown_connections 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.030+0000 7efc0f07fb80 1 -- 172.21.15.46:0/2653965293 wait complete. 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.030+0000 7efc0f07fb80 1 -- start start 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.030+0000 7efc0f07fb80 1 Processor -- start 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a482b20 con 0x56254a4869f0 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a482c90 con 0x56254a4860d0 2024-05-08T23:07:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254a482e50 con 0x56254a480910 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efc0f07fb80 10 client.0 _ll_register_callbacks cb 0x56254a375bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51198/0 (socket says 172.21.15.46:51198) 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 -- 172.21.15.46:0/48240023 learned_addr learned my addr 172.21.15.46:0/48240023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf3e00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf9600700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 -- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 msgr2=0x56254a4820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 -- 172.21.15.46:0/48240023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 msgr2=0x56254a4825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf8c00700 1 -- 172.21.15.46:0/48240023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbe800d9d0 con 0x56254a480910 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.031+0000 7efbf9600700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbf8c00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efbe801eda0 tx=0x7efbe801eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c89777117de076c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbda000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800dee0 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbec004d10 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7efbe8026740 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe801ebc0 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efc0f07fb80 10 client.26782 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56254a3965e0 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.032+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbec004d10 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efbe8026050 con 0x56254a480910 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 10 client.26782 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 20 client.26782 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a4f1990 0x56254a4f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 4 client.26782 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:07:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56254a3c8f40 con 0x56254a4f1990 2024-05-08T23:07:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efbf3e00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a4f1990 0x56254a4f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efbf3e00700 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a4f1990 0x56254a4f3db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb62618cc294223 server_cookie=f536e20bf78d4922 in_seq=0 out_seq=0 2024-05-08T23:07:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.033+0000 7efbda000700 10 client.26782 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efbda000700 1 -- 172.21.15.46:0/48240023 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56254a3c8f40 con 0x56254a4f1990 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efbda000700 10 client.26782 handle_command_reply: tid=0 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 client.26782 shutdown 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 msgr2=0x56254a4915a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efbe801eda0 tx=0x7efbe801eb20 comp rx=0 tx=0).stop 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 shutdown_connections 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56254a4f1990 0x56254a4f3db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efbda000700 0 client.26782 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56254a4869f0 0x56254a4825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56254a480910 0x56254a4915a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.035+0000 7efc0f07fb80 1 --2- 172.21.15.46:0/48240023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56254a4860d0 0x56254a4820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 >> 172.21.15.46:0/48240023 conn(0x56254a3de290 msgr2=0x56254a4a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 shutdown_connections 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efc0f07fb80 1 -- 172.21.15.46:0/48240023 wait complete. 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efc0f07fb80 20 client.26782 trim_cache size 0 max 16384 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6961efdd-505a-42a4-a66c-44c10042704a", 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:07:28.039 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 msgr2=0x7efbdc0a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 0x7efbdc0a39a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005fd0 tx=0x7efbe8005e20 comp rx=0 tx=0).stop 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 msgr2=0x7efbf41a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.036+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efbe00098e0 tx=0x7efbe00057b0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 shutdown_connections 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf4115fa0 0x7efbf41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efbdc0a14e0 0x7efbdc0a39a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbf41084e0 0x7efbf41a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 --2- 172.21.15.46:0/770741679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efbf410b390 0x7efbf41a7b40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 >> 172.21.15.46:0/770741679 conn(0x7efbf406bd20 msgr2=0x7efbf41174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 shutdown_connections 2024-05-08T23:07:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.037+0000 7efbfa000700 1 -- 172.21.15.46:0/770741679 wait complete. 2024-05-08T23:07:28.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 -- 172.21.15.46:0/1007917420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0073370 msgr2=0x7fb2b0073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/1007917420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0073370 0x7fb2b0073750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009990 tx=0x7fb2a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 -- 172.21.15.46:0/1007917420 shutdown_connections 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/1007917420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b010d500 0x7fb2b010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/1007917420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0073370 0x7fb2b0073750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/1007917420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 0x7fb2b010cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 -- 172.21.15.46:0/1007917420 >> 172.21.15.46:0/1007917420 conn(0x7fb2b00fc750 msgr2=0x7fb2b00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 -- 172.21.15.46:0/1007917420 shutdown_connections 2024-05-08T23:07:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.288+0000 7fb2b4c00700 1 -- 172.21.15.46:0/1007917420 wait complete. 2024-05-08T23:07:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.289+0000 7fb2b4c00700 1 Processor -- start 2024-05-08T23:07:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.289+0000 7fb2b4c00700 1 -- start start 2024-05-08T23:07:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 0x7fb2b01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 0x7fb2b019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019e1c0 con 0x7fb2b0073370 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019e330 con 0x7fb2b0073c90 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2b4c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019e4f0 con 0x7fb2b010d500 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2af400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2af400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57784/0 (socket says 172.21.15.46:57784) 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2af400700 1 -- 172.21.15.46:0/658444515 learned_addr learned my addr 172.21.15.46:0/658444515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2afe00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 0x7fb2b019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.290+0000 7fb2aea00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 0x7fb2b01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 -- 172.21.15.46:0/658444515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 msgr2=0x7fb2b01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 0x7fb2b01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 -- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 msgr2=0x7fb2b01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 -- 172.21.15.46:0/658444515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00095f0 con 0x7fb2b010d500 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2af400700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2afe00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 0x7fb2b019db80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007ee0 tx=0x7fb2a800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2a7e00700 1 -- 172.21.15.46:0/658444515 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8009ea0 con 0x7fb2b010d500 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2a7e00700 1 -- 172.21.15.46:0/658444515 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2a800f040 con 0x7fb2b010d500 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b019e7d0 con 0x7fb2b010d500 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2a7e00700 1 -- 172.21.15.46:0/658444515 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8004280 con 0x7fb2b010d500 2024-05-08T23:07:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.291+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b01a6010 con 0x7fb2b010d500 2024-05-08T23:07:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.295+0000 7fb2a7e00700 1 -- 172.21.15.46:0/658444515 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb2a800db20 con 0x7fb2b010d500 2024-05-08T23:07:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.296+0000 7fb2c9cdcb80 1 Processor -- start 2024-05-08T23:07:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.296+0000 7fb2a7e00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 0x7fb2940a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.296+0000 7fb2a7e00700 1 -- 172.21.15.46:0/658444515 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb2a80cc8a0 con 0x7fb2b010d500 2024-05-08T23:07:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.296+0000 7fb2c9cdcb80 1 -- start start 2024-05-08T23:07:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.296+0000 7fb2af400700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 0x7fb2940a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4113430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4064150 con 0x564dc410d890 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4061b10 con 0x564dc4113970 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc405d400 con 0x564dc4113050 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2afe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2afe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57794/0 (socket says 172.21.15.46:57794) 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57886/0 (socket says 172.21.15.46:57886) 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.297+0000 7fb2afe00700 1 -- 172.21.15.46:0/2387242167 learned_addr learned my addr 172.21.15.46:0/2387242167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2af400700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 0x7fb2940a3870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0006010 tx=0x7fb2a000b560 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2af400700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 -- 172.21.15.46:0/2387242167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 msgr2=0x564dc4113430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4113430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 -- 172.21.15.46:0/2387242167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 msgr2=0x564dc410d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 -- 172.21.15.46:0/2387242167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc3ede3b0 con 0x564dc410d890 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2aea00700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2af400700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb2afe00700 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8003fd0 tx=0x7fb2a8095960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a77940637512eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb29be00700 1 -- 172.21.15.46:0/2387242167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8098f00 con 0x564dc410d890 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb29be00700 1 -- 172.21.15.46:0/2387242167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2a8097df0 con 0x564dc410d890 2024-05-08T23:07:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.298+0000 7fb29be00700 1 -- 172.21.15.46:0/2387242167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a809d3d0 con 0x564dc410d890 2024-05-08T23:07:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/2387242167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 msgr2=0x564dc410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8003fd0 tx=0x7fb2a8095960 comp rx=0 tx=0).stop 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/2387242167 shutdown_connections 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410dcd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/2387242167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/2387242167 >> 172.21.15.46:0/2387242167 conn(0x564dc406b210 msgr2=0x564dc41082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/2387242167 shutdown_connections 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.299+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/2387242167 wait complete. 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.300+0000 7fb2c9cdcb80 1 -- start start 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.300+0000 7fb2c9cdcb80 1 Processor -- start 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113050 0x564dc4116bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4117300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3ff9560 con 0x564dc4113050 2024-05-08T23:07:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3ff9200 con 0x564dc4113970 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3ee06a0 con 0x564dc410d890 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2c9cdcb80 10 client.0 _ll_register_callbacks cb 0x564dc4002b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51252/0 (socket says 172.21.15.46:51252) 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 -- 172.21.15.46:0/376088410 learned_addr learned my addr 172.21.15.46:0/376088410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2aea00700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113050 0x564dc4116bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2afe00700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4117300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 -- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 msgr2=0x564dc4117300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4117300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 -- 172.21.15.46:0/376088410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113050 msgr2=0x564dc4116bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113050 0x564dc4116bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.301+0000 7fb2af400700 1 -- 172.21.15.46:0/376088410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0078a0 con 0x564dc410d890 2024-05-08T23:07:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb2af400700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00057c0 tx=0x7fb2a0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=895da3531fa70770 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb298c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000f870 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb2a000f9d0 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dc3fb6270 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a000fb90 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb2c9cdcb80 10 client.26791 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.302+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fb2b01a6010 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564dc3fb6270 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb2a002dcb0 con 0x564dc410d890 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 10 client.26791 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 20 client.26791 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc4127110 0x564dc4129530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 4 client.26791 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dc40dc810 con 0x564dc4127110 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.303+0000 7fb2aea00700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc4127110 0x564dc4129530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.304+0000 7fb2aea00700 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc4127110 0x564dc4129530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc380698b395538c server_cookie=2f77360c391e7774 in_seq=0 out_seq=0 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.304+0000 7fb298c00700 10 client.26791 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.304+0000 7fb298c00700 1 -- 172.21.15.46:0/376088410 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564dc40dc810 con 0x564dc4127110 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb298c00700 10 client.26791 handle_command_reply: tid=0 2024-05-08T23:07:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 client.26791 shutdown 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 msgr2=0x564dc41198d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00057c0 tx=0x7fb2a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 shutdown_connections 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4117300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc41198d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc4127110 0x564dc4129530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/376088410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113050 0x564dc4116bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb298c00700 0 client.26791 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 >> 172.21.15.46:0/376088410 conn(0x564dc406b210 msgr2=0x564dc41080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 shutdown_connections 2024-05-08T23:07:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/376088410 wait complete. 2024-05-08T23:07:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.305+0000 7fb2c9cdcb80 20 client.26791 trim_cache size 0 max 16384 2024-05-08T23:07:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 Processor -- start 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 -- start start 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4122d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.325+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc41134a0 con 0x564dc410d890 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc412e720 con 0x564dc4113970 2024-05-08T23:07:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4100040 con 0x564dc4113050 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51260/0 (socket says 172.21.15.46:51260) 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 -- 172.21.15.46:0/835368064 learned_addr learned my addr 172.21.15.46:0/835368064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2af400700 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2afe00700 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 -- 172.21.15.46:0/835368064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 msgr2=0x564dc4122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4122d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 -- 172.21.15.46:0/835368064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 msgr2=0x564dc410d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.326+0000 7fb2aea00700 1 -- 172.21.15.46:0/835368064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc3e8ab60 con 0x564dc4113050 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.327+0000 7fb2aea00700 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x564dc4118100 tx=0x7fb29c0073d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf7b2d3e01f3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.327+0000 7fb2a5200700 1 -- 172.21.15.46:0/835368064 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c016070 con 0x564dc4113050 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.327+0000 7fb2a5200700 1 -- 172.21.15.46:0/835368064 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29c017040 con 0x564dc4113050 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.327+0000 7fb2a5200700 1 -- 172.21.15.46:0/835368064 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c01f350 con 0x564dc4113050 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/835368064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 msgr2=0x564dc4108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x564dc4118100 tx=0x7fb29c0073d0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/835368064 shutdown_connections 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113970 0x564dc4122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc410d890 0x564dc410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/835368064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4113050 0x564dc4108dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/835368064 >> 172.21.15.46:0/835368064 conn(0x564dc406b210 msgr2=0x564dc4106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/835368064 shutdown_connections 2024-05-08T23:07:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.328+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/835368064 wait complete. 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.329+0000 7fb2c9cdcb80 1 -- start start 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.329+0000 7fb2c9cdcb80 1 Processor -- start 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc411e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 0x564dc410ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc41107a0 con 0x564dc4113970 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc410ef40 con 0x564dc4113050 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc410f0b0 con 0x564dc410d890 2024-05-08T23:07:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2c9cdcb80 10 client.0 _ll_register_callbacks cb 0x564dc4002b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57894/0 (socket says 172.21.15.46:57894) 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 -- 172.21.15.46:0/847302163 learned_addr learned my addr 172.21.15.46:0/847302163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2afe00700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 0x564dc410ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2af400700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc411e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 -- 172.21.15.46:0/847302163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 msgr2=0x564dc411e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc411e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 -- 172.21.15.46:0/847302163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 msgr2=0x564dc410ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.330+0000 7fb2aea00700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 0x564dc410ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb2aea00700 1 -- 172.21.15.46:0/847302163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0114a0 con 0x564dc4113050 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb2afe00700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 0x564dc410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb2af400700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc411e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb2aea00700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb29c005fa0 tx=0x7fb29c003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e02e40d878b319d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb298c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c016040 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb29c01f2f0 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.331+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2a8097df0 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb29c01f4b0 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb2c9cdcb80 10 client.26747 fetch_fsmap learned FSMap version 28 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564dc4023560 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2a8097df0 con 0x564dc4113050 2024-05-08T23:07:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb29c029400 con 0x564dc4113050 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb2c9cdcb80 10 client.26747 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.332+0000 7fb2c9cdcb80 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.333+0000 7fb2c9cdcb80 20 client.26747 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.333+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc417e300 0x564dc4180720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.333+0000 7fb2c9cdcb80 4 client.26747 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.333+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564dc4055ec0 con 0x564dc417e300 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.333+0000 7fb2af400700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc417e300 0x564dc4180720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2af400700 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc417e300 0x564dc4180720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5316fb82c787be2 server_cookie=b7f381fac7e9cd6f in_seq=0 out_seq=0 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb298c00700 10 client.26747 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb298c00700 1 -- 172.21.15.46:0/847302163 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564dc4055ec0 con 0x564dc417e300 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb298c00700 10 client.26747 handle_command_reply: tid=0 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 client.26747 shutdown 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 msgr2=0x564dc410fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb29c005fa0 tx=0x7fb29c003800 comp rx=0 tx=0).stop 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 shutdown_connections 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc417e300 0x564dc4180720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4113970 0x564dc410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc410d890 0x564dc411e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 --2- 172.21.15.46:0/847302163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4113050 0x564dc410fec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 >> 172.21.15.46:0/847302163 conn(0x564dc406b210 msgr2=0x564dc4110c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.334+0000 7fb298c00700 0 client.26747 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 shutdown_connections 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2c9cdcb80 1 -- 172.21.15.46:0/847302163 wait complete. 2024-05-08T23:07:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2c9cdcb80 20 client.26747 trim_cache size 0 max 16384 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "6961efdd-505a-42a4-a66c-44c10042704a": { 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6961efdd-505a-42a4-a66c-44c10042704a", 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:07:28.338 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 msgr2=0x7fb2940a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 0x7fb2940a3870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0006010 tx=0x7fb2a000b560 comp rx=0 tx=0).stop 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 msgr2=0x7fb2b019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.335+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 0x7fb2b019db80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007ee0 tx=0x7fb2a800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 shutdown_connections 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010d500 0x7fb2b019db80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2940a13b0 0x7fb2940a3870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2b0073370 0x7fb2b01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 --2- 172.21.15.46:0/658444515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2b0073c90 0x7fb2b01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 >> 172.21.15.46:0/658444515 conn(0x7fb2b00fc750 msgr2=0x7fb2b00fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 shutdown_connections 2024-05-08T23:07:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:28.336+0000 7fb2b4c00700 1 -- 172.21.15.46:0/658444515 wait complete. 2024-05-08T23:07:28.359 INFO:tasks.cephfs.filesystem:scrub status for tag:6961efdd-505a-42a4-a66c-44c10042704a - {'path': '/', 'tag': '6961efdd-505a-42a4-a66c-44c10042704a', 'options': 'recursive,force'} 2024-05-08T23:07:28.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 192025 9.58 MB/sec warmup 110 sec latency 11.622 ms 2024-05-08T23:07:28.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:07:28.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:28.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:07:28.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:28.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:07:28.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:28 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:07:29.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:07:29.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:07:29.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:28 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:07:29.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:28 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:29.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 194476 9.60 MB/sec warmup 111 sec latency 12.004 ms 2024-05-08T23:07:30.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:29 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:07:30.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:29 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:30.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:29 smithi053 ceph-mon[77587]: pgmap v113: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:29 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:07:30.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:29 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:29 smithi046 ceph-mon[36898]: pgmap v113: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:29 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:07:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:29 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:07:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:29 smithi086 ceph-mon[78610]: pgmap v113: 129 pgs: 129 active+clean; 112 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-08T23:07:30.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 196086 9.59 MB/sec warmup 112 sec latency 9.964 ms 2024-05-08T23:07:31.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 198777 9.65 MB/sec warmup 113 sec latency 18.037 ms 2024-05-08T23:07:32.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:31 smithi053 ceph-mon[77587]: pgmap v114: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-05-08T23:07:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:31 smithi046 ceph-mon[36898]: pgmap v114: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-05-08T23:07:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:31 smithi086 ceph-mon[78610]: pgmap v114: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-05-08T23:07:32.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 201017 9.64 MB/sec warmup 114 sec latency 9.321 ms 2024-05-08T23:07:33.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 202814 9.66 MB/sec warmup 115 sec latency 10.338 ms 2024-05-08T23:07:34.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:33 smithi053 ceph-mon[77587]: pgmap v115: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-08T23:07:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:33 smithi046 ceph-mon[36898]: pgmap v115: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-08T23:07:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:33 smithi086 ceph-mon[78610]: pgmap v115: 129 pgs: 129 active+clean; 119 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-08T23:07:34.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 205354 9.69 MB/sec warmup 116 sec latency 47.598 ms 2024-05-08T23:07:35.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 206908 9.68 MB/sec warmup 117 sec latency 37.281 ms 2024-05-08T23:07:36.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:35 smithi053 ceph-mon[77587]: pgmap v116: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-05-08T23:07:36.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:35 smithi046 ceph-mon[36898]: pgmap v116: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-05-08T23:07:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:35 smithi086 ceph-mon[78610]: pgmap v116: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-05-08T23:07:36.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 208324 9.64 MB/sec warmup 118 sec latency 259.751 ms 2024-05-08T23:07:37.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 209685 9.66 MB/sec warmup 119 sec latency 288.729 ms 2024-05-08T23:07:38.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:37 smithi053 ceph-mon[77587]: pgmap v117: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:38.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:37 smithi046 ceph-mon[36898]: pgmap v117: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:37 smithi086 ceph-mon[78610]: pgmap v117: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:39.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 213502 12.18 MB/sec execute 1 sec latency 37.125 ms 2024-05-08T23:07:40.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:39 smithi053 ceph-mon[77587]: pgmap v118: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:40.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:39 smithi046 ceph-mon[36898]: pgmap v118: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:39 smithi086 ceph-mon[78610]: pgmap v118: 129 pgs: 129 active+clean; 125 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-08T23:07:40.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 216372 14.71 MB/sec execute 2 sec latency 9.927 ms 2024-05-08T23:07:41.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:41.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:41.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:41.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:41.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:41.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:41.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 218117 12.14 MB/sec execute 3 sec latency 9.967 ms 2024-05-08T23:07:42.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:41 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:07:42.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:41 smithi053 ceph-mon[77587]: pgmap v119: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-05-08T23:07:42.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:41 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:07:42.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:41 smithi046 ceph-mon[36898]: pgmap v119: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-05-08T23:07:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:41 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:07:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:41 smithi086 ceph-mon[78610]: pgmap v119: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-05-08T23:07:42.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 219778 11.41 MB/sec execute 4 sec latency 226.114 ms 2024-05-08T23:07:43.878 INFO:tasks.workunit.client.0.smithi046.stdout: 1 220839 10.18 MB/sec execute 5 sec latency 38.386 ms 2024-05-08T23:07:44.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:43 smithi053 ceph-mon[77587]: pgmap v120: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-08T23:07:44.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:43 smithi046 ceph-mon[36898]: pgmap v120: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-08T23:07:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:43 smithi086 ceph-mon[78610]: pgmap v120: 129 pgs: 129 active+clean; 126 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-08T23:07:44.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 223589 11.32 MB/sec execute 6 sec latency 11.201 ms 2024-05-08T23:07:45.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 225654 10.73 MB/sec execute 7 sec latency 12.336 ms 2024-05-08T23:07:46.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:45 smithi053 ceph-mon[77587]: pgmap v121: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 402 op/s 2024-05-08T23:07:46.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:45 smithi046 ceph-mon[36898]: pgmap v121: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 402 op/s 2024-05-08T23:07:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:45 smithi086 ceph-mon[78610]: pgmap v121: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 402 op/s 2024-05-08T23:07:46.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 227520 11.06 MB/sec execute 8 sec latency 54.610 ms 2024-05-08T23:07:47.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 229934 11.08 MB/sec execute 9 sec latency 10.742 ms 2024-05-08T23:07:48.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:47 smithi053 ceph-mon[77587]: pgmap v122: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:48.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:47 smithi046 ceph-mon[36898]: pgmap v122: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:47 smithi086 ceph-mon[78610]: pgmap v122: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:48.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 231344 10.80 MB/sec execute 10 sec latency 32.119 ms 2024-05-08T23:07:49.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 233936 11.10 MB/sec execute 11 sec latency 13.271 ms 2024-05-08T23:07:50.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:49 smithi053 ceph-mon[77587]: pgmap v123: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:50.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:49 smithi046 ceph-mon[36898]: pgmap v123: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:49 smithi086 ceph-mon[78610]: pgmap v123: 129 pgs: 129 active+clean; 128 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 319 op/s 2024-05-08T23:07:50.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 235431 10.75 MB/sec execute 12 sec latency 8.389 ms 2024-05-08T23:07:51.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 237934 11.20 MB/sec execute 13 sec latency 10.235 ms 2024-05-08T23:07:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:51 smithi053 ceph-mon[77587]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.8 MiB/s wr, 397 op/s 2024-05-08T23:07:52.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:51 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:07:52.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:51 smithi046 ceph-mon[36898]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.8 MiB/s wr, 397 op/s 2024-05-08T23:07:52.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:51 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:07:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:51 smithi086 ceph-mon[78610]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.8 MiB/s wr, 397 op/s 2024-05-08T23:07:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:51 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:07:52.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 240137 10.93 MB/sec execute 14 sec latency 14.609 ms 2024-05-08T23:07:53.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 241840 11.00 MB/sec execute 15 sec latency 26.837 ms 2024-05-08T23:07:54.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:53 smithi053 ceph-mon[77587]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-05-08T23:07:54.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:53 smithi046 ceph-mon[36898]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-05-08T23:07:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:53 smithi086 ceph-mon[78610]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-05-08T23:07:54.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 244445 11.15 MB/sec execute 16 sec latency 12.720 ms 2024-05-08T23:07:55.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 246076 10.96 MB/sec execute 17 sec latency 10.343 ms 2024-05-08T23:07:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:55 smithi046 ceph-mon[36898]: pgmap v126: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-08T23:07:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:55 smithi086 ceph-mon[78610]: pgmap v126: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-08T23:07:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:55 smithi053 ceph-mon[77587]: pgmap v126: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-08T23:07:56.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 248617 11.27 MB/sec execute 18 sec latency 11.781 ms 2024-05-08T23:07:57.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 250819 11.10 MB/sec execute 19 sec latency 11.715 ms 2024-05-08T23:07:58.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:57 smithi046 ceph-mon[36898]: pgmap v127: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:07:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:57 smithi086 ceph-mon[78610]: pgmap v127: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:07:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:57 smithi053 ceph-mon[77587]: pgmap v127: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:07:58.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:07:58.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.588+0000 7f5391200700 1 -- 172.21.15.46:0/2729750157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 msgr2=0x7f538c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.588+0000 7f5391200700 1 --2- 172.21.15.46:0/2729750157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c100170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f537c009990 tx=0x7f537c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 -- 172.21.15.46:0/2729750157 shutdown_connections 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 --2- 172.21.15.46:0/2729750157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c105190 0x7f538c105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 --2- 172.21.15.46:0/2729750157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c100170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 --2- 172.21.15.46:0/2729750157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c1006b0 0x7f538c104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 -- 172.21.15.46:0/2729750157 >> 172.21.15.46:0/2729750157 conn(0x7f538c0fb760 msgr2=0x7f538c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 -- 172.21.15.46:0/2729750157 shutdown_connections 2024-05-08T23:07:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.589+0000 7f5391200700 1 -- 172.21.15.46:0/2729750157 wait complete. 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 Processor -- start 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 -- start start 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c1006b0 0x7f538c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19ddd0 con 0x7f538c1006b0 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19df40 con 0x7f538c105190 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f5391200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19e100 con 0x7f538c0ffd90 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.590+0000 7f538a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5389e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53440/0 (socket says 172.21.15.46:53440) 2024-05-08T23:07:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 -- 172.21.15.46:0/721284608 learned_addr learned my addr 172.21.15.46:0/721284608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5389400700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c1006b0 0x7f538c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 -- 172.21.15.46:0/721284608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 msgr2=0x7f538c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 -- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c1006b0 msgr2=0x7f538c1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c1006b0 0x7f538c1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 -- 172.21.15.46:0/721284608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0095f0 con 0x7f538c105190 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f538a800700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f538400ef30 tx=0x7f538400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5382a00700 1 -- 172.21.15.46:0/721284608 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384018070 con 0x7f538c105190 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5382a00700 1 -- 172.21.15.46:0/721284608 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538400f040 con 0x7f538c105190 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c19e380 con 0x7f538c105190 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.591+0000 7f5382a00700 1 -- 172.21.15.46:0/721284608 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538400cdb0 con 0x7f538c105190 2024-05-08T23:07:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.592+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1aa630 con 0x7f538c105190 2024-05-08T23:07:58.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.595+0000 7f5382a00700 1 -- 172.21.15.46:0/721284608 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5384003a00 con 0x7f538c105190 2024-05-08T23:07:58.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.596+0000 7f5382a00700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 0x7f53700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.596+0000 7f53a4d99b80 1 Processor -- start 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.596+0000 7f5382a00700 1 -- 172.21.15.46:0/721284608 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f53840ccbf0 con 0x7f538c105190 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f5389e00700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 0x7f53700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 -- start start 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 0x5583cc254350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc254890 0x5583cc254cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc1ab150 con 0x5583cc259e00 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc1a8b10 con 0x5583cc25a720 2024-05-08T23:07:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.597+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc1a4400 con 0x5583cc254890 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51898/0 (socket says 172.21.15.46:51898) 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 -- 172.21.15.46:0/1327298724 learned_addr learned my addr 172.21.15.46:0/1327298724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389400700 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 0x5583cc254350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 0x7f53700a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b2e0 tx=0x7f537c005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f538a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc254890 0x5583cc254cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 -- 172.21.15.46:0/1327298724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc254890 msgr2=0x5583cc254cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc254890 0x5583cc254cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 -- 172.21.15.46:0/1327298724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 msgr2=0x5583cc254350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 0x5583cc254350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389e00700 1 -- 172.21.15.46:0/1327298724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583cc0253b0 con 0x5583cc259e00 2024-05-08T23:07:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.598+0000 7f5389400700 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 0x5583cc254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.599+0000 7f5389e00700 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f537c015000 tx=0x7f537c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2aae5c678feb985 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.599+0000 7f5377400700 1 -- 172.21.15.46:0/1327298724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c004500 con 0x5583cc259e00 2024-05-08T23:07:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.599+0000 7f5377400700 1 -- 172.21.15.46:0/1327298724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537c01e040 con 0x5583cc259e00 2024-05-08T23:07:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.599+0000 7f5377400700 1 -- 172.21.15.46:0/1327298724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00fa10 con 0x5583cc259e00 2024-05-08T23:07:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1327298724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 msgr2=0x5583cc25a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f537c015000 tx=0x7f537c020000 comp rx=0 tx=0).stop 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1327298724 shutdown_connections 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc254890 0x5583cc254cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25a1e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1327298724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc25a720 0x5583cc254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1327298724 >> 172.21.15.46:0/1327298724 conn(0x5583cc1b2210 msgr2=0x5583cc24edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1327298724 shutdown_connections 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.600+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1327298724 wait complete. 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.601+0000 7f53a4d99b80 1 -- start start 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.601+0000 7f53a4d99b80 1 Processor -- start 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc140560 con 0x5583cc259e00 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc140200 con 0x5583cc254890 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc0276a0 con 0x5583cc25a720 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f53a4d99b80 10 client.0 _ll_register_callbacks cb 0x5583cc149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f5389e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f5389e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53460/0 (socket says 172.21.15.46:53460) 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f5389e00700 1 -- 172.21.15.46:0/3667556824 learned_addr learned my addr 172.21.15.46:0/3667556824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.602+0000 7f538a800700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 -- 172.21.15.46:0/3667556824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 msgr2=0x5583cc25e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 -- 172.21.15.46:0/3667556824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 msgr2=0x5583cc25e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 -- 172.21.15.46:0/3667556824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5378009820 con 0x5583cc254890 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f5389e00700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f537c0098e0 tx=0x7f537c004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19cc50eb02ab8c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f536be00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c034070 con 0x5583cc254890 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f537c004e90 con 0x5583cc254890 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.603+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583cc0fd270 con 0x5583cc254890 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f537c00f610 con 0x5583cc254890 2024-05-08T23:07:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 10 client.26759 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f538c114b40 con 0x5583cc254890 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583cc0fd270 con 0x5583cc254890 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f537c0309a0 con 0x5583cc254890 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 20 client.26759 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc26dd80 0x5583cc270170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 4 client.26759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.604+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583cc223810 con 0x5583cc26dd80 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.605+0000 7f5389400700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc26dd80 0x5583cc270170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.605+0000 7f5389400700 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc26dd80 0x5583cc270170 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eca876629d508c7 server_cookie=21081d3133492858 in_seq=0 out_seq=0 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.605+0000 7f536be00700 10 client.26759 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f536be00700 1 -- 172.21.15.46:0/3667556824 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583cc223810 con 0x5583cc26dd80 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f536be00700 10 client.26759 handle_command_reply: tid=0 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 client.26759 shutdown 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 msgr2=0x5583cc25db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f537c0098e0 tx=0x7f537c004c00 comp rx=0 tx=0).stop 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 shutdown_connections 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc254890 0x5583cc25db60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc26dd80 0x5583cc270170 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/3667556824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc259e00 0x5583cc25e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 >> 172.21.15.46:0/3667556824 conn(0x5583cc1b2210 msgr2=0x5583cc24e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.606+0000 7f536be00700 0 client.26759 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.607+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 shutdown_connections 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.607+0000 7f53a4d99b80 1 -- 172.21.15.46:0/3667556824 wait complete. 2024-05-08T23:07:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.607+0000 7f53a4d99b80 20 client.26759 trim_cache size 0 max 16384 2024-05-08T23:07:58.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.626+0000 7f53a4d99b80 1 Processor -- start 2024-05-08T23:07:58.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 -- start start 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc259350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc24ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc24fa90 con 0x5583cc254890 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc25c0b0 con 0x5583cc259e00 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc255230 con 0x5583cc25a720 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc259350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f5389400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc24ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc259350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51918/0 (socket says 172.21.15.46:51918) 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f5389e00700 1 -- 172.21.15.46:0/190349217 learned_addr learned my addr 172.21.15.46:0/190349217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.627+0000 7f538a800700 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 -- 172.21.15.46:0/190349217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 msgr2=0x5583cc25e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 -- 172.21.15.46:0/190349217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 msgr2=0x5583cc259350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc259350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 -- 172.21.15.46:0/190349217 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583cbfd1b60 con 0x5583cc259e00 2024-05-08T23:07:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.628+0000 7f5389400700 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc24ed20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5583cc250910 tx=0x7f5378014750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0304d88c5a69895 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.629+0000 7f5377e00700 1 -- 172.21.15.46:0/190349217 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378017040 con 0x5583cc259e00 2024-05-08T23:07:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.629+0000 7f5377e00700 1 -- 172.21.15.46:0/190349217 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5378021070 con 0x5583cc259e00 2024-05-08T23:07:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.629+0000 7f5377e00700 1 -- 172.21.15.46:0/190349217 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378004c10 con 0x5583cc259e00 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- 172.21.15.46:0/190349217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 msgr2=0x5583cc24ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc24ed20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5583cc250910 tx=0x7f5378014750 comp rx=0 tx=0).stop 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- 172.21.15.46:0/190349217 shutdown_connections 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc25e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc259350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/190349217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc24ed20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- 172.21.15.46:0/190349217 >> 172.21.15.46:0/190349217 conn(0x5583cc1b2210 msgr2=0x5583cc258c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- 172.21.15.46:0/190349217 shutdown_connections 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- 172.21.15.46:0/190349217 wait complete. 2024-05-08T23:07:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.630+0000 7f53a4d99b80 1 -- start start 2024-05-08T23:07:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 Processor -- start 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc256cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc2573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc255b40 con 0x5583cc254890 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc255cb0 con 0x5583cc259e00 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583cc255e70 con 0x5583cc25a720 2024-05-08T23:07:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.631+0000 7f53a4d99b80 10 client.0 _ll_register_callbacks cb 0x5583cc149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51928/0 (socket says 172.21.15.46:51928) 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 -- 172.21.15.46:0/1996400984 learned_addr learned my addr 172.21.15.46:0/1996400984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389400700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc256cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 -- 172.21.15.46:0/1996400984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 msgr2=0x5583cc2573f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc2573f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 -- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 msgr2=0x5583cc256cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc256cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 -- 172.21.15.46:0/1996400984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5378009ad0 con 0x5583cc254890 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f5389e00700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f537c0095c0 tx=0x7f537c030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38f40ccb5956e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.632+0000 7f536be00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c034070 con 0x5583cc254890 2024-05-08T23:07:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f537c039540 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f537c01e040 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f537c030bb0 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f53a4d99b80 10 client.16767 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5583cc16a560 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.633+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537c01e040 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f537c00f770 con 0x5583cc254890 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 20 client.16767 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc2c4940 0x5583cc2c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 4 client.16767 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5583cc19cec0 con 0x5583cc2c4940 2024-05-08T23:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.634+0000 7f5389400700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc2c4940 0x5583cc2c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f5389400700 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc2c4940 0x5583cc2c6d60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db2c569f0b3a4f0 server_cookie=4328b55abe930725 in_seq=0 out_seq=0 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f536be00700 10 client.16767 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f536be00700 1 -- 172.21.15.46:0/1996400984 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5583cc19cec0 con 0x5583cc2c4940 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f536be00700 10 client.16767 handle_command_reply: tid=0 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 client.16767 shutdown 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 msgr2=0x5583cc2655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f537c0095c0 tx=0x7f537c030700 comp rx=0 tx=0).stop 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 shutdown_connections 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583cc25a720 0x5583cc2573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583cc2c4940 0x5583cc2c6d60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cc254890 0x5583cc2655f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 --2- 172.21.15.46:0/1996400984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583cc259e00 0x5583cc256cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f536be00700 0 client.16767 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 >> 172.21.15.46:0/1996400984 conn(0x5583cc1b2210 msgr2=0x5583cc25c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.635+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 shutdown_connections 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f53a4d99b80 1 -- 172.21.15.46:0/1996400984 wait complete. 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f53a4d99b80 20 client.16767 trim_cache size 0 max 16384 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:07:58.638 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:07:58.639 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:07:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 msgr2=0x7f53700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 0x7f53700a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b2e0 tx=0x7f537c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 msgr2=0x7f538c19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.636+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f538400ef30 tx=0x7f538400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 shutdown_connections 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c105190 0x7f538c19d770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538c0ffd90 0x7f538c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53700a1290 0x7f53700a3750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 --2- 172.21.15.46:0/721284608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f538c1006b0 0x7f538c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 >> 172.21.15.46:0/721284608 conn(0x7f538c0fb760 msgr2=0x7f538c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 shutdown_connections 2024-05-08T23:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.637+0000 7f5391200700 1 -- 172.21.15.46:0/721284608 wait complete. 2024-05-08T23:07:58.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:07:58.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 -- 172.21.15.46:0/3371047806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0683d0 msgr2=0x7f8ebc0687b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/3371047806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc0687b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009990 tx=0x7f8eac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 -- 172.21.15.46:0/3371047806 shutdown_connections 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/3371047806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc105f90 0x7f8ebc10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/3371047806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc0687b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/3371047806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc068d80 0x7f8ebc105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 -- 172.21.15.46:0/3371047806 >> 172.21.15.46:0/3371047806 conn(0x7f8ebc075690 msgr2=0x7f8ebc075aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 -- 172.21.15.46:0/3371047806 shutdown_connections 2024-05-08T23:07:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.867+0000 7f8ec0c00700 1 -- 172.21.15.46:0/3371047806 wait complete. 2024-05-08T23:07:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 Processor -- start 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 -- start start 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 0x7f8ebc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e710 con 0x7f8ebc105f90 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e880 con 0x7f8ebc0683d0 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.869+0000 7f8ec0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19ea40 con 0x7f8ebc19dc90 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53502/0 (socket says 172.21.15.46:53502) 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52300/0 (socket says 172.21.15.46:52300) 2024-05-08T23:07:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebb400700 1 -- 172.21.15.46:0/640492607 learned_addr learned my addr 172.21.15.46:0/640492607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebaa00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 0x7f8ebc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 -- 172.21.15.46:0/640492607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 msgr2=0x7f8ebc199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 -- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 msgr2=0x7f8ebc19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 0x7f8ebc19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 -- 172.21.15.46:0/640492607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac0095f0 con 0x7f8ebc19dc90 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebaa00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 0x7f8ebc19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.870+0000 7f8ebbe00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb00 tx=0x7f8eb400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.871+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400ca30 con 0x7f8ebc19dc90 2024-05-08T23:07:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.871+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb400cb90 con 0x7f8ebc19dc90 2024-05-08T23:07:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.871+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc1a5fa0 con 0x7f8ebc19dc90 2024-05-08T23:07:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.871+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40125a0 con 0x7f8ebc19dc90 2024-05-08T23:07:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.871+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc1a6470 con 0x7f8ebc19dc90 2024-05-08T23:07:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.872+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc102340 con 0x7f8ebc19dc90 2024-05-08T23:07:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.874+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8eb400cd00 con 0x7f8ebc19dc90 2024-05-08T23:07:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.876+0000 7f8eb3e00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.876+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8eb40cc660 con 0x7f8ebc19dc90 2024-05-08T23:07:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.876+0000 7f8ebb400700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.877+0000 7f8ebb400700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 0x7f8ea00a37d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006010 tx=0x7f8eac005ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:58.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:58.880+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8eb40957b0 con 0x7f8ebc19dc90 2024-05-08T23:07:58.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 252479 11.12 MB/sec execute 20 sec latency 10.485 ms 2024-05-08T23:07:59.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:58 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:07:59.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:58 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:07:59.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.240+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ebc04ecd0 con 0x7f8ebc19dc90 2024-05-08T23:07:59.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.241+0000 7f8eb3e00700 1 -- 172.21.15.46:0/640492607 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10905 (secure 0 0 0) 0x7f8eb4098dd0 con 0x7f8ebc19dc90 2024-05-08T23:07:59.240 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:07:59.240 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:07:50.751461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 msgr2=0x7f8ea00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 0x7f8ea00a37d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006010 tx=0x7f8eac005ab0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 msgr2=0x7f8ebc19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb00 tx=0x7f8eb400be10 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 shutdown_connections 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc19dc90 0x7f8ebc19e110 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ebc0683d0 0x7f8ebc199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 --2- 172.21.15.46:0/640492607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ebc105f90 0x7f8ebc19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 >> 172.21.15.46:0/640492607 conn(0x7f8ebc075690 msgr2=0x7f8ebc104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.243+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 shutdown_connections 2024-05-08T23:07:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.244+0000 7f8ec0c00700 1 -- 172.21.15.46:0/640492607 wait complete. 2024-05-08T23:07:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-05-08T23:07:59.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:58 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:07:59.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:07:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 -- 172.21.15.46:0/592386729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581083d0 msgr2=0x7f035810ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 --2- 172.21.15.46:0/592386729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581083d0 0x7f035810ff50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f035000b0a0 tx=0x7f035000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 -- 172.21.15.46:0/592386729 shutdown_connections 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 --2- 172.21.15.46:0/592386729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581083d0 0x7f035810ff50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 --2- 172.21.15.46:0/592386729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581032b0 0x7f0358103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 --2- 172.21.15.46:0/592386729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0358103c60 0x7f0358107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 -- 172.21.15.46:0/592386729 >> 172.21.15.46:0/592386729 conn(0x7f03580feb10 msgr2=0x7f0358100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 -- 172.21.15.46:0/592386729 shutdown_connections 2024-05-08T23:07:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.478+0000 7f035d600700 1 -- 172.21.15.46:0/592386729 wait complete. 2024-05-08T23:07:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.479+0000 7f035d600700 1 Processor -- start 2024-05-08T23:07:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 -- start start 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581083d0 0x7f0358199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035819de60 0x7f035819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819e8e0 con 0x7f03581083d0 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819ea50 con 0x7f035819de60 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f035d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819ec10 con 0x7f03581032b0 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f0356a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f0356a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52320/0 (socket says 172.21.15.46:52320) 2024-05-08T23:07:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.480+0000 7f0356a00700 1 -- 172.21.15.46:0/3546853420 learned_addr learned my addr 172.21.15.46:0/3546853420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356000700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581083d0 0x7f0358199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0357400700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035819de60 0x7f035819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 -- 172.21.15.46:0/3546853420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035819de60 msgr2=0x7f035819e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035819de60 0x7f035819e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 -- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581083d0 msgr2=0x7f0358199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581083d0 0x7f0358199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 -- 172.21.15.46:0/3546853420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350009d00 con 0x7f03581032b0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.481+0000 7f0356a00700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f034800d950 tx=0x7f034800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.482+0000 7f034f400700 1 -- 172.21.15.46:0/3546853420 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800f890 con 0x7f03581032b0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.482+0000 7f034f400700 1 -- 172.21.15.46:0/3546853420 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0348010460 con 0x7f03581032b0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.482+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03581a6170 con 0x7f03581032b0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.482+0000 7f034f400700 1 -- 172.21.15.46:0/3546853420 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800e610 con 0x7f03581032b0 2024-05-08T23:07:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.482+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03581a6650 con 0x7f03581032b0 2024-05-08T23:07:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.485+0000 7f034f400700 1 -- 172.21.15.46:0/3546853420 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f03480108d0 con 0x7f03581032b0 2024-05-08T23:07:59.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.487+0000 7f034f400700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 0x7f033c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.487+0000 7f034f400700 1 -- 172.21.15.46:0/3546853420 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f03480cceb0 con 0x7f03581032b0 2024-05-08T23:07:59.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.487+0000 7f0356000700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 0x7f033c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.488+0000 7f0356000700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 0x7f033c0a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f035819aad0 tx=0x7f0344009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.488+0000 7f0371137b80 1 Processor -- start 2024-05-08T23:07:59.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.489+0000 7f0371137b80 1 -- start start 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.489+0000 7f0371137b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.489+0000 7f0371137b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 0x55d56fa15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.489+0000 7f0371137b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0371137b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f96c150 con 0x55d56fa1b720 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0371137b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f969b10 con 0x55d56fa1ae00 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0371137b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f965400 con 0x55d56fa15890 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 0x55d56fa15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53536/0 (socket says 172.21.15.46:53536) 2024-05-08T23:07:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356a00700 1 -- 172.21.15.46:0/1121076984 learned_addr learned my addr 172.21.15.46:0/1121076984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0357400700 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 -- 172.21.15.46:0/1121076984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 msgr2=0x55d56fa15cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa15cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 -- 172.21.15.46:0/1121076984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 msgr2=0x55d56fa1b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356000700 1 -- 172.21.15.46:0/1121076984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56f7e63b0 con 0x55d56fa1b720 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.490+0000 7f0356a00700 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.491+0000 7f0356000700 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 0x55d56fa15350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0344011c30 tx=0x7f0344014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c78ec733bf4e9c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.491+0000 7f0343400700 1 -- 172.21.15.46:0/1121076984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344004050 con 0x55d56fa1b720 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.491+0000 7f0343400700 1 -- 172.21.15.46:0/1121076984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f034400e840 con 0x55d56fa1b720 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.492+0000 7f0371137b80 1 -- 172.21.15.46:0/1121076984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 msgr2=0x55d56fa15350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.492+0000 7f0371137b80 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 0x55d56fa15350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0344011c30 tx=0x7f0344014000 comp rx=0 tx=0).stop 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 -- 172.21.15.46:0/1121076984 shutdown_connections 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 --2- 172.21.15.46:0/1121076984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1b720 0x55d56fa15350 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 -- 172.21.15.46:0/1121076984 >> 172.21.15.46:0/1121076984 conn(0x55d56f973210 msgr2=0x55d56fa0fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 -- 172.21.15.46:0/1121076984 shutdown_connections 2024-05-08T23:07:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 -- 172.21.15.46:0/1121076984 wait complete. 2024-05-08T23:07:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.493+0000 7f0371137b80 1 -- start start 2024-05-08T23:07:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.494+0000 7f0371137b80 1 Processor -- start 2024-05-08T23:07:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f901560 con 0x55d56fa15890 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f901200 con 0x55d56fa1ae00 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56f7e86a0 con 0x55d56fa1b720 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.495+0000 7f0371137b80 10 client.0 _ll_register_callbacks cb 0x55d56f90ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.496+0000 7f0356000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.496+0000 7f0356000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53538/0 (socket says 172.21.15.46:53538) 2024-05-08T23:07:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.496+0000 7f0356000700 1 -- 172.21.15.46:0/2067330912 learned_addr learned my addr 172.21.15.46:0/2067330912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.496+0000 7f0357400700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.497+0000 7f0356a00700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.497+0000 7f0356000700 1 -- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 msgr2=0x55d56fa1f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0356000700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa1f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0356000700 1 -- 172.21.15.46:0/2067330912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 msgr2=0x55d56fa216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0356000700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0356000700 1 -- 172.21.15.46:0/2067330912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0344017070 con 0x55d56fa1ae00 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0356000700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0344009340 tx=0x7f0344003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433a5fbf4ddfb5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0337e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034402c070 con 0x55d56fa1ae00 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0344004260 con 0x55d56fa1ae00 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.498+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d56f8be270 con 0x55d56fa1ae00 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.499+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0344004420 con 0x55d56fa1ae00 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.499+0000 7f0371137b80 10 client.26780 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.499+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d56f792b60 con 0x55d56fa1ae00 2024-05-08T23:07:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.499+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d56f8be270 con 0x55d56fa1ae00 2024-05-08T23:07:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f034401e4b0 con 0x55d56fa1ae00 2024-05-08T23:07:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 10 client.26780 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 10 client.26780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 20 client.26780 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa2f010 0x55d56fa31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 4 client.26780 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d56f9e4810 con 0x55d56fa2f010 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.500+0000 7f0356a00700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa2f010 0x55d56fa31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.501+0000 7f0337e00700 10 client.26780 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.501+0000 7f0356a00700 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa2f010 0x55d56fa31430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=730182a7844d3921 server_cookie=a1b5a2bf30cb7b46 in_seq=0 out_seq=0 2024-05-08T23:07:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.502+0000 7f0337e00700 1 -- 172.21.15.46:0/2067330912 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d56f9e4810 con 0x55d56fa2f010 2024-05-08T23:07:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.502+0000 7f0337e00700 10 client.26780 handle_command_reply: tid=0 2024-05-08T23:07:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.502+0000 7f0371137b80 1 client.26780 shutdown 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.502+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 msgr2=0x55d56fa1e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0344009340 tx=0x7f0344003f30 comp rx=0 tx=0).stop 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 shutdown_connections 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa2f010 0x55d56fa31430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa1f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 --2- 172.21.15.46:0/2067330912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa1e950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 >> 172.21.15.46:0/2067330912 conn(0x55d56f973210 msgr2=0x55d56fa0f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0337e00700 0 client.26780 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 shutdown_connections 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 1 -- 172.21.15.46:0/2067330912 wait complete. 2024-05-08T23:07:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.503+0000 7f0371137b80 20 client.26780 trim_cache size 0 max 16384 2024-05-08T23:07:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 Processor -- start 2024-05-08T23:07:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 -- start start 2024-05-08T23:07:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa365e0 con 0x55d56fa15890 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa0ab40 con 0x55d56fa1ae00 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0371137b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa2b630 con 0x55d56fa1b720 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0357400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0356a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0356000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0357400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52334/0 (socket says 172.21.15.46:52334) 2024-05-08T23:07:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.522+0000 7f0357400700 1 -- 172.21.15.46:0/439629860 learned_addr learned my addr 172.21.15.46:0/439629860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 -- 172.21.15.46:0/439629860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 msgr2=0x55d56fa0fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa0fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 -- 172.21.15.46:0/439629860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 msgr2=0x55d56fa15350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa15350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 -- 172.21.15.46:0/439629860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56f92b560 con 0x55d56fa1b720 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0357400700 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0350007cb0 tx=0x7f0350007ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae85cdf3eb1424bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0343e00700 1 -- 172.21.15.46:0/439629860 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035000e040 con 0x55d56fa1b720 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0343e00700 1 -- 172.21.15.46:0/439629860 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0350022070 con 0x55d56fa1b720 2024-05-08T23:07:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.523+0000 7f0343e00700 1 -- 172.21.15.46:0/439629860 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350027470 con 0x55d56fa1b720 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 -- 172.21.15.46:0/439629860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 msgr2=0x55d56fa21200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0350007cb0 tx=0x7f0350007ce0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 -- 172.21.15.46:0/439629860 shutdown_connections 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa1b720 0x55d56fa21200 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa15890 0x55d56fa15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 --2- 172.21.15.46:0/439629860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1ae00 0x55d56fa0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 -- 172.21.15.46:0/439629860 >> 172.21.15.46:0/439629860 conn(0x55d56f973210 msgr2=0x55d56fa18ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 -- 172.21.15.46:0/439629860 shutdown_connections 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.524+0000 7f0371137b80 1 -- 172.21.15.46:0/439629860 wait complete. 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 -- start start 2024-05-08T23:07:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 Processor -- start 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1ae00 0x55d56fa267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1b720 0x55d56fa16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa17010 con 0x55d56fa1ae00 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa17180 con 0x55d56fa1b720 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56fa17340 con 0x55d56fa15890 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.525+0000 7f0371137b80 10 client.0 _ll_register_callbacks cb 0x55d56f90ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52350/0 (socket says 172.21.15.46:52350) 2024-05-08T23:07:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 -- 172.21.15.46:0/2579543580 learned_addr learned my addr 172.21.15.46:0/2579543580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0357400700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1b720 0x55d56fa16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356000700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1ae00 0x55d56fa267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 -- 172.21.15.46:0/2579543580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1b720 msgr2=0x55d56fa16930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1b720 0x55d56fa16930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 -- 172.21.15.46:0/2579543580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1ae00 msgr2=0x55d56fa267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1ae00 0x55d56fa267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 -- 172.21.15.46:0/2579543580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03581107c0 con 0x55d56fa15890 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0356a00700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0348099b40 tx=0x7f0348005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3166514ba3a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0337e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03480ca670 con 0x55d56fa15890 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0348099e10 con 0x55d56fa15890 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.526+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034400e840 con 0x55d56fa15890 2024-05-08T23:07:59.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0348099fd0 con 0x55d56fa15890 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 10 client.26836 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d56f7eacd0 con 0x55d56fa15890 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0348063080 con 0x55d56fa15890 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f034809abc0 con 0x55d56fa15890 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 10 client.26836 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 10 client.26836 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 20 client.26836 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa8ed00 0x55d56fa91120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 4 client.26836 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d56f95dec0 con 0x55d56fa8ed00 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.527+0000 7f0356000700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa8ed00 0x55d56fa91120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0356000700 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa8ed00 0x55d56fa91120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7b97076002ab459 server_cookie=7f1059d898b5ce4f in_seq=0 out_seq=0 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0337e00700 10 client.26836 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:59.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0337e00700 1 -- 172.21.15.46:0/2579543580 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d56f95dec0 con 0x55d56fa8ed00 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0337e00700 10 client.26836 handle_command_reply: tid=0 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 client.26836 shutdown 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 msgr2=0x55d56fa26290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0348099b40 tx=0x7f0348005910 comp rx=0 tx=0).stop 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 shutdown_connections 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55d56fa8ed00 0x55d56fa91120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d56fa1b720 0x55d56fa16930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d56fa15890 0x55d56fa26290 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0337e00700 0 client.26836 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 --2- 172.21.15.46:0/2579543580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d56fa1ae00 0x55d56fa267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 >> 172.21.15.46:0/2579543580 conn(0x55d56f973210 msgr2=0x55d56fa19a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.528+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 shutdown_connections 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f0371137b80 1 -- 172.21.15.46:0/2579543580 wait complete. 2024-05-08T23:07:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f0371137b80 20 client.26836 trim_cache size 0 max 16384 2024-05-08T23:07:59.533 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 msgr2=0x7f033c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 0x7f033c0a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f035819aad0 tx=0x7f0344009260 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 msgr2=0x7f03581995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f034800d950 tx=0x7f034800dc60 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 shutdown_connections 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f035819de60 0x7f035819e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03581032b0 0x7f03581995d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f033c0a1290 0x7f033c0a3750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 --2- 172.21.15.46:0/3546853420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03581083d0 0x7f0358199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.529+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 >> 172.21.15.46:0/3546853420 conn(0x7f03580feb10 msgr2=0x7f0358100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.530+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 shutdown_connections 2024-05-08T23:07:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.530+0000 7f035d600700 1 -- 172.21.15.46:0/3546853420 wait complete. 2024-05-08T23:07:59.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:07:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 -- 172.21.15.46:0/514019610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 msgr2=0x7f4fd010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 --2- 172.21.15.46:0/514019610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd010ada0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009990 tx=0x7f4fbc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 -- 172.21.15.46:0/514019610 shutdown_connections 2024-05-08T23:07:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 --2- 172.21.15.46:0/514019610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 0x7f4fd010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 --2- 172.21.15.46:0/514019610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 --2- 172.21.15.46:0/514019610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd010ada0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 -- 172.21.15.46:0/514019610 >> 172.21.15.46:0/514019610 conn(0x7f4fd00fb760 msgr2=0x7f4fd00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 -- 172.21.15.46:0/514019610 shutdown_connections 2024-05-08T23:07:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.766+0000 7f4fd7600700 1 -- 172.21.15.46:0/514019610 wait complete. 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.767+0000 7f4fd7600700 1 Processor -- start 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 -- start start 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 0x7f4fd019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd019e390 con 0x7f4fd0101fa0 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd019e500 con 0x7f4fd010b3c0 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.768+0000 7f4fd7600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd019e6c0 con 0x7f4fd01028c0 2024-05-08T23:07:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51988/0 (socket says 172.21.15.46:51988) 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fd4c00700 1 -- 172.21.15.46:0/1111118096 learned_addr learned my addr 172.21.15.46:0/1111118096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fd5600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 0x7f4fd019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 -- 172.21.15.46:0/1111118096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 msgr2=0x7f4fd019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 0x7f4fd019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 -- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 msgr2=0x7f4fd0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 -- 172.21.15.46:0/1111118096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0095f0 con 0x7f4fd01028c0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.769+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd01999b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc006010 tx=0x7f4fbc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.770+0000 7f4fcd600700 1 -- 172.21.15.46:0/1111118096 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc01d070 con 0x7f4fd01028c0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.770+0000 7f4fcd600700 1 -- 172.21.15.46:0/1111118096 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fbc022470 con 0x7f4fd01028c0 2024-05-08T23:07:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.770+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd019e940 con 0x7f4fd01028c0 2024-05-08T23:07:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.770+0000 7f4fcd600700 1 -- 172.21.15.46:0/1111118096 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00bee0 con 0x7f4fd01028c0 2024-05-08T23:07:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.770+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd01a61e0 con 0x7f4fd01028c0 2024-05-08T23:07:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.774+0000 7f4fcd600700 1 -- 172.21.15.46:0/1111118096 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4fbc022890 con 0x7f4fd01028c0 2024-05-08T23:07:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.775+0000 7f4feb078b80 1 Processor -- start 2024-05-08T23:07:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.775+0000 7f4feb078b80 1 -- start start 2024-05-08T23:07:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a56d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a56dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a4c4150 con 0x555c5a573720 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a4c1b10 con 0x555c5a56d890 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4feb078b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a4bd400 con 0x555c5a572e00 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fcd600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 0x7f4fb80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fcfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a56dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fd5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fd5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52388/0 (socket says 172.21.15.46:52388) 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fcfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a56dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53576/0 (socket says 172.21.15.46:53576) 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fd5600700 1 -- 172.21.15.46:0/645623860 learned_addr learned my addr 172.21.15.46:0/645623860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fd5600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 0x7f4fb80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fcd600700 1 -- 172.21.15.46:0/1111118096 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fbc0ce220 con 0x7f4fd01028c0 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.776+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a56d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 -- 172.21.15.46:0/645623860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 msgr2=0x555c5a56dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a56dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 -- 172.21.15.46:0/645623860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 msgr2=0x555c5a56d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a56d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 -- 172.21.15.46:0/645623860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5a33e3b0 con 0x555c5a572e00 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8008fa0 tx=0x7f4fc801a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f1ca9592917510 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.777+0000 7f4fd5600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 0x7f4fb80a37d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc801e000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.778+0000 7f4fc1c00700 1 -- 172.21.15.46:0/645623860 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8027070 con 0x555c5a572e00 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.778+0000 7f4fc1c00700 1 -- 172.21.15.46:0/645623860 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc800f040 con 0x555c5a572e00 2024-05-08T23:07:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.778+0000 7f4fc1c00700 1 -- 172.21.15.46:0/645623860 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800e030 con 0x555c5a572e00 2024-05-08T23:07:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.778+0000 7f4feb078b80 1 -- 172.21.15.46:0/645623860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 msgr2=0x555c5a5731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8008fa0 tx=0x7f4fc801a6c0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 -- 172.21.15.46:0/645623860 shutdown_connections 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a56dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a5731e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 --2- 172.21.15.46:0/645623860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a56d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 -- 172.21.15.46:0/645623860 >> 172.21.15.46:0/645623860 conn(0x555c5a4cb210 msgr2=0x555c5a567df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 -- 172.21.15.46:0/645623860 shutdown_connections 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 -- 172.21.15.46:0/645623860 wait complete. 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.779+0000 7f4feb078b80 1 -- start start 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 Processor -- start 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a576a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a576fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a577510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a459560 con 0x555c5a56d890 2024-05-08T23:07:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a459200 con 0x555c5a572e00 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.780+0000 7f4feb078b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a3406a0 con 0x555c5a573720 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4feb078b80 10 client.0 _ll_register_callbacks cb 0x555c5a462b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a576a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a577510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a576a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52006/0 (socket says 172.21.15.46:52006) 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd4c00700 1 -- 172.21.15.46:0/4234641760 learned_addr learned my addr 172.21.15.46:0/4234641760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a576fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 -- 172.21.15.46:0/4234641760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 msgr2=0x555c5a576fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a576fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 -- 172.21.15.46:0/4234641760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 msgr2=0x555c5a576a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a576a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.781+0000 7f4fd5600700 1 -- 172.21.15.46:0/4234641760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc8008b50 con 0x555c5a573720 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fd5600700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a577510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8008fa0 tx=0x7f4fc801bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c2562d06464901 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800e070 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fc8005270 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5a416270 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc8027040 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4feb078b80 10 client.26848 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555c5a2eab60 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c5a416270 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.782+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4fc8033070 con 0x555c5a573720 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 10 client.26848 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 10 client.26848 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 20 client.26848 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a587150 0x555c5a589570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 4 client.26848 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c5a53c810 con 0x555c5a587150 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a587150 0x555c5a589570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4fb6000700 10 client.26848 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.783+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a587150 0x555c5a589570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d371a0e4c4311a93 server_cookie=7dc13ed4bd04f1cf in_seq=0 out_seq=0 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4fb6000700 1 -- 172.21.15.46:0/4234641760 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c5a53c810 con 0x555c5a587150 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4fb6000700 10 client.26848 handle_command_reply: tid=0 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 client.26848 shutdown 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 msgr2=0x555c5a577510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a577510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8008fa0 tx=0x7f4fc801bd50 comp rx=0 tx=0).stop 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 shutdown_connections 2024-05-08T23:07:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a587150 0x555c5a589570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a577510 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a576a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 --2- 172.21.15.46:0/4234641760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a576fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.784+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 >> 172.21.15.46:0/4234641760 conn(0x555c5a4cb210 msgr2=0x555c5a5675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.785+0000 7f4fb6000700 0 client.26848 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.785+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 shutdown_connections 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.785+0000 7f4feb078b80 1 -- 172.21.15.46:0/4234641760 wait complete. 2024-05-08T23:07:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.785+0000 7f4feb078b80 20 client.26848 trim_cache size 0 max 16384 2024-05-08T23:07:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.802+0000 7f4feb078b80 1 Processor -- start 2024-05-08T23:07:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.802+0000 7f4feb078b80 1 -- start start 2024-05-08T23:07:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.802+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a583260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a567d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4feb078b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a5697e0 con 0x555c5a573720 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4feb078b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a586ea0 con 0x555c5a56d890 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4feb078b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a568e80 con 0x555c5a572e00 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a583260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52406/0 (socket says 172.21.15.46:52406) 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 -- 172.21.15.46:0/2244048489 learned_addr learned my addr 172.21.15.46:0/2244048489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a583260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53594/0 (socket says 172.21.15.46:53594) 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fd5600700 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a567d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 -- 172.21.15.46:0/2244048489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 msgr2=0x555c5a583260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a583260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 -- 172.21.15.46:0/2244048489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 msgr2=0x555c5a567d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a567d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.803+0000 7f4fcfe00700 1 -- 172.21.15.46:0/2244048489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5a483560 con 0x555c5a572e00 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.804+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098e0 tx=0x7f4fbc06b130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8d7dd9664e0592 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.804+0000 7f4fc2600700 1 -- 172.21.15.46:0/2244048489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc09ab50 con 0x555c5a572e00 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.804+0000 7f4fc2600700 1 -- 172.21.15.46:0/2244048489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fbc09a970 con 0x555c5a572e00 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.804+0000 7f4fc2600700 1 -- 172.21.15.46:0/2244048489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc09eed0 con 0x555c5a572e00 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- 172.21.15.46:0/2244048489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 msgr2=0x555c5a569a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098e0 tx=0x7f4fbc06b130 comp rx=0 tx=0).stop 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- 172.21.15.46:0/2244048489 shutdown_connections 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a573720 0x555c5a567d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a56d890 0x555c5a583260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2244048489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a572e00 0x555c5a569a00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- 172.21.15.46:0/2244048489 >> 172.21.15.46:0/2244048489 conn(0x555c5a4cb210 msgr2=0x555c5a571b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- 172.21.15.46:0/2244048489 shutdown_connections 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- 172.21.15.46:0/2244048489 wait complete. 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.805+0000 7f4feb078b80 1 -- start start 2024-05-08T23:07:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 Processor -- start 2024-05-08T23:07:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a57a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a57a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a57ea20 con 0x555c5a56d890 2024-05-08T23:07:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a57ab60 con 0x555c5a572e00 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5a57ad20 con 0x555c5a573720 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.806+0000 7f4feb078b80 10 client.0 _ll_register_callbacks cb 0x555c5a462b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52024/0 (socket says 172.21.15.46:52024) 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd4c00700 1 -- 172.21.15.46:0/2090893087 learned_addr learned my addr 172.21.15.46:0/2090893087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fcfe00700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a57a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a57a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 -- 172.21.15.46:0/2090893087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 msgr2=0x555c5a57a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a57a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 -- 172.21.15.46:0/2090893087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 msgr2=0x555c5a57e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 -- 172.21.15.46:0/2090893087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd01a61e0 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fd5600700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a57a620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4fc801b010 tx=0x7f4fc80095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8942cd63dc6be33f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.807+0000 7f4fb6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800e030 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fc801bdc0 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fc800f040 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc8024ea0 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 10 client.26854 fetch_fsmap learned FSMap version 29 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555c5a342cd0 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc800f040 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4fc801a730 con 0x555c5a573720 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 10 client.26854 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 10 client.26854 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:07:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.808+0000 7f4feb078b80 20 client.26854 populate_metadata read hostname 'smithi046' 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a5dd740 0x555c5a5dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4feb078b80 4 client.26854 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c5a4b5ec0 con 0x555c5a5dd740 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a5dd740 0x555c5a5dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4fd4c00700 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a5dd740 0x555c5a5dfb30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32354e224a64f446 server_cookie=c84cf78fc8f15bed in_seq=0 out_seq=0 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.809+0000 7f4fb6000700 10 client.26854 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4fb6000700 1 -- 172.21.15.46:0/2090893087 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c5a4b5ec0 con 0x555c5a5dd740 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4fb6000700 10 client.26854 handle_command_reply: tid=0 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 client.26854 shutdown 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 msgr2=0x555c5a57a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a57a620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4fc801b010 tx=0x7f4fc80095e0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 shutdown_connections 2024-05-08T23:07:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x555c5a5dd740 0x555c5a5dfb30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5a573720 0x555c5a57a620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c5a56d890 0x555c5a57e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 --2- 172.21.15.46:0/2090893087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c5a572e00 0x555c5a57a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4fb6000700 0 client.26854 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 >> 172.21.15.46:0/2090893087 conn(0x555c5a4cb210 msgr2=0x555c5a574390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 shutdown_connections 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 1 -- 172.21.15.46:0/2090893087 wait complete. 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.810+0000 7f4feb078b80 20 client.26854 trim_cache size 0 max 16384 2024-05-08T23:07:59.814 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:07:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 msgr2=0x7f4fb80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 0x7f4fb80a37d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc801e000 comp rx=0 tx=0).stop 2024-05-08T23:07:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 msgr2=0x7f4fd01999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:07:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd01999b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc006010 tx=0x7f4fbc005740 comp rx=0 tx=0).stop 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 shutdown_connections 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd010b3c0 0x7f4fd019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1310 0x7f4fb80a37d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd0101fa0 0x7f4fd0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 --2- 172.21.15.46:0/1111118096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd01028c0 0x7f4fd01999b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 >> 172.21.15.46:0/1111118096 conn(0x7f4fd00fb760 msgr2=0x7f4fd010c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 shutdown_connections 2024-05-08T23:07:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:07:59.811+0000 7f4fd7600700 1 -- 172.21.15.46:0/1111118096 wait complete. 2024-05-08T23:07:59.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:59 smithi046 ceph-mon[36898]: pgmap v128: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:07:59.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:59 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:07:59.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:59 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:07:59.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:59 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:07:59.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:07:59 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/640492607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:07:59.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:07:59.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 254865 11.19 MB/sec execute 21 sec latency 19.372 ms 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 -- 172.21.15.46:0/3560068438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c30101fa0 msgr2=0x7f0c30102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 --2- 172.21.15.46:0/3560068438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c30101fa0 0x7f0c30102380 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009990 tx=0x7f0c20009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 -- 172.21.15.46:0/3560068438 shutdown_connections 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 --2- 172.21.15.46:0/3560068438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3010b3c0 0x7f0c3010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 --2- 172.21.15.46:0/3560068438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c30101fa0 0x7f0c30102380 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 --2- 172.21.15.46:0/3560068438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c301028c0 0x7f0c3010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 -- 172.21.15.46:0/3560068438 >> 172.21.15.46:0/3560068438 conn(0x7f0c300fb760 msgr2=0x7f0c300fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 -- 172.21.15.46:0/3560068438 shutdown_connections 2024-05-08T23:08:00.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.055+0000 7f0c34e00700 1 -- 172.21.15.46:0/3560068438 wait complete. 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.056+0000 7f0c34e00700 1 Processor -- start 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.056+0000 7f0c34e00700 1 -- start start 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c301028c0 0x7f0c30199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3010b3c0 0x7f0c3019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3019e400 con 0x7f0c301028c0 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3019e570 con 0x7f0c3010b3c0 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c34e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3019e730 con 0x7f0c30101fa0 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52422/0 (socket says 172.21.15.46:52422) 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 -- 172.21.15.46:0/4004074974 learned_addr learned my addr 172.21.15.46:0/4004074974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2d400700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c301028c0 0x7f0c30199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2e800700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3010b3c0 0x7f0c3019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 -- 172.21.15.46:0/4004074974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3010b3c0 msgr2=0x7f0c3019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3010b3c0 0x7f0c3019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 -- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c301028c0 msgr2=0x7f0c30199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c301028c0 0x7f0c30199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.057+0000 7f0c2de00700 1 -- 172.21.15.46:0/4004074974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c28009c40 con 0x7f0c30101fa0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c2de00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c20005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c26a00700 1 -- 172.21.15.46:0/4004074974 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2001d070 con 0x7f0c30101fa0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c26a00700 1 -- 172.21.15.46:0/4004074974 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c20022470 con 0x7f0c30101fa0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c26a00700 1 -- 172.21.15.46:0/4004074974 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20017650 con 0x7f0c30101fa0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200095f0 con 0x7f0c30101fa0 2024-05-08T23:08:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.058+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c301a6070 con 0x7f0c30101fa0 2024-05-08T23:08:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.061+0000 7f0c26a00700 1 -- 172.21.15.46:0/4004074974 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0c200225e0 con 0x7f0c30101fa0 2024-05-08T23:08:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.062+0000 7f0c4895bb80 1 Processor -- start 2024-05-08T23:08:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 -- start start 2024-05-08T23:08:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 0x560e352191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 0x560e35213350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 0x560e35213cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3516a150 con 0x560e35213890 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e35167b10 con 0x560e35219720 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e35163400 con 0x560e35218e00 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c26a00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 0x7f0c140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c2e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 0x560e352191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c2e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 0x560e352191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52424/0 (socket says 172.21.15.46:52424) 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c2e800700 1 -- 172.21.15.46:0/3282086132 learned_addr learned my addr 172.21.15.46:0/3282086132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c2de00700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 0x560e35213350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c2d400700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 0x560e35213cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.063+0000 7f0c26a00700 1 -- 172.21.15.46:0/4004074974 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0c200ce4a0 con 0x7f0c30101fa0 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 -- 172.21.15.46:0/3282086132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 msgr2=0x560e352191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 0x560e352191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 -- 172.21.15.46:0/3282086132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 msgr2=0x560e35213350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 0x560e35213350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 -- 172.21.15.46:0/3282086132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e34fe43b0 con 0x560e35213890 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2de00700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 0x560e35213350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2e800700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 0x7f0c140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c2d400700 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 0x560e35213cd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009c70 tx=0x7f0c1c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7193c6f57450e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c1b400700 1 -- 172.21.15.46:0/3282086132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c011cb0 con 0x560e35213890 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c1b400700 1 -- 172.21.15.46:0/3282086132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1c011e10 con 0x560e35213890 2024-05-08T23:08:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.064+0000 7f0c1b400700 1 -- 172.21.15.46:0/3282086132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0193e0 con 0x560e35213890 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 -- 172.21.15.46:0/3282086132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 msgr2=0x560e35213cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c2e800700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 0x7f0c140a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009b50 tx=0x7f0c28004b40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 0x560e35213cd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009c70 tx=0x7f0c1c009f80 comp rx=0 tx=0).stop 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 -- 172.21.15.46:0/3282086132 shutdown_connections 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35213890 0x560e35213cd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35218e00 0x560e352191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/3282086132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35219720 0x560e35213350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 -- 172.21.15.46:0/3282086132 >> 172.21.15.46:0/3282086132 conn(0x560e35171210 msgr2=0x560e3520ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 -- 172.21.15.46:0/3282086132 shutdown_connections 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.065+0000 7f0c4895bb80 1 -- 172.21.15.46:0/3282086132 wait complete. 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.066+0000 7f0c4895bb80 1 -- start start 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.066+0000 7f0c4895bb80 1 Processor -- start 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e3521d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e3521d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e350ff560 con 0x560e35219720 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e350ff200 con 0x560e35218e00 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34fe66a0 con 0x560e35213890 2024-05-08T23:08:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52436/0 (socket says 172.21.15.46:52436) 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 -- 172.21.15.46:0/2093569762 learned_addr learned my addr 172.21.15.46:0/2093569762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2d400700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e3521d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c4895bb80 10 client.0 _ll_register_callbacks cb 0x560e35108b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2e800700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e3521d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 -- 172.21.15.46:0/2093569762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 msgr2=0x560e3521d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e3521d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 -- 172.21.15.46:0/2093569762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 msgr2=0x560e3521d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e3521d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.067+0000 7f0c2de00700 1 -- 172.21.15.46:0/2093569762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c301a6070 con 0x560e35213890 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c2de00700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cb840 tx=0x7f0c2006a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b473db3572b0e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2009ae90 con 0x560e35213890 2024-05-08T23:08:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0c20066e10 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e350bc270 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2009acb0 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c4895bb80 10 client.26866 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f0c1c0098d0 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e350bc270 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.068+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0c200b3030 con 0x560e35213890 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 10 client.26866 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 10 client.26866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 20 client.26866 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e3522cd50 0x560e3522f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 4 client.26866 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e351e2810 con 0x560e3522cd50 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.069+0000 7f0c2d400700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e3522cd50 0x560e3522f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.070+0000 7f0c2d400700 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e3522cd50 0x560e3522f170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bddc7ddb362683e4 server_cookie=a4a7309526330027 in_seq=0 out_seq=0 2024-05-08T23:08:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.070+0000 7f0c0fe00700 10 client.26866 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c0fe00700 1 -- 172.21.15.46:0/2093569762 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e351e2810 con 0x560e3522cd50 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c0fe00700 10 client.26866 handle_command_reply: tid=0 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 client.26866 shutdown 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 msgr2=0x560e3521cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cb840 tx=0x7f0c2006a730 comp rx=0 tx=0).stop 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 shutdown_connections 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e3522cd50 0x560e3522f170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e3521d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e3521cb80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2093569762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e3521d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c0fe00700 0 client.26866 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 >> 172.21.15.46:0/2093569762 conn(0x560e35171210 msgr2=0x560e3520d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 shutdown_connections 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.071+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2093569762 wait complete. 2024-05-08T23:08:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.072+0000 7f0c4895bb80 20 client.26866 trim_cache size 0 max 16384 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.086+0000 7f0c4895bb80 1 Processor -- start 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 -- start start 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35213890 0x560e35213350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35218e00 0x560e35218360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3520adc0 con 0x560e35218e00 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3520ee80 con 0x560e35213890 2024-05-08T23:08:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e3521fc40 con 0x560e35219720 2024-05-08T23:08:00.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c2e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c2d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35218e00 0x560e35218360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c2e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52452/0 (socket says 172.21.15.46:52452) 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c2e800700 1 -- 172.21.15.46:0/2715604265 learned_addr learned my addr 172.21.15.46:0/2715604265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.087+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35213890 0x560e35213350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 -- 172.21.15.46:0/2715604265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35213890 msgr2=0x560e35213350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35213890 0x560e35213350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 -- 172.21.15.46:0/2715604265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35218e00 msgr2=0x560e35218360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35218e00 0x560e35218360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 -- 172.21.15.46:0/2715604265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e34f90b60 con 0x560e35219720 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c2e800700 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009240 tx=0x7f0c28009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71dabbf1d768e6fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c1be00700 1 -- 172.21.15.46:0/2715604265 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800e040 con 0x560e35219720 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c1be00700 1 -- 172.21.15.46:0/2715604265 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c28021040 con 0x560e35219720 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.088+0000 7f0c1be00700 1 -- 172.21.15.46:0/2715604265 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280266a0 con 0x560e35219720 2024-05-08T23:08:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2715604265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 msgr2=0x560e3520dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009240 tx=0x7f0c28009920 comp rx=0 tx=0).stop 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2715604265 shutdown_connections 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35219720 0x560e3520dd20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35213890 0x560e35213350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/2715604265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35218e00 0x560e35218360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2715604265 >> 172.21.15.46:0/2715604265 conn(0x560e35171210 msgr2=0x560e35217c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2715604265 shutdown_connections 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.089+0000 7f0c4895bb80 1 -- 172.21.15.46:0/2715604265 wait complete. 2024-05-08T23:08:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 -- start start 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 Processor -- start 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e35222710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e35222c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e35223190 con 0x560e35219720 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e35223300 con 0x560e35218e00 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e352234c0 con 0x560e35213890 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.090+0000 7f0c4895bb80 10 client.0 _ll_register_callbacks cb 0x560e35108b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e35222c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52466/0 (socket says 172.21.15.46:52466) 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 -- 172.21.15.46:0/162992852 learned_addr learned my addr 172.21.15.46:0/162992852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2d400700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e35222710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 -- 172.21.15.46:0/162992852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 msgr2=0x560e35222710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e35222710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 -- 172.21.15.46:0/162992852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 msgr2=0x560e35222c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e35222c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 -- 172.21.15.46:0/162992852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2800eda0 con 0x560e35213890 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c2de00700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005b40 tx=0x7f0c20005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65b3b3a65b0d6a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200ce2c0 con 0x560e35213890 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0c200b0020 con 0x560e35213890 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.091+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c1c011e10 con 0x560e35213890 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2009ae90 con 0x560e35213890 2024-05-08T23:08:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 10 client.26872 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560e35129560 con 0x560e35213890 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1c011e10 con 0x560e35213890 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f0c2001f2d0 con 0x560e35213890 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 10 client.26872 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 10 client.26872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 20 client.26872 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e35284230 0x560e35286620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 4 client.26872 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e3515bec0 con 0x560e35284230 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.092+0000 7f0c2d400700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e35284230 0x560e35286620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.093+0000 7f0c2d400700 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e35284230 0x560e35286620 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=974413028f6a2ced server_cookie=c027a77dd62ef85e in_seq=0 out_seq=0 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.093+0000 7f0c0fe00700 10 client.26872 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.093+0000 7f0c0fe00700 1 -- 172.21.15.46:0/162992852 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e3515bec0 con 0x560e35284230 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.093+0000 7f0c0fe00700 10 client.26872 handle_command_reply: tid=0 2024-05-08T23:08:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.093+0000 7f0c4895bb80 1 client.26872 shutdown 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 msgr2=0x560e35224270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005b40 tx=0x7f0c20005e80 comp rx=0 tx=0).stop 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 shutdown_connections 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560e35284230 0x560e35286620 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e35219720 0x560e35222c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e35213890 0x560e35224270 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 --2- 172.21.15.46:0/162992852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e35218e00 0x560e35222710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c0fe00700 0 client.26872 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 >> 172.21.15.46:0/162992852 conn(0x560e35171210 msgr2=0x560e35219b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 shutdown_connections 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 1 -- 172.21.15.46:0/162992852 wait complete. 2024-05-08T23:08:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.094+0000 7f0c4895bb80 20 client.26872 trim_cache size 0 max 16384 2024-05-08T23:08:00.098 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 msgr2=0x7f0c140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 0x7f0c140a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009b50 tx=0x7f0c28004b40 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 msgr2=0x7f0c301994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c20005740 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 shutdown_connections 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3010b3c0 0x7f0c3019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c30101fa0 0x7f0c301994e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c140a1290 0x7f0c140a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 --2- 172.21.15.46:0/4004074974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c301028c0 0x7f0c30199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 >> 172.21.15.46:0/4004074974 conn(0x7f0c300fb760 msgr2=0x7f0c300fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 shutdown_connections 2024-05-08T23:08:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.095+0000 7f0c34e00700 1 -- 172.21.15.46:0/4004074974 wait complete. 2024-05-08T23:08:00.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:08:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:59 smithi086 ceph-mon[78610]: pgmap v128: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:08:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:59 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:08:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:59 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:08:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:59 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:08:00.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:07:59 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/640492607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:08:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:59 smithi053 ceph-mon[77587]: pgmap v128: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-08T23:08:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:59 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:08:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:59 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:08:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:59 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:08:00.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:07:59 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/640492607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 -- 172.21.15.46:0/3696705721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 msgr2=0x7f41a410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 --2- 172.21.15.46:0/3696705721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a410f930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f419c009f90 tx=0x7f419c00b330 comp rx=0 tx=0).stop 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 -- 172.21.15.46:0/3696705721 shutdown_connections 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 --2- 172.21.15.46:0/3696705721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a410f930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 --2- 172.21.15.46:0/3696705721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 0x7f41a4073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 --2- 172.21.15.46:0/3696705721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 -- 172.21.15.46:0/3696705721 >> 172.21.15.46:0/3696705721 conn(0x7f41a40fc7b0 msgr2=0x7f41a40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 -- 172.21.15.46:0/3696705721 shutdown_connections 2024-05-08T23:08:00.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.333+0000 7f41a3e00700 1 -- 172.21.15.46:0/3696705721 wait complete. 2024-05-08T23:08:00.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.334+0000 7f41a3e00700 1 Processor -- start 2024-05-08T23:08:00.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 -- start start 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 0x7f41a4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419e330 con 0x7f41a410d500 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419e4a0 con 0x7f41a4073370 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419e660 con 0x7f41a4073c90 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 0x7f41a4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52488/0 (socket says 172.21.15.46:52488) 2024-05-08T23:08:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.335+0000 7f41a2000700 1 -- 172.21.15.46:0/139090512 learned_addr learned my addr 172.21.15.46:0/139090512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 -- 172.21.15.46:0/139090512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 msgr2=0x7f41a4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 0x7f41a4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 -- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 msgr2=0x7f41a419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 -- 172.21.15.46:0/139090512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4194009520 con 0x7f41a4073c90 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.336+0000 7f41a2000700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f419000e9c0 tx=0x7f419000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.337+0000 7f419b400700 1 -- 172.21.15.46:0/139090512 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000cbd0 con 0x7f41a4073c90 2024-05-08T23:08:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.337+0000 7f419b400700 1 -- 172.21.15.46:0/139090512 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419000eeb0 con 0x7f41a4073c90 2024-05-08T23:08:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.337+0000 7f419b400700 1 -- 172.21.15.46:0/139090512 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190005580 con 0x7f41a4073c90 2024-05-08T23:08:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.337+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c009c40 con 0x7f41a4073c90 2024-05-08T23:08:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.337+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a41a5fa0 con 0x7f41a4073c90 2024-05-08T23:08:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.341+0000 7f419b400700 1 -- 172.21.15.46:0/139090512 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4190004750 con 0x7f41a4073c90 2024-05-08T23:08:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.342+0000 7f41bd375b80 1 Processor -- start 2024-05-08T23:08:00.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.342+0000 7f41bd375b80 1 -- start start 2024-05-08T23:08:00.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b5fb150 con 0x55a15b6b2870 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b5f8b10 con 0x55a15b6ad520 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f419b400700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 0x7f41880a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41bd375b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b5f4400 con 0x55a15b6acb60 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f419b400700 1 -- 172.21.15.46:0/139090512 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4190014070 con 0x7f41a4073c90 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53674/0 (socket says 172.21.15.46:53674) 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a2000700 1 -- 172.21.15.46:0/3533296207 learned_addr learned my addr 172.21.15.46:0/3533296207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52496/0 (socket says 172.21.15.46:52496) 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.343+0000 7f41a3400700 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 -- 172.21.15.46:0/3533296207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 msgr2=0x55a15b6acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 -- 172.21.15.46:0/3533296207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 msgr2=0x55a15b6b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 -- 172.21.15.46:0/3533296207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a15b4753b0 con 0x55a15b6ad520 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2a00700 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a3400700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 0x7f41880a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f41a2000700 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4190096b50 tx=0x7f4190099d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15b04e257d31aae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f418fe00700 1 -- 172.21.15.46:0/3533296207 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900a6f70 con 0x55a15b6ad520 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f418fe00700 1 -- 172.21.15.46:0/3533296207 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419000cd30 con 0x55a15b6ad520 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.344+0000 7f418fe00700 1 -- 172.21.15.46:0/3533296207 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900a9900 con 0x55a15b6ad520 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 -- 172.21.15.46:0/3533296207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 msgr2=0x55a15b6b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41a3400700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 0x7f41880a3870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f419c009c10 tx=0x7f419c009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4190096b50 tx=0x7f4190099d50 comp rx=0 tx=0).stop 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 -- 172.21.15.46:0/3533296207 shutdown_connections 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b9630 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.345+0000 7f41bd375b80 1 --2- 172.21.15.46:0/3533296207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.346+0000 7f41bd375b80 1 -- 172.21.15.46:0/3533296207 >> 172.21.15.46:0/3533296207 conn(0x55a15b602210 msgr2=0x55a15b69f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.346+0000 7f41bd375b80 1 -- 172.21.15.46:0/3533296207 shutdown_connections 2024-05-08T23:08:00.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.346+0000 7f41bd375b80 1 -- 172.21.15.46:0/3533296207 wait complete. 2024-05-08T23:08:00.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.346+0000 7f41bd375b80 1 -- start start 2024-05-08T23:08:00.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.347+0000 7f41bd375b80 1 Processor -- start 2024-05-08T23:08:00.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.347+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b590560 con 0x55a15b6b2870 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b590200 con 0x55a15b6ad520 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b4776a0 con 0x55a15b6acb60 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41bd375b80 10 client.0 _ll_register_callbacks cb 0x55a15b599b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52510/0 (socket says 172.21.15.46:52510) 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 -- 172.21.15.46:0/1361306750 learned_addr learned my addr 172.21.15.46:0/1361306750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2000700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a3400700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 -- 172.21.15.46:0/1361306750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 msgr2=0x55a15b6b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 -- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 msgr2=0x55a15b6b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.348+0000 7f41a2a00700 1 -- 172.21.15.46:0/1361306750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4194009a70 con 0x55a15b6acb60 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f41a2000700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f41a2a00700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4194012cf0 tx=0x7f419400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1acb7945de123ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f418cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194017070 con 0x55a15b6acb60 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4194009c60 con 0x55a15b6acb60 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.349+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a15b54d270 con 0x55a15b6acb60 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.350+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4194009e20 con 0x55a15b6acb60 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.350+0000 7f41bd375b80 10 client.26884 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.350+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f41a41a5fa0 con 0x55a15b6acb60 2024-05-08T23:08:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.350+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a15b54d270 con 0x55a15b6acb60 2024-05-08T23:08:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4194010430 con 0x55a15b6acb60 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 10 client.26884 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 10 client.26884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 20 client.26884 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b6c6bf0 0x55a15b6c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 4 client.26884 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a15b673810 con 0x55a15b6c6bf0 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.351+0000 7f41a2000700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b6c6bf0 0x55a15b6c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.352+0000 7f41a2000700 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b6c6bf0 0x55a15b6c9010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75a9a26051221255 server_cookie=8b58c6c135609144 in_seq=0 out_seq=0 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.352+0000 7f418cc00700 10 client.26884 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f418cc00700 1 -- 172.21.15.46:0/1361306750 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a15b673810 con 0x55a15b6c6bf0 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f418cc00700 10 client.26884 handle_command_reply: tid=0 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 client.26884 shutdown 2024-05-08T23:08:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 msgr2=0x55a15b6b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4194012cf0 tx=0x7f419400f6a0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 shutdown_connections 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b6c6bf0 0x55a15b6c9010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6acb60 0x55a15b6b69d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f418cc00700 0 client.26884 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1361306750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6ad520 0x55a15b6b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 >> 172.21.15.46:0/1361306750 conn(0x55a15b602210 msgr2=0x55a15b69f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 shutdown_connections 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.353+0000 7f41bd375b80 1 -- 172.21.15.46:0/1361306750 wait complete. 2024-05-08T23:08:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.354+0000 7f41bd375b80 20 client.26884 trim_cache size 0 max 16384 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 Processor -- start 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 -- start start 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6ad520 0x55a15b6b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b6a0e60 con 0x55a15b6ad520 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b69bdc0 con 0x55a15b6acb60 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41bd375b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b6c69d0 con 0x55a15b6b2870 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52518/0 (socket says 172.21.15.46:52518) 2024-05-08T23:08:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.371+0000 7f41a3400700 1 -- 172.21.15.46:0/1898311827 learned_addr learned my addr 172.21.15.46:0/1898311827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a2a00700 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a2000700 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6ad520 0x55a15b6b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 -- 172.21.15.46:0/1898311827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 msgr2=0x55a15b6b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 -- 172.21.15.46:0/1898311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6ad520 msgr2=0x55a15b6b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6ad520 0x55a15b6b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 -- 172.21.15.46:0/1898311827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a15b421b60 con 0x55a15b6b2870 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f41a3400700 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f419c007860 tx=0x7f419c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81553bd500439ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f4198c00700 1 -- 172.21.15.46:0/1898311827 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00e030 con 0x55a15b6b2870 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f4198c00700 1 -- 172.21.15.46:0/1898311827 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419c003e90 con 0x55a15b6b2870 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.372+0000 7f4198c00700 1 -- 172.21.15.46:0/1898311827 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c005610 con 0x55a15b6b2870 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 -- 172.21.15.46:0/1898311827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 msgr2=0x55a15b6bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f419c007860 tx=0x7f419c01d000 comp rx=0 tx=0).stop 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 -- 172.21.15.46:0/1898311827 shutdown_connections 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6b2870 0x55a15b6bdb00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 --2- 172.21.15.46:0/1898311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6ad520 0x55a15b6b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.373+0000 7f41bd375b80 1 -- 172.21.15.46:0/1898311827 >> 172.21.15.46:0/1898311827 conn(0x55a15b602210 msgr2=0x55a15b6cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.374+0000 7f41bd375b80 1 -- 172.21.15.46:0/1898311827 shutdown_connections 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.374+0000 7f41bd375b80 1 -- 172.21.15.46:0/1898311827 wait complete. 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.374+0000 7f41bd375b80 1 -- start start 2024-05-08T23:08:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.374+0000 7f41bd375b80 1 Processor -- start 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b71c120 con 0x55a15b6b2870 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b71c290 con 0x55a15b6acb60 2024-05-08T23:08:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15b71c450 con 0x55a15b6ad520 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41bd375b80 10 client.0 _ll_register_callbacks cb 0x55a15b599b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52532/0 (socket says 172.21.15.46:52532) 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 -- 172.21.15.46:0/286356729 learned_addr learned my addr 172.21.15.46:0/286356729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a3400700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2a00700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 -- 172.21.15.46:0/286356729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 msgr2=0x55a15b6b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 -- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 msgr2=0x55a15b6ba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6ba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.375+0000 7f41a2000700 1 -- 172.21.15.46:0/286356729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a4110640 con 0x55a15b6ad520 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41a2000700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f419000cec0 tx=0x7f419009a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261c317ef17af61a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900a2560 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f419009d040 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f419000cd30 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419009df10 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 10 client.26890 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a15b5ba560 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41900b2020 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f41900995d0 con 0x55a15b6ad520 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 10 client.26890 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 10 client.26890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 20 client.26890 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b71d970 0x55a15b71fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 4 client.26890 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.376+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a15b5ecec0 con 0x55a15b71d970 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.377+0000 7f41a2a00700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b71d970 0x55a15b71fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.377+0000 7f41a2a00700 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b71d970 0x55a15b71fd90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95b424c974e823d9 server_cookie=738e8cb8d9391741 in_seq=0 out_seq=0 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.377+0000 7f418cc00700 10 client.26890 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f418cc00700 1 -- 172.21.15.46:0/286356729 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a15b5ecec0 con 0x55a15b71d970 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f418cc00700 10 client.26890 handle_command_reply: tid=0 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 client.26890 shutdown 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 msgr2=0x55a15b6ba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f419000cec0 tx=0x7f419009a1d0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 shutdown_connections 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a15b71d970 0x55a15b71fd90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a15b6b2870 0x55a15b6ba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a15b6acb60 0x55a15b6b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f418cc00700 0 client.26890 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 --2- 172.21.15.46:0/286356729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a15b6ad520 0x55a15b6ba020 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 >> 172.21.15.46:0/286356729 conn(0x55a15b602210 msgr2=0x55a15b69e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 shutdown_connections 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 1 -- 172.21.15.46:0/286356729 wait complete. 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.378+0000 7f41bd375b80 20 client.26890 trim_cache size 0 max 16384 2024-05-08T23:08:00.382 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 msgr2=0x7f41880a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 0x7f41880a3870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f419c009c10 tx=0x7f419c009920 comp rx=0 tx=0).stop 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 msgr2=0x7f41a4199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f419000e9c0 tx=0x7f419000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 shutdown_connections 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f41880a13b0 0x7f41880a3870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a410d500 0x7f41a419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4073370 0x7f41a4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 --2- 172.21.15.46:0/139090512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a4073c90 0x7f41a4199950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 >> 172.21.15.46:0/139090512 conn(0x7f41a40fc7b0 msgr2=0x7f41a40fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 shutdown_connections 2024-05-08T23:08:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.379+0000 7f41a3e00700 1 -- 172.21.15.46:0/139090512 wait complete. 2024-05-08T23:08:00.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:08:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 -- 172.21.15.46:0/3038117328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c101080 msgr2=0x7faf0c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 --2- 172.21.15.46:0/3038117328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c101080 0x7faf0c101460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faf00009990 tx=0x7faf00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 -- 172.21.15.46:0/3038117328 shutdown_connections 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 --2- 172.21.15.46:0/3038117328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c1061a0 0x7faf0c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 --2- 172.21.15.46:0/3038117328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c101080 0x7faf0c101460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 --2- 172.21.15.46:0/3038117328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101a30 0x7faf0c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 -- 172.21.15.46:0/3038117328 >> 172.21.15.46:0/3038117328 conn(0x7faf0c0fc8e0 msgr2=0x7faf0c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 -- 172.21.15.46:0/3038117328 shutdown_connections 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.612+0000 7faf14800700 1 -- 172.21.15.46:0/3038117328 wait complete. 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.613+0000 7faf14800700 1 Processor -- start 2024-05-08T23:08:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.613+0000 7faf14800700 1 -- start start 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 0x7faf0c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c19de20 0x7faf0c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c19e8a0 con 0x7faf0c101080 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c19ea10 con 0x7faf0c19de20 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf14800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c19ebd0 con 0x7faf0c1061a0 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 0x7faf0c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52550/0 (socket says 172.21.15.46:52550) 2024-05-08T23:08:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 -- 172.21.15.46:0/2101404668 learned_addr learned my addr 172.21.15.46:0/2101404668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 0x7faf0c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52160/0 (socket says 172.21.15.46:52160) 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf12800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c19de20 0x7faf0c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 -- 172.21.15.46:0/2101404668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c19de20 msgr2=0x7faf0c19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c19de20 0x7faf0c19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 -- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 msgr2=0x7faf0c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 0x7faf0c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.614+0000 7faf11400700 1 -- 172.21.15.46:0/2101404668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009520 con 0x7faf0c1061a0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf11400700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faefc00ea30 tx=0x7faefc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf06a00700 1 -- 172.21.15.46:0/2101404668 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc009820 con 0x7faf0c1061a0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf06a00700 1 -- 172.21.15.46:0/2101404668 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc0044e0 con 0x7faf0c1061a0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf06a00700 1 -- 172.21.15.46:0/2101404668 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc003dc0 con 0x7faf0c1061a0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000095f0 con 0x7faf0c1061a0 2024-05-08T23:08:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.615+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1a6480 con 0x7faf0c1061a0 2024-05-08T23:08:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.618+0000 7faf06a00700 1 -- 172.21.15.46:0/2101404668 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faefc003f80 con 0x7faf0c1061a0 2024-05-08T23:08:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.619+0000 7faf06a00700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 0x7faef40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.620+0000 7faf06a00700 1 -- 172.21.15.46:0/2101404668 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faefc014070 con 0x7faf0c1061a0 2024-05-08T23:08:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.620+0000 7faf11e00700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 0x7faef40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.620+0000 7faf28242b80 1 Processor -- start 2024-05-08T23:08:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.620+0000 7faf28242b80 1 -- start start 2024-05-08T23:08:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.620+0000 7faf11e00700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 0x7faef40a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf00006010 tx=0x7faf0000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1609e00 0x55c8f160a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1604cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f155b150 con 0x55c8f1609e00 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1558b10 con 0x55c8f1604890 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf28242b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1554400 con 0x55c8f160a720 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1609e00 0x55c8f160a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52562/0 (socket says 172.21.15.46:52562) 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 -- 172.21.15.46:0/301241974 learned_addr learned my addr 172.21.15.46:0/301241974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf12800700 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1604cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 -- 172.21.15.46:0/301241974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 msgr2=0x55c8f1604cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1604cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 -- 172.21.15.46:0/301241974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1609e00 msgr2=0x55c8f160a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1609e00 0x55c8f160a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf11400700 1 -- 172.21.15.46:0/301241974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f13d53b0 con 0x55c8f160a720 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.621+0000 7faf12800700 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1604cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faf11400700 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faefc0a2080 tx=0x7faefc0057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8c1b3ef27282bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faefb000700 1 -- 172.21.15.46:0/301241974 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc09a130 con 0x55c8f160a720 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faefb000700 1 -- 172.21.15.46:0/301241974 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc09c070 con 0x55c8f160a720 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faefb000700 1 -- 172.21.15.46:0/301241974 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0984d0 con 0x55c8f160a720 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faf28242b80 1 -- 172.21.15.46:0/301241974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 msgr2=0x55c8f1604350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.622+0000 7faf28242b80 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faefc0a2080 tx=0x7faefc0057e0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 -- 172.21.15.46:0/301241974 shutdown_connections 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1604cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1609e00 0x55c8f160a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 --2- 172.21.15.46:0/301241974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f1604350 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 -- 172.21.15.46:0/301241974 >> 172.21.15.46:0/301241974 conn(0x55c8f1562210 msgr2=0x55c8f15fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 -- 172.21.15.46:0/301241974 shutdown_connections 2024-05-08T23:08:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 -- 172.21.15.46:0/301241974 wait complete. 2024-05-08T23:08:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.623+0000 7faf28242b80 1 -- start start 2024-05-08T23:08:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 Processor -- start 2024-05-08T23:08:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f160e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f160e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f14f0560 con 0x55c8f1604890 2024-05-08T23:08:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f14f0200 con 0x55c8f1609e00 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f13d76a0 con 0x55c8f160a720 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.624+0000 7faf28242b80 10 client.0 _ll_register_callbacks cb 0x55c8f14f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52190/0 (socket says 172.21.15.46:52190) 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f160e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 -- 172.21.15.46:0/2147631039 learned_addr learned my addr 172.21.15.46:0/2147631039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf12800700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f160e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 -- 172.21.15.46:0/2147631039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 msgr2=0x55c8f160e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f160e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 -- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 msgr2=0x55c8f160e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f160e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 -- 172.21.15.46:0/2147631039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1a6480 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11400700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f160e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faf11e00700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf0000be00 tx=0x7faf00009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365b3685e3450fca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.625+0000 7faeef400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00004d20 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faf00004e80 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8f14ad270 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf0000f460 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faf28242b80 10 client.16854 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8f1381b60 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8f14ad270 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.626+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faf0002f070 con 0x55c8f1604890 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 20 client.16854 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f161e2d0 0x55c8f16206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 4 client.16854 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8f15d3810 con 0x55c8f161e2d0 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.627+0000 7faf11400700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f161e2d0 0x55c8f16206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.628+0000 7faf11400700 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f161e2d0 0x55c8f16206f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70743f7b7e1847cb server_cookie=8f3a484db9daed5c in_seq=0 out_seq=0 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.628+0000 7faeef400700 10 client.16854 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.628+0000 7faeef400700 1 -- 172.21.15.46:0/2147631039 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8f15d3810 con 0x55c8f161e2d0 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.628+0000 7faeef400700 10 client.16854 handle_command_reply: tid=0 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 client.16854 shutdown 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 msgr2=0x55c8f160dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf0000be00 tx=0x7faf00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 shutdown_connections 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f160e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f161e2d0 0x55c8f16206f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f160dad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 --2- 172.21.15.46:0/2147631039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f160e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faeef400700 0 client.16854 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 >> 172.21.15.46:0/2147631039 conn(0x55c8f1562210 msgr2=0x55c8f15fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 shutdown_connections 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 1 -- 172.21.15.46:0/2147631039 wait complete. 2024-05-08T23:08:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.629+0000 7faf28242b80 20 client.16854 trim_cache size 0 max 16384 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.644+0000 7faf28242b80 1 Processor -- start 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 -- start start 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f1619c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f15fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f15fbdc0 con 0x55c8f1604890 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1600e50 con 0x55c8f1609e00 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf28242b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f16258c0 con 0x55c8f160a720 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf12800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f15fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52200/0 (socket says 172.21.15.46:52200) 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf11e00700 1 -- 172.21.15.46:0/3582944024 learned_addr learned my addr 172.21.15.46:0/3582944024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.645+0000 7faf11400700 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f1619c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 -- 172.21.15.46:0/3582944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 msgr2=0x55c8f15fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f15fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 -- 172.21.15.46:0/3582944024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 msgr2=0x55c8f1619c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f1619c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 -- 172.21.15.46:0/3582944024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f151a560 con 0x55c8f1604890 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faf11e00700 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b690 tx=0x7faf0000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91464fbcf95a46a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faefba00700 1 -- 172.21.15.46:0/3582944024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00024910 con 0x55c8f1604890 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faefba00700 1 -- 172.21.15.46:0/3582944024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf00028070 con 0x55c8f1604890 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.646+0000 7faefba00700 1 -- 172.21.15.46:0/3582944024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000f7d0 con 0x55c8f1604890 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 -- 172.21.15.46:0/3582944024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 msgr2=0x55c8f1604350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b690 tx=0x7faf0000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 -- 172.21.15.46:0/3582944024 shutdown_connections 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f160a720 0x55c8f15fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f1604890 0x55c8f1604350 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 --2- 172.21.15.46:0/3582944024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1609e00 0x55c8f1619c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 -- 172.21.15.46:0/3582944024 >> 172.21.15.46:0/3582944024 conn(0x55c8f1562210 msgr2=0x55c8f160c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 -- 172.21.15.46:0/3582944024 shutdown_connections 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.647+0000 7faf28242b80 1 -- 172.21.15.46:0/3582944024 wait complete. 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 -- start start 2024-05-08T23:08:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 Processor -- start 2024-05-08T23:08:00.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1615460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f160a720 0x55c8f1613690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1613d30 con 0x55c8f160a720 2024-05-08T23:08:00.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1613ea0 con 0x55c8f1604890 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f1673790 con 0x55c8f1609e00 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf28242b80 10 client.0 _ll_register_callbacks cb 0x55c8f14f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.648+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1615460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:52590/0 (socket says 172.21.15.46:52590) 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 -- 172.21.15.46:0/1869640543 learned_addr learned my addr 172.21.15.46:0/1869640543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf12800700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f160a720 0x55c8f1613690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 -- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 msgr2=0x55c8f1615460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1615460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 -- 172.21.15.46:0/1869640543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f160a720 msgr2=0x55c8f1613690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f160a720 0x55c8f1613690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 -- 172.21.15.46:0/1869640543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8f1381b60 con 0x55c8f1609e00 2024-05-08T23:08:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf11400700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faefc099e30 tx=0x7faefc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8acdd287723f1c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faeef400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc09a130 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faefc09c070 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.649+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7faefc083070 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7faefc0a25b0 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 10 client.26908 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8f13d9cd0 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc09c070 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faefc09a390 con 0x55c8f1609e00 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 20 client.26908 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f16749d0 0x55c8f1676df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 4 client.26908 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8f154cec0 con 0x55c8f16749d0 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.650+0000 7faf11e00700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f16749d0 0x55c8f1676df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf11e00700 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f16749d0 0x55c8f1676df0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2300b436fa6ce41a server_cookie=8f1df59eb35b79a9 in_seq=0 out_seq=0 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faeef400700 10 client.26908 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faeef400700 1 -- 172.21.15.46:0/1869640543 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8f154cec0 con 0x55c8f16749d0 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faeef400700 10 client.26908 handle_command_reply: tid=0 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 client.26908 shutdown 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 msgr2=0x55c8f1613150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faefc099e30 tx=0x7faefc000bc0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 shutdown_connections 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c8f16749d0 0x55c8f1676df0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8f160a720 0x55c8f1613690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8f1604890 0x55c8f1615460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faeef400700 0 client.26908 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 --2- 172.21.15.46:0/1869640543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8f1609e00 0x55c8f1613150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 >> 172.21.15.46:0/1869640543 conn(0x55c8f1562210 msgr2=0x55c8f1608bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.651+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 shutdown_connections 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf28242b80 1 -- 172.21.15.46:0/1869640543 wait complete. 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf28242b80 20 client.26908 trim_cache size 0 max 16384 2024-05-08T23:08:00.657 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 msgr2=0x7faef40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 0x7faef40a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf00006010 tx=0x7faf0000b260 comp rx=0 tx=0).stop 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 msgr2=0x7faf0c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faefc00ea30 tx=0x7faefc00ed40 comp rx=0 tx=0).stop 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 shutdown_connections 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf0c19de20 0x7faf0c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faef40a1310 0x7faef40a37d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf0c101080 0x7faf0c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 --2- 172.21.15.46:0/2101404668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf0c1061a0 0x7faf0c199b70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.652+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 >> 172.21.15.46:0/2101404668 conn(0x7faf0c0fc8e0 msgr2=0x7faf0c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.653+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 shutdown_connections 2024-05-08T23:08:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:00.653+0000 7faf14800700 1 -- 172.21.15.46:0/2101404668 wait complete. 2024-05-08T23:08:00.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 256428 11.02 MB/sec execute 22 sec latency 14.366 ms 2024-05-08T23:08:01.676 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:08:01.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:08:01.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 257839 10.81 MB/sec execute 23 sec latency 58.279 ms 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 -- 172.21.15.46:0/430390123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 msgr2=0x7f9fcc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 --2- 172.21.15.46:0/430390123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc10dd20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b0a0 tx=0x7f9fc800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 -- 172.21.15.46:0/430390123 shutdown_connections 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 --2- 172.21.15.46:0/430390123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc10dd20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 --2- 172.21.15.46:0/430390123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 --2- 172.21.15.46:0/430390123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc101a30 0x7f9fcc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 -- 172.21.15.46:0/430390123 >> 172.21.15.46:0/430390123 conn(0x7f9fcc0fc8e0 msgr2=0x7f9fcc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 -- 172.21.15.46:0/430390123 shutdown_connections 2024-05-08T23:08:01.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.909+0000 7f9fd4800700 1 -- 172.21.15.46:0/430390123 wait complete. 2024-05-08T23:08:01.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.911+0000 7f9fd4800700 1 Processor -- start 2024-05-08T23:08:01.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.911+0000 7f9fd4800700 1 -- start start 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.911+0000 7f9fd4800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.911+0000 7f9fd4800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd4800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc19bd10 0x7f9fcc19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd4800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc19c790 con 0x7f9fcc101080 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd4800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc19c900 con 0x7f9fcc19bd10 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd4800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc19cac0 con 0x7f9fcc1061a0 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42174/0 (socket says 172.21.15.46:42174) 2024-05-08T23:08:01.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 -- 172.21.15.46:0/1628924508 learned_addr learned my addr 172.21.15.46:0/1628924508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:01.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc19bd10 0x7f9fcc19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 -- 172.21.15.46:0/1628924508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc19bd10 msgr2=0x7f9fcc19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc19bd10 0x7f9fcc19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 -- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 msgr2=0x7f9fcc197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 -- 172.21.15.46:0/1628924508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc8009d00 con 0x7f9fcc1061a0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.912+0000 7f9fd1400700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00e970 tx=0x7f9fbc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.913+0000 7f9fc6a00700 1 -- 172.21.15.46:0/1628924508 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc009790 con 0x7f9fcc1061a0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.913+0000 7f9fc6a00700 1 -- 172.21.15.46:0/1628924508 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fbc00c7e0 con 0x7f9fcc1061a0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.913+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fcc1a3fc0 con 0x7f9fcc1061a0 2024-05-08T23:08:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.913+0000 7f9fc6a00700 1 -- 172.21.15.46:0/1628924508 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0054e0 con 0x7f9fcc1061a0 2024-05-08T23:08:01.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.913+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fcc1a44f0 con 0x7f9fcc1061a0 2024-05-08T23:08:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.917+0000 7f9fc6a00700 1 -- 172.21.15.46:0/1628924508 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9fbc00c950 con 0x7f9fcc1061a0 2024-05-08T23:08:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fe82b7b80 1 Processor -- start 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fc6a00700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 0x7f9fb40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fe82b7b80 1 -- start start 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fc6a00700 1 -- 172.21.15.46:0/1628924508 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fbc014070 con 0x7f9fcc1061a0 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 0x7f9fb40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.918+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe782723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe78272d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe781c91d0 con 0x55fe782787a0 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe781c6b90 con 0x55fe78277e80 2024-05-08T23:08:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe781c2480 con 0x55fe78272910 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe782723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52692/0 (socket says 172.21.15.46:52692) 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 -- 172.21.15.46:0/2209396947 learned_addr learned my addr 172.21.15.46:0/2209396947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd2800700 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe78272d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 -- 172.21.15.46:0/2209396947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 msgr2=0x55fe78272d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe78272d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 0x7f9fb40a37d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b780 tx=0x7f9fc80095c0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 -- 172.21.15.46:0/2209396947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 msgr2=0x55fe782723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe782723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.919+0000 7f9fd1e00700 1 -- 172.21.15.46:0/2209396947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe78043430 con 0x55fe78277e80 2024-05-08T23:08:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.920+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00115f0 tx=0x7f9fc0011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8553791a411ba3d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.920+0000 7f9fbb000700 1 -- 172.21.15.46:0/2209396947 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0019630 con 0x55fe78277e80 2024-05-08T23:08:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.920+0000 7f9fbb000700 1 -- 172.21.15.46:0/2209396947 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc0019c50 con 0x55fe78277e80 2024-05-08T23:08:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.920+0000 7f9fbb000700 1 -- 172.21.15.46:0/2209396947 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ebe0 con 0x55fe78277e80 2024-05-08T23:08:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2209396947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 msgr2=0x55fe78278260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00115f0 tx=0x7f9fc0011900 comp rx=0 tx=0).stop 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2209396947 shutdown_connections 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe78272d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe78278260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2209396947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe782723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2209396947 >> 172.21.15.46:0/2209396947 conn(0x55fe781d0290 msgr2=0x55fe7826ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2209396947 shutdown_connections 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.921+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2209396947 wait complete. 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.922+0000 7f9fe82b7b80 1 -- start start 2024-05-08T23:08:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.922+0000 7f9fe82b7b80 1 Processor -- start 2024-05-08T23:08:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78272910 0x55fe7827e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe7827baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7815e5e0 con 0x55fe78272910 2024-05-08T23:08:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7815e280 con 0x55fe78277e80 2024-05-08T23:08:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78045720 con 0x55fe782787a0 2024-05-08T23:08:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fe82b7b80 10 client.0 _ll_register_callbacks cb 0x55fe78167bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78272910 0x55fe7827e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42196/0 (socket says 172.21.15.46:42196) 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 -- 172.21.15.46:0/1418580282 learned_addr learned my addr 172.21.15.46:0/1418580282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd1400700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe7827baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 -- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 msgr2=0x55fe7827baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe7827baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 -- 172.21.15.46:0/1418580282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78272910 msgr2=0x55fe7827e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78272910 0x55fe7827e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.923+0000 7f9fd2800700 1 -- 172.21.15.46:0/1418580282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc0009dd0 con 0x55fe782787a0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9fd2800700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8015000 tx=0x7f9fc800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f939634868689e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9faf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800e040 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe7811b2f0 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fc801e670 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc8003e90 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9fe82b7b80 10 client.26920 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.924+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe77fefbe0 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe7811b2f0 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9fc8005400 con 0x55fe782787a0 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 10 client.26920 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 10 client.26920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 20 client.26920 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe7828c220 0x55fe7828e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 4 client.26920 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.925+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe78241890 con 0x55fe7828c220 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.926+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe7828c220 0x55fe7828e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.926+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe7828c220 0x55fe7828e640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3c244f8ffaf1ef server_cookie=745fe4a0b5b83928 in_seq=0 out_seq=0 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.926+0000 7f9faf400700 10 client.26920 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.927+0000 7f9faf400700 1 -- 172.21.15.46:0/1418580282 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe78241890 con 0x55fe7828c220 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.927+0000 7f9faf400700 10 client.26920 handle_command_reply: tid=0 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.927+0000 7f9fe82b7b80 1 client.26920 shutdown 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 msgr2=0x55fe7827c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8015000 tx=0x7f9fc800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 shutdown_connections 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe782787a0 0x55fe7827c230 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe7828c220 0x55fe7828e640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78272910 0x55fe7827e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1418580282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78277e80 0x55fe7827baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 >> 172.21.15.46:0/1418580282 conn(0x55fe781d0290 msgr2=0x55fe7826c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9faf400700 0 client.26920 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 shutdown_connections 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1418580282 wait complete. 2024-05-08T23:08:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.928+0000 7f9fe82b7b80 20 client.26920 trim_cache size 0 max 16384 2024-05-08T23:08:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 Processor -- start 2024-05-08T23:08:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 -- start start 2024-05-08T23:08:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78277e80 0x55fe7826cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe782787a0 0x55fe7827e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe782650c0 con 0x55fe78277e80 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7827a060 con 0x55fe782787a0 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78267b10 con 0x55fe78272910 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42204/0 (socket says 172.21.15.46:42204) 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fd1e00700 1 -- 172.21.15.46:0/1087698679 learned_addr learned my addr 172.21.15.46:0/1087698679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.950+0000 7f9fd1400700 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78277e80 0x55fe7826cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 -- 172.21.15.46:0/1087698679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe782787a0 msgr2=0x55fe7827e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe782787a0 0x55fe7827e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 -- 172.21.15.46:0/1087698679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78277e80 msgr2=0x55fe7826cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78277e80 0x55fe7826cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 -- 172.21.15.46:0/1087698679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe781885e0 con 0x55fe78272910 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.951+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9fc001c040 tx=0x7f9fc0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5b27c2d5b60f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.952+0000 7f9fbba00700 1 -- 172.21.15.46:0/1087698679 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002ad00 con 0x55fe78272910 2024-05-08T23:08:01.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.952+0000 7f9fbba00700 1 -- 172.21.15.46:0/1087698679 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc0019e00 con 0x55fe78272910 2024-05-08T23:08:01.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.952+0000 7f9fbba00700 1 -- 172.21.15.46:0/1087698679 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0019890 con 0x55fe78272910 2024-05-08T23:08:01.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.952+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1087698679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 msgr2=0x55fe7827ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.952+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9fc001c040 tx=0x7f9fc0011900 comp rx=0 tx=0).stop 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1087698679 shutdown_connections 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe782787a0 0x55fe7827e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78272910 0x55fe7827ba90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/1087698679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe78277e80 0x55fe7826cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1087698679 >> 172.21.15.46:0/1087698679 conn(0x55fe781d0290 msgr2=0x55fe7827b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1087698679 shutdown_connections 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/1087698679 wait complete. 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 -- start start 2024-05-08T23:08:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.953+0000 7f9fe82b7b80 1 Processor -- start 2024-05-08T23:08:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78272910 0x55fe782831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe78273f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78274640 con 0x55fe782787a0 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe782747b0 con 0x55fe78272910 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78274970 con 0x55fe78277e80 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fe82b7b80 10 client.0 _ll_register_callbacks cb 0x55fe78167bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78272910 0x55fe782831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42218/0 (socket says 172.21.15.46:42218) 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fd1400700 1 -- 172.21.15.46:0/2091730329 learned_addr learned my addr 172.21.15.46:0/2091730329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.954+0000 7f9fd2800700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe78273f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 -- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78272910 msgr2=0x55fe782831f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78272910 0x55fe782831f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 -- 172.21.15.46:0/2091730329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 msgr2=0x55fe78273f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe78273f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 -- 172.21.15.46:0/2091730329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe78043430 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd2800700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe78273f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fd1400700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc096980 tx=0x7f9fbc09a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a13d67eb07effd83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9faf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0a2390 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9fbc0b81e0 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe781baf40 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fbc09a170 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fe82b7b80 10 client.26926 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.955+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe78047d50 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fbc0a6020 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9fbc0a1f00 con 0x55fe78277e80 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 10 client.26926 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 10 client.26926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 20 client.26926 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe782e3a40 0x55fe782e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 4 client.26926 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:08:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fe78010d70 con 0x55fe782e3a40 2024-05-08T23:08:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.956+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe782e3a40 0x55fe782e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.957+0000 7f9fd1e00700 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe782e3a40 0x55fe782e5e60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703393b7d3819686 server_cookie=ebb6124ace012cf3 in_seq=0 out_seq=0 2024-05-08T23:08:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.957+0000 7f9faf400700 10 client.26926 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.958+0000 7f9faf400700 1 -- 172.21.15.46:0/2091730329 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe78010d70 con 0x55fe782e3a40 2024-05-08T23:08:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.958+0000 7f9faf400700 10 client.26926 handle_command_reply: tid=0 2024-05-08T23:08:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.958+0000 7f9fe82b7b80 1 client.26926 shutdown 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.958+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 msgr2=0x55fe78283730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.958+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc096980 tx=0x7f9fbc09a690 comp rx=0 tx=0).stop 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 shutdown_connections 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fe782e3a40 0x55fe782e5e60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe782787a0 0x55fe78273f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe78272910 0x55fe782831f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 --2- 172.21.15.46:0/2091730329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe78277e80 0x55fe78283730 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9faf400700 0 client.26926 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 >> 172.21.15.46:0/2091730329 conn(0x55fe781d0290 msgr2=0x55fe7827a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 shutdown_connections 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 1 -- 172.21.15.46:0/2091730329 wait complete. 2024-05-08T23:08:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.959+0000 7f9fe82b7b80 20 client.26926 trim_cache size 0 max 16384 2024-05-08T23:08:01.961 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:08:01.961 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:08:01.961 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6a79e379-daf1-434e-9ad0-ec49122f0bc2", 2024-05-08T23:08:01.961 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:08:01.961 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 msgr2=0x7f9fb40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 0x7f9fb40a37d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b780 tx=0x7f9fc80095c0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 msgr2=0x7f9fcc197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00e970 tx=0x7f9fbc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 shutdown_connections 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fcc19bd10 0x7f9fcc19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fb40a1310 0x7f9fb40a37d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fcc101080 0x7f9fcc197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 --2- 172.21.15.46:0/1628924508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc1061a0 0x7f9fcc197970 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 >> 172.21.15.46:0/1628924508 conn(0x7f9fcc0fc8e0 msgr2=0x7f9fcc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 shutdown_connections 2024-05-08T23:08:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:01.960+0000 7f9fd4800700 1 -- 172.21.15.46:0/1628924508 wait complete. 2024-05-08T23:08:01.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:08:02.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:01 smithi046 ceph-mon[36898]: pgmap v129: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 -- 172.21.15.46:0/2070515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75541030f0 msgr2=0x7f75541034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 --2- 172.21.15.46:0/2070515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75541030f0 0x7f75541034d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7548009990 tx=0x7f7548009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 -- 172.21.15.46:0/2070515028 shutdown_connections 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 --2- 172.21.15.46:0/2070515028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f755410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 --2- 172.21.15.46:0/2070515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75541030f0 0x7f75541034d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 --2- 172.21.15.46:0/2070515028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7554103aa0 0x7f7554107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 -- 172.21.15.46:0/2070515028 >> 172.21.15.46:0/2070515028 conn(0x7f75540fe930 msgr2=0x7f7554100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.206+0000 7f755a000700 1 -- 172.21.15.46:0/2070515028 shutdown_connections 2024-05-08T23:08:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.207+0000 7f755a000700 1 -- 172.21.15.46:0/2070515028 wait complete. 2024-05-08T23:08:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.208+0000 7f755a000700 1 Processor -- start 2024-05-08T23:08:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.208+0000 7f755a000700 1 -- start start 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 0x7f75541992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f7554199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 0x7f755419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419e640 con 0x7f75541030f0 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419e7b0 con 0x7f7554108210 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f755a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419e970 con 0x7f755419dbc0 2024-05-08T23:08:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7558c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 0x7f75541992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7553e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f7554199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 0x7f755419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7553e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f7554199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52746/0 (socket says 172.21.15.46:52746) 2024-05-08T23:08:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7553e00700 1 -- 172.21.15.46:0/1138036854 learned_addr learned my addr 172.21.15.46:0/1138036854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 -- 172.21.15.46:0/1138036854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 msgr2=0x7f7554199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f7554199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 -- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 msgr2=0x7f75541992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 0x7f75541992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.209+0000 7f7559600700 1 -- 172.21.15.46:0/1138036854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540009520 con 0x7f755419dbc0 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f7558c00700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 0x7f75541992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f7559600700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 0x7f755419e040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f754c00efc0 tx=0x7f754c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f7551600700 1 -- 172.21.15.46:0/1138036854 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c018070 con 0x7f755419dbc0 2024-05-08T23:08:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f7551600700 1 -- 172.21.15.46:0/1138036854 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754c00f040 con 0x7f755419dbc0 2024-05-08T23:08:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75480095f0 con 0x7f755419dbc0 2024-05-08T23:08:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f7551600700 1 -- 172.21.15.46:0/1138036854 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00cd40 con 0x7f755419dbc0 2024-05-08T23:08:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.210+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75541a61c0 con 0x7f755419dbc0 2024-05-08T23:08:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.213+0000 7f7551600700 1 -- 172.21.15.46:0/1138036854 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f754c003a00 con 0x7f755419dbc0 2024-05-08T23:08:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:01 smithi086 ceph-mon[78610]: pgmap v129: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-05-08T23:08:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.215+0000 7f7551600700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 0x7f753c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.215+0000 7f7551600700 1 -- 172.21.15.46:0/1138036854 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f754c0ccbf0 con 0x7f755419dbc0 2024-05-08T23:08:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.215+0000 7f756f1dab80 1 Processor -- start 2024-05-08T23:08:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.215+0000 7f7558c00700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 0x7f753c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.215+0000 7f756f1dab80 1 -- start start 2024-05-08T23:08:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e643430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e63d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e594150 con 0x55ff6e643050 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e591b10 con 0x55ff6e643970 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f756f1dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e58d400 con 0x55ff6e63d890 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f7559600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f7559600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42246/0 (socket says 172.21.15.46:42246) 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f7559600700 1 -- 172.21.15.46:0/1470807310 learned_addr learned my addr 172.21.15.46:0/1470807310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f7558c00700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e643430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.216+0000 7f7553e00700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e63d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7558c00700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 0x7f753c0a3870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7548006010 tx=0x7f754800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 -- 172.21.15.46:0/1470807310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 msgr2=0x55ff6e63d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e63d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 -- 172.21.15.46:0/1470807310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 msgr2=0x55ff6e643430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e643430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 -- 172.21.15.46:0/1470807310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6e40e3b0 con 0x55ff6e63d890 2024-05-08T23:08:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7558c00700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e643430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7559600700 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f754c0a1840 tx=0x7f754c0997d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2e5762521f772a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7545c00700 1 -- 172.21.15.46:0/1470807310 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c01b320 con 0x55ff6e63d890 2024-05-08T23:08:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7545c00700 1 -- 172.21.15.46:0/1470807310 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754c004ad0 con 0x55ff6e63d890 2024-05-08T23:08:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.217+0000 7f7545c00700 1 -- 172.21.15.46:0/1470807310 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c099be0 con 0x55ff6e63d890 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 -- 172.21.15.46:0/1470807310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 msgr2=0x55ff6e63dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f754c0a1840 tx=0x7f754c0997d0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 -- 172.21.15.46:0/1470807310 shutdown_connections 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63dcd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e643430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1470807310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e63d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 -- 172.21.15.46:0/1470807310 >> 172.21.15.46:0/1470807310 conn(0x55ff6e59b210 msgr2=0x55ff6e6382d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 -- 172.21.15.46:0/1470807310 shutdown_connections 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.218+0000 7f756f1dab80 1 -- 172.21.15.46:0/1470807310 wait complete. 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.219+0000 7f756f1dab80 1 -- start start 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.219+0000 7f756f1dab80 1 Processor -- start 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e6471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e647700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e529560 con 0x55ff6e643050 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e529200 con 0x55ff6e63d890 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e4106a0 con 0x55ff6e643970 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f756f1dab80 10 client.0 _ll_register_callbacks cb 0x55ff6e532b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:02.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7553e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e6471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7558c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7558c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52756/0 (socket says 172.21.15.46:52756) 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7558c00700 1 -- 172.21.15.46:0/3824422726 learned_addr learned my addr 172.21.15.46:0/3824422726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7559600700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e647700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7553e00700 1 -- 172.21.15.46:0/3824422726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 msgr2=0x55ff6e647700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7553e00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e647700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7553e00700 1 -- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 msgr2=0x55ff6e646c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.220+0000 7f7553e00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f7553e00700 1 -- 172.21.15.46:0/3824422726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75541a61c0 con 0x55ff6e643050 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f7558c00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f7559600700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e647700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f7553e00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e6471c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7540012930 tx=0x7f7540012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647f64843d44497c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f753a000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540017070 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7540004020 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6e4e6270 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75400041e0 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f756f1dab80 10 client.16890 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.221+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff6e3bab60 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff6e4e6270 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f754000f640 con 0x55ff6e643050 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 10 client.16890 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 20 client.16890 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6574b0 0x55ff6e6598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 4 client.16890 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6e60c810 con 0x55ff6e6574b0 2024-05-08T23:08:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.222+0000 7f7558c00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6574b0 0x55ff6e6598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.223+0000 7f7558c00700 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6574b0 0x55ff6e6598d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7b0a8bc7f9d008 server_cookie=6aee1024bb150dfa in_seq=0 out_seq=0 2024-05-08T23:08:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.223+0000 7f753a000700 10 client.16890 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f753a000700 1 -- 172.21.15.46:0/3824422726 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff6e60c810 con 0x55ff6e6574b0 2024-05-08T23:08:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f753a000700 10 client.16890 handle_command_reply: tid=0 2024-05-08T23:08:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 client.16890 shutdown 2024-05-08T23:08:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 msgr2=0x55ff6e6471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e6471c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7540012930 tx=0x7f7540012c40 comp rx=0 tx=0).stop 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 shutdown_connections 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e647700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e646c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6574b0 0x55ff6e6598d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3824422726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e6471c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 >> 172.21.15.46:0/3824422726 conn(0x55ff6e59b210 msgr2=0x55ff6e6380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f753a000700 0 client.16890 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.224+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 shutdown_connections 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.225+0000 7f756f1dab80 1 -- 172.21.15.46:0/3824422726 wait complete. 2024-05-08T23:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.225+0000 7f756f1dab80 20 client.16890 trim_cache size 0 max 16384 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.245+0000 7f756f1dab80 1 Processor -- start 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.245+0000 7f756f1dab80 1 -- start start 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.245+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.245+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e64e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.245+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e64e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f756f1dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e632a90 con 0x55ff6e643050 2024-05-08T23:08:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f756f1dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e648280 con 0x55ff6e643970 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f756f1dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e638ef0 con 0x55ff6e63d890 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42270/0 (socket says 172.21.15.46:42270) 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 -- 172.21.15.46:0/1669837622 learned_addr learned my addr 172.21.15.46:0/1669837622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7553e00700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e64e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7559600700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e64e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 -- 172.21.15.46:0/1669837622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 msgr2=0x55ff6e64e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e64e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 -- 172.21.15.46:0/1669837622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 msgr2=0x55ff6e64e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e64e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 -- 172.21.15.46:0/1669837622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6e553560 con 0x55ff6e63d890 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7553e00700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e64e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7559600700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.246+0000 7f7558c00700 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55ff6e65a1a0 tx=0x7f7548020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9101dbe3de3df96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.247+0000 7f7546600700 1 -- 172.21.15.46:0/1669837622 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548005960 con 0x55ff6e63d890 2024-05-08T23:08:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.247+0000 7f7546600700 1 -- 172.21.15.46:0/1669837622 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754801e040 con 0x55ff6e63d890 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.247+0000 7f7546600700 1 -- 172.21.15.46:0/1669837622 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548033890 con 0x55ff6e63d890 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.247+0000 7f756f1dab80 1 -- 172.21.15.46:0/1669837622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 msgr2=0x55ff6e63d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.247+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55ff6e65a1a0 tx=0x7f7548020000 comp rx=0 tx=0).stop 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 -- 172.21.15.46:0/1669837622 shutdown_connections 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e643970 0x55ff6e64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e63d890 0x55ff6e63d350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 --2- 172.21.15.46:0/1669837622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e64e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 -- 172.21.15.46:0/1669837622 >> 172.21.15.46:0/1669837622 conn(0x55ff6e59b210 msgr2=0x55ff6e636d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 -- 172.21.15.46:0/1669837622 shutdown_connections 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 -- 172.21.15.46:0/1669837622 wait complete. 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 -- start start 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.248+0000 7f756f1dab80 1 Processor -- start 2024-05-08T23:08:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e64e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e640420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e63ebd0 con 0x55ff6e643050 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e63ed40 con 0x55ff6e63d890 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e63ef00 con 0x55ff6e643970 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f756f1dab80 10 client.0 _ll_register_callbacks cb 0x55ff6e532b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f7553e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f7559600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e640420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f7558c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e64e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.249+0000 7f7553e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35512/0 (socket says 172.21.15.46:35512) 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 -- 172.21.15.46:0/3975134336 learned_addr learned my addr 172.21.15.46:0/3975134336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 -- 172.21.15.46:0/3975134336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 msgr2=0x55ff6e640420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e640420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 -- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 msgr2=0x55ff6e64e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e64e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 -- 172.21.15.46:0/3975134336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7554110600 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7558c00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f7553e00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ff6e634e90 tx=0x7f7540009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5078633d5d96816b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.250+0000 7f753a000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540017070 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f754c004ad0 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f754000fcc0 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7540005240 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f756f1dab80 10 client.16896 fetch_fsmap learned FSMap version 29 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff6e412cd0 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754c004ad0 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.251+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7540004b20 con 0x55ff6e643050 2024-05-08T23:08:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 10 client.16896 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 20 client.16896 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6b6d40 0x55ff6e6b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 4 client.16896 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ff6e585ec0 con 0x55ff6e6b6d40 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f7558c00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6b6d40 0x55ff6e6b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f7558c00700 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6b6d40 0x55ff6e6b9160 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a7d6c167697da2 server_cookie=5352e1ea7f91a95a in_seq=0 out_seq=0 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.252+0000 7f753a000700 10 client.16896 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f753a000700 1 -- 172.21.15.46:0/3975134336 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ff6e585ec0 con 0x55ff6e6b6d40 2024-05-08T23:08:02.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f753a000700 10 client.16896 handle_command_reply: tid=0 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 client.16896 shutdown 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 msgr2=0x55ff6e63fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ff6e634e90 tx=0x7f7540009aa0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 shutdown_connections 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6e643970 0x55ff6e640420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ff6e63d890 0x55ff6e64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ff6e6b6d40 0x55ff6e6b9160 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 --2- 172.21.15.46:0/3975134336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff6e643050 0x55ff6e63fce0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 >> 172.21.15.46:0/3975134336 conn(0x55ff6e59b210 msgr2=0x55ff6e645490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f753a000700 0 client.16896 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 shutdown_connections 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.253+0000 7f756f1dab80 1 -- 172.21.15.46:0/3975134336 wait complete. 2024-05-08T23:08:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f756f1dab80 20 client.16896 trim_cache size 0 max 16384 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "6a79e379-daf1-434e-9ad0-ec49122f0bc2": { 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6a79e379-daf1-434e-9ad0-ec49122f0bc2", 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:08:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:08:02.258 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:08:02.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 msgr2=0x7f753c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 0x7f753c0a3870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7548006010 tx=0x7f754800b260 comp rx=0 tx=0).stop 2024-05-08T23:08:02.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 msgr2=0x7f755419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:02.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 0x7f755419e040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f754c00efc0 tx=0x7f754c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 shutdown_connections 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755419dbc0 0x7f755419e040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f753c0a13b0 0x7f753c0a3870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75541030f0 0x7f75541992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 --2- 172.21.15.46:0/1138036854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7554108210 0x7f7554199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.254+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 >> 172.21.15.46:0/1138036854 conn(0x7f75540fe930 msgr2=0x7f7554100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.255+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 shutdown_connections 2024-05-08T23:08:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:02.255+0000 7f755a000700 1 -- 172.21.15.46:0/1138036854 wait complete. 2024-05-08T23:08:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:01 smithi053 ceph-mon[77587]: pgmap v129: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-05-08T23:08:02.273 INFO:tasks.cephfs.filesystem:scrub status for tag:6a79e379-daf1-434e-9ad0-ec49122f0bc2 - {'path': '/', 'tag': '6a79e379-daf1-434e-9ad0-ec49122f0bc2', 'options': 'recursive,force'} 2024-05-08T23:08:02.879 INFO:tasks.workunit.client.0.smithi046.stdout: 1 259520 10.86 MB/sec execute 24 sec latency 13.033 ms 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:08:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:03.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:03.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:08:03.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:03.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 261986 10.93 MB/sec execute 25 sec latency 14.886 ms 2024-05-08T23:08:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:03 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:03 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:03 smithi086 ceph-mon[78610]: pgmap v130: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-08T23:08:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:03 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:03 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:04.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:03 smithi053 ceph-mon[77587]: pgmap v130: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-08T23:08:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:03 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:03 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:03 smithi046 ceph-mon[36898]: pgmap v130: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-08T23:08:04.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 263750 10.84 MB/sec execute 26 sec latency 10.184 ms 2024-05-08T23:08:05.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 266315 11.06 MB/sec execute 27 sec latency 12.005 ms 2024-05-08T23:08:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:05 smithi086 ceph-mon[78610]: pgmap v131: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-05-08T23:08:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:05 smithi053 ceph-mon[77587]: pgmap v131: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-05-08T23:08:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:05 smithi046 ceph-mon[36898]: pgmap v131: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-05-08T23:08:06.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 268528 10.95 MB/sec execute 28 sec latency 9.193 ms 2024-05-08T23:08:07.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 270313 10.97 MB/sec execute 29 sec latency 21.547 ms 2024-05-08T23:08:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:07 smithi086 ceph-mon[78610]: pgmap v132: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:07 smithi053 ceph-mon[77587]: pgmap v132: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:08.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:07 smithi046 ceph-mon[36898]: pgmap v132: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:08.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 272939 11.08 MB/sec execute 30 sec latency 11.994 ms 2024-05-08T23:08:09.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 274604 11.04 MB/sec execute 31 sec latency 10.095 ms 2024-05-08T23:08:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:09 smithi086 ceph-mon[78610]: pgmap v133: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:09 smithi053 ceph-mon[77587]: pgmap v133: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:10.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:09 smithi046 ceph-mon[36898]: pgmap v133: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-05-08T23:08:10.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 276326 10.94 MB/sec execute 32 sec latency 373.229 ms 2024-05-08T23:08:11.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 278103 10.95 MB/sec execute 33 sec latency 9.366 ms 2024-05-08T23:08:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:11 smithi086 ceph-mon[78610]: pgmap v134: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-05-08T23:08:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:11 smithi053 ceph-mon[77587]: pgmap v134: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-05-08T23:08:12.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:11 smithi046 ceph-mon[36898]: pgmap v134: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-05-08T23:08:12.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 280474 11.03 MB/sec execute 34 sec latency 20.333 ms 2024-05-08T23:08:13.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 281120 10.74 MB/sec execute 35 sec latency 258.061 ms 2024-05-08T23:08:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:13 smithi086 ceph-mon[78610]: pgmap v135: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-08T23:08:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:13 smithi053 ceph-mon[77587]: pgmap v135: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-08T23:08:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:13 smithi046 ceph-mon[36898]: pgmap v135: 129 pgs: 129 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-08T23:08:14.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 283251 10.77 MB/sec execute 36 sec latency 30.979 ms 2024-05-08T23:08:15.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 284764 10.68 MB/sec execute 37 sec latency 10.002 ms 2024-05-08T23:08:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:15 smithi086 ceph-mon[78610]: pgmap v136: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-05-08T23:08:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:15 smithi053 ceph-mon[77587]: pgmap v136: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-05-08T23:08:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:15 smithi046 ceph-mon[36898]: pgmap v136: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-05-08T23:08:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:16.880 INFO:tasks.workunit.client.0.smithi046.stdout: 1 287469 10.85 MB/sec execute 38 sec latency 10.098 ms 2024-05-08T23:08:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:17.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:16 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:08:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:16 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:08:17.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:17.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:16 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:08:17.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 289668 10.78 MB/sec execute 39 sec latency 9.729 ms 2024-05-08T23:08:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:17 smithi086 ceph-mon[78610]: pgmap v137: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:17 smithi053 ceph-mon[77587]: pgmap v137: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:17 smithi046 ceph-mon[36898]: pgmap v137: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:18.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 291132 10.81 MB/sec execute 40 sec latency 194.567 ms 2024-05-08T23:08:19.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 292701 10.68 MB/sec execute 41 sec latency 24.359 ms 2024-05-08T23:08:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:19 smithi086 ceph-mon[78610]: pgmap v138: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:19 smithi053 ceph-mon[77587]: pgmap v138: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:20.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:19 smithi046 ceph-mon[36898]: pgmap v138: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-05-08T23:08:20.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 294861 10.76 MB/sec execute 42 sec latency 14.128 ms 2024-05-08T23:08:21.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 297330 10.78 MB/sec execute 43 sec latency 13.770 ms 2024-05-08T23:08:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:21 smithi086 ceph-mon[78610]: pgmap v139: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 364 op/s 2024-05-08T23:08:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:21 smithi053 ceph-mon[77587]: pgmap v139: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 364 op/s 2024-05-08T23:08:22.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:21 smithi046 ceph-mon[36898]: pgmap v139: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 364 op/s 2024-05-08T23:08:22.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 298917 10.72 MB/sec execute 44 sec latency 11.404 ms 2024-05-08T23:08:23.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 301615 10.86 MB/sec execute 45 sec latency 11.312 ms 2024-05-08T23:08:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:23 smithi086 ceph-mon[78610]: pgmap v140: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 279 op/s 2024-05-08T23:08:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:23 smithi053 ceph-mon[77587]: pgmap v140: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 279 op/s 2024-05-08T23:08:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:23 smithi046 ceph-mon[36898]: pgmap v140: 129 pgs: 129 active+clean; 192 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 279 op/s 2024-05-08T23:08:24.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 303333 10.77 MB/sec execute 46 sec latency 17.638 ms 2024-05-08T23:08:25.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 305456 10.83 MB/sec execute 47 sec latency 34.087 ms 2024-05-08T23:08:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:25 smithi086 ceph-mon[78610]: pgmap v141: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-05-08T23:08:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:25 smithi053 ceph-mon[77587]: pgmap v141: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-05-08T23:08:26.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:25 smithi046 ceph-mon[36898]: pgmap v141: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-05-08T23:08:26.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 307562 10.77 MB/sec execute 48 sec latency 16.857 ms 2024-05-08T23:08:27.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 309170 10.78 MB/sec execute 49 sec latency 20.267 ms 2024-05-08T23:08:28.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:27 smithi086 ceph-mon[78610]: pgmap v142: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:27 smithi053 ceph-mon[77587]: pgmap v142: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:27 smithi046 ceph-mon[36898]: pgmap v142: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:28.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 311589 10.82 MB/sec execute 50 sec latency 12.616 ms 2024-05-08T23:08:29.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 313022 10.75 MB/sec execute 51 sec latency 48.511 ms 2024-05-08T23:08:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:29 smithi086 ceph-mon[78610]: pgmap v143: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:29 smithi053 ceph-mon[77587]: pgmap v143: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:29 smithi046 ceph-mon[36898]: pgmap v143: 129 pgs: 129 active+clean; 176 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-05-08T23:08:30.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 315652 10.86 MB/sec execute 52 sec latency 12.168 ms 2024-05-08T23:08:31.881 INFO:tasks.workunit.client.0.smithi046.stdout: 1 316979 10.73 MB/sec execute 53 sec latency 11.684 ms 2024-05-08T23:08:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:31 smithi086 ceph-mon[78610]: pgmap v144: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-05-08T23:08:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:31 smithi053 ceph-mon[77587]: pgmap v144: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-05-08T23:08:32.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:08:32.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:31 smithi046 ceph-mon[36898]: pgmap v144: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 -- 172.21.15.46:0/3308213785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 msgr2=0x7f889410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 --2- 172.21.15.46:0/3308213785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f889410ada0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f88800098f0 tx=0x7f8880009c00 comp rx=0 tx=0).stop 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 -- 172.21.15.46:0/3308213785 shutdown_connections 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 --2- 172.21.15.46:0/3308213785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f889410b3c0 0x7f889410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 --2- 172.21.15.46:0/3308213785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101fa0 0x7f8894102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 --2- 172.21.15.46:0/3308213785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f889410ada0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 -- 172.21.15.46:0/3308213785 >> 172.21.15.46:0/3308213785 conn(0x7f88940fb760 msgr2=0x7f88940fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 -- 172.21.15.46:0/3308213785 shutdown_connections 2024-05-08T23:08:32.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.508+0000 7f889a600700 1 -- 172.21.15.46:0/3308213785 wait complete. 2024-05-08T23:08:32.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.510+0000 7f889a600700 1 Processor -- start 2024-05-08T23:08:32.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.510+0000 7f889a600700 1 -- start start 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.510+0000 7f889a600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8894101fa0 0x7f88940ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.510+0000 7f889a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f889a600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 0x7f8894100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f889a600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941a56d0 con 0x7f8894101fa0 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f889a600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941a5840 con 0x7f889410b3c0 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f889a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941a5a00 con 0x7f88941028c0 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8893400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8894101fa0 0x7f88940ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33886/0 (socket says 172.21.15.46:33886) 2024-05-08T23:08:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 -- 172.21.15.46:0/1750822438 learned_addr learned my addr 172.21.15.46:0/1750822438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8893e00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 0x7f8894100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 -- 172.21.15.46:0/1750822438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 msgr2=0x7f8894100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 0x7f8894100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 -- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8894101fa0 msgr2=0x7f88940ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8894101fa0 0x7f88940ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8892a00700 1 -- 172.21.15.46:0/1750822438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c009d00 con 0x7f88941028c0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.511+0000 7f8893e00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 0x7f8894100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f8892a00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f888000f950 tx=0x7f888000f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f888be00700 1 -- 172.21.15.46:0/1750822438 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888001c070 con 0x7f88941028c0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8880009520 con 0x7f88941028c0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f888be00700 1 -- 172.21.15.46:0/1750822438 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8880005040 con 0x7f88941028c0 2024-05-08T23:08:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f888be00700 1 -- 172.21.15.46:0/1750822438 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880004a90 con 0x7f88941028c0 2024-05-08T23:08:32.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.512+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88941a5fa0 con 0x7f88941028c0 2024-05-08T23:08:32.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.515+0000 7f888be00700 1 -- 172.21.15.46:0/1750822438 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f88800051b0 con 0x7f88941028c0 2024-05-08T23:08:32.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.518+0000 7f888be00700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 0x7f88780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.518+0000 7f88ae150b80 1 Processor -- start 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.518+0000 7f888be00700 1 -- 172.21.15.46:0/1750822438 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f88800ce240 con 0x7f88941028c0 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.518+0000 7f8893400700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 0x7f88780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.518+0000 7f88ae150b80 1 -- start start 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4719720 0x5640c4713350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4713890 0x5640c4713cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c466a150 con 0x5640c4719720 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c4667b10 con 0x5640c4718e00 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f88ae150b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c4663400 con 0x5640c4713890 2024-05-08T23:08:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8893400700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 0x7f88780a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8884009c60 tx=0x7f8884009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8893400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8892a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4719720 0x5640c4713350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8893400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47134/0 (socket says 172.21.15.46:47134) 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8893e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4713890 0x5640c4713cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.519+0000 7f8893400700 1 -- 172.21.15.46:0/1318749388 learned_addr learned my addr 172.21.15.46:0/1318749388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 -- 172.21.15.46:0/1318749388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4713890 msgr2=0x5640c4713cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4713890 0x5640c4713cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 -- 172.21.15.46:0/1318749388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4719720 msgr2=0x5640c4713350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4719720 0x5640c4713350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 -- 172.21.15.46:0/1318749388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c44e43b0 con 0x5640c4718e00 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8893400700 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f888400e4e0 tx=0x7f8884014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0df3c35dc644fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8888c00700 1 -- 172.21.15.46:0/1318749388 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888400ee00 con 0x5640c4718e00 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8888c00700 1 -- 172.21.15.46:0/1318749388 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884021070 con 0x5640c4718e00 2024-05-08T23:08:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.520+0000 7f8888c00700 1 -- 172.21.15.46:0/1318749388 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884025740 con 0x5640c4718e00 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 -- 172.21.15.46:0/1318749388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 msgr2=0x5640c47191e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f888400e4e0 tx=0x7f8884014000 comp rx=0 tx=0).stop 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 -- 172.21.15.46:0/1318749388 shutdown_connections 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4713890 0x5640c4713cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47191e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1318749388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4719720 0x5640c4713350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 -- 172.21.15.46:0/1318749388 >> 172.21.15.46:0/1318749388 conn(0x5640c4671210 msgr2=0x5640c470ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 -- 172.21.15.46:0/1318749388 shutdown_connections 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.521+0000 7f88ae150b80 1 -- 172.21.15.46:0/1318749388 wait complete. 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.522+0000 7f88ae150b80 1 -- start start 2024-05-08T23:08:32.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.522+0000 7f88ae150b80 1 Processor -- start 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c471d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c471d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c45ff560 con 0x5640c4713890 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c45ff200 con 0x5640c4718e00 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c44e66a0 con 0x5640c4719720 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f88ae150b80 10 client.0 _ll_register_callbacks cb 0x5640c4608b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42738/0 (socket says 172.21.15.46:42738) 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 -- 172.21.15.46:0/3560935193 learned_addr learned my addr 172.21.15.46:0/3560935193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893e00700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c471d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8892a00700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c471d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 -- 172.21.15.46:0/3560935193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 msgr2=0x5640c471d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c471d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 -- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 msgr2=0x5640c471d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.523+0000 7f8893400700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c471d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f8893400700 1 -- 172.21.15.46:0/3560935193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8884017070 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f8893e00700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c471d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f8893400700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f888400e4b0 tx=0x7f888400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fe791f1b8b3275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f887d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884005260 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f88840053c0 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.524+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640c45bc270 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88840055c0 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 10 client.16908 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5640c4490b60 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640c45bc270 con 0x5640c4713890 2024-05-08T23:08:32.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f888401d820 con 0x5640c4713890 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 20 client.16908 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c472d0c0 0x5640c472f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 4 client.16908 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.525+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640c46e2810 con 0x5640c472d0c0 2024-05-08T23:08:32.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.526+0000 7f8892a00700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c472d0c0 0x5640c472f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.526+0000 7f887d600700 10 client.16908 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.526+0000 7f8892a00700 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c472d0c0 0x5640c472f4b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b40e2a174d4194 server_cookie=c011a1e78af598ec in_seq=0 out_seq=0 2024-05-08T23:08:32.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f887d600700 1 -- 172.21.15.46:0/3560935193 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640c46e2810 con 0x5640c472d0c0 2024-05-08T23:08:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f887d600700 10 client.16908 handle_command_reply: tid=0 2024-05-08T23:08:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 client.16908 shutdown 2024-05-08T23:08:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 msgr2=0x5640c471cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f888400e4b0 tx=0x7f888400ed10 comp rx=0 tx=0).stop 2024-05-08T23:08:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 shutdown_connections 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c471d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c472d0c0 0x5640c472f4b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c471cb50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 --2- 172.21.15.46:0/3560935193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c471d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 >> 172.21.15.46:0/3560935193 conn(0x5640c4671210 msgr2=0x5640c470d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.527+0000 7f887d600700 0 client.16908 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.528+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 shutdown_connections 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.528+0000 7f88ae150b80 1 -- 172.21.15.46:0/3560935193 wait complete. 2024-05-08T23:08:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.528+0000 7f88ae150b80 20 client.16908 trim_cache size 0 max 16384 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.547+0000 7f88ae150b80 1 Processor -- start 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.547+0000 7f88ae150b80 1 -- start start 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4713890 0x5640c4713350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 0x5640c47184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c471e100 con 0x5640c4718e00 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c4706040 con 0x5640c4713890 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f88ae150b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c470adc0 con 0x5640c4719720 2024-05-08T23:08:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f8893e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f8893e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33912/0 (socket says 172.21.15.46:33912) 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f8893e00700 1 -- 172.21.15.46:0/1669787187 learned_addr learned my addr 172.21.15.46:0/1669787187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f8892a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 0x5640c47184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.548+0000 7f8893400700 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4713890 0x5640c4713350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 -- 172.21.15.46:0/1669787187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4713890 msgr2=0x5640c4713350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4713890 0x5640c4713350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 -- 172.21.15.46:0/1669787187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 msgr2=0x5640c47184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 0x5640c47184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 -- 172.21.15.46:0/1669787187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c4713f40 con 0x5640c4719720 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8892a00700 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 0x5640c47184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8893e00700 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f888c007a60 tx=0x7f888c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c25e1e9dd8e2c9e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8889600700 1 -- 172.21.15.46:0/1669787187 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c014ac0 con 0x5640c4719720 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8889600700 1 -- 172.21.15.46:0/1669787187 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f888c011040 con 0x5640c4719720 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.549+0000 7f8889600700 1 -- 172.21.15.46:0/1669787187 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c018420 con 0x5640c4719720 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 -- 172.21.15.46:0/1669787187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 msgr2=0x5640c470dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f888c007a60 tx=0x7f888c00b040 comp rx=0 tx=0).stop 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 -- 172.21.15.46:0/1669787187 shutdown_connections 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c470dd20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4713890 0x5640c4713350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1669787187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4718e00 0x5640c47184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 -- 172.21.15.46:0/1669787187 >> 172.21.15.46:0/1669787187 conn(0x5640c4671210 msgr2=0x5640c471b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 -- 172.21.15.46:0/1669787187 shutdown_connections 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.550+0000 7f88ae150b80 1 -- 172.21.15.46:0/1669787187 wait complete. 2024-05-08T23:08:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.551+0000 7f88ae150b80 1 -- start start 2024-05-08T23:08:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.551+0000 7f88ae150b80 1 Processor -- start 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c4724310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c4722bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c4723130 con 0x5640c4713890 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c47232a0 con 0x5640c4718e00 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c4723460 con 0x5640c4719720 2024-05-08T23:08:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f88ae150b80 10 client.0 _ll_register_callbacks cb 0x5640c4608b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c4724310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c4722bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c4724310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42742/0 (socket says 172.21.15.46:42742) 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893400700 1 -- 172.21.15.46:0/1448838231 learned_addr learned my addr 172.21.15.46:0/1448838231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8892a00700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893e00700 1 -- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 msgr2=0x5640c47226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.552+0000 7f8893e00700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f8893e00700 1 -- 172.21.15.46:0/1448838231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 msgr2=0x5640c4724310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f8893e00700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c4724310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f8893e00700 1 -- 172.21.15.46:0/1448838231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c007710 con 0x5640c4719720 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f8893e00700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c4722bf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f888c009a20 tx=0x7f888c00afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7fc4f422b816f2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f887d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c01a880 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8884021070 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f888c0148c0 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888c014aa0 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f88ae150b80 10 client.26962 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.553+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5640c4629560 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884021070 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f888c016040 con 0x5640c4719720 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 10 client.26962 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 10 client.26962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 20 client.26962 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c478d1a0 0x5640c478f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 4 client.26962 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640c465bec0 con 0x5640c478d1a0 2024-05-08T23:08:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.554+0000 7f8893400700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c478d1a0 0x5640c478f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.555+0000 7f8893400700 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c478d1a0 0x5640c478f590 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd39fde29fb20008 server_cookie=28ed7d7cdbdda8c4 in_seq=0 out_seq=0 2024-05-08T23:08:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.555+0000 7f887d600700 10 client.26962 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.555+0000 7f887d600700 1 -- 172.21.15.46:0/1448838231 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5640c465bec0 con 0x5640c478d1a0 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.555+0000 7f887d600700 10 client.26962 handle_command_reply: tid=0 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.555+0000 7f88ae150b80 1 client.26962 shutdown 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 msgr2=0x5640c4722bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c4722bf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f888c009a20 tx=0x7f888c00afd0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 shutdown_connections 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c4719720 0x5640c4722bf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5640c478d1a0 0x5640c478f590 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640c4713890 0x5640c4724310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 --2- 172.21.15.46:0/1448838231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c4718e00 0x5640c47226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f887d600700 0 client.26962 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 >> 172.21.15.46:0/1448838231 conn(0x5640c4671210 msgr2=0x5640c471d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 shutdown_connections 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 1 -- 172.21.15.46:0/1448838231 wait complete. 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.556+0000 7f88ae150b80 20 client.26962 trim_cache size 0 max 16384 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:08:32.560 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 msgr2=0x7f88780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 0x7f88780a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8884009c60 tx=0x7f8884009260 comp rx=0 tx=0).stop 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 msgr2=0x7f8894100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f888000f950 tx=0x7f888000f980 comp rx=0 tx=0).stop 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 shutdown_connections 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f889410b3c0 0x7f8894100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f88780a1290 0x7f88780a3750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8894101fa0 0x7f88940ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 --2- 172.21.15.46:0/1750822438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88941028c0 0x7f8894100130 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 >> 172.21.15.46:0/1750822438 conn(0x7f88940fb760 msgr2=0x7f88940fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 shutdown_connections 2024-05-08T23:08:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.557+0000 7f889a600700 1 -- 172.21.15.46:0/1750822438 wait complete. 2024-05-08T23:08:32.578 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:08:32.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.801+0000 7ff362800700 1 -- 172.21.15.46:0/2791229043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 msgr2=0x7ff35c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.801+0000 7ff362800700 1 --2- 172.21.15.46:0/2791229043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c101450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009990 tx=0x7ff34c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 -- 172.21.15.46:0/2791229043 shutdown_connections 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 --2- 172.21.15.46:0/2791229043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c106190 0x7ff35c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 --2- 172.21.15.46:0/2791229043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c101450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 --2- 172.21.15.46:0/2791229043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c101a20 0x7ff35c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 -- 172.21.15.46:0/2791229043 >> 172.21.15.46:0/2791229043 conn(0x7ff35c0fc930 msgr2=0x7ff35c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 -- 172.21.15.46:0/2791229043 shutdown_connections 2024-05-08T23:08:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.802+0000 7ff362800700 1 -- 172.21.15.46:0/2791229043 wait complete. 2024-05-08T23:08:32.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 Processor -- start 2024-05-08T23:08:32.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 -- start start 2024-05-08T23:08:32.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 0x7ff35c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 0x7ff35c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19e8a0 con 0x7ff35c106190 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19ea10 con 0x7ff35c101070 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.803+0000 7ff362800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c19ebd0 con 0x7ff35c19de20 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47162/0 (socket says 172.21.15.46:47162) 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 0x7ff35c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 -- 172.21.15.46:0/2243026726 learned_addr learned my addr 172.21.15.46:0/2243026726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35aa00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 0x7ff35c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 -- 172.21.15.46:0/2243026726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 msgr2=0x7ff35c19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 0x7ff35c19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 -- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 msgr2=0x7ff35c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 0x7ff35c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 -- 172.21.15.46:0/2243026726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009520 con 0x7ff35c101070 2024-05-08T23:08:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35aa00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 0x7ff35c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35be00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 0x7ff35c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.804+0000 7ff35b400700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005910 tx=0x7ff34c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.805+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c01d070 con 0x7ff35c101070 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.805+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34c022470 con 0x7ff35c101070 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.805+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c0095f0 con 0x7ff35c101070 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.805+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c017650 con 0x7ff35c101070 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.805+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c1a63c0 con 0x7ff35c101070 2024-05-08T23:08:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.806+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35c102540 con 0x7ff35c101070 2024-05-08T23:08:32.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.808+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff34c0225e0 con 0x7ff35c101070 2024-05-08T23:08:32.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.809+0000 7ff353e00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 0x7ff3400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:32.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.810+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff34c0ce700 con 0x7ff35c101070 2024-05-08T23:08:32.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.810+0000 7ff35aa00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 0x7ff3400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:32.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.810+0000 7ff35aa00700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 0x7ff3400a3750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff35c19aa90 tx=0x7ff348009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:32.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:32.812+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff34c0977d0 con 0x7ff35c101070 2024-05-08T23:08:32.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 319491 10.84 MB/sec execute 54 sec latency 14.850 ms 2024-05-08T23:08:33.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.115+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff35c19a8e0 con 0x7ff35c101070 2024-05-08T23:08:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.116+0000 7ff353e00700 1 -- 172.21.15.46:0/2243026726 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7ff34c09adf0 con 0x7ff35c101070 2024-05-08T23:08:33.116 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:08:33.117 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:08:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 msgr2=0x7ff3400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 0x7ff3400a3750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff35c19aa90 tx=0x7ff348009260 comp rx=0 tx=0).stop 2024-05-08T23:08:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 msgr2=0x7ff35c199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005910 tx=0x7ff34c004730 comp rx=0 tx=0).stop 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 shutdown_connections 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff35c19de20 0x7ff35c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff35c101070 0x7ff35c199590 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff3400a1290 0x7ff3400a3750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 --2- 172.21.15.46:0/2243026726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff35c106190 0x7ff35c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 >> 172.21.15.46:0/2243026726 conn(0x7ff35c0fc930 msgr2=0x7ff35c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 shutdown_connections 2024-05-08T23:08:33.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.119+0000 7ff362800700 1 -- 172.21.15.46:0/2243026726 wait complete. 2024-05-08T23:08:33.121 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:08:33.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 -- 172.21.15.46:0/450890824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 msgr2=0x7fab2c10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 --2- 172.21.15.46:0/450890824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c10ada0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fab18009990 tx=0x7fab18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 -- 172.21.15.46:0/450890824 shutdown_connections 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 --2- 172.21.15.46:0/450890824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c10b3c0 0x7fab2c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 --2- 172.21.15.46:0/450890824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c101fa0 0x7fab2c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 --2- 172.21.15.46:0/450890824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c10ada0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.378+0000 7fab33200700 1 -- 172.21.15.46:0/450890824 >> 172.21.15.46:0/450890824 conn(0x7fab2c0fb760 msgr2=0x7fab2c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.379+0000 7fab33200700 1 -- 172.21.15.46:0/450890824 shutdown_connections 2024-05-08T23:08:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.379+0000 7fab33200700 1 -- 172.21.15.46:0/450890824 wait complete. 2024-05-08T23:08:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 Processor -- start 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 -- start start 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 0x7fab2c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c10b3c0 0x7fab2c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1a5680 con 0x7fab2c10b3c0 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1a57f0 con 0x7fab2c101fa0 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab33200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1a59b0 con 0x7fab2c1028c0 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 0x7fab2c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33974/0 (socket says 172.21.15.46:33974) 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.380+0000 7fab2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 0x7fab2c0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47184/0 (socket says 172.21.15.46:47184) 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 -- 172.21.15.46:0/1786906928 learned_addr learned my addr 172.21.15.46:0/1786906928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab31200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c10b3c0 0x7fab2c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 -- 172.21.15.46:0/1786906928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 msgr2=0x7fab2c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 0x7fab2c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 -- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c10b3c0 msgr2=0x7fab2c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c10b3c0 0x7fab2c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 -- 172.21.15.46:0/1786906928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180095f0 con 0x7fab2c1028c0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab2b400700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab18005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab28c00700 1 -- 172.21.15.46:0/1786906928 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1801d070 con 0x7fab2c1028c0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab28c00700 1 -- 172.21.15.46:0/1786906928 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab18022470 con 0x7fab2c1028c0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.381+0000 7fab28c00700 1 -- 172.21.15.46:0/1786906928 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800bee0 con 0x7fab2c1028c0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.382+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1a5c30 con 0x7fab2c1028c0 2024-05-08T23:08:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.382+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1a6090 con 0x7fab2c1028c0 2024-05-08T23:08:33.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.385+0000 7fab28c00700 1 -- 172.21.15.46:0/1786906928 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab180225e0 con 0x7fab2c1028c0 2024-05-08T23:08:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.386+0000 7fab28c00700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 0x7fab100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.386+0000 7fab28c00700 1 -- 172.21.15.46:0/1786906928 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab180ce300 con 0x7fab2c1028c0 2024-05-08T23:08:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.386+0000 7fab2be00700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 0x7fab100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.386+0000 7fab46c27b80 1 Processor -- start 2024-05-08T23:08:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 -- start start 2024-05-08T23:08:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab2be00700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 0x7fab100a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009c60 tx=0x7fab1c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d42acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d381150 con 0x55637d42a890 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d37eb10 con 0x55637d42fe00 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab46c27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d37a400 con 0x55637d430720 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33986/0 (socket says 172.21.15.46:33986) 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab2b400700 1 -- 172.21.15.46:0/1402087787 learned_addr learned my addr 172.21.15.46:0/1402087787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.387+0000 7fab31200700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d42acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 -- 172.21.15.46:0/1402087787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 msgr2=0x55637d4301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 -- 172.21.15.46:0/1402087787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 msgr2=0x55637d42acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d42acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 -- 172.21.15.46:0/1402087787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637d1fb3b0 con 0x55637d430720 2024-05-08T23:08:33.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab31200700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d42acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2be00700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab2b400700 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fab180976b0 tx=0x7fab1809b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489d90261e506e11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab21600700 1 -- 172.21.15.46:0/1402087787 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1809b1b0 con 0x55637d430720 2024-05-08T23:08:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab21600700 1 -- 172.21.15.46:0/1402087787 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1809efe0 con 0x55637d430720 2024-05-08T23:08:33.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.388+0000 7fab21600700 1 -- 172.21.15.46:0/1402087787 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab180229e0 con 0x55637d430720 2024-05-08T23:08:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 -- 172.21.15.46:0/1402087787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 msgr2=0x55637d42a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fab180976b0 tx=0x7fab1809b000 comp rx=0 tx=0).stop 2024-05-08T23:08:33.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 -- 172.21.15.46:0/1402087787 shutdown_connections 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d42acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1402087787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d42a350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 -- 172.21.15.46:0/1402087787 >> 172.21.15.46:0/1402087787 conn(0x55637d388210 msgr2=0x55637d424df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 -- 172.21.15.46:0/1402087787 shutdown_connections 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.389+0000 7fab46c27b80 1 -- 172.21.15.46:0/1402087787 wait complete. 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.390+0000 7fab46c27b80 1 -- start start 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.390+0000 7fab46c27b80 1 Processor -- start 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 0x55637d436730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d430720 0x55637d434220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d316560 con 0x55637d430720 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d316200 con 0x55637d42a890 2024-05-08T23:08:33.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d1fd6a0 con 0x55637d42fe00 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab46c27b80 10 client.0 _ll_register_callbacks cb 0x55637d31fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 0x55637d436730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab31200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d430720 0x55637d434220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34002/0 (socket says 172.21.15.46:34002) 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 0x55637d436730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47214/0 (socket says 172.21.15.46:47214) 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.391+0000 7fab2b400700 1 -- 172.21.15.46:0/1139552495 learned_addr learned my addr 172.21.15.46:0/1139552495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 -- 172.21.15.46:0/1139552495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 msgr2=0x55637d436730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 0x55637d436730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 -- 172.21.15.46:0/1139552495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d430720 msgr2=0x55637d434220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d430720 0x55637d434220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 -- 172.21.15.46:0/1139552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c017070 con 0x55637d42fe00 2024-05-08T23:08:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab2b400700 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab180a3070 tx=0x7fab18022b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b10a43fb708e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab16000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1809b1b0 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab1809b410 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55637d2d3270 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1809b5d0 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab46c27b80 10 client.26980 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fab2c1a6090 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.392+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab180a7020 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab180a3110 con 0x55637d42fe00 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 10 client.26980 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 10 client.26980 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 20 client.26980 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d4439f0 0x55637d445de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 4 client.26980 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55637d3f9810 con 0x55637d4439f0 2024-05-08T23:08:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab2be00700 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d4439f0 0x55637d445de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab16000700 10 client.26980 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:08:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.393+0000 7fab2be00700 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d4439f0 0x55637d445de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9d4ad688c396583 server_cookie=bb3f141b6788afb5 in_seq=0 out_seq=0 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab16000700 1 -- 172.21.15.46:0/1139552495 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55637d3f9810 con 0x55637d4439f0 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab16000700 10 client.26980 handle_command_reply: tid=0 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 client.26980 shutdown 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 msgr2=0x55637d433ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab180a3070 tx=0x7fab18022b90 comp rx=0 tx=0).stop 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 shutdown_connections 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d4439f0 0x55637d445de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d430720 0x55637d434220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab16000700 0 client.26980 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42a890 0x55637d436730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1139552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d42fe00 0x55637d433ae0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.394+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 >> 172.21.15.46:0/1139552495 conn(0x55637d388210 msgr2=0x55637d4245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.395+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 shutdown_connections 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.395+0000 7fab46c27b80 1 -- 172.21.15.46:0/1139552495 wait complete. 2024-05-08T23:08:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.395+0000 7fab46c27b80 20 client.26980 trim_cache size 0 max 16384 2024-05-08T23:08:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 Processor -- start 2024-05-08T23:08:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 -- start start 2024-05-08T23:08:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d424d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d43aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d434f20 con 0x55637d42a890 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d421dc0 con 0x55637d42fe00 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.412+0000 7fab46c27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d42a6d0 con 0x55637d430720 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42804/0 (socket says 172.21.15.46:42804) 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 -- 172.21.15.46:0/1211512523 learned_addr learned my addr 172.21.15.46:0/1211512523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab31200700 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d43aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2b400700 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d424d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 -- 172.21.15.46:0/1211512523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 msgr2=0x55637d43aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d43aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 -- 172.21.15.46:0/1211512523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 msgr2=0x55637d424d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d424d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 -- 172.21.15.46:0/1211512523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637d1a7b60 con 0x55637d42a890 2024-05-08T23:08:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.413+0000 7fab2be00700 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fab1c013010 tx=0x7fab1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66570a0f1f50468e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.414+0000 7fab22000700 1 -- 172.21.15.46:0/1211512523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c022030 con 0x55637d42a890 2024-05-08T23:08:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.414+0000 7fab22000700 1 -- 172.21.15.46:0/1211512523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c023040 con 0x55637d42a890 2024-05-08T23:08:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.414+0000 7fab22000700 1 -- 172.21.15.46:0/1211512523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c01d8d0 con 0x55637d42a890 2024-05-08T23:08:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- 172.21.15.46:0/1211512523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 msgr2=0x55637d434220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fab1c013010 tx=0x7fab1c014000 comp rx=0 tx=0).stop 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- 172.21.15.46:0/1211512523 shutdown_connections 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d43aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d434220 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 --2- 172.21.15.46:0/1211512523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d424d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- 172.21.15.46:0/1211512523 >> 172.21.15.46:0/1211512523 conn(0x55637d388210 msgr2=0x55637d44a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- 172.21.15.46:0/1211512523 shutdown_connections 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- 172.21.15.46:0/1211512523 wait complete. 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.415+0000 7fab46c27b80 1 -- start start 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 Processor -- start 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d436eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d498eb0 con 0x55637d42a890 2024-05-08T23:08:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d498ff0 con 0x55637d42fe00 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637d4991b0 con 0x55637d430720 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab46c27b80 10 client.0 _ll_register_callbacks cb 0x55637d31fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab2b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab2be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d436eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab31200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34016/0 (socket says 172.21.15.46:34016) 2024-05-08T23:08:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.416+0000 7fab31200700 1 -- 172.21.15.46:0/2185959334 learned_addr learned my addr 172.21.15.46:0/2185959334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 -- 172.21.15.46:0/2185959334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 msgr2=0x55637d4373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 -- 172.21.15.46:0/2185959334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 msgr2=0x55637d436eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d436eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 -- 172.21.15.46:0/2185959334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c004400 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab2be00700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d436eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab31200700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab24007f00 tx=0x7fab24007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=248b8d4179a00238 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab16000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e040 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fab24018d00 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab1809efe0 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab24003a00 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab46c27b80 10 client.26986 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.417+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55637d340560 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1809efe0 con 0x55637d430720 2024-05-08T23:08:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab24021480 con 0x55637d430720 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 10 client.26986 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 10 client.26986 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 20 client.26986 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d49aa90 0x55637d49ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 4 client.26986 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55637d372ec0 con 0x55637d49aa90 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.418+0000 7fab2be00700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d49aa90 0x55637d49ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab2be00700 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d49aa90 0x55637d49ceb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a61ed019c98a06c server_cookie=893479c4037cd6f0 in_seq=0 out_seq=0 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab16000700 10 client.26986 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab16000700 1 -- 172.21.15.46:0/2185959334 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55637d372ec0 con 0x55637d49aa90 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab16000700 10 client.26986 handle_command_reply: tid=0 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 client.26986 shutdown 2024-05-08T23:08:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 msgr2=0x55637d437930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fab24007f00 tx=0x7fab24007f30 comp rx=0 tx=0).stop 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 shutdown_connections 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55637d49aa90 0x55637d49ceb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55637d430720 0x55637d437930 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637d42a890 0x55637d436eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 --2- 172.21.15.46:0/2185959334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55637d42fe00 0x55637d4373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab16000700 0 client.26986 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 >> 172.21.15.46:0/2185959334 conn(0x55637d388210 msgr2=0x55637d42dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 shutdown_connections 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.419+0000 7fab46c27b80 1 -- 172.21.15.46:0/2185959334 wait complete. 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab46c27b80 20 client.26986 trim_cache size 0 max 16384 2024-05-08T23:08:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:33.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 msgr2=0x7fab100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 0x7fab100a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009c60 tx=0x7fab1c009260 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 msgr2=0x7fab2c100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab18000c00 tx=0x7fab18005740 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 shutdown_connections 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fab100a1310 0x7fab100a37d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab2c10b3c0 0x7fab2c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab2c101fa0 0x7fab2c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 --2- 172.21.15.46:0/1786906928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab2c1028c0 0x7fab2c100130 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.420+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 >> 172.21.15.46:0/1786906928 conn(0x7fab2c0fb760 msgr2=0x7fab2c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.421+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 shutdown_connections 2024-05-08T23:08:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.421+0000 7fab33200700 1 -- 172.21.15.46:0/1786906928 wait complete. 2024-05-08T23:08:33.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:08:33.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 -- 172.21.15.46:0/2037514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c101880 msgr2=0x7f8b4c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 --2- 172.21.15.46:0/2037514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c101880 0x7f8b4c1058c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009990 tx=0x7f8b38009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 -- 172.21.15.46:0/2037514868 shutdown_connections 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 --2- 172.21.15.46:0/2037514868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 --2- 172.21.15.46:0/2037514868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 --2- 172.21.15.46:0/2037514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c101880 0x7f8b4c1058c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 -- 172.21.15.46:0/2037514868 >> 172.21.15.46:0/2037514868 conn(0x7f8b4c0fc7b0 msgr2=0x7f8b4c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 -- 172.21.15.46:0/2037514868 shutdown_connections 2024-05-08T23:08:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.672+0000 7f8b4b200700 1 -- 172.21.15.46:0/2037514868 wait complete. 2024-05-08T23:08:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 Processor -- start 2024-05-08T23:08:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 -- start start 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c19b950 0x7f8b4c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19c3d0 con 0x7f8b4c19b950 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19c540 con 0x7f8b4c100ed0 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.674+0000 7f8b4b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c19c700 con 0x7f8b4c105ff0 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47236/0 (socket says 172.21.15.46:47236) 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b4a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c19b950 0x7f8b4c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49e00700 1 -- 172.21.15.46:0/4248218225 learned_addr learned my addr 172.21.15.46:0/4248218225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 -- 172.21.15.46:0/4248218225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 msgr2=0x7f8b4c197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 -- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c19b950 msgr2=0x7f8b4c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c19b950 0x7f8b4c19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49400700 1 -- 172.21.15.46:0/4248218225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380095f0 con 0x7f8b4c105ff0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.675+0000 7f8b49e00700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b49400700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c197670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b42a00700 1 -- 172.21.15.46:0/4248218225 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3801d070 con 0x7f8b4c105ff0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b42a00700 1 -- 172.21.15.46:0/4248218225 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b38021410 con 0x7f8b4c105ff0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c1a3c00 con 0x7f8b4c105ff0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b42a00700 1 -- 172.21.15.46:0/4248218225 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38017650 con 0x7f8b4c105ff0 2024-05-08T23:08:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.676+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c1a4080 con 0x7f8b4c105ff0 2024-05-08T23:08:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.679+0000 7f8b42a00700 1 -- 172.21.15.46:0/4248218225 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8b38021840 con 0x7f8b4c105ff0 2024-05-08T23:08:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.680+0000 7f8b42a00700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 0x7f8b300a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.680+0000 7f8b64da1b80 1 Processor -- start 2024-05-08T23:08:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.680+0000 7f8b42a00700 1 -- 172.21.15.46:0/4248218225 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8b380ce1e0 con 0x7f8b4c105ff0 2024-05-08T23:08:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 -- start start 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x562440856850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x56244085d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624407a3150 con 0x562440856470 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624407a0b10 con 0x562440851120 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244079c400 con 0x562440850760 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b49e00700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 0x7f8b300a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b49400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b49400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34054/0 (socket says 172.21.15.46:34054) 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.681+0000 7f8b49400700 1 -- 172.21.15.46:0/330157374 learned_addr learned my addr 172.21.15.46:0/330157374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b4a800700 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x56244085d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 -- 172.21.15.46:0/330157374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 msgr2=0x56244085d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x56244085d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 -- 172.21.15.46:0/330157374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 msgr2=0x562440856850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x562440856850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 -- 172.21.15.46:0/330157374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244061d3b0 con 0x562440850760 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b4a800700 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x56244085d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b49400700 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a2e30 tx=0x7f8b3809ad40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a6f162d7cadf5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b37400700 1 -- 172.21.15.46:0/330157374 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3809a8d0 con 0x562440850760 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b37400700 1 -- 172.21.15.46:0/330157374 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3809b670 con 0x562440850760 2024-05-08T23:08:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.682+0000 7f8b37400700 1 -- 172.21.15.46:0/330157374 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3809b430 con 0x562440850760 2024-05-08T23:08:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 -- 172.21.15.46:0/330157374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 msgr2=0x562440850be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b49e00700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 0x7f8b300a3870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c011680 tx=0x7f8b3c009e50 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a2e30 tx=0x7f8b3809ad40 comp rx=0 tx=0).stop 2024-05-08T23:08:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 -- 172.21.15.46:0/330157374 shutdown_connections 2024-05-08T23:08:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x56244085d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x562440856850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/330157374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440850be0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.683+0000 7f8b64da1b80 1 -- 172.21.15.46:0/330157374 >> 172.21.15.46:0/330157374 conn(0x5624407aa210 msgr2=0x5624408471a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.684+0000 7f8b64da1b80 1 -- 172.21.15.46:0/330157374 shutdown_connections 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.684+0000 7f8b64da1b80 1 -- 172.21.15.46:0/330157374 wait complete. 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.684+0000 7f8b64da1b80 1 -- start start 2024-05-08T23:08:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.684+0000 7f8b64da1b80 1 Processor -- start 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x562440860350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x56244085a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440738560 con 0x562440856470 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440738200 con 0x562440851120 2024-05-08T23:08:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244061f6a0 con 0x562440850760 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b64da1b80 10 client.0 _ll_register_callbacks cb 0x562440741b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34064/0 (socket says 172.21.15.46:34064) 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b49e00700 1 -- 172.21.15.46:0/1285258283 learned_addr learned my addr 172.21.15.46:0/1285258283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b49400700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x562440860350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.685+0000 7f8b4a800700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x56244085a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 -- 172.21.15.46:0/1285258283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 msgr2=0x562440860350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x562440860350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 -- 172.21.15.46:0/1285258283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 msgr2=0x56244085a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x56244085a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 -- 172.21.15.46:0/1285258283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c1a4080 con 0x562440850760 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0092a0 tx=0x7f8b3c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ed92f44ae3637f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b2be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c01c8a0 con 0x562440850760 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8b3c01ca00 con 0x562440850760 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.686+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624406f5270 con 0x562440850760 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3c01cbc0 con 0x562440850760 2024-05-08T23:08:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 10 client.26998 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5624405c9b60 con 0x562440850760 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624406f5270 con 0x562440850760 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b3c00e3b0 con 0x562440850760 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 10 client.26998 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 10 client.26998 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 20 client.26998 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56244086a3c0 0x56244086c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 4 client.26998 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56244081b810 con 0x56244086a3c0 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.687+0000 7f8b49400700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56244086a3c0 0x56244086c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.688+0000 7f8b49400700 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56244086a3c0 0x56244086c7e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac1755e1afb3f565 server_cookie=4216491381d76486 in_seq=0 out_seq=0 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.688+0000 7f8b2be00700 10 client.26998 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.688+0000 7f8b2be00700 1 -- 172.21.15.46:0/1285258283 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56244081b810 con 0x56244086a3c0 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.688+0000 7f8b2be00700 10 client.26998 handle_command_reply: tid=0 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.688+0000 7f8b64da1b80 1 client.26998 shutdown 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 msgr2=0x56244085fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0092a0 tx=0x7f8b3c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 shutdown_connections 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56244086a3c0 0x56244086c7e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440856470 0x56244085a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x56244085fe10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1285258283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440851120 0x562440860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b2be00700 0 client.26998 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 >> 172.21.15.46:0/1285258283 conn(0x5624407aa210 msgr2=0x562440847040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 shutdown_connections 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1285258283 wait complete. 2024-05-08T23:08:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.689+0000 7f8b64da1b80 20 client.26998 trim_cache size 0 max 16384 2024-05-08T23:08:33.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.706+0000 7f8b64da1b80 1 Processor -- start 2024-05-08T23:08:33.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 -- start start 2024-05-08T23:08:33.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x562440858f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 0x56244085fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440867ac0 con 0x562440851120 2024-05-08T23:08:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440856ed0 con 0x562440850760 2024-05-08T23:08:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562440843dc0 con 0x562440856470 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47264/0 (socket says 172.21.15.46:47264) 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.707+0000 7f8b49e00700 1 -- 172.21.15.46:0/1542734328 learned_addr learned my addr 172.21.15.46:0/1542734328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49400700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x562440858f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b4a800700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 0x56244085fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 -- 172.21.15.46:0/1542734328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 msgr2=0x56244085fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 0x56244085fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 -- 172.21.15.46:0/1542734328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 msgr2=0x562440858f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x562440858f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 -- 172.21.15.46:0/1542734328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562440762560 con 0x562440850760 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b4a800700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 0x56244085fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b49e00700 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e960 tx=0x7f8b3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6eeccb3b3d569d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.708+0000 7f8b37e00700 1 -- 172.21.15.46:0/1542734328 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00e3b0 con 0x562440850760 2024-05-08T23:08:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b37e00700 1 -- 172.21.15.46:0/1542734328 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3c00e510 con 0x562440850760 2024-05-08T23:08:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b37e00700 1 -- 172.21.15.46:0/1542734328 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c026680 con 0x562440850760 2024-05-08T23:08:33.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1542734328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 msgr2=0x562440858a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e960 tx=0x7f8b3c004750 comp rx=0 tx=0).stop 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1542734328 shutdown_connections 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440856470 0x56244085fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440850760 0x562440858a40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/1542734328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x562440858f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1542734328 >> 172.21.15.46:0/1542734328 conn(0x5624407aa210 msgr2=0x562440845d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1542734328 shutdown_connections 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.709+0000 7f8b64da1b80 1 -- 172.21.15.46:0/1542734328 wait complete. 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.710+0000 7f8b64da1b80 1 -- start start 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.710+0000 7f8b64da1b80 1 Processor -- start 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x5624408655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440856470 0x5624408bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624408bfdc0 con 0x562440851120 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624408bff00 con 0x562440856470 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624408c0040 con 0x562440850760 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b64da1b80 10 client.0 _ll_register_callbacks cb 0x562440741b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34072/0 (socket says 172.21.15.46:34072) 2024-05-08T23:08:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 -- 172.21.15.46:0/3892460907 learned_addr learned my addr 172.21.15.46:0/3892460907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49400700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x5624408655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b4a800700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440856470 0x5624408bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 -- 172.21.15.46:0/3892460907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440856470 msgr2=0x5624408bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440856470 0x5624408bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 -- 172.21.15.46:0/3892460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 msgr2=0x5624408655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x5624408655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 -- 172.21.15.46:0/3892460907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624405c9b60 con 0x562440850760 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.711+0000 7f8b49e00700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e6c0 tx=0x7f8b3c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2455a50e5438889d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c01c6f0 con 0x562440850760 2024-05-08T23:08:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b3809b670 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8b3c01c850 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3c01ca30 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b64da1b80 10 client.27004 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562440621cd0 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3809b670 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b3c03e030 con 0x562440850760 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.712+0000 7f8b64da1b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b64da1b80 10 client.27004 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b64da1b80 20 client.27004 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5624408c1450 0x5624408c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b64da1b80 4 client.27004 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562440794ec0 con 0x5624408c1450 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b49400700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5624408c1450 0x5624408c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b49400700 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5624408c1450 0x5624408c3870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e734872fb6602186 server_cookie=13e27d8525b3ad7e in_seq=0 out_seq=0 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b2be00700 10 client.27004 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.713+0000 7f8b2be00700 1 -- 172.21.15.46:0/3892460907 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562440794ec0 con 0x5624408c1450 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b2be00700 10 client.27004 handle_command_reply: tid=0 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 client.27004 shutdown 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 msgr2=0x562440865070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e6c0 tx=0x7f8b3c00eea0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 shutdown_connections 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5624408c1450 0x5624408c3870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562440856470 0x5624408bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440850760 0x562440865070 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 --2- 172.21.15.46:0/3892460907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562440851120 0x5624408655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b2be00700 0 client.27004 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 >> 172.21.15.46:0/3892460907 conn(0x5624407aa210 msgr2=0x5624408588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 shutdown_connections 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 1 -- 172.21.15.46:0/3892460907 wait complete. 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.714+0000 7f8b64da1b80 20 client.27004 trim_cache size 0 max 16384 2024-05-08T23:08:33.727 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 msgr2=0x7f8b300a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 0x7f8b300a3870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c011680 tx=0x7f8b3c009e50 comp rx=0 tx=0).stop 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 msgr2=0x7f8b4c197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c197670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000c00 tx=0x7f8b38004730 comp rx=0 tx=0).stop 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 shutdown_connections 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b300a13b0 0x7f8b300a3870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c19b950 0x7f8b4c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b4c100ed0 0x7f8b4c197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 --2- 172.21.15.46:0/4248218225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4c105ff0 0x7f8b4c197670 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 >> 172.21.15.46:0/4248218225 conn(0x7f8b4c0fc7b0 msgr2=0x7f8b4c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 shutdown_connections 2024-05-08T23:08:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.715+0000 7f8b4b200700 1 -- 172.21.15.46:0/4248218225 wait complete. 2024-05-08T23:08:33.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:08:33.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 321641 10.78 MB/sec execute 55 sec latency 15.019 ms 2024-05-08T23:08:33.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 -- 172.21.15.46:0/2528534114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 msgr2=0x7fb0d40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2528534114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d40fffa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009990 tx=0x7fb0c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 -- 172.21.15.46:0/2528534114 shutdown_connections 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2528534114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d410d770 0x7fb0d410fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2528534114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d40fffa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2528534114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d4100570 0x7fb0d410d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 -- 172.21.15.46:0/2528534114 >> 172.21.15.46:0/2528534114 conn(0x7fb0d40fb760 msgr2=0x7fb0d40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 -- 172.21.15.46:0/2528534114 shutdown_connections 2024-05-08T23:08:33.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.974+0000 7fb0dac00700 1 -- 172.21.15.46:0/2528534114 wait complete. 2024-05-08T23:08:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.975+0000 7fb0dac00700 1 Processor -- start 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 -- start start 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d4100570 0x7fb0d4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d410d770 0x7fb0d419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d419e400 con 0x7fb0d4100570 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d419e570 con 0x7fb0d410d770 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0dac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d419e730 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34086/0 (socket says 172.21.15.46:34086) 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.976+0000 7fb0d3e00700 1 -- 172.21.15.46:0/2912999171 learned_addr learned my addr 172.21.15.46:0/2912999171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3400700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d4100570 0x7fb0d4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d410d770 0x7fb0d419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 -- 172.21.15.46:0/2912999171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d410d770 msgr2=0x7fb0d419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d410d770 0x7fb0d419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 -- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d4100570 msgr2=0x7fb0d4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d4100570 0x7fb0d4199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 -- 172.21.15.46:0/2912999171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40095f0 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000c00 tx=0x7fb0c4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d0c00700 1 -- 172.21.15.46:0/2912999171 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c401d070 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d0c00700 1 -- 172.21.15.46:0/2912999171 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0c4022470 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d419e9b0 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.977+0000 7fb0d0c00700 1 -- 172.21.15.46:0/2912999171 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4017650 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.978+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d41a6250 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.981+0000 7fb0d0c00700 1 -- 172.21.15.46:0/2912999171 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb0c40225e0 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.983+0000 7fb0ee6a8b80 1 Processor -- start 2024-05-08T23:08:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.983+0000 7fb0d0c00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 0x7fb0b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.983+0000 7fb0d0c00700 1 -- 172.21.15.46:0/2912999171 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb0c40ce390 con 0x7fb0d40ffbc0 2024-05-08T23:08:33.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.983+0000 7fb0ee6a8b80 1 -- start start 2024-05-08T23:08:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d3400700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 0x7fb0b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c3484511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 0x55c34844b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3483a2150 con 0x55c34844b890 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34839fb10 con 0x55c348451720 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34839b400 con 0x55c348450e00 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c3484511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c3484511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34092/0 (socket says 172.21.15.46:34092) 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d3e00700 1 -- 172.21.15.46:0/2200793681 learned_addr learned my addr 172.21.15.46:0/2200793681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d3400700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 0x55c34844b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.984+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d3400700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 0x7fb0b80a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb0d419aa30 tx=0x7fb0c0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 -- 172.21.15.46:0/2200793681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 msgr2=0x55c3484511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c3484511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 -- 172.21.15.46:0/2200793681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 msgr2=0x55c34844b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 0x55c34844b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 -- 172.21.15.46:0/2200793681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34821c3b0 con 0x55c34844b890 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d3400700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 0x55c34844b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844bcd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40690f0 tx=0x7fb0cc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e1325285d97ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0c9200700 1 -- 172.21.15.46:0/2200793681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00e070 con 0x55c34844b890 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0c9200700 1 -- 172.21.15.46:0/2200793681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0cc00f040 con 0x55c34844b890 2024-05-08T23:08:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.985+0000 7fb0c9200700 1 -- 172.21.15.46:0/2200793681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc014820 con 0x55c34844b890 2024-05-08T23:08:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/2200793681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 msgr2=0x55c34844bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844bcd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40690f0 tx=0x7fb0cc009920 comp rx=0 tx=0).stop 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/2200793681 shutdown_connections 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844bcd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c3484511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/2200793681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348451720 0x55c34844b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/2200793681 >> 172.21.15.46:0/2200793681 conn(0x55c3483a9210 msgr2=0x55c348445df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.986+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/2200793681 shutdown_connections 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.987+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/2200793681 wait complete. 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.987+0000 7fb0ee6a8b80 1 -- start start 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.987+0000 7fb0ee6a8b80 1 Processor -- start 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c34844b890 0x55c348454ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c348455550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c348337560 con 0x55c348451720 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c348337200 con 0x55c348450e00 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34821e6a0 con 0x55c34844b890 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0ee6a8b80 10 client.0 _ll_register_callbacks cb 0x55c348340b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c34844b890 0x55c348454ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47298/0 (socket says 172.21.15.46:47298) 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0d3400700 1 -- 172.21.15.46:0/1741948866 learned_addr learned my addr 172.21.15.46:0/1741948866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.988+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c348455550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 -- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c34844b890 msgr2=0x55c348454ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c34844b890 0x55c348454ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 -- 172.21.15.46:0/1741948866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 msgr2=0x55c348455550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c348455550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 -- 172.21.15.46:0/1741948866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c0011ac0 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0d3400700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0012c30 tx=0x7fb0c00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a2e0b92216ffcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0bd600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0004220 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.989+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3482f4270 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb0c0018070 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c0004380 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0ee6a8b80 10 client.26957 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb0cc009050 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3482f4270 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c0003b40 con 0x55c348450e00 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0ee6a8b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0ee6a8b80 10 client.26957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:08:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.990+0000 7fb0ee6a8b80 20 client.26957 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:33.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484652a0 0x55c3484676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:33.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0ee6a8b80 4 client.26957 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:33.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c34841a810 con 0x55c3484652a0 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484652a0 0x55c3484676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0bd600700 10 client.26957 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.991+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484652a0 0x55c3484676c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a74e8a650d886900 server_cookie=c5a3a567def521a2 in_seq=0 out_seq=0 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.992+0000 7fb0bd600700 1 -- 172.21.15.46:0/1741948866 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c34841a810 con 0x55c3484652a0 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.992+0000 7fb0bd600700 10 client.26957 handle_command_reply: tid=0 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.992+0000 7fb0ee6a8b80 1 client.26957 shutdown 2024-05-08T23:08:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 msgr2=0x55c348455010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0012c30 tx=0x7fb0c00047c0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 shutdown_connections 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484652a0 0x55c3484676c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c348455550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c34844b890 0x55c348454ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0bd600700 0 client.26957 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/1741948866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348455010 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 >> 172.21.15.46:0/1741948866 conn(0x55c3483a9210 msgr2=0x55c3484455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 shutdown_connections 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/1741948866 wait complete. 2024-05-08T23:08:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:33.993+0000 7fb0ee6a8b80 20 client.26957 trim_cache size 0 max 16384 2024-05-08T23:08:34.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.009+0000 7fb0ee6a8b80 1 Processor -- start 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.009+0000 7fb0ee6a8b80 1 -- start start 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.009+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 0x55c3484553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3482f4270 con 0x55c34844b890 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3484560f0 con 0x55c348450e00 2024-05-08T23:08:34.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3484515b0 con 0x55c348451720 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 0x55c3484553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47308/0 (socket says 172.21.15.46:47308) 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 0x55c3484553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34110/0 (socket says 172.21.15.46:34110) 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 -- 172.21.15.46:0/3262701360 learned_addr learned my addr 172.21.15.46:0/3262701360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 -- 172.21.15.46:0/3262701360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 msgr2=0x55c3484553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 0x55c3484553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 -- 172.21.15.46:0/3262701360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 msgr2=0x55c34844b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.010+0000 7fb0d3400700 1 -- 172.21.15.46:0/3262701360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3481c8b60 con 0x55c348450e00 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.011+0000 7fb0d3400700 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005cb0 tx=0x7fb0c00117f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b059e6a2fb04d633 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.011+0000 7fb0c9c00700 1 -- 172.21.15.46:0/3262701360 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00095c0 con 0x55c348450e00 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.011+0000 7fb0c9c00700 1 -- 172.21.15.46:0/3262701360 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0c001ce90 con 0x55c348450e00 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.011+0000 7fb0c9c00700 1 -- 172.21.15.46:0/3262701360 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003cb0 con 0x55c348450e00 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/3262701360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 msgr2=0x55c348445d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005cb0 tx=0x7fb0c00117f0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/3262701360 shutdown_connections 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348451720 0x55c3484553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c34844b890 0x55c34844b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/3262701360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c348450e00 0x55c348445d20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/3262701360 >> 172.21.15.46:0/3262701360 conn(0x55c3483a9210 msgr2=0x55c34844bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/3262701360 shutdown_connections 2024-05-08T23:08:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/3262701360 wait complete. 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 -- start start 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.012+0000 7fb0ee6a8b80 1 Processor -- start 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c34845a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c3484ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34845afd0 con 0x55c348451720 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3484bad10 con 0x55c34844b890 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3484baed0 con 0x55c348450e00 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0ee6a8b80 10 client.0 _ll_register_callbacks cb 0x55c348340b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:34.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0d3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47322/0 (socket says 172.21.15.46:47322) 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0d3e00700 1 -- 172.21.15.46:0/80132402 learned_addr learned my addr 172.21.15.46:0/80132402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0d3400700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c34845a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.013+0000 7fb0d8c00700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c3484ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 -- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 msgr2=0x55c34845a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c34845a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 -- 172.21.15.46:0/80132402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 msgr2=0x55c3484ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c3484ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 -- 172.21.15.46:0/80132402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc009050 con 0x55c34844b890 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0d3e00700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400f740 tx=0x7fb0c40a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49225b0db8cf9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0bd600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c409d040 con 0x55c34844b890 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb0c4084040 con 0x55c34844b890 2024-05-08T23:08:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.014+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0cc00f040 con 0x55c34844b890 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c409b120 con 0x55c34844b890 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 10 client.26963 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c348361560 con 0x55c34844b890 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0cc00f040 con 0x55c34844b890 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c40cb950 con 0x55c34844b890 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 10 client.26963 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 10 client.26963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 20 client.26963 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484bc440 0x55c3484be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 4 client.26963 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c348393ec0 con 0x55c3484bc440 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.015+0000 7fb0d3400700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484bc440 0x55c3484be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0bd600700 10 client.26963 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0d3400700 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484bc440 0x55c3484be830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8769ae41b8ba39e0 server_cookie=40c1f5e278e50259 in_seq=0 out_seq=0 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0bd600700 1 -- 172.21.15.46:0/80132402 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c348393ec0 con 0x55c3484bc440 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0bd600700 10 client.26963 handle_command_reply: tid=0 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0ee6a8b80 1 client.26963 shutdown 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 msgr2=0x55c34845a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.016+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400f740 tx=0x7fb0c40a1040 comp rx=0 tx=0).stop 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 shutdown_connections 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55c3484bc440 0x55c3484be830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c348451720 0x55c3484ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c34844b890 0x55c34845a300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0bd600700 0 client.26963 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 --2- 172.21.15.46:0/80132402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c348450e00 0x55c34845a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 >> 172.21.15.46:0/80132402 conn(0x55c3483a9210 msgr2=0x55c3484553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 shutdown_connections 2024-05-08T23:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 1 -- 172.21.15.46:0/80132402 wait complete. 2024-05-08T23:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.017+0000 7fb0ee6a8b80 20 client.26963 trim_cache size 0 max 16384 2024-05-08T23:08:34.022 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 msgr2=0x7fb0b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 0x7fb0b80a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb0d419aa30 tx=0x7fb0c0009260 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 msgr2=0x7fb0d41994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000c00 tx=0x7fb0c4005740 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 shutdown_connections 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0d410d770 0x7fb0d419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d40ffbc0 0x7fb0d41994e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb0b80a1290 0x7fb0b80a3750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 --2- 172.21.15.46:0/2912999171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0d4100570 0x7fb0d4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 >> 172.21.15.46:0/2912999171 conn(0x7fb0d40fb760 msgr2=0x7fb0d40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 shutdown_connections 2024-05-08T23:08:34.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.018+0000 7fb0dac00700 1 -- 172.21.15.46:0/2912999171 wait complete. 2024-05-08T23:08:34.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:33 smithi046 ceph-mon[36898]: pgmap v145: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-05-08T23:08:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:33 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2243026726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:08:34.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:08:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:33 smithi086 ceph-mon[78610]: pgmap v145: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-05-08T23:08:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:33 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2243026726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:08:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:33 smithi053 ceph-mon[77587]: pgmap v145: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-05-08T23:08:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:33 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2243026726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:08:34.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 -- 172.21.15.46:0/3837813757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41083d0 msgr2=0x7ff0b410ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/3837813757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41083d0 0x7ff0b410ff50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b0a0 tx=0x7ff0ac00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 -- 172.21.15.46:0/3837813757 shutdown_connections 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/3837813757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41083d0 0x7ff0b410ff50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/3837813757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41032b0 0x7ff0b4103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/3837813757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b4103c60 0x7ff0b4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 -- 172.21.15.46:0/3837813757 >> 172.21.15.46:0/3837813757 conn(0x7ff0b40feb10 msgr2=0x7ff0b4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 -- 172.21.15.46:0/3837813757 shutdown_connections 2024-05-08T23:08:34.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.287+0000 7ff0b8a00700 1 -- 172.21.15.46:0/3837813757 wait complete. 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.288+0000 7ff0b8a00700 1 Processor -- start 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.288+0000 7ff0b8a00700 1 -- start start 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 0x7ff0b4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b419df40 0x7ff0b419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419e9c0 con 0x7ff0b419df40 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419eb30 con 0x7ff0b41083d0 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419ecf0 con 0x7ff0b41032b0 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 0x7ff0b4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34128/0 (socket says 172.21.15.46:34128) 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b1e00700 1 -- 172.21.15.46:0/2251499437 learned_addr learned my addr 172.21.15.46:0/2251499437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 0x7ff0b4199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47328/0 (socket says 172.21.15.46:47328) 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.289+0000 7ff0b2800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b419df40 0x7ff0b419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 -- 172.21.15.46:0/2251499437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 msgr2=0x7ff0b4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 0x7ff0b4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 -- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b419df40 msgr2=0x7ff0b419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b419df40 0x7ff0b419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 -- 172.21.15.46:0/2251499437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0009520 con 0x7ff0b41032b0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9c0 tx=0x7ff0a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0aaa00700 1 -- 172.21.15.46:0/2251499437 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40097b0 con 0x7ff0b41032b0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0aaa00700 1 -- 172.21.15.46:0/2251499437 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a400ee60 con 0x7ff0b41032b0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0aaa00700 1 -- 172.21.15.46:0/2251499437 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40054f0 con 0x7ff0b41032b0 2024-05-08T23:08:34.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac009d00 con 0x7ff0b41032b0 2024-05-08T23:08:34.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.290+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41a6550 con 0x7ff0b41032b0 2024-05-08T23:08:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.294+0000 7ff0aaa00700 1 -- 172.21.15.46:0/2251499437 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff0a4004750 con 0x7ff0b41032b0 2024-05-08T23:08:34.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.295+0000 7ff0cc5acb80 1 Processor -- start 2024-05-08T23:08:34.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.295+0000 7ff0aaa00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.295+0000 7ff0aaa00700 1 -- 172.21.15.46:0/2251499437 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0a4014070 con 0x7ff0b41032b0 2024-05-08T23:08:34.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.295+0000 7ff0cc5acb80 1 -- start start 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b1400700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9053f150 con 0x561d905ede00 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9053cb10 con 0x561d905ee720 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d90538400 con 0x561d905e8890 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34130/0 (socket says 172.21.15.46:34130) 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.296+0000 7ff0b2800700 1 -- 172.21.15.46:0/3917060920 learned_addr learned my addr 172.21.15.46:0/3917060920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b1400700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0b419abb0 tx=0x7ff0a0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 -- 172.21.15.46:0/3917060920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 msgr2=0x561d905e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 -- 172.21.15.46:0/3917060920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 msgr2=0x561d905ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 -- 172.21.15.46:0/3917060920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d903b93b0 con 0x561d905e8890 2024-05-08T23:08:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff0b2800700 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b0a0 tx=0x7ff0ac009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7555c7be5003f92b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff09f000700 1 -- 172.21.15.46:0/3917060920 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00e040 con 0x561d905e8890 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff09f000700 1 -- 172.21.15.46:0/3917060920 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0ac0094c0 con 0x561d905e8890 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.297+0000 7ff09f000700 1 -- 172.21.15.46:0/3917060920 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac003c60 con 0x561d905e8890 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/3917060920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 msgr2=0x561d905e8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b0a0 tx=0x7ff0ac009c60 comp rx=0 tx=0).stop 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/3917060920 shutdown_connections 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905e8cd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/3917060920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/3917060920 >> 172.21.15.46:0/3917060920 conn(0x561d90546210 msgr2=0x561d905e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/3917060920 shutdown_connections 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.298+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/3917060920 wait complete. 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.299+0000 7ff0cc5acb80 1 -- start start 2024-05-08T23:08:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.299+0000 7ff0cc5acb80 1 Processor -- start 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ee720 0x561d905f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d904d4560 con 0x561d905ede00 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d904d4200 con 0x561d905e8890 2024-05-08T23:08:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d903bb6a0 con 0x561d905ee720 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0cc5acb80 10 client.0 _ll_register_callbacks cb 0x561d904ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47338/0 (socket says 172.21.15.46:47338) 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 -- 172.21.15.46:0/531195775 learned_addr learned my addr 172.21.15.46:0/531195775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b2800700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ee720 0x561d905f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1400700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 -- 172.21.15.46:0/531195775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ee720 msgr2=0x561d905f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ee720 0x561d905f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 -- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 msgr2=0x561d905f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.300+0000 7ff0b1e00700 1 -- 172.21.15.46:0/531195775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41107c0 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff0b1400700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c5b0 tx=0x7ff0a409a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350e1c298e7e1a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff093400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40a2430 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0a409a210 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d905b7810 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0a409a3d0 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff0cc5acb80 10 client.26975 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.301+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d90365b60 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a407f080 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0a40a1fa0 con 0x561d905e8890 2024-05-08T23:08:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 10 client.26975 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 10 client.26975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 20 client.26975 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d906022a0 0x561d906046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 4 client.26975 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d90530ec0 con 0x561d906022a0 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.302+0000 7ff0b1400700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d906022a0 0x561d906046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.303+0000 7ff093400700 10 client.26975 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.303+0000 7ff0b1400700 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d906022a0 0x561d906046c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7829a5f70bf921e6 server_cookie=3bbc8053ba28d4db in_seq=0 out_seq=0 2024-05-08T23:08:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff093400700 1 -- 172.21.15.46:0/531195775 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d90530ec0 con 0x561d906022a0 2024-05-08T23:08:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff093400700 10 client.26975 handle_command_reply: tid=0 2024-05-08T23:08:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 client.26975 shutdown 2024-05-08T23:08:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 msgr2=0x561d905f1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c5b0 tx=0x7ff0a409a030 comp rx=0 tx=0).stop 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 shutdown_connections 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ee720 0x561d905f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905e8890 0x561d905f1b60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d906022a0 0x561d906046c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/531195775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ede00 0x561d905f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff093400700 0 client.26975 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 >> 172.21.15.46:0/531195775 conn(0x561d90546210 msgr2=0x561d905e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 shutdown_connections 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.304+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/531195775 wait complete. 2024-05-08T23:08:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.305+0000 7ff0cc5acb80 20 client.26975 trim_cache size 0 max 16384 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.321+0000 7ff0cc5acb80 1 Processor -- start 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.321+0000 7ff0cc5acb80 1 -- start start 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 0x561d905e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d905ff940 con 0x561d905e8890 2024-05-08T23:08:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d906020c0 con 0x561d905ee720 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d905f3190 con 0x561d905ede00 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 0x561d905e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34154/0 (socket says 172.21.15.46:34154) 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1400700 1 -- 172.21.15.46:0/394886700 learned_addr learned my addr 172.21.15.46:0/394886700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1e00700 1 -- 172.21.15.46:0/394886700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 msgr2=0x561d905ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.322+0000 7ff0b1e00700 1 -- 172.21.15.46:0/394886700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 msgr2=0x561d905e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff0b1e00700 1 -- 172.21.15.46:0/394886700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d904fe560 con 0x561d905e8890 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff0b2800700 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff0b1400700 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 0x561d905e8350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005c10 tx=0x7ff0a400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519f1fb305391d10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff09fa00700 1 -- 172.21.15.46:0/394886700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4094c60 con 0x561d905e8890 2024-05-08T23:08:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff09fa00700 1 -- 172.21.15.46:0/394886700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a407c070 con 0x561d905e8890 2024-05-08T23:08:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.323+0000 7ff09fa00700 1 -- 172.21.15.46:0/394886700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4097d20 con 0x561d905e8890 2024-05-08T23:08:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/394886700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 msgr2=0x561d905e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 0x561d905e8350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005c10 tx=0x7ff0a400efd0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/394886700 shutdown_connections 2024-05-08T23:08:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ee720 0x561d905e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905e8890 0x561d905e8350 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/394886700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905ede00 0x561d905ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/394886700 >> 172.21.15.46:0/394886700 conn(0x561d90546210 msgr2=0x561d905f0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/394886700 shutdown_connections 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.324+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/394886700 wait complete. 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.325+0000 7ff0cc5acb80 1 -- start start 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.325+0000 7ff0cc5acb80 1 Processor -- start 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ede00 0x561d905f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ee720 0x561d9065ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d905f7f60 con 0x561d905ee720 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d906604f0 con 0x561d905ede00 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d90660630 con 0x561d905e8890 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0cc5acb80 10 client.0 _ll_register_callbacks cb 0x561d904ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34168/0 (socket says 172.21.15.46:34168) 2024-05-08T23:08:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0b2800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ee720 0x561d9065ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0b1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ede00 0x561d905f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.326+0000 7ff0b1e00700 1 -- 172.21.15.46:0/1370362810 learned_addr learned my addr 172.21.15.46:0/1370362810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 -- 172.21.15.46:0/1370362810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ede00 msgr2=0x561d905f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ede00 0x561d905f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 -- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ee720 msgr2=0x561d9065ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ee720 0x561d9065ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 -- 172.21.15.46:0/1370362810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a0017050 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0b1e00700 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40cd010 tx=0x7ff0a4099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=216e154e7eb9f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff093400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409dde0 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff0a4083420 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.327+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d90491270 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0a40994a0 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 10 client.27040 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d903b93b0 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d90491270 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0a40b2b60 con 0x561d905e8890 2024-05-08T23:08:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 10 client.27040 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 10 client.27040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 20 client.27040 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.328+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d90661820 0x561d90663c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.329+0000 7ff0cc5acb80 4 client.27040 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.329+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d90386cf0 con 0x561d90661820 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.329+0000 7ff0b1400700 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d90661820 0x561d90663c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.329+0000 7ff093400700 10 client.27040 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.329+0000 7ff0b1400700 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d90661820 0x561d90663c10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64cd81624ca0079 server_cookie=52e7fc74ad3ff2d0 in_seq=0 out_seq=0 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff093400700 1 -- 172.21.15.46:0/1370362810 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d90386cf0 con 0x561d90661820 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff093400700 10 client.27040 handle_command_reply: tid=0 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 client.27040 shutdown 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 msgr2=0x561d905f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40cd010 tx=0x7ff0a4099c20 comp rx=0 tx=0).stop 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 shutdown_connections 2024-05-08T23:08:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x561d90661820 0x561d90663c10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d905ee720 0x561d9065ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d905e8890 0x561d905f7310 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 --2- 172.21.15.46:0/1370362810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d905ede00 0x561d905f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff093400700 0 client.27040 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 >> 172.21.15.46:0/1370362810 conn(0x561d90546210 msgr2=0x561d905ed330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 shutdown_connections 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 1 -- 172.21.15.46:0/1370362810 wait complete. 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.330+0000 7ff0cc5acb80 20 client.27040 trim_cache size 0 max 16384 2024-05-08T23:08:34.333 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 msgr2=0x7ff0980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0b419abb0 tx=0x7ff0a0009260 comp rx=0 tx=0).stop 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 msgr2=0x7ff0b4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9c0 tx=0x7ff0a400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 shutdown_connections 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0980a1310 0x7ff0980a37d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b419df40 0x7ff0b419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0b41032b0 0x7ff0b4199660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 --2- 172.21.15.46:0/2251499437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0b41083d0 0x7ff0b4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 >> 172.21.15.46:0/2251499437 conn(0x7ff0b40feb10 msgr2=0x7ff0b4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 shutdown_connections 2024-05-08T23:08:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.331+0000 7ff0b8a00700 1 -- 172.21.15.46:0/2251499437 wait complete. 2024-05-08T23:08:34.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:08:34.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 -- 172.21.15.46:0/3056510103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048101060 msgr2=0x7fb048101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 --2- 172.21.15.46:0/3056510103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048101060 0x7fb048101440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb038009990 tx=0x7fb038009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 -- 172.21.15.46:0/3056510103 shutdown_connections 2024-05-08T23:08:34.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 --2- 172.21.15.46:0/3056510103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb048106180 0x7fb04810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 --2- 172.21.15.46:0/3056510103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048101060 0x7fb048101440 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 --2- 172.21.15.46:0/3056510103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101a10 0x7fb048105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.567+0000 7fb04ea00700 1 -- 172.21.15.46:0/3056510103 >> 172.21.15.46:0/3056510103 conn(0x7fb0480fc900 msgr2=0x7fb0480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.568+0000 7fb04ea00700 1 -- 172.21.15.46:0/3056510103 shutdown_connections 2024-05-08T23:08:34.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.568+0000 7fb04ea00700 1 -- 172.21.15.46:0/3056510103 wait complete. 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 Processor -- start 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 -- start start 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101060 0x7fb0481995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04819de90 0x7fb04819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819e910 con 0x7fb04819de90 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819ea80 con 0x7fb048101060 2024-05-08T23:08:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.569+0000 7fb04ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819ec40 con 0x7fb048106180 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34184/0 (socket says 172.21.15.46:34184) 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 -- 172.21.15.46:0/1794457456 learned_addr learned my addr 172.21.15.46:0/1794457456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb04ca00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04819de90 0x7fb04819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047e00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101060 0x7fb0481995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 -- 172.21.15.46:0/1794457456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101060 msgr2=0x7fb0481995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101060 0x7fb0481995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 -- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04819de90 msgr2=0x7fb04819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04819de90 0x7fb04819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 -- 172.21.15.46:0/1794457456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380095f0 con 0x7fb048106180 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.570+0000 7fb047400700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb03400d950 tx=0x7fb03400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.571+0000 7fb044c00700 1 -- 172.21.15.46:0/1794457456 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034004220 con 0x7fb048106180 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.571+0000 7fb044c00700 1 -- 172.21.15.46:0/1794457456 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb034007620 con 0x7fb048106180 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.571+0000 7fb044c00700 1 -- 172.21.15.46:0/1794457456 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340056c0 con 0x7fb048106180 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.571+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481a61a0 con 0x7fb048106180 2024-05-08T23:08:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.571+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a6680 con 0x7fb048106180 2024-05-08T23:08:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.575+0000 7fb044c00700 1 -- 172.21.15.46:0/1794457456 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb034004380 con 0x7fb048106180 2024-05-08T23:08:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 Processor -- start 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 -- start start 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb044c00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 0x7fb02c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb044c00700 1 -- 172.21.15.46:0/1794457456 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb0340ccd30 con 0x7fb048106180 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 0x55e57366d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb04ca00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 0x7fb02c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5735be150 con 0x55e573667890 2024-05-08T23:08:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5735bbb10 con 0x55e57366ce00 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.577+0000 7fb062483b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5735b7400 con 0x55e57366d720 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34198/0 (socket says 172.21.15.46:34198) 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 -- 172.21.15.46:0/3675591149 learned_addr learned my addr 172.21.15.46:0/3675591149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047e00700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 0x55e57366d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb04ca00700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 -- 172.21.15.46:0/3675591149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 msgr2=0x55e57366d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 0x55e57366d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 -- 172.21.15.46:0/3675591149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 msgr2=0x55e573667cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 -- 172.21.15.46:0/3675591149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5734383b0 con 0x55e57366d720 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047e00700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 0x55e57366d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb047400700 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb034005cd0 tx=0x7fb0340223d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af39e123450ef36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb04ca00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 0x7fb02c0a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb04000f600 tx=0x7fb04000f9c0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb03d200700 1 -- 172.21.15.46:0/3675591149 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03409d8a0 con 0x55e57366d720 2024-05-08T23:08:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb03d200700 1 -- 172.21.15.46:0/3675591149 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0340a1ef0 con 0x55e57366d720 2024-05-08T23:08:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.578+0000 7fb03d200700 1 -- 172.21.15.46:0/3675591149 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340a1a60 con 0x55e57366d720 2024-05-08T23:08:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 -- 172.21.15.46:0/3675591149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 msgr2=0x55e573667350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb034005cd0 tx=0x7fb0340223d0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 -- 172.21.15.46:0/3675591149 shutdown_connections 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366ce00 0x55e57366d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 --2- 172.21.15.46:0/3675591149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366d720 0x55e573667350 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 -- 172.21.15.46:0/3675591149 >> 172.21.15.46:0/3675591149 conn(0x55e5735c5210 msgr2=0x55e573661df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 -- 172.21.15.46:0/3675591149 shutdown_connections 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.579+0000 7fb062483b80 1 -- 172.21.15.46:0/3675591149 wait complete. 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.580+0000 7fb062483b80 1 -- start start 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.580+0000 7fb062483b80 1 Processor -- start 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 0x55e573671060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e573553560 con 0x55e57366ce00 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e573553200 con 0x55e57366d720 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57343a6a0 con 0x55e573667890 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb062483b80 10 client.0 _ll_register_callbacks cb 0x55e57355cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34202/0 (socket says 172.21.15.46:34202) 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 -- 172.21.15.46:0/3919014300 learned_addr learned my addr 172.21.15.46:0/3919014300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb04ca00700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047400700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 0x55e573671060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 -- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 msgr2=0x55e5736715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 -- 172.21.15.46:0/3919014300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 msgr2=0x55e573671060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 0x55e573671060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.581+0000 7fb047e00700 1 -- 172.21.15.46:0/3919014300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a6680 con 0x55e573667890 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb047400700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 0x55e573671060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb047e00700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb038005800 tx=0x7fb03800b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa88432305dd429 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb031600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03801d070 con 0x55e573667890 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb038004500 con 0x55e573667890 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e573510270 con 0x55e573667890 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0380046c0 con 0x55e573667890 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb062483b80 10 client.27052 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.582+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e5733e4b60 con 0x55e573667890 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e573510270 con 0x55e573667890 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb038024680 con 0x55e573667890 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 10 client.27052 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 10 client.27052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 20 client.27052 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e573681260 0x55e573683680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 4 client.27052 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e573636810 con 0x55e573681260 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.583+0000 7fb047400700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e573681260 0x55e573683680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.584+0000 7fb047400700 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e573681260 0x55e573683680 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3d772a07c0a870 server_cookie=a5430a4893a6c92a in_seq=0 out_seq=0 2024-05-08T23:08:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.584+0000 7fb031600700 10 client.27052 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb031600700 1 -- 172.21.15.46:0/3919014300 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e573636810 con 0x55e573681260 2024-05-08T23:08:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb031600700 10 client.27052 handle_command_reply: tid=0 2024-05-08T23:08:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 client.27052 shutdown 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 msgr2=0x55e573670b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb038005800 tx=0x7fb03800b5b0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 shutdown_connections 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e573667890 0x55e573670b20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e573681260 0x55e573683680 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 --2- 172.21.15.46:0/3919014300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57366ce00 0x55e573671060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 >> 172.21.15.46:0/3919014300 conn(0x55e5735c5210 msgr2=0x55e5736615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb031600700 0 client.27052 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.585+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 shutdown_connections 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.586+0000 7fb062483b80 1 -- 172.21.15.46:0/3919014300 wait complete. 2024-05-08T23:08:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.586+0000 7fb062483b80 20 client.27052 trim_cache size 0 max 16384 2024-05-08T23:08:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.601+0000 7fb062483b80 1 Processor -- start 2024-05-08T23:08:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.601+0000 7fb062483b80 1 -- start start 2024-05-08T23:08:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e573661d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57365c800 con 0x55e573667890 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e573663a00 con 0x55e57366d720 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb062483b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57366f070 con 0x55e57366ce00 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34218/0 (socket says 172.21.15.46:34218) 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 -- 172.21.15.46:0/3326893161 learned_addr learned my addr 172.21.15.46:0/3326893161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047e00700 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb04ca00700 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e573661d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 -- 172.21.15.46:0/3326893161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 msgr2=0x55e573661d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e573661d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 -- 172.21.15.46:0/3326893161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 msgr2=0x55e573667350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.602+0000 7fb047400700 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.603+0000 7fb047400700 1 -- 172.21.15.46:0/3326893161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e57357d560 con 0x55e57366ce00 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.603+0000 7fb047400700 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb034097dc0 tx=0x7fb034095ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4241330e84319ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.603+0000 7fb03dc00700 1 -- 172.21.15.46:0/3326893161 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03407c070 con 0x55e57366ce00 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.603+0000 7fb03dc00700 1 -- 172.21.15.46:0/3326893161 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0340b6070 con 0x55e57366ce00 2024-05-08T23:08:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.603+0000 7fb03dc00700 1 -- 172.21.15.46:0/3326893161 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340991c0 con 0x55e57366ce00 2024-05-08T23:08:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 -- 172.21.15.46:0/3326893161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 msgr2=0x55e57366c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb034097dc0 tx=0x7fb034095ee0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 -- 172.21.15.46:0/3326893161 shutdown_connections 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e573661d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573667350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 --2- 172.21.15.46:0/3326893161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e57366c4a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 -- 172.21.15.46:0/3326893161 >> 172.21.15.46:0/3326893161 conn(0x55e5735c5210 msgr2=0x55e57366f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 -- 172.21.15.46:0/3326893161 shutdown_connections 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.604+0000 7fb062483b80 1 -- 172.21.15.46:0/3326893161 wait complete. 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.605+0000 7fb062483b80 1 -- start start 2024-05-08T23:08:34.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.605+0000 7fb062483b80 1 Processor -- start 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573675d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5736d6890 con 0x55e573667890 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5736d6a00 con 0x55e57366d720 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5736d6bc0 con 0x55e57366ce00 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb062483b80 10 client.0 _ll_register_callbacks cb 0x55e57355cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb047400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34232/0 (socket says 172.21.15.46:34232) 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb047400700 1 -- 172.21.15.46:0/3872849894 learned_addr learned my addr 172.21.15.46:0/3872849894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb04ca00700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.606+0000 7fb047e00700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573675d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 -- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 msgr2=0x55e5736767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 -- 172.21.15.46:0/3872849894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 msgr2=0x55e573675d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573675d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 -- 172.21.15.46:0/3872849894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0340b1040 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb047400700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb034099770 tx=0x7fb034097fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea66cd065cd8a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb031600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03407c070 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fb0340b22e0 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0340a1ef0 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb034099cd0 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb062483b80 10 client.27058 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.607+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e57343ccd0 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0340a1ef0 con 0x55e57366ce00 2024-05-08T23:08:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0340a1a60 con 0x55e57366ce00 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 10 client.27058 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 10 client.27058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 20 client.27058 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e5736d7f40 0x55e5736da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 4 client.27058 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5735afec0 con 0x55e5736d7f40 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.608+0000 7fb047e00700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e5736d7f40 0x55e5736da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb047e00700 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e5736d7f40 0x55e5736da360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0196e6f4ae8229 server_cookie=fb18386279de87ab in_seq=0 out_seq=0 2024-05-08T23:08:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb031600700 10 client.27058 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:34.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb031600700 1 -- 172.21.15.46:0/3872849894 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5735afec0 con 0x55e5736d7f40 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb031600700 10 client.27058 handle_command_reply: tid=0 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 client.27058 shutdown 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 msgr2=0x55e5736762a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb034099770 tx=0x7fb034097fb0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 shutdown_connections 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e57366d720 0x55e5736767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e5736d7f40 0x55e5736da360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e573667890 0x55e573675d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 --2- 172.21.15.46:0/3872849894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e57366ce00 0x55e5736762a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 >> 172.21.15.46:0/3872849894 conn(0x55e5735c5210 msgr2=0x55e57366e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.609+0000 7fb031600700 0 client.27058 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 shutdown_connections 2024-05-08T23:08:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb062483b80 1 -- 172.21.15.46:0/3872849894 wait complete. 2024-05-08T23:08:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb062483b80 20 client.27058 trim_cache size 0 max 16384 2024-05-08T23:08:34.615 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:08:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 msgr2=0x7fb02c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 0x7fb02c0a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb04000f600 tx=0x7fb04000f9c0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.610+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 msgr2=0x7fb048199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb03400d950 tx=0x7fb03400dc60 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 shutdown_connections 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb02c0a1310 0x7fb02c0a37d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04819de90 0x7fb04819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb048101060 0x7fb0481995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 --2- 172.21.15.46:0/1794457456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb048106180 0x7fb048199af0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 >> 172.21.15.46:0/1794457456 conn(0x7fb0480fc900 msgr2=0x7fb0480fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 shutdown_connections 2024-05-08T23:08:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:34.611+0000 7fb04ea00700 1 -- 172.21.15.46:0/1794457456 wait complete. 2024-05-08T23:08:34.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 323217 10.80 MB/sec execute 56 sec latency 49.799 ms 2024-05-08T23:08:35.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:08:35.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:08:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 -- 172.21.15.46:0/1164118452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c10103aa0 msgr2=0x7f0c10107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 --2- 172.21.15.46:0/1164118452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c10103aa0 0x7f0c10107ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0098f0 tx=0x7f0bfc009c00 comp rx=0 tx=0).stop 2024-05-08T23:08:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 -- 172.21.15.46:0/1164118452 shutdown_connections 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 --2- 172.21.15.46:0/1164118452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c10108210 0x7f0c1010fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 --2- 172.21.15.46:0/1164118452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c101030f0 0x7f0c101034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 --2- 172.21.15.46:0/1164118452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c10103aa0 0x7f0c10107ae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 -- 172.21.15.46:0/1164118452 >> 172.21.15.46:0/1164118452 conn(0x7f0c100fe930 msgr2=0x7f0c10100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 -- 172.21.15.46:0/1164118452 shutdown_connections 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.851+0000 7f0c0f200700 1 -- 172.21.15.46:0/1164118452 wait complete. 2024-05-08T23:08:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.852+0000 7f0c0f200700 1 Processor -- start 2024-05-08T23:08:35.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 -- start start 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c101030f0 0x7f0c10199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c1019dc10 0x7f0c1019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019e690 con 0x7f0c101030f0 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019e800 con 0x7f0c10108210 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019e9c0 con 0x7f0c1019dc10 2024-05-08T23:08:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.853+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c1019dc10 0x7f0c1019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c101030f0 0x7f0c10199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47430/0 (socket says 172.21.15.46:47430) 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 -- 172.21.15.46:0/2068456803 learned_addr learned my addr 172.21.15.46:0/2068456803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 -- 172.21.15.46:0/2068456803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c1019dc10 msgr2=0x7f0c1019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c1019dc10 0x7f0c1019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 -- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c101030f0 msgr2=0x7f0c10199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c101030f0 0x7f0c10199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 -- 172.21.15.46:0/2068456803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc009520 con 0x7f0c10108210 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.854+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009230 tx=0x7f0bfc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.855+0000 7f0c06a00700 1 -- 172.21.15.46:0/2068456803 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01c070 con 0x7f0c10108210 2024-05-08T23:08:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.855+0000 7f0c06a00700 1 -- 172.21.15.46:0/2068456803 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bfc00ba40 con 0x7f0c10108210 2024-05-08T23:08:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.855+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c101a5ec0 con 0x7f0c10108210 2024-05-08T23:08:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.855+0000 7f0c06a00700 1 -- 172.21.15.46:0/2068456803 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0038e0 con 0x7f0c10108210 2024-05-08T23:08:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.855+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101a6230 con 0x7f0c10108210 2024-05-08T23:08:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.859+0000 7f0c06a00700 1 -- 172.21.15.46:0/2068456803 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0bfc020070 con 0x7f0c10108210 2024-05-08T23:08:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.860+0000 7f0c06a00700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 0x7f0bf40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.860+0000 7f0c06a00700 1 -- 172.21.15.46:0/2068456803 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0bfc0cdb40 con 0x7f0c10108210 2024-05-08T23:08:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.861+0000 7f0c0de00700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 0x7f0bf40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.861+0000 7f0c0de00700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 0x7f0bf40a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005fd0 tx=0x7f0c00009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 Processor -- start 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 -- start start 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a98c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a986d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a8dd1d0 con 0x56247a98c9f0 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.862+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a8dab90 con 0x56247a98c0d0 2024-05-08T23:08:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a8d6480 con 0x56247a986910 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a986d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a98c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43048/0 (socket says 172.21.15.46:43048) 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a98c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47432/0 (socket says 172.21.15.46:47432) 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 -- 172.21.15.46:0/867206223 learned_addr learned my addr 172.21.15.46:0/867206223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 -- 172.21.15.46:0/867206223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 msgr2=0x56247a986d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a986d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 -- 172.21.15.46:0/867206223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 msgr2=0x56247a98c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a98c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.863+0000 7f0c0d400700 1 -- 172.21.15.46:0/867206223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247a757430 con 0x56247a98c9f0 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.864+0000 7f0c0d400700 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc096dd0 tx=0x7f0bfc09a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c8291dd6e800cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.864+0000 7f0bfb400700 1 -- 172.21.15.46:0/867206223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc09a8d0 con 0x56247a98c9f0 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.864+0000 7f0bfb400700 1 -- 172.21.15.46:0/867206223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bfc09e700 con 0x56247a98c9f0 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.864+0000 7f0bfb400700 1 -- 172.21.15.46:0/867206223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0ba330 con 0x56247a98c9f0 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/867206223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 msgr2=0x56247a9863d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc096dd0 tx=0x7f0bfc09a4e0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/867206223 shutdown_connections 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a986d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a98c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/867206223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9863d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/867206223 >> 172.21.15.46:0/867206223 conn(0x56247a8e4290 msgr2=0x56247a981350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/867206223 shutdown_connections 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/867206223 wait complete. 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.865+0000 7f0c28d0fb80 1 -- start start 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 Processor -- start 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 0x56247a992990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c9f0 0x56247a9904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a8725e0 con 0x56247a986910 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a872280 con 0x56247a98c9f0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a759720 con 0x56247a98c0d0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.866+0000 7f0c28d0fb80 10 client.0 _ll_register_callbacks cb 0x56247a87bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 0x56247a992990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 0x56247a992990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43052/0 (socket says 172.21.15.46:43052) 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34272/0 (socket says 172.21.15.46:34272) 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0de00700 1 -- 172.21.15.46:0/2063685238 learned_addr learned my addr 172.21.15.46:0/2063685238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0e800700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c9f0 0x56247a9904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 -- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c9f0 msgr2=0x56247a9904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c9f0 0x56247a9904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 -- 172.21.15.46:0/2063685238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 msgr2=0x56247a992990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 0x56247a992990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.867+0000 7f0c0d400700 1 -- 172.21.15.46:0/2063685238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101a6230 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0c0d400700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x56247a987630 tx=0x7f0bfc0cae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9b0f2f519b791e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0befe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0ba2d0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0bfc09a8d0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247a82f2f0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bfc029350 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0c28d0fb80 10 client.27070 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.868+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56247a703be0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56247a82f2f0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0bfc099dc0 con 0x56247a98c0d0 2024-05-08T23:08:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 10 client.27070 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 10 client.27070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 20 client.27070 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247a9b9430 0x56247a9bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 4 client.27070 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56247a955890 con 0x56247a9b9430 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.869+0000 7f0c0de00700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247a9b9430 0x56247a9bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.870+0000 7f0c0de00700 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247a9b9430 0x56247a9bb850 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220bf02f735c20ce server_cookie=ed802a233c9fdcde in_seq=0 out_seq=0 2024-05-08T23:08:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.870+0000 7f0befe00700 10 client.27070 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.870+0000 7f0befe00700 1 -- 172.21.15.46:0/2063685238 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56247a955890 con 0x56247a9b9430 2024-05-08T23:08:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.870+0000 7f0befe00700 10 client.27070 handle_command_reply: tid=0 2024-05-08T23:08:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.870+0000 7f0c28d0fb80 1 client.27070 shutdown 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 msgr2=0x56247a98fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x56247a987630 tx=0x7f0bfc0cae60 comp rx=0 tx=0).stop 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 shutdown_connections 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c9f0 0x56247a9904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247a9b9430 0x56247a9bb850 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a986910 0x56247a992990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/2063685238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98fd80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0befe00700 0 client.27070 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 >> 172.21.15.46:0/2063685238 conn(0x56247a8e4290 msgr2=0x56247a981130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 shutdown_connections 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/2063685238 wait complete. 2024-05-08T23:08:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.871+0000 7f0c28d0fb80 20 client.27070 trim_cache size 0 max 16384 2024-05-08T23:08:35.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 325694 10.82 MB/sec execute 57 sec latency 9.632 ms 2024-05-08T23:08:35.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 Processor -- start 2024-05-08T23:08:35.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 -- start start 2024-05-08T23:08:35.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a986910 0x56247a9863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a97de40 con 0x56247a98c9f0 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a980410 con 0x56247a986910 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a9790c0 con 0x56247a98c0d0 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34288/0 (socket says 172.21.15.46:34288) 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c0d400700 1 -- 172.21.15.46:0/3305478193 learned_addr learned my addr 172.21.15.46:0/3305478193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.895+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a986910 0x56247a9863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0e800700 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 -- 172.21.15.46:0/3305478193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a986910 msgr2=0x56247a9863d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a986910 0x56247a9863d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 -- 172.21.15.46:0/3305478193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 msgr2=0x56247a9c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 -- 172.21.15.46:0/3305478193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247a97d560 con 0x56247a98c0d0 2024-05-08T23:08:35.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009fd0 tx=0x7f0bfc0b3320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc615d0a47015cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0bfbe00700 1 -- 172.21.15.46:0/3305478193 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0a2d40 con 0x56247a98c0d0 2024-05-08T23:08:35.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0bfbe00700 1 -- 172.21.15.46:0/3305478193 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bfc09a230 con 0x56247a98c0d0 2024-05-08T23:08:35.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.896+0000 7f0bfbe00700 1 -- 172.21.15.46:0/3305478193 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc029350 con 0x56247a98c0d0 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3305478193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 msgr2=0x56247a98f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009fd0 tx=0x7f0bfc0b3320 comp rx=0 tx=0).stop 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3305478193 shutdown_connections 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a986910 0x56247a9863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3305478193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a98c0d0 0x56247a98f640 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3305478193 >> 172.21.15.46:0/3305478193 conn(0x56247a8e4290 msgr2=0x56247a992520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3305478193 shutdown_connections 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.897+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3305478193 wait complete. 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 -- start start 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 Processor -- start 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a988060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a988ae0 con 0x56247a98c9f0 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a988c50 con 0x56247a98c0d0 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247a988e10 con 0x56247a986910 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.898+0000 7f0c28d0fb80 10 client.0 _ll_register_callbacks cb 0x56247a87bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:35.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34298/0 (socket says 172.21.15.46:34298) 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 -- 172.21.15.46:0/3787313468 learned_addr learned my addr 172.21.15.46:0/3787313468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a988060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 -- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 msgr2=0x56247a988060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a988060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 -- 172.21.15.46:0/3787313468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 msgr2=0x56247a9885a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9885a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 -- 172.21.15.46:0/3787313468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56247a703be0 con 0x56247a986910 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0e800700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9885a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0c0de00700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009f30 tx=0x7f0c000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9612c3c2febaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.899+0000 7f0befe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00024040 con 0x56247a986910 2024-05-08T23:08:35.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f0c00004a30 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bfc09e700 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c00004bf0 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 10 client.27076 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56247a89c5e0 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bfc09e700 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c00003a90 con 0x56247a986910 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 10 client.27076 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 20 client.27076 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247aa10ea0 0x56247aa132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 4 client.27076 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.900+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56247a8cef40 con 0x56247aa10ea0 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.901+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247aa10ea0 0x56247aa132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.901+0000 7f0c0d400700 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247aa10ea0 0x56247aa132c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918188f6588a1b8d server_cookie=345f5fe5f1a52381 in_seq=0 out_seq=0 2024-05-08T23:08:35.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.901+0000 7f0befe00700 10 client.27076 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:35.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.902+0000 7f0befe00700 1 -- 172.21.15.46:0/3787313468 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56247a8cef40 con 0x56247aa10ea0 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.902+0000 7f0befe00700 10 client.27076 handle_command_reply: tid=0 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 client.27076 shutdown 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 msgr2=0x56247a9a8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009f30 tx=0x7f0c000047c0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 shutdown_connections 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56247aa10ea0 0x56247aa132c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56247a98c9f0 0x56247a9885a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56247a986910 0x56247a9a8320 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 --2- 172.21.15.46:0/3787313468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247a98c0d0 0x56247a988060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 >> 172.21.15.46:0/3787313468 conn(0x56247a8e4290 msgr2=0x56247a989d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0befe00700 0 client.27076 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 shutdown_connections 2024-05-08T23:08:35.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 1 -- 172.21.15.46:0/3787313468 wait complete. 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.903+0000 7f0c28d0fb80 20 client.27076 trim_cache size 0 max 16384 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "37fdd2c6-1317-420a-9e4c-94c3d1907219", 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:08:35.907 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 msgr2=0x7f0bf40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 0x7f0bf40a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005fd0 tx=0x7f0c00009310 comp rx=0 tx=0).stop 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 msgr2=0x7f0c10199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009230 tx=0x7f0bfc00fa00 comp rx=0 tx=0).stop 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 shutdown_connections 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c1019dc10 0x7f0c1019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0bf40a1330 0x7f0bf40a37f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c101030f0 0x7f0c10199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 --2- 172.21.15.46:0/2068456803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10108210 0x7f0c10199870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 >> 172.21.15.46:0/2068456803 conn(0x7f0c100fe930 msgr2=0x7f0c10100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 shutdown_connections 2024-05-08T23:08:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:35.904+0000 7f0c0f200700 1 -- 172.21.15.46:0/2068456803 wait complete. 2024-05-08T23:08:35.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:08:35.993 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:35 smithi046 ceph-mon[36898]: pgmap v146: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.151+0000 7f96dca00700 1 -- 172.21.15.46:0/2712596139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 msgr2=0x7f96d8073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.151+0000 7f96dca00700 1 --2- 172.21.15.46:0/2712596139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d8073720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009990 tx=0x7f96c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.151+0000 7f96dca00700 1 -- 172.21.15.46:0/2712596139 shutdown_connections 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 --2- 172.21.15.46:0/2712596139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8111e20 0x7f96d8114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 --2- 172.21.15.46:0/2712596139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d8073720 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 --2- 172.21.15.46:0/2712596139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8073c60 0x7f96d81118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 -- 172.21.15.46:0/2712596139 >> 172.21.15.46:0/2712596139 conn(0x7f96d80fc940 msgr2=0x7f96d80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 -- 172.21.15.46:0/2712596139 shutdown_connections 2024-05-08T23:08:36.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.152+0000 7f96dca00700 1 -- 172.21.15.46:0/2712596139 wait complete. 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 Processor -- start 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 -- start start 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8073c60 0x7f96d81a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8111e20 0x7f96d819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.153+0000 7f96dca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819dd90 con 0x7f96d8111e20 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96dca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819df00 con 0x7f96d8073c60 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96dca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819e0c0 con 0x7f96d8073340 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34322/0 (socket says 172.21.15.46:34322) 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 -- 172.21.15.46:0/451106061 learned_addr learned my addr 172.21.15.46:0/451106061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5400700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8073c60 0x7f96d81a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d6800700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8111e20 0x7f96d819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 -- 172.21.15.46:0/451106061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8073c60 msgr2=0x7f96d81a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8073c60 0x7f96d81a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 -- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8111e20 msgr2=0x7f96d819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8111e20 0x7f96d819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 -- 172.21.15.46:0/451106061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80095f0 con 0x7f96d8073340 2024-05-08T23:08:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.154+0000 7f96d5e00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005910 tx=0x7f96c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.155+0000 7f96cea00700 1 -- 172.21.15.46:0/451106061 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c801d070 con 0x7f96d8073340 2024-05-08T23:08:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.155+0000 7f96cea00700 1 -- 172.21.15.46:0/451106061 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c8022470 con 0x7f96d8073340 2024-05-08T23:08:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.155+0000 7f96cea00700 1 -- 172.21.15.46:0/451106061 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8017650 con 0x7f96d8073340 2024-05-08T23:08:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.155+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d819e340 con 0x7f96d8073340 2024-05-08T23:08:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.155+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d81aa5f0 con 0x7f96d8073340 2024-05-08T23:08:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.158+0000 7f96cea00700 1 -- 172.21.15.46:0/451106061 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f96c80225e0 con 0x7f96d8073340 2024-05-08T23:08:36.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.159+0000 7f96cea00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 0x7f96bc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.159+0000 7f96f0427b80 1 Processor -- start 2024-05-08T23:08:36.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96cea00700 1 -- 172.21.15.46:0/451106061 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f96c80ce2a0 con 0x7f96d8073340 2024-05-08T23:08:36.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 -- start start 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96d5400700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 0x7f96bc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e585350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e585890 0x55ec9e585cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e4dc150 con 0x55ec9e585890 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e4d9b10 con 0x55ec9e58b720 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96f0427b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e4d5400 con 0x55ec9e58ae00 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96d5400700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 0x7f96bc0a37d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96d819ee40 tx=0x7f96c4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.160+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e585350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34324/0 (socket says 172.21.15.46:34324) 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 -- 172.21.15.46:0/192131149 learned_addr learned my addr 172.21.15.46:0/192131149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d6800700 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e585890 0x55ec9e585cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 -- 172.21.15.46:0/192131149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 msgr2=0x55ec9e585350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e585350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 -- 172.21.15.46:0/192131149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e585890 msgr2=0x55ec9e585cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e585890 0x55ec9e585cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 -- 172.21.15.46:0/192131149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9e3563b0 con 0x55ec9e58ae00 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96d5e00700 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96c80a3010 tx=0x7f96c8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43318778c4a51dda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96c3000700 1 -- 172.21.15.46:0/192131149 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c809fea0 con 0x55ec9e58ae00 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96c3000700 1 -- 172.21.15.46:0/192131149 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c809b850 con 0x55ec9e58ae00 2024-05-08T23:08:36.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.161+0000 7f96c3000700 1 -- 172.21.15.46:0/192131149 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8099610 con 0x55ec9e58ae00 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 -- 172.21.15.46:0/192131149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 msgr2=0x55ec9e58b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96c80a3010 tx=0x7f96c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 -- 172.21.15.46:0/192131149 shutdown_connections 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e585890 0x55ec9e585cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58ae00 0x55ec9e58b1e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 --2- 172.21.15.46:0/192131149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e585350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 -- 172.21.15.46:0/192131149 >> 172.21.15.46:0/192131149 conn(0x55ec9e4e3210 msgr2=0x55ec9e57fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 -- 172.21.15.46:0/192131149 shutdown_connections 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.162+0000 7f96f0427b80 1 -- 172.21.15.46:0/192131149 wait complete. 2024-05-08T23:08:36.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.163+0000 7f96f0427b80 1 -- start start 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.163+0000 7f96f0427b80 1 Processor -- start 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58ae00 0x55ec9e58ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 0x55ec9e58f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e471560 con 0x55ec9e58b720 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e471200 con 0x55ec9e58ae00 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e3586a0 con 0x55ec9e585890 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96f0427b80 10 client.0 _ll_register_callbacks cb 0x55ec9e47ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34340/0 (socket says 172.21.15.46:34340) 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96d5e00700 1 -- 172.21.15.46:0/876219252 learned_addr learned my addr 172.21.15.46:0/876219252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96d6800700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 0x55ec9e58f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.164+0000 7f96d5400700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58ae00 0x55ec9e58ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 -- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58ae00 msgr2=0x55ec9e58ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58ae00 0x55ec9e58ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 -- 172.21.15.46:0/876219252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 msgr2=0x55ec9e58f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 0x55ec9e58f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 -- 172.21.15.46:0/876219252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d81aa5f0 con 0x55ec9e585890 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d6800700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 0x55ec9e58f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96d5e00700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005f50 tx=0x7f96c8099700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da5977e2e65a142 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96b7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c809fea0 con 0x55ec9e585890 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f96c809ef80 con 0x55ec9e585890 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec9e42e270 con 0x55ec9e585890 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c80a3540 con 0x55ec9e585890 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96f0427b80 10 client.27088 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:36.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.165+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ec9e302b60 con 0x55ec9e585890 2024-05-08T23:08:36.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec9e42e270 con 0x55ec9e585890 2024-05-08T23:08:36.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96c80a30b0 con 0x55ec9e585890 2024-05-08T23:08:36.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 10 client.27088 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 10 client.27088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 20 client.27088 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e59e990 0x55ec9e5a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 4 client.27088 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.166+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec9e554810 con 0x55ec9e59e990 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.167+0000 7f96d5400700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e59e990 0x55ec9e5a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.167+0000 7f96d5400700 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e59e990 0x55ec9e5a0d80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6212e43a04d1b4 server_cookie=efb2dcfdd0cff25 in_seq=0 out_seq=0 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.167+0000 7f96b7400700 10 client.27088 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.168+0000 7f96b7400700 1 -- 172.21.15.46:0/876219252 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec9e554810 con 0x55ec9e59e990 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.168+0000 7f96b7400700 10 client.27088 handle_command_reply: tid=0 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.168+0000 7f96f0427b80 1 client.27088 shutdown 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 msgr2=0x55ec9e58ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005f50 tx=0x7f96c8099700 comp rx=0 tx=0).stop 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 shutdown_connections 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e59e990 0x55ec9e5a0d80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58b720 0x55ec9e58f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e58ea50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 --2- 172.21.15.46:0/876219252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58ae00 0x55ec9e58ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96b7400700 0 client.27088 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 >> 172.21.15.46:0/876219252 conn(0x55ec9e4e3210 msgr2=0x55ec9e57f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 shutdown_connections 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 1 -- 172.21.15.46:0/876219252 wait complete. 2024-05-08T23:08:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.169+0000 7f96f0427b80 20 client.27088 trim_cache size 0 max 16384 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.189+0000 7f96f0427b80 1 Processor -- start 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.189+0000 7f96f0427b80 1 -- start start 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e58a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e57fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e57cdc0 con 0x55ec9e58ae00 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e580ae0 con 0x55ec9e58b720 2024-05-08T23:08:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96f0427b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e5a5eb0 con 0x55ec9e585890 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e58a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34356/0 (socket says 172.21.15.46:34356) 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 -- 172.21.15.46:0/3671639496 learned_addr learned my addr 172.21.15.46:0/3671639496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d6800700 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e57fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 -- 172.21.15.46:0/3671639496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 msgr2=0x55ec9e57fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e57fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 -- 172.21.15.46:0/3671639496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 msgr2=0x55ec9e58a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e58a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5e00700 1 -- 172.21.15.46:0/3671639496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9e585f40 con 0x55ec9e585890 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.190+0000 7f96d5400700 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e58a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.191+0000 7f96d5e00700 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c809b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a0171442183b5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.191+0000 7f96c3a00700 1 -- 172.21.15.46:0/3671639496 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c809fea0 con 0x55ec9e585890 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.191+0000 7f96c3a00700 1 -- 172.21.15.46:0/3671639496 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c80a3540 con 0x55ec9e585890 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.191+0000 7f96c3a00700 1 -- 172.21.15.46:0/3671639496 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80b92c0 con 0x55ec9e585890 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- 172.21.15.46:0/3671639496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 msgr2=0x55ec9e585350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c809b770 comp rx=0 tx=0).stop 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- 172.21.15.46:0/3671639496 shutdown_connections 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e58b720 0x55ec9e57fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e585890 0x55ec9e585350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 --2- 172.21.15.46:0/3671639496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e58a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- 172.21.15.46:0/3671639496 >> 172.21.15.46:0/3671639496 conn(0x55ec9e4e3210 msgr2=0x55ec9e58d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- 172.21.15.46:0/3671639496 shutdown_connections 2024-05-08T23:08:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- 172.21.15.46:0/3671639496 wait complete. 2024-05-08T23:08:36.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.192+0000 7f96f0427b80 1 -- start start 2024-05-08T23:08:36.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 Processor -- start 2024-05-08T23:08:36.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e585890 0x55ec9e595ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e594600 con 0x55ec9e58ae00 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e594770 con 0x55ec9e585890 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9e594930 con 0x55ec9e58b720 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96f0427b80 10 client.0 _ll_register_callbacks cb 0x55ec9e47ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34360/0 (socket says 172.21.15.46:34360) 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43132/0 (socket says 172.21.15.46:43132) 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d6800700 1 -- 172.21.15.46:0/2288832711 learned_addr learned my addr 172.21.15.46:0/2288832711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.193+0000 7f96d5e00700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e585890 0x55ec9e595ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 -- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e585890 msgr2=0x55ec9e595ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e585890 0x55ec9e595ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 -- 172.21.15.46:0/2288832711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 msgr2=0x55ec9e596010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 -- 172.21.15.46:0/2288832711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c4017070 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d5400700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96d6800700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96d0007b60 tx=0x7f96d0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19000c2640630fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96b7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000e040 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96c809b850 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.194+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f96d0003e10 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d0004030 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 10 client.27094 fetch_fsmap learned FSMap version 30 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ec9e302b60 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c809b850 con 0x55ec9e58b720 2024-05-08T23:08:36.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96d0018aa0 con 0x55ec9e58b720 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 10 client.27094 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 10 client.27094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 20 client.27094 populate_metadata read hostname 'smithi046' 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e5f5f70 0x55ec9e5f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 4 client.27094 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec9e4cdec0 con 0x55ec9e5f5f70 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.195+0000 7f96d5e00700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e5f5f70 0x55ec9e5f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96d5e00700 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e5f5f70 0x55ec9e5f8390 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1808171c162a5c65 server_cookie=710f4d6c030c616f in_seq=0 out_seq=0 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96b7400700 10 client.27094 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96b7400700 1 -- 172.21.15.46:0/2288832711 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ec9e4cdec0 con 0x55ec9e5f5f70 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96b7400700 10 client.27094 handle_command_reply: tid=0 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 client.27094 shutdown 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 msgr2=0x55ec9e593fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96d0007b60 tx=0x7f96d0009510 comp rx=0 tx=0).stop 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 shutdown_connections 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec9e58b720 0x55ec9e593fe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec9e585890 0x55ec9e595ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ec9e5f5f70 0x55ec9e5f8390 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 --2- 172.21.15.46:0/2288832711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec9e58ae00 0x55ec9e596010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96b7400700 0 client.27094 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.196+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 >> 172.21.15.46:0/2288832711 conn(0x55ec9e4e3210 msgr2=0x55ec9e58a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 shutdown_connections 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96f0427b80 1 -- 172.21.15.46:0/2288832711 wait complete. 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96f0427b80 20 client.27094 trim_cache size 0 max 16384 2024-05-08T23:08:36.201 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "37fdd2c6-1317-420a-9e4c-94c3d1907219": { 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "37fdd2c6-1317-420a-9e4c-94c3d1907219", 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:08:36.202 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 msgr2=0x7f96bc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 0x7f96bc0a37d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96d819ee40 tx=0x7f96c4009260 comp rx=0 tx=0).stop 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 msgr2=0x7f96d81a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.197+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005910 tx=0x7f96c8004730 comp rx=0 tx=0).stop 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 shutdown_connections 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96bc0a1310 0x7f96bc0a37d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d8111e20 0x7f96d819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96d8073340 0x7f96d81a3600 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 --2- 172.21.15.46:0/451106061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d8073c60 0x7f96d81a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:08:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 >> 172.21.15.46:0/451106061 conn(0x7f96d80fc940 msgr2=0x7f96d80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:08:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 shutdown_connections 2024-05-08T23:08:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:08:36.198+0000 7f96dca00700 1 -- 172.21.15.46:0/451106061 wait complete. 2024-05-08T23:08:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:35 smithi086 ceph-mon[78610]: pgmap v146: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-05-08T23:08:36.218 INFO:tasks.cephfs.filesystem:scrub status for tag:37fdd2c6-1317-420a-9e4c-94c3d1907219 - {'path': '/', 'tag': '37fdd2c6-1317-420a-9e4c-94c3d1907219', 'options': 'recursive,force'} 2024-05-08T23:08:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:35 smithi053 ceph-mon[77587]: pgmap v146: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-05-08T23:08:36.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 327250 10.77 MB/sec execute 58 sec latency 15.062 ms 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:08:37.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:08:37.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:08:37.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 329864 10.87 MB/sec execute 59 sec latency 59.269 ms 2024-05-08T23:08:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:37 smithi086 ceph-mon[78610]: pgmap v147: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:37 smithi053 ceph-mon[77587]: pgmap v147: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:37 smithi046 ceph-mon[36898]: pgmap v147: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:38.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 331580 10.80 MB/sec execute 60 sec latency 14.429 ms 2024-05-08T23:08:39.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 333824 10.85 MB/sec execute 61 sec latency 12.526 ms 2024-05-08T23:08:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:39 smithi086 ceph-mon[78610]: pgmap v148: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:39 smithi053 ceph-mon[77587]: pgmap v148: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:39 smithi046 ceph-mon[36898]: pgmap v148: 129 pgs: 129 active+clean; 190 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-08T23:08:40.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 336187 10.86 MB/sec execute 62 sec latency 8.874 ms 2024-05-08T23:08:41.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 337757 10.82 MB/sec execute 63 sec latency 11.302 ms 2024-05-08T23:08:42.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:41 smithi086 ceph-mon[78610]: pgmap v149: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-05-08T23:08:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:41 smithi053 ceph-mon[77587]: pgmap v149: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-05-08T23:08:42.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:41 smithi046 ceph-mon[36898]: pgmap v149: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-05-08T23:08:42.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 340325 10.87 MB/sec execute 64 sec latency 16.751 ms 2024-05-08T23:08:43.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 341497 10.79 MB/sec execute 65 sec latency 177.110 ms 2024-05-08T23:08:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:43 smithi086 ceph-mon[78610]: pgmap v150: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-08T23:08:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:43 smithi053 ceph-mon[77587]: pgmap v150: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-08T23:08:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:43 smithi046 ceph-mon[36898]: pgmap v150: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-08T23:08:44.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 342865 10.72 MB/sec execute 66 sec latency 115.154 ms 2024-05-08T23:08:45.882 INFO:tasks.workunit.client.0.smithi046.stdout: 1 344203 10.72 MB/sec execute 67 sec latency 63.314 ms 2024-05-08T23:08:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:45 smithi086 ceph-mon[78610]: pgmap v151: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-05-08T23:08:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:45 smithi053 ceph-mon[77587]: pgmap v151: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-05-08T23:08:46.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:45 smithi046 ceph-mon[36898]: pgmap v151: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-05-08T23:08:46.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 346360 10.68 MB/sec execute 68 sec latency 14.002 ms 2024-05-08T23:08:47.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 348075 10.70 MB/sec execute 69 sec latency 19.891 ms 2024-05-08T23:08:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:47 smithi086 ceph-mon[78610]: pgmap v152: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:47 smithi053 ceph-mon[77587]: pgmap v152: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:48.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:47 smithi046 ceph-mon[36898]: pgmap v152: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:48.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 350078 10.66 MB/sec execute 70 sec latency 11.214 ms 2024-05-08T23:08:49.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 351650 10.67 MB/sec execute 71 sec latency 58.023 ms 2024-05-08T23:08:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:49 smithi086 ceph-mon[78610]: pgmap v153: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:49 smithi053 ceph-mon[77587]: pgmap v153: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:50.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:49 smithi046 ceph-mon[36898]: pgmap v153: 129 pgs: 129 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-08T23:08:50.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 354058 10.69 MB/sec execute 72 sec latency 13.798 ms 2024-05-08T23:08:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:08:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:08:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:08:51.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 355613 10.65 MB/sec execute 73 sec latency 10.404 ms 2024-05-08T23:08:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:51 smithi086 ceph-mon[78610]: pgmap v154: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-05-08T23:08:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:51 smithi053 ceph-mon[77587]: pgmap v154: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-05-08T23:08:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:51 smithi046 ceph-mon[36898]: pgmap v154: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-05-08T23:08:52.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 358329 10.74 MB/sec execute 74 sec latency 11.792 ms 2024-05-08T23:08:53.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 360170 10.69 MB/sec execute 75 sec latency 10.726 ms 2024-05-08T23:08:54.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:53 smithi086 ceph-mon[78610]: pgmap v155: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-08T23:08:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:53 smithi053 ceph-mon[77587]: pgmap v155: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-08T23:08:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:53 smithi046 ceph-mon[36898]: pgmap v155: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 272 op/s 2024-05-08T23:08:54.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 362206 10.72 MB/sec execute 76 sec latency 23.822 ms 2024-05-08T23:08:55.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 364393 10.69 MB/sec execute 77 sec latency 17.931 ms 2024-05-08T23:08:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:55 smithi086 ceph-mon[78610]: pgmap v156: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 372 op/s 2024-05-08T23:08:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:55 smithi053 ceph-mon[77587]: pgmap v156: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 372 op/s 2024-05-08T23:08:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:55 smithi046 ceph-mon[36898]: pgmap v156: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 372 op/s 2024-05-08T23:08:56.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 366081 10.70 MB/sec execute 78 sec latency 9.776 ms 2024-05-08T23:08:57.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 368676 10.74 MB/sec execute 79 sec latency 11.411 ms 2024-05-08T23:08:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:57 smithi086 ceph-mon[78610]: pgmap v157: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:08:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:57 smithi053 ceph-mon[77587]: pgmap v157: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:08:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:57 smithi046 ceph-mon[36898]: pgmap v157: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:08:58.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 370308 10.73 MB/sec execute 80 sec latency 15.089 ms 2024-05-08T23:08:59.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 372669 10.77 MB/sec execute 81 sec latency 12.936 ms 2024-05-08T23:09:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:59 smithi086 ceph-mon[78610]: pgmap v158: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:09:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:08:59 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:09:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:59 smithi053 ceph-mon[77587]: pgmap v158: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:09:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:08:59 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:09:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:59 smithi046 ceph-mon[36898]: pgmap v158: 129 pgs: 129 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 293 op/s 2024-05-08T23:09:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:08:59 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:09:00.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 373461 10.66 MB/sec execute 82 sec latency 49.807 ms 2024-05-08T23:09:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:00 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:09:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:00 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:09:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:00 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:09:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:00 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:09:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:00 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:09:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:00 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:09:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:00 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:09:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:00 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:09:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:00 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:09:01.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 375441 10.66 MB/sec execute 83 sec latency 57.387 ms 2024-05-08T23:09:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:01 smithi086 ceph-mon[78610]: pgmap v159: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.2 MiB/s wr, 371 op/s 2024-05-08T23:09:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:01 smithi053 ceph-mon[77587]: pgmap v159: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.2 MiB/s wr, 371 op/s 2024-05-08T23:09:02.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:01 smithi046 ceph-mon[36898]: pgmap v159: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.2 MiB/s wr, 371 op/s 2024-05-08T23:09:02.883 INFO:tasks.workunit.client.0.smithi046.stdout: 1 377034 10.64 MB/sec execute 84 sec latency 14.645 ms 2024-05-08T23:09:03.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 379717 10.70 MB/sec execute 85 sec latency 12.968 ms 2024-05-08T23:09:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:03 smithi086 ceph-mon[78610]: pgmap v160: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:09:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:03 smithi053 ceph-mon[77587]: pgmap v160: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:09:04.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:03 smithi046 ceph-mon[36898]: pgmap v160: 129 pgs: 129 active+clean; 217 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-05-08T23:09:04.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 381655 10.66 MB/sec execute 86 sec latency 7.355 ms 2024-05-08T23:09:05.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 383585 10.69 MB/sec execute 87 sec latency 18.870 ms 2024-05-08T23:09:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:05 smithi086 ceph-mon[78610]: pgmap v161: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-08T23:09:06.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:09:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:05 smithi053 ceph-mon[77587]: pgmap v161: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-08T23:09:06.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:05 smithi046 ceph-mon[36898]: pgmap v161: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-08T23:09:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2677026117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd40ff610 msgr2=0x7f4fd40ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2677026117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd40ff610 0x7f4fd40ff9f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009990 tx=0x7f4fc4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2677026117 shutdown_connections 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2677026117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd41061b0 0x7f4fd410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2677026117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd40ff610 0x7f4fd40ff9f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2677026117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40fffc0 0x7f4fd4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2677026117 >> 172.21.15.46:0/2677026117 conn(0x7f4fd4076360 msgr2=0x7f4fd40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2677026117 shutdown_connections 2024-05-08T23:09:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.460+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2677026117 wait complete. 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.462+0000 7f4fd9e00700 1 Processor -- start 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.462+0000 7f4fd9e00700 1 -- start start 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40ff610 0x7f4fd4100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd4101000 0x7f4fd41a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41a5a40 con 0x7f4fd40ff610 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41a5bb0 con 0x7f4fd4101000 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41a5d70 con 0x7f4fd41061b0 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33482/0 (socket says 172.21.15.46:33482) 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 -- 172.21.15.46:0/2992492158 learned_addr learned my addr 172.21.15.46:0/2992492158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40ff610 0x7f4fd4100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd3400700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd4101000 0x7f4fd41a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 -- 172.21.15.46:0/2992492158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd4101000 msgr2=0x7f4fd41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd4101000 0x7f4fd41a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 -- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40ff610 msgr2=0x7f4fd4100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40ff610 0x7f4fd4100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 -- 172.21.15.46:0/2992492158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40095f0 con 0x7f4fd41061b0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd2000700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e970 tx=0x7f4fc000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd8a00700 1 -- 172.21.15.46:0/2992492158 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0009790 con 0x7f4fd41061b0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd8a00700 1 -- 172.21.15.46:0/2992492158 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc000eec0 con 0x7f4fd41061b0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd41a5ff0 con 0x7f4fd41061b0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd8a00700 1 -- 172.21.15.46:0/2992492158 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0005590 con 0x7f4fd41061b0 2024-05-08T23:09:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.463+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41a64d0 con 0x7f4fd41061b0 2024-05-08T23:09:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.466+0000 7f4fd8a00700 1 -- 172.21.15.46:0/2992492158 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4fc000cbd0 con 0x7f4fd41061b0 2024-05-08T23:09:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fd8a00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 0x7f4fb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fd8a00700 1 -- 172.21.15.46:0/2992492158 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fc0014070 con 0x7f4fd41061b0 2024-05-08T23:09:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 0x7f4fb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fed94eb80 1 Processor -- start 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fed94eb80 1 -- start start 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.468+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 0x7f4fb80a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 0x5585b25ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b2504150 con 0x5585b25b3720 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b2501b10 con 0x5585b25b2e00 2024-05-08T23:09:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b24fd400 con 0x5585b25ad890 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 0x5585b25ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33494/0 (socket says 172.21.15.46:33494) 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 0x5585b25ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47788/0 (socket says 172.21.15.46:47788) 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 -- 172.21.15.46:0/1539524082 learned_addr learned my addr 172.21.15.46:0/1539524082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 -- 172.21.15.46:0/1539524082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 msgr2=0x5585b25b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 -- 172.21.15.46:0/1539524082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 msgr2=0x5585b25ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 0x5585b25ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.469+0000 7f4fd3400700 1 -- 172.21.15.46:0/1539524082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585b237e3b0 con 0x5585b25ad890 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009440 tx=0x7f4fcc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b30e493811b4355 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fbfe00700 1 -- 172.21.15.46:0/1539524082 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00e070 con 0x5585b25ad890 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fbfe00700 1 -- 172.21.15.46:0/1539524082 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc00f040 con 0x5585b25ad890 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fbfe00700 1 -- 172.21.15.46:0/1539524082 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc015b60 con 0x5585b25ad890 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1539524082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 msgr2=0x5585b25adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.470+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009440 tx=0x7f4fcc009920 comp rx=0 tx=0).stop 2024-05-08T23:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1539524082 shutdown_connections 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25ad890 0x5585b25adcd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1539524082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25b3720 0x5585b25ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1539524082 >> 172.21.15.46:0/1539524082 conn(0x5585b250b210 msgr2=0x5585b25a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1539524082 shutdown_connections 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1539524082 wait complete. 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 -- start start 2024-05-08T23:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.471+0000 7f4fed94eb80 1 Processor -- start 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 0x5585b25b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b2499560 con 0x5585b25ad890 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b2499200 con 0x5585b25b2e00 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b23806a0 con 0x5585b25b3720 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fed94eb80 10 client.0 _ll_register_callbacks cb 0x5585b24a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fd2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49316/0 (socket says 172.21.15.46:49316) 2024-05-08T23:09:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fd2000700 1 -- 172.21.15.46:0/1063889406 learned_addr learned my addr 172.21.15.46:0/1063889406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.472+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 0x5585b25b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 -- 172.21.15.46:0/1063889406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 msgr2=0x5585b25b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 0x5585b25b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 -- 172.21.15.46:0/1063889406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 msgr2=0x5585b25b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 -- 172.21.15.46:0/1063889406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc0090f0 con 0x5585b25b2e00 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 0x5585b25b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00ca720 tx=0x7f4fc009a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956417cf67207543 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc009d050 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fc0016530 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.473+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585b2456270 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc00a2580 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 10 client.27044 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585b232ab60 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585b2456270 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4fc0099570 con 0x5585b25b2e00 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 10 client.27044 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 10 client.27044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 20 client.27044 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b25c7200 0x5585b25c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 4 client.27044 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585b257c810 con 0x5585b25c7200 2024-05-08T23:09:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.474+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b25c7200 0x5585b25c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.475+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b25c7200 0x5585b25c9620 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7e4ce787de0781 server_cookie=7d6a17470a961a0f in_seq=0 out_seq=0 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.475+0000 7f4fbcc00700 10 client.27044 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fbcc00700 1 -- 172.21.15.46:0/1063889406 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585b257c810 con 0x5585b25c7200 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fbcc00700 10 client.27044 handle_command_reply: tid=0 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 client.27044 shutdown 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 msgr2=0x5585b25b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00ca720 tx=0x7f4fc009a1c0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 shutdown_connections 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b3720 0x5585b25b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b25c7200 0x5585b25c9620 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1063889406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b2e00 0x5585b25b7070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 >> 172.21.15.46:0/1063889406 conn(0x5585b250b210 msgr2=0x5585b25a75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fbcc00700 0 client.27044 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 shutdown_connections 2024-05-08T23:09:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.476+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1063889406 wait complete. 2024-05-08T23:09:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.477+0000 7f4fed94eb80 20 client.27044 trim_cache size 0 max 16384 2024-05-08T23:09:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.498+0000 7f4fed94eb80 1 Processor -- start 2024-05-08T23:09:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 -- start start 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25b3560 con 0x5585b25ad890 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25ce830 con 0x5585b25b3720 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25a0040 con 0x5585b25b2e00 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fd2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fd2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33524/0 (socket says 172.21.15.46:33524) 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fd2000700 1 -- 172.21.15.46:0/1462778281 learned_addr learned my addr 172.21.15.46:0/1462778281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.499+0000 7f4fd3400700 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 -- 172.21.15.46:0/1462778281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 msgr2=0x5585b25a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 -- 172.21.15.46:0/1462778281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 msgr2=0x5585b25ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 -- 172.21.15.46:0/1462778281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585b24c3560 con 0x5585b25b2e00 2024-05-08T23:09:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fd2000700 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fc009a010 tx=0x7f4fc0099bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154f9f17e76d9c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fc9200700 1 -- 172.21.15.46:0/1462778281 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc009d070 con 0x5585b25b2e00 2024-05-08T23:09:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fc9200700 1 -- 172.21.15.46:0/1462778281 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc00a2110 con 0x5585b25b2e00 2024-05-08T23:09:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.500+0000 7f4fc9200700 1 -- 172.21.15.46:0/1462778281 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0016330 con 0x5585b25b2e00 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1462778281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 msgr2=0x5585b25adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fc009a010 tx=0x7f4fc0099bc0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1462778281 shutdown_connections 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/1462778281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25adc70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1462778281 >> 172.21.15.46:0/1462778281 conn(0x5585b250b210 msgr2=0x5585b25b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1462778281 shutdown_connections 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.501+0000 7f4fed94eb80 1 -- 172.21.15.46:0/1462778281 wait complete. 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.502+0000 7f4fed94eb80 1 -- start start 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.502+0000 7f4fed94eb80 1 Processor -- start 2024-05-08T23:09:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25af7d0 con 0x5585b25ad890 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25af940 con 0x5585b25b3720 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b25afb00 con 0x5585b25b2e00 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fed94eb80 10 client.0 _ll_register_callbacks cb 0x5585b24a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47810/0 (socket says 172.21.15.46:47810) 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 -- 172.21.15.46:0/2202597957 learned_addr learned my addr 172.21.15.46:0/2202597957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 -- 172.21.15.46:0/2202597957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 msgr2=0x5585b25ae9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25ae9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 -- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 msgr2=0x5585b25af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.503+0000 7f4fd2a00700 1 -- 172.21.15.46:0/2202597957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585b232ab60 con 0x5585b25ad890 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fd3400700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fd2a00700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009990 tx=0x7f4fc4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf15627b2dbdb7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40362c0 con 0x5585b25ad890 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fcc00f040 con 0x5585b25ad890 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4fc4036420 con 0x5585b25ad890 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc40365e0 con 0x5585b25ad890 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fed94eb80 10 client.17058 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.504+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585b2382cd0 con 0x5585b25ad890 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc00f040 con 0x5585b25ad890 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4fc4004d40 con 0x5585b25ad890 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 20 client.17058 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b261df00 0x5585b2620320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 4 client.17058 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585b24f5ec0 con 0x5585b261df00 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.505+0000 7f4fd2000700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b261df00 0x5585b2620320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fd2000700 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b261df00 0x5585b2620320 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a001daef33b7aaf server_cookie=1b3cfe3995b9885c in_seq=0 out_seq=0 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fbcc00700 10 client.17058 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fbcc00700 1 -- 172.21.15.46:0/2202597957 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5585b24f5ec0 con 0x5585b261df00 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fbcc00700 10 client.17058 handle_command_reply: tid=0 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 client.17058 shutdown 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 msgr2=0x5585b25be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009990 tx=0x7f4fc4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 shutdown_connections 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585b25b3720 0x5585b25af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5585b261df00 0x5585b2620320 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585b25ad890 0x5585b25be650 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 --2- 172.21.15.46:0/2202597957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585b25b2e00 0x5585b25ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fbcc00700 0 client.17058 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.506+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 >> 172.21.15.46:0/2202597957 conn(0x5585b250b210 msgr2=0x5585b25b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.507+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 shutdown_connections 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.507+0000 7f4fed94eb80 1 -- 172.21.15.46:0/2202597957 wait complete. 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.507+0000 7f4fed94eb80 20 client.17058 trim_cache size 0 max 16384 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:09:06.511 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:09:06.512 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 msgr2=0x7f4fb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 0x7f4fb80a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc400b260 comp rx=0 tx=0).stop 2024-05-08T23:09:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 msgr2=0x7f4fd4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e970 tx=0x7f4fc000ec80 comp rx=0 tx=0).stop 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 shutdown_connections 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd4101000 0x7f4fd41a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4fb80a1290 0x7f4fb80a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fd40ff610 0x7f4fd4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 --2- 172.21.15.46:0/2992492158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd41061b0 0x7f4fd4100ac0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 >> 172.21.15.46:0/2992492158 conn(0x7f4fd4076360 msgr2=0x7f4fd40fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 shutdown_connections 2024-05-08T23:09:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.508+0000 7f4fd9e00700 1 -- 172.21.15.46:0/2992492158 wait complete. 2024-05-08T23:09:06.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:09:06.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 -- 172.21.15.46:0/1511618473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4974101fb0 msgr2=0x7f4974102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 --2- 172.21.15.46:0/1511618473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4974101fb0 0x7f4974102390 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4968009990 tx=0x7f4968009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 -- 172.21.15.46:0/1511618473 shutdown_connections 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 --2- 172.21.15.46:0/1511618473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497410b3d0 0x7f497410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 --2- 172.21.15.46:0/1511618473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4974101fb0 0x7f4974102390 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 --2- 172.21.15.46:0/1511618473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f497410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 -- 172.21.15.46:0/1511618473 >> 172.21.15.46:0/1511618473 conn(0x7f49740fb7f0 msgr2=0x7f49740fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:06.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 -- 172.21.15.46:0/1511618473 shutdown_connections 2024-05-08T23:09:06.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.749+0000 7f497c800700 1 -- 172.21.15.46:0/1511618473 wait complete. 2024-05-08T23:09:06.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.751+0000 7f497c800700 1 Processor -- start 2024-05-08T23:09:06.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.751+0000 7f497c800700 1 -- start start 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.751+0000 7f497c800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.751+0000 7f497c800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f4974197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497c800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 0x7f497419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497c800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419c230 con 0x7f4974101fb0 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497c800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419c3a0 con 0x7f49741028d0 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497c800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419c560 con 0x7f497410b3d0 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f4974197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47832/0 (socket says 172.21.15.46:47832) 2024-05-08T23:09:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979e00700 1 -- 172.21.15.46:0/1752171626 learned_addr learned my addr 172.21.15.46:0/1752171626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497a800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 0x7f497419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 -- 172.21.15.46:0/1752171626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 msgr2=0x7f497419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 0x7f497419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 -- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 msgr2=0x7f4974197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979400700 1 -- 172.21.15.46:0/1752171626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680095f0 con 0x7f49741028d0 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f4979e00700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.752+0000 7f497a800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 0x7f497419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f4979400700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f4974197850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f496400e9f0 tx=0x7f496400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964004220 con 0x7f49741028d0 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497419c840 con 0x7f49741028d0 2024-05-08T23:09:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49640095c0 con 0x7f49741028d0 2024-05-08T23:09:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400cda0 con 0x7f49741028d0 2024-05-08T23:09:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.753+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49741a4080 con 0x7f49741028d0 2024-05-08T23:09:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.754+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49741064d0 con 0x7f49741028d0 2024-05-08T23:09:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.757+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4964004750 con 0x7f49741028d0 2024-05-08T23:09:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.758+0000 7f496ea00700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 0x7f495c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.758+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4964014070 con 0x7f49741028d0 2024-05-08T23:09:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.758+0000 7f4979e00700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 0x7f495c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.759+0000 7f4979e00700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 0x7f495c0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4968006010 tx=0x7f496800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:06.760+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4964096310 con 0x7f49741028d0 2024-05-08T23:09:06.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 386028 10.70 MB/sec execute 88 sec latency 10.390 ms 2024-05-08T23:09:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.082+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4974069d10 con 0x7f49741028d0 2024-05-08T23:09:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.084+0000 7f496ea00700 1 -- 172.21.15.46:0/1752171626 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f4964099930 con 0x7f49741028d0 2024-05-08T23:09:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:09:07.083 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:09:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 msgr2=0x7f495c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 0x7f495c0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4968006010 tx=0x7f496800b260 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 msgr2=0x7f4974197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f4974197850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f496400e9f0 tx=0x7f496400ed00 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 shutdown_connections 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f497410b3d0 0x7f497419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f495c0a1290 0x7f495c0a3750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4974101fb0 0x7f4974197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 --2- 172.21.15.46:0/1752171626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741028d0 0x7f4974197850 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 >> 172.21.15.46:0/1752171626 conn(0x7f49740fb7f0 msgr2=0x7f49740fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 shutdown_connections 2024-05-08T23:09:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.087+0000 7f497c800700 1 -- 172.21.15.46:0/1752171626 wait complete. 2024-05-08T23:09:07.088 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:09:07.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:09:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.331+0000 7f53c9c00700 1 -- 172.21.15.46:0/783133907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073340 msgr2=0x7f53c4073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.331+0000 7f53c9c00700 1 --2- 172.21.15.46:0/783133907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073340 0x7f53c4073720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009990 tx=0x7f53b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 -- 172.21.15.46:0/783133907 shutdown_connections 2024-05-08T23:09:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 --2- 172.21.15.46:0/783133907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4111e20 0x7f53c4114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 --2- 172.21.15.46:0/783133907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073340 0x7f53c4073720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 --2- 172.21.15.46:0/783133907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4073c60 0x7f53c41118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 -- 172.21.15.46:0/783133907 >> 172.21.15.46:0/783133907 conn(0x7f53c40fc8e0 msgr2=0x7f53c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 -- 172.21.15.46:0/783133907 shutdown_connections 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.332+0000 7f53c9c00700 1 -- 172.21.15.46:0/783133907 wait complete. 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 Processor -- start 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 -- start start 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073c60 0x7f53c41a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4111e20 0x7f53c419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c419dd90 con 0x7f53c4111e20 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c419df00 con 0x7f53c4073340 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.333+0000 7f53c9c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c419e0c0 con 0x7f53c4073c60 2024-05-08T23:09:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49392/0 (socket says 172.21.15.46:49392) 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 -- 172.21.15.46:0/775850449 learned_addr learned my addr 172.21.15.46:0/775850449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53bbe00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073c60 0x7f53c41a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c3400700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4111e20 0x7f53c419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 -- 172.21.15.46:0/775850449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073c60 msgr2=0x7f53c41a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073c60 0x7f53c41a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 -- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4111e20 msgr2=0x7f53c419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4111e20 0x7f53c419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 -- 172.21.15.46:0/775850449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40095f0 con 0x7f53c4073340 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.334+0000 7f53c2a00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000c00 tx=0x7f53b4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.335+0000 7f53bb400700 1 -- 172.21.15.46:0/775850449 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b401d070 con 0x7f53c4073340 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.335+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c419e340 con 0x7f53c4073340 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.335+0000 7f53bb400700 1 -- 172.21.15.46:0/775850449 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b4022470 con 0x7f53c4073340 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.335+0000 7f53bb400700 1 -- 172.21.15.46:0/775850449 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4017650 con 0x7f53c4073340 2024-05-08T23:09:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.335+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41aa5f0 con 0x7f53c4073340 2024-05-08T23:09:07.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.338+0000 7f53bb400700 1 -- 172.21.15.46:0/775850449 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f53b40225e0 con 0x7f53c4073340 2024-05-08T23:09:07.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.339+0000 7f53dd79db80 1 Processor -- start 2024-05-08T23:09:07.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 -- start start 2024-05-08T23:09:07.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9bc7150 con 0x564ba9c70890 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9bc4b10 con 0x564ba9c76720 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53dd79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9bc0400 con 0x564ba9c75e00 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53bb400700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 0x7f53a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33568/0 (socket says 172.21.15.46:33568) 2024-05-08T23:09:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53c3400700 1 -- 172.21.15.46:0/1177762237 learned_addr learned my addr 172.21.15.46:0/1177762237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53bbe00700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53bb400700 1 -- 172.21.15.46:0/775850449 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f53b40ce4a0 con 0x7f53c4073340 2024-05-08T23:09:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.340+0000 7f53c2a00700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 -- 172.21.15.46:0/1177762237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 msgr2=0x564ba9c70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 -- 172.21.15.46:0/1177762237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 msgr2=0x564ba9c70cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 -- 172.21.15.46:0/1177762237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba9a413b0 con 0x564ba9c75e00 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53bbe00700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 0x7f53a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53c3400700 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f53bc013690 tx=0x7f53bc0139a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae1d3dab4b266a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.341+0000 7f53b3e00700 1 -- 172.21.15.46:0/1177762237 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00e040 con 0x564ba9c75e00 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53b3e00700 1 -- 172.21.15.46:0/1177762237 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53bc027070 con 0x564ba9c75e00 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53b3e00700 1 -- 172.21.15.46:0/1177762237 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0238f0 con 0x564ba9c75e00 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53c3400700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 0x7f53a80a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53bc022000 tx=0x7f53bc01f000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53dd79db80 1 -- 172.21.15.46:0/1177762237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 msgr2=0x564ba9c761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53dd79db80 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f53bc013690 tx=0x7f53bc0139a0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53dd79db80 1 -- 172.21.15.46:0/1177762237 shutdown_connections 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.342+0000 7f53dd79db80 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c761e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 --2- 172.21.15.46:0/1177762237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 -- 172.21.15.46:0/1177762237 >> 172.21.15.46:0/1177762237 conn(0x564ba9bce210 msgr2=0x564ba9c6adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 -- 172.21.15.46:0/1177762237 shutdown_connections 2024-05-08T23:09:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 -- 172.21.15.46:0/1177762237 wait complete. 2024-05-08T23:09:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.343+0000 7f53dd79db80 1 -- start start 2024-05-08T23:09:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 Processor -- start 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c75e00 0x564ba9c7a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c76720 0x564ba9c7a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9b5c560 con 0x564ba9c76720 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9b5c200 con 0x564ba9c75e00 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9a436a0 con 0x564ba9c70890 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53dd79db80 10 client.0 _ll_register_callbacks cb 0x564ba9b65b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.344+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33580/0 (socket says 172.21.15.46:33580) 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 -- 172.21.15.46:0/2102501552 learned_addr learned my addr 172.21.15.46:0/2102501552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c3400700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c76720 0x564ba9c7a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c75e00 0x564ba9c7a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 -- 172.21.15.46:0/2102501552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c75e00 msgr2=0x564ba9c7a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c75e00 0x564ba9c7a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 -- 172.21.15.46:0/2102501552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c76720 msgr2=0x564ba9c7a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c76720 0x564ba9c7a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 -- 172.21.15.46:0/2102501552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc013370 con 0x564ba9c70890 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f53b4097dc0 tx=0x7f53b40cbd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949e4c9cf6f8e05f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b409acb0 con 0x564ba9c70890 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f53b40b0030 con 0x564ba9c70890 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.345+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ba9b19270 con 0x564ba9c70890 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b409ba50 con 0x564ba9c70890 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 10 client.27127 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564ba99edb60 con 0x564ba9c70890 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ba9b19270 con 0x564ba9c70890 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53b409a840 con 0x564ba9c70890 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 10 client.27127 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 10 client.27127 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 20 client.27127 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9c89dc0 0x564ba9c8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 4 client.27127 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ba9c3f810 con 0x564ba9c89dc0 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.346+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9c89dc0 0x564ba9c8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.347+0000 7f53b0c00700 10 client.27127 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.347+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9c89dc0 0x564ba9c8c1e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aea8cb85dcf32715 server_cookie=949601ba3641535e in_seq=0 out_seq=0 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.347+0000 7f53b0c00700 1 -- 172.21.15.46:0/2102501552 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ba9c3f810 con 0x564ba9c89dc0 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.347+0000 7f53b0c00700 10 client.27127 handle_command_reply: tid=0 2024-05-08T23:09:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 client.27127 shutdown 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 msgr2=0x564ba9c79b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f53b4097dc0 tx=0x7f53b40cbd40 comp rx=0 tx=0).stop 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 shutdown_connections 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9c89dc0 0x564ba9c8c1e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c76720 0x564ba9c7a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c70890 0x564ba9c79b80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2102501552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c75e00 0x564ba9c7a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53b0c00700 0 client.27127 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 >> 172.21.15.46:0/2102501552 conn(0x564ba9bce210 msgr2=0x564ba9c6a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 shutdown_connections 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 1 -- 172.21.15.46:0/2102501552 wait complete. 2024-05-08T23:09:07.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.348+0000 7f53dd79db80 20 client.27127 trim_cache size 0 max 16384 2024-05-08T23:09:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.362+0000 7f53dd79db80 1 Processor -- start 2024-05-08T23:09:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.362+0000 7f53dd79db80 1 -- start start 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c78ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c76560 con 0x564ba9c70890 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c913f0 con 0x564ba9c76720 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53dd79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c79220 con 0x564ba9c75e00 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c78ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47882/0 (socket says 172.21.15.46:47882) 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53c2a00700 1 -- 172.21.15.46:0/2265803411 learned_addr learned my addr 172.21.15.46:0/2265803411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53c3400700 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 -- 172.21.15.46:0/2265803411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 msgr2=0x564ba9c6ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c6ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 -- 172.21.15.46:0/2265803411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 msgr2=0x564ba9c70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53bbe00700 1 -- 172.21.15.46:0/2265803411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba9b86560 con 0x564ba9c75e00 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.363+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.364+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c78ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x564ba9c8ca90 tx=0x7f53ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74accec527c453de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.364+0000 7f53b8c00700 1 -- 172.21.15.46:0/2265803411 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac016030 con 0x564ba9c75e00 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.364+0000 7f53b8c00700 1 -- 172.21.15.46:0/2265803411 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac017040 con 0x564ba9c75e00 2024-05-08T23:09:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.364+0000 7f53b8c00700 1 -- 172.21.15.46:0/2265803411 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac010720 con 0x564ba9c75e00 2024-05-08T23:09:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- 172.21.15.46:0/2265803411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 msgr2=0x564ba9c78ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c78ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x564ba9c8ca90 tx=0x7f53ac009260 comp rx=0 tx=0).stop 2024-05-08T23:09:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- 172.21.15.46:0/2265803411 shutdown_connections 2024-05-08T23:09:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c76720 0x564ba9c6ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c70890 0x564ba9c70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2265803411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c75e00 0x564ba9c78ce0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- 172.21.15.46:0/2265803411 >> 172.21.15.46:0/2265803411 conn(0x564ba9bce210 msgr2=0x564ba9c74c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- 172.21.15.46:0/2265803411 shutdown_connections 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- 172.21.15.46:0/2265803411 wait complete. 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 -- start start 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.365+0000 7f53dd79db80 1 Processor -- start 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c75e00 0x564ba9c813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 0x564ba9c7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c7fc20 con 0x564ba9c75e00 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c7fd90 con 0x564ba9c70890 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba9c7ff50 con 0x564ba9c76720 2024-05-08T23:09:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53dd79db80 10 client.0 _ll_register_callbacks cb 0x564ba9b65b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53c2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49418/0 (socket says 172.21.15.46:49418) 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 0x564ba9c7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53c2a00700 1 -- 172.21.15.46:0/2812291196 learned_addr learned my addr 172.21.15.46:0/2812291196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.366+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c75e00 0x564ba9c813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 -- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 msgr2=0x564ba9c7f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 0x564ba9c7f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 -- 172.21.15.46:0/2812291196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c75e00 msgr2=0x564ba9c813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c75e00 0x564ba9c813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 -- 172.21.15.46:0/2812291196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ba99edb60 con 0x564ba9c70890 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c3400700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 0x564ba9c7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53c2a00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53b4097dc0 tx=0x7f53b40ce380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fb5efb9a47f8ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400fd00 con 0x564ba9c70890 2024-05-08T23:09:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53bc027070 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f53b40b0070 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b400fe60 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53dd79db80 10 client.27074 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.367+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f53ac009b40 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53bc027070 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53b4097140 con 0x564ba9c70890 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 10 client.27074 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 10 client.27074 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 20 client.27074 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9ce1080 0x564ba9ce34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 4 client.27074 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ba9bb8ec0 con 0x564ba9ce1080 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.368+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9ce1080 0x564ba9ce34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53bbe00700 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9ce1080 0x564ba9ce34a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53507b9b4485a982 server_cookie=2b6ada783aaf3e85 in_seq=0 out_seq=0 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53b0c00700 10 client.27074 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53b0c00700 1 -- 172.21.15.46:0/2812291196 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ba9bb8ec0 con 0x564ba9ce1080 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53b0c00700 10 client.27074 handle_command_reply: tid=0 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 client.27074 shutdown 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 msgr2=0x564ba9c80e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53b4097dc0 tx=0x7f53b40ce380 comp rx=0 tx=0).stop 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 shutdown_connections 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564ba9ce1080 0x564ba9ce34a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ba9c76720 0x564ba9c7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba9c70890 0x564ba9c80e80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 --2- 172.21.15.46:0/2812291196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba9c75e00 0x564ba9c813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53b0c00700 0 client.27074 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 >> 172.21.15.46:0/2812291196 conn(0x564ba9bce210 msgr2=0x564ba9c75490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 shutdown_connections 2024-05-08T23:09:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.369+0000 7f53dd79db80 1 -- 172.21.15.46:0/2812291196 wait complete. 2024-05-08T23:09:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53dd79db80 20 client.27074 trim_cache size 0 max 16384 2024-05-08T23:09:07.374 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 msgr2=0x7f53a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 0x7f53a80a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53bc022000 tx=0x7f53bc01f000 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 msgr2=0x7f53c41a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000c00 tx=0x7f53b4005740 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 shutdown_connections 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f53a80a1290 0x7f53a80a3750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c4111e20 0x7f53c419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4073340 0x7f53c41a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 --2- 172.21.15.46:0/775850449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53c4073c60 0x7f53c41a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 >> 172.21.15.46:0/775850449 conn(0x7f53c40fc8e0 msgr2=0x7f53c40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.370+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 shutdown_connections 2024-05-08T23:09:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.371+0000 7f53c9c00700 1 -- 172.21.15.46:0/775850449 wait complete. 2024-05-08T23:09:07.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 -- 172.21.15.46:0/2624261437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3938101080 msgr2=0x7f3938101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 --2- 172.21.15.46:0/2624261437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3938101080 0x7f3938101460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 -- 172.21.15.46:0/2624261437 shutdown_connections 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 --2- 172.21.15.46:0/2624261437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39381061a0 0x7f393810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 --2- 172.21.15.46:0/2624261437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3938101080 0x7f3938101460 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 --2- 172.21.15.46:0/2624261437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3938101a30 0x7f3938105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 -- 172.21.15.46:0/2624261437 >> 172.21.15.46:0/2624261437 conn(0x7f39380fc8e0 msgr2=0x7f39380fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 -- 172.21.15.46:0/2624261437 shutdown_connections 2024-05-08T23:09:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.603+0000 7f393ee00700 1 -- 172.21.15.46:0/2624261437 wait complete. 2024-05-08T23:09:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.604+0000 7f393ee00700 1 Processor -- start 2024-05-08T23:09:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.604+0000 7f393ee00700 1 -- start start 2024-05-08T23:09:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 0x7f3938199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 0x7f393819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393819e890 con 0x7f3938101080 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393819ea00 con 0x7f393819de10 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393819ebc0 con 0x7f39381061a0 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47914/0 (socket says 172.21.15.46:47914) 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 -- 172.21.15.46:0/2827650461 learned_addr learned my addr 172.21.15.46:0/2827650461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937400700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 0x7f3938199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f393ce00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 0x7f393819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 -- 172.21.15.46:0/2827650461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 msgr2=0x7f3938199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 0x7f3938199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 -- 172.21.15.46:0/2827650461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 msgr2=0x7f393819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.605+0000 7f3937e00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 0x7f393819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3937e00700 1 -- 172.21.15.46:0/2827650461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280095f0 con 0x7f3938101080 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f393ce00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 0x7f393819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3937400700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 0x7f3938199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3937e00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3934c00700 1 -- 172.21.15.46:0/2827650461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392801d070 con 0x7f3938101080 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39381a60c0 con 0x7f3938101080 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3934c00700 1 -- 172.21.15.46:0/2827650461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3928022470 con 0x7f3938101080 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f3934c00700 1 -- 172.21.15.46:0/2827650461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928017650 con 0x7f3938101080 2024-05-08T23:09:07.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.606+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381a64e0 con 0x7f3938101080 2024-05-08T23:09:07.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.609+0000 7f3934c00700 1 -- 172.21.15.46:0/2827650461 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39280225e0 con 0x7f3938101080 2024-05-08T23:09:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.611+0000 7f3934c00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 0x7f391c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.611+0000 7f3934c00700 1 -- 172.21.15.46:0/2827650461 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f39280ce250 con 0x7f3938101080 2024-05-08T23:09:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.611+0000 7f3937400700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 0x7f391c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f3937400700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 0x7f391c0a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f393819aa80 tx=0x7f3924009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 Processor -- start 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 -- start start 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5c720 0x559099b56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099aad150 con 0x559099b5be00 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099aaab10 con 0x559099b5c720 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f39529eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099aa6400 con 0x559099b56890 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f393ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.612+0000 7f3937e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33636/0 (socket says 172.21.15.46:33636) 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f3937e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47924/0 (socket says 172.21.15.46:47924) 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 -- 172.21.15.46:0/2657265556 learned_addr learned my addr 172.21.15.46:0/2657265556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f3937400700 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5c720 0x559099b56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 -- 172.21.15.46:0/2657265556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5c720 msgr2=0x559099b56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5c720 0x559099b56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 -- 172.21.15.46:0/2657265556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 msgr2=0x559099b5c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 -- 172.21.15.46:0/2657265556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590999273b0 con 0x559099b56890 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f3937e00700 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f393ce00700 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3930009440 tx=0x7f3930009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1af3d0ea8fd27d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f392d600700 1 -- 172.21.15.46:0/2657265556 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393000e070 con 0x559099b56890 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f392d600700 1 -- 172.21.15.46:0/2657265556 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f393000f040 con 0x559099b56890 2024-05-08T23:09:07.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.613+0000 7f392d600700 1 -- 172.21.15.46:0/2657265556 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930015c10 con 0x559099b56890 2024-05-08T23:09:07.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 -- 172.21.15.46:0/2657265556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 msgr2=0x559099b56cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3930009440 tx=0x7f3930009920 comp rx=0 tx=0).stop 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 -- 172.21.15.46:0/2657265556 shutdown_connections 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56cd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2657265556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5c720 0x559099b56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 -- 172.21.15.46:0/2657265556 >> 172.21.15.46:0/2657265556 conn(0x559099ab4210 msgr2=0x559099b50df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 -- 172.21.15.46:0/2657265556 shutdown_connections 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.614+0000 7f39529eeb80 1 -- 172.21.15.46:0/2657265556 wait complete. 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.615+0000 7f39529eeb80 1 -- start start 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.615+0000 7f39529eeb80 1 Processor -- start 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 0x559099b604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099a42560 con 0x559099b5be00 2024-05-08T23:09:07.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099a42200 con 0x559099b56890 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590999296a0 con 0x559099b5c720 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f39529eeb80 10 client.0 _ll_register_callbacks cb 0x559099a4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f3937400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 0x559099b604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f3937400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47926/0 (socket says 172.21.15.46:47926) 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f3937400700 1 -- 172.21.15.46:0/2716704852 learned_addr learned my addr 172.21.15.46:0/2716704852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f3937e00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 -- 172.21.15.46:0/2716704852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 msgr2=0x559099b5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 -- 172.21.15.46:0/2716704852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 msgr2=0x559099b5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f393ce00700 1 -- 172.21.15.46:0/2716704852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381a64e0 con 0x559099b5c720 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.616+0000 7f3937400700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f393ce00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 0x559099b604f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3930009410 tx=0x7f3930016a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ccfed927ae1329 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f3922000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393000e040 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590999ff270 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3930003c00 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39300095e0 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f39529eeb80 10 client.27145 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.617+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f39300090f0 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590999ff270 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f393001c050 con 0x559099b5c720 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 10 client.27145 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 10 client.27145 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 20 client.27145 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099b70180 0x559099b725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 4 client.27145 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:07.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559099b25810 con 0x559099b70180 2024-05-08T23:09:07.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.618+0000 7f3937e00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099b70180 0x559099b725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.619+0000 7f3937e00700 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099b70180 0x559099b725a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2a335f987d0013b server_cookie=60c6fef44dfe3a0b in_seq=0 out_seq=0 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.619+0000 7f3922000700 10 client.27145 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.619+0000 7f3922000700 1 -- 172.21.15.46:0/2716704852 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559099b25810 con 0x559099b70180 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.619+0000 7f3922000700 10 client.27145 handle_command_reply: tid=0 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.619+0000 7f39529eeb80 1 client.27145 shutdown 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 msgr2=0x559099b604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 0x559099b604f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3930009410 tx=0x7f3930016a30 comp rx=0 tx=0).stop 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 shutdown_connections 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099b70180 0x559099b725a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5c720 0x559099b604f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2716704852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5be00 0x559099b5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f3922000700 0 client.27145 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 >> 172.21.15.46:0/2716704852 conn(0x559099ab4210 msgr2=0x559099b505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 shutdown_connections 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 1 -- 172.21.15.46:0/2716704852 wait complete. 2024-05-08T23:09:07.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.620+0000 7f39529eeb80 20 client.27145 trim_cache size 0 max 16384 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 Processor -- start 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 -- start start 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 0x559099b50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099b5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099b51a90 con 0x559099b5c720 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099b4ddc0 con 0x559099b5be00 2024-05-08T23:09:07.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f39529eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099b62b30 con 0x559099b56890 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f3937e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f3937400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 0x559099b50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f3937e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33660/0 (socket says 172.21.15.46:33660) 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f3937e00700 1 -- 172.21.15.46:0/1535521259 learned_addr learned my addr 172.21.15.46:0/1535521259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.636+0000 7f393ce00700 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099b5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 -- 172.21.15.46:0/1535521259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 msgr2=0x559099b56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 -- 172.21.15.46:0/1535521259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 msgr2=0x559099b5b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099b5b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 -- 172.21.15.46:0/1535521259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590998d3b60 con 0x559099b5be00 2024-05-08T23:09:07.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f3937400700 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 0x559099b50d20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559099b60fb0 tx=0x7f3924014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a004e324c1be56d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f392e000700 1 -- 172.21.15.46:0/1535521259 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924024070 con 0x559099b5be00 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f392e000700 1 -- 172.21.15.46:0/1535521259 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392401e8b0 con 0x559099b5be00 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.637+0000 7f392e000700 1 -- 172.21.15.46:0/1535521259 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39240287c0 con 0x559099b5be00 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- 172.21.15.46:0/1535521259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 msgr2=0x559099b50d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 0x559099b50d20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559099b60fb0 tx=0x7f3924014000 comp rx=0 tx=0).stop 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- 172.21.15.46:0/1535521259 shutdown_connections 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099b5b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b56890 0x559099b56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 --2- 172.21.15.46:0/1535521259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b5be00 0x559099b50d20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- 172.21.15.46:0/1535521259 >> 172.21.15.46:0/1535521259 conn(0x559099ab4210 msgr2=0x559099b5ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- 172.21.15.46:0/1535521259 shutdown_connections 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- 172.21.15.46:0/1535521259 wait complete. 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.638+0000 7f39529eeb80 1 -- start start 2024-05-08T23:09:07.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 Processor -- start 2024-05-08T23:09:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5be00 0x559099b65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099bbf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099bbfbc0 con 0x559099b5c720 2024-05-08T23:09:07.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099bbfd30 con 0x559099b56890 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559099bbfef0 con 0x559099b5be00 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f39529eeb80 10 client.0 _ll_register_callbacks cb 0x559099a4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f3937e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f3937400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5be00 0x559099b65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f3937e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49480/0 (socket says 172.21.15.46:49480) 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f3937e00700 1 -- 172.21.15.46:0/2909369636 learned_addr learned my addr 172.21.15.46:0/2909369636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.639+0000 7f393ce00700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099bbf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 -- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5be00 msgr2=0x559099b65a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5be00 0x559099b65a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 -- 172.21.15.46:0/2909369636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 msgr2=0x559099bbf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099bbf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 -- 172.21.15.46:0/2909369636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3924017070 con 0x559099b56890 2024-05-08T23:09:07.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3937e00700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3928097a50 tx=0x7f39280a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca2308625b9fb20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3922000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392809b6e0 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f393000f040 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f3928084040 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f392809afe0 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f39529eeb80 10 client.27092 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.640+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559099a6c560 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f393000f040 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f392809b240 con 0x559099b56890 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 10 client.27092 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 10 client.27092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:09:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 20 client.27092 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099bc73e0 0x559099bc97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 4 client.27092 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559099a9eec0 con 0x559099bc73e0 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.641+0000 7f3937400700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099bc73e0 0x559099bc97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f3937400700 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099bc73e0 0x559099bc97d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd468c49d0324153 server_cookie=37515b5a9f4473b3 in_seq=0 out_seq=0 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f3922000700 10 client.27092 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f3922000700 1 -- 172.21.15.46:0/2909369636 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559099a9eec0 con 0x559099bc73e0 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f3922000700 10 client.27092 handle_command_reply: tid=0 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 client.27092 shutdown 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 msgr2=0x559099b65520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3928097a50 tx=0x7f39280a1040 comp rx=0 tx=0).stop 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 shutdown_connections 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559099bc73e0 0x559099bc97d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559099b5c720 0x559099bbf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559099b56890 0x559099b65520 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 --2- 172.21.15.46:0/2909369636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559099b5be00 0x559099b65a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f3922000700 0 client.27092 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 >> 172.21.15.46:0/2909369636 conn(0x559099ab4210 msgr2=0x559099b5df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 shutdown_connections 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 1 -- 172.21.15.46:0/2909369636 wait complete. 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.642+0000 7f39529eeb80 20 client.27092 trim_cache size 0 max 16384 2024-05-08T23:09:07.648 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 msgr2=0x7f391c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 0x7f391c0a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f393819aa80 tx=0x7f3924009260 comp rx=0 tx=0).stop 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 msgr2=0x7f3938199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928005740 comp rx=0 tx=0).stop 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 shutdown_connections 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393819de10 0x7f393819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f391c0a1290 0x7f391c0a3750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3938101080 0x7f3938199580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 --2- 172.21.15.46:0/2827650461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39381061a0 0x7f3938199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 >> 172.21.15.46:0/2827650461 conn(0x7f39380fc8e0 msgr2=0x7f39380fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 shutdown_connections 2024-05-08T23:09:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.643+0000 7f393ee00700 1 -- 172.21.15.46:0/2827650461 wait complete. 2024-05-08T23:09:07.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:09:07.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 387579 10.67 MB/sec execute 89 sec latency 11.735 ms 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 -- 172.21.15.46:0/1958939234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8101080 msgr2=0x7f5bd8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1958939234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8101080 0x7f5bd8101460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80098f0 tx=0x7f5bc8009c00 comp rx=0 tx=0).stop 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 -- 172.21.15.46:0/1958939234 shutdown_connections 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1958939234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1958939234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8101080 0x7f5bd8101460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1958939234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101a30 0x7f5bd8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.899+0000 7f5bdce00700 1 -- 172.21.15.46:0/1958939234 >> 172.21.15.46:0/1958939234 conn(0x7f5bd80fc8e0 msgr2=0x7f5bd80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.900+0000 7f5bdce00700 1 -- 172.21.15.46:0/1958939234 shutdown_connections 2024-05-08T23:09:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.900+0000 7f5bdce00700 1 -- 172.21.15.46:0/1958939234 wait complete. 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.900+0000 7f5bdce00700 1 Processor -- start 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 -- start start 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 0x7f5bd80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8073d20 0x7f5bd81a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a5b50 con 0x7f5bd8101080 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a5cc0 con 0x7f5bd8073d20 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bdce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a5e80 con 0x7f5bd81061a0 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 0x7f5bd80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bd5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bd5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33686/0 (socket says 172.21.15.46:33686) 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bd5400700 1 -- 172.21.15.46:0/1574066335 learned_addr learned my addr 172.21.15.46:0/1574066335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.901+0000 7f5bd6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8073d20 0x7f5bd81a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 -- 172.21.15.46:0/1574066335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8073d20 msgr2=0x7f5bd81a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8073d20 0x7f5bd81a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 -- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 msgr2=0x7f5bd80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 0x7f5bd80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 -- 172.21.15.46:0/1574066335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009520 con 0x7f5bd81061a0 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 0x7f5bd80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bd5400700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400ea30 tx=0x7f5bc400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bcea00700 1 -- 172.21.15.46:0/1574066335 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400cbd0 con 0x7f5bd81061a0 2024-05-08T23:09:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8009520 con 0x7f5bd81061a0 2024-05-08T23:09:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bcea00700 1 -- 172.21.15.46:0/1574066335 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc400eed0 con 0x7f5bd81061a0 2024-05-08T23:09:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bcea00700 1 -- 172.21.15.46:0/1574066335 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003d20 con 0x7f5bd81061a0 2024-05-08T23:09:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.902+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81a64e0 con 0x7f5bd81061a0 2024-05-08T23:09:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.906+0000 7f5bcea00700 1 -- 172.21.15.46:0/1574066335 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5bc4003ee0 con 0x7f5bd81061a0 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bf0914b80 1 Processor -- start 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bcea00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 0x7f5bbc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bf0914b80 1 -- start start 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bcea00700 1 -- 172.21.15.46:0/1574066335 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5bc4014070 con 0x7f5bd81061a0 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bd6800700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 0x7f5bbc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.907+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582f720 0x561915829350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x561915829cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561915780150 con 0x56191582f720 2024-05-08T23:09:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191577db10 con 0x56191582ee00 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561915779400 con 0x561915829890 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49498/0 (socket says 172.21.15.46:49498) 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 -- 172.21.15.46:0/3400712712 learned_addr learned my addr 172.21.15.46:0/3400712712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd6800700 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x561915829cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5400700 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582f720 0x561915829350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 -- 172.21.15.46:0/3400712712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 msgr2=0x561915829cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd6800700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 0x7f5bbc0a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000f590 tx=0x7f5bd000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x561915829cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 -- 172.21.15.46:0/3400712712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582f720 msgr2=0x561915829350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582f720 0x561915829350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.908+0000 7f5bd5e00700 1 -- 172.21.15.46:0/3400712712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619155fa3b0 con 0x56191582ee00 2024-05-08T23:09:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.909+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009230 tx=0x7f5bc8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a583dd6d526f7f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.909+0000 7f5bc3400700 1 -- 172.21.15.46:0/3400712712 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc801c050 con 0x56191582ee00 2024-05-08T23:09:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.909+0000 7f5bc3400700 1 -- 172.21.15.46:0/3400712712 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc801d040 con 0x56191582ee00 2024-05-08T23:09:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.909+0000 7f5bc3400700 1 -- 172.21.15.46:0/3400712712 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80052a0 con 0x56191582ee00 2024-05-08T23:09:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- 172.21.15.46:0/3400712712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 msgr2=0x56191582f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009230 tx=0x7f5bc8005dc0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- 172.21.15.46:0/3400712712 shutdown_connections 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x561915829cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582ee00 0x56191582f1e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/3400712712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582f720 0x561915829350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- 172.21.15.46:0/3400712712 >> 172.21.15.46:0/3400712712 conn(0x561915787210 msgr2=0x561915823df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- 172.21.15.46:0/3400712712 shutdown_connections 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- 172.21.15.46:0/3400712712 wait complete. 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.910+0000 7f5bf0914b80 1 -- start start 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 Processor -- start 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 0x561915832a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x561915832fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561915715560 con 0x56191582ee00 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561915715200 con 0x561915829890 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619155fc6a0 con 0x56191582f720 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.911+0000 7f5bf0914b80 10 client.0 _ll_register_callbacks cb 0x56191571eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 0x561915832a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33698/0 (socket says 172.21.15.46:33698) 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 -- 172.21.15.46:0/234989284 learned_addr learned my addr 172.21.15.46:0/234989284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 0x561915832a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49506/0 (socket says 172.21.15.46:49506) 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd5400700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x561915832fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 -- 172.21.15.46:0/234989284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 msgr2=0x561915832a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 0x561915832a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 -- 172.21.15.46:0/234989284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 msgr2=0x561915832fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x561915832fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 -- 172.21.15.46:0/234989284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd810e590 con 0x56191582f720 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bd6800700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0008b80 tx=0x7f5bd0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb50f188914905 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.912+0000 7f5bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000e040 con 0x56191582f720 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619156d2270 con 0x56191582f720 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5bd0017ce0 con 0x56191582f720 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd0017ea0 con 0x56191582f720 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 10 client.27163 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619155a6b60 con 0x56191582f720 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619156d2270 con 0x56191582f720 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bd0027030 con 0x56191582f720 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 10 client.27163 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 10 client.27163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.913+0000 7f5bf0914b80 20 client.27163 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915843200 0x561915845620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bf0914b80 4 client.27163 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619157f8810 con 0x561915843200 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915843200 0x561915845620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915843200 0x561915845620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=581072b9df77b8b2 server_cookie=4c690b7dd7402166 in_seq=0 out_seq=0 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.914+0000 7f5bb7e00700 10 client.27163 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bb7e00700 1 -- 172.21.15.46:0/234989284 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619157f8810 con 0x561915843200 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bb7e00700 10 client.27163 handle_command_reply: tid=0 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 client.27163 shutdown 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 msgr2=0x5619158334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0008b80 tx=0x7f5bd0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 shutdown_connections 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582f720 0x5619158334f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561915829890 0x561915832a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915843200 0x561915845620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/234989284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x561915832fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 >> 172.21.15.46:0/234989284 conn(0x561915787210 msgr2=0x5619158235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bb7e00700 0 client.27163 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 shutdown_connections 2024-05-08T23:09:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.915+0000 7f5bf0914b80 1 -- 172.21.15.46:0/234989284 wait complete. 2024-05-08T23:09:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.916+0000 7f5bf0914b80 20 client.27163 trim_cache size 0 max 16384 2024-05-08T23:09:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.930+0000 7f5bf0914b80 1 Processor -- start 2024-05-08T23:09:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.930+0000 7f5bf0914b80 1 -- start start 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.930+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 0x56191583ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.930+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x5619158257e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619158340e0 con 0x561915829890 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561915820dc0 con 0x56191582f720 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191581e800 con 0x56191582ee00 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 0x56191583ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33704/0 (socket says 172.21.15.46:33704) 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 -- 172.21.15.46:0/2299365850 learned_addr learned my addr 172.21.15.46:0/2299365850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 0x56191583ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47982/0 (socket says 172.21.15.46:47982) 2024-05-08T23:09:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd6800700 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x5619158257e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 -- 172.21.15.46:0/2299365850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 msgr2=0x5619158257e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x5619158257e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 -- 172.21.15.46:0/2299365850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 msgr2=0x56191583ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 0x56191583ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.931+0000 7f5bd5400700 1 -- 172.21.15.46:0/2299365850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56191573f560 con 0x56191582ee00 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.932+0000 7f5bd5400700 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4096b90 tx=0x7f5bc40045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b0b867ef7289d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.932+0000 7f5bc3e00700 1 -- 172.21.15.46:0/2299365850 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc409c070 con 0x56191582ee00 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.932+0000 7f5bc3e00700 1 -- 172.21.15.46:0/2299365850 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc409d040 con 0x56191582ee00 2024-05-08T23:09:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.932+0000 7f5bc3e00700 1 -- 172.21.15.46:0/2299365850 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40ca9e0 con 0x56191582ee00 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- 172.21.15.46:0/2299365850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 msgr2=0x561915823d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4096b90 tx=0x7f5bc40045e0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- 172.21.15.46:0/2299365850 shutdown_connections 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x5619158257e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561915829890 0x56191583ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/2299365850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56191582ee00 0x561915823d20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- 172.21.15.46:0/2299365850 >> 172.21.15.46:0/2299365850 conn(0x561915787210 msgr2=0x561915820210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- 172.21.15.46:0/2299365850 shutdown_connections 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- 172.21.15.46:0/2299365850 wait complete. 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 -- start start 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.933+0000 7f5bf0914b80 1 Processor -- start 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x56191582b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x56191582bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191582c510 con 0x56191582ee00 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191582c650 con 0x56191582f720 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191582a930 con 0x561915829890 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bf0914b80 10 client.0 _ll_register_callbacks cb 0x56191571eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bd5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33716/0 (socket says 172.21.15.46:33716) 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bd5e00700 1 -- 172.21.15.46:0/762425269 learned_addr learned my addr 172.21.15.46:0/762425269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bd6800700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x56191582bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.934+0000 7f5bd5400700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x56191582b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 -- 172.21.15.46:0/762425269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 msgr2=0x56191582bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x56191582bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 -- 172.21.15.46:0/762425269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 msgr2=0x56191582b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x56191582b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 -- 172.21.15.46:0/762425269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81a64e0 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5400700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x56191582b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bd5e00700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800f7e0 tx=0x7f5bc800f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60899e8753e8934a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8032040 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5bc8034d70 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc801d040 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc801c040 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bf0914b80 10 client.27169 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.935+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619155fecd0 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc801d040 con 0x561915829890 2024-05-08T23:09:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bc8009c70 con 0x561915829890 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 10 client.27169 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 10 client.27169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 20 client.27169 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915899930 0x56191589bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 4 client.27169 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561915771ec0 con 0x561915899930 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.936+0000 7f5bd5400700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915899930 0x56191589bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.937+0000 7f5bd5400700 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915899930 0x56191589bd20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd40618f668c156b server_cookie=8b012d79489e6a1a in_seq=0 out_seq=0 2024-05-08T23:09:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.937+0000 7f5bb7e00700 10 client.27169 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.937+0000 7f5bb7e00700 1 -- 172.21.15.46:0/762425269 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561915771ec0 con 0x561915899930 2024-05-08T23:09:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.937+0000 7f5bb7e00700 10 client.27169 handle_command_reply: tid=0 2024-05-08T23:09:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.937+0000 7f5bf0914b80 1 client.27169 shutdown 2024-05-08T23:09:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 msgr2=0x56191583a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800f7e0 tx=0x7f5bc800f770 comp rx=0 tx=0).stop 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 shutdown_connections 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56191582f720 0x56191582bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561915829890 0x56191583a390 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x561915899930 0x56191589bd20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 --2- 172.21.15.46:0/762425269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56191582ee00 0x56191582b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bb7e00700 0 client.27169 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 >> 172.21.15.46:0/762425269 conn(0x561915787210 msgr2=0x5619158257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 shutdown_connections 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 1 -- 172.21.15.46:0/762425269 wait complete. 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bf0914b80 20 client.27169 trim_cache size 0 max 16384 2024-05-08T23:09:07.941 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 msgr2=0x7f5bbc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.938+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 0x7f5bbc0a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000f590 tx=0x7f5bd000b040 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 msgr2=0x7f5bd80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400ea30 tx=0x7f5bc400ed40 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 shutdown_connections 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8073d20 0x7f5bd81a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5bbc0a1290 0x7f5bbc0a3750 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd8101080 0x7f5bd80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 --2- 172.21.15.46:0/1574066335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd81061a0 0x7f5bd80737e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 >> 172.21.15.46:0/1574066335 conn(0x7f5bd80fc8e0 msgr2=0x7f5bd80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 shutdown_connections 2024-05-08T23:09:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:07.939+0000 7f5bdce00700 1 -- 172.21.15.46:0/1574066335 wait complete. 2024-05-08T23:09:07.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:07 smithi046 ceph-mon[36898]: pgmap v162: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:07.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:07 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1752171626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:07.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 -- 172.21.15.46:0/4213238771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11241061a0 msgr2=0x7f112410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 --2- 172.21.15.46:0/4213238771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11241061a0 0x7f112410dd20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f111c00b0a0 tx=0x7f111c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 -- 172.21.15.46:0/4213238771 shutdown_connections 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 --2- 172.21.15.46:0/4213238771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11241061a0 0x7f112410dd20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 --2- 172.21.15.46:0/4213238771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f1124101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 --2- 172.21.15.46:0/4213238771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1124101a30 0x7f1124105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 -- 172.21.15.46:0/4213238771 >> 172.21.15.46:0/4213238771 conn(0x7f11240fc8e0 msgr2=0x7f11240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 -- 172.21.15.46:0/4213238771 shutdown_connections 2024-05-08T23:09:08.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.184+0000 7f112a400700 1 -- 172.21.15.46:0/4213238771 wait complete. 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.185+0000 7f112a400700 1 Processor -- start 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 -- start start 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11241061a0 0x7f1124197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 0x7f112419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419c750 con 0x7f1124101080 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419c8c0 con 0x7f11241061a0 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f112a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112419ca80 con 0x7f112419bcd0 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f1123400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f1123400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48010/0 (socket says 172.21.15.46:48010) 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f1123400700 1 -- 172.21.15.46:0/3119100006 learned_addr learned my addr 172.21.15.46:0/3119100006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f111be00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11241061a0 0x7f1124197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.186+0000 7f1123e00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 0x7f112419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 -- 172.21.15.46:0/3119100006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 msgr2=0x7f112419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 0x7f112419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 -- 172.21.15.46:0/3119100006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11241061a0 msgr2=0x7f1124197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11241061a0 0x7f1124197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 -- 172.21.15.46:0/3119100006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c009d00 con 0x7f1124101080 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123e00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 0x7f112419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.187+0000 7f1123400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f111400e9a0 tx=0x7f111400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.188+0000 7f1120c00700 1 -- 172.21.15.46:0/3119100006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114009790 con 0x7f1124101080 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.188+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11241a3fe0 con 0x7f1124101080 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.188+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11241a44b0 con 0x7f1124101080 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.188+0000 7f1120c00700 1 -- 172.21.15.46:0/3119100006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f111400cbd0 con 0x7f1124101080 2024-05-08T23:09:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.188+0000 7f1120c00700 1 -- 172.21.15.46:0/3119100006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114018680 con 0x7f1124101080 2024-05-08T23:09:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.192+0000 7f1120c00700 1 -- 172.21.15.46:0/3119100006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f11140187e0 con 0x7f1124101080 2024-05-08T23:09:08.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 Processor -- start 2024-05-08T23:09:08.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 -- start start 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f1120c00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 0x7f11080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f88a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f8850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f7d7150 con 0x55be9f88a210 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f7d4b10 con 0x55be9f884ca0 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f113ded7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f7d0400 con 0x55be9f88ab30 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.193+0000 7f1120c00700 1 -- 172.21.15.46:0/3119100006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1114014070 con 0x7f1124101080 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f1123e00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 0x7f11080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f1123400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f88a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f1123e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f8850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33762/0 (socket says 172.21.15.46:33762) 2024-05-08T23:09:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 -- 172.21.15.46:0/3206205959 learned_addr learned my addr 172.21.15.46:0/3206205959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 -- 172.21.15.46:0/3206205959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 msgr2=0x55be9f8850e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f8850e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 -- 172.21.15.46:0/3206205959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 msgr2=0x55be9f88a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f88a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 -- 172.21.15.46:0/3206205959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be9f6513b0 con 0x55be9f88ab30 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f1123e00700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 0x7f11080a37d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f111c009830 tx=0x7f111c009360 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.194+0000 7f111be00700 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f110c009c60 tx=0x7f110c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf144cd69f330c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.195+0000 7f1118c00700 1 -- 172.21.15.46:0/3206205959 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c009730 con 0x55be9f88ab30 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.195+0000 7f1118c00700 1 -- 172.21.15.46:0/3206205959 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110c015070 con 0x55be9f88ab30 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.195+0000 7f1118c00700 1 -- 172.21.15.46:0/3206205959 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c0116a0 con 0x55be9f88ab30 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.195+0000 7f113ded7b80 1 -- 172.21.15.46:0/3206205959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 msgr2=0x55be9f884760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.195+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f110c009c60 tx=0x7f110c009f70 comp rx=0 tx=0).stop 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 -- 172.21.15.46:0/3206205959 shutdown_connections 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f8850e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f88a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3206205959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f884760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 -- 172.21.15.46:0/3206205959 >> 172.21.15.46:0/3206205959 conn(0x55be9f7de210 msgr2=0x55be9f87adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 -- 172.21.15.46:0/3206205959 shutdown_connections 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 -- 172.21.15.46:0/3206205959 wait complete. 2024-05-08T23:09:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 -- start start 2024-05-08T23:09:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.196+0000 7f113ded7b80 1 Processor -- start 2024-05-08T23:09:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 0x55be9f88dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f88e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f76c560 con 0x55be9f884ca0 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f76c200 con 0x55be9f88a210 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f6536a0 con 0x55be9f88ab30 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f113ded7b80 10 client.0 _ll_register_callbacks cb 0x55be9f775b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f1123400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f111be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 0x55be9f88dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f1123400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48034/0 (socket says 172.21.15.46:48034) 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f1123400700 1 -- 172.21.15.46:0/2925241992 learned_addr learned my addr 172.21.15.46:0/2925241992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.197+0000 7f1123e00700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f88e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 -- 172.21.15.46:0/2925241992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 msgr2=0x55be9f88e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f88e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 -- 172.21.15.46:0/2925241992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 msgr2=0x55be9f88dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 0x55be9f88dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 -- 172.21.15.46:0/2925241992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c009890 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f111be00700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 0x55be9f88dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1123400700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11140ca570 tx=0x7f111409de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48081571755097cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1111600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140996a0 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1114098410 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be9f729270 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.198+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11140a1ec0 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 10 client.17130 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be9f5fdb60 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be9f729270 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1114098f10 con 0x55be9f884ca0 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 10 client.17130 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:09:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 20 client.17130 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f89e6d0 0x55be9f8a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 4 client.17130 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be9f84f810 con 0x55be9f89e6d0 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.199+0000 7f111be00700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f89e6d0 0x55be9f8a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.200+0000 7f111be00700 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f89e6d0 0x55be9f8a0af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c4735715cf04f08 server_cookie=4b9f1368c2eb69df in_seq=0 out_seq=0 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.200+0000 7f1111600700 10 client.17130 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.200+0000 7f1111600700 1 -- 172.21.15.46:0/2925241992 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be9f84f810 con 0x55be9f89e6d0 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.200+0000 7f1111600700 10 client.17130 handle_command_reply: tid=0 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 client.17130 shutdown 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 msgr2=0x55be9f890bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11140ca570 tx=0x7f111409de40 comp rx=0 tx=0).stop 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 shutdown_connections 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f88e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f89e6d0 0x55be9f8a0af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f884ca0 0x55be9f890bb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 --2- 172.21.15.46:0/2925241992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f88a210 0x55be9f88dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f1111600700 0 client.17130 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 >> 172.21.15.46:0/2925241992 conn(0x55be9f7de210 msgr2=0x55be9f87a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 shutdown_connections 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 1 -- 172.21.15.46:0/2925241992 wait complete. 2024-05-08T23:09:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.201+0000 7f113ded7b80 20 client.17130 trim_cache size 0 max 16384 2024-05-08T23:09:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:07 smithi086 ceph-mon[78610]: pgmap v162: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:07 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1752171626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:08.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.216+0000 7f113ded7b80 1 Processor -- start 2024-05-08T23:09:08.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.216+0000 7f113ded7b80 1 -- start start 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f884760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88ab30 0x55be9f87ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f891070 con 0x55be9f88ab30 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f877dc0 con 0x55be9f884ca0 2024-05-08T23:09:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f113ded7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f875ac0 con 0x55be9f88a210 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33774/0 (socket says 172.21.15.46:33774) 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 -- 172.21.15.46:0/3380919994 learned_addr learned my addr 172.21.15.46:0/3380919994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f1123400700 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f884760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f1123e00700 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88ab30 0x55be9f87ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 -- 172.21.15.46:0/3380919994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 msgr2=0x55be9f884760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f884760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 -- 172.21.15.46:0/3380919994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88ab30 msgr2=0x55be9f87ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88ab30 0x55be9f87ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.217+0000 7f111be00700 1 -- 172.21.15.46:0/3380919994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be9f796560 con 0x55be9f88a210 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.218+0000 7f111be00700 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f110c009340 tx=0x7f110c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4714c6112d5d436c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.218+0000 7f1119600700 1 -- 172.21.15.46:0/3380919994 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c020040 con 0x55be9f88a210 2024-05-08T23:09:08.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.218+0000 7f1119600700 1 -- 172.21.15.46:0/3380919994 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110c01c620 con 0x55be9f88a210 2024-05-08T23:09:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.218+0000 7f1119600700 1 -- 172.21.15.46:0/3380919994 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c003680 con 0x55be9f88a210 2024-05-08T23:09:08.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- 172.21.15.46:0/3380919994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 msgr2=0x55be9f89a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f110c009340 tx=0x7f110c0092d0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- 172.21.15.46:0/3380919994 shutdown_connections 2024-05-08T23:09:08.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88ab30 0x55be9f87ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f884760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 --2- 172.21.15.46:0/3380919994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88a210 0x55be9f89a760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- 172.21.15.46:0/3380919994 >> 172.21.15.46:0/3380919994 conn(0x55be9f7de210 msgr2=0x55be9f885080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- 172.21.15.46:0/3380919994 shutdown_connections 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- 172.21.15.46:0/3380919994 wait complete. 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.219+0000 7f113ded7b80 1 -- start start 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 Processor -- start 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f885d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f886280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f886850 con 0x55be9f88a210 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f8869c0 con 0x55be9f884ca0 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9f886b80 con 0x55be9f88ab30 2024-05-08T23:09:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f113ded7b80 10 client.0 _ll_register_callbacks cb 0x55be9f775b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f1123400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f1123e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f886280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f1123400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49574/0 (socket says 172.21.15.46:49574) 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f1123400700 1 -- 172.21.15.46:0/14247949 learned_addr learned my addr 172.21.15.46:0/14247949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.220+0000 7f111be00700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f885d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 -- 172.21.15.46:0/14247949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 msgr2=0x55be9f895870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 -- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 msgr2=0x55be9f885d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f885d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 -- 172.21.15.46:0/14247949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be9f6513b0 con 0x55be9f88ab30 2024-05-08T23:09:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123400700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.221+0000 7f1123e00700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f886280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f111c00b780 tx=0x7f111c006210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39ed6f8d3eea67a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f1111600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c025070 con 0x55be9f88ab30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110c015070 con 0x55be9f88ab30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f111c0216f0 con 0x55be9f88ab30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f111c0218f0 con 0x55be9f88ab30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.222+0000 7f113ded7b80 10 client.27187 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.223+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be9f655cd0 con 0x55be9f88ab30 2024-05-08T23:09:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.224+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110c015070 con 0x55be9f88ab30 2024-05-08T23:09:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f111c034080 con 0x55be9f88ab30 2024-05-08T23:09:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 10 client.27187 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 10 client.27187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:09:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 20 client.27187 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f8f5240 0x55be9f8f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 4 client.27187 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.226+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be9f7c8ec0 con 0x55be9f8f5240 2024-05-08T23:09:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.227+0000 7f1123400700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f8f5240 0x55be9f8f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.228+0000 7f1123400700 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f8f5240 0x55be9f8f7660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f09e9a0eb6b1bca4 server_cookie=c528555c215a1b51 in_seq=0 out_seq=0 2024-05-08T23:09:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.228+0000 7f1111600700 10 client.27187 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.229+0000 7f1111600700 1 -- 172.21.15.46:0/14247949 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be9f7c8ec0 con 0x55be9f8f5240 2024-05-08T23:09:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.229+0000 7f1111600700 10 client.27187 handle_command_reply: tid=0 2024-05-08T23:09:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.229+0000 7f113ded7b80 1 client.27187 shutdown 2024-05-08T23:09:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 msgr2=0x55be9f886280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f886280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f111c00b780 tx=0x7f111c006210 comp rx=0 tx=0).stop 2024-05-08T23:09:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 shutdown_connections 2024-05-08T23:09:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be9f88ab30 0x55be9f886280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be9f884ca0 0x55be9f895870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55be9f8f5240 0x55be9f8f7660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 --2- 172.21.15.46:0/14247949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be9f88a210 0x55be9f885d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 >> 172.21.15.46:0/14247949 conn(0x55be9f7de210 msgr2=0x55be9f887fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.230+0000 7f1111600700 0 client.27187 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.231+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 shutdown_connections 2024-05-08T23:09:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.231+0000 7f113ded7b80 1 -- 172.21.15.46:0/14247949 wait complete. 2024-05-08T23:09:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.232+0000 7f113ded7b80 20 client.27187 trim_cache size 0 max 16384 2024-05-08T23:09:08.232 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:08.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 msgr2=0x7f11080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 0x7f11080a37d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f111c009830 tx=0x7f111c009360 comp rx=0 tx=0).stop 2024-05-08T23:09:08.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 msgr2=0x7f11241973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f111400e9a0 tx=0x7f111400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 shutdown_connections 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f112419bcd0 0x7f112419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f11080a1310 0x7f11080a37d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124101080 0x7f11241973f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 --2- 172.21.15.46:0/3119100006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11241061a0 0x7f1124197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.233+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 >> 172.21.15.46:0/3119100006 conn(0x7f11240fc8e0 msgr2=0x7f11240fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.234+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 shutdown_connections 2024-05-08T23:09:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.234+0000 7f112a400700 1 -- 172.21.15.46:0/3119100006 wait complete. 2024-05-08T23:09:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:07 smithi053 ceph-mon[77587]: pgmap v162: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:07 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1752171626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:08.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 -- 172.21.15.46:0/1020107396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b141028c0 msgr2=0x7f2b1410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1020107396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b141028c0 0x7f2b1410ada0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009990 tx=0x7f2b00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 -- 172.21.15.46:0/1020107396 shutdown_connections 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1020107396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1410b3c0 0x7f2b1410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1020107396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 0x7f2b14102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1020107396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b141028c0 0x7f2b1410ada0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 -- 172.21.15.46:0/1020107396 >> 172.21.15.46:0/1020107396 conn(0x7f2b140fb760 msgr2=0x7f2b140fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 -- 172.21.15.46:0/1020107396 shutdown_connections 2024-05-08T23:09:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.499+0000 7f2b19a00700 1 -- 172.21.15.46:0/1020107396 wait complete. 2024-05-08T23:09:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.501+0000 7f2b19a00700 1 Processor -- start 2024-05-08T23:09:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.501+0000 7f2b19a00700 1 -- start start 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 0x7f2b140ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b141028c0 0x7f2b14100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141a5710 con 0x7f2b141028c0 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141a5880 con 0x7f2b14101fa0 2024-05-08T23:09:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b19a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141a5a40 con 0x7f2b1410b3c0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b12000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b141028c0 0x7f2b14100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33804/0 (socket says 172.21.15.46:33804) 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 -- 172.21.15.46:0/1777404476 learned_addr learned my addr 172.21.15.46:0/1777404476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 0x7f2b140ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 -- 172.21.15.46:0/1777404476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 msgr2=0x7f2b140ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 0x7f2b140ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 -- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b141028c0 msgr2=0x7f2b14100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b141028c0 0x7f2b14100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.502+0000 7f2b13400700 1 -- 172.21.15.46:0/1777404476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000095f0 con 0x7f2b1410b3c0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b13400700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00ef30 tx=0x7f2b0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b0b400700 1 -- 172.21.15.46:0/1777404476 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c018070 con 0x7f2b1410b3c0 2024-05-08T23:09:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b0b400700 1 -- 172.21.15.46:0/1777404476 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0c00f040 con 0x7f2b1410b3c0 2024-05-08T23:09:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b141a5cc0 con 0x7f2b1410b3c0 2024-05-08T23:09:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b0b400700 1 -- 172.21.15.46:0/1777404476 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00cd00 con 0x7f2b1410b3c0 2024-05-08T23:09:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.503+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b141a6150 con 0x7f2b1410b3c0 2024-05-08T23:09:08.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.508+0000 7f2b0b400700 1 -- 172.21.15.46:0/1777404476 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2b0c003a00 con 0x7f2b1410b3c0 2024-05-08T23:09:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.509+0000 7f2b0b400700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 0x7f2af80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.509+0000 7f2b0b400700 1 -- 172.21.15.46:0/1777404476 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2b0c0ccb50 con 0x7f2b1410b3c0 2024-05-08T23:09:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.509+0000 7f2b2d508b80 1 Processor -- start 2024-05-08T23:09:08.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.509+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 0x7f2af80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 -- start start 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 0x7f2af80a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009c60 tx=0x7f2b04009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a94b150 con 0x55841a9fa720 2024-05-08T23:09:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a948b10 con 0x55841a9f4890 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a944400 con 0x55841a9f9e00 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33812/0 (socket says 172.21.15.46:33812) 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b12a00700 1 -- 172.21.15.46:0/224258442 learned_addr learned my addr 172.21.15.46:0/224258442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b13400700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.510+0000 7f2b12000700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 -- 172.21.15.46:0/224258442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 msgr2=0x55841a9f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 -- 172.21.15.46:0/224258442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 msgr2=0x55841a9f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 -- 172.21.15.46:0/224258442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55841a7c53b0 con 0x55841a9f9e00 2024-05-08T23:09:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b13400700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2b12a00700 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b04004140 tx=0x7f2b04014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a357231cce2a7079 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2affe00700 1 -- 172.21.15.46:0/224258442 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04003b10 con 0x55841a9f9e00 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2affe00700 1 -- 172.21.15.46:0/224258442 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b04004cf0 con 0x55841a9f9e00 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.511+0000 7f2affe00700 1 -- 172.21.15.46:0/224258442 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b040056e0 con 0x55841a9f9e00 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 -- 172.21.15.46:0/224258442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 msgr2=0x55841a9fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b04004140 tx=0x7f2b04014000 comp rx=0 tx=0).stop 2024-05-08T23:09:08.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 -- 172.21.15.46:0/224258442 shutdown_connections 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fa1e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/224258442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 -- 172.21.15.46:0/224258442 >> 172.21.15.46:0/224258442 conn(0x55841a952210 msgr2=0x55841a9eedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 -- 172.21.15.46:0/224258442 shutdown_connections 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.512+0000 7f2b2d508b80 1 -- 172.21.15.46:0/224258442 wait complete. 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.513+0000 7f2b2d508b80 1 -- start start 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.513+0000 7f2b2d508b80 1 Processor -- start 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a8e0560 con 0x55841a9fa720 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a8e0200 con 0x55841a9f4890 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a7c76a0 con 0x55841a9f9e00 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b2d508b80 10 client.0 _ll_register_callbacks cb 0x55841a8e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b12000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49596/0 (socket says 172.21.15.46:49596) 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b12000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33822/0 (socket says 172.21.15.46:33822) 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.514+0000 7f2b12a00700 1 -- 172.21.15.46:0/1645644537 learned_addr learned my addr 172.21.15.46:0/1645644537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b13400700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 -- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 msgr2=0x55841a9fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 -- 172.21.15.46:0/1645644537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 msgr2=0x55841a9fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 -- 172.21.15.46:0/1645644537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b141a6150 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2b12000700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b000059d0 tx=0x7f2b00005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f5f0ff1d6cdf29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.515+0000 7f2afcc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0001d070 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2b00004500 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55841a89d270 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b000046c0 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2b2d508b80 10 client.27199 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2b04017070 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55841a89d270 con 0x55841a9f9e00 2024-05-08T23:09:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b0000f6a0 con 0x55841a9f9e00 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.516+0000 7f2b2d508b80 10 client.27199 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b2d508b80 10 client.27199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b2d508b80 20 client.27199 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa0e2b0 0x55841aa106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b2d508b80 4 client.27199 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55841a9c3810 con 0x55841aa0e2b0 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa0e2b0 0x55841aa106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa0e2b0 0x55841aa106d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a650f9c785ccc0 server_cookie=d1e9223779e9b259 in_seq=0 out_seq=0 2024-05-08T23:09:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.517+0000 7f2afcc00700 10 client.27199 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2afcc00700 1 -- 172.21.15.46:0/1645644537 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55841a9c3810 con 0x55841aa0e2b0 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2afcc00700 10 client.27199 handle_command_reply: tid=0 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2b2d508b80 1 client.27199 shutdown 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 msgr2=0x55841a9fe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b000059d0 tx=0x7f2b00005750 comp rx=0 tx=0).stop 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 shutdown_connections 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.518+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa0e2b0 0x55841aa106d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9fa720 0x55841a9fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9f4890 0x55841a9fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2afcc00700 0 client.27199 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1645644537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f9e00 0x55841a9fe0a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 >> 172.21.15.46:0/1645644537 conn(0x55841a952210 msgr2=0x55841a9ee5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 shutdown_connections 2024-05-08T23:09:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1645644537 wait complete. 2024-05-08T23:09:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.519+0000 7f2b2d508b80 20 client.27199 trim_cache size 0 max 16384 2024-05-08T23:09:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.533+0000 7f2b2d508b80 1 Processor -- start 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.533+0000 7f2b2d508b80 1 -- start start 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa09c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841a9eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a89d270 con 0x55841a9f9e00 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a9f0870 con 0x55841a9fa720 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841a9ff160 con 0x55841a9f4890 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33830/0 (socket says 172.21.15.46:33830) 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b12a00700 1 -- 172.21.15.46:0/3861032533 learned_addr learned my addr 172.21.15.46:0/3861032533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b12000700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa09c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.534+0000 7f2b13400700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841a9eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 -- 172.21.15.46:0/3861032533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 msgr2=0x55841a9eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841a9eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 -- 172.21.15.46:0/3861032533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 msgr2=0x55841aa09c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa09c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 -- 172.21.15.46:0/3861032533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55841a771b60 con 0x55841a9f4890 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12000700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa09c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.535+0000 7f2b12a00700 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b04003960 tx=0x7f2b04013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec2b5f7c76a5925 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b08c00700 1 -- 172.21.15.46:0/3861032533 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04003d10 con 0x55841a9f4890 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b08c00700 1 -- 172.21.15.46:0/3861032533 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b0402d040 con 0x55841a9f4890 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b08c00700 1 -- 172.21.15.46:0/3861032533 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04004450 con 0x55841a9f4890 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 -- 172.21.15.46:0/3861032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 msgr2=0x55841a9f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b04003960 tx=0x7f2b04013040 comp rx=0 tx=0).stop 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 -- 172.21.15.46:0/3861032533 shutdown_connections 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841a9eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841a9f4c70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/3861032533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa09c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.536+0000 7f2b2d508b80 1 -- 172.21.15.46:0/3861032533 >> 172.21.15.46:0/3861032533 conn(0x55841a952210 msgr2=0x55841a9eb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.537+0000 7f2b2d508b80 1 -- 172.21.15.46:0/3861032533 shutdown_connections 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.537+0000 7f2b2d508b80 1 -- 172.21.15.46:0/3861032533 wait complete. 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.537+0000 7f2b2d508b80 1 -- start start 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.537+0000 7f2b2d508b80 1 Processor -- start 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa03150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841aa03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841aa03e00 con 0x55841a9f9e00 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841aa03f70 con 0x55841a9fa720 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55841aa637b0 con 0x55841a9f4890 2024-05-08T23:09:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b2d508b80 10 client.0 _ll_register_callbacks cb 0x55841a8e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33836/0 (socket says 172.21.15.46:33836) 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 -- 172.21.15.46:0/1795627688 learned_addr learned my addr 172.21.15.46:0/1795627688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12000700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa03150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b13400700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841aa03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 -- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 msgr2=0x55841aa03760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841aa03760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 -- 172.21.15.46:0/1795627688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 msgr2=0x55841aa03150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa03150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b12a00700 1 -- 172.21.15.46:0/1795627688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b04017070 con 0x55841a9f4890 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.538+0000 7f2b13400700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841aa03760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2b12a00700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005250 tx=0x7f2b04011bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1496d4d1290f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2afcc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04003c90 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f2b04032890 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b04004cf0 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b04003df0 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2b2d508b80 10 client.27205 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55841a7c53b0 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.539+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b04004cf0 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b04024030 con 0x55841a9f4890 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 10 client.27205 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 10 client.27205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 20 client.27205 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa64b10 0x55841aa66f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 4 client.27205 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55841a93cec0 con 0x55841aa64b10 2024-05-08T23:09:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.540+0000 7f2b12000700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa64b10 0x55841aa66f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b12000700 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa64b10 0x55841aa66f30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e11342a1ebf455 server_cookie=d97a3bb469e60f3e in_seq=0 out_seq=0 2024-05-08T23:09:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2afcc00700 10 client.27205 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2afcc00700 1 -- 172.21.15.46:0/1795627688 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55841a93cec0 con 0x55841aa64b10 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2afcc00700 10 client.27205 handle_command_reply: tid=0 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 client.27205 shutdown 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 msgr2=0x55841aa055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2b04005250 tx=0x7f2b04011bc0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 shutdown_connections 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55841a9fa720 0x55841aa03760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55841a9f4890 0x55841aa055c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55841aa64b10 0x55841aa66f30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 --2- 172.21.15.46:0/1795627688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55841a9f9e00 0x55841aa03150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2afcc00700 0 client.27205 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 >> 172.21.15.46:0/1795627688 conn(0x55841a952210 msgr2=0x55841a9fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.541+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 shutdown_connections 2024-05-08T23:09:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.542+0000 7f2b2d508b80 1 -- 172.21.15.46:0/1795627688 wait complete. 2024-05-08T23:09:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.542+0000 7f2b2d508b80 20 client.27205 trim_cache size 0 max 16384 2024-05-08T23:09:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 msgr2=0x7f2af80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 0x7f2af80a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009c60 tx=0x7f2b04009260 comp rx=0 tx=0).stop 2024-05-08T23:09:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 msgr2=0x7f2b14100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:08.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00ef30 tx=0x7f2b0c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 shutdown_connections 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b1410b3c0 0x7f2b14100670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b14101fa0 0x7f2b140ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2af80a1290 0x7f2af80a3750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 --2- 172.21.15.46:0/1777404476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b141028c0 0x7f2b14100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 >> 172.21.15.46:0/1777404476 conn(0x7f2b140fb760 msgr2=0x7f2b140fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 shutdown_connections 2024-05-08T23:09:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:08.543+0000 7f2b19a00700 1 -- 172.21.15.46:0/1777404476 wait complete. 2024-05-08T23:09:08.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 390045 10.71 MB/sec execute 90 sec latency 15.603 ms 2024-05-08T23:09:09.563 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:09:09.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 -- 172.21.15.46:0/358060554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 msgr2=0x7ff9d010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/358060554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d010d880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b0a0 tx=0x7ff9c800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 -- 172.21.15.46:0/358060554 shutdown_connections 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/358060554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d010d880 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/358060554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/358060554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 0x7ff9d010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.800+0000 7ff9d5a00700 1 -- 172.21.15.46:0/358060554 >> 172.21.15.46:0/358060554 conn(0x7ff9d00fb760 msgr2=0x7ff9d00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.801+0000 7ff9d5a00700 1 -- 172.21.15.46:0/358060554 shutdown_connections 2024-05-08T23:09:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.801+0000 7ff9d5a00700 1 -- 172.21.15.46:0/358060554 wait complete. 2024-05-08T23:09:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.801+0000 7ff9d5a00700 1 Processor -- start 2024-05-08T23:09:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 -- start start 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 0x7ff9d0100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d0100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a57f0 con 0x7ff9d0101fa0 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a5960 con 0x7ff9d01028c0 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.802+0000 7ff9d5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a5b20 con 0x7ff9d010b3c0 2024-05-08T23:09:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48122/0 (socket says 172.21.15.46:48122) 2024-05-08T23:09:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cea00700 1 -- 172.21.15.46:0/4261138883 learned_addr learned my addr 172.21.15.46:0/4261138883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d0100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9ce000700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 0x7ff9d0100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 -- 172.21.15.46:0/4261138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 msgr2=0x7ff9d0100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 0x7ff9d0100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 -- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 msgr2=0x7ff9d00ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cf400700 1 -- 172.21.15.46:0/4261138883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009d00 con 0x7ff9d010b3c0 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.803+0000 7ff9cea00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9cf400700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d0100670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b070 tx=0x7ff9c8007860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9c7400700 1 -- 172.21.15.46:0/4261138883 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800e040 con 0x7ff9d010b3c0 2024-05-08T23:09:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9c7400700 1 -- 172.21.15.46:0/4261138883 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c801d470 con 0x7ff9d010b3c0 2024-05-08T23:09:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d01a5da0 con 0x7ff9d010b3c0 2024-05-08T23:09:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9c7400700 1 -- 172.21.15.46:0/4261138883 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8005280 con 0x7ff9d010b3c0 2024-05-08T23:09:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.804+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d01a62b0 con 0x7ff9d010b3c0 2024-05-08T23:09:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.808+0000 7ff9c7400700 1 -- 172.21.15.46:0/4261138883 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff9c801d5e0 con 0x7ff9d010b3c0 2024-05-08T23:09:09.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 Processor -- start 2024-05-08T23:09:09.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 -- start start 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9c7400700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9c7400700 1 -- 172.21.15.46:0/4261138883 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff9c80ce390 con 0x7ff9d010b3c0 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x5586476683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647668d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586475bf1d0 con 0x55864766e7a0 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586475bcb90 con 0x558647668910 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586475b8480 con 0x55864766de80 2024-05-08T23:09:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9cea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9cea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33870/0 (socket says 172.21.15.46:33870) 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9cea00700 1 -- 172.21.15.46:0/2490530599 learned_addr learned my addr 172.21.15.46:0/2490530599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.809+0000 7ff9ce000700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9ce000700 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x5586476683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 -- 172.21.15.46:0/2490530599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 msgr2=0x558647668d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647668d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 -- 172.21.15.46:0/2490530599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 msgr2=0x5586476683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x5586476683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 -- 172.21.15.46:0/2490530599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558647439430 con 0x55864766de80 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9cea00700 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00117e0 tx=0x7ff9c0011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cecfe7b6abaec2e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9ce000700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 0x7ff9b40a37d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x558647664c50 tx=0x7ff9bc009980 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.810+0000 7ff9bbe00700 1 -- 172.21.15.46:0/2490530599 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000f480 con 0x55864766de80 2024-05-08T23:09:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.811+0000 7ff9bbe00700 1 -- 172.21.15.46:0/2490530599 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c000faa0 con 0x55864766de80 2024-05-08T23:09:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.811+0000 7ff9bbe00700 1 -- 172.21.15.46:0/2490530599 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0019b60 con 0x55864766de80 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.811+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2490530599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 msgr2=0x55864766e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.811+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00117e0 tx=0x7ff9c0011af0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2490530599 shutdown_connections 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647668d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766e260 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2490530599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x5586476683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2490530599 >> 172.21.15.46:0/2490530599 conn(0x5586475c6290 msgr2=0x558647662e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2490530599 shutdown_connections 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2490530599 wait complete. 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.812+0000 7ff9e95beb80 1 -- start start 2024-05-08T23:09:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 Processor -- start 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647671bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x5586476720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x558647672630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586475545e0 con 0x55864766e7a0 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558647554280 con 0x558647668910 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864743b720 con 0x55864766de80 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.813+0000 7ff9e95beb80 10 client.0 _ll_register_callbacks cb 0x55864755dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9ce000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x5586476720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9ce000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x5586476720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33872/0 (socket says 172.21.15.46:33872) 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9ce000700 1 -- 172.21.15.46:0/248054964 learned_addr learned my addr 172.21.15.46:0/248054964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647671bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cf400700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x558647672630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 -- 172.21.15.46:0/248054964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 msgr2=0x5586476720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x5586476720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 -- 172.21.15.46:0/248054964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 msgr2=0x558647672630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x558647672630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 -- 172.21.15.46:0/248054964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c0011440 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9cea00700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647671bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011ef0 tx=0x7ff9c000f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158cc354c2a54d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.814+0000 7ff9b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0005ce0 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586475112f0 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff9c0004240 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c0004400 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9e95beb80 10 client.27155 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5586473e5be0 con 0x558647668910 2024-05-08T23:09:09.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.815+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586475112f0 con 0x558647668910 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9c0004980 con 0x558647668910 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 10 client.27155 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 20 client.27155 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558647682270 0x558647684690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 4 client.27155 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558647637890 con 0x558647682270 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.816+0000 7ff9ce000700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558647682270 0x558647684690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.817+0000 7ff9ce000700 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558647682270 0x558647684690 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df132f36ccb84035 server_cookie=5eab817520d3f42c in_seq=0 out_seq=0 2024-05-08T23:09:09.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.817+0000 7ff9b8c00700 10 client.27155 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.817+0000 7ff9b8c00700 1 -- 172.21.15.46:0/248054964 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558647637890 con 0x558647682270 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.817+0000 7ff9b8c00700 10 client.27155 handle_command_reply: tid=0 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 client.27155 shutdown 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 msgr2=0x558647671bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647671bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011ef0 tx=0x7ff9c000f6b0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 shutdown_connections 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558647682270 0x558647684690 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864766e7a0 0x558647672630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558647668910 0x558647671bb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/248054964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x5586476720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 >> 172.21.15.46:0/248054964 conn(0x5586475c6290 msgr2=0x558647662620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9b8c00700 0 client.27155 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 shutdown_connections 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 1 -- 172.21.15.46:0/248054964 wait complete. 2024-05-08T23:09:09.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.818+0000 7ff9e95beb80 20 client.27155 trim_cache size 0 max 16384 2024-05-08T23:09:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 Processor -- start 2024-05-08T23:09:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 -- start start 2024-05-08T23:09:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x55864767dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766e7a0 0x558647662da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558647674720 con 0x558647668910 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864765fe40 con 0x55864766e7a0 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.841+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558647664ed0 con 0x55864766de80 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33882/0 (socket says 172.21.15.46:33882) 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 -- 172.21.15.46:0/3042603923 learned_addr learned my addr 172.21.15.46:0/3042603923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9cf400700 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766e7a0 0x558647662da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9cea00700 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x55864767dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 -- 172.21.15.46:0/3042603923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766e7a0 msgr2=0x558647662da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766e7a0 0x558647662da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 -- 172.21.15.46:0/3042603923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 msgr2=0x55864767dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x55864767dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 -- 172.21.15.46:0/3042603923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55864765f560 con 0x55864766de80 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.842+0000 7ff9ce000700 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0058e0 tx=0x7ff9bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5e57d3043824d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.843+0000 7ff9c4c00700 1 -- 172.21.15.46:0/3042603923 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc025070 con 0x55864766de80 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.843+0000 7ff9c4c00700 1 -- 172.21.15.46:0/3042603923 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc011e10 con 0x55864766de80 2024-05-08T23:09:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.843+0000 7ff9c4c00700 1 -- 172.21.15.46:0/3042603923 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0204f0 con 0x55864766de80 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.843+0000 7ff9e95beb80 1 -- 172.21.15.46:0/3042603923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 msgr2=0x55864766d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.843+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0058e0 tx=0x7ff9bc009260 comp rx=0 tx=0).stop 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 -- 172.21.15.46:0/3042603923 shutdown_connections 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766e7a0 0x558647662da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x55864767dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/3042603923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766de80 0x55864766d3b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 -- 172.21.15.46:0/3042603923 >> 172.21.15.46:0/3042603923 conn(0x5586475c6290 msgr2=0x55864766cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 -- 172.21.15.46:0/3042603923 shutdown_connections 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 -- 172.21.15.46:0/3042603923 wait complete. 2024-05-08T23:09:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.844+0000 7ff9e95beb80 1 -- start start 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 Processor -- start 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x5586476795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766de80 0x558647677960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586476785b0 con 0x558647668910 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558647678720 con 0x55864766de80 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586476d8780 con 0x55864766e7a0 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.845+0000 7ff9e95beb80 10 client.0 _ll_register_callbacks cb 0x55864755dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x5586476795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9ce000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766de80 0x558647677960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33890/0 (socket says 172.21.15.46:33890) 2024-05-08T23:09:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 -- 172.21.15.46:0/2183250292 learned_addr learned my addr 172.21.15.46:0/2183250292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 -- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766de80 msgr2=0x558647677960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766de80 0x558647677960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 -- 172.21.15.46:0/2183250292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 msgr2=0x5586476795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x5586476795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 -- 172.21.15.46:0/2183250292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc028b00 con 0x55864766e7a0 2024-05-08T23:09:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9cf400700 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8097620 tx=0x7ff9c809acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab178259fc8278b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c809d070 con 0x55864766e7a0 2024-05-08T23:09:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.846+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff9c809b820 con 0x55864766e7a0 2024-05-08T23:09:09.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c000faa0 con 0x55864766e7a0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c80cbb60 con 0x55864766e7a0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 10 client.27223 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55864757e5e0 con 0x55864766e7a0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c000faa0 con 0x55864766e7a0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9c801d8c0 con 0x55864766e7a0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 10 client.27223 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 10 client.27223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 20 client.27223 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586476d9ce0 0x5586476dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 4 client.27223 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.847+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5586475b0f40 con 0x5586476d9ce0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.848+0000 7ff9cea00700 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586476d9ce0 0x5586476dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.848+0000 7ff9cea00700 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586476d9ce0 0x5586476dc100 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eab6f6877930b98 server_cookie=a4f38d52cfbac7bc in_seq=0 out_seq=0 2024-05-08T23:09:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.848+0000 7ff9b8c00700 10 client.27223 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9b8c00700 1 -- 172.21.15.46:0/2183250292 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5586475b0f40 con 0x5586476d9ce0 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9b8c00700 10 client.27223 handle_command_reply: tid=0 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 client.27223 shutdown 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 msgr2=0x558647677ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8097620 tx=0x7ff9c809acd0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 shutdown_connections 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55864766e7a0 0x558647677ea0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5586476d9ce0 0x5586476dc100 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558647668910 0x5586476795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 --2- 172.21.15.46:0/2183250292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55864766de80 0x558647677960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9b8c00700 0 client.27223 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 >> 172.21.15.46:0/2183250292 conn(0x5586475c6290 msgr2=0x558647673450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 shutdown_connections 2024-05-08T23:09:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 1 -- 172.21.15.46:0/2183250292 wait complete. 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.850+0000 7ff9e95beb80 20 client.27223 trim_cache size 0 max 16384 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "91e3fb79-3f5c-4b53-9cae-40c6758675a4", 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:09:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 msgr2=0x7ff9b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 0x7ff9b40a37d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x558647664c50 tx=0x7ff9bc009980 comp rx=0 tx=0).stop 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 msgr2=0x7ff9d0100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d0100670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b070 tx=0x7ff9c8007860 comp rx=0 tx=0).stop 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 shutdown_connections 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9d010b3c0 0x7ff9d0100670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9b40a1310 0x7ff9b40a37d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9d0101fa0 0x7ff9d00ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 --2- 172.21.15.46:0/4261138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9d01028c0 0x7ff9d0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:09.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 >> 172.21.15.46:0/4261138883 conn(0x7ff9d00fb760 msgr2=0x7ff9d00fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:09.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.851+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 shutdown_connections 2024-05-08T23:09:09.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:09.852+0000 7ff9d5a00700 1 -- 172.21.15.46:0/4261138883 wait complete. 2024-05-08T23:09:09.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:09:09.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 391620 10.69 MB/sec execute 91 sec latency 12.301 ms 2024-05-08T23:09:10.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 -- 172.21.15.46:0/647753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 msgr2=0x7f34240ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 --2- 172.21.15.46:0/647753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f34240ffde0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 -- 172.21.15.46:0/647753832 shutdown_connections 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 --2- 172.21.15.46:0/647753832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f342410d5b0 0x7f342410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 --2- 172.21.15.46:0/647753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f34240ffde0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 --2- 172.21.15.46:0/647753832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241003b0 0x7f342410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 -- 172.21.15.46:0/647753832 >> 172.21.15.46:0/647753832 conn(0x7f34240fb640 msgr2=0x7f34240fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 -- 172.21.15.46:0/647753832 shutdown_connections 2024-05-08T23:09:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.103+0000 7f3423e00700 1 -- 172.21.15.46:0/647753832 wait complete. 2024-05-08T23:09:10.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.105+0000 7f3423e00700 1 Processor -- start 2024-05-08T23:09:10.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.105+0000 7f3423e00700 1 -- start start 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.105+0000 7f3423e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f3424101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 0x7f3424102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 0x7f3424102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241a9890 con 0x7f342410d5b0 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241a99d0 con 0x7f34241003b0 2024-05-08T23:09:10.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241a9b40 con 0x7f34240ffa00 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3422000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 0x7f3424102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3422000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 0x7f3424102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49652/0 (socket says 172.21.15.46:49652) 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3422000700 1 -- 172.21.15.46:0/4012392208 learned_addr learned my addr 172.21.15.46:0/4012392208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3423400700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 0x7f3424102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.106+0000 7f3422a00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f3424101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.107+0000 7f3423400700 1 -- 172.21.15.46:0/4012392208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 msgr2=0x7f3424101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.107+0000 7f3423400700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f3424101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.107+0000 7f3423400700 1 -- 172.21.15.46:0/4012392208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 msgr2=0x7f3424102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.107+0000 7f3423400700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 0x7f3424102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.107+0000 7f3423400700 1 -- 172.21.15.46:0/4012392208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c009c40 con 0x7f342410d5b0 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.108+0000 7f3423400700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 0x7f3424102760 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f341c00ef70 tx=0x7f341c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.108+0000 7f3428a00700 1 -- 172.21.15.46:0/4012392208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c018070 con 0x7f342410d5b0 2024-05-08T23:09:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.108+0000 7f3428a00700 1 -- 172.21.15.46:0/4012392208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341c00f040 con 0x7f342410d5b0 2024-05-08T23:09:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.108+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140095f0 con 0x7f342410d5b0 2024-05-08T23:09:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.109+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34241aa190 con 0x7f342410d5b0 2024-05-08T23:09:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.109+0000 7f3428a00700 1 -- 172.21.15.46:0/4012392208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c004280 con 0x7f342410d5b0 2024-05-08T23:09:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.117+0000 7f3428a00700 1 -- 172.21.15.46:0/4012392208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f341c00c9f0 con 0x7f342410d5b0 2024-05-08T23:09:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.121+0000 7f3428a00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 0x7f34080a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.121+0000 7f3428a00700 1 -- 172.21.15.46:0/4012392208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f341c0cc970 con 0x7f342410d5b0 2024-05-08T23:09:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.122+0000 7f343d81db80 1 Processor -- start 2024-05-08T23:09:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.122+0000 7f343d81db80 1 -- start start 2024-05-08T23:09:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f3422a00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 0x7f34080a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c8350430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c834acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c82a1150 con 0x5573c8350970 2024-05-08T23:09:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c829eb10 con 0x5573c8350050 2024-05-08T23:09:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f343d81db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c829a400 con 0x5573c834a890 2024-05-08T23:09:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.123+0000 7f3422000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c8350430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48188/0 (socket says 172.21.15.46:48188) 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 -- 172.21.15.46:0/4063466265 learned_addr learned my addr 172.21.15.46:0/4063466265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 -- 172.21.15.46:0/4063466265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 msgr2=0x5573c834acd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c834acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 -- 172.21.15.46:0/4063466265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 msgr2=0x5573c8350430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c8350430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.124+0000 7f3422000700 1 -- 172.21.15.46:0/4063466265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c811b3b0 con 0x5573c8350970 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.125+0000 7f3422000700 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3410009df0 tx=0x7f3410011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1584f49f423fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.126+0000 7f340fe00700 1 -- 172.21.15.46:0/4063466265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410004230 con 0x5573c8350970 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.126+0000 7f340fe00700 1 -- 172.21.15.46:0/4063466265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3410004cf0 con 0x5573c8350970 2024-05-08T23:09:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.126+0000 7f340fe00700 1 -- 172.21.15.46:0/4063466265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410004230 con 0x5573c8350970 2024-05-08T23:09:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 -- 172.21.15.46:0/4063466265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 msgr2=0x5573c834a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f3422a00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 0x7f34080a3870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f341400b6f0 tx=0x7f3414005c00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3410009df0 tx=0x7f3410011720 comp rx=0 tx=0).stop 2024-05-08T23:09:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 -- 172.21.15.46:0/4063466265 shutdown_connections 2024-05-08T23:09:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c834acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c8350430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 --2- 172.21.15.46:0/4063466265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834a350 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 -- 172.21.15.46:0/4063466265 >> 172.21.15.46:0/4063466265 conn(0x5573c82a8210 msgr2=0x5573c83452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 -- 172.21.15.46:0/4063466265 shutdown_connections 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.128+0000 7f343d81db80 1 -- 172.21.15.46:0/4063466265 wait complete. 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.129+0000 7f343d81db80 1 -- start start 2024-05-08T23:09:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.130+0000 7f343d81db80 1 Processor -- start 2024-05-08T23:09:10.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c8350050 0x5573c8353c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c8354360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c8236560 con 0x5573c834a890 2024-05-08T23:09:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c8236200 con 0x5573c8350970 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c811d6a0 con 0x5573c8350050 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f343d81db80 10 client.0 _ll_register_callbacks cb 0x5573c823fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f3422a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f3423400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c8354360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f3422a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48204/0 (socket says 172.21.15.46:48204) 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f3422a00700 1 -- 172.21.15.46:0/273068237 learned_addr learned my addr 172.21.15.46:0/273068237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.131+0000 7f3422000700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c8350050 0x5573c8353c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3423400700 1 -- 172.21.15.46:0/273068237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c8350050 msgr2=0x5573c8353c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3423400700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c8350050 0x5573c8353c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3423400700 1 -- 172.21.15.46:0/273068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 msgr2=0x5573c8356930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3423400700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3423400700 1 -- 172.21.15.46:0/273068237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3410009a50 con 0x5573c8350970 2024-05-08T23:09:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.132+0000 7f3422a00700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.133+0000 7f3423400700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c8354360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c9cb0 tx=0x7f341c099240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b41c003c67713b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.133+0000 7f340cc00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.133+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c09d320 con 0x5573c8350970 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.133+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f341c099e00 con 0x5573c8350970 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.134+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573c81f3270 con 0x5573c8350970 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.134+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f341c099700 con 0x5573c8350970 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.134+0000 7f343d81db80 10 client.27170 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.134+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5573c80c7b60 con 0x5573c8350970 2024-05-08T23:09:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573c81f3270 con 0x5573c8350970 2024-05-08T23:09:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f341c0a1660 con 0x5573c8350970 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 10 client.27170 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 20 client.27170 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c8364330 0x5573c8366750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 4 client.27170 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573c8319810 con 0x5573c8364330 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.135+0000 7f3422a00700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c8364330 0x5573c8366750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.136+0000 7f3422a00700 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c8364330 0x5573c8366750 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58dd02d84858fdd5 server_cookie=aae1c941ef43af8d in_seq=0 out_seq=0 2024-05-08T23:09:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.136+0000 7f340cc00700 10 client.27170 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.137+0000 7f340cc00700 1 -- 172.21.15.46:0/273068237 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573c8319810 con 0x5573c8364330 2024-05-08T23:09:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.137+0000 7f340cc00700 10 client.27170 handle_command_reply: tid=0 2024-05-08T23:09:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.137+0000 7f343d81db80 1 client.27170 shutdown 2024-05-08T23:09:10.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 msgr2=0x5573c8354360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c8354360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f341c0c9cb0 tx=0x7f341c099240 comp rx=0 tx=0).stop 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 shutdown_connections 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c8354360 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c8364330 0x5573c8366750 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c834a890 0x5573c8356930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 --2- 172.21.15.46:0/273068237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c8350050 0x5573c8353c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 >> 172.21.15.46:0/273068237 conn(0x5573c82a8210 msgr2=0x5573c83450b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.139+0000 7f340cc00700 0 client.27170 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.140+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 shutdown_connections 2024-05-08T23:09:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.140+0000 7f343d81db80 1 -- 172.21.15.46:0/273068237 wait complete. 2024-05-08T23:09:10.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.140+0000 7f343d81db80 20 client.27170 trim_cache size 0 max 16384 2024-05-08T23:09:10.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 Processor -- start 2024-05-08T23:09:10.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 -- start start 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350050 0x5573c835b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c835b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c833f800 con 0x5573c8350050 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c8354960 con 0x5573c8350970 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f343d81db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c834b3d0 con 0x5573c834a890 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f3422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f3422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33954/0 (socket says 172.21.15.46:33954) 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.166+0000 7f3422a00700 1 -- 172.21.15.46:0/3128016863 learned_addr learned my addr 172.21.15.46:0/3128016863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:10.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422000700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350050 0x5573c835b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3423400700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c835b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 -- 172.21.15.46:0/3128016863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 msgr2=0x5573c835b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c835b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 -- 172.21.15.46:0/3128016863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350050 msgr2=0x5573c835b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350050 0x5573c835b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 -- 172.21.15.46:0/3128016863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c8260560 con 0x5573c834a890 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3423400700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c835b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.167+0000 7f3422a00700 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abac668dd8530c55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.168+0000 7f3419200700 1 -- 172.21.15.46:0/3128016863 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341402c040 con 0x5573c834a890 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.168+0000 7f3419200700 1 -- 172.21.15.46:0/3128016863 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341401e040 con 0x5573c834a890 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.168+0000 7f3419200700 1 -- 172.21.15.46:0/3128016863 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414031840 con 0x5573c834a890 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- 172.21.15.46:0/3128016863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 msgr2=0x5573c8353520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414023000 comp rx=0 tx=0).stop 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- 172.21.15.46:0/3128016863 shutdown_connections 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350970 0x5573c835b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c8353520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 --2- 172.21.15.46:0/3128016863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350050 0x5573c835b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- 172.21.15.46:0/3128016863 >> 172.21.15.46:0/3128016863 conn(0x5573c82a8210 msgr2=0x5573c8343d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- 172.21.15.46:0/3128016863 shutdown_connections 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- 172.21.15.46:0/3128016863 wait complete. 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.169+0000 7f343d81db80 1 -- start start 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 Processor -- start 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c834ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c834c540 con 0x5573c8350970 2024-05-08T23:09:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c834c6b0 con 0x5573c8350050 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.170+0000 7f343d81db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c834c870 con 0x5573c834a890 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f343d81db80 10 client.0 _ll_register_callbacks cb 0x5573c823fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3423400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33958/0 (socket says 172.21.15.46:33958) 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 -- 172.21.15.46:0/2763711748 learned_addr learned my addr 172.21.15.46:0/2763711748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422000700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c834ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 -- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 msgr2=0x5573c834ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c834ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 -- 172.21.15.46:0/2763711748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 msgr2=0x5573c834bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.171+0000 7f3422a00700 1 -- 172.21.15.46:0/2763711748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573c80c7b60 con 0x5573c834a890 2024-05-08T23:09:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f3422a00700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3414005ad0 tx=0x7f3414005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30b5dfb9264a8a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f340cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341402c040 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f34140313b0 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3410004cf0 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3414031590 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f343d81db80 10 client.27238 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f34241105d0 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.172+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3410004cf0 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f341400f830 con 0x5573c834a890 2024-05-08T23:09:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 10 client.27238 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 10 client.27238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 20 client.27238 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c83baa30 0x5573c83bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 4 client.27238 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:09:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573c8292ec0 con 0x5573c83baa30 2024-05-08T23:09:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.173+0000 7f3422000700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c83baa30 0x5573c83bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f3422000700 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c83baa30 0x5573c83bce20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198c296a1bb864b server_cookie=66f991e1f9cc8333 in_seq=0 out_seq=0 2024-05-08T23:09:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f340cc00700 10 client.27238 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f340cc00700 1 -- 172.21.15.46:0/2763711748 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5573c8292ec0 con 0x5573c83baa30 2024-05-08T23:09:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f340cc00700 10 client.27238 handle_command_reply: tid=0 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f343d81db80 1 client.27238 shutdown 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 msgr2=0x5573c835b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.174+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3414005ad0 tx=0x7f3414005b00 comp rx=0 tx=0).stop 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 shutdown_connections 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5573c83baa30 0x5573c83bce20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c8350970 0x5573c834bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573c834a890 0x5573c835b620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 --2- 172.21.15.46:0/2763711748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573c8350050 0x5573c834ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 >> 172.21.15.46:0/2763711748 conn(0x5573c82a8210 msgr2=0x5573c834dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f340cc00700 0 client.27238 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 shutdown_connections 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 1 -- 172.21.15.46:0/2763711748 wait complete. 2024-05-08T23:09:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.175+0000 7f343d81db80 20 client.27238 trim_cache size 0 max 16384 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "91e3fb79-3f5c-4b53-9cae-40c6758675a4": { 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "91e3fb79-3f5c-4b53-9cae-40c6758675a4", 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:09:10.178 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 msgr2=0x7f34080a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 0x7f34080a3870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f341400b6f0 tx=0x7f3414005c00 comp rx=0 tx=0).stop 2024-05-08T23:09:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 msgr2=0x7f3424102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 0x7f3424102760 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f341c00ef70 tx=0x7f341c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 shutdown_connections 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34080a13b0 0x7f34080a3870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f342410d5b0 0x7f3424102760 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34240ffa00 0x7f3424101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 --2- 172.21.15.46:0/4012392208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34241003b0 0x7f3424102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 >> 172.21.15.46:0/4012392208 conn(0x7f34240fb640 msgr2=0x7f342410e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 shutdown_connections 2024-05-08T23:09:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:10.176+0000 7f3423e00700 1 -- 172.21.15.46:0/4012392208 wait complete. 2024-05-08T23:09:10.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:09 smithi046 ceph-mon[36898]: pgmap v163: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:10.196 INFO:tasks.cephfs.filesystem:scrub status for tag:91e3fb79-3f5c-4b53-9cae-40c6758675a4 - {'path': '/', 'tag': '91e3fb79-3f5c-4b53-9cae-40c6758675a4', 'options': 'recursive,force'} 2024-05-08T23:09:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:09 smithi086 ceph-mon[78610]: pgmap v163: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:09 smithi053 ceph-mon[77587]: pgmap v163: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-05-08T23:09:10.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 393950 10.72 MB/sec execute 92 sec latency 16.095 ms 2024-05-08T23:09:11.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:09:11.285 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.286 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:10 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:10 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.287 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.288 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:10 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:11.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 395641 10.67 MB/sec execute 93 sec latency 23.817 ms 2024-05-08T23:09:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:11 smithi086 ceph-mon[78610]: pgmap v164: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 258 op/s 2024-05-08T23:09:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:11 smithi053 ceph-mon[77587]: pgmap v164: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 258 op/s 2024-05-08T23:09:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:11 smithi046 ceph-mon[36898]: pgmap v164: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 258 op/s 2024-05-08T23:09:12.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 397720 10.71 MB/sec execute 94 sec latency 10.415 ms 2024-05-08T23:09:13.217 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:12 smithi086 ceph-mon[78610]: pgmap v165: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-08T23:09:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:12 smithi053 ceph-mon[77587]: pgmap v165: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-08T23:09:13.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:12 smithi046 ceph-mon[36898]: pgmap v165: 129 pgs: 129 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-08T23:09:13.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 400094 10.71 MB/sec execute 95 sec latency 11.879 ms 2024-05-08T23:09:14.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 401650 10.69 MB/sec execute 96 sec latency 13.015 ms 2024-05-08T23:09:15.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:15 smithi053 ceph-mon[77587]: pgmap v166: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-05-08T23:09:15.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:15 smithi046 ceph-mon[36898]: pgmap v166: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-05-08T23:09:15.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:15 smithi086 ceph-mon[78610]: pgmap v166: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 259 op/s 2024-05-08T23:09:15.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 404160 10.72 MB/sec execute 97 sec latency 18.900 ms 2024-05-08T23:09:16.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 405835 10.71 MB/sec execute 98 sec latency 9.522 ms 2024-05-08T23:09:17.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:17 smithi053 ceph-mon[77587]: pgmap v167: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:17.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:17 smithi046 ceph-mon[36898]: pgmap v167: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:17.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:17 smithi086 ceph-mon[78610]: pgmap v167: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:17.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 407687 10.70 MB/sec execute 99 sec latency 163.355 ms 2024-05-08T23:09:18.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 408951 10.65 MB/sec execute 100 sec latency 185.553 ms 2024-05-08T23:09:19.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:19 smithi053 ceph-mon[77587]: pgmap v168: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:19.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:19 smithi046 ceph-mon[36898]: pgmap v168: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:19 smithi086 ceph-mon[78610]: pgmap v168: 129 pgs: 129 active+clean; 238 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-08T23:09:19.884 INFO:tasks.workunit.client.0.smithi046.stdout: 1 411613 10.71 MB/sec execute 101 sec latency 11.535 ms 2024-05-08T23:09:20.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:20 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:20.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:20 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:20.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:20 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:20.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 412243 10.62 MB/sec execute 102 sec latency 263.277 ms 2024-05-08T23:09:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:21 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:21 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:21.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:21 smithi053 ceph-mon[77587]: pgmap v169: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 232 op/s 2024-05-08T23:09:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:21 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:21 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:21.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:21 smithi046 ceph-mon[36898]: pgmap v169: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 232 op/s 2024-05-08T23:09:21.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:21 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:21.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:21 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:21.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:21 smithi086 ceph-mon[78610]: pgmap v169: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 232 op/s 2024-05-08T23:09:21.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 413902 10.58 MB/sec execute 103 sec latency 55.222 ms 2024-05-08T23:09:22.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 415593 10.60 MB/sec execute 104 sec latency 12.233 ms 2024-05-08T23:09:23.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:23 smithi053 ceph-mon[77587]: pgmap v170: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-08T23:09:23.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:23 smithi046 ceph-mon[36898]: pgmap v170: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-08T23:09:23.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:23 smithi086 ceph-mon[78610]: pgmap v170: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-08T23:09:23.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 417538 10.57 MB/sec execute 105 sec latency 15.029 ms 2024-05-08T23:09:24.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 419172 10.58 MB/sec execute 106 sec latency 55.724 ms 2024-05-08T23:09:25.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:25 smithi046 ceph-mon[36898]: pgmap v171: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 313 op/s 2024-05-08T23:09:25.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:25 smithi086 ceph-mon[78610]: pgmap v171: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 313 op/s 2024-05-08T23:09:25.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:25 smithi053 ceph-mon[77587]: pgmap v171: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 313 op/s 2024-05-08T23:09:25.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 420287 10.53 MB/sec execute 107 sec latency 163.603 ms 2024-05-08T23:09:26.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 422238 10.54 MB/sec execute 108 sec latency 43.767 ms 2024-05-08T23:09:27.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:27 smithi046 ceph-mon[36898]: pgmap v172: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:27.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:27 smithi086 ceph-mon[78610]: pgmap v172: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:27.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:27 smithi053 ceph-mon[77587]: pgmap v172: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:27.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 423690 10.51 MB/sec execute 109 sec latency 20.684 ms 2024-05-08T23:09:28.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 425884 10.54 MB/sec execute 110 sec latency 16.289 ms 2024-05-08T23:09:29.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:29 smithi046 ceph-mon[36898]: pgmap v173: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:29.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:29 smithi086 ceph-mon[78610]: pgmap v173: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:29.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:29 smithi053 ceph-mon[77587]: pgmap v173: 129 pgs: 129 active+clean; 249 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:09:29.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 427449 10.49 MB/sec execute 111 sec latency 14.544 ms 2024-05-08T23:09:30.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 429589 10.52 MB/sec execute 112 sec latency 17.086 ms 2024-05-08T23:09:31.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:31 smithi046 ceph-mon[36898]: pgmap v174: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 367 op/s 2024-05-08T23:09:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:31 smithi086 ceph-mon[78610]: pgmap v174: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 367 op/s 2024-05-08T23:09:31.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:31 smithi053 ceph-mon[77587]: pgmap v174: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 367 op/s 2024-05-08T23:09:31.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 431422 10.48 MB/sec execute 113 sec latency 16.127 ms 2024-05-08T23:09:32.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 433200 10.50 MB/sec execute 114 sec latency 20.747 ms 2024-05-08T23:09:33.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:33 smithi086 ceph-mon[78610]: pgmap v175: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2024-05-08T23:09:33.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:33 smithi053 ceph-mon[77587]: pgmap v175: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2024-05-08T23:09:33.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:33 smithi046 ceph-mon[36898]: pgmap v175: 129 pgs: 129 active+clean; 252 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2024-05-08T23:09:33.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 435143 10.47 MB/sec execute 115 sec latency 15.235 ms 2024-05-08T23:09:34.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 436860 10.49 MB/sec execute 116 sec latency 12.481 ms 2024-05-08T23:09:35.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:35 smithi086 ceph-mon[78610]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 364 op/s 2024-05-08T23:09:35.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:35 smithi053 ceph-mon[77587]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 364 op/s 2024-05-08T23:09:35.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:35 smithi046 ceph-mon[36898]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 364 op/s 2024-05-08T23:09:35.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 438895 10.47 MB/sec execute 117 sec latency 19.935 ms 2024-05-08T23:09:36.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 440595 10.48 MB/sec execute 118 sec latency 14.167 ms 2024-05-08T23:09:37.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:37 smithi086 ceph-mon[78610]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:37.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:37 smithi053 ceph-mon[77587]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:37.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:37 smithi046 ceph-mon[36898]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:37.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 442749 10.48 MB/sec execute 119 sec latency 28.376 ms 2024-05-08T23:09:38.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 444348 10.46 MB/sec execute 120 sec latency 16.205 ms 2024-05-08T23:09:39.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:39 smithi086 ceph-mon[78610]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:39 smithi053 ceph-mon[77587]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:39.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:39 smithi046 ceph-mon[36898]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-05-08T23:09:39.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 446599 10.48 MB/sec execute 121 sec latency 24.212 ms 2024-05-08T23:09:40.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:09:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 -- 172.21.15.46:0/214864566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c90101080 msgr2=0x7f5c90101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 --2- 172.21.15.46:0/214864566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c90101080 0x7f5c90101460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009990 tx=0x7f5c84009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 -- 172.21.15.46:0/214864566 shutdown_connections 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 --2- 172.21.15.46:0/214864566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c901061a0 0x7f5c9010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 --2- 172.21.15.46:0/214864566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c90101080 0x7f5c90101460 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 --2- 172.21.15.46:0/214864566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101a30 0x7f5c90105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 -- 172.21.15.46:0/214864566 >> 172.21.15.46:0/214864566 conn(0x7f5c900fc8e0 msgr2=0x7f5c900fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 -- 172.21.15.46:0/214864566 shutdown_connections 2024-05-08T23:09:40.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.448+0000 7f5c97a00700 1 -- 172.21.15.46:0/214864566 wait complete. 2024-05-08T23:09:40.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.449+0000 7f5c97a00700 1 Processor -- start 2024-05-08T23:09:40.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.449+0000 7f5c97a00700 1 -- start start 2024-05-08T23:09:40.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c901061a0 0x7f5c90199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 0x7f5c9019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9019e840 con 0x7f5c90101080 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9019e9b0 con 0x7f5c901061a0 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c97a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9019eb70 con 0x7f5c9019ddc0 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55886/0 (socket says 172.21.15.46:55886) 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 -- 172.21.15.46:0/1809683120 learned_addr learned my addr 172.21.15.46:0/1809683120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 0x7f5c9019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c901061a0 0x7f5c90199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 -- 172.21.15.46:0/1809683120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 msgr2=0x7f5c9019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 0x7f5c9019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 -- 172.21.15.46:0/1809683120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c901061a0 msgr2=0x7f5c90199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c901061a0 0x7f5c90199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95000700 1 -- 172.21.15.46:0/1809683120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009520 con 0x7f5c90101080 2024-05-08T23:09:40.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.450+0000 7f5c95a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 0x7f5c9019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c95000700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005910 tx=0x7f5c84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c8d600700 1 -- 172.21.15.46:0/1809683120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8401d070 con 0x7f5c90101080 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c8d600700 1 -- 172.21.15.46:0/1809683120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c84021410 con 0x7f5c90101080 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c840095f0 con 0x7f5c90101080 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c8d600700 1 -- 172.21.15.46:0/1809683120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84017650 con 0x7f5c90101080 2024-05-08T23:09:40.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.451+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c901a63b0 con 0x7f5c90101080 2024-05-08T23:09:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.456+0000 7f5c8d600700 1 -- 172.21.15.46:0/1809683120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c84021580 con 0x7f5c90101080 2024-05-08T23:09:40.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 Processor -- start 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 -- start start 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5c8d600700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 0x7f5c780a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d900b150 con 0x5604d90ba720 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d9008b10 con 0x5604d90b9e00 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5c8d600700 1 -- 172.21.15.46:0/1809683120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c840cdfe0 con 0x7f5c90101080 2024-05-08T23:09:40.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d9004400 con 0x5604d90b4890 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.457+0000 7f5c95a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 0x7f5c780a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60492/0 (socket says 172.21.15.46:60492) 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 -- 172.21.15.46:0/2917149855 learned_addr learned my addr 172.21.15.46:0/2917149855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95a00700 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 -- 172.21.15.46:0/2917149855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 msgr2=0x5604d90b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 -- 172.21.15.46:0/2917149855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 msgr2=0x5604d90b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 -- 172.21.15.46:0/2917149855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d8e853b0 con 0x5604d90b9e00 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95000700 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c84097840 tx=0x7f5c84097110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74666bb68ebb45e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c95a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 0x7f5c780a3810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009fd0 tx=0x7f5c88009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.458+0000 7f5c82000700 1 -- 172.21.15.46:0/2917149855 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8409a730 con 0x5604d90b9e00 2024-05-08T23:09:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5c82000700 1 -- 172.21.15.46:0/2917149855 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8409b4d0 con 0x5604d90b9e00 2024-05-08T23:09:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5c82000700 1 -- 172.21.15.46:0/2917149855 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8409b030 con 0x5604d90b9e00 2024-05-08T23:09:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/2917149855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 msgr2=0x5604d90ba1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c84097840 tx=0x7f5c84097110 comp rx=0 tx=0).stop 2024-05-08T23:09:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/2917149855 shutdown_connections 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b9e00 0x5604d90ba1e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/2917149855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.459+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/2917149855 >> 172.21.15.46:0/2917149855 conn(0x5604d9012210 msgr2=0x5604d90aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.460+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/2917149855 shutdown_connections 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.460+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/2917149855 wait complete. 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.460+0000 7f5cab5d1b80 1 -- start start 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.460+0000 7f5cab5d1b80 1 Processor -- start 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 0x5604d90c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d8fa0560 con 0x5604d90ba720 2024-05-08T23:09:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d8fa0200 con 0x5604d90b4890 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d8e876a0 con 0x5604d90b9e00 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5cab5d1b80 10 client.0 _ll_register_callbacks cb 0x5604d8fa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c95000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 0x5604d90c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c95a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c95a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90be110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55896/0 (socket says 172.21.15.46:55896) 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38618/0 (socket says 172.21.15.46:38618) 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c95000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 0x5604d90c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60506/0 (socket says 172.21.15.46:60506) 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.461+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3232436220 learned_addr learned my addr 172.21.15.46:0/3232436220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 msgr2=0x5604d90c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 0x5604d90c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3232436220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 msgr2=0x5604d90be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3232436220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c901a63b0 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009fd0 tx=0x7f5c7c012af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa6e2d9ba56a233 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c76a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c017070 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.462+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604d8f5d270 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c7c00f4e0 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7c00f6a0 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 10 client.27250 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5604d8e31b60 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604d8f5d270 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c7c0108a0 con 0x5604d90b9e00 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 10 client.27250 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 10 client.27250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 20 client.27250 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d90cde80 0x5604d90d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 4 client.27250 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.463+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604d9083810 con 0x5604d90cde80 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.464+0000 7f5c95000700 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d90cde80 0x5604d90d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.464+0000 7f5c95000700 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d90cde80 0x5604d90d02a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352ea0e63bd171b server_cookie=3fb8a1f766413cb4 in_seq=0 out_seq=0 2024-05-08T23:09:40.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.464+0000 7f5c76a00700 10 client.27250 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5c76a00700 1 -- 172.21.15.46:0/3232436220 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604d9083810 con 0x5604d90cde80 2024-05-08T23:09:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5c76a00700 10 client.27250 handle_command_reply: tid=0 2024-05-08T23:09:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 client.27250 shutdown 2024-05-08T23:09:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 msgr2=0x5604d90bd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009fd0 tx=0x7f5c7c012af0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 shutdown_connections 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d90cde80 0x5604d90d02a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90ba720 0x5604d90be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90b4890 0x5604d90c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5c76a00700 0 client.27250 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3232436220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90bd9d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 >> 172.21.15.46:0/3232436220 conn(0x5604d9012210 msgr2=0x5604d90ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 shutdown_connections 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.465+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3232436220 wait complete. 2024-05-08T23:09:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.466+0000 7f5cab5d1b80 20 client.27250 trim_cache size 0 max 16384 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 Processor -- start 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 -- start start 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 0x5604d90b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d90aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d90d5470 con 0x5604d90b9e00 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d90ca200 con 0x5604d90ba720 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.484+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d90b8410 con 0x5604d90b4890 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c8fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 0x5604d90b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38620/0 (socket says 172.21.15.46:38620) 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 -- 172.21.15.46:0/4100664692 learned_addr learned my addr 172.21.15.46:0/4100664692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95a00700 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d90aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 -- 172.21.15.46:0/4100664692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 msgr2=0x5604d90aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d90aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 -- 172.21.15.46:0/4100664692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 msgr2=0x5604d90b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 0x5604d90b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 -- 172.21.15.46:0/4100664692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d8fca560 con 0x5604d90b4890 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 0x5604d90b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.485+0000 7f5c95000700 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5c8409aad0 tx=0x7f5c8409b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7df848bc79efaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5c82a00700 1 -- 172.21.15.46:0/4100664692 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8409add0 con 0x5604d90b4890 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5c82a00700 1 -- 172.21.15.46:0/4100664692 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8409ec00 con 0x5604d90b4890 2024-05-08T23:09:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5c82a00700 1 -- 172.21.15.46:0/4100664692 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8409b290 con 0x5604d90b4890 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/4100664692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 msgr2=0x5604d90be0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5c8409aad0 tx=0x7f5c8409b640 comp rx=0 tx=0).stop 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/4100664692 shutdown_connections 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d90aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b4890 0x5604d90be0b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/4100664692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b9e00 0x5604d90b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/4100664692 >> 172.21.15.46:0/4100664692 conn(0x5604d9012210 msgr2=0x5604d90ab210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/4100664692 shutdown_connections 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.486+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/4100664692 wait complete. 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.487+0000 7f5cab5d1b80 1 -- start start 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.487+0000 7f5cab5d1b80 1 Processor -- start 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d911d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d911d900 con 0x5604d90b4890 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d911da70 con 0x5604d90ba720 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d911dc30 con 0x5604d90b9e00 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5cab5d1b80 10 client.0 _ll_register_callbacks cb 0x5604d8fa9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55922/0 (socket says 172.21.15.46:55922) 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95000700 1 -- 172.21.15.46:0/3271088835 learned_addr learned my addr 172.21.15.46:0/3271088835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95a00700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d911d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 msgr2=0x5604d911d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d911d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3271088835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 msgr2=0x5604d90c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 -- 172.21.15.46:0/3271088835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604d8e31b60 con 0x5604d90b9e00 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95000700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c95a00700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d911d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c8fe00700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90c37d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c010d50 tx=0x7f5c7c012e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f067413b543e166e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.488+0000 7f5c76a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c017040 con 0x5604d90b9e00 2024-05-08T23:09:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c8409b4d0 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5c7c00f9b0 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7c00fb70 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5cab5d1b80 10 client.27256 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5604d8e853b0 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8409b4d0 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c7c003680 con 0x5604d90b9e00 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.489+0000 7f5cab5d1b80 10 client.27256 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5cab5d1b80 10 client.27256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5cab5d1b80 20 client.27256 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d9125440 0x5604d9127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5cab5d1b80 4 client.27256 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604d8ffcec0 con 0x5604d9125440 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5c95000700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d9125440 0x5604d9127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5c95000700 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d9125440 0x5604d9127860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b727423e0841a8fb server_cookie=391dab2131265a12 in_seq=0 out_seq=0 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.490+0000 7f5c76a00700 10 client.27256 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5c76a00700 1 -- 172.21.15.46:0/3271088835 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5604d8ffcec0 con 0x5604d9125440 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5c76a00700 10 client.27256 handle_command_reply: tid=0 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 client.27256 shutdown 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 msgr2=0x5604d90c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90c37d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c010d50 tx=0x7f5c7c012e40 comp rx=0 tx=0).stop 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 shutdown_connections 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5604d90ba720 0x5604d911d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5604d9125440 0x5604d9127860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604d90b4890 0x5604d90c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 --2- 172.21.15.46:0/3271088835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604d90b9e00 0x5604d90c37d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5c76a00700 0 client.27256 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 >> 172.21.15.46:0/3271088835 conn(0x5604d9012210 msgr2=0x5604d90b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 shutdown_connections 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.491+0000 7f5cab5d1b80 1 -- 172.21.15.46:0/3271088835 wait complete. 2024-05-08T23:09:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.492+0000 7f5cab5d1b80 20 client.27256 trim_cache size 0 max 16384 2024-05-08T23:09:40.496 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:09:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:09:40.496 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.492+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 msgr2=0x7f5c780a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.492+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 0x7f5c780a3810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009fd0 tx=0x7f5c88009920 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.492+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 msgr2=0x7f5c90199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.492+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005910 tx=0x7f5c84004730 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 shutdown_connections 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9019ddc0 0x7f5c9019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c780a1350 0x7f5c780a3810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c90101080 0x7f5c90199530 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 --2- 172.21.15.46:0/1809683120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c901061a0 0x7f5c90199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 >> 172.21.15.46:0/1809683120 conn(0x7f5c900fc8e0 msgr2=0x7f5c900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 shutdown_connections 2024-05-08T23:09:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.493+0000 7f5c97a00700 1 -- 172.21.15.46:0/1809683120 wait complete. 2024-05-08T23:09:40.513 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:09:40.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:09:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 -- 172.21.15.46:0/3999836428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 msgr2=0x7f768c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 --2- 172.21.15.46:0/3999836428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c1021f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f767c009990 tx=0x7f767c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 -- 172.21.15.46:0/3999836428 shutdown_connections 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 --2- 172.21.15.46:0/3999836428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c10b230 0x7f768c10d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 --2- 172.21.15.46:0/3999836428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c1021f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 --2- 172.21.15.46:0/3999836428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c102730 0x7f768c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 -- 172.21.15.46:0/3999836428 >> 172.21.15.46:0/3999836428 conn(0x7f768c0fb610 msgr2=0x7f768c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 -- 172.21.15.46:0/3999836428 shutdown_connections 2024-05-08T23:09:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.741+0000 7f7690e00700 1 -- 172.21.15.46:0/3999836428 wait complete. 2024-05-08T23:09:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.742+0000 7f7690e00700 1 Processor -- start 2024-05-08T23:09:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.742+0000 7f7690e00700 1 -- start start 2024-05-08T23:09:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c102730 0x7f768c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 0x7f768c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c1a5540 con 0x7f768c102730 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c1a56b0 con 0x7f768c10b230 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f7690e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c1a5870 con 0x7f768c101e10 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f768b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f768b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38650/0 (socket says 172.21.15.46:38650) 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f768b400700 1 -- 172.21.15.46:0/3401573309 learned_addr learned my addr 172.21.15.46:0/3401573309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f768be00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 0x7f768c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.743+0000 7f768aa00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c102730 0x7f768c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 -- 172.21.15.46:0/3401573309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 msgr2=0x7f768c0ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 -- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c102730 msgr2=0x7f768c0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c102730 0x7f768c0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 -- 172.21.15.46:0/3401573309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0095f0 con 0x7f768c10b230 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768b400700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.744+0000 7f768be00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 0x7f768c1004e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f768400efc0 tx=0x7f768400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.745+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684018070 con 0x7f768c10b230 2024-05-08T23:09:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.745+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768400f040 con 0x7f768c10b230 2024-05-08T23:09:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.745+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400cd40 con 0x7f768c10b230 2024-05-08T23:09:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.745+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c1a5af0 con 0x7f768c10b230 2024-05-08T23:09:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.745+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c1a5fb0 con 0x7f768c10b230 2024-05-08T23:09:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.746+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c106360 con 0x7f768c10b230 2024-05-08T23:09:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.749+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7684003a00 con 0x7f768c10b230 2024-05-08T23:09:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.750+0000 7f7683e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 0x7f76700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.750+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f76840cd480 con 0x7f768c10b230 2024-05-08T23:09:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.750+0000 7f768b400700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 0x7f76700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.751+0000 7f768b400700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 0x7f76700a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:40.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:40.753+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7684096650 con 0x7f768c10b230 2024-05-08T23:09:40.885 INFO:tasks.workunit.client.0.smithi046.stdout: 1 448116 10.46 MB/sec execute 122 sec latency 14.538 ms 2024-05-08T23:09:41.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.059+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f768c069670 con 0x7f768c10b230 2024-05-08T23:09:41.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.060+0000 7f7683e00700 1 -- 172.21.15.46:0/3401573309 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f7684099c70 con 0x7f768c10b230 2024-05-08T23:09:41.059 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:09:41.060 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 msgr2=0x7f76700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 0x7f76700a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c00b260 comp rx=0 tx=0).stop 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 msgr2=0x7f768c1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 0x7f768c1004e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f768400efc0 tx=0x7f768400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 shutdown_connections 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c10b230 0x7f768c1004e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c101e10 0x7f768c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76700a1290 0x7f76700a3750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 --2- 172.21.15.46:0/3401573309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f768c102730 0x7f768c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 >> 172.21.15.46:0/3401573309 conn(0x7f768c0fb610 msgr2=0x7f768c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 shutdown_connections 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.063+0000 7f7690e00700 1 -- 172.21.15.46:0/3401573309 wait complete. 2024-05-08T23:09:41.063 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:09:41.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:09:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 -- 172.21.15.46:0/1352203470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8101a10 msgr2=0x7fa6b8105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 --2- 172.21.15.46:0/1352203470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8101a10 0x7fa6b8105a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009990 tx=0x7fa6a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 -- 172.21.15.46:0/1352203470 shutdown_connections 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 --2- 172.21.15.46:0/1352203470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 0x7fa6b810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 --2- 172.21.15.46:0/1352203470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 --2- 172.21.15.46:0/1352203470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8101a10 0x7fa6b8105a50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 -- 172.21.15.46:0/1352203470 >> 172.21.15.46:0/1352203470 conn(0x7fa6b80fc900 msgr2=0x7fa6b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 -- 172.21.15.46:0/1352203470 shutdown_connections 2024-05-08T23:09:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.308+0000 7fa6bda00700 1 -- 172.21.15.46:0/1352203470 wait complete. 2024-05-08T23:09:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 Processor -- start 2024-05-08T23:09:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 -- start start 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 0x7fa6b80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 0x7fa6b81a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a5b90 con 0x7fa6b8106180 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a5d00 con 0x7fa6b8101060 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.309+0000 7fa6bda00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a5ec0 con 0x7fa6b8073d20 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60572/0 (socket says 172.21.15.46:60572) 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b6a00700 1 -- 172.21.15.46:0/2669326019 learned_addr learned my addr 172.21.15.46:0/2669326019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 0x7fa6b81a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 0x7fa6b80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 -- 172.21.15.46:0/2669326019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 msgr2=0x7fa6b80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 -- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 msgr2=0x7fa6b80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 0x7fa6b80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 -- 172.21.15.46:0/2669326019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40095f0 con 0x7fa6b8073d20 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.310+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 0x7fa6b81a5590 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0007f10 tx=0x7fa6b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.311+0000 7fa6af400700 1 -- 172.21.15.46:0/2669326019 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0009ea0 con 0x7fa6b8073d20 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.311+0000 7fa6af400700 1 -- 172.21.15.46:0/2669326019 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6b000f040 con 0x7fa6b8073d20 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.311+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b81a61a0 con 0x7fa6b8073d20 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.311+0000 7fa6af400700 1 -- 172.21.15.46:0/2669326019 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0004350 con 0x7fa6b8073d20 2024-05-08T23:09:41.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.311+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b81a66d0 con 0x7fa6b8073d20 2024-05-08T23:09:41.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.314+0000 7fa6af400700 1 -- 172.21.15.46:0/2669326019 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa6b000db20 con 0x7fa6b8073d20 2024-05-08T23:09:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.316+0000 7fa6af400700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 0x7fa69c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.316+0000 7fa6d1528b80 1 Processor -- start 2024-05-08T23:09:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.316+0000 7fa6af400700 1 -- 172.21.15.46:0/2669326019 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa6b00cc8c0 con 0x7fa6b8073d20 2024-05-08T23:09:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.316+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 0x7fa69c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.316+0000 7fa6d1528b80 1 -- start start 2024-05-08T23:09:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 0x55b8148c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b814817150 con 0x55b8148c5e00 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b814814b10 con 0x55b8148c0890 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b814810400 con 0x55b8148c6720 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 0x55b8148c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38682/0 (socket says 172.21.15.46:38682) 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b6000700 1 -- 172.21.15.46:0/2420772435 learned_addr learned my addr 172.21.15.46:0/2420772435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 0x7fa69c0a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8011590 tx=0x7fa6a8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.317+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 -- 172.21.15.46:0/2420772435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 msgr2=0x55b8148c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 -- 172.21.15.46:0/2420772435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 msgr2=0x55b8148c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 0x55b8148c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 -- 172.21.15.46:0/2420772435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8146913b0 con 0x55b8148c6720 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 0x55b8148c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009990 tx=0x7fa6a400b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce095c7859437b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6a3e00700 1 -- 172.21.15.46:0/2420772435 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401d070 con 0x55b8148c6720 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6a3e00700 1 -- 172.21.15.46:0/2420772435 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a401e040 con 0x55b8148c6720 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.318+0000 7fa6a3e00700 1 -- 172.21.15.46:0/2420772435 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40047f0 con 0x55b8148c6720 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2420772435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 msgr2=0x55b8148c0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009990 tx=0x7fa6a400b5b0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2420772435 shutdown_connections 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c5e00 0x55b8148c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2420772435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c6720 0x55b8148c0350 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2420772435 >> 172.21.15.46:0/2420772435 conn(0x55b81481e210 msgr2=0x55b8148badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2420772435 shutdown_connections 2024-05-08T23:09:41.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.319+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2420772435 wait complete. 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 -- start start 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 Processor -- start 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8147ac560 con 0x55b8148c6720 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8147ac200 con 0x55b8148c0890 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8146936a0 con 0x55b8148c5e00 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.320+0000 7fa6d1528b80 10 client.0 _ll_register_callbacks cb 0x55b8147b5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38686/0 (socket says 172.21.15.46:38686) 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 -- 172.21.15.46:0/424814113 learned_addr learned my addr 172.21.15.46:0/424814113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b7400700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 -- 172.21.15.46:0/424814113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 msgr2=0x55b8148c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 -- 172.21.15.46:0/424814113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 msgr2=0x55b8148ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 -- 172.21.15.46:0/424814113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a8017070 con 0x55b8148c5e00 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6b6000700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400b2e0 tx=0x7fa6a400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53245a89a302342 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.321+0000 7fa6a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401d030 con 0x55b8148c5e00 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa6a4004700 con 0x55b8148c5e00 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b814769270 con 0x55b8148c5e00 2024-05-08T23:09:41.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a400beb0 con 0x55b8148c5e00 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 10 client.27274 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa6b810e570 con 0x55b8148c5e00 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b814769270 con 0x55b8148c5e00 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6a400f810 con 0x55b8148c5e00 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 10 client.27274 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 10 client.27274 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 20 client.27274 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.322+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8148d9d30 0x55b8148dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.323+0000 7fa6d1528b80 4 client.27274 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.323+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b81488f810 con 0x55b8148d9d30 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.323+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8148d9d30 0x55b8148dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.323+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8148d9d30 0x55b8148dc150 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e77f56b3143547b server_cookie=5a9edd117d0f8e0 in_seq=0 out_seq=0 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.323+0000 7fa6a0c00700 10 client.27274 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6a0c00700 1 -- 172.21.15.46:0/424814113 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b81488f810 con 0x55b8148d9d30 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6a0c00700 10 client.27274 handle_command_reply: tid=0 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 client.27274 shutdown 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 msgr2=0x55b8148ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400b2e0 tx=0x7fa6a400b920 comp rx=0 tx=0).stop 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 shutdown_connections 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8148d9d30 0x55b8148dc150 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6a0c00700 0 client.27274 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/424814113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148ca030 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 >> 172.21.15.46:0/424814113 conn(0x55b81481e210 msgr2=0x55b8148ba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 shutdown_connections 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.324+0000 7fa6d1528b80 1 -- 172.21.15.46:0/424814113 wait complete. 2024-05-08T23:09:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.325+0000 7fa6d1528b80 20 client.27274 trim_cache size 0 max 16384 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 Processor -- start 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 -- start start 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c5e00 0x55b8148d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148d6d60 con 0x55b8148c6720 2024-05-08T23:09:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148bb8e0 con 0x55b8148c5e00 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148cc660 con 0x55b8148c0890 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38702/0 (socket says 172.21.15.46:38702) 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6b6a00700 1 -- 172.21.15.46:0/711196352 learned_addr learned my addr 172.21.15.46:0/711196352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6b6000700 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c5e00 0x55b8148d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.340+0000 7fa6b7400700 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 -- 172.21.15.46:0/711196352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c5e00 msgr2=0x55b8148d5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c5e00 0x55b8148d5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 -- 172.21.15.46:0/711196352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 msgr2=0x55b8148bad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148bad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 -- 172.21.15.46:0/711196352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b81463db60 con 0x55b8148c0890 2024-05-08T23:09:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55b8148dc9f0 tx=0x7fa6a8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f16b07bf8f6a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6acc00700 1 -- 172.21.15.46:0/711196352 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8022030 con 0x55b8148c0890 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6acc00700 1 -- 172.21.15.46:0/711196352 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a8023040 con 0x55b8148c0890 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.341+0000 7fa6acc00700 1 -- 172.21.15.46:0/711196352 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a802c2b0 con 0x55b8148c0890 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- 172.21.15.46:0/711196352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 msgr2=0x55b8148bca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55b8148dc9f0 tx=0x7fa6a8016000 comp rx=0 tx=0).stop 2024-05-08T23:09:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- 172.21.15.46:0/711196352 shutdown_connections 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148bad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c0890 0x55b8148bca90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/711196352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c5e00 0x55b8148d5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- 172.21.15.46:0/711196352 >> 172.21.15.46:0/711196352 conn(0x55b81481e210 msgr2=0x55b8148c83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- 172.21.15.46:0/711196352 shutdown_connections 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- 172.21.15.46:0/711196352 wait complete. 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.342+0000 7fa6d1528b80 1 -- start start 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 Processor -- start 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148cfbb0 con 0x55b8148c6720 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148cfd20 con 0x55b8148c0890 2024-05-08T23:09:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8148cfee0 con 0x55b8148c5e00 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6d1528b80 10 client.0 _ll_register_callbacks cb 0x55b8147b5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.343+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38712/0 (socket says 172.21.15.46:38712) 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 -- 172.21.15.46:0/2546231899 learned_addr learned my addr 172.21.15.46:0/2546231899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b7400700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 -- 172.21.15.46:0/2546231899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 msgr2=0x55b8148d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 -- 172.21.15.46:0/2546231899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 msgr2=0x55b8148cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 -- 172.21.15.46:0/2546231899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a801d770 con 0x55b8148c5e00 2024-05-08T23:09:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6b6000700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005b00 tx=0x7fa6a4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b27cebe5874657 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.344+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401d040 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa6a4000e40 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6a401e040 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a400b990 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 10 client.27280 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b8147d6560 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a401e040 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6a402ba60 con 0x55b8148c5e00 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 10 client.27280 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 10 client.27280 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:09:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 20 client.27280 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8149303a0 0x55b8149327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 4 client.27280 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.345+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b814808ec0 con 0x55b8149303a0 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8149303a0 0x55b8149327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6b6a00700 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8149303a0 0x55b8149327c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f77b68d9b073a0b server_cookie=7698065a1335718 in_seq=0 out_seq=0 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6a0c00700 10 client.27280 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6a0c00700 1 -- 172.21.15.46:0/2546231899 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b814808ec0 con 0x55b8149303a0 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6a0c00700 10 client.27280 handle_command_reply: tid=0 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.346+0000 7fa6d1528b80 1 client.27280 shutdown 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 msgr2=0x55b8148cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005b00 tx=0x7fa6a4000bc0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 shutdown_connections 2024-05-08T23:09:41.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b8149303a0 0x55b8149327c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8148c6720 0x55b8148cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8148c0890 0x55b8148d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6a0c00700 0 client.27280 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 --2- 172.21.15.46:0/2546231899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8148c5e00 0x55b8148cf0f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 >> 172.21.15.46:0/2546231899 conn(0x55b81481e210 msgr2=0x55b8148c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 shutdown_connections 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 1 -- 172.21.15.46:0/2546231899 wait complete. 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.347+0000 7fa6d1528b80 20 client.27280 trim_cache size 0 max 16384 2024-05-08T23:09:41.352 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 msgr2=0x7fa69c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 0x7fa69c0a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8011590 tx=0x7fa6a8009260 comp rx=0 tx=0).stop 2024-05-08T23:09:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 msgr2=0x7fa6b81a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 0x7fa6b81a5590 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0007f10 tx=0x7fa6b000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 shutdown_connections 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b8073d20 0x7fa6b81a5590 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6b8101060 0x7fa6b80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa69c0a1310 0x7fa69c0a37d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 --2- 172.21.15.46:0/2669326019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b8106180 0x7fa6b80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 >> 172.21.15.46:0/2669326019 conn(0x7fa6b80fc900 msgr2=0x7fa6b80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 shutdown_connections 2024-05-08T23:09:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.348+0000 7fa6bda00700 1 -- 172.21.15.46:0/2669326019 wait complete. 2024-05-08T23:09:41.364 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:41 smithi046 ceph-mon[36898]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 277 op/s 2024-05-08T23:09:41.364 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:41 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3401573309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:41.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:09:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.586+0000 7f76a5600700 1 -- 172.21.15.46:0/2258227448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0106d30 msgr2=0x7f76a0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.586+0000 7f76a5600700 1 --2- 172.21.15.46:0/2258227448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0106d30 0x7f76a0107110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7690009990 tx=0x7f7690009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 -- 172.21.15.46:0/2258227448 shutdown_connections 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 --2- 172.21.15.46:0/2258227448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068b20 0x7f76a010dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 --2- 172.21.15.46:0/2258227448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0106d30 0x7f76a0107110 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 --2- 172.21.15.46:0/2258227448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0068160 0x7f76a00685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 -- 172.21.15.46:0/2258227448 >> 172.21.15.46:0/2258227448 conn(0x7f76a0075710 msgr2=0x7f76a0075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 -- 172.21.15.46:0/2258227448 shutdown_connections 2024-05-08T23:09:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.587+0000 7f76a5600700 1 -- 172.21.15.46:0/2258227448 wait complete. 2024-05-08T23:09:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.588+0000 7f76a5600700 1 Processor -- start 2024-05-08T23:09:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.588+0000 7f76a5600700 1 -- start start 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068160 0x7f76a00ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 0x7f76a01003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01a5360 con 0x7f76a0068b20 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01a54d0 con 0x7f76a0068160 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a5600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01a5690 con 0x7f76a0106d30 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f769fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068160 0x7f76a00ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38738/0 (socket says 172.21.15.46:38738) 2024-05-08T23:09:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f76a4c00700 1 -- 172.21.15.46:0/1985342189 learned_addr learned my addr 172.21.15.46:0/1985342189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.589+0000 7f769f400700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 0x7f76a01003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 -- 172.21.15.46:0/1985342189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068160 msgr2=0x7f76a00ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068160 0x7f76a00ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 -- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 msgr2=0x7f76a01003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 0x7f76a01003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 -- 172.21.15.46:0/1985342189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900095f0 con 0x7f76a0106d30 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f769f400700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 0x7f76a01003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a4c00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f769800ef30 tx=0x7f769800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f769cc00700 1 -- 172.21.15.46:0/1985342189 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698018070 con 0x7f76a0106d30 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01a5910 con 0x7f76a0106d30 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f769cc00700 1 -- 172.21.15.46:0/1985342189 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f769800f040 con 0x7f76a0106d30 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.590+0000 7f769cc00700 1 -- 172.21.15.46:0/1985342189 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800cdb0 con 0x7f76a0106d30 2024-05-08T23:09:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.591+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01a5e70 con 0x7f76a0106d30 2024-05-08T23:09:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.593+0000 7f769cc00700 1 -- 172.21.15.46:0/1985342189 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7698003a00 con 0x7f76a0106d30 2024-05-08T23:09:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.595+0000 7f769cc00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 0x7f76840a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.595+0000 7f769cc00700 1 -- 172.21.15.46:0/1985342189 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f76980ccc40 con 0x7f76a0106d30 2024-05-08T23:09:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.595+0000 7f769fe00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 0x7f76840a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.596+0000 7f769fe00700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 0x7f76840a3870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f769000b2e0 tx=0x7f7690005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.596+0000 7f76ba68ab80 1 Processor -- start 2024-05-08T23:09:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.596+0000 7f76ba68ab80 1 -- start start 2024-05-08T23:09:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5050 0x55dfa11d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa1126150 con 0x55dfa11d5050 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa1123b10 con 0x55dfa11cf890 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa111f400 con 0x55dfa11d5970 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38742/0 (socket says 172.21.15.46:38742) 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5050 0x55dfa11d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 -- 172.21.15.46:0/4206137696 learned_addr learned my addr 172.21.15.46:0/4206137696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f76a4c00700 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 -- 172.21.15.46:0/4206137696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 msgr2=0x55dfa11cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 -- 172.21.15.46:0/4206137696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5050 msgr2=0x55dfa11d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5050 0x55dfa11d5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.597+0000 7f769f400700 1 -- 172.21.15.46:0/4206137696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa0fa03b0 con 0x55dfa11d5970 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.598+0000 7f769f400700 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c009d80 tx=0x7f768c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786323cc07fb80b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.598+0000 7f7695200700 1 -- 172.21.15.46:0/4206137696 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011e20 con 0x55dfa11d5970 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.598+0000 7f7695200700 1 -- 172.21.15.46:0/4206137696 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768c012910 con 0x55dfa11d5970 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.598+0000 7f7695200700 1 -- 172.21.15.46:0/4206137696 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00f780 con 0x55dfa11d5970 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 -- 172.21.15.46:0/4206137696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 msgr2=0x55dfa11cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c009d80 tx=0x7f768c0116c0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 -- 172.21.15.46:0/4206137696 shutdown_connections 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5050 0x55dfa11d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/4206137696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11cf350 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 -- 172.21.15.46:0/4206137696 >> 172.21.15.46:0/4206137696 conn(0x55dfa112d210 msgr2=0x55dfa11ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.599+0000 7f76ba68ab80 1 -- 172.21.15.46:0/4206137696 shutdown_connections 2024-05-08T23:09:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.600+0000 7f76ba68ab80 1 -- 172.21.15.46:0/4206137696 wait complete. 2024-05-08T23:09:41.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.600+0000 7f76ba68ab80 1 -- start start 2024-05-08T23:09:41.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 Processor -- start 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa10bb560 con 0x55dfa11d5970 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa10bb200 con 0x55dfa11cf890 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa0fa26a0 con 0x55dfa11d5050 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.601+0000 7f76ba68ab80 10 client.0 _ll_register_callbacks cb 0x55dfa10c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38748/0 (socket says 172.21.15.46:38748) 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 -- 172.21.15.46:0/1315692014 learned_addr learned my addr 172.21.15.46:0/1315692014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f76a4c00700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769fe00700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 -- 172.21.15.46:0/1315692014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 msgr2=0x55dfa11d8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11d8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 -- 172.21.15.46:0/1315692014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 msgr2=0x55dfa11d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 -- 172.21.15.46:0/1315692014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c0099b0 con 0x55dfa11d5050 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769fe00700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f769f400700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f768c000c00 tx=0x7f768c012cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71250312a56f9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.602+0000 7f7689600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c012660 con 0x55dfa11d5050 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f768c004040 con 0x55dfa11d5050 2024-05-08T23:09:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfa1078270 con 0x55dfa11d5050 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f768c004200 con 0x55dfa11d5050 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 10 client.27292 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dfa0f4cb60 con 0x55dfa11d5050 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfa1078270 con 0x55dfa11d5050 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f768c022070 con 0x55dfa11d5050 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 10 client.27292 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 10 client.27292 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 20 client.27292 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa11e9220 0x55dfa11eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 4 client.27292 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.603+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfa119e810 con 0x55dfa11e9220 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.604+0000 7f769fe00700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa11e9220 0x55dfa11eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.604+0000 7f769fe00700 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa11e9220 0x55dfa11eb640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6bc432f28d08975 server_cookie=958cd01349308515 in_seq=0 out_seq=0 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.604+0000 7f7689600700 10 client.27292 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f7689600700 1 -- 172.21.15.46:0/1315692014 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfa119e810 con 0x55dfa11e9220 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f7689600700 10 client.27292 handle_command_reply: tid=0 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 client.27292 shutdown 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 msgr2=0x55dfa11d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f768c000c00 tx=0x7f768c012cb0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 shutdown_connections 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa11e9220 0x55dfa11eb640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11cf890 0x55dfa11d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1315692014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5050 0x55dfa11d9160 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 >> 172.21.15.46:0/1315692014 conn(0x55dfa112d210 msgr2=0x55dfa11ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.605+0000 7f7689600700 0 client.27292 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.606+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 shutdown_connections 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.606+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1315692014 wait complete. 2024-05-08T23:09:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.606+0000 7f76ba68ab80 20 client.27292 trim_cache size 0 max 16384 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.622+0000 7f76ba68ab80 1 Processor -- start 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 -- start start 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11cf890 0x55dfa11cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11c4ad0 con 0x55dfa11cf890 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11da250 con 0x55dfa11d5050 2024-05-08T23:09:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11c4880 con 0x55dfa11d5970 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f769f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f769fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11cf890 0x55dfa11cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f769f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60652/0 (socket says 172.21.15.46:60652) 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f769f400700 1 -- 172.21.15.46:0/1367284641 learned_addr learned my addr 172.21.15.46:0/1367284641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.623+0000 7f76a4c00700 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 -- 172.21.15.46:0/1367284641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 msgr2=0x55dfa11e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 -- 172.21.15.46:0/1367284641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11cf890 msgr2=0x55dfa11cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11cf890 0x55dfa11cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 -- 172.21.15.46:0/1367284641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa10e5560 con 0x55dfa11d5050 2024-05-08T23:09:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f769f400700 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f768c005d90 tx=0x7f768c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e54a76d1627dc1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f7695c00700 1 -- 172.21.15.46:0/1367284641 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0095c0 con 0x55dfa11d5050 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f7695c00700 1 -- 172.21.15.46:0/1367284641 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768c009c80 con 0x55dfa11d5050 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.624+0000 7f7695c00700 1 -- 172.21.15.46:0/1367284641 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c02d8c0 con 0x55dfa11d5050 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1367284641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 msgr2=0x55dfa11e0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f768c005d90 tx=0x7f768c009870 comp rx=0 tx=0).stop 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1367284641 shutdown_connections 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11d5970 0x55dfa11e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11cf890 0x55dfa11cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/1367284641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11e0320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1367284641 >> 172.21.15.46:0/1367284641 conn(0x55dfa112d210 msgr2=0x55dfa11cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1367284641 shutdown_connections 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.625+0000 7f76ba68ab80 1 -- 172.21.15.46:0/1367284641 wait complete. 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.626+0000 7f76ba68ab80 1 -- start start 2024-05-08T23:09:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.626+0000 7f76ba68ab80 1 Processor -- start 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.626+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 0x55dfa11e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.626+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76ba68ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11d1480 con 0x55dfa11d5970 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11d15f0 con 0x55dfa11d5050 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76ba68ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa11d17b0 con 0x55dfa11cf890 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76ba68ab80 10 client.0 _ll_register_callbacks cb 0x55dfa10c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f76a4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60654/0 (socket says 172.21.15.46:60654) 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 -- 172.21.15.46:0/871638025 learned_addr learned my addr 172.21.15.46:0/871638025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769fe00700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 0x55dfa11e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 -- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 msgr2=0x55dfa11e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 0x55dfa11e0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 -- 172.21.15.46:0/871638025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 msgr2=0x55dfa11d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769f400700 1 -- 172.21.15.46:0/871638025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfa0f4cb60 con 0x55dfa11d5050 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.627+0000 7f769fe00700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 0x55dfa11e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f769f400700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f768c005a40 tx=0x7f768c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740a097d2dc37467 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f7689600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c024520 con 0x55dfa11d5050 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f768c02d5b0 con 0x55dfa11d5050 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f768c012910 con 0x55dfa11d5050 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f768c0050c0 con 0x55dfa11d5050 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f76ba68ab80 10 client.27236 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.628+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dfa0fa4cd0 con 0x55dfa11d5050 2024-05-08T23:09:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768c012910 con 0x55dfa11d5050 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f768c04a020 con 0x55dfa11d5050 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 10 client.27236 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 10 client.27236 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 20 client.27236 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa123f8b0 0x55dfa1241cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 4 client.27236 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfa1117ec0 con 0x55dfa123f8b0 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f769fe00700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa123f8b0 0x55dfa1241cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f769fe00700 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa123f8b0 0x55dfa1241cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=699408a645392686 server_cookie=f19d0cce34f6e784 in_seq=0 out_seq=0 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.629+0000 7f7689600700 10 client.27236 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f7689600700 1 -- 172.21.15.46:0/871638025 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfa1117ec0 con 0x55dfa123f8b0 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f7689600700 10 client.27236 handle_command_reply: tid=0 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 client.27236 shutdown 2024-05-08T23:09:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 msgr2=0x55dfa11d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f768c005a40 tx=0x7f768c009490 comp rx=0 tx=0).stop 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 shutdown_connections 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55dfa123f8b0 0x55dfa1241cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dfa11d5970 0x55dfa11d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfa11cf890 0x55dfa11e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 --2- 172.21.15.46:0/871638025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfa11d5050 0x55dfa11d0a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f7689600700 0 client.27236 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 >> 172.21.15.46:0/871638025 conn(0x55dfa112d210 msgr2=0x55dfa11cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 shutdown_connections 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 1 -- 172.21.15.46:0/871638025 wait complete. 2024-05-08T23:09:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.630+0000 7f76ba68ab80 20 client.27236 trim_cache size 0 max 16384 2024-05-08T23:09:41.634 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 msgr2=0x7f76840a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 0x7f76840a3870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f769000b2e0 tx=0x7f7690005fd0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 msgr2=0x7f76a01008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f769800ef30 tx=0x7f769800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 shutdown_connections 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a0106d30 0x7f76a01008e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0068160 0x7f76a00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f76840a13b0 0x7f76840a3870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 --2- 172.21.15.46:0/1985342189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0068b20 0x7f76a01003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 >> 172.21.15.46:0/1985342189 conn(0x7f76a0075710 msgr2=0x7f76a00fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.631+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 shutdown_connections 2024-05-08T23:09:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.632+0000 7f76a5600700 1 -- 172.21.15.46:0/1985342189 wait complete. 2024-05-08T23:09:41.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:09:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:41 smithi086 ceph-mon[78610]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 277 op/s 2024-05-08T23:09:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:41 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3401573309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:41 smithi053 ceph-mon[77587]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 277 op/s 2024-05-08T23:09:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:41 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3401573309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 -- 172.21.15.46:0/2582662244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 msgr2=0x7f946c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 --2- 172.21.15.46:0/2582662244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c101460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098f0 tx=0x7f945c009c00 comp rx=0 tx=0).stop 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 -- 172.21.15.46:0/2582662244 shutdown_connections 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 --2- 172.21.15.46:0/2582662244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 0x7f946c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 --2- 172.21.15.46:0/2582662244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c101460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 --2- 172.21.15.46:0/2582662244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c101a30 0x7f946c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.886+0000 7f9473000700 1 -- 172.21.15.46:0/2582662244 >> 172.21.15.46:0/2582662244 conn(0x7f946c0fc8e0 msgr2=0x7f946c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 450241 10.47 MB/sec execute 123 sec latency 15.717 ms 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.887+0000 7f9473000700 1 -- 172.21.15.46:0/2582662244 shutdown_connections 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.887+0000 7f9473000700 1 -- 172.21.15.46:0/2582662244 wait complete. 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.888+0000 7f9473000700 1 Processor -- start 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.888+0000 7f9473000700 1 -- start start 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 0x7f946c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c073d20 0x7f946c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a5b50 con 0x7f946c073d20 2024-05-08T23:09:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a5cc0 con 0x7f946c1061a0 2024-05-08T23:09:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9473000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1a5e80 con 0x7f946c101080 2024-05-08T23:09:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f946be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f946be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42840/0 (socket says 172.21.15.46:42840) 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f946be00700 1 -- 172.21.15.46:0/1426976451 learned_addr learned my addr 172.21.15.46:0/1426976451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f9471000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c073d20 0x7f946c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.889+0000 7f946b400700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 0x7f946c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 -- 172.21.15.46:0/1426976451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 msgr2=0x7f946c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 0x7f946c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 -- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c073d20 msgr2=0x7f946c1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c073d20 0x7f946c1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 -- 172.21.15.46:0/1426976451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580095f0 con 0x7f946c101080 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f946be00700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f945c005950 tx=0x7f945c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f9468c00700 1 -- 172.21.15.46:0/1426976451 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c01c070 con 0x7f946c101080 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f9468c00700 1 -- 172.21.15.46:0/1426976451 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f945c00b9c0 con 0x7f946c101080 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f9468c00700 1 -- 172.21.15.46:0/1426976451 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0038e0 con 0x7f946c101080 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.890+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c009520 con 0x7f946c101080 2024-05-08T23:09:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.891+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1a6340 con 0x7f946c101080 2024-05-08T23:09:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.894+0000 7f9468c00700 1 -- 172.21.15.46:0/1426976451 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f945c020030 con 0x7f946c101080 2024-05-08T23:09:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.895+0000 7f9486b00b80 1 Processor -- start 2024-05-08T23:09:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 -- start start 2024-05-08T23:09:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9468c00700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 0x7f94500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9468c00700 1 -- 172.21.15.46:0/1426976451 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f945c0cddc0 con 0x7f946c101080 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd187350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd187cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd0de150 con 0x560ccd187890 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd0dbb10 con 0x560ccd18d720 2024-05-08T23:09:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9486b00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd0d7400 con 0x560ccd18ce00 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f9471000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 0x7f94500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.896+0000 7f946be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42842/0 (socket says 172.21.15.46:42842) 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 -- 172.21.15.46:0/3760324984 learned_addr learned my addr 172.21.15.46:0/3760324984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946b400700 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd187350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f9471000700 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd187cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 -- 172.21.15.46:0/3760324984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 msgr2=0x560ccd187350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd187350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 -- 172.21.15.46:0/3760324984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 msgr2=0x560ccd187cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd187cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 -- 172.21.15.46:0/3760324984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cccf583b0 con 0x560ccd18ce00 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f946be00700 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f945c09aae0 tx=0x7f945c09a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66dfca024112e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.897+0000 7f9461600700 1 -- 172.21.15.46:0/3760324984 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c09ab50 con 0x560ccd18ce00 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.898+0000 7f9471000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 0x7f94500a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ec0 tx=0x7f9464009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.898+0000 7f9461600700 1 -- 172.21.15.46:0/3760324984 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f945c09e980 con 0x560ccd18ce00 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.898+0000 7f9461600700 1 -- 172.21.15.46:0/3760324984 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0973f0 con 0x560ccd18ce00 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 -- 172.21.15.46:0/3760324984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 msgr2=0x560ccd18d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f945c09aae0 tx=0x7f945c09a530 comp rx=0 tx=0).stop 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 -- 172.21.15.46:0/3760324984 shutdown_connections 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd187cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd18d1e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 --2- 172.21.15.46:0/3760324984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd187350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 -- 172.21.15.46:0/3760324984 >> 172.21.15.46:0/3760324984 conn(0x560ccd0e5210 msgr2=0x560ccd181df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 -- 172.21.15.46:0/3760324984 shutdown_connections 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.899+0000 7f9486b00b80 1 -- 172.21.15.46:0/3760324984 wait complete. 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.900+0000 7f9486b00b80 1 -- start start 2024-05-08T23:09:41.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.900+0000 7f9486b00b80 1 Processor -- start 2024-05-08T23:09:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd190ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd190ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd073560 con 0x560ccd187890 2024-05-08T23:09:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd073200 con 0x560ccd18d720 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cccf5a6a0 con 0x560ccd18ce00 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9486b00b80 10 client.0 _ll_register_callbacks cb 0x560ccd07cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46530/0 (socket says 172.21.15.46:46530) 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 -- 172.21.15.46:0/1086907769 learned_addr learned my addr 172.21.15.46:0/1086907769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f946be00700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd190ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f946b400700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd190ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 -- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 msgr2=0x560ccd190ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd190ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 -- 172.21.15.46:0/1086907769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 msgr2=0x560ccd190ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd190ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f9471000700 1 -- 172.21.15.46:0/1086907769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1a6340 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.901+0000 7f946b400700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd190ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9471000700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9464018970 tx=0x7f9464004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47f7bfd694f561d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9456000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464020070 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ccd030270 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f946401b500 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946400e070 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9486b00b80 10 client.27248 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.902+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cccf04b60 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ccd030270 con 0x560ccd18d720 2024-05-08T23:09:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94640268a0 con 0x560ccd18d720 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 10 client.27248 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 10 client.27248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 20 client.27248 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1a0ed0 0x560ccd1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 4 client.27248 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ccd156810 con 0x560ccd1a0ed0 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.903+0000 7f946be00700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1a0ed0 0x560ccd1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.904+0000 7f9456000700 10 client.27248 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:41.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.904+0000 7f946be00700 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1a0ed0 0x560ccd1a32f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa3971c719d67452 server_cookie=3b970e0985d9abab in_seq=0 out_seq=0 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.905+0000 7f9456000700 1 -- 172.21.15.46:0/1086907769 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560ccd156810 con 0x560ccd1a0ed0 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.905+0000 7f9456000700 10 client.27248 handle_command_reply: tid=0 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.905+0000 7f9486b00b80 1 client.27248 shutdown 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 msgr2=0x560ccd191530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9464018970 tx=0x7f9464004b40 comp rx=0 tx=0).stop 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 shutdown_connections 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd18d720 0x560ccd191530 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1a0ed0 0x560ccd1a32f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd187890 0x560ccd190ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 --2- 172.21.15.46:0/1086907769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd190ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 >> 172.21.15.46:0/1086907769 conn(0x560ccd0e5210 msgr2=0x560ccd1815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9456000700 0 client.27248 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 shutdown_connections 2024-05-08T23:09:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 1 -- 172.21.15.46:0/1086907769 wait complete. 2024-05-08T23:09:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.906+0000 7f9486b00b80 20 client.27248 trim_cache size 0 max 16384 2024-05-08T23:09:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.920+0000 7f9486b00b80 1 Processor -- start 2024-05-08T23:09:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 -- start start 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd187350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 0x560ccd18c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd183a20 con 0x560ccd18ce00 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd17c800 con 0x560ccd187890 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9486b00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd183e50 con 0x560ccd18d720 2024-05-08T23:09:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42856/0 (socket says 172.21.15.46:42856) 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 -- 172.21.15.46:0/2565220147 learned_addr learned my addr 172.21.15.46:0/2565220147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f946be00700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd187350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f946b400700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 0x560ccd18c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 -- 172.21.15.46:0/2565220147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 msgr2=0x560ccd187350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd187350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 -- 172.21.15.46:0/2565220147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 msgr2=0x560ccd18c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 0x560ccd18c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f9471000700 1 -- 172.21.15.46:0/2565220147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ccd09d560 con 0x560ccd18d720 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.921+0000 7f946b400700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 0x560ccd18c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.922+0000 7f9471000700 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9464008740 tx=0x7f9464026dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc0b5c74cbab5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.922+0000 7f9462000700 1 -- 172.21.15.46:0/2565220147 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464020070 con 0x560ccd18d720 2024-05-08T23:09:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.922+0000 7f9462000700 1 -- 172.21.15.46:0/2565220147 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f946400f040 con 0x560ccd18d720 2024-05-08T23:09:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.922+0000 7f9462000700 1 -- 172.21.15.46:0/2565220147 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464004020 con 0x560ccd18d720 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- 172.21.15.46:0/2565220147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 msgr2=0x560ccd181d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9464008740 tx=0x7f9464026dd0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- 172.21.15.46:0/2565220147 shutdown_connections 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18d720 0x560ccd181d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd187350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2565220147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18ce00 0x560ccd18c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- 172.21.15.46:0/2565220147 >> 172.21.15.46:0/2565220147 conn(0x560ccd0e5210 msgr2=0x560ccd18f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- 172.21.15.46:0/2565220147 shutdown_connections 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- 172.21.15.46:0/2565220147 wait complete. 2024-05-08T23:09:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 -- start start 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.923+0000 7f9486b00b80 1 Processor -- start 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd189010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18d720 0x560ccd189550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd189c10 con 0x560ccd18d720 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd189d80 con 0x560ccd187890 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ccd1f73d0 con 0x560ccd18ce00 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9486b00b80 10 client.0 _ll_register_callbacks cb 0x560ccd07cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46538/0 (socket says 172.21.15.46:46538) 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 -- 172.21.15.46:0/2451338771 learned_addr learned my addr 172.21.15.46:0/2451338771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd189010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f9471000700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18d720 0x560ccd189550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 -- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 msgr2=0x560ccd189010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd189010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 -- 172.21.15.46:0/2451338771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18d720 msgr2=0x560ccd189550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18d720 0x560ccd189550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.924+0000 7f946be00700 1 -- 172.21.15.46:0/2451338771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cccf04b60 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f946be00700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098c0 tx=0x7f945c09a5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82400d7a6d6ba469 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9456000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00bb30 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f945c097260 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f945c09e980 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f945c084170 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9486b00b80 10 client.27254 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560cccf5ccd0 con 0x560ccd187890 2024-05-08T23:09:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.925+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f945c09e980 con 0x560ccd187890 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f945c0a6020 con 0x560ccd187890 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 10 client.27254 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 10 client.27254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 20 client.27254 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1f85e0 0x560ccd1fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 4 client.27254 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560ccd0cfec0 con 0x560ccd1f85e0 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.926+0000 7f946b400700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1f85e0 0x560ccd1fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f946b400700 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1f85e0 0x560ccd1fa9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82fce733476c3fa5 server_cookie=1780878c10842c3a in_seq=0 out_seq=0 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9456000700 10 client.27254 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9456000700 1 -- 172.21.15.46:0/2451338771 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560ccd0cfec0 con 0x560ccd1f85e0 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9456000700 10 client.27254 handle_command_reply: tid=0 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 client.27254 shutdown 2024-05-08T23:09:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 msgr2=0x560ccd1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098c0 tx=0x7f945c09a5a0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 shutdown_connections 2024-05-08T23:09:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x560ccd1f85e0 0x560ccd1fa9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ccd18d720 0x560ccd189550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ccd187890 0x560ccd1982b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 --2- 172.21.15.46:0/2451338771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ccd18ce00 0x560ccd189010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9456000700 0 client.27254 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 >> 172.21.15.46:0/2451338771 conn(0x560ccd0e5210 msgr2=0x560ccd18dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 shutdown_connections 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 1 -- 172.21.15.46:0/2451338771 wait complete. 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.927+0000 7f9486b00b80 20 client.27254 trim_cache size 0 max 16384 2024-05-08T23:09:41.931 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 msgr2=0x7f94500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 0x7f94500a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ec0 tx=0x7f9464009920 comp rx=0 tx=0).stop 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 msgr2=0x7f946c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f945c005950 tx=0x7f945c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 shutdown_connections 2024-05-08T23:09:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94500a1310 0x7f94500a37d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f946c073d20 0x7f946c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c101080 0x7f946c0732a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 --2- 172.21.15.46:0/1426976451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c1061a0 0x7f946c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 >> 172.21.15.46:0/1426976451 conn(0x7f946c0fc8e0 msgr2=0x7f946c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 shutdown_connections 2024-05-08T23:09:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:41.928+0000 7f9473000700 1 -- 172.21.15.46:0/1426976451 wait complete. 2024-05-08T23:09:41.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.179+0000 7f5f34a00700 1 -- 172.21.15.46:0/2404274564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 msgr2=0x7f5f30104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.179+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2404274564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f30104660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fc0 tx=0x7f5f2800b330 comp rx=0 tx=0).stop 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.179+0000 7f5f34a00700 1 -- 172.21.15.46:0/2404274564 shutdown_connections 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.179+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2404274564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f30104660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.179+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2404274564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 0x7f5f300ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.180+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2404274564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f30103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.180+0000 7f5f34a00700 1 -- 172.21.15.46:0/2404274564 >> 172.21.15.46:0/2404274564 conn(0x7f5f30075eb0 msgr2=0x7f5f300fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.180+0000 7f5f34a00700 1 -- 172.21.15.46:0/2404274564 shutdown_connections 2024-05-08T23:09:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.180+0000 7f5f34a00700 1 -- 172.21.15.46:0/2404274564 wait complete. 2024-05-08T23:09:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.181+0000 7f5f34a00700 1 Processor -- start 2024-05-08T23:09:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.181+0000 7f5f34a00700 1 -- start start 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 0x7f5f301a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f3019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019dea0 con 0x7f5f301041b0 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019e010 con 0x7f5f300ff280 2024-05-08T23:09:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f34a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019e1d0 con 0x7f5f300ffba0 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42868/0 (socket says 172.21.15.46:42868) 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f2d400700 1 -- 172.21.15.46:0/2239634540 learned_addr learned my addr 172.21.15.46:0/2239634540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f2e800700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f3019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.182+0000 7f5f2de00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 0x7f5f301a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 -- 172.21.15.46:0/2239634540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 msgr2=0x7f5f301a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 0x7f5f301a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 -- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 msgr2=0x7f5f3019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f3019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 -- 172.21.15.46:0/2239634540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009c40 con 0x7f5f300ffba0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000baf0 tx=0x7f5f2000be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f26a00700 1 -- 172.21.15.46:0/2239634540 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004830 con 0x7f5f300ffba0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f26a00700 1 -- 172.21.15.46:0/2239634540 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f20004e50 con 0x7f5f300ffba0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3019e4b0 con 0x7f5f300ffba0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.183+0000 7f5f26a00700 1 -- 172.21.15.46:0/2239634540 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20009930 con 0x7f5f300ffba0 2024-05-08T23:09:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.184+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301aa750 con 0x7f5f300ffba0 2024-05-08T23:09:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.187+0000 7f5f26a00700 1 -- 172.21.15.46:0/2239634540 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5f20014070 con 0x7f5f300ffba0 2024-05-08T23:09:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.188+0000 7f5f26a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 0x7f5f100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.188+0000 7f5f26a00700 1 -- 172.21.15.46:0/2239634540 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5f200cc630 con 0x7f5f300ffba0 2024-05-08T23:09:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.188+0000 7f5f2de00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 0x7f5f100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 Processor -- start 2024-05-08T23:09:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 -- start start 2024-05-08T23:09:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f2de00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 0x7f5f100a37d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f300fe510 tx=0x7f5f180073d0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b5c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5ab3150 con 0x55caa5b62720 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5ab0b10 con 0x55caa5b5c890 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.189+0000 7f5f48569b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5aac400 con 0x55caa5b61e00 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42874/0 (socket says 172.21.15.46:42874) 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 -- 172.21.15.46:0/3538615771 learned_addr learned my addr 172.21.15.46:0/3538615771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2d400700 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b5c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2e800700 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 -- 172.21.15.46:0/3538615771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 msgr2=0x55caa5b5ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 -- 172.21.15.46:0/3538615771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 msgr2=0x55caa5b5c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b5c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 -- 172.21.15.46:0/3538615771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caa592d3b0 con 0x55caa5b61e00 2024-05-08T23:09:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.190+0000 7f5f2de00700 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f18010e00 tx=0x7f5f18013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4920964450518d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.191+0000 7f5f1f000700 1 -- 172.21.15.46:0/3538615771 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18011bb0 con 0x55caa5b61e00 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.191+0000 7f5f1f000700 1 -- 172.21.15.46:0/3538615771 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f18015070 con 0x55caa5b61e00 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.191+0000 7f5f1f000700 1 -- 172.21.15.46:0/3538615771 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1801c610 con 0x55caa5b61e00 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.191+0000 7f5f48569b80 1 -- 172.21.15.46:0/3538615771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 msgr2=0x55caa5b621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.191+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f18010e00 tx=0x7f5f18013000 comp rx=0 tx=0).stop 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 -- 172.21.15.46:0/3538615771 shutdown_connections 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b621e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3538615771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b5c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 -- 172.21.15.46:0/3538615771 >> 172.21.15.46:0/3538615771 conn(0x55caa5aba210 msgr2=0x55caa5b56df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 -- 172.21.15.46:0/3538615771 shutdown_connections 2024-05-08T23:09:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 -- 172.21.15.46:0/3538615771 wait complete. 2024-05-08T23:09:42.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.192+0000 7f5f48569b80 1 -- start start 2024-05-08T23:09:42.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 Processor -- start 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b65a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b65f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5a48560 con 0x55caa5b61e00 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5a48200 con 0x55caa5b62720 2024-05-08T23:09:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa592f6a0 con 0x55caa5b5c890 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f48569b80 10 client.0 _ll_register_callbacks cb 0x55caa5a51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b65a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b65a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42890/0 (socket says 172.21.15.46:42890) 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b65f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f2de00700 1 -- 172.21.15.46:0/3754788468 learned_addr learned my addr 172.21.15.46:0/3754788468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.193+0000 7f5f2e800700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 -- 172.21.15.46:0/3754788468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 msgr2=0x55caa5b65a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b65a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 -- 172.21.15.46:0/3754788468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 msgr2=0x55caa5b664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 -- 172.21.15.46:0/3754788468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f18010ae0 con 0x55caa5b61e00 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f2d400700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b65f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5f200c9970 tx=0x7f5f20098d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2473bbbfc67c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f17400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20099ac0 con 0x55caa5b61e00 2024-05-08T23:09:42.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5f200993c0 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caa5a05270 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f2009d1f0 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.194+0000 7f5f48569b80 10 client.17280 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55caa58d9b60 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f2007c080 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f20099740 con 0x55caa5b61e00 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 10 client.17280 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 10 client.17280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 20 client.17280 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5b76100 0x55caa5b78520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 4 client.17280 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caa5b2b810 con 0x55caa5b76100 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.195+0000 7f5f2de00700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5b76100 0x55caa5b78520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.196+0000 7f5f2de00700 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5b76100 0x55caa5b78520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df140e0d3f006db7 server_cookie=35ddb52805ca901d in_seq=0 out_seq=0 2024-05-08T23:09:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.196+0000 7f5f17400700 10 client.17280 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.196+0000 7f5f17400700 1 -- 172.21.15.46:0/3754788468 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55caa5b2b810 con 0x55caa5b76100 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.196+0000 7f5f17400700 10 client.17280 handle_command_reply: tid=0 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 client.17280 shutdown 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 msgr2=0x55caa5b65f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b65f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5f200c9970 tx=0x7f5f20098d20 comp rx=0 tx=0).stop 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 shutdown_connections 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b65a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5b76100 0x55caa5b78520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 --2- 172.21.15.46:0/3754788468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b65f90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 >> 172.21.15.46:0/3754788468 conn(0x55caa5aba210 msgr2=0x55caa5b565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f17400700 0 client.17280 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 shutdown_connections 2024-05-08T23:09:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 1 -- 172.21.15.46:0/3754788468 wait complete. 2024-05-08T23:09:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.197+0000 7f5f48569b80 20 client.17280 trim_cache size 0 max 16384 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.213+0000 7f5f48569b80 1 Processor -- start 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.213+0000 7f5f48569b80 1 -- start start 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b53dc0 con 0x55caa5b62720 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b51a90 con 0x55caa5b5c890 2024-05-08T23:09:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f48569b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b51800 con 0x55caa5b61e00 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42892/0 (socket says 172.21.15.46:42892) 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 -- 172.21.15.46:0/2297960723 learned_addr learned my addr 172.21.15.46:0/2297960723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2de00700 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 -- 172.21.15.46:0/2297960723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 msgr2=0x55caa5b5c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 -- 172.21.15.46:0/2297960723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 msgr2=0x55caa5b56d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b56d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.214+0000 7f5f2d400700 1 -- 172.21.15.46:0/2297960723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caa5a72560 con 0x55caa5b61e00 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.215+0000 7f5f2d400700 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f200990c0 tx=0x7f5f20097970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d693a27012cfb0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.215+0000 7f5f1fa00700 1 -- 172.21.15.46:0/2297960723 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20099ac0 con 0x55caa5b61e00 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.215+0000 7f5f1fa00700 1 -- 172.21.15.46:0/2297960723 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55caa5a05270 con 0x55caa5b61e00 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.215+0000 7f5f1fa00700 1 -- 172.21.15.46:0/2297960723 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200a17b0 con 0x55caa5b61e00 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- 172.21.15.46:0/2297960723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 msgr2=0x55caa5b58a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f200990c0 tx=0x7f5f20097970 comp rx=0 tx=0).stop 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- 172.21.15.46:0/2297960723 shutdown_connections 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b62720 0x55caa5b56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b5c890 0x55caa5b5c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 --2- 172.21.15.46:0/2297960723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b61e00 0x55caa5b58a20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- 172.21.15.46:0/2297960723 >> 172.21.15.46:0/2297960723 conn(0x55caa5aba210 msgr2=0x55caa5b5cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- 172.21.15.46:0/2297960723 shutdown_connections 2024-05-08T23:09:42.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- 172.21.15.46:0/2297960723 wait complete. 2024-05-08T23:09:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 -- start start 2024-05-08T23:09:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.216+0000 7f5f48569b80 1 Processor -- start 2024-05-08T23:09:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b6b720 con 0x55caa5b61e00 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b6b890 con 0x55caa5b62720 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa5b6ba50 con 0x55caa5b5c890 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f48569b80 10 client.0 _ll_register_callbacks cb 0x55caa5a51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f2d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f2de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42900/0 (socket says 172.21.15.46:42900) 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f2de00700 1 -- 172.21.15.46:0/343311062 learned_addr learned my addr 172.21.15.46:0/343311062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.217+0000 7f5f2e800700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 -- 172.21.15.46:0/343311062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 msgr2=0x55caa5b6b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b6b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 -- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 msgr2=0x55caa5b6d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b6d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 -- 172.21.15.46:0/343311062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caa58d9b60 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2e800700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b6b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f2de00700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5f18006010 tx=0x7f5f18005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faede036428286c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f17400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1802d040 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f5f1801fe40 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f18015070 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f180343f0 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f48569b80 10 client.27334 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.218+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55caa5931cd0 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f18015070 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f1801e420 con 0x55caa5b5c890 2024-05-08T23:09:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 10 client.27334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 20 client.27334 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5bcc5b0 0x55caa5bce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 4 client.27334 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55caa5aa4ec0 con 0x55caa5bcc5b0 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.219+0000 7f5f2d400700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5bcc5b0 0x55caa5bce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f17400700 10 client.27334 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f2d400700 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5bcc5b0 0x55caa5bce9d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a568569b347d10a9 server_cookie=efe353d4ab9fc120 in_seq=0 out_seq=0 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f17400700 1 -- 172.21.15.46:0/343311062 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55caa5aa4ec0 con 0x55caa5bcc5b0 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f17400700 10 client.27334 handle_command_reply: tid=0 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 client.27334 shutdown 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 msgr2=0x55caa5b6d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5f18006010 tx=0x7f5f18005b70 comp rx=0 tx=0).stop 2024-05-08T23:09:42.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 shutdown_connections 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caa5b62720 0x55caa5b6b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caa5b5c890 0x55caa5b6d170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55caa5bcc5b0 0x55caa5bce9d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 --2- 172.21.15.46:0/343311062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55caa5b61e00 0x55caa5b6d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f17400700 0 client.27334 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 >> 172.21.15.46:0/343311062 conn(0x55caa5aba210 msgr2=0x55caa5b5cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 shutdown_connections 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.220+0000 7f5f48569b80 1 -- 172.21.15.46:0/343311062 wait complete. 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f48569b80 20 client.27334 trim_cache size 0 max 16384 2024-05-08T23:09:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 msgr2=0x7f5f100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 0x7f5f100a37d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f300fe510 tx=0x7f5f180073d0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 msgr2=0x7f5f301a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000baf0 tx=0x7f5f2000be00 comp rx=0 tx=0).stop 2024-05-08T23:09:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 shutdown_connections 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5f100a1310 0x7f5f100a37d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f301041b0 0x7f5f3019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f300ff280 0x7f5f301a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 --2- 172.21.15.46:0/2239634540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f300ffba0 0x7f5f301a3bf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.221+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 >> 172.21.15.46:0/2239634540 conn(0x7f5f30075eb0 msgr2=0x7f5f30101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.222+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 shutdown_connections 2024-05-08T23:09:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.222+0000 7f5f34a00700 1 -- 172.21.15.46:0/2239634540 wait complete. 2024-05-08T23:09:42.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:09:42.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 -- 172.21.15.46:0/1449445065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8105ff0 msgr2=0x7f08a810db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 --2- 172.21.15.46:0/1449445065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8105ff0 0x7f08a810db70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009f90 tx=0x7f08a000b330 comp rx=0 tx=0).stop 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 -- 172.21.15.46:0/1449445065 shutdown_connections 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 --2- 172.21.15.46:0/1449445065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8105ff0 0x7f08a810db70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 --2- 172.21.15.46:0/1449445065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8100ed0 0x7f08a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 --2- 172.21.15.46:0/1449445065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8101880 0x7f08a81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.479+0000 7f08ae000700 1 -- 172.21.15.46:0/1449445065 >> 172.21.15.46:0/1449445065 conn(0x7f08a80fc770 msgr2=0x7f08a80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.480+0000 7f08ae000700 1 -- 172.21.15.46:0/1449445065 shutdown_connections 2024-05-08T23:09:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.480+0000 7f08ae000700 1 -- 172.21.15.46:0/1449445065 wait complete. 2024-05-08T23:09:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 Processor -- start 2024-05-08T23:09:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 -- start start 2024-05-08T23:09:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8100ed0 0x7f08a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 0x7f08a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 0x7f08a81a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a5970 con 0x7f08a8100ed0 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a5ae0 con 0x7f08a8105ff0 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ae000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a5ca0 con 0x7f08a8073d20 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 0x7f08a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08ad600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 0x7f08a81a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08acc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8100ed0 0x7f08a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.481+0000 7f08a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 0x7f08a80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46618/0 (socket says 172.21.15.46:46618) 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08a7e00700 1 -- 172.21.15.46:0/2263638092 learned_addr learned my addr 172.21.15.46:0/2263638092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 -- 172.21.15.46:0/2263638092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 msgr2=0x7f08a80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 0x7f08a80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 -- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8100ed0 msgr2=0x7f08a80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8100ed0 0x7f08a80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 -- 172.21.15.46:0/2263638092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a0009c40 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ad600700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 0x7f08a81a53a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b6c0 tx=0x7f08a0009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08a5600700 1 -- 172.21.15.46:0/2263638092 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a000e040 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08a5600700 1 -- 172.21.15.46:0/2263638092 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a000be50 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a81a5f20 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.482+0000 7f08a5600700 1 -- 172.21.15.46:0/2263638092 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0005040 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.483+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a81a63a0 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.486+0000 7f08a5600700 1 -- 172.21.15.46:0/2263638092 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f08a001d600 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.487+0000 7f08a5600700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 0x7f08900a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.487+0000 7f08c3023b80 1 Processor -- start 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.487+0000 7f08a5600700 1 -- 172.21.15.46:0/2263638092 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f08a00ce360 con 0x7f08a8073d20 2024-05-08T23:09:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.487+0000 7f08acc00700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 0x7f08900a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.487+0000 7f08c3023b80 1 -- start start 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c7cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516bcb150 con 0x560516c830f0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516bc8b10 con 0x560516c7d0a0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08c3023b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516bc4400 con 0x560516c827d0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08acc00700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 0x7f08900a3870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f089c005fd0 tx=0x7f089c009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08acc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08acc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42922/0 (socket says 172.21.15.46:42922) 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08acc00700 1 -- 172.21.15.46:0/2314014576 learned_addr learned my addr 172.21.15.46:0/2314014576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08ad600700 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.488+0000 7f08a7e00700 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c7cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 -- 172.21.15.46:0/2314014576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 msgr2=0x560516c7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 -- 172.21.15.46:0/2314014576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 msgr2=0x560516c7cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c7cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 -- 172.21.15.46:0/2314014576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560516a453b0 con 0x560516c827d0 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f08acc00700 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e4b0 tx=0x7f089c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef47cbe0681897f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f0899c00700 1 -- 172.21.15.46:0/2314014576 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00ee50 con 0x560516c827d0 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f0899c00700 1 -- 172.21.15.46:0/2314014576 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089c021070 con 0x560516c827d0 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.489+0000 7f0899c00700 1 -- 172.21.15.46:0/2314014576 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c025790 con 0x560516c827d0 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 -- 172.21.15.46:0/2314014576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 msgr2=0x560516c82bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e4b0 tx=0x7f089c014000 comp rx=0 tx=0).stop 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 -- 172.21.15.46:0/2314014576 shutdown_connections 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c82bb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2314014576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c7cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 -- 172.21.15.46:0/2314014576 >> 172.21.15.46:0/2314014576 conn(0x560516bd2210 msgr2=0x560516c6f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 -- 172.21.15.46:0/2314014576 shutdown_connections 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.490+0000 7f08c3023b80 1 -- 172.21.15.46:0/2314014576 wait complete. 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.491+0000 7f08c3023b80 1 -- start start 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.491+0000 7f08c3023b80 1 Processor -- start 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c827d0 0x560516c86400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c830f0 0x560516c86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516b60560 con 0x560516c827d0 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516b60200 con 0x560516c830f0 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516a476a0 con 0x560516c7d0a0 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08c3023b80 10 client.0 _ll_register_callbacks cb 0x560516b69b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08acc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08a7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c827d0 0x560516c86400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08acc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42936/0 (socket says 172.21.15.46:42936) 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08acc00700 1 -- 172.21.15.46:0/1468897490 learned_addr learned my addr 172.21.15.46:0/1468897490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.492+0000 7f08ad600700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c830f0 0x560516c86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 -- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c830f0 msgr2=0x560516c86b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c830f0 0x560516c86b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 -- 172.21.15.46:0/1468897490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c827d0 msgr2=0x560516c86400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c827d0 0x560516c86400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 -- 172.21.15.46:0/1468897490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c017070 con 0x560516c7d0a0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08acc00700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f089c0096d0 tx=0x7f089c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba95ca4e95d5b46b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f088e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c005250 con 0x560516c7d0a0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560516b1d270 con 0x560516c7d0a0 2024-05-08T23:09:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.493+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f089c0053b0 con 0x560516c7d0a0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f089c005570 con 0x560516c7d0a0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 10 client.27346 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5605169f1b60 con 0x560516c7d0a0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560516b1d270 con 0x560516c7d0a0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f089c01d550 con 0x560516c7d0a0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 10 client.27346 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 10 client.27346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 20 client.27346 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516c96b10 0x560516c98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 4 client.27346 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560516c43810 con 0x560516c96b10 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.494+0000 7f08a7e00700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516c96b10 0x560516c98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.495+0000 7f088e000700 10 client.27346 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.495+0000 7f08a7e00700 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516c96b10 0x560516c98f30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c52bd0bc6c0a5a server_cookie=c06f03276eb73426 in_seq=0 out_seq=0 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f088e000700 1 -- 172.21.15.46:0/1468897490 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560516c43810 con 0x560516c96b10 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f088e000700 10 client.27346 handle_command_reply: tid=0 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 client.27346 shutdown 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 msgr2=0x560516c89110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f089c0096d0 tx=0x7f089c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 shutdown_connections 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c830f0 0x560516c86b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c7d0a0 0x560516c89110 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516c96b10 0x560516c98f30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1468897490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c827d0 0x560516c86400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 >> 172.21.15.46:0/1468897490 conn(0x560516bd2210 msgr2=0x560516c6f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f088e000700 0 client.27346 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 shutdown_connections 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.496+0000 7f08c3023b80 1 -- 172.21.15.46:0/1468897490 wait complete. 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.497+0000 7f08c3023b80 20 client.27346 trim_cache size 0 max 16384 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 Processor -- start 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 -- start start 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c69800 con 0x560516c830f0 2024-05-08T23:09:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c9e120 con 0x560516c7d0a0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.512+0000 7f08c3023b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c84950 con 0x560516c827d0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08acc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08ad600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42940/0 (socket says 172.21.15.46:42940) 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 -- 172.21.15.46:0/1389147855 learned_addr learned my addr 172.21.15.46:0/1389147855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08acc00700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46644/0 (socket says 172.21.15.46:46644) 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 -- 172.21.15.46:0/1389147855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 msgr2=0x560516c8db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 -- 172.21.15.46:0/1389147855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 msgr2=0x560516c92610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c92610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 -- 172.21.15.46:0/1389147855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560516b8a560 con 0x560516c827d0 2024-05-08T23:09:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08acc00700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08ad600700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.513+0000 7f08a7e00700 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0894005950 tx=0x7f0894004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a5e65206d2de56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f089a600700 1 -- 172.21.15.46:0/1389147855 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894016030 con 0x560516c827d0 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f089a600700 1 -- 172.21.15.46:0/1389147855 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0894017040 con 0x560516c827d0 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f089a600700 1 -- 172.21.15.46:0/1389147855 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894004d60 con 0x560516c827d0 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 -- 172.21.15.46:0/1389147855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 msgr2=0x560516c8e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0894005950 tx=0x7f0894004750 comp rx=0 tx=0).stop 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 -- 172.21.15.46:0/1389147855 shutdown_connections 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c830f0 0x560516c92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c7d0a0 0x560516c8db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 --2- 172.21.15.46:0/1389147855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c827d0 0x560516c8e090 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 -- 172.21.15.46:0/1389147855 >> 172.21.15.46:0/1389147855 conn(0x560516bd2210 msgr2=0x560516c80550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.514+0000 7f08c3023b80 1 -- 172.21.15.46:0/1389147855 shutdown_connections 2024-05-08T23:09:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.515+0000 7f08c3023b80 1 -- 172.21.15.46:0/1389147855 wait complete. 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.515+0000 7f08c3023b80 1 -- start start 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.515+0000 7f08c3023b80 1 Processor -- start 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c827d0 0x560516c7f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 0x560516c7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c7e340 con 0x560516c7d0a0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c7e4b0 con 0x560516c827d0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560516c7e670 con 0x560516c830f0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08c3023b80 10 client.0 _ll_register_callbacks cb 0x560516b69b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08acc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08acc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48680/0 (socket says 172.21.15.46:48680) 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08acc00700 1 -- 172.21.15.46:0/2190824959 learned_addr learned my addr 172.21.15.46:0/2190824959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 0x560516c7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08a7e00700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c827d0 0x560516c7f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 -- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c827d0 msgr2=0x560516c7f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c827d0 0x560516c7f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 -- 172.21.15.46:0/2190824959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 msgr2=0x560516c8dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08ad600700 1 -- 172.21.15.46:0/2190824959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605169f1b60 con 0x560516c830f0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.516+0000 7f08acc00700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f08ad600700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 0x560516c7fb50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08a00cb6a0 tx=0x7f08a0006020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3eca939dd0502a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00cbb20 con 0x560516c830f0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f08a009fe40 con 0x560516c830f0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f089c021070 con 0x560516c830f0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a009b0f0 con 0x560516c830f0 2024-05-08T23:09:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f08c3023b80 10 client.27352 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f08a81a63a0 con 0x560516c830f0 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089c021070 con 0x560516c830f0 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.517+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08a009a5e0 con 0x560516c830f0 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 10 client.27352 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 20 client.27352 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516ced920 0x560516cefd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 4 client.27352 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560516bbcec0 con 0x560516ced920 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08acc00700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516ced920 0x560516cefd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f08acc00700 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516ced920 0x560516cefd10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c1b0e811852a3d server_cookie=4a16a53d811b8edf in_seq=0 out_seq=0 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.518+0000 7f088e000700 10 client.27352 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f088e000700 1 -- 172.21.15.46:0/2190824959 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560516bbcec0 con 0x560516ced920 2024-05-08T23:09:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f088e000700 10 client.27352 handle_command_reply: tid=0 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 client.27352 shutdown 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 msgr2=0x560516c7fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 0x560516c7fb50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08a00cb6a0 tx=0x7f08a0006020 comp rx=0 tx=0).stop 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 shutdown_connections 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560516c830f0 0x560516c7fb50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560516ced920 0x560516cefd10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560516c7d0a0 0x560516c8dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 --2- 172.21.15.46:0/2190824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560516c827d0 0x560516c7f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f088e000700 0 client.27352 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 >> 172.21.15.46:0/2190824959 conn(0x560516bd2210 msgr2=0x560516c84c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 shutdown_connections 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.519+0000 7f08c3023b80 1 -- 172.21.15.46:0/2190824959 wait complete. 2024-05-08T23:09:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08c3023b80 20 client.27352 trim_cache size 0 max 16384 2024-05-08T23:09:42.693 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:09:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 msgr2=0x7f08900a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 0x7f08900a3870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f089c005fd0 tx=0x7f089c009310 comp rx=0 tx=0).stop 2024-05-08T23:09:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 msgr2=0x7f08a81a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 0x7f08a81a53a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b6c0 tx=0x7f08a0009440 comp rx=0 tx=0).stop 2024-05-08T23:09:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 shutdown_connections 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8073d20 0x7f08a81a53a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f08900a13b0 0x7f08900a3870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a8100ed0 0x7f08a80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 --2- 172.21.15.46:0/2263638092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a8105ff0 0x7f08a80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 >> 172.21.15.46:0/2263638092 conn(0x7f08a80fc770 msgr2=0x7f08a80fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 shutdown_connections 2024-05-08T23:09:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:42.520+0000 7f08ae000700 1 -- 172.21.15.46:0/2263638092 wait complete. 2024-05-08T23:09:42.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 451650 10.44 MB/sec execute 124 sec latency 16.580 ms 2024-05-08T23:09:43.696 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:09:43.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:09:43.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:43 smithi086 ceph-mon[78610]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-08T23:09:43.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:43 smithi053 ceph-mon[77587]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-08T23:09:43.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:43 smithi046 ceph-mon[36898]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-08T23:09:43.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 453856 10.46 MB/sec execute 125 sec latency 16.618 ms 2024-05-08T23:09:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 -- 172.21.15.46:0/2256324464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 msgr2=0x7f4a2810cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 --2- 172.21.15.46:0/2256324464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a2810cf50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009990 tx=0x7f4a14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 -- 172.21.15.46:0/2256324464 shutdown_connections 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 --2- 172.21.15.46:0/2256324464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2810d490 0x7f4a2810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 --2- 172.21.15.46:0/2256324464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a280733d0 0x7f4a280737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 --2- 172.21.15.46:0/2256324464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a2810cf50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 -- 172.21.15.46:0/2256324464 >> 172.21.15.46:0/2256324464 conn(0x7f4a280fc740 msgr2=0x7f4a280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 -- 172.21.15.46:0/2256324464 shutdown_connections 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.933+0000 7f4a2e400700 1 -- 172.21.15.46:0/2256324464 wait complete. 2024-05-08T23:09:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.934+0000 7f4a2e400700 1 Processor -- start 2024-05-08T23:09:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 -- start start 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a280733d0 0x7f4a28199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a2810d490 0x7f4a2819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2819e490 con 0x7f4a280733d0 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2819e600 con 0x7f4a2810d490 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a2e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2819e7c0 con 0x7f4a28073cf0 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.935+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42966/0 (socket says 172.21.15.46:42966) 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 -- 172.21.15.46:0/88481546 learned_addr learned my addr 172.21.15.46:0/88481546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a2d000700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a280733d0 0x7f4a28199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a2da00700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a2810d490 0x7f4a2819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 -- 172.21.15.46:0/88481546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a2810d490 msgr2=0x7f4a2819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a2810d490 0x7f4a2819de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 -- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a280733d0 msgr2=0x7f4a28199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a280733d0 0x7f4a28199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 -- 172.21.15.46:0/88481546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140095f0 con 0x7f4a28073cf0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a27e00700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000c00 tx=0x7f4a14005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a25600700 1 -- 172.21.15.46:0/88481546 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1401d070 con 0x7f4a28073cf0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a25600700 1 -- 172.21.15.46:0/88481546 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14022470 con 0x7f4a28073cf0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.936+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2819ea40 con 0x7f4a28073cf0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.937+0000 7f4a25600700 1 -- 172.21.15.46:0/88481546 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14017650 con 0x7f4a28073cf0 2024-05-08T23:09:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.937+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a281a62e0 con 0x7f4a28073cf0 2024-05-08T23:09:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.940+0000 7f4a25600700 1 -- 172.21.15.46:0/88481546 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4a140225e0 con 0x7f4a28073cf0 2024-05-08T23:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.942+0000 7f4a25600700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 0x7f4a100a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.942+0000 7f4a25600700 1 -- 172.21.15.46:0/88481546 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a140ce3d0 con 0x7f4a28073cf0 2024-05-08T23:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.942+0000 7f4a43564b80 1 Processor -- start 2024-05-08T23:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.942+0000 7f4a2d000700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 0x7f4a100a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.942+0000 7f4a43564b80 1 -- start start 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd141d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd0981d0 con 0x5557bd141910 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd095b90 con 0x5557bd1470d0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a43564b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd091480 con 0x5557bd1479f0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42978/0 (socket says 172.21.15.46:42978) 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 -- 172.21.15.46:0/1284590394 learned_addr learned my addr 172.21.15.46:0/1284590394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a2d000700 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd1474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a2da00700 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a2d000700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 0x7f4a100a38f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009cb0 tx=0x7f4a1c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 -- 172.21.15.46:0/1284590394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 msgr2=0x5557bd1474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd1474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 -- 172.21.15.46:0/1284590394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 msgr2=0x5557bd141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.943+0000 7f4a27e00700 1 -- 172.21.15.46:0/1284590394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557bcf12430 con 0x5557bd1479f0 2024-05-08T23:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.944+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a1409ffd0 tx=0x7f4a1409b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207bdae644d19d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.944+0000 7f4a1a000700 1 -- 172.21.15.46:0/1284590394 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1409b980 con 0x5557bd1479f0 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.944+0000 7f4a1a000700 1 -- 172.21.15.46:0/1284590394 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14095e90 con 0x5557bd1479f0 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.944+0000 7f4a1a000700 1 -- 172.21.15.46:0/1284590394 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14022990 con 0x5557bd1479f0 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 -- 172.21.15.46:0/1284590394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 msgr2=0x5557bd1413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a1409ffd0 tx=0x7f4a1409b050 comp rx=0 tx=0).stop 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 -- 172.21.15.46:0/1284590394 shutdown_connections 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1284590394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd1413d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 -- 172.21.15.46:0/1284590394 >> 172.21.15.46:0/1284590394 conn(0x5557bd09f290 msgr2=0x5557bd13c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 -- 172.21.15.46:0/1284590394 shutdown_connections 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.945+0000 7f4a43564b80 1 -- 172.21.15.46:0/1284590394 wait complete. 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 -- start start 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 Processor -- start 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd14d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1479f0 0x5557bd14b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd02d5e0 con 0x5557bd141910 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd02d280 con 0x5557bd1479f0 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bcf14720 con 0x5557bd1470d0 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.946+0000 7f4a43564b80 10 client.0 _ll_register_callbacks cb 0x5557bd036bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a2d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42988/0 (socket says 172.21.15.46:42988) 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 -- 172.21.15.46:0/4275557181 learned_addr learned my addr 172.21.15.46:0/4275557181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a2da00700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1479f0 0x5557bd14b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 -- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1479f0 msgr2=0x5557bd14b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1479f0 0x5557bd14b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 -- 172.21.15.46:0/4275557181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 msgr2=0x5557bd14d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd14d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 -- 172.21.15.46:0/4275557181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a281a62e0 con 0x5557bd1470d0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a27e00700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400f740 tx=0x7f4a14009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b28faa0523d9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1409b280 con 0x5557bd1470d0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.947+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a1409f0b0 con 0x5557bd1470d0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557bcfea2f0 con 0x5557bd1470d0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1409b6f0 con 0x5557bd1470d0 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 10 client.27364 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5557bcebebe0 con 0x5557bd1470d0 2024-05-08T23:09:43.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14084020 con 0x5557bd1470d0 2024-05-08T23:09:43.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a140a1100 con 0x5557bd1470d0 2024-05-08T23:09:43.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 10 client.27364 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:43.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:43.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 20 client.27364 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:43.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd15aaf0 0x5557bd15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 4 client.27364 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.948+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557bd110890 con 0x5557bd15aaf0 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.949+0000 7f4a2d000700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd15aaf0 0x5557bd15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.949+0000 7f4a2d000700 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd15aaf0 0x5557bd15cee0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8e50a79269baff server_cookie=2c98892e724b23fc in_seq=0 out_seq=0 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.949+0000 7f4a0ea00700 10 client.27364 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a0ea00700 1 -- 172.21.15.46:0/4275557181 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557bd110890 con 0x5557bd15aaf0 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a0ea00700 10 client.27364 handle_command_reply: tid=0 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 client.27364 shutdown 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 msgr2=0x5557bd14ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400f740 tx=0x7f4a14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 shutdown_connections 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1479f0 0x5557bd14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd15aaf0 0x5557bd15cee0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd141910 0x5557bd14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 --2- 172.21.15.46:0/4275557181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1470d0 0x5557bd14ac40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a0ea00700 0 client.27364 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 >> 172.21.15.46:0/4275557181 conn(0x5557bd09f290 msgr2=0x5557bd13c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 shutdown_connections 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.950+0000 7f4a43564b80 1 -- 172.21.15.46:0/4275557181 wait complete. 2024-05-08T23:09:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.951+0000 7f4a43564b80 20 client.27364 trim_cache size 0 max 16384 2024-05-08T23:09:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.972+0000 7f4a43564b80 1 Processor -- start 2024-05-08T23:09:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.972+0000 7f4a43564b80 1 -- start start 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd141910 0x5557bd14d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 0x5557bd1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd14c240 con 0x5557bd1470d0 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd13d8a0 con 0x5557bd141910 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a43564b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd14bb50 con 0x5557bd1479f0 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 0x5557bd1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42994/0 (socket says 172.21.15.46:42994) 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 -- 172.21.15.46:0/2674753455 learned_addr learned my addr 172.21.15.46:0/2674753455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2d000700 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd141910 0x5557bd14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 -- 172.21.15.46:0/2674753455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd141910 msgr2=0x5557bd14d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd141910 0x5557bd14d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 -- 172.21.15.46:0/2674753455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 msgr2=0x5557bd1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 0x5557bd1520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.973+0000 7f4a2da00700 1 -- 172.21.15.46:0/2674753455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557bd138560 con 0x5557bd1479f0 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.974+0000 7f4a27e00700 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 0x5557bd1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.974+0000 7f4a2da00700 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a20007630 tx=0x7f4a2000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479b173a0fb8b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.974+0000 7f4a1aa00700 1 -- 172.21.15.46:0/2674753455 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000e040 con 0x5557bd1479f0 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.974+0000 7f4a1aa00700 1 -- 172.21.15.46:0/2674753455 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2000f040 con 0x5557bd1479f0 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.974+0000 7f4a1aa00700 1 -- 172.21.15.46:0/2674753455 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017650 con 0x5557bd1479f0 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- 172.21.15.46:0/2674753455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 msgr2=0x5557bd152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a20007630 tx=0x7f4a2000b040 comp rx=0 tx=0).stop 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- 172.21.15.46:0/2674753455 shutdown_connections 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd1479f0 0x5557bd152600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd141910 0x5557bd14d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 --2- 172.21.15.46:0/2674753455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1470d0 0x5557bd1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- 172.21.15.46:0/2674753455 >> 172.21.15.46:0/2674753455 conn(0x5557bd09f290 msgr2=0x5557bd144ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- 172.21.15.46:0/2674753455 shutdown_connections 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- 172.21.15.46:0/2674753455 wait complete. 2024-05-08T23:09:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.975+0000 7f4a43564b80 1 -- start start 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 Processor -- start 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd152600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1479f0 0x5557bd150190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd150830 con 0x5557bd1479f0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd1509a0 con 0x5557bd1470d0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557bd150b60 con 0x5557bd141910 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a43564b80 10 client.0 _ll_register_callbacks cb 0x5557bd036bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a2d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a2d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43006/0 (socket says 172.21.15.46:43006) 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a2d000700 1 -- 172.21.15.46:0/1768395829 learned_addr learned my addr 172.21.15.46:0/1768395829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.976+0000 7f4a2da00700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1479f0 0x5557bd150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 -- 172.21.15.46:0/1768395829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 msgr2=0x5557bd1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 -- 172.21.15.46:0/1768395829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1479f0 msgr2=0x5557bd150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1479f0 0x5557bd150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 -- 172.21.15.46:0/1768395829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557bcebebe0 con 0x5557bd1470d0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a2d000700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a27e00700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd152600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009580 tx=0x7f4a14022ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9eefc9b0a6c0e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1409b050 con 0x5557bd1470d0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a14095e90 con 0x5557bd1470d0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a140a3670 con 0x5557bd1470d0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1409f0b0 con 0x5557bd1470d0 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a43564b80 10 client.27308 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.977+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f4a20009300 con 0x5557bd1470d0 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14095e90 con 0x5557bd1470d0 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a1409a770 con 0x5557bd1470d0 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 10 client.27308 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 20 client.27308 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd1b2e00 0x5557bd1b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 4 client.27308 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5557bd089f40 con 0x5557bd1b2e00 2024-05-08T23:09:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.978+0000 7f4a2d000700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd1b2e00 0x5557bd1b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.979+0000 7f4a2d000700 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd1b2e00 0x5557bd1b5220 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9bd00c0a845f64 server_cookie=676079d1403adbb2 in_seq=0 out_seq=0 2024-05-08T23:09:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.979+0000 7f4a0ea00700 10 client.27308 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.980+0000 7f4a0ea00700 1 -- 172.21.15.46:0/1768395829 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5557bd089f40 con 0x5557bd1b2e00 2024-05-08T23:09:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.980+0000 7f4a0ea00700 10 client.27308 handle_command_reply: tid=0 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.980+0000 7f4a43564b80 1 client.27308 shutdown 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 msgr2=0x5557bd152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd152600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009580 tx=0x7f4a14022ad0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 shutdown_connections 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5557bd1b2e00 0x5557bd1b5220 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557bd1479f0 0x5557bd150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557bd141910 0x5557bd1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a0ea00700 0 client.27308 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 --2- 172.21.15.46:0/1768395829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557bd1470d0 0x5557bd152600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 >> 172.21.15.46:0/1768395829 conn(0x5557bd09f290 msgr2=0x5557bd14a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 shutdown_connections 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 1 -- 172.21.15.46:0/1768395829 wait complete. 2024-05-08T23:09:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.981+0000 7f4a43564b80 20 client.27308 trim_cache size 0 max 16384 2024-05-08T23:09:43.984 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:43.984 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:09:43.984 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "44da8395-bd20-4a22-b9fb-048619dbd91c", 2024-05-08T23:09:43.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:09:43.984 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 msgr2=0x7f4a100a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 0x7f4a100a38f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009cb0 tx=0x7f4a1c009260 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 msgr2=0x7f4a28199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000c00 tx=0x7f4a14005740 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 shutdown_connections 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a2810d490 0x7f4a2819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a100a1430 0x7f4a100a38f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a280733d0 0x7f4a28199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 --2- 172.21.15.46:0/88481546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a28073cf0 0x7f4a28199ab0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 >> 172.21.15.46:0/88481546 conn(0x7f4a280fc740 msgr2=0x7f4a280fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 shutdown_connections 2024-05-08T23:09:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:43.982+0000 7f4a2e400700 1 -- 172.21.15.46:0/88481546 wait complete. 2024-05-08T23:09:44.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 -- 172.21.15.46:0/786863380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc100ec0 msgr2=0x7ffafc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 --2- 172.21.15.46:0/786863380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc100ec0 0x7ffafc1012a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009990 tx=0x7ffaec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 -- 172.21.15.46:0/786863380 shutdown_connections 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 --2- 172.21.15.46:0/786863380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc105fe0 0x7ffafc10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 --2- 172.21.15.46:0/786863380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc100ec0 0x7ffafc1012a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 --2- 172.21.15.46:0/786863380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc101870 0x7ffafc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 -- 172.21.15.46:0/786863380 >> 172.21.15.46:0/786863380 conn(0x7ffafc0fc740 msgr2=0x7ffafc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 -- 172.21.15.46:0/786863380 shutdown_connections 2024-05-08T23:09:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.216+0000 7ffb01400700 1 -- 172.21.15.46:0/786863380 wait complete. 2024-05-08T23:09:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.217+0000 7ffb01400700 1 Processor -- start 2024-05-08T23:09:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 -- start start 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc105fe0 0x7ffafc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc19de80 0x7ffafc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc19e900 con 0x7ffafc19de80 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc19ea70 con 0x7ffafc105fe0 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb01400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc19ec30 con 0x7ffafc100ec0 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43024/0 (socket says 172.21.15.46:43024) 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffafbe00700 1 -- 172.21.15.46:0/2603436614 learned_addr learned my addr 172.21.15.46:0/2603436614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffb00a00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc19de80 0x7ffafc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.218+0000 7ffafb400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc105fe0 0x7ffafc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 -- 172.21.15.46:0/2603436614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc105fe0 msgr2=0x7ffafc199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc105fe0 0x7ffafc199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 -- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc19de80 msgr2=0x7ffafc19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc19de80 0x7ffafc19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 -- 172.21.15.46:0/2603436614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0095f0 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffafbe00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffaf8c00700 1 -- 172.21.15.46:0/2603436614 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec01d070 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffaf8c00700 1 -- 172.21.15.46:0/2603436614 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaec022470 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc1a6130 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffaf8c00700 1 -- 172.21.15.46:0/2603436614 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec017650 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.219+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc1a6600 con 0x7ffafc100ec0 2024-05-08T23:09:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.222+0000 7ffaf8c00700 1 -- 172.21.15.46:0/2603436614 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffaec0225e0 con 0x7ffafc100ec0 2024-05-08T23:09:44.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.223+0000 7ffb1653ab80 1 Processor -- start 2024-05-08T23:09:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 -- start start 2024-05-08T23:09:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffaf8c00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 0x7ffae00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffaf8c00700 1 -- 172.21.15.46:0/2603436614 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ffaec0ce330 con 0x7ffafc100ec0 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a41cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0998150 con 0x5613b0a47050 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0995b10 con 0x5613b0a47970 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0991400 con 0x5613b0a41890 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffafb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a47430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48758/0 (socket says 172.21.15.46:48758) 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffafb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46738/0 (socket says 172.21.15.46:46738) 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffafbe00700 1 -- 172.21.15.46:0/1718471249 learned_addr learned my addr 172.21.15.46:0/1718471249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.224+0000 7ffb00a00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 0x7ffae00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffb00a00700 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a41cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 -- 172.21.15.46:0/1718471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 msgr2=0x5613b0a41cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a41cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 -- 172.21.15.46:0/1718471249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 msgr2=0x5613b0a47430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a47430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 -- 172.21.15.46:0/1718471249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613b08123b0 con 0x5613b0a47970 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffafb400700 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06a260 tx=0x7ffae8009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d9a7c70fc7bcda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffb00a00700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 0x7ffae00a3870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003fd0 tx=0x7ffaf4003a00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffaf1200700 1 -- 172.21.15.46:0/1718471249 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011d10 con 0x5613b0a47970 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffaf1200700 1 -- 172.21.15.46:0/1718471249 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8011e70 con 0x5613b0a47970 2024-05-08T23:09:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.225+0000 7ffaf1200700 1 -- 172.21.15.46:0/1718471249 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8005280 con 0x5613b0a47970 2024-05-08T23:09:44.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1718471249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 msgr2=0x5613b0a41350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06a260 tx=0x7ffae8009f40 comp rx=0 tx=0).stop 2024-05-08T23:09:44.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1718471249 shutdown_connections 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a47430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1718471249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a41350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1718471249 >> 172.21.15.46:0/1718471249 conn(0x5613b099f210 msgr2=0x5613b0a3c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1718471249 shutdown_connections 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.226+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1718471249 wait complete. 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 -- start start 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 Processor -- start 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b092d560 con 0x5613b0a47050 2024-05-08T23:09:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b092d200 con 0x5613b0a47970 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.227+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b08146a0 con 0x5613b0a41890 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffb1653ab80 10 client.0 _ll_register_callbacks cb 0x5613b0936b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43046/0 (socket says 172.21.15.46:43046) 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 -- 172.21.15.46:0/119231109 learned_addr learned my addr 172.21.15.46:0/119231109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafb400700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffb00a00700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 -- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 msgr2=0x5613b0a4b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a4b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 -- 172.21.15.46:0/119231109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 msgr2=0x5613b0a4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 -- 172.21.15.46:0/119231109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae80098c0 con 0x5613b0a41890 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffb00a00700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a4b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffafbe00700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b6f0 tx=0x7ffaec09af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab62c1a913e120d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.228+0000 7ffae5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec09e070 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ffaec09b7c0 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613b08ea270 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaec09b0c0 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 10 client.27382 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5613b07beb60 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613b08ea270 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffaec0b9240 con 0x5613b0a41890 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 10 client.27382 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 10 client.27382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 20 client.27382 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0a5af10 0x5613b0a5d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 4 client.27382 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.229+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613b0a10810 con 0x5613b0a5af10 2024-05-08T23:09:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.230+0000 7ffafb400700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0a5af10 0x5613b0a5d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.230+0000 7ffafb400700 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0a5af10 0x5613b0a5d330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2ed7e31206f499 server_cookie=26ee912cd5406798 in_seq=0 out_seq=0 2024-05-08T23:09:44.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.230+0000 7ffae5600700 10 client.27382 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffae5600700 1 -- 172.21.15.46:0/119231109 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613b0a10810 con 0x5613b0a5af10 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffae5600700 10 client.27382 handle_command_reply: tid=0 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 client.27382 shutdown 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 msgr2=0x5613b0a4d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b6f0 tx=0x7ffaec09af10 comp rx=0 tx=0).stop 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 shutdown_connections 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a47970 0x5613b0a4b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a41890 0x5613b0a4d930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0a5af10 0x5613b0a5d330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/119231109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47050 0x5613b0a4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffae5600700 0 client.27382 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 >> 172.21.15.46:0/119231109 conn(0x5613b099f210 msgr2=0x5613b0a3c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 shutdown_connections 2024-05-08T23:09:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.231+0000 7ffb1653ab80 1 -- 172.21.15.46:0/119231109 wait complete. 2024-05-08T23:09:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.232+0000 7ffb1653ab80 20 client.27382 trim_cache size 0 max 16384 2024-05-08T23:09:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.251+0000 7ffb1653ab80 1 Processor -- start 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.251+0000 7ffb1653ab80 1 -- start start 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.251+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a4d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a4c1d0 con 0x5613b0a47970 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a3be30 con 0x5613b0a41890 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a47430 con 0x5613b0a47050 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffb00a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a4d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43054/0 (socket says 172.21.15.46:43054) 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 -- 172.21.15.46:0/1666470526 learned_addr learned my addr 172.21.15.46:0/1666470526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafbe00700 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 -- 172.21.15.46:0/1666470526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 msgr2=0x5613b0a44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a44d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 -- 172.21.15.46:0/1666470526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 msgr2=0x5613b0a4d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a4d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 -- 172.21.15.46:0/1666470526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613b0957560 con 0x5613b0a47050 2024-05-08T23:09:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.252+0000 7ffafb400700 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffae80097a0 tx=0x7ffae8009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5509871474e1e740 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.253+0000 7ffaf1c00700 1 -- 172.21.15.46:0/1666470526 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8029070 con 0x5613b0a47050 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.253+0000 7ffaf1c00700 1 -- 172.21.15.46:0/1666470526 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8005ce0 con 0x5613b0a47050 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.253+0000 7ffaf1c00700 1 -- 172.21.15.46:0/1666470526 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8003dc0 con 0x5613b0a47050 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1666470526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 msgr2=0x5613b0a45240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffae80097a0 tx=0x7ffae8009730 comp rx=0 tx=0).stop 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1666470526 shutdown_connections 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a4d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/1666470526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a45240 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1666470526 >> 172.21.15.46:0/1666470526 conn(0x5613b099f210 msgr2=0x5613b0a61c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1666470526 shutdown_connections 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- 172.21.15.46:0/1666470526 wait complete. 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 -- start start 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.254+0000 7ffb1653ab80 1 Processor -- start 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a52550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a43500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a43a40 con 0x5613b0a47970 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a43bb0 con 0x5613b0a41890 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b0a43d70 con 0x5613b0a47050 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb1653ab80 10 client.0 _ll_register_callbacks cb 0x5613b0936b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:09:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a52550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:43070/0 (socket says 172.21.15.46:43070) 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 -- 172.21.15.46:0/2951321053 learned_addr learned my addr 172.21.15.46:0/2951321053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb00a00700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a43500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 -- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 msgr2=0x5613b0a52550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a52550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 -- 172.21.15.46:0/2951321053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 msgr2=0x5613b0a43500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a43500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffafb400700 1 -- 172.21.15.46:0/2951321053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae801d040 con 0x5613b0a47050 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.255+0000 7ffb00a00700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a43500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffafb400700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ffae8011720 tx=0x7ffae8011860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa055eaf050db7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffae5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8037070 con 0x5613b0a47050 2024-05-08T23:09:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffae8011e70 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ffae80039a0 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffae8029040 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffb1653ab80 10 client.27388 fetch_fsmap learned FSMap version 30 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5613b0816cd0 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.256+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8011e70 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffae803a020 con 0x5613b0a47050 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 10 client.27388 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 10 client.27388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 20 client.27388 populate_metadata read hostname 'smithi046' 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0ab25f0 0x5613b0ab4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 4 client.27388 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613b0989ec0 con 0x5613b0ab25f0 2024-05-08T23:09:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffafbe00700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0ab25f0 0x5613b0ab4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffae5600700 10 client.27388 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.257+0000 7ffafbe00700 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0ab25f0 0x5613b0ab4a10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d557a86f068ae869 server_cookie=feb4ec12351d8c40 in_seq=0 out_seq=0 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffae5600700 1 -- 172.21.15.46:0/2951321053 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5613b0989ec0 con 0x5613b0ab25f0 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffae5600700 10 client.27388 handle_command_reply: tid=0 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 client.27388 shutdown 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 msgr2=0x5613b0a42fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ffae8011720 tx=0x7ffae8011860 comp rx=0 tx=0).stop 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 shutdown_connections 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5613b0ab25f0 0x5613b0ab4a10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613b0a47970 0x5613b0a43500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffae5600700 0 client.27388 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613b0a41890 0x5613b0a52550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 --2- 172.21.15.46:0/2951321053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b0a47050 0x5613b0a42fc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 >> 172.21.15.46:0/2951321053 conn(0x5613b099f210 msgr2=0x5613b0a44d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 shutdown_connections 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 1 -- 172.21.15.46:0/2951321053 wait complete. 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.258+0000 7ffb1653ab80 20 client.27388 trim_cache size 0 max 16384 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: "44da8395-bd20-4a22-b9fb-048619dbd91c": { 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:09:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "44da8395-bd20-4a22-b9fb-048619dbd91c", 2024-05-08T23:09:44.263 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:09:44.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:09:44.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:09:44.263 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:09:44.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 msgr2=0x7ffae00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 0x7ffae00a3870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003fd0 tx=0x7ffaf4003a00 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 msgr2=0x7ffafc1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec005740 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 shutdown_connections 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffae00a13b0 0x7ffae00a3870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffafc19de80 0x7ffafc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc100ec0 0x7ffafc1995a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 --2- 172.21.15.46:0/2603436614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffafc105fe0 0x7ffafc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 >> 172.21.15.46:0/2603436614 conn(0x7ffafc0fc740 msgr2=0x7ffafc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 shutdown_connections 2024-05-08T23:09:44.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:09:44.259+0000 7ffb01400700 1 -- 172.21.15.46:0/2603436614 wait complete. 2024-05-08T23:09:44.276 INFO:tasks.cephfs.filesystem:scrub status for tag:44da8395-bd20-4a22-b9fb-048619dbd91c - {'path': '/', 'tag': '44da8395-bd20-4a22-b9fb-048619dbd91c', 'options': 'recursive,force'} 2024-05-08T23:09:44.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:44 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:09:44.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:44 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:44 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:44.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:44 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:44 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:09:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:44 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:44 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:44.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:44 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:44 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:09:44.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:44 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:44 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:44.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:44 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:44.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 455447 10.45 MB/sec execute 126 sec latency 14.901 ms 2024-05-08T23:09:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:45 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:45 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:09:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:45 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:45 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:45 smithi086 ceph-mon[78610]: pgmap v181: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:45 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:45.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:45 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:09:45.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:45 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:45 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:45 smithi053 ceph-mon[77587]: pgmap v181: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:45.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:45 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:45 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:09:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:45 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:45 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:09:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:45 smithi046 ceph-mon[36898]: pgmap v181: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:45.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 457655 10.47 MB/sec execute 127 sec latency 15.534 ms 2024-05-08T23:09:46.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 458701 10.50 MB/sec execute 128 sec latency 86.052 ms 2024-05-08T23:09:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:47 smithi086 ceph-mon[78610]: pgmap v182: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:47.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:47 smithi053 ceph-mon[77587]: pgmap v182: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:47.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:47 smithi046 ceph-mon[36898]: pgmap v182: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:47.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 460400 10.53 MB/sec execute 129 sec latency 15.563 ms 2024-05-08T23:09:48.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 462443 10.55 MB/sec execute 130 sec latency 13.993 ms 2024-05-08T23:09:49.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:49 smithi086 ceph-mon[78610]: pgmap v183: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:49 smithi053 ceph-mon[77587]: pgmap v183: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:49 smithi046 ceph-mon[36898]: pgmap v183: 129 pgs: 129 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-08T23:09:49.886 INFO:tasks.workunit.client.0.smithi046.stdout: 1 464620 10.55 MB/sec execute 131 sec latency 15.997 ms 2024-05-08T23:09:50.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 466057 10.54 MB/sec execute 132 sec latency 72.811 ms 2024-05-08T23:09:51.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:51 smithi086 ceph-mon[78610]: pgmap v184: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:51 smithi053 ceph-mon[77587]: pgmap v184: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:51 smithi046 ceph-mon[36898]: pgmap v184: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-08T23:09:51.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 466934 10.50 MB/sec execute 133 sec latency 325.389 ms 2024-05-08T23:09:52.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 468647 10.48 MB/sec execute 134 sec latency 171.414 ms 2024-05-08T23:09:53.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:53 smithi086 ceph-mon[78610]: pgmap v185: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-08T23:09:53.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:53 smithi053 ceph-mon[77587]: pgmap v185: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-08T23:09:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:53 smithi046 ceph-mon[36898]: pgmap v185: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-08T23:09:53.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 469971 10.45 MB/sec execute 135 sec latency 339.403 ms 2024-05-08T23:09:54.887 INFO:tasks.workunit.client.0.smithi046.stdout: 1 472319 10.47 MB/sec execute 136 sec latency 16.332 ms 2024-05-08T23:09:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:55 smithi086 ceph-mon[78610]: pgmap v186: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-08T23:09:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:55 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:55 smithi053 ceph-mon[77587]: pgmap v186: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-08T23:09:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:55 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:55.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:55 smithi046 ceph-mon[36898]: pgmap v186: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-08T23:09:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:55 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:55.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 474069 10.47 MB/sec execute 137 sec latency 15.230 ms 2024-05-08T23:09:56.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:56 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:56.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:56 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:09:56.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:56 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:56.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:56 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:09:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:56 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:56.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:56 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:09:56.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 475526 10.45 MB/sec execute 138 sec latency 199.269 ms 2024-05-08T23:09:57.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:57 smithi086 ceph-mon[78610]: pgmap v187: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:57.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:57 smithi053 ceph-mon[77587]: pgmap v187: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:57.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:57 smithi046 ceph-mon[36898]: pgmap v187: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:57.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 477110 10.43 MB/sec execute 139 sec latency 17.295 ms 2024-05-08T23:09:58.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 479333 10.44 MB/sec execute 140 sec latency 24.310 ms 2024-05-08T23:09:59.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:09:59 smithi086 ceph-mon[78610]: pgmap v188: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:59.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:09:59 smithi053 ceph-mon[77587]: pgmap v188: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:09:59 smithi046 ceph-mon[36898]: pgmap v188: 129 pgs: 129 active+clean; 281 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-05-08T23:09:59.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 480852 10.42 MB/sec execute 141 sec latency 15.072 ms 2024-05-08T23:10:00.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:00 smithi086 ceph-mon[78610]: overall HEALTH_OK 2024-05-08T23:10:00.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:00 smithi053 ceph-mon[77587]: overall HEALTH_OK 2024-05-08T23:10:00.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:00 smithi046 ceph-mon[36898]: overall HEALTH_OK 2024-05-08T23:10:00.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 482902 10.43 MB/sec execute 142 sec latency 17.562 ms 2024-05-08T23:10:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:01 smithi086 ceph-mon[78610]: pgmap v189: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 328 op/s 2024-05-08T23:10:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:01 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:10:01.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:01 smithi053 ceph-mon[77587]: pgmap v189: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 328 op/s 2024-05-08T23:10:01.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:01 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:10:01.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:01 smithi046 ceph-mon[36898]: pgmap v189: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 328 op/s 2024-05-08T23:10:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:01 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:10:01.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 483483 10.39 MB/sec execute 143 sec latency 57.248 ms 2024-05-08T23:10:02.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:10:02.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:10:02.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:10:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:10:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:10:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:10:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:10:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:10:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:10:02.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 485045 10.36 MB/sec execute 144 sec latency 17.493 ms 2024-05-08T23:10:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:03 smithi086 ceph-mon[78610]: pgmap v190: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 257 op/s 2024-05-08T23:10:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:03 smithi053 ceph-mon[77587]: pgmap v190: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 257 op/s 2024-05-08T23:10:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:03 smithi046 ceph-mon[36898]: pgmap v190: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 257 op/s 2024-05-08T23:10:03.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 487134 10.38 MB/sec execute 145 sec latency 27.422 ms 2024-05-08T23:10:04.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 488962 10.35 MB/sec execute 146 sec latency 19.265 ms 2024-05-08T23:10:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:05 smithi086 ceph-mon[78610]: pgmap v191: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 352 op/s 2024-05-08T23:10:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:05 smithi053 ceph-mon[77587]: pgmap v191: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 352 op/s 2024-05-08T23:10:05.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:05 smithi046 ceph-mon[36898]: pgmap v191: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 352 op/s 2024-05-08T23:10:05.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 490849 10.37 MB/sec execute 147 sec latency 10.462 ms 2024-05-08T23:10:06.889 INFO:tasks.workunit.client.0.smithi046.stdout: 1 492877 10.36 MB/sec execute 148 sec latency 21.774 ms 2024-05-08T23:10:07.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:07 smithi086 ceph-mon[78610]: pgmap v192: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:07.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:07 smithi053 ceph-mon[77587]: pgmap v192: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:07.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:07 smithi046 ceph-mon[36898]: pgmap v192: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:07.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 494537 10.36 MB/sec execute 149 sec latency 19.169 ms 2024-05-08T23:10:08.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 496608 10.37 MB/sec execute 150 sec latency 32.087 ms 2024-05-08T23:10:09.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:09 smithi086 ceph-mon[78610]: pgmap v193: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:09.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:09 smithi053 ceph-mon[77587]: pgmap v193: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:09 smithi046 ceph-mon[36898]: pgmap v193: 129 pgs: 129 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-05-08T23:10:09.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 498168 10.35 MB/sec execute 151 sec latency 13.727 ms 2024-05-08T23:10:10.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 500367 10.36 MB/sec execute 152 sec latency 29.110 ms 2024-05-08T23:10:11.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:11 smithi086 ceph-mon[78610]: pgmap v194: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-08T23:10:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:11 smithi053 ceph-mon[77587]: pgmap v194: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-08T23:10:11.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:11 smithi046 ceph-mon[36898]: pgmap v194: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-08T23:10:11.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 501865 10.35 MB/sec execute 153 sec latency 15.174 ms 2024-05-08T23:10:12.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 504128 10.36 MB/sec execute 154 sec latency 19.988 ms 2024-05-08T23:10:13.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:13 smithi086 ceph-mon[78610]: pgmap v195: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-08T23:10:13.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:13 smithi053 ceph-mon[77587]: pgmap v195: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-08T23:10:13.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:13 smithi046 ceph-mon[36898]: pgmap v195: 129 pgs: 129 active+clean; 320 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-08T23:10:13.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 505671 10.34 MB/sec execute 155 sec latency 14.544 ms 2024-05-08T23:10:14.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:10:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 -- 172.21.15.46:0/4079489546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c10d630 msgr2=0x7f8f5c10faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 --2- 172.21.15.46:0/4079489546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c10d630 0x7f8f5c10faa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400b0a0 tx=0x7f8f5400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 -- 172.21.15.46:0/4079489546 shutdown_connections 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 --2- 172.21.15.46:0/4079489546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c10d630 0x7f8f5c10faa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 --2- 172.21.15.46:0/4079489546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 --2- 172.21.15.46:0/4079489546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c073cf0 0x7f8f5c10d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 -- 172.21.15.46:0/4079489546 >> 172.21.15.46:0/4079489546 conn(0x7f8f5c0fc900 msgr2=0x7f8f5c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 -- 172.21.15.46:0/4079489546 shutdown_connections 2024-05-08T23:10:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.528+0000 7f8f61a00700 1 -- 172.21.15.46:0/4079489546 wait complete. 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.529+0000 7f8f61a00700 1 Processor -- start 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.529+0000 7f8f61a00700 1 -- start start 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c073cf0 0x7f8f5c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 0x7f8f5c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19e550 con 0x7f8f5c073cf0 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19e6c0 con 0x7f8f5c10d630 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f61a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c19e880 con 0x7f8f5c0733d0 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49984/0 (socket says 172.21.15.46:49984) 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 -- 172.21.15.46:0/1935837109 learned_addr learned my addr 172.21.15.46:0/1935837109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5a000700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c073cf0 0x7f8f5c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5b400700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 0x7f8f5c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 -- 172.21.15.46:0/1935837109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 msgr2=0x7f8f5c19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 0x7f8f5c19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 -- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c073cf0 msgr2=0x7f8f5c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c073cf0 0x7f8f5c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5aa00700 1 -- 172.21.15.46:0/1935837109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c009520 con 0x7f8f5c0733d0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.530+0000 7f8f5b400700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 0x7f8f5c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00eb30 tx=0x7f8f4c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f53400700 1 -- 172.21.15.46:0/1935837109 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c009880 con 0x7f8f5c0733d0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f53400700 1 -- 172.21.15.46:0/1935837109 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4c0044e0 con 0x7f8f5c0733d0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f53400700 1 -- 172.21.15.46:0/1935837109 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c003e90 con 0x7f8f5c0733d0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f54009d00 con 0x7f8f5c0733d0 2024-05-08T23:10:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.531+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c1a61c0 con 0x7f8f5c0733d0 2024-05-08T23:10:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.535+0000 7f8f53400700 1 -- 172.21.15.46:0/1935837109 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f4c004000 con 0x7f8f5c0733d0 2024-05-08T23:10:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.536+0000 7f8f75534b80 1 Processor -- start 2024-05-08T23:10:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.536+0000 7f8f75534b80 1 -- start start 2024-05-08T23:10:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.536+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.536+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f75534b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c258446150 con 0x55c2584ef890 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f75534b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c258443b10 con 0x55c2584f4e00 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f75534b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25843f400 con 0x55c2584f5720 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f53400700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 0x7f8f400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49988/0 (socket says 172.21.15.46:49988) 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5aa00700 1 -- 172.21.15.46:0/3744428002 learned_addr learned my addr 172.21.15.46:0/3744428002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5b400700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f53400700 1 -- 172.21.15.46:0/1935837109 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8f4c014070 con 0x7f8f5c0733d0 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5b400700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 0x7f8f400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 -- 172.21.15.46:0/3744428002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 msgr2=0x55c2584ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 -- 172.21.15.46:0/3744428002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 msgr2=0x55c2584f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.537+0000 7f8f5a000700 1 -- 172.21.15.46:0/3744428002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2582c03b0 con 0x55c2584ef890 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584efcd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009c60 tx=0x7f8f48009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e046d8212041a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f5b400700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 0x7f8f400a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009850 tx=0x7f8f5400b3b0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f47e00700 1 -- 172.21.15.46:0/3744428002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011cb0 con 0x55c2584ef890 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f47e00700 1 -- 172.21.15.46:0/3744428002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f48011e10 con 0x55c2584ef890 2024-05-08T23:10:14.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.538+0000 7f8f47e00700 1 -- 172.21.15.46:0/3744428002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48004ea0 con 0x55c2584ef890 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 -- 172.21.15.46:0/3744428002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 msgr2=0x55c2584efcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584efcd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009c60 tx=0x7f8f48009f70 comp rx=0 tx=0).stop 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 -- 172.21.15.46:0/3744428002 shutdown_connections 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584efcd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3744428002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f5720 0x55c2584ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 -- 172.21.15.46:0/3744428002 >> 172.21.15.46:0/3744428002 conn(0x55c25844d210 msgr2=0x55c2584e9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 -- 172.21.15.46:0/3744428002 shutdown_connections 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.539+0000 7f8f75534b80 1 -- 172.21.15.46:0/3744428002 wait complete. 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.540+0000 7f8f75534b80 1 -- start start 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.540+0000 7f8f75534b80 1 Processor -- start 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.540+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.540+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2584f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.540+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f5720 0x55c2584f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f75534b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2583db560 con 0x55c2584ef890 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f75534b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2583db200 con 0x55c2584f5720 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f75534b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2582c26a0 con 0x55c2584f4e00 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f75534b80 10 client.0 _ll_register_callbacks cb 0x55c2583e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2584f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48822/0 (socket says 172.21.15.46:48822) 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 -- 172.21.15.46:0/3431844798 learned_addr learned my addr 172.21.15.46:0/3431844798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f5720 0x55c2584f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 -- 172.21.15.46:0/3431844798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 msgr2=0x55c2584f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2584f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 -- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f5720 msgr2=0x55c2584f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f5720 0x55c2584f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.541+0000 7f8f5aa00700 1 -- 172.21.15.46:0/3431844798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c1a61c0 con 0x55c2584ef890 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c000c00 tx=0x7f8f4c0694d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9155962b4e980bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f44c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c065bb0 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8f4c065e40 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c258398270 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f4c00cc90 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f75534b80 10 client.17352 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8f480098c0 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.542+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c258398270 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f4c0691b0 con 0x55c2584ef890 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 10 client.17352 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 20 client.17352 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c2585091d0 0x55c25850b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 4 client.17352 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2584be810 con 0x55c2585091d0 2024-05-08T23:10:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.543+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c2585091d0 0x55c25850b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.544+0000 7f8f5a000700 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c2585091d0 0x55c25850b5c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc81de63926fde6 server_cookie=3b9970063eae75d0 in_seq=0 out_seq=0 2024-05-08T23:10:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.544+0000 7f8f44c00700 10 client.17352 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f44c00700 1 -- 172.21.15.46:0/3431844798 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2584be810 con 0x55c2585091d0 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f44c00700 10 client.17352 handle_command_reply: tid=0 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 client.17352 shutdown 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 msgr2=0x55c2584f8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c000c00 tx=0x7f8f4c0694d0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 shutdown_connections 2024-05-08T23:10:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f5720 0x55c2584f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c2585091d0 0x55c25850b5c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584ef890 0x55c2584f8aa0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 --2- 172.21.15.46:0/3431844798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2584f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f44c00700 0 client.17352 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 >> 172.21.15.46:0/3431844798 conn(0x55c25844d210 msgr2=0x55c2584e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 shutdown_connections 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 1 -- 172.21.15.46:0/3431844798 wait complete. 2024-05-08T23:10:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.545+0000 7f8f75534b80 20 client.17352 trim_cache size 0 max 16384 2024-05-08T23:10:14.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.565+0000 7f8f75534b80 1 Processor -- start 2024-05-08T23:10:14.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.565+0000 7f8f75534b80 1 -- start start 2024-05-08T23:10:14.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2584f5560 con 0x55c2584f5720 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2585107a0 con 0x55c2584f4e00 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f75534b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2584f8140 con 0x55c2584ef890 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 -- 172.21.15.46:0/105919813 learned_addr learned my addr 172.21.15.46:0/105919813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5a000700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5b400700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 -- 172.21.15.46:0/105919813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 msgr2=0x55c2584f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 -- 172.21.15.46:0/105919813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 msgr2=0x55c2584e9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584e9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5aa00700 1 -- 172.21.15.46:0/105919813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c25826cb60 con 0x55c2584ef890 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.566+0000 7f8f5b400700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584e9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c06c2c0 tx=0x7f8f4c00ea80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35addbfcfbd4c87e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f50c00700 1 -- 172.21.15.46:0/105919813 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0b0030 con 0x55c2584ef890 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f50c00700 1 -- 172.21.15.46:0/105919813 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4c003680 con 0x55c2584ef890 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f50c00700 1 -- 172.21.15.46:0/105919813 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0a2690 con 0x55c2584ef890 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f75534b80 1 -- 172.21.15.46:0/105919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 msgr2=0x55c2584ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.567+0000 7f8f75534b80 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c06c2c0 tx=0x7f8f4c00ea80 comp rx=0 tx=0).stop 2024-05-08T23:10:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 -- 172.21.15.46:0/105919813 shutdown_connections 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584e9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584ef890 0x55c2584ef350 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 --2- 172.21.15.46:0/105919813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584f4e00 0x55c2584f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 -- 172.21.15.46:0/105919813 >> 172.21.15.46:0/105919813 conn(0x55c25844d210 msgr2=0x55c2584f3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 -- 172.21.15.46:0/105919813 shutdown_connections 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 -- 172.21.15.46:0/105919813 wait complete. 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 -- start start 2024-05-08T23:10:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.568+0000 7f8f75534b80 1 Processor -- start 2024-05-08T23:10:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584ef890 0x55c258500170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2584f1550 con 0x55c2584f5720 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2584f16c0 con 0x55c2584ef890 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2584f1880 con 0x55c2584f4e00 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f75534b80 10 client.0 _ll_register_callbacks cb 0x55c2583e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50012/0 (socket says 172.21.15.46:50012) 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 -- 172.21.15.46:0/2348679449 learned_addr learned my addr 172.21.15.46:0/2348679449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5b400700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584ef890 0x55c258500170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 -- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584ef890 msgr2=0x55c258500170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584ef890 0x55c258500170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 -- 172.21.15.46:0/2348679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 msgr2=0x55c2584f0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584f0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.569+0000 7f8f5a000700 1 -- 172.21.15.46:0/2348679449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480098c0 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f5a000700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009230 tx=0x7f8f48004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c395bd96a9502a1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f44c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011ab0 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f8f48030030 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c258437ec0 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f48011c10 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f75534b80 10 client.27406 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c258405560 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.570+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c258437ec0 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f480039e0 con 0x55c2584f4e00 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 10 client.27406 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 10 client.27406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 20 client.27406 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c258560900 0x55c258562d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 4 client.27406 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c25828dcf0 con 0x55c258560900 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.571+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c258560900 0x55c258562d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f5aa00700 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c258560900 0x55c258562d20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f8e7baa08b3e0 server_cookie=2eba1a39bd37b459 in_seq=0 out_seq=0 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f44c00700 10 client.27406 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f44c00700 1 -- 172.21.15.46:0/2348679449 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c25828dcf0 con 0x55c258560900 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f44c00700 10 client.27406 handle_command_reply: tid=0 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 client.27406 shutdown 2024-05-08T23:10:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 msgr2=0x55c2585006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009230 tx=0x7f8f48004a30 comp rx=0 tx=0).stop 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 shutdown_connections 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c258560900 0x55c258562d20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2584f5720 0x55c2584f0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2584ef890 0x55c258500170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 --2- 172.21.15.46:0/2348679449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2584f4e00 0x55c2585006b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f44c00700 0 client.27406 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 >> 172.21.15.46:0/2348679449 conn(0x55c25844d210 msgr2=0x55c2584f6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 shutdown_connections 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.572+0000 7f8f75534b80 1 -- 172.21.15.46:0/2348679449 wait complete. 2024-05-08T23:10:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.573+0000 7f8f75534b80 20 client.27406 trim_cache size 0 max 16384 2024-05-08T23:10:14.580 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:10:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:10:14.580 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.573+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 msgr2=0x7f8f400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.573+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 0x7f8f400a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009850 tx=0x7f8f5400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.573+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 msgr2=0x7f8f5c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.573+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00eb30 tx=0x7f8f4c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 shutdown_connections 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f5c10d630 0x7f8f5c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5c0733d0 0x7f8f5c199680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f400a1310 0x7f8f400a37d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 --2- 172.21.15.46:0/1935837109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5c073cf0 0x7f8f5c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 >> 172.21.15.46:0/1935837109 conn(0x7f8f5c0fc900 msgr2=0x7f8f5c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 shutdown_connections 2024-05-08T23:10:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.574+0000 7f8f61a00700 1 -- 172.21.15.46:0/1935837109 wait complete. 2024-05-08T23:10:14.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:10:14.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 -- 172.21.15.46:0/242746654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368073c60 msgr2=0x7fc3681116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 --2- 172.21.15.46:0/242746654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368073c60 0x7fc3681116d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc3540098f0 tx=0x7fc354009c00 comp rx=0 tx=0).stop 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 -- 172.21.15.46:0/242746654 shutdown_connections 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 --2- 172.21.15.46:0/242746654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368111c10 0x7fc368114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 --2- 172.21.15.46:0/242746654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 0x7fc368073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 --2- 172.21.15.46:0/242746654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368073c60 0x7fc3681116d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 -- 172.21.15.46:0/242746654 >> 172.21.15.46:0/242746654 conn(0x7fc3680fc750 msgr2=0x7fc3680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.823+0000 7fc36e000700 1 -- 172.21.15.46:0/242746654 shutdown_connections 2024-05-08T23:10:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.824+0000 7fc36e000700 1 -- 172.21.15.46:0/242746654 wait complete. 2024-05-08T23:10:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 Processor -- start 2024-05-08T23:10:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 -- start start 2024-05-08T23:10:14.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 0x7fc36819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 0x7fc3681a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a15e0 con 0x7fc368111c10 2024-05-08T23:10:14.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a1750 con 0x7fc368073340 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.825+0000 7fc36e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a1910 con 0x7fc368073c60 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc36cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 0x7fc36819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50040/0 (socket says 172.21.15.46:50040) 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 -- 172.21.15.46:0/2044041369 learned_addr learned my addr 172.21.15.46:0/2044041369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc36d600700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 0x7fc3681a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 -- 172.21.15.46:0/2044041369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 msgr2=0x7fc36819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 0x7fc36819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 -- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 msgr2=0x7fc3681a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 0x7fc3681a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc367e00700 1 -- 172.21.15.46:0/2044041369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354009520 con 0x7fc368073c60 2024-05-08T23:10:14.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.826+0000 7fc36d600700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 0x7fc3681a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc367e00700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc354005950 tx=0x7fc354010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35401c070 con 0x7fc368073c60 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35400bba0 con 0x7fc368073c60 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3681a1b90 con 0x7fc368073c60 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354003a00 con 0x7fc368073c60 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.827+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681aa550 con 0x7fc368073c60 2024-05-08T23:10:14.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.828+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368104150 con 0x7fc368073c60 2024-05-08T23:10:14.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.831+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc354020030 con 0x7fc368073c60 2024-05-08T23:10:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.832+0000 7fc365600700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.832+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc3540cdeb0 con 0x7fc368073c60 2024-05-08T23:10:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.832+0000 7fc36cc00700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.833+0000 7fc36cc00700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 0x7fc3500a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:14.837+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc354096f80 con 0x7fc368073c60 2024-05-08T23:10:14.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 507787 10.36 MB/sec execute 156 sec latency 15.631 ms 2024-05-08T23:10:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.144+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc36819eab0 con 0x7fc368073c60 2024-05-08T23:10:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.145+0000 7fc365600700 1 -- 172.21.15.46:0/2044041369 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fc35409a5a0 con 0x7fc368073c60 2024-05-08T23:10:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:10:15.146 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 msgr2=0x7fc3500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 0x7fc3500a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c009310 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 msgr2=0x7fc36819dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc354005950 tx=0x7fc354010ff0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 shutdown_connections 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368111c10 0x7fc3681a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc368073340 0x7fc36819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 --2- 172.21.15.46:0/2044041369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc368073c60 0x7fc36819dcf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 >> 172.21.15.46:0/2044041369 conn(0x7fc3680fc750 msgr2=0x7fc3680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.148+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 shutdown_connections 2024-05-08T23:10:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.149+0000 7fc36e000700 1 -- 172.21.15.46:0/2044041369 wait complete. 2024-05-08T23:10:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:10:15.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:10:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.412+0000 7f4a35c00700 1 -- 172.21.15.46:0/1059807793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 msgr2=0x7f4a30114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.412+0000 7f4a35c00700 1 --2- 172.21.15.46:0/1059807793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a30114210 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009f90 tx=0x7f4a2800b330 comp rx=0 tx=0).stop 2024-05-08T23:10:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 -- 172.21.15.46:0/1059807793 shutdown_connections 2024-05-08T23:10:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 --2- 172.21.15.46:0/1059807793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a30114210 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 --2- 172.21.15.46:0/1059807793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 0x7f4a30073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 --2- 172.21.15.46:0/1059807793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a301118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 -- 172.21.15.46:0/1059807793 >> 172.21.15.46:0/1059807793 conn(0x7f4a300fc8e0 msgr2=0x7f4a300fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 -- 172.21.15.46:0/1059807793 shutdown_connections 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.413+0000 7f4a35c00700 1 -- 172.21.15.46:0/1059807793 wait complete. 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.414+0000 7f4a35c00700 1 Processor -- start 2024-05-08T23:10:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.414+0000 7f4a35c00700 1 -- start start 2024-05-08T23:10:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 0x7f4a30072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a3006b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a3006be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3006c370 con 0x7f4a30073c60 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3006c4e0 con 0x7f4a30073340 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a35c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3006c6a0 con 0x7f4a30111e20 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a3006b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a3006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a3006b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48882/0 (socket says 172.21.15.46:48882) 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2e000700 1 -- 172.21.15.46:0/823262417 learned_addr learned my addr 172.21.15.46:0/823262417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 0x7f4a30072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 -- 172.21.15.46:0/823262417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 msgr2=0x7f4a30072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 0x7f4a30072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 -- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 msgr2=0x7f4a3006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a3006b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.415+0000 7f4a2f400700 1 -- 172.21.15.46:0/823262417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009c40 con 0x7f4a30111e20 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a2f400700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a3006be00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b6c0 tx=0x7f4a280094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a27400700 1 -- 172.21.15.46:0/823262417 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800e040 con 0x7f4a30111e20 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a27400700 1 -- 172.21.15.46:0/823262417 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2801b450 con 0x7f4a30111e20 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a30070410 con 0x7f4a30111e20 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a27400700 1 -- 172.21.15.46:0/823262417 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280052c0 con 0x7f4a30111e20 2024-05-08T23:10:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.416+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a30070970 con 0x7f4a30111e20 2024-05-08T23:10:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.419+0000 7f4a27400700 1 -- 172.21.15.46:0/823262417 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4a2801b5c0 con 0x7f4a30111e20 2024-05-08T23:10:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 Processor -- start 2024-05-08T23:10:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 -- start start 2024-05-08T23:10:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a27400700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 0x7f4a140a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a27400700 1 -- 172.21.15.46:0/823262417 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a280ce200 con 0x7f4a30111e20 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc373e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a2e000700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 0x7f4a140a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc3695150 con 0x558dc3743e00 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc3692b10 con 0x558dc3744720 2024-05-08T23:10:15.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.421+0000 7f4a49729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc368e400 con 0x558dc373e890 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc373e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50070/0 (socket says 172.21.15.46:50070) 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 -- 172.21.15.46:0/2434763744 learned_addr learned my addr 172.21.15.46:0/2434763744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 -- 172.21.15.46:0/2434763744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 msgr2=0x558dc373e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc373e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 -- 172.21.15.46:0/2434763744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 msgr2=0x558dc37441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 -- 172.21.15.46:0/2434763744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc350f3b0 con 0x558dc373e890 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:15.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2f400700 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009790 tx=0x7f4a280972d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f194a7b8e7560f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.422+0000 7f4a2e000700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 0x7f4a140a37d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x558dc373abd0 tx=0x7f4a1c009860 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.423+0000 7f4a1be00700 1 -- 172.21.15.46:0/2434763744 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280ce020 con 0x558dc373e890 2024-05-08T23:10:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.423+0000 7f4a1be00700 1 -- 172.21.15.46:0/2434763744 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a280a10c0 con 0x558dc373e890 2024-05-08T23:10:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.423+0000 7f4a1be00700 1 -- 172.21.15.46:0/2434763744 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280a3380 con 0x558dc373e890 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.423+0000 7f4a49729b80 1 -- 172.21.15.46:0/2434763744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 msgr2=0x558dc373ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.423+0000 7f4a49729b80 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009790 tx=0x7f4a280972d0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 -- 172.21.15.46:0/2434763744 shutdown_connections 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc373ecd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 --2- 172.21.15.46:0/2434763744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc373e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 -- 172.21.15.46:0/2434763744 >> 172.21.15.46:0/2434763744 conn(0x558dc369c210 msgr2=0x558dc3738df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 -- 172.21.15.46:0/2434763744 shutdown_connections 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 -- 172.21.15.46:0/2434763744 wait complete. 2024-05-08T23:10:15.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.424+0000 7f4a49729b80 1 -- start start 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 Processor -- start 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc3747fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc3748510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc362a560 con 0x558dc3743e00 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc362a200 con 0x558dc3744720 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc35116a0 con 0x558dc373e890 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a49729b80 10 client.0 _ll_register_callbacks cb 0x558dc3633b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50076/0 (socket says 172.21.15.46:50076) 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.425+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1397327768 learned_addr learned my addr 172.21.15.46:0/1397327768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2f400700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc3748510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2e000700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc3747fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1397327768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 msgr2=0x558dc3748510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc3748510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1397327768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 msgr2=0x558dc3747fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc3747fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1397327768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a30070970 con 0x558dc373e890 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4a20012810 tx=0x7f4a20012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e45d2cf7b0f5626c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a18c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017070 con 0x558dc373e890 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a2000f5e0 con 0x558dc373e890 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.426+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dc35e7270 con 0x558dc373e890 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a200122e0 con 0x558dc373e890 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 10 client.27424 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558dc34bbb60 con 0x558dc373e890 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dc35e7270 con 0x558dc373e890 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a20010990 con 0x558dc373e890 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 10 client.27424 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 10 client.27424 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 20 client.27424 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc375a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 4 client.27424 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc370d810 con 0x558dc37580f0 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.427+0000 7f4a2e000700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc375a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.428+0000 7f4a2e000700 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc375a510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fddfdeccf1c0fd9 server_cookie=79362196bcd7bf3a in_seq=0 out_seq=0 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.428+0000 7f4a18c00700 10 client.27424 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:15.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.428+0000 7f4a18c00700 1 -- 172.21.15.46:0/1397327768 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dc370d810 con 0x558dc37580f0 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.428+0000 7f4a18c00700 10 client.27424 handle_command_reply: tid=0 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 client.27424 shutdown 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 msgr2=0x558dc3747a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4a20012810 tx=0x7f4a20012b20 comp rx=0 tx=0).stop 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 shutdown_connections 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc375a510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc3744720 0x558dc3748510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a18c00700 0 client.27424 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc373e890 0x558dc3747a90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1397327768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc3747fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 >> 172.21.15.46:0/1397327768 conn(0x558dc369c210 msgr2=0x558dc37385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 shutdown_connections 2024-05-08T23:10:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 1 -- 172.21.15.46:0/1397327768 wait complete. 2024-05-08T23:10:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.429+0000 7f4a49729b80 20 client.27424 trim_cache size 0 max 16384 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.445+0000 7f4a49729b80 1 Processor -- start 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.445+0000 7f4a49729b80 1 -- start start 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.445+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.445+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.445+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc3738d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a49729b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc373a840 con 0x558dc3743e00 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a49729b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc3735dc0 con 0x558dc373e890 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a49729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc375f750 con 0x558dc3744720 2024-05-08T23:10:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34948/0 (socket says 172.21.15.46:34948) 2024-05-08T23:10:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1135669793 learned_addr learned my addr 172.21.15.46:0/1135669793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2e000700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2f400700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc3738d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1135669793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 msgr2=0x558dc3738d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc3738d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1135669793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 msgr2=0x558dc37433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2ea00700 1 -- 172.21.15.46:0/1135669793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc3654560 con 0x558dc373e890 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.446+0000 7f4a2f400700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc3738d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.447+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a20010d40 tx=0x7f4a20009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad09342c3a5e5766 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.447+0000 7f4a24c00700 1 -- 172.21.15.46:0/1135669793 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017040 con 0x558dc373e890 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.447+0000 7f4a24c00700 1 -- 172.21.15.46:0/1135669793 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a200092d0 con 0x558dc373e890 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.447+0000 7f4a24c00700 1 -- 172.21.15.46:0/1135669793 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a200038e0 con 0x558dc373e890 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- 172.21.15.46:0/1135669793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 msgr2=0x558dc373e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a20010d40 tx=0x7f4a20009730 comp rx=0 tx=0).stop 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- 172.21.15.46:0/1135669793 shutdown_connections 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc3738d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc373e350 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 --2- 172.21.15.46:0/1135669793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc37433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- 172.21.15.46:0/1135669793 >> 172.21.15.46:0/1135669793 conn(0x558dc369c210 msgr2=0x558dc3746360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- 172.21.15.46:0/1135669793 shutdown_connections 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- 172.21.15.46:0/1135669793 wait complete. 2024-05-08T23:10:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.448+0000 7f4a49729b80 1 -- start start 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 Processor -- start 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc374b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc374ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc374c050 con 0x558dc3743e00 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc374c1c0 con 0x558dc373e890 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc374c380 con 0x558dc3744720 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a49729b80 10 client.0 _ll_register_callbacks cb 0x558dc3633b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc374b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a2f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc374ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a2ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34960/0 (socket says 172.21.15.46:34960) 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.449+0000 7f4a2ea00700 1 -- 172.21.15.46:0/576507414 learned_addr learned my addr 172.21.15.46:0/576507414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 -- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 msgr2=0x558dc374ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc374ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 -- 172.21.15.46:0/576507414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 msgr2=0x558dc374f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 -- 172.21.15.46:0/576507414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dc34bbb60 con 0x558dc3743e00 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2f400700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc374ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a2e000700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc374b540 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x558dc3744b80 tx=0x7f4a1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffce4170fd1f9631 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a18c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c028730 con 0x558dc3743e00 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f4a1c026040 con 0x558dc3743e00 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.450+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a280a10c0 con 0x558dc3743e00 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1c028890 con 0x558dc3743e00 2024-05-08T23:10:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 10 client.17382 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558dc3513cd0 con 0x558dc3743e00 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a280a10c0 con 0x558dc3743e00 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a1c005400 con 0x558dc3743e00 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 10 client.17382 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 10 client.17382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 20 client.17382 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc37b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 4 client.17382 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558dc3686ec0 con 0x558dc37580f0 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.451+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc37b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a2ea00700 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc37b1d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24a4ed19fd018067 server_cookie=8dc6a55f220dc76e in_seq=0 out_seq=0 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a18c00700 10 client.17382 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a18c00700 1 -- 172.21.15.46:0/576507414 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558dc3686ec0 con 0x558dc37580f0 2024-05-08T23:10:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a18c00700 10 client.17382 handle_command_reply: tid=0 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a49729b80 1 client.17382 shutdown 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 msgr2=0x558dc374b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc374b540 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x558dc3744b80 tx=0x7f4a1c009310 comp rx=0 tx=0).stop 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 shutdown_connections 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.452+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558dc37580f0 0x558dc37b1d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc3744720 0x558dc374ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc373e890 0x558dc374f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 --2- 172.21.15.46:0/576507414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558dc3743e00 0x558dc374b540 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 >> 172.21.15.46:0/576507414 conn(0x558dc369c210 msgr2=0x558dc3745440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a18c00700 0 client.17382 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 shutdown_connections 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 1 -- 172.21.15.46:0/576507414 wait complete. 2024-05-08T23:10:15.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a49729b80 20 client.17382 trim_cache size 0 max 16384 2024-05-08T23:10:15.462 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 msgr2=0x7f4a140a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 0x7f4a140a37d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x558dc373abd0 tx=0x7f4a1c009860 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 msgr2=0x7f4a3006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.453+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a3006be00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b6c0 tx=0x7f4a280094b0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 shutdown_connections 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30111e20 0x7f4a3006be00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a30073340 0x7f4a30072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a140a1310 0x7f4a140a37d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 --2- 172.21.15.46:0/823262417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a30073c60 0x7f4a3006b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 >> 172.21.15.46:0/823262417 conn(0x7f4a300fc8e0 msgr2=0x7f4a300fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 shutdown_connections 2024-05-08T23:10:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.454+0000 7f4a35c00700 1 -- 172.21.15.46:0/823262417 wait complete. 2024-05-08T23:10:15.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:15 smithi046 ceph-mon[36898]: pgmap v196: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-08T23:10:15.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:15 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2044041369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:15.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:10:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 -- 172.21.15.46:0/2218164082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc10b3c0 msgr2=0x7ff8bc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 --2- 172.21.15.46:0/2218164082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc10d880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400b0a0 tx=0x7ff8b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 -- 172.21.15.46:0/2218164082 shutdown_connections 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 --2- 172.21.15.46:0/2218164082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc10d880 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 --2- 172.21.15.46:0/2218164082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc101fa0 0x7ff8bc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 --2- 172.21.15.46:0/2218164082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 0x7ff8bc10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 -- 172.21.15.46:0/2218164082 >> 172.21.15.46:0/2218164082 conn(0x7ff8bc0fb760 msgr2=0x7ff8bc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 -- 172.21.15.46:0/2218164082 shutdown_connections 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.708+0000 7ff8c2000700 1 -- 172.21.15.46:0/2218164082 wait complete. 2024-05-08T23:10:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.709+0000 7ff8c2000700 1 Processor -- start 2024-05-08T23:10:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.709+0000 7ff8c2000700 1 -- start start 2024-05-08T23:10:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc101fa0 0x7ff8bc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 0x7ff8bc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc19e4b0 con 0x7ff8bc1028c0 2024-05-08T23:10:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc19e620 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.710+0000 7ff8c2000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc19e7e0 con 0x7ff8bc101fa0 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34976/0 (socket says 172.21.15.46:34976) 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 -- 172.21.15.46:0/1227980222 learned_addr learned my addr 172.21.15.46:0/1227980222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8baa00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 0x7ff8bc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bb400700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc101fa0 0x7ff8bc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 -- 172.21.15.46:0/1227980222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc101fa0 msgr2=0x7ff8bc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc101fa0 0x7ff8bc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 -- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 msgr2=0x7ff8bc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 0x7ff8bc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 -- 172.21.15.46:0/1227980222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8009520 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.711+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400b070 tx=0x7ff8b40078c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.712+0000 7ff8b3e00700 1 -- 172.21.15.46:0/1227980222 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400e040 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.712+0000 7ff8b3e00700 1 -- 172.21.15.46:0/1227980222 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b401d470 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.712+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b4009d00 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.712+0000 7ff8b3e00700 1 -- 172.21.15.46:0/1227980222 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4005280 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.712+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc1a6120 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:15 smithi086 ceph-mon[78610]: pgmap v196: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-08T23:10:15.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:15 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2044041369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.717+0000 7ff8b3e00700 1 -- 172.21.15.46:0/1227980222 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8b401d5e0 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.719+0000 7ff8d5ba1b80 1 Processor -- start 2024-05-08T23:10:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.719+0000 7ff8b3e00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 0x7ff8a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.719+0000 7ff8b3e00700 1 -- 172.21.15.46:0/1227980222 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff8b40ce420 con 0x7ff8bc10b3c0 2024-05-08T23:10:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.720+0000 7ff8baa00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 0x7ff8a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.720+0000 7ff8d5ba1b80 1 -- start start 2024-05-08T23:10:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 0x560e1318c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e130da150 con 0x560e13191f80 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e130d7b10 con 0x560e1318c0f0 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e130d3400 con 0x560e13191660 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 0x560e1318c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50130/0 (socket says 172.21.15.46:50130) 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 -- 172.21.15.46:0/1167084856 learned_addr learned my addr 172.21.15.46:0/1167084856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 -- 172.21.15.46:0/1167084856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 msgr2=0x560e1318c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 0x560e1318c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.721+0000 7ff8bb400700 1 -- 172.21.15.46:0/1167084856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 msgr2=0x560e1318bb60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.722+0000 7ff8bb400700 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.722+0000 7ff8bb400700 1 -- 172.21.15.46:0/1167084856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e12f543b0 con 0x560e13191660 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.722+0000 7ff8baa00700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 0x7ff8a00a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc10dd20 tx=0x7ff8a8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.722+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 0x560e1318c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.722+0000 7ff8bb400700 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009db0 tx=0x7ff8ac011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd583dc9635bb62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.723+0000 7ff8a7e00700 1 -- 172.21.15.46:0/1167084856 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac009ea0 con 0x560e13191660 2024-05-08T23:10:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.723+0000 7ff8a7e00700 1 -- 172.21.15.46:0/1167084856 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8ac019af0 con 0x560e13191660 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.723+0000 7ff8a7e00700 1 -- 172.21.15.46:0/1167084856 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00ebd0 con 0x560e13191660 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.723+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/1167084856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 msgr2=0x560e13191a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.723+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009db0 tx=0x7ff8ac011720 comp rx=0 tx=0).stop 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/1167084856 shutdown_connections 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e1318c0f0 0x560e1318c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191660 0x560e13191a40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/1167084856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/1167084856 >> 172.21.15.46:0/1167084856 conn(0x560e130e1210 msgr2=0x560e1317ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/1167084856 shutdown_connections 2024-05-08T23:10:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.724+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/1167084856 wait complete. 2024-05-08T23:10:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.725+0000 7ff8d5ba1b80 1 -- start start 2024-05-08T23:10:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.725+0000 7ff8d5ba1b80 1 Processor -- start 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13195a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1306f560 con 0x560e13191f80 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1306f200 con 0x560e13191660 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e12f566a0 con 0x560e1318c0f0 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8d5ba1b80 10 client.0 _ll_register_callbacks cb 0x560e13078b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35002/0 (socket says 172.21.15.46:35002) 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8baa00700 1 -- 172.21.15.46:0/243078035 learned_addr learned my addr 172.21.15.46:0/243078035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.726+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8baa00700 1 -- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 msgr2=0x560e13195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8baa00700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8baa00700 1 -- 172.21.15.46:0/243078035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 msgr2=0x560e1318a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8baa00700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8baa00700 1 -- 172.21.15.46:0/243078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc1a6120 con 0x560e13191660 2024-05-08T23:10:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.727+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.729+0000 7ff8baa00700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800ebe0 tx=0x7ff8a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=befce7a61fa7faba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.729+0000 7ff8a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:10:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.729+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801cc40 con 0x560e13191660 2024-05-08T23:10:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.729+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff8a8023070 con 0x560e13191660 2024-05-08T23:10:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.729+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e1302c270 con 0x560e13191660 2024-05-08T23:10:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.730+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a801cda0 con 0x560e13191660 2024-05-08T23:10:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.730+0000 7ff8d5ba1b80 10 client.27380 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.730+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff8ac0099e0 con 0x560e13191660 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.730+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e1302c270 con 0x560e13191660 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.730+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8a801d4a0 con 0x560e13191660 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 10 client.27380 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 10 client.27380 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:10:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 20 client.27380 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e131ae3c0 0x560e131b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 4 client.27380 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e13152810 con 0x560e131ae3c0 2024-05-08T23:10:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.731+0000 7ff8bb400700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e131ae3c0 0x560e131b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.732+0000 7ff8bb400700 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e131ae3c0 0x560e131b07e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4ed0953184df85c server_cookie=847ef7b04574fc53 in_seq=0 out_seq=0 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.732+0000 7ff8a4c00700 10 client.27380 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.732+0000 7ff8a4c00700 1 -- 172.21.15.46:0/243078035 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e13152810 con 0x560e131ae3c0 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.732+0000 7ff8a4c00700 10 client.27380 handle_command_reply: tid=0 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.732+0000 7ff8d5ba1b80 1 client.27380 shutdown 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 msgr2=0x560e13195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800ebe0 tx=0x7ff8a8004750 comp rx=0 tx=0).stop 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 shutdown_connections 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e131ae3c0 0x560e131b07e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8a4c00700 0 client.27380 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191f80 0x560e1318a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/243078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13195fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 >> 172.21.15.46:0/243078035 conn(0x560e130e1210 msgr2=0x560e1317d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 shutdown_connections 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.733+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/243078035 wait complete. 2024-05-08T23:10:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.734+0000 7ff8d5ba1b80 20 client.27380 trim_cache size 0 max 16384 2024-05-08T23:10:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.750+0000 7ff8d5ba1b80 1 Processor -- start 2024-05-08T23:10:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 -- start start 2024-05-08T23:10:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191f80 0x560e13195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e131b59f0 con 0x560e1318c0f0 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e13196300 con 0x560e13191660 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1317f800 con 0x560e13191f80 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35018/0 (socket says 172.21.15.46:35018) 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8baa00700 1 -- 172.21.15.46:0/283127211 learned_addr learned my addr 172.21.15.46:0/283127211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48944/0 (socket says 172.21.15.46:48944) 2024-05-08T23:10:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.751+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191f80 0x560e13195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 -- 172.21.15.46:0/283127211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191f80 msgr2=0x560e13195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191f80 0x560e13195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 -- 172.21.15.46:0/283127211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 msgr2=0x560e13193370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 -- 172.21.15.46:0/283127211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e12f00b60 con 0x560e1318c0f0 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8baa00700 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8bb400700 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x560e131b10f0 tx=0x7ff8ac00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f367c26677f1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.752+0000 7ff8b1200700 1 -- 172.21.15.46:0/283127211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac02bd70 con 0x560e1318c0f0 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.753+0000 7ff8b1200700 1 -- 172.21.15.46:0/283127211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8ac005ce0 con 0x560e1318c0f0 2024-05-08T23:10:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.753+0000 7ff8b1200700 1 -- 172.21.15.46:0/283127211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac02b5a0 con 0x560e1318c0f0 2024-05-08T23:10:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/283127211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 msgr2=0x560e1318f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x560e131b10f0 tx=0x7ff8ac00e710 comp rx=0 tx=0).stop 2024-05-08T23:10:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/283127211 shutdown_connections 2024-05-08T23:10:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e13191f80 0x560e13195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e1318c0f0 0x560e1318f070 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/283127211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191660 0x560e13193370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/283127211 >> 172.21.15.46:0/283127211 conn(0x560e130e1210 msgr2=0x560e1317a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/283127211 shutdown_connections 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/283127211 wait complete. 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.754+0000 7ff8d5ba1b80 1 -- start start 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 Processor -- start 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191660 0x560e131857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 0x560e13185d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e13186460 con 0x560e13191660 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e131865d0 con 0x560e13191f80 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e132038c0 con 0x560e1318c0f0 2024-05-08T23:10:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8d5ba1b80 10 client.0 _ll_register_callbacks cb 0x560e13078b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50154/0 (socket says 172.21.15.46:50154) 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 0x560e13185d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8baa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191660 0x560e131857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.755+0000 7ff8bb400700 1 -- 172.21.15.46:0/4104617887 learned_addr learned my addr 172.21.15.46:0/4104617887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 -- 172.21.15.46:0/4104617887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 msgr2=0x560e13185d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 0x560e13185d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 -- 172.21.15.46:0/4104617887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191660 msgr2=0x560e131857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191660 0x560e131857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 -- 172.21.15.46:0/4104617887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac0099e0 con 0x560e1318c0f0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bbe00700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 0x560e13185d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8bb400700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005e90 tx=0x7ff8ac00e470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0eaaa29d922e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:15.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.756+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac005240 con 0x560e1318c0f0 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff8ac0053a0 con 0x560e1318c0f0 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8ac019af0 con 0x560e1318c0f0 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ac005560 con 0x560e1318c0f0 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 10 client.27448 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560e12f543b0 con 0x560e1318c0f0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8ac019af0 con 0x560e1318c0f0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8ac004730 con 0x560e1318c0f0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 10 client.27448 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 10 client.27448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 20 client.27448 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e13204ca0 0x560e132070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 4 client.27448 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.757+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e130cbec0 con 0x560e13204ca0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8baa00700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e13204ca0 0x560e132070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8baa00700 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e13204ca0 0x560e132070c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da4d35649ca78e9c server_cookie=637a6b5d66554e2a in_seq=0 out_seq=0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8a4c00700 10 client.27448 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8a4c00700 1 -- 172.21.15.46:0/4104617887 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e130cbec0 con 0x560e13204ca0 2024-05-08T23:10:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8a4c00700 10 client.27448 handle_command_reply: tid=0 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 client.27448 shutdown 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 msgr2=0x560e13187c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005e90 tx=0x7ff8ac00e470 comp rx=0 tx=0).stop 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 shutdown_connections 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560e13204ca0 0x560e132070c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e13191f80 0x560e13185d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8a4c00700 0 client.27448 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e1318c0f0 0x560e13187c40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 --2- 172.21.15.46:0/4104617887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e13191660 0x560e131857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.758+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 >> 172.21.15.46:0/4104617887 conn(0x560e130e1210 msgr2=0x560e13190900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 shutdown_connections 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8d5ba1b80 1 -- 172.21.15.46:0/4104617887 wait complete. 2024-05-08T23:10:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8d5ba1b80 20 client.27448 trim_cache size 0 max 16384 2024-05-08T23:10:15.764 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:15.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:15 smithi053 ceph-mon[77587]: pgmap v196: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-08T23:10:15.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:15 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2044041369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 msgr2=0x7ff8a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 0x7ff8a00a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc10dd20 tx=0x7ff8a8009260 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 msgr2=0x7ff8bc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.759+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400b070 tx=0x7ff8b40078c0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 shutdown_connections 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8bc10b3c0 0x7ff8bc19de70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8bc101fa0 0x7ff8bc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8a00a1310 0x7ff8a00a37d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 --2- 172.21.15.46:0/1227980222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8bc1028c0 0x7ff8bc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 >> 172.21.15.46:0/1227980222 conn(0x7ff8bc0fb760 msgr2=0x7ff8bc0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 shutdown_connections 2024-05-08T23:10:15.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:15.760+0000 7ff8c2000700 1 -- 172.21.15.46:0/1227980222 wait complete. 2024-05-08T23:10:15.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:10:15.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 509184 10.33 MB/sec execute 157 sec latency 19.148 ms 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 -- 172.21.15.46:0/3078390580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9154101a30 msgr2=0x7f9154105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 --2- 172.21.15.46:0/3078390580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9154101a30 0x7f9154105a70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9140009990 tx=0x7f9140009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 -- 172.21.15.46:0/3078390580 shutdown_connections 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 --2- 172.21.15.46:0/3078390580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91541061a0 0x7f915410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 --2- 172.21.15.46:0/3078390580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9154101080 0x7f9154101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 --2- 172.21.15.46:0/3078390580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9154101a30 0x7f9154105a70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.010+0000 7f915a800700 1 -- 172.21.15.46:0/3078390580 >> 172.21.15.46:0/3078390580 conn(0x7f91540fc8e0 msgr2=0x7f91540fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.011+0000 7f915a800700 1 -- 172.21.15.46:0/3078390580 shutdown_connections 2024-05-08T23:10:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.011+0000 7f915a800700 1 -- 172.21.15.46:0/3078390580 wait complete. 2024-05-08T23:10:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.012+0000 7f915a800700 1 Processor -- start 2024-05-08T23:10:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.012+0000 7f915a800700 1 -- start start 2024-05-08T23:10:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915419deb0 0x7f915419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915419e930 con 0x7f9154101080 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915419eaa0 con 0x7f915419deb0 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f915a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915419ec60 con 0x7f91541061a0 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9153400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50178/0 (socket says 172.21.15.46:50178) 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9153400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48982/0 (socket says 172.21.15.46:48982) 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 -- 172.21.15.46:0/1045828819 learned_addr learned my addr 172.21.15.46:0/1045828819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9153e00700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915419deb0 0x7f915419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 -- 172.21.15.46:0/1045828819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915419deb0 msgr2=0x7f915419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915419deb0 0x7f915419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 -- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 msgr2=0x7f91541995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9152a00700 1 -- 172.21.15.46:0/1045828819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c009c40 con 0x7f91541061a0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.013+0000 7f9153400700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f9152a00700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91400059d0 tx=0x7f9140004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f914be00700 1 -- 172.21.15.46:0/1045828819 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914001d070 con 0x7f91541061a0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400095f0 con 0x7f91541061a0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f914be00700 1 -- 172.21.15.46:0/1045828819 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9140022470 con 0x7f91541061a0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f914be00700 1 -- 172.21.15.46:0/1045828819 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140017650 con 0x7f91541061a0 2024-05-08T23:10:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.014+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91541a6450 con 0x7f91541061a0 2024-05-08T23:10:16.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.017+0000 7f914be00700 1 -- 172.21.15.46:0/1045828819 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f91400225e0 con 0x7f91541061a0 2024-05-08T23:10:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 Processor -- start 2024-05-08T23:10:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 -- start start 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab9989d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f914be00700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 0x7f91380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 0x55ab99897cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab997ee150 con 0x55ab9989d720 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab997ebb10 con 0x55ab9989ce00 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab997e7400 con 0x55ab99897890 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.019+0000 7f914be00700 1 -- 172.21.15.46:0/1045828819 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f91400ce1a0 con 0x7f91541061a0 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9153e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab9989d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9153400700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 0x7f91380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9153e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 0x55ab99897cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48994/0 (socket says 172.21.15.46:48994) 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 -- 172.21.15.46:0/2127686084 learned_addr learned my addr 172.21.15.46:0/2127686084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 -- 172.21.15.46:0/2127686084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 msgr2=0x55ab99897cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 0x55ab99897cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 -- 172.21.15.46:0/2127686084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 msgr2=0x55ab9989d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab9989d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 -- 172.21.15.46:0/2127686084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab996683b0 con 0x55ab9989d720 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9153e00700 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 0x55ab99897cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9152a00700 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f91400cb540 tx=0x7f9140009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c46f5125c25d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9153400700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 0x7f91380a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9144011490 tx=0x7f9144009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.020+0000 7f9148c00700 1 -- 172.21.15.46:0/2127686084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914009aa70 con 0x55ab9989d720 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f9148c00700 1 -- 172.21.15.46:0/2127686084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f914009a890 con 0x55ab9989d720 2024-05-08T23:10:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f9148c00700 1 -- 172.21.15.46:0/2127686084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140098620 con 0x55ab9989d720 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 -- 172.21.15.46:0/2127686084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 msgr2=0x55ab99897350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f91400cb540 tx=0x7f9140009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 -- 172.21.15.46:0/2127686084 shutdown_connections 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab99897890 0x55ab99897cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab9989d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/2127686084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989d720 0x55ab99897350 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 -- 172.21.15.46:0/2127686084 >> 172.21.15.46:0/2127686084 conn(0x55ab997f5210 msgr2=0x55ab99891df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.021+0000 7f916e3eeb80 1 -- 172.21.15.46:0/2127686084 shutdown_connections 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.022+0000 7f916e3eeb80 1 -- 172.21.15.46:0/2127686084 wait complete. 2024-05-08T23:10:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.022+0000 7f916e3eeb80 1 -- start start 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.022+0000 7f916e3eeb80 1 Processor -- start 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab99897890 0x55ab998a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 0x55ab998a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab99783560 con 0x55ab9989ce00 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab99783200 con 0x55ab99897890 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9966a6a0 con 0x55ab9989d720 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f916e3eeb80 10 client.0 _ll_register_callbacks cb 0x55ab9978cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f9153e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f9153e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50202/0 (socket says 172.21.15.46:50202) 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f9153e00700 1 -- 172.21.15.46:0/694981465 learned_addr learned my addr 172.21.15.46:0/694981465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.023+0000 7f9153400700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab99897890 0x55ab998a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 -- 172.21.15.46:0/694981465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab99897890 msgr2=0x55ab998a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab99897890 0x55ab998a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9152a00700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 0x55ab998a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 -- 172.21.15.46:0/694981465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 msgr2=0x55ab998a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 0x55ab998a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 -- 172.21.15.46:0/694981465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c008950 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9152a00700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 0x55ab998a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f9153e00700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f914c003c30 tx=0x7f914c008530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be4a414100f0ea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f913d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00e040 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f914c004830 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.024+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab99740270 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.025+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f914c004bb0 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.025+0000 7f916e3eeb80 10 client.27460 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.025+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f91541a6450 con 0x55ab9989d720 2024-05-08T23:10:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.025+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab99740270 con 0x55ab9989d720 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.025+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f914c01a450 con 0x55ab9989d720 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 10 client.27460 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 10 client.27460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 20 client.27460 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab998b0ed0 0x55ab998b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 4 client.27460 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab99866810 con 0x55ab998b0ed0 2024-05-08T23:10:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f9153400700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab998b0ed0 0x55ab998b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f9153400700 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab998b0ed0 0x55ab998b32f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3605b0ef87306e71 server_cookie=6e36eb3cf641efee in_seq=0 out_seq=0 2024-05-08T23:10:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.026+0000 7f913d600700 10 client.27460 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f913d600700 1 -- 172.21.15.46:0/694981465 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab99866810 con 0x55ab998b0ed0 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f913d600700 10 client.27460 handle_command_reply: tid=0 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 client.27460 shutdown 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 msgr2=0x55ab998a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f914c003c30 tx=0x7f914c008530 comp rx=0 tx=0).stop 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 shutdown_connections 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1560 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab99897890 0x55ab998a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab998b0ed0 0x55ab998b32f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/694981465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab9989ce00 0x55ab998a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f913d600700 0 client.27460 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.028+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 >> 172.21.15.46:0/694981465 conn(0x55ab997f5210 msgr2=0x55ab998915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.029+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 shutdown_connections 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.029+0000 7f916e3eeb80 1 -- 172.21.15.46:0/694981465 wait complete. 2024-05-08T23:10:16.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.029+0000 7f916e3eeb80 20 client.27460 trim_cache size 0 max 16384 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.046+0000 7f916e3eeb80 1 Processor -- start 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.046+0000 7f916e3eeb80 1 -- start start 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab99891d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab998a1f70 con 0x55ab99897890 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab998a3b10 con 0x55ab9989ce00 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9988edc0 con 0x55ab9989d720 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9152a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50214/0 (socket says 172.21.15.46:50214) 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 -- 172.21.15.46:0/531006725 learned_addr learned my addr 172.21.15.46:0/531006725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9152a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35084/0 (socket says 172.21.15.46:35084) 2024-05-08T23:10:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153400700 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab99891d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 -- 172.21.15.46:0/531006725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 msgr2=0x55ab998a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 -- 172.21.15.46:0/531006725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 msgr2=0x55ab99891d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab99891d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9153e00700 1 -- 172.21.15.46:0/531006725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab99614b60 con 0x55ab9989d720 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.047+0000 7f9152a00700 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.048+0000 7f9153e00700 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f914c01a800 tx=0x7f914c008680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12fddcdce5674404 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.048+0000 7f9149600700 1 -- 172.21.15.46:0/531006725 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00e040 con 0x55ab9989d720 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.048+0000 7f9149600700 1 -- 172.21.15.46:0/531006725 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f914c00f040 con 0x55ab9989d720 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.048+0000 7f9149600700 1 -- 172.21.15.46:0/531006725 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01b450 con 0x55ab9989d720 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- 172.21.15.46:0/531006725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 msgr2=0x55ab998a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f914c01a800 tx=0x7f914c008680 comp rx=0 tx=0).stop 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- 172.21.15.46:0/531006725 shutdown_connections 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989d720 0x55ab998a1590 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab99891d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/531006725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989ce00 0x55ab998a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- 172.21.15.46:0/531006725 >> 172.21.15.46:0/531006725 conn(0x55ab997f5210 msgr2=0x55ab9989f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- 172.21.15.46:0/531006725 shutdown_connections 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- 172.21.15.46:0/531006725 wait complete. 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.049+0000 7f916e3eeb80 1 -- start start 2024-05-08T23:10:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 Processor -- start 2024-05-08T23:10:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab998a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 0x55ab99906390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab999068d0 con 0x55ab99897890 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab99906a10 con 0x55ab9989d720 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab99906bd0 con 0x55ab9989ce00 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f916e3eeb80 10 client.0 _ll_register_callbacks cb 0x55ab9978cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f9152a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f9152a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50228/0 (socket says 172.21.15.46:50228) 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f9152a00700 1 -- 172.21.15.46:0/3813392129 learned_addr learned my addr 172.21.15.46:0/3813392129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f9153e00700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 0x55ab99906390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.050+0000 7f9153400700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab998a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 -- 172.21.15.46:0/3813392129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 msgr2=0x55ab99906390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 0x55ab99906390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 -- 172.21.15.46:0/3813392129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 msgr2=0x55ab998a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab998a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 -- 172.21.15.46:0/3813392129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab996683b0 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9153e00700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 0x55ab99906390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f9152a00700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9140097430 tx=0x7f9140099a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc67c5a830606ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f913d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914009aa70 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f914009b630 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.051+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f914009a890 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9140099780 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 10 client.27466 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f91541a6450 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f914009a890 con 0x55ab9989ce00 2024-05-08T23:10:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f91400cb760 con 0x55ab9989ce00 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 10 client.27466 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 10 client.27466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 20 client.27466 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab99907fe0 0x55ab9990a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 4 client.27466 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab997dfec0 con 0x55ab99907fe0 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.052+0000 7f9153400700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab99907fe0 0x55ab9990a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.053+0000 7f9153400700 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab99907fe0 0x55ab9990a400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12f56c13113f4b97 server_cookie=a047ea97a1d807e4 in_seq=0 out_seq=0 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.053+0000 7f913d600700 10 client.27466 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.053+0000 7f913d600700 1 -- 172.21.15.46:0/3813392129 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab997dfec0 con 0x55ab99907fe0 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.053+0000 7f913d600700 10 client.27466 handle_command_reply: tid=0 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.053+0000 7f916e3eeb80 1 client.27466 shutdown 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 msgr2=0x55ab998a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9140097430 tx=0x7f9140099a30 comp rx=0 tx=0).stop 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 shutdown_connections 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab9989d720 0x55ab99906390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ab99907fe0 0x55ab9990a400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab99897890 0x55ab998a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 --2- 172.21.15.46:0/3813392129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9989ce00 0x55ab998a6630 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f913d600700 0 client.27466 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 >> 172.21.15.46:0/3813392129 conn(0x55ab997f5210 msgr2=0x55ab9989bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 shutdown_connections 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 1 -- 172.21.15.46:0/3813392129 wait complete. 2024-05-08T23:10:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.054+0000 7f916e3eeb80 20 client.27466 trim_cache size 0 max 16384 2024-05-08T23:10:16.059 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:16.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 msgr2=0x7f91380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 0x7f91380a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9144011490 tx=0x7f9144009260 comp rx=0 tx=0).stop 2024-05-08T23:10:16.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 msgr2=0x7f9154199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91400059d0 tx=0x7f9140004730 comp rx=0 tx=0).stop 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 shutdown_connections 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915419deb0 0x7f915419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91380a1290 0x7f91380a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9154101080 0x7f91541995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 --2- 172.21.15.46:0/1045828819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91541061a0 0x7f9154199b10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 >> 172.21.15.46:0/1045828819 conn(0x7f91540fc8e0 msgr2=0x7f91540fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 shutdown_connections 2024-05-08T23:10:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.055+0000 7f915a800700 1 -- 172.21.15.46:0/1045828819 wait complete. 2024-05-08T23:10:16.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:10:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.302+0000 7f9dfdc00700 1 -- 172.21.15.46:0/2239150833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df8101080 msgr2=0x7f9df8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.302+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/2239150833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df8101080 0x7f9df8101460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098f0 tx=0x7f9de8009c00 comp rx=0 tx=0).stop 2024-05-08T23:10:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 -- 172.21.15.46:0/2239150833 shutdown_connections 2024-05-08T23:10:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/2239150833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df81061a0 0x7f9df810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/2239150833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df8101080 0x7f9df8101460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/2239150833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df8101a30 0x7f9df8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 -- 172.21.15.46:0/2239150833 >> 172.21.15.46:0/2239150833 conn(0x7f9df80fc8e0 msgr2=0x7f9df80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 -- 172.21.15.46:0/2239150833 shutdown_connections 2024-05-08T23:10:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.303+0000 7f9dfdc00700 1 -- 172.21.15.46:0/2239150833 wait complete. 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 Processor -- start 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 -- start start 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 0x7f9df8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df81061a0 0x7f9df8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df819e980 con 0x7f9df8101080 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df819eaf0 con 0x7f9df81061a0 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.304+0000 7f9dfdc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df819ecb0 con 0x7f9df819df00 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 0x7f9df8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50242/0 (socket says 172.21.15.46:50242) 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 -- 172.21.15.46:0/1053861889 learned_addr learned my addr 172.21.15.46:0/1053861889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df6000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df81061a0 0x7f9df8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 0x7f9df8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49026/0 (socket says 172.21.15.46:49026) 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 -- 172.21.15.46:0/1053861889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df81061a0 msgr2=0x7f9df8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df81061a0 0x7f9df8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 -- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 msgr2=0x7f9df8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 0x7f9df8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 -- 172.21.15.46:0/1053861889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8009520 con 0x7f9df819df00 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.305+0000 7f9df7400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b970 tx=0x7f9df000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.306+0000 7f9def400700 1 -- 172.21.15.46:0/1053861889 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000c870 con 0x7f9df819df00 2024-05-08T23:10:16.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.306+0000 7f9def400700 1 -- 172.21.15.46:0/1053861889 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df000ce90 con 0x7f9df819df00 2024-05-08T23:10:16.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.306+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df81a6210 con 0x7f9df819df00 2024-05-08T23:10:16.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.306+0000 7f9def400700 1 -- 172.21.15.46:0/1053861889 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0007370 con 0x7f9df819df00 2024-05-08T23:10:16.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.306+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df81a6740 con 0x7f9df819df00 2024-05-08T23:10:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.309+0000 7f9def400700 1 -- 172.21.15.46:0/1053861889 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9df000c9d0 con 0x7f9df819df00 2024-05-08T23:10:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.310+0000 7f9e117f1b80 1 Processor -- start 2024-05-08T23:10:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 -- start start 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fa71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa1890 0x5574c3fa1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3ef8150 con 0x5574c3fa6e00 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3ef5b10 con 0x5574c3fa1890 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.311+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3ef1400 con 0x5574c3fa7720 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50252/0 (socket says 172.21.15.46:50252) 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 -- 172.21.15.46:0/427397470 learned_addr learned my addr 172.21.15.46:0/427397470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6000700 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fa71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6000700 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa1890 0x5574c3fa1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9def400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 0x7f9ddc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 -- 172.21.15.46:0/427397470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa1890 msgr2=0x5574c3fa1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa1890 0x5574c3fa1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 -- 172.21.15.46:0/427397470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 msgr2=0x5574c3fa71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fa71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 -- 172.21.15.46:0/427397470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c3d723b0 con 0x5574c3fa7720 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9def400700 1 -- 172.21.15.46:0/1053861889 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9df0006e40 con 0x7f9df819df00 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df7400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 0x7f9ddc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.312+0000 7f9df6a00700 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009230 tx=0x7f9de8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daedec1b889bc381 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.313+0000 7f9de3e00700 1 -- 172.21.15.46:0/427397470 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800fe70 con 0x5574c3fa7720 2024-05-08T23:10:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.313+0000 7f9de3e00700 1 -- 172.21.15.46:0/427397470 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9de801d040 con 0x5574c3fa7720 2024-05-08T23:10:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.313+0000 7f9de3e00700 1 -- 172.21.15.46:0/427397470 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80052a0 con 0x5574c3fa7720 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- 172.21.15.46:0/427397470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 msgr2=0x5574c3fa1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9df7400700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 0x7f9ddc0a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9df00a13f0 tx=0x7f9df0004ad0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009230 tx=0x7f9de8005dc0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- 172.21.15.46:0/427397470 shutdown_connections 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa1890 0x5574c3fa1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fa71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/427397470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa7720 0x5574c3fa1350 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- 172.21.15.46:0/427397470 >> 172.21.15.46:0/427397470 conn(0x5574c3eff210 msgr2=0x5574c3f9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- 172.21.15.46:0/427397470 shutdown_connections 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- 172.21.15.46:0/427397470 wait complete. 2024-05-08T23:10:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.314+0000 7f9e117f1b80 1 -- start start 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 Processor -- start 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3e8d560 con 0x5574c3fa6e00 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3e8d200 con 0x5574c3fa7720 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3d746a0 con 0x5574c3fa1890 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9e117f1b80 10 client.0 _ll_register_callbacks cb 0x5574c3e96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.315+0000 7f9df6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50266/0 (socket says 172.21.15.46:50266) 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 -- 172.21.15.46:0/3250176570 learned_addr learned my addr 172.21.15.46:0/3250176570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6a00700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 -- 172.21.15.46:0/3250176570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 msgr2=0x5574c3fab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 -- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 msgr2=0x5574c3fab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 -- 172.21.15.46:0/3250176570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df810e590 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9df6000700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9de400f460 tx=0x7f9de4015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d656c533dc020961 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9de0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.316+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400fd40 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9de4023430 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574c3e4a270 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de400fea0 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 10 client.27478 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574c3d1eb60 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574c3e4a270 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9de4022500 con 0x5574c3fa1890 2024-05-08T23:10:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 10 client.27478 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 10 client.27478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 20 client.27478 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c3fbb1c0 0x5574c3fbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 4 client.27478 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.317+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574c3f70810 con 0x5574c3fbb1c0 2024-05-08T23:10:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.318+0000 7f9df6a00700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c3fbb1c0 0x5574c3fbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.318+0000 7f9de0c00700 10 client.27478 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:16.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.318+0000 7f9df6a00700 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c3fbb1c0 0x5574c3fbd5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ba6eb6488b528a0 server_cookie=fa214599543f7d94 in_seq=0 out_seq=0 2024-05-08T23:10:16.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9de0c00700 1 -- 172.21.15.46:0/3250176570 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574c3f70810 con 0x5574c3fbb1c0 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9de0c00700 10 client.27478 handle_command_reply: tid=0 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 client.27478 shutdown 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 msgr2=0x5574c3faaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9de400f460 tx=0x7f9de4015040 comp rx=0 tx=0).stop 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 shutdown_connections 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa1890 0x5574c3faaaf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c3fbb1c0 0x5574c3fbd5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/3250176570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa6e00 0x5574c3fab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 >> 172.21.15.46:0/3250176570 conn(0x5574c3eff210 msgr2=0x5574c3f9b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.319+0000 7f9de0c00700 0 client.27478 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.320+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 shutdown_connections 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.320+0000 7f9e117f1b80 1 -- 172.21.15.46:0/3250176570 wait complete. 2024-05-08T23:10:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.320+0000 7f9e117f1b80 20 client.27478 trim_cache size 0 max 16384 2024-05-08T23:10:16.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 Processor -- start 2024-05-08T23:10:16.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 -- start start 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3f9de50 con 0x5574c3fa1890 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3fc27b0 con 0x5574c3fa7720 2024-05-08T23:10:16.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.335+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3f96800 con 0x5574c3fa6e00 2024-05-08T23:10:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50276/0 (socket says 172.21.15.46:50276) 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 -- 172.21.15.46:0/223459450 learned_addr learned my addr 172.21.15.46:0/223459450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6000700 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 -- 172.21.15.46:0/223459450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 msgr2=0x5574c3fab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 -- 172.21.15.46:0/223459450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 msgr2=0x5574c3fbd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fbd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 -- 172.21.15.46:0/223459450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c3eb7560 con 0x5574c3fa6e00 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9df6a00700 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9de80092a0 tx=0x7f9de8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66cc3036b3b384f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.336+0000 7f9decc00700 1 -- 172.21.15.46:0/223459450 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800bb60 con 0x5574c3fa6e00 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9decc00700 1 -- 172.21.15.46:0/223459450 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9de800fc70 con 0x5574c3fa6e00 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9decc00700 1 -- 172.21.15.46:0/223459450 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800f7c0 con 0x5574c3fa6e00 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 -- 172.21.15.46:0/223459450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 msgr2=0x5574c3f9bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9de80092a0 tx=0x7f9de8005dc0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 -- 172.21.15.46:0/223459450 shutdown_connections 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fbd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/223459450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3f9bd20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.337+0000 7f9e117f1b80 1 -- 172.21.15.46:0/223459450 >> 172.21.15.46:0/223459450 conn(0x5574c3eff210 msgr2=0x5574c3f98210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.338+0000 7f9e117f1b80 1 -- 172.21.15.46:0/223459450 shutdown_connections 2024-05-08T23:10:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.338+0000 7f9e117f1b80 1 -- 172.21.15.46:0/223459450 wait complete. 2024-05-08T23:10:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.338+0000 7f9e117f1b80 1 -- start start 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.338+0000 7f9e117f1b80 1 Processor -- start 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fb0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3fb0cf0 con 0x5574c3fa1890 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c3fb0e60 con 0x5574c3fa7720 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c4010680 con 0x5574c3fa6e00 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9e117f1b80 10 client.0 _ll_register_callbacks cb 0x5574c3e96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 -- 172.21.15.46:0/1915352978 learned_addr learned my addr 172.21.15.46:0/1915352978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6000700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fb0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 -- 172.21.15.46:0/1915352978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 msgr2=0x5574c3fb0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fb0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 -- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 msgr2=0x5574c3fb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fb2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6a00700 1 -- 172.21.15.46:0/1915352978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574c3d723b0 con 0x5574c3fa6e00 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.339+0000 7f9df6000700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9df6a00700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005c50 tx=0x7f9de800bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b0ffb47c0353ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9de0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800b710 con 0x5574c3fa6e00 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f9de800f460 con 0x5574c3fa6e00 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9de801d040 con 0x5574c3fa6e00 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de800f620 con 0x5574c3fa6e00 2024-05-08T23:10:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9e117f1b80 10 client.27484 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574c3d76cd0 con 0x5574c3fa6e00 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.340+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9de801d040 con 0x5574c3fa6e00 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9de8032070 con 0x5574c3fa6e00 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 10 client.27484 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 10 client.27484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 20 client.27484 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c40118e0 0x5574c4013cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 4 client.27484 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574c3ee9ec0 con 0x5574c40118e0 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.341+0000 7f9df6000700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c40118e0 0x5574c4013cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9df6000700 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c40118e0 0x5574c4013cd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12c6dee4b18bb076 server_cookie=2a3b9558b2a9793 in_seq=0 out_seq=0 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9de0c00700 10 client.27484 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9de0c00700 1 -- 172.21.15.46:0/1915352978 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574c3ee9ec0 con 0x5574c40118e0 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9de0c00700 10 client.27484 handle_command_reply: tid=0 2024-05-08T23:10:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 client.27484 shutdown 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 msgr2=0x5574c3fb00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005c50 tx=0x7f9de800bc90 comp rx=0 tx=0).stop 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 shutdown_connections 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574c3fa7720 0x5574c3fb0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5574c40118e0 0x5574c4013cd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574c3fa1890 0x5574c3fb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 --2- 172.21.15.46:0/1915352978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c3fa6e00 0x5574c3fb00f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 >> 172.21.15.46:0/1915352978 conn(0x5574c3eff210 msgr2=0x5574c3fa9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9de0c00700 0 client.27484 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 shutdown_connections 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 1 -- 172.21.15.46:0/1915352978 wait complete. 2024-05-08T23:10:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.342+0000 7f9e117f1b80 20 client.27484 trim_cache size 0 max 16384 2024-05-08T23:10:16.350 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 msgr2=0x7f9ddc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 0x7f9ddc0a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9df00a13f0 tx=0x7f9df0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 msgr2=0x7f9df819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b970 tx=0x7f9df000bc80 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 shutdown_connections 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df819df00 0x7f9df819e380 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9ddc0a1310 0x7f9ddc0a37d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df8101080 0x7f9df8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 --2- 172.21.15.46:0/1053861889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9df81061a0 0x7f9df8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 >> 172.21.15.46:0/1053861889 conn(0x7f9df80fc8e0 msgr2=0x7f9df80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 shutdown_connections 2024-05-08T23:10:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.343+0000 7f9dfdc00700 1 -- 172.21.15.46:0/1053861889 wait complete. 2024-05-08T23:10:16.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:10:16.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 -- 172.21.15.46:0/2605684902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee41083d0 msgr2=0x7f1ee410ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 --2- 172.21.15.46:0/2605684902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee41083d0 0x7f1ee410ff50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00b0a0 tx=0x7f1edc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 -- 172.21.15.46:0/2605684902 shutdown_connections 2024-05-08T23:10:16.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 --2- 172.21.15.46:0/2605684902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee41083d0 0x7f1ee410ff50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 --2- 172.21.15.46:0/2605684902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41032b0 0x7f1ee4103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 --2- 172.21.15.46:0/2605684902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee4103c60 0x7f1ee4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.600+0000 7f1eebc00700 1 -- 172.21.15.46:0/2605684902 >> 172.21.15.46:0/2605684902 conn(0x7f1ee40feb10 msgr2=0x7f1ee4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.601+0000 7f1eebc00700 1 -- 172.21.15.46:0/2605684902 shutdown_connections 2024-05-08T23:10:16.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.601+0000 7f1eebc00700 1 -- 172.21.15.46:0/2605684902 wait complete. 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 Processor -- start 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 -- start start 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 0x7f1ee4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 0x7f1ee419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419e820 con 0x7f1ee41032b0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419e990 con 0x7f1ee419dda0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1eebc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee419eb50 con 0x7f1ee41083d0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.602+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 0x7f1ee4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49074/0 (socket says 172.21.15.46:49074) 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee9200700 1 -- 172.21.15.46:0/1747257831 learned_addr learned my addr 172.21.15.46:0/1747257831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 0x7f1ee419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 -- 172.21.15.46:0/1747257831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 msgr2=0x7f1ee419e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 0x7f1ee419e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 -- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 msgr2=0x7f1ee41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 -- 172.21.15.46:0/1747257831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009520 con 0x7f1ee41083d0 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 0x7f1ee419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:16.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 0x7f1ee4199a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e9c0 tx=0x7f1ed000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.603+0000 7f1ee1600700 1 -- 172.21.15.46:0/1747257831 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000cb70 con 0x7f1ee41083d0 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.604+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc009d00 con 0x7f1ee41083d0 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.604+0000 7f1ee1600700 1 -- 172.21.15.46:0/1747257831 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed00044e0 con 0x7f1ee41083d0 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.604+0000 7f1ee1600700 1 -- 172.21.15.46:0/1747257831 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003d20 con 0x7f1ee41083d0 2024-05-08T23:10:16.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.604+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee41a6450 con 0x7f1ee41083d0 2024-05-08T23:10:16.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.607+0000 7f1ee1600700 1 -- 172.21.15.46:0/1747257831 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1ed0003ec0 con 0x7f1ee41083d0 2024-05-08T23:10:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.608+0000 7f1ee1600700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 0x7f1ecc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 Processor -- start 2024-05-08T23:10:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1ee1600700 1 -- 172.21.15.46:0/1747257831 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1ed0014070 con 0x7f1ee41083d0 2024-05-08T23:10:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 -- start start 2024-05-08T23:10:16.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 0x7f1ecc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cc9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1c12150 con 0x557eb1cc9660 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1c0fb10 con 0x557eb1cc9f80 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.609+0000 7f1eff79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1c0b400 con 0x557eb1cc40f0 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cc9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35176/0 (socket says 172.21.15.46:35176) 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 -- 172.21.15.46:0/261957270 learned_addr learned my addr 172.21.15.46:0/261957270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 0x7f1ecc0a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8011590 tx=0x7f1ed8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 -- 172.21.15.46:0/261957270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 msgr2=0x557eb1cc4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cc4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 -- 172.21.15.46:0/261957270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 msgr2=0x557eb1cc9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cc9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee3e00700 1 -- 172.21.15.46:0/261957270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb1a8c3b0 con 0x557eb1cc9f80 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cc4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.610+0000 7f1ee9200700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cc9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00a20c0 tx=0x7f1ed0099d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876665140954ca36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1ed6000700 1 -- 172.21.15.46:0/261957270 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed009a170 con 0x557eb1cc9f80 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1ed6000700 1 -- 172.21.15.46:0/261957270 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed00b1020 con 0x557eb1cc9f80 2024-05-08T23:10:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1ed6000700 1 -- 172.21.15.46:0/261957270 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0004170 con 0x557eb1cc9f80 2024-05-08T23:10:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1eff79db80 1 -- 172.21.15.46:0/261957270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 msgr2=0x557eb1cc3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.611+0000 7f1eff79db80 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00a20c0 tx=0x7f1ed0099d80 comp rx=0 tx=0).stop 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 -- 172.21.15.46:0/261957270 shutdown_connections 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cc4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cc9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 --2- 172.21.15.46:0/261957270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1cc3b60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 -- 172.21.15.46:0/261957270 >> 172.21.15.46:0/261957270 conn(0x557eb1c19210 msgr2=0x557eb1cb5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 -- 172.21.15.46:0/261957270 shutdown_connections 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 -- 172.21.15.46:0/261957270 wait complete. 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.612+0000 7f1eff79db80 1 -- start start 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 Processor -- start 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1ccd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1ba7560 con 0x557eb1cc9660 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1ba7200 con 0x557eb1cc9f80 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1a8e6a0 con 0x557eb1cc40f0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.613+0000 7f1eff79db80 10 client.0 _ll_register_callbacks cb 0x557eb1bb0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50328/0 (socket says 172.21.15.46:50328) 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 -- 172.21.15.46:0/1907976242 learned_addr learned my addr 172.21.15.46:0/1907976242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1ccd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 -- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 msgr2=0x557eb1ccde70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 -- 172.21.15.46:0/1907976242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 msgr2=0x557eb1ccd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1ccd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 -- 172.21.15.46:0/1907976242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed8017070 con 0x557eb1cc40f0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ee9200700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801cc80 tx=0x7f1ed80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13acebe12cb0a7df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.614+0000 7f1ecaa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8022070 con 0x557eb1cc40f0 2024-05-08T23:10:16.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1ed801dbb0 con 0x557eb1cc40f0 2024-05-08T23:10:16.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557eb1b64270 con 0x557eb1cc40f0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ed801dd70 con 0x557eb1cc40f0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1eff79db80 10 client.27496 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.615+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1ee41a6450 con 0x557eb1cc40f0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557eb1b64270 con 0x557eb1cc40f0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ed802e070 con 0x557eb1cc40f0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 10 client.27496 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 10 client.27496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 20 client.27496 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1cdda20 0x557eb1cdfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 4 client.27496 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557eb1c8a810 con 0x557eb1cdda20 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.616+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1cdda20 0x557eb1cdfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.617+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1cdda20 0x557eb1cdfe40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e59f71c0f8c99c server_cookie=387faa6ba91d4479 in_seq=0 out_seq=0 2024-05-08T23:10:16.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.617+0000 7f1ecaa00700 10 client.27496 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1ecaa00700 1 -- 172.21.15.46:0/1907976242 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557eb1c8a810 con 0x557eb1cdda20 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1ecaa00700 10 client.27496 handle_command_reply: tid=0 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 client.27496 shutdown 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 msgr2=0x557eb1ccd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801cc80 tx=0x7f1ed80047c0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 shutdown_connections 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1ccd3f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1cdda20 0x557eb1cdfe40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 --2- 172.21.15.46:0/1907976242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1ccd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1ecaa00700 0 client.27496 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 >> 172.21.15.46:0/1907976242 conn(0x557eb1c19210 msgr2=0x557eb1cb55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 shutdown_connections 2024-05-08T23:10:16.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.618+0000 7f1eff79db80 1 -- 172.21.15.46:0/1907976242 wait complete. 2024-05-08T23:10:16.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.619+0000 7f1eff79db80 20 client.27496 trim_cache size 0 max 16384 2024-05-08T23:10:16.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.634+0000 7f1eff79db80 1 Processor -- start 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.634+0000 7f1eff79db80 1 -- start start 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.634+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cdfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1eff79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1ce5050 con 0x557eb1cc9660 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1eff79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1cd0370 con 0x557eb1cc9f80 2024-05-08T23:10:16.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1eff79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1cc4720 con 0x557eb1cc40f0 2024-05-08T23:10:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cdfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35190/0 (socket says 172.21.15.46:35190) 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9c00700 1 -- 172.21.15.46:0/545481255 learned_addr learned my addr 172.21.15.46:0/545481255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49086/0 (socket says 172.21.15.46:49086) 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 -- 172.21.15.46:0/545481255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 msgr2=0x557eb1ccdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 -- 172.21.15.46:0/545481255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 msgr2=0x557eb1cb5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee9200700 1 -- 172.21.15.46:0/545481255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb1a38b60 con 0x557eb1cc40f0 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.635+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.636+0000 7f1ee9200700 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cdfe10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ed802e320 tx=0x7f1ed8009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00c7251a21b716b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.636+0000 7f1ed6a00700 1 -- 172.21.15.46:0/545481255 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8022070 con 0x557eb1cc40f0 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.636+0000 7f1ed6a00700 1 -- 172.21.15.46:0/545481255 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed802f640 con 0x557eb1cc40f0 2024-05-08T23:10:16.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.636+0000 7f1ed6a00700 1 -- 172.21.15.46:0/545481255 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed802fe80 con 0x557eb1cc40f0 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- 172.21.15.46:0/545481255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 msgr2=0x557eb1cdfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cdfe10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ed802e320 tx=0x7f1ed8009370 comp rx=0 tx=0).stop 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- 172.21.15.46:0/545481255 shutdown_connections 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1ccdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc40f0 0x557eb1cdfe10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 --2- 172.21.15.46:0/545481255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc9660 0x557eb1cb5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- 172.21.15.46:0/545481255 >> 172.21.15.46:0/545481255 conn(0x557eb1c19210 msgr2=0x557eb1cb2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- 172.21.15.46:0/545481255 shutdown_connections 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- 172.21.15.46:0/545481255 wait complete. 2024-05-08T23:10:16.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.637+0000 7f1eff79db80 1 -- start start 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 Processor -- start 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc40f0 0x557eb1cd2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1d32f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1cd37f0 con 0x557eb1cc40f0 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1d33460 con 0x557eb1cc9f80 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1d33620 con 0x557eb1cc9660 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1eff79db80 10 client.0 _ll_register_callbacks cb 0x557eb1bb0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.638+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50340/0 (socket says 172.21.15.46:50340) 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 -- 172.21.15.46:0/3511518415 learned_addr learned my addr 172.21.15.46:0/3511518415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee9200700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc40f0 0x557eb1cd2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1d32f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 -- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 msgr2=0x557eb1d32f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1d32f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 -- 172.21.15.46:0/3511518415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc40f0 msgr2=0x557eb1cd2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc40f0 0x557eb1cd2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 -- 172.21.15.46:0/3511518415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557eb1a8c3b0 con 0x557eb1cc9660 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee9c00700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1d32f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ee3e00700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0099e70 tx=0x7f1ed009e120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a978b36040b0b4c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.639+0000 7f1ecaa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed009a170 con 0x557eb1cc9660 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1ed00b2030 con 0x557eb1cc9660 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ed00b1020 con 0x557eb1cc9660 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ed00cd1b0 con 0x557eb1cc9660 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 10 client.27502 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:16.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1ee41a6450 con 0x557eb1cc9660 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ed00b1020 con 0x557eb1cc9660 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ed00b2d20 con 0x557eb1cc9660 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 10 client.27502 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 10 client.27502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 20 client.27502 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1d34a30 0x557eb1d36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 4 client.27502 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.640+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557eb1c03ec0 con 0x557eb1d34a30 2024-05-08T23:10:16.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.641+0000 7f1ee9200700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1d34a30 0x557eb1d36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.641+0000 7f1ee9200700 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1d34a30 0x557eb1d36e50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=686661541167d19d server_cookie=c9548501728195f0 in_seq=0 out_seq=0 2024-05-08T23:10:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.641+0000 7f1ecaa00700 10 client.27502 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1ecaa00700 1 -- 172.21.15.46:0/3511518415 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557eb1c03ec0 con 0x557eb1d34a30 2024-05-08T23:10:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1ecaa00700 10 client.27502 handle_command_reply: tid=0 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 client.27502 shutdown 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 msgr2=0x557eb1cd3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0099e70 tx=0x7f1ed009e120 comp rx=0 tx=0).stop 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 shutdown_connections 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eb1cc9f80 0x557eb1d32f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557eb1d34a30 0x557eb1d36e50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557eb1cc40f0 0x557eb1cd2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 --2- 172.21.15.46:0/3511518415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb1cc9660 0x557eb1cd3080 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1ecaa00700 0 client.27502 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 >> 172.21.15.46:0/3511518415 conn(0x557eb1c19210 msgr2=0x557eb1ccbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 shutdown_connections 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 1 -- 172.21.15.46:0/3511518415 wait complete. 2024-05-08T23:10:16.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.642+0000 7f1eff79db80 20 client.27502 trim_cache size 0 max 16384 2024-05-08T23:10:16.647 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 msgr2=0x7f1ecc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 0x7f1ecc0a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8011590 tx=0x7f1ed8009260 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 msgr2=0x7f1ee4199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 0x7f1ee4199a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e9c0 tx=0x7f1ed000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 shutdown_connections 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee419dda0 0x7f1ee419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ecc0a1290 0x7f1ecc0a3750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee41032b0 0x7f1ee41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 --2- 172.21.15.46:0/1747257831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41083d0 0x7f1ee4199a00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 >> 172.21.15.46:0/1747257831 conn(0x7f1ee40feb10 msgr2=0x7f1ee4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 shutdown_connections 2024-05-08T23:10:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:16.643+0000 7f1eebc00700 1 -- 172.21.15.46:0/1747257831 wait complete. 2024-05-08T23:10:16.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 511390 10.35 MB/sec execute 158 sec latency 17.649 ms 2024-05-08T23:10:17.664 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:10:17.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:10:17.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:17 smithi086 ceph-mon[78610]: pgmap v197: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:17.726 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:17 smithi046 ceph-mon[36898]: pgmap v197: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:17 smithi053 ceph-mon[77587]: pgmap v197: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:17.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 -- 172.21.15.46:0/300820214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 msgr2=0x7fa01c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 --2- 172.21.15.46:0/300820214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c10dd20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa01400b0a0 tx=0x7fa01400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 -- 172.21.15.46:0/300820214 shutdown_connections 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 --2- 172.21.15.46:0/300820214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c10dd20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 --2- 172.21.15.46:0/300820214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c101080 0x7fa01c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 --2- 172.21.15.46:0/300820214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101a30 0x7fa01c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 -- 172.21.15.46:0/300820214 >> 172.21.15.46:0/300820214 conn(0x7fa01c0fc8e0 msgr2=0x7fa01c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 -- 172.21.15.46:0/300820214 shutdown_connections 2024-05-08T23:10:17.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.888+0000 7fa022a00700 1 -- 172.21.15.46:0/300820214 wait complete. 2024-05-08T23:10:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 Processor -- start 2024-05-08T23:10:17.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 -- start start 2024-05-08T23:10:17.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 512965 10.34 MB/sec execute 159 sec latency 15.494 ms 2024-05-08T23:10:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 0x7fa01c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 0x7fa01c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19e990 con 0x7fa01c19df10 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19eb00 con 0x7fa01c101080 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa022a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19ecc0 con 0x7fa01c1061a0 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.890+0000 7fa01be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 0x7fa01c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa020a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 0x7fa01c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 0x7fa01c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35224/0 (socket says 172.21.15.46:35224) 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01be00700 1 -- 172.21.15.46:0/2520961392 learned_addr learned my addr 172.21.15.46:0/2520961392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 -- 172.21.15.46:0/2520961392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 msgr2=0x7fa01c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 0x7fa01c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 -- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 msgr2=0x7fa01c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 0x7fa01c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 -- 172.21.15.46:0/2520961392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014009d00 con 0x7fa01c1061a0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa020a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 0x7fa01c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa01b400700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c199bc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d720 tx=0x7fa00800da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.891+0000 7fa018c00700 1 -- 172.21.15.46:0/2520961392 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008004a30 con 0x7fa01c1061a0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.892+0000 7fa018c00700 1 -- 172.21.15.46:0/2520961392 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0080054e0 con 0x7fa01c1061a0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.892+0000 7fa018c00700 1 -- 172.21.15.46:0/2520961392 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800fb90 con 0x7fa01c1061a0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.892+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c1a6220 con 0x7fa01c1061a0 2024-05-08T23:10:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.893+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1a66b0 con 0x7fa01c1061a0 2024-05-08T23:10:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.895+0000 7fa018c00700 1 -- 172.21.15.46:0/2520961392 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa008004b90 con 0x7fa01c1061a0 2024-05-08T23:10:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.897+0000 7fa018c00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 0x7fa0000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.897+0000 7fa018c00700 1 -- 172.21.15.46:0/2520961392 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa0080cc790 con 0x7fa01c1061a0 2024-05-08T23:10:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.897+0000 7fa01be00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 0x7fa0000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.897+0000 7fa036465b80 1 Processor -- start 2024-05-08T23:10:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.897+0000 7fa036465b80 1 -- start start 2024-05-08T23:10:17.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb68183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb6818910 0x556eb6818d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb676f1d0 con 0x556eb6818910 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb676cb90 con 0x556eb681e7a0 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa036465b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb6768480 con 0x556eb681de80 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa01be00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 0x7fa0000a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009d20 tx=0x7fa00c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50378/0 (socket says 172.21.15.46:50378) 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa01be00700 1 -- 172.21.15.46:0/2135493805 learned_addr learned my addr 172.21.15.46:0/2135493805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa020a00700 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb6818910 0x556eb6818d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.898+0000 7fa01b400700 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb68183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 -- 172.21.15.46:0/2135493805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 msgr2=0x556eb68183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb68183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 -- 172.21.15.46:0/2135493805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb6818910 msgr2=0x556eb6818d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb6818910 0x556eb6818d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 -- 172.21.15.46:0/2135493805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eb65e9430 con 0x556eb681de80 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa01be00700 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e570 tx=0x7fa00c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d663a2d8f54242c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa011200700 1 -- 172.21.15.46:0/2135493805 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c026430 con 0x556eb681de80 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa011200700 1 -- 172.21.15.46:0/2135493805 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c021070 con 0x556eb681de80 2024-05-08T23:10:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.899+0000 7fa011200700 1 -- 172.21.15.46:0/2135493805 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c025610 con 0x556eb681de80 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 -- 172.21.15.46:0/2135493805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 msgr2=0x556eb681e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e570 tx=0x7fa00c014000 comp rx=0 tx=0).stop 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 -- 172.21.15.46:0/2135493805 shutdown_connections 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb6818910 0x556eb6818d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb681de80 0x556eb681e260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 --2- 172.21.15.46:0/2135493805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb68183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 -- 172.21.15.46:0/2135493805 >> 172.21.15.46:0/2135493805 conn(0x556eb6776290 msgr2=0x556eb6812e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 -- 172.21.15.46:0/2135493805 shutdown_connections 2024-05-08T23:10:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.900+0000 7fa036465b80 1 -- 172.21.15.46:0/2135493805 wait complete. 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.901+0000 7fa036465b80 1 -- start start 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.901+0000 7fa036465b80 1 Processor -- start 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6822060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb68225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb67045e0 con 0x556eb681e7a0 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb6704280 con 0x556eb681de80 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb65eb720 con 0x556eb6818910 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa036465b80 10 client.0 _ll_register_callbacks cb 0x556eb670dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50390/0 (socket says 172.21.15.46:50390) 2024-05-08T23:10:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.902+0000 7fa01be00700 1 -- 172.21.15.46:0/3311227115 learned_addr learned my addr 172.21.15.46:0/3311227115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa020a00700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb68225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01b400700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6822060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 -- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 msgr2=0x556eb6822060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6822060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 -- 172.21.15.46:0/3311227115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 msgr2=0x556eb68225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb68225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 -- 172.21.15.46:0/3311227115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c017070 con 0x556eb6818910 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa020a00700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb68225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa01be00700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00eb60 tx=0x7fa00c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86ffbdade434cce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa005600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.903+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0053d0 con 0x556eb6818910 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556eb66c12f0 con 0x556eb6818910 2024-05-08T23:10:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa00c005530 con 0x556eb6818910 2024-05-08T23:10:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa00c0056f0 con 0x556eb6818910 2024-05-08T23:10:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 10 client.27514 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556eb6595be0 con 0x556eb6818910 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556eb66c12f0 con 0x556eb6818910 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa00c01d8e0 con 0x556eb6818910 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 10 client.27514 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 10 client.27514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 20 client.27514 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6832050 0x556eb6834470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 4 client.27514 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.904+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556eb67e7890 con 0x556eb6832050 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.905+0000 7fa01b400700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6832050 0x556eb6834470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.905+0000 7fa01b400700 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6832050 0x556eb6834470 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1799567c4c032f server_cookie=91caeaf8fae7b811 in_seq=0 out_seq=0 2024-05-08T23:10:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.905+0000 7fa005600700 10 client.27514 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.906+0000 7fa005600700 1 -- 172.21.15.46:0/3311227115 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556eb67e7890 con 0x556eb6832050 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.906+0000 7fa005600700 10 client.27514 handle_command_reply: tid=0 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.906+0000 7fa036465b80 1 client.27514 shutdown 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.906+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 msgr2=0x556eb6821b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.906+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00eb60 tx=0x7fa00c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 shutdown_connections 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6832050 0x556eb6834470 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb68225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6821b20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa005600700 0 client.27514 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 --2- 172.21.15.46:0/3311227115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6822060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 >> 172.21.15.46:0/3311227115 conn(0x556eb6776290 msgr2=0x556eb6812620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 shutdown_connections 2024-05-08T23:10:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 1 -- 172.21.15.46:0/3311227115 wait complete. 2024-05-08T23:10:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.907+0000 7fa036465b80 20 client.27514 trim_cache size 0 max 16384 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 Processor -- start 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 -- start start 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681de80 0x556eb681d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb6812da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb68396c0 con 0x556eb681de80 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb6824ca0 con 0x556eb681e7a0 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa036465b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb680d880 con 0x556eb6818910 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50392/0 (socket says 172.21.15.46:50392) 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa01be00700 1 -- 172.21.15.46:0/3210193426 learned_addr learned my addr 172.21.15.46:0/3210193426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa020a00700 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb6812da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.927+0000 7fa01b400700 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681de80 0x556eb681d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 -- 172.21.15.46:0/3210193426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 msgr2=0x556eb6812da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb6812da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 -- 172.21.15.46:0/3210193426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681de80 msgr2=0x556eb681d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681de80 0x556eb681d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 -- 172.21.15.46:0/3210193426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eb680f560 con 0x556eb6818910 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa01be00700 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa00c01ddb0 tx=0x7fa00c026480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9d9d45b90c6051 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa011c00700 1 -- 172.21.15.46:0/3210193426 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00eb90 con 0x556eb6818910 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa011c00700 1 -- 172.21.15.46:0/3210193426 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c0265d0 con 0x556eb6818910 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.928+0000 7fa011c00700 1 -- 172.21.15.46:0/3210193426 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01d670 con 0x556eb6818910 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 -- 172.21.15.46:0/3210193426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 msgr2=0x556eb68183d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa00c01ddb0 tx=0x7fa00c026480 comp rx=0 tx=0).stop 2024-05-08T23:10:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 -- 172.21.15.46:0/3210193426 shutdown_connections 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681e7a0 0x556eb6812da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb68183d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 --2- 172.21.15.46:0/3210193426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681de80 0x556eb681d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 -- 172.21.15.46:0/3210193426 >> 172.21.15.46:0/3210193426 conn(0x556eb6776290 msgr2=0x556eb68202c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 -- 172.21.15.46:0/3210193426 shutdown_connections 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.929+0000 7fa036465b80 1 -- 172.21.15.46:0/3210193426 wait complete. 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 -- start start 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 Processor -- start 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6828140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb6828680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb6888560 con 0x556eb681e7a0 2024-05-08T23:10:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb68886a0 con 0x556eb681de80 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb68887e0 con 0x556eb6818910 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.930+0000 7fa036465b80 10 client.0 _ll_register_callbacks cb 0x556eb670dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50408/0 (socket says 172.21.15.46:50408) 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 -- 172.21.15.46:0/1490979841 learned_addr learned my addr 172.21.15.46:0/1490979841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01b400700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6828140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa020a00700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb6828680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 -- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 msgr2=0x556eb6828140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6828140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 -- 172.21.15.46:0/1490979841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 msgr2=0x556eb6828680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb6828680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 -- 172.21.15.46:0/1490979841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eb6595be0 con 0x556eb6818910 2024-05-08T23:10:17.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa01be00700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009be0 tx=0x7fa00c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d22f1be2dcf70a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.931+0000 7fa005600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c005700 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fa00c0053d0 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa00c021070 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa00c00eb60 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 10 client.27520 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556eb6818fc0 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c021070 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa00c048030 con 0x556eb6818910 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 10 client.27520 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 10 client.27520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.932+0000 7fa036465b80 20 client.27520 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6889aa0 0x556eb688be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa036465b80 4 client.27520 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:10:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556eb6760f40 con 0x556eb6889aa0 2024-05-08T23:10:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa01b400700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6889aa0 0x556eb688be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa01b400700 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6889aa0 0x556eb688be90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0b4178c7c247d5 server_cookie=38bbb4207372c5e in_seq=0 out_seq=0 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.933+0000 7fa005600700 10 client.27520 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.934+0000 7fa005600700 1 -- 172.21.15.46:0/1490979841 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556eb6760f40 con 0x556eb6889aa0 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.934+0000 7fa005600700 10 client.27520 handle_command_reply: tid=0 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.934+0000 7fa036465b80 1 client.27520 shutdown 2024-05-08T23:10:17.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 msgr2=0x556eb6827c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009be0 tx=0x7fa00c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 shutdown_connections 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556eb6889aa0 0x556eb688be90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556eb681e7a0 0x556eb6828680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa005600700 0 client.27520 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eb6818910 0x556eb6827c00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 --2- 172.21.15.46:0/1490979841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eb681de80 0x556eb6828140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 >> 172.21.15.46:0/1490979841 conn(0x556eb6776290 msgr2=0x556eb6820b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 shutdown_connections 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 1 -- 172.21.15.46:0/1490979841 wait complete. 2024-05-08T23:10:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.935+0000 7fa036465b80 20 client.27520 trim_cache size 0 max 16384 2024-05-08T23:10:17.940 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:10:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "50608b99-a534-4887-a6eb-3357173d91b0", 2024-05-08T23:10:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:10:17.940 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 msgr2=0x7fa0000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 0x7fa0000a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009d20 tx=0x7fa00c009260 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 msgr2=0x7fa01c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c199bc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d720 tx=0x7fa00800da30 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 shutdown_connections 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa0000a1290 0x7fa0000a3750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa01c19df10 0x7fa01c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa01c101080 0x7fa01c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 --2- 172.21.15.46:0/2520961392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa01c1061a0 0x7fa01c199bc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 >> 172.21.15.46:0/2520961392 conn(0x7fa01c0fc8e0 msgr2=0x7fa01c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 shutdown_connections 2024-05-08T23:10:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:17.936+0000 7fa022a00700 1 -- 172.21.15.46:0/2520961392 wait complete. 2024-05-08T23:10:17.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 -- 172.21.15.46:0/1382351445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c103c60 msgr2=0x7f176c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 --2- 172.21.15.46:0/1382351445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c103c60 0x7f176c107ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1758009990 tx=0x7f1758009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 -- 172.21.15.46:0/1382351445 shutdown_connections 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 --2- 172.21.15.46:0/1382351445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1083d0 0x7f176c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 --2- 172.21.15.46:0/1382351445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1032b0 0x7f176c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 --2- 172.21.15.46:0/1382351445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c103c60 0x7f176c107ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 -- 172.21.15.46:0/1382351445 >> 172.21.15.46:0/1382351445 conn(0x7f176c0feb10 msgr2=0x7f176c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 -- 172.21.15.46:0/1382351445 shutdown_connections 2024-05-08T23:10:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.188+0000 7f1773600700 1 -- 172.21.15.46:0/1382351445 wait complete. 2024-05-08T23:10:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.190+0000 7f1773600700 1 Processor -- start 2024-05-08T23:10:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.190+0000 7f1773600700 1 -- start start 2024-05-08T23:10:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1083d0 0x7f176c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c19dea0 0x7f176c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19e920 con 0x7f176c19dea0 2024-05-08T23:10:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19ea90 con 0x7f176c1083d0 2024-05-08T23:10:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1773600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19ec50 con 0x7f176c1032b0 2024-05-08T23:10:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50424/0 (socket says 172.21.15.46:50424) 2024-05-08T23:10:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 -- 172.21.15.46:0/2497793136 learned_addr learned my addr 172.21.15.46:0/2497793136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1771600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c19dea0 0x7f176c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f176be00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1083d0 0x7f176c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 -- 172.21.15.46:0/2497793136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1083d0 msgr2=0x7f176c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1083d0 0x7f176c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 -- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c19dea0 msgr2=0x7f176c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c19dea0 0x7f176c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.191+0000 7f1770c00700 1 -- 172.21.15.46:0/2497793136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580095f0 con 0x7f176c1032b0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1770c00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9a0 tx=0x7f176000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1769600700 1 -- 172.21.15.46:0/2497793136 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760009790 con 0x7f176c1032b0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c075440 con 0x7f176c1032b0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1769600700 1 -- 172.21.15.46:0/2497793136 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1760010430 con 0x7f176c1032b0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1769600700 1 -- 172.21.15.46:0/2497793136 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760018680 con 0x7f176c1032b0 2024-05-08T23:10:18.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.192+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c075950 con 0x7f176c1032b0 2024-05-08T23:10:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.195+0000 7f1769600700 1 -- 172.21.15.46:0/2497793136 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f17600105a0 con 0x7f176c1032b0 2024-05-08T23:10:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.196+0000 7f1769600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 0x7f17540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.197+0000 7f1769600700 1 -- 172.21.15.46:0/2497793136 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1760014070 con 0x7f176c1032b0 2024-05-08T23:10:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.197+0000 7f1787091b80 1 Processor -- start 2024-05-08T23:10:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.197+0000 7f176be00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 0x7f17540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.197+0000 7f1787091b80 1 -- start start 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee1d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee128150 con 0x55a5ee1d6e00 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee125b10 con 0x55a5ee1d7720 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1787091b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee121400 con 0x55a5ee1d1890 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f176be00700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 0x7f17540a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f176c19ab10 tx=0x7f1758024040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1770c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f176be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee1d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1770c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49180/0 (socket says 172.21.15.46:49180) 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1770c00700 1 -- 172.21.15.46:0/2341792786 learned_addr learned my addr 172.21.15.46:0/2341792786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 -- 172.21.15.46:0/2341792786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 msgr2=0x55a5ee1d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee1d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 -- 172.21.15.46:0/2341792786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 msgr2=0x55a5ee1d71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.198+0000 7f1771600700 1 -- 172.21.15.46:0/2341792786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5edfa23b0 con 0x55a5ee1d1890 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.199+0000 7f1771600700 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1cd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1764011580 tx=0x7f1764009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f373e462af2dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.199+0000 7f175dc00700 1 -- 172.21.15.46:0/2341792786 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400e040 con 0x55a5ee1d1890 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.199+0000 7f175dc00700 1 -- 172.21.15.46:0/2341792786 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176400f040 con 0x55a5ee1d1890 2024-05-08T23:10:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.199+0000 7f175dc00700 1 -- 172.21.15.46:0/2341792786 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764004490 con 0x55a5ee1d1890 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 -- 172.21.15.46:0/2341792786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 msgr2=0x55a5ee1d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1cd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1764011580 tx=0x7f1764009920 comp rx=0 tx=0).stop 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 -- 172.21.15.46:0/2341792786 shutdown_connections 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1cd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 --2- 172.21.15.46:0/2341792786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee1d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 -- 172.21.15.46:0/2341792786 >> 172.21.15.46:0/2341792786 conn(0x55a5ee12f210 msgr2=0x55a5ee1cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 -- 172.21.15.46:0/2341792786 shutdown_connections 2024-05-08T23:10:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.200+0000 7f1787091b80 1 -- 172.21.15.46:0/2341792786 wait complete. 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.201+0000 7f1787091b80 1 -- start start 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.201+0000 7f1787091b80 1 Processor -- start 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1db070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee0bd560 con 0x55a5ee1d7720 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee0bd200 con 0x55a5ee1d6e00 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5edfa46a0 con 0x55a5ee1d1890 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1787091b80 10 client.0 _ll_register_callbacks cb 0x55a5ee0c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50456/0 (socket says 172.21.15.46:50456) 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 -- 172.21.15.46:0/623408592 learned_addr learned my addr 172.21.15.46:0/623408592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1771600700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f176be00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 -- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 msgr2=0x55a5ee1db070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1db070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 -- 172.21.15.46:0/623408592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 msgr2=0x55a5ee1db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.202+0000 7f1770c00700 1 -- 172.21.15.46:0/623408592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1764011230 con 0x55a5ee1d1890 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1770c00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f176000c2d0 tx=0x7f176009a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74fd9c6598956b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1752000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600996b0 con 0x55a5ee1d1890 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f176009ddf0 con 0x55a5ee1d1890 2024-05-08T23:10:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5ee07a270 con 0x55a5ee1d1890 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17600a2440 con 0x55a5ee1d1890 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1787091b80 10 client.27532 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a5edf4eb60 con 0x55a5ee1d1890 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.203+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5ee07a270 con 0x55a5ee1d1890 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17600a1fb0 con 0x55a5ee1d1890 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 10 client.27532 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 20 client.27532 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee1eb200 0x55a5ee1ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 4 client.27532 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5ee1a0810 con 0x55a5ee1eb200 2024-05-08T23:10:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.204+0000 7f176be00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee1eb200 0x55a5ee1ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.205+0000 7f1752000700 10 client.27532 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.205+0000 7f176be00700 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee1eb200 0x55a5ee1ed620 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73d25dc620fe273 server_cookie=f279cc586aca25f0 in_seq=0 out_seq=0 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.205+0000 7f1752000700 1 -- 172.21.15.46:0/623408592 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5ee1a0810 con 0x55a5ee1eb200 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.205+0000 7f1752000700 10 client.27532 handle_command_reply: tid=0 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 client.27532 shutdown 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 msgr2=0x55a5ee1dab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f176000c2d0 tx=0x7f176009a740 comp rx=0 tx=0).stop 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 shutdown_connections 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee1eb200 0x55a5ee1ed620 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1752000700 0 client.27532 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1dab30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 --2- 172.21.15.46:0/623408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 >> 172.21.15.46:0/623408592 conn(0x55a5ee12f210 msgr2=0x55a5ee1cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 shutdown_connections 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 1 -- 172.21.15.46:0/623408592 wait complete. 2024-05-08T23:10:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.206+0000 7f1787091b80 20 client.27532 trim_cache size 0 max 16384 2024-05-08T23:10:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.226+0000 7f1787091b80 1 Processor -- start 2024-05-08T23:10:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 -- start start 2024-05-08T23:10:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee1d8b10 con 0x55a5ee1d7720 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee1c8dc0 con 0x55a5ee1d6e00 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1787091b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee1f2830 con 0x55a5ee1d1890 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50458/0 (socket says 172.21.15.46:50458) 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 -- 172.21.15.46:0/1890019300 learned_addr learned my addr 172.21.15.46:0/1890019300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f176be00700 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1771600700 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 -- 172.21.15.46:0/1890019300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 msgr2=0x55a5ee1d64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 -- 172.21.15.46:0/1890019300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 msgr2=0x55a5ee1cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.227+0000 7f1770c00700 1 -- 172.21.15.46:0/1890019300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5ee0e7560 con 0x55a5ee1d1890 2024-05-08T23:10:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.228+0000 7f1770c00700 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f17600cc010 tx=0x7f1760010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c89576ba787904 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.228+0000 7f175e600700 1 -- 172.21.15.46:0/1890019300 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760063070 con 0x55a5ee1d1890 2024-05-08T23:10:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.228+0000 7f175e600700 1 -- 172.21.15.46:0/1890019300 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17600ade90 con 0x55a5ee1d1890 2024-05-08T23:10:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.228+0000 7f175e600700 1 -- 172.21.15.46:0/1890019300 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600a1fb0 con 0x55a5ee1d1890 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- 172.21.15.46:0/1890019300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 msgr2=0x55a5ee1d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f17600cc010 tx=0x7f1760010a90 comp rx=0 tx=0).stop 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- 172.21.15.46:0/1890019300 shutdown_connections 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d7720 0x55a5ee1cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d1890 0x55a5ee1d1350 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 --2- 172.21.15.46:0/1890019300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- 172.21.15.46:0/1890019300 >> 172.21.15.46:0/1890019300 conn(0x55a5ee12f210 msgr2=0x55a5ee1d92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- 172.21.15.46:0/1890019300 shutdown_connections 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- 172.21.15.46:0/1890019300 wait complete. 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.229+0000 7f1787091b80 1 -- start start 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 Processor -- start 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d1890 0x55a5ee1e0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee240700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee1e0fa0 con 0x55a5ee1d1890 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee240c40 con 0x55a5ee1d7720 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.230+0000 7f1787091b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5ee240d80 con 0x55a5ee1d6e00 2024-05-08T23:10:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f1787091b80 10 client.0 _ll_register_callbacks cb 0x55a5ee0c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50464/0 (socket says 172.21.15.46:50464) 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 -- 172.21.15.46:0/1712084205 learned_addr learned my addr 172.21.15.46:0/1712084205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f1770c00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d1890 0x55a5ee1e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f1771600700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee240700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 -- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 msgr2=0x55a5ee240700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee240700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 -- 172.21.15.46:0/1712084205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d1890 msgr2=0x55a5ee1e0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d1890 0x55a5ee1e0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.231+0000 7f176be00700 1 -- 172.21.15.46:0/1712084205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5edf4eb60 con 0x55a5ee1d6e00 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f176be00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f176c068d20 tx=0x7f1758009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128d93104f2e48f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1752000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758034040 con 0x55a5ee1d6e00 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f176400f040 con 0x55a5ee1d6e00 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f175803d450 con 0x55a5ee1d6e00 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1758027040 con 0x55a5ee1d6e00 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1787091b80 10 client.27538 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a5edfa6cd0 con 0x55a5ee1d6e00 2024-05-08T23:10:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.232+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176400f040 con 0x55a5ee1d6e00 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.233+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1758043020 con 0x55a5ee1d6e00 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.233+0000 7f1787091b80 10 client.27538 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.233+0000 7f1787091b80 10 client.27538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.233+0000 7f1787091b80 20 client.27538 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.233+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee242390 0x55a5ee2447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.234+0000 7f1787091b80 4 client.27538 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.234+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a5ee119ec0 con 0x55a5ee242390 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.234+0000 7f1770c00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee242390 0x55a5ee2447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:18.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1770c00700 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee242390 0x55a5ee2447b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6c9d1b9fcb816d server_cookie=e35ed037bdf851e7 in_seq=0 out_seq=0 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1752000700 10 client.27538 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1752000700 1 -- 172.21.15.46:0/1712084205 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a5ee119ec0 con 0x55a5ee242390 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1752000700 10 client.27538 handle_command_reply: tid=0 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 client.27538 shutdown 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 msgr2=0x55a5ee1e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f176c068d20 tx=0x7f1758009310 comp rx=0 tx=0).stop 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 shutdown_connections 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5ee1d7720 0x55a5ee240700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a5ee242390 0x55a5ee2447b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee1d1890 0x55a5ee1e0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 --2- 172.21.15.46:0/1712084205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5ee1d6e00 0x55a5ee1e0690 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1752000700 0 client.27538 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 >> 172.21.15.46:0/1712084205 conn(0x55a5ee12f210 msgr2=0x55a5ee1d9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.235+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 shutdown_connections 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.236+0000 7f1787091b80 1 -- 172.21.15.46:0/1712084205 wait complete. 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.236+0000 7f1787091b80 20 client.27538 trim_cache size 0 max 16384 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:10:18.240 INFO:teuthology.orchestra.run.smithi046.stdout: "50608b99-a534-4887-a6eb-3357173d91b0": { 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "50608b99-a534-4887-a6eb-3357173d91b0", 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:10:18.241 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 msgr2=0x7f17540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 0x7f17540a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f176c19ab10 tx=0x7f1758024040 comp rx=0 tx=0).stop 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 msgr2=0x7f176c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9a0 tx=0x7f176000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 shutdown_connections 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17540a1290 0x7f17540a3750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c19dea0 0x7f176c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f176c1032b0 0x7f176c199610 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 --2- 172.21.15.46:0/2497793136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c1083d0 0x7f176c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.237+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 >> 172.21.15.46:0/2497793136 conn(0x7f176c0feb10 msgr2=0x7f176c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.238+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 shutdown_connections 2024-05-08T23:10:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:18.238+0000 7f1773600700 1 -- 172.21.15.46:0/2497793136 wait complete. 2024-05-08T23:10:18.257 INFO:tasks.cephfs.filesystem:scrub status for tag:50608b99-a534-4887-a6eb-3357173d91b0 - {'path': '/', 'tag': '50608b99-a534-4887-a6eb-3357173d91b0', 'options': 'recursive,force'} 2024-05-08T23:10:18.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:18 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:10:18.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:18 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:18 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:18.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:18 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:18 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:10:18.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:18 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:18 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:18.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:18 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:18 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:10:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:18 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:18 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:18.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:18 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:18.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 515123 10.36 MB/sec execute 160 sec latency 25.517 ms 2024-05-08T23:10:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:19 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:19 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:19 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:19 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:19 smithi086 ceph-mon[78610]: pgmap v198: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:19 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:19 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:19 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:19 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:19 smithi053 ceph-mon[77587]: pgmap v198: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:19.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:19 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:19 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:19 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:19 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:19 smithi046 ceph-mon[36898]: pgmap v198: 129 pgs: 129 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-05-08T23:10:19.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 516612 10.33 MB/sec execute 161 sec latency 17.295 ms 2024-05-08T23:10:20.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 518778 10.35 MB/sec execute 162 sec latency 22.736 ms 2024-05-08T23:10:21.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:21 smithi086 ceph-mon[78610]: pgmap v199: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-05-08T23:10:21.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:21 smithi053 ceph-mon[77587]: pgmap v199: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-05-08T23:10:21.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:21 smithi046 ceph-mon[36898]: pgmap v199: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-05-08T23:10:21.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 520332 10.32 MB/sec execute 163 sec latency 14.545 ms 2024-05-08T23:10:22.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 522486 10.34 MB/sec execute 164 sec latency 15.525 ms 2024-05-08T23:10:23.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:23 smithi086 ceph-mon[78610]: pgmap v200: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:10:23.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:23 smithi053 ceph-mon[77587]: pgmap v200: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:10:23.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:23 smithi046 ceph-mon[36898]: pgmap v200: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:10:23.890 INFO:tasks.workunit.client.0.smithi046.stdout: 1 524040 10.32 MB/sec execute 165 sec latency 15.441 ms 2024-05-08T23:10:24.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 526132 10.34 MB/sec execute 166 sec latency 17.613 ms 2024-05-08T23:10:25.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:25 smithi086 ceph-mon[78610]: pgmap v201: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-08T23:10:25.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:25 smithi053 ceph-mon[77587]: pgmap v201: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-08T23:10:25.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:25 smithi046 ceph-mon[36898]: pgmap v201: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-08T23:10:25.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 527824 10.31 MB/sec execute 167 sec latency 18.261 ms 2024-05-08T23:10:26.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 529424 10.32 MB/sec execute 168 sec latency 170.948 ms 2024-05-08T23:10:27.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:27 smithi086 ceph-mon[78610]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:27.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:27 smithi053 ceph-mon[77587]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:27.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:27 smithi046 ceph-mon[36898]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:27.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 529973 10.27 MB/sec execute 169 sec latency 207.592 ms 2024-05-08T23:10:28.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 531911 10.25 MB/sec execute 170 sec latency 15.362 ms 2024-05-08T23:10:29.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:29 smithi086 ceph-mon[78610]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:29.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:29 smithi053 ceph-mon[77587]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:29.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:29 smithi046 ceph-mon[36898]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:29.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 533569 10.26 MB/sec execute 171 sec latency 15.672 ms 2024-05-08T23:10:30.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:10:30.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:10:30.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:10:30.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 535766 10.26 MB/sec execute 172 sec latency 16.851 ms 2024-05-08T23:10:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:31 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:10:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:31 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:10:31.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:31 smithi086 ceph-mon[78610]: pgmap v204: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 325 op/s 2024-05-08T23:10:31.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:31 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:10:31.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:31 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:10:31.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:31 smithi053 ceph-mon[77587]: pgmap v204: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 325 op/s 2024-05-08T23:10:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:31 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:10:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:31 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:10:31.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:31 smithi046 ceph-mon[36898]: pgmap v204: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 325 op/s 2024-05-08T23:10:31.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 537107 10.25 MB/sec execute 173 sec latency 77.870 ms 2024-05-08T23:10:32.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 538909 10.24 MB/sec execute 174 sec latency 35.001 ms 2024-05-08T23:10:33.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:33 smithi086 ceph-mon[78610]: pgmap v205: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:10:33.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:33 smithi053 ceph-mon[77587]: pgmap v205: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:10:33.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:33 smithi046 ceph-mon[36898]: pgmap v205: 129 pgs: 129 active+clean; 346 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:10:33.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 540575 10.24 MB/sec execute 175 sec latency 14.757 ms 2024-05-08T23:10:34.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 542753 10.25 MB/sec execute 176 sec latency 18.467 ms 2024-05-08T23:10:35.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:35 smithi086 ceph-mon[78610]: pgmap v206: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-05-08T23:10:35.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:35 smithi053 ceph-mon[77587]: pgmap v206: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-05-08T23:10:35.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:35 smithi046 ceph-mon[36898]: pgmap v206: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-05-08T23:10:35.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 544347 10.24 MB/sec execute 177 sec latency 14.155 ms 2024-05-08T23:10:36.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 546531 10.25 MB/sec execute 178 sec latency 23.255 ms 2024-05-08T23:10:37.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:37 smithi086 ceph-mon[78610]: pgmap v207: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:37.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:37 smithi053 ceph-mon[77587]: pgmap v207: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:37.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:37 smithi046 ceph-mon[36898]: pgmap v207: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:37.891 INFO:tasks.workunit.client.0.smithi046.stdout: 1 548039 10.23 MB/sec execute 179 sec latency 18.006 ms 2024-05-08T23:10:38.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 550236 10.24 MB/sec execute 180 sec latency 18.022 ms 2024-05-08T23:10:39.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:39 smithi086 ceph-mon[78610]: pgmap v208: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:39.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:39 smithi053 ceph-mon[77587]: pgmap v208: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:39.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:39 smithi046 ceph-mon[36898]: pgmap v208: 129 pgs: 129 active+clean; 352 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-05-08T23:10:39.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 551652 10.23 MB/sec execute 181 sec latency 16.268 ms 2024-05-08T23:10:40.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 553780 10.24 MB/sec execute 182 sec latency 17.966 ms 2024-05-08T23:10:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:41 smithi086 ceph-mon[78610]: pgmap v209: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 350 op/s 2024-05-08T23:10:41.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:41 smithi053 ceph-mon[77587]: pgmap v209: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 350 op/s 2024-05-08T23:10:41.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:41 smithi046 ceph-mon[36898]: pgmap v209: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 350 op/s 2024-05-08T23:10:41.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 555150 10.22 MB/sec execute 183 sec latency 23.643 ms 2024-05-08T23:10:42.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 557348 10.23 MB/sec execute 184 sec latency 25.814 ms 2024-05-08T23:10:43.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:43 smithi086 ceph-mon[78610]: pgmap v210: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-08T23:10:43.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:43 smithi053 ceph-mon[77587]: pgmap v210: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-08T23:10:43.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:43 smithi046 ceph-mon[36898]: pgmap v210: 129 pgs: 129 active+clean; 360 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-08T23:10:43.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 558872 10.22 MB/sec execute 185 sec latency 15.820 ms 2024-05-08T23:10:44.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 561073 10.23 MB/sec execute 186 sec latency 16.037 ms 2024-05-08T23:10:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:45 smithi086 ceph-mon[78610]: pgmap v211: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-08T23:10:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:45 smithi053 ceph-mon[77587]: pgmap v211: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-08T23:10:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:45 smithi046 ceph-mon[36898]: pgmap v211: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-08T23:10:45.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 562679 10.23 MB/sec execute 187 sec latency 20.464 ms 2024-05-08T23:10:46.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 564917 10.24 MB/sec execute 188 sec latency 15.400 ms 2024-05-08T23:10:47.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:47 smithi086 ceph-mon[78610]: pgmap v212: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:47.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:47 smithi053 ceph-mon[77587]: pgmap v212: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:47.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:47 smithi046 ceph-mon[36898]: pgmap v212: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:47.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 566313 10.22 MB/sec execute 189 sec latency 15.918 ms 2024-05-08T23:10:48.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:10:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.477+0000 7f5803600700 1 -- 172.21.15.46:0/1249425759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc101fa0 msgr2=0x7f57fc102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.477+0000 7f5803600700 1 --2- 172.21.15.46:0/1249425759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc101fa0 0x7f57fc102380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009990 tx=0x7f57f0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 -- 172.21.15.46:0/1249425759 shutdown_connections 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 --2- 172.21.15.46:0/1249425759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc10b3c0 0x7f57fc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 --2- 172.21.15.46:0/1249425759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc101fa0 0x7f57fc102380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 --2- 172.21.15.46:0/1249425759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 0x7f57fc10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 -- 172.21.15.46:0/1249425759 >> 172.21.15.46:0/1249425759 conn(0x7f57fc0fb760 msgr2=0x7f57fc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 -- 172.21.15.46:0/1249425759 shutdown_connections 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.478+0000 7f5803600700 1 -- 172.21.15.46:0/1249425759 wait complete. 2024-05-08T23:10:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.479+0000 7f5803600700 1 Processor -- start 2024-05-08T23:10:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.479+0000 7f5803600700 1 -- start start 2024-05-08T23:10:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 0x7f57fc199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc10b3c0 0x7f57fc19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19e410 con 0x7f57fc1028c0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19e580 con 0x7f57fc10b3c0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5803600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19e740 con 0x7f57fc101fa0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f57fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 0x7f57fc199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44280/0 (socket says 172.21.15.46:44280) 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 -- 172.21.15.46:0/3027828094 learned_addr learned my addr 172.21.15.46:0/3027828094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5801600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc10b3c0 0x7f57fc19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 -- 172.21.15.46:0/3027828094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc10b3c0 msgr2=0x7f57fc19ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc10b3c0 0x7f57fc19ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 -- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 msgr2=0x7f57fc199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 0x7f57fc199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.480+0000 7f5800c00700 1 -- 172.21.15.46:0/3027828094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8009520 con 0x7f57fc101fa0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f5800c00700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f57f0000c00 tx=0x7f57f0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f57f9600700 1 -- 172.21.15.46:0/3027828094 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f001d070 con 0x7f57fc101fa0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f57f9600700 1 -- 172.21.15.46:0/3027828094 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f0022470 con 0x7f57fc101fa0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f4009c40 con 0x7f57fc101fa0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f57f9600700 1 -- 172.21.15.46:0/3027828094 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017650 con 0x7f57fc101fa0 2024-05-08T23:10:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.481+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00095f0 con 0x7f57fc101fa0 2024-05-08T23:10:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.484+0000 7f57f9600700 1 -- 172.21.15.46:0/3027828094 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57f00225e0 con 0x7f57fc101fa0 2024-05-08T23:10:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f58170cdb80 1 Processor -- start 2024-05-08T23:10:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f57f9600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 0x7f57e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f58170cdb80 1 -- start start 2024-05-08T23:10:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f57f9600700 1 -- 172.21.15.46:0/3027828094 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f57f00ce330 con 0x7f57fc101fa0 2024-05-08T23:10:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f5801600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 0x7f57e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.486+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5458720 0x5574e5452350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5452cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f58170cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e53a9150 con 0x5574e5458720 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f58170cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e53a6b10 con 0x5574e5457e00 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f58170cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e53a2400 con 0x5574e5452890 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33972/0 (socket says 172.21.15.46:33972) 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 -- 172.21.15.46:0/4080692212 learned_addr learned my addr 172.21.15.46:0/4080692212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f57fbe00700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5458720 0x5574e5452350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5801600700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5452cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5801600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 0x7f57e40a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009fd0 tx=0x7f57f400b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 -- 172.21.15.46:0/4080692212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 msgr2=0x5574e5452cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5452cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 -- 172.21.15.46:0/4080692212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5458720 msgr2=0x5574e5452350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5458720 0x5574e5452350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5800c00700 1 -- 172.21.15.46:0/4080692212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e52233b0 con 0x5574e5457e00 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.487+0000 7f5801600700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5452cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.488+0000 7f5800c00700 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f57f0097c50 tx=0x7f57f009afb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302f02f4fc1aee8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.488+0000 7f57edc00700 1 -- 172.21.15.46:0/4080692212 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f009b8e0 con 0x5574e5457e00 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.488+0000 7f57edc00700 1 -- 172.21.15.46:0/4080692212 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f009b1e0 con 0x5574e5457e00 2024-05-08T23:10:48.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.488+0000 7f57edc00700 1 -- 172.21.15.46:0/4080692212 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00ce020 con 0x5574e5457e00 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 -- 172.21.15.46:0/4080692212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 msgr2=0x5574e54581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f57f0097c50 tx=0x7f57f009afb0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 -- 172.21.15.46:0/4080692212 shutdown_connections 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5452cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5457e00 0x5574e54581e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 --2- 172.21.15.46:0/4080692212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5458720 0x5574e5452350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 -- 172.21.15.46:0/4080692212 >> 172.21.15.46:0/4080692212 conn(0x5574e53b0210 msgr2=0x5574e544cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 -- 172.21.15.46:0/4080692212 shutdown_connections 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.489+0000 7f58170cdb80 1 -- 172.21.15.46:0/4080692212 wait complete. 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 -- start start 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 Processor -- start 2024-05-08T23:10:48.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 0x5574e545bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e533e560 con 0x5574e5452890 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e533e200 con 0x5574e5458720 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e52256a0 con 0x5574e5457e00 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.490+0000 7f58170cdb80 10 client.0 _ll_register_callbacks cb 0x5574e5347b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f57fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 0x5574e545bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33306/0 (socket says 172.21.15.46:33306) 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 -- 172.21.15.46:0/968642910 learned_addr learned my addr 172.21.15.46:0/968642910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5801600700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f57fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 0x5574e545bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44300/0 (socket says 172.21.15.46:44300) 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 -- 172.21.15.46:0/968642910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 msgr2=0x5574e545bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 0x5574e545bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 -- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 msgr2=0x5574e545c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 -- 172.21.15.46:0/968642910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00095f0 con 0x5574e5452890 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f5800c00700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57f00a30a0 tx=0x7f57f00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7190a961da7fd910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f57e2000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.491+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00a35d0 con 0x5574e5452890 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f57f00228c0 con 0x5574e5452890 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574e52fb270 con 0x5574e5452890 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f00b92c0 con 0x5574e5452890 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 10 client.17493 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574e51cfb60 con 0x5574e5452890 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574e52fb270 con 0x5574e5452890 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57f009a6d0 con 0x5574e5452890 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 10 client.17493 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 20 client.17493 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e546ba20 0x5574e546de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 4 client.17493 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.492+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574e5421810 con 0x5574e546ba20 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.493+0000 7f57fbe00700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e546ba20 0x5574e546de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.493+0000 7f57fbe00700 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e546ba20 0x5574e546de10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562a4d5c0dd1cf93 server_cookie=16afe07a0eb5aa30 in_seq=0 out_seq=0 2024-05-08T23:10:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.493+0000 7f57e2000700 10 client.17493 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f57e2000700 1 -- 172.21.15.46:0/968642910 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574e5421810 con 0x5574e546ba20 2024-05-08T23:10:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f57e2000700 10 client.17493 handle_command_reply: tid=0 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 client.17493 shutdown 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 msgr2=0x5574e545bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f57f00a30a0 tx=0x7f57f00092d0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 shutdown_connections 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e546ba20 0x5574e546de10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5452890 0x5574e545bab0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 --2- 172.21.15.46:0/968642910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5457e00 0x5574e545bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 >> 172.21.15.46:0/968642910 conn(0x5574e53b0210 msgr2=0x5574e544c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.494+0000 7f57e2000700 0 client.17493 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.495+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 shutdown_connections 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.495+0000 7f58170cdb80 1 -- 172.21.15.46:0/968642910 wait complete. 2024-05-08T23:10:48.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.495+0000 7f58170cdb80 20 client.17493 trim_cache size 0 max 16384 2024-05-08T23:10:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 Processor -- start 2024-05-08T23:10:48.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 -- start start 2024-05-08T23:10:48.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e544cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e544ea30 con 0x5574e5457e00 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e544ee50 con 0x5574e5458720 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.519+0000 7f58170cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e545ea80 con 0x5574e5452890 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f5800c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f5800c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44302/0 (socket says 172.21.15.46:44302) 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f5800c00700 1 -- 172.21.15.46:0/1650136614 learned_addr learned my addr 172.21.15.46:0/1650136614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e544cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f5801600700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 -- 172.21.15.46:0/1650136614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 msgr2=0x5574e5457430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 -- 172.21.15.46:0/1650136614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 msgr2=0x5574e545aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 -- 172.21.15.46:0/1650136614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e5368560 con 0x5574e5457e00 2024-05-08T23:10:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f5800c00700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:48.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.520+0000 7f57fbe00700 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e544cd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5574e545d080 tx=0x7f57e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45aba226fa94d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.521+0000 7f57ee600700 1 -- 172.21.15.46:0/1650136614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017040 con 0x5574e5457e00 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.521+0000 7f57ee600700 1 -- 172.21.15.46:0/1650136614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57e8018040 con 0x5574e5457e00 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.521+0000 7f57ee600700 1 -- 172.21.15.46:0/1650136614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80056c0 con 0x5574e5457e00 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- 172.21.15.46:0/1650136614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 msgr2=0x5574e544cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e544cd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5574e545d080 tx=0x7f57e8009260 comp rx=0 tx=0).stop 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- 172.21.15.46:0/1650136614 shutdown_connections 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5458720 0x5574e545aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5452890 0x5574e5457430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 --2- 172.21.15.46:0/1650136614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e544cd20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- 172.21.15.46:0/1650136614 >> 172.21.15.46:0/1650136614 conn(0x5574e53b0210 msgr2=0x5574e545a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- 172.21.15.46:0/1650136614 shutdown_connections 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- 172.21.15.46:0/1650136614 wait complete. 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.522+0000 7f58170cdb80 1 -- start start 2024-05-08T23:10:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.523+0000 7f58170cdb80 1 Processor -- start 2024-05-08T23:10:48.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.523+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 0x5574e545f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.523+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.523+0000 7f58170cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 0x5574e54baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.523+0000 7f58170cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e54bb460 con 0x5574e5457e00 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f58170cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e54bb5a0 con 0x5574e5452890 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f58170cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e54bb760 con 0x5574e5458720 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f58170cdb80 10 client.0 _ll_register_callbacks cb 0x5574e5347b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33318/0 (socket says 172.21.15.46:33318) 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 -- 172.21.15.46:0/2430276870 learned_addr learned my addr 172.21.15.46:0/2430276870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f5801600700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 0x5574e54baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f5800c00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 0x5574e545f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 -- 172.21.15.46:0/2430276870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 msgr2=0x5574e54baf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 0x5574e54baf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 -- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 msgr2=0x5574e545f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:48.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 0x5574e545f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f57fbe00700 1 -- 172.21.15.46:0/2430276870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574e51cfb60 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f5800c00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 0x5574e545f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.524+0000 7f5801600700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 0x5574e54baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57fbe00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e8005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02a5115f3bd28c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57e2000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80052a0 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f57e8005400 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57f009b1e0 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e8005600 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f58170cdb80 10 client.17499 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f57e80124b0 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.525+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f009b1e0 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57e80214f0 con 0x5574e5457e00 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 10 client.17499 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 20 client.17499 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e54c2f00 0x5574e54c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 4 client.17499 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:10:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574e539aec0 con 0x5574e54c2f00 2024-05-08T23:10:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.526+0000 7f5800c00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e54c2f00 0x5574e54c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f5800c00700 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e54c2f00 0x5574e54c5320 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2803d2898601b15 server_cookie=1e9d8c4778538439 in_seq=0 out_seq=0 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f57e2000700 10 client.17499 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f57e2000700 1 -- 172.21.15.46:0/2430276870 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574e539aec0 con 0x5574e54c2f00 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f57e2000700 10 client.17499 handle_command_reply: tid=0 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 client.17499 shutdown 2024-05-08T23:10:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 msgr2=0x5574e545f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e8005160 comp rx=0 tx=0).stop 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 shutdown_connections 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574e5458720 0x5574e54baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574e5452890 0x5574e545f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5574e54c2f00 0x5574e54c5320 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 --2- 172.21.15.46:0/2430276870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574e5457e00 0x5574e545f7c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f57e2000700 0 client.17499 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 >> 172.21.15.46:0/2430276870 conn(0x5574e53b0210 msgr2=0x5574e5458bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 shutdown_connections 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.527+0000 7f58170cdb80 1 -- 172.21.15.46:0/2430276870 wait complete. 2024-05-08T23:10:48.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f58170cdb80 20 client.17499 trim_cache size 0 max 16384 2024-05-08T23:10:48.531 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:10:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:10:48.531 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 msgr2=0x7f57e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 0x7f57e40a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009fd0 tx=0x7f57f400b040 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 msgr2=0x7f57fc1994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f57f0000c00 tx=0x7f57f0005740 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 shutdown_connections 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57fc10b3c0 0x7f57fc19ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57fc101fa0 0x7f57fc1994f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f57e40a1290 0x7f57e40a3750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 --2- 172.21.15.46:0/3027828094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57fc1028c0 0x7f57fc199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 >> 172.21.15.46:0/3027828094 conn(0x7f57fc0fb760 msgr2=0x7f57fc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 shutdown_connections 2024-05-08T23:10:48.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.528+0000 7f5803600700 1 -- 172.21.15.46:0/3027828094 wait complete. 2024-05-08T23:10:48.548 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:10:48.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 -- 172.21.15.46:0/3603384830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470101a20 msgr2=0x7f1470105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 --2- 172.21.15.46:0/3603384830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470101a20 0x7f1470105a60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14580098f0 tx=0x7f1458009c00 comp rx=0 tx=0).stop 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 -- 172.21.15.46:0/3603384830 shutdown_connections 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 --2- 172.21.15.46:0/3603384830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470106190 0x7f147010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 --2- 172.21.15.46:0/3603384830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1470101070 0x7f1470101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 --2- 172.21.15.46:0/3603384830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470101a20 0x7f1470105a60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 -- 172.21.15.46:0/3603384830 >> 172.21.15.46:0/3603384830 conn(0x7f14700fc930 msgr2=0x7f14700fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 -- 172.21.15.46:0/3603384830 shutdown_connections 2024-05-08T23:10:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.776+0000 7f1477200700 1 -- 172.21.15.46:0/3603384830 wait complete. 2024-05-08T23:10:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.777+0000 7f1477200700 1 Processor -- start 2024-05-08T23:10:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.777+0000 7f1477200700 1 -- start start 2024-05-08T23:10:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470106190 0x7f1470199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147019e960 con 0x7f1470101070 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147019ead0 con 0x7f147019dee0 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1477200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147019ec90 con 0x7f1470106190 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f146fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1475200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1475200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34020/0 (socket says 172.21.15.46:34020) 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f146fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33340/0 (socket says 172.21.15.46:33340) 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.778+0000 7f1475200700 1 -- 172.21.15.46:0/1999764908 learned_addr learned my addr 172.21.15.46:0/1999764908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1467400700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470106190 0x7f1470199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 -- 172.21.15.46:0/1999764908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470106190 msgr2=0x7f1470199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470106190 0x7f1470199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 -- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 msgr2=0x7f1470199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 -- 172.21.15.46:0/1999764908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458009520 con 0x7f147019dee0 2024-05-08T23:10:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f146fe00700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.779+0000 7f1475200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f146800e940 tx=0x7f146800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.780+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680097b0 con 0x7f147019dee0 2024-05-08T23:10:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.780+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1468005020 con 0x7f147019dee0 2024-05-08T23:10:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.780+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14701a61f0 con 0x7f147019dee0 2024-05-08T23:10:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.780+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800cc40 con 0x7f147019dee0 2024-05-08T23:10:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.780+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14701a6720 con 0x7f147019dee0 2024-05-08T23:10:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.781+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1470102540 con 0x7f147019dee0 2024-05-08T23:10:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.783+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1468005400 con 0x7f147019dee0 2024-05-08T23:10:48.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.785+0000 7f146d600700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 0x7f14540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:48.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.785+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f1468014070 con 0x7f147019dee0 2024-05-08T23:10:48.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.785+0000 7f146fe00700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 0x7f14540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.785+0000 7f146fe00700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 0x7f14540a37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1460003e30 tx=0x7f146000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:48.787+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f14680963d0 con 0x7f147019dee0 2024-05-08T23:10:48.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 568462 10.23 MB/sec execute 190 sec latency 14.506 ms 2024-05-08T23:10:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.096+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1470067d10 con 0x7f147019dee0 2024-05-08T23:10:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.097+0000 7f146d600700 1 -- 172.21.15.46:0/1999764908 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f1468005190 con 0x7f147019dee0 2024-05-08T23:10:49.097 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:10:49.097 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:10:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 msgr2=0x7f14540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 0x7f14540a37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1460003e30 tx=0x7f146000b040 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 msgr2=0x7f147019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f146800e940 tx=0x7f146800ec50 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 shutdown_connections 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f147019dee0 0x7f147019e360 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14540a1310 0x7f14540a37d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1470101070 0x7f1470199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 --2- 172.21.15.46:0/1999764908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1470106190 0x7f1470199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 >> 172.21.15.46:0/1999764908 conn(0x7f14700fc930 msgr2=0x7f14700fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 shutdown_connections 2024-05-08T23:10:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.100+0000 7f1477200700 1 -- 172.21.15.46:0/1999764908 wait complete. 2024-05-08T23:10:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:10:49.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 -- 172.21.15.46:0/3867722253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de4101080 msgr2=0x7f7de4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 --2- 172.21.15.46:0/3867722253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de4101080 0x7f7de4101460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009990 tx=0x7f7dd4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 -- 172.21.15.46:0/3867722253 shutdown_connections 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 --2- 172.21.15.46:0/3867722253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 --2- 172.21.15.46:0/3867722253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de4101080 0x7f7de4101460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 --2- 172.21.15.46:0/3867722253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101a30 0x7f7de4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 -- 172.21.15.46:0/3867722253 >> 172.21.15.46:0/3867722253 conn(0x7f7de40fc8e0 msgr2=0x7f7de40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 -- 172.21.15.46:0/3867722253 shutdown_connections 2024-05-08T23:10:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.344+0000 7f7deaa00700 1 -- 172.21.15.46:0/3867722253 wait complete. 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.345+0000 7f7deaa00700 1 Processor -- start 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 -- start start 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101080 0x7f7de41a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de410cdb0 0x7f7de410d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de410d830 con 0x7f7de4101080 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de410d970 con 0x7f7de41061a0 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7deaa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de410db00 con 0x7f7de410cdb0 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7de3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7de3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34038/0 (socket says 172.21.15.46:34038) 2024-05-08T23:10:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7de3400700 1 -- 172.21.15.46:0/1824776057 learned_addr learned my addr 172.21.15.46:0/1824776057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.346+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101080 0x7f7de41a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de8a00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de410cdb0 0x7f7de410d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 -- 172.21.15.46:0/1824776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de410cdb0 msgr2=0x7f7de410d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de410cdb0 0x7f7de410d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 -- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101080 msgr2=0x7f7de41a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101080 0x7f7de41a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 -- 172.21.15.46:0/1824776057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40095f0 con 0x7f7de41061a0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de3400700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d980 tx=0x7f7dd000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.347+0000 7f7de0c00700 1 -- 172.21.15.46:0/1824776057 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000f890 con 0x7f7de41061a0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.348+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de41aa550 con 0x7f7de41061a0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.348+0000 7f7de0c00700 1 -- 172.21.15.46:0/1824776057 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd0010460 con 0x7f7de41061a0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.348+0000 7f7de0c00700 1 -- 172.21.15.46:0/1824776057 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000e610 con 0x7f7de41061a0 2024-05-08T23:10:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.348+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de41aaa30 con 0x7f7de41061a0 2024-05-08T23:10:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.351+0000 7f7de0c00700 1 -- 172.21.15.46:0/1824776057 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7dd00105d0 con 0x7f7de41061a0 2024-05-08T23:10:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.352+0000 7f7dfe50bb80 1 Processor -- start 2024-05-08T23:10:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.352+0000 7f7dfe50bb80 1 -- start start 2024-05-08T23:10:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.352+0000 7f7de0c00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 0x7f7dc80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de0c00700 1 -- 172.21.15.46:0/1824776057 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7dd00ccae0 con 0x7f7de41061a0 2024-05-08T23:10:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de8a00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 0x7f7dc80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 0x55c58e1d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e128150 con 0x55c58e1d1890 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e125b10 con 0x55c58e1d7720 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e121400 con 0x55c58e1d6e00 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44362/0 (socket says 172.21.15.46:44362) 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de3e00700 1 -- 172.21.15.46:0/4029852015 learned_addr learned my addr 172.21.15.46:0/4029852015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de8a00700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 0x55c58e1d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.353+0000 7f7de3400700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 -- 172.21.15.46:0/4029852015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 msgr2=0x55c58e1d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 -- 172.21.15.46:0/4029852015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 msgr2=0x55c58e1d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 0x55c58e1d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 -- 172.21.15.46:0/4029852015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58dfa23b0 con 0x55c58e1d6e00 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de8a00700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 0x55c58e1d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de8a00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 0x7f7dc80a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00f870 tx=0x7f7ddc00fc30 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7de3e00700 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4015000 tx=0x7f7dd4005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef759da467b8c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7dd9200700 1 -- 172.21.15.46:0/4029852015 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd401d070 con 0x55c58e1d6e00 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7dd9200700 1 -- 172.21.15.46:0/4029852015 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd401e040 con 0x55c58e1d6e00 2024-05-08T23:10:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.354+0000 7f7dd9200700 1 -- 172.21.15.46:0/4029852015 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4004740 con 0x55c58e1d6e00 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/4029852015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 msgr2=0x55c58e1d71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4015000 tx=0x7f7dd4005ee0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/4029852015 shutdown_connections 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d1890 0x55c58e1d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d71e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/4029852015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/4029852015 >> 172.21.15.46:0/4029852015 conn(0x55c58e12f210 msgr2=0x55c58e1cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/4029852015 shutdown_connections 2024-05-08T23:10:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.355+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/4029852015 wait complete. 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.356+0000 7f7dfe50bb80 1 -- start start 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.356+0000 7f7dfe50bb80 1 Processor -- start 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.356+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.356+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.356+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e0bd560 con 0x55c58e1d6e00 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e0bd200 con 0x55c58e1d7720 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58dfa46a0 con 0x55c58e1d1890 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7dfe50bb80 10 client.0 _ll_register_callbacks cb 0x55c58e0c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44378/0 (socket says 172.21.15.46:44378) 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 -- 172.21.15.46:0/364981148 learned_addr learned my addr 172.21.15.46:0/364981148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3400700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de8a00700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 -- 172.21.15.46:0/364981148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 msgr2=0x55c58e1db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 -- 172.21.15.46:0/364981148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 msgr2=0x55c58e1dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1dafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3e00700 1 -- 172.21.15.46:0/364981148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de410e590 con 0x55c58e1d1890 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.357+0000 7f7de3400700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7de3e00700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006010 tx=0x7f7dd400b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb1995480d06750 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dcd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd401d030 con 0x55c58e1d1890 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7dd4004700 con 0x55c58e1d1890 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c58e07a270 con 0x55c58e1d1890 2024-05-08T23:10:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dd40048c0 con 0x55c58e1d1890 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dfe50bb80 10 client.27574 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.358+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c58df4eb60 con 0x55c58e1d1890 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c58e07a270 con 0x55c58e1d1890 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dd400fb90 con 0x55c58e1d1890 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 10 client.27574 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 10 client.27574 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 20 client.27574 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e1eac90 0x55c58e1ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 4 client.27574 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c58e1a0810 con 0x55c58e1eac90 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.359+0000 7f7de3400700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e1eac90 0x55c58e1ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.360+0000 7f7de3400700 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e1eac90 0x55c58e1ed080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82641297e6733795 server_cookie=5c7c7eac70d048d4 in_seq=0 out_seq=0 2024-05-08T23:10:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.360+0000 7f7dcd600700 10 client.27574 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dcd600700 1 -- 172.21.15.46:0/364981148 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c58e1a0810 con 0x55c58e1eac90 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dcd600700 10 client.27574 handle_command_reply: tid=0 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 client.27574 shutdown 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 msgr2=0x55c58e1daa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006010 tx=0x7f7dd400b7a0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 shutdown_connections 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e1eac90 0x55c58e1ed080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d7720 0x55c58e1db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d1890 0x55c58e1daa70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dcd600700 0 client.27574 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/364981148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 >> 172.21.15.46:0/364981148 conn(0x55c58e12f210 msgr2=0x55c58e1cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 shutdown_connections 2024-05-08T23:10:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/364981148 wait complete. 2024-05-08T23:10:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.361+0000 7f7dfe50bb80 20 client.27574 trim_cache size 0 max 16384 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.377+0000 7f7dfe50bb80 1 Processor -- start 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 -- start start 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d7720 0x55c58e1e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1d9050 con 0x55c58e1d6e00 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1c6a90 con 0x55c58e1d1890 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1c4040 con 0x55c58e1d7720 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34056/0 (socket says 172.21.15.46:34056) 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3e00700 1 -- 172.21.15.46:0/2901633950 learned_addr learned my addr 172.21.15.46:0/2901633950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de8a00700 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d7720 0x55c58e1e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3400700 1 -- 172.21.15.46:0/2901633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d7720 msgr2=0x55c58e1e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3400700 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d7720 0x55c58e1e6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3400700 1 -- 172.21.15.46:0/2901633950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 msgr2=0x55c58e1d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.378+0000 7f7de3400700 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.379+0000 7f7de3400700 1 -- 172.21.15.46:0/2901633950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58e0e7560 con 0x55c58e1d6e00 2024-05-08T23:10:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.379+0000 7f7de3400700 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1ed080 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c9e20 tx=0x7f7dd00996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331d76805a28ba60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.379+0000 7f7dd9c00700 1 -- 172.21.15.46:0/2901633950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00a1c60 con 0x55c58e1d6e00 2024-05-08T23:10:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.379+0000 7f7dd9c00700 1 -- 172.21.15.46:0/2901633950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd0097b80 con 0x55c58e1d6e00 2024-05-08T23:10:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.379+0000 7f7dd9c00700 1 -- 172.21.15.46:0/2901633950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0099ca0 con 0x55c58e1d6e00 2024-05-08T23:10:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/2901633950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 msgr2=0x55c58e1ed080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1ed080 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c9e20 tx=0x7f7dd00996e0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/2901633950 shutdown_connections 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d7720 0x55c58e1e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/2901633950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d6e00 0x55c58e1ed080 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/2901633950 >> 172.21.15.46:0/2901633950 conn(0x55c58e12f210 msgr2=0x55c58e1d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/2901633950 shutdown_connections 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.380+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/2901633950 wait complete. 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.381+0000 7f7dfe50bb80 1 -- start start 2024-05-08T23:10:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.381+0000 7f7dfe50bb80 1 Processor -- start 2024-05-08T23:10:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d7720 0x55c58e1d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1d4600 con 0x55c58e1d7720 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1d2930 con 0x55c58e1d1890 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58e1d2af0 con 0x55c58e1d6e00 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7dfe50bb80 10 client.0 _ll_register_callbacks cb 0x55c58e0c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44396/0 (socket says 172.21.15.46:44396) 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 -- 172.21.15.46:0/270032570 learned_addr learned my addr 172.21.15.46:0/270032570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de8a00700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d7720 0x55c58e1d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 -- 172.21.15.46:0/270032570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 msgr2=0x55c58e1e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 -- 172.21.15.46:0/270032570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d7720 msgr2=0x55c58e1d3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d7720 0x55c58e1d3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.382+0000 7f7de3400700 1 -- 172.21.15.46:0/270032570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de41aaa30 con 0x55c58e1d6e00 2024-05-08T23:10:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7de3400700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c9e80 tx=0x7f7dd009d790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f38e90bc5ee233 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dcd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00b92c0 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7f7dd0098cd0 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dd401e040 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dd0098e90 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dfe50bb80 10 client.27580 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.383+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c58dfa6cd0 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd401e040 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dd009d170 con 0x55c58e1d6e00 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 10 client.27580 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 10 client.27580 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 20 client.27580 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e241400 0x55c58e243820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 4 client.27580 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c58e119ec0 con 0x55c58e241400 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.384+0000 7f7de3e00700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e241400 0x55c58e243820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dcd600700 10 client.27580 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7de3e00700 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e241400 0x55c58e243820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=490361f1d61a35cc server_cookie=8f4435ca51408a89 in_seq=0 out_seq=0 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dcd600700 1 -- 172.21.15.46:0/270032570 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c58e119ec0 con 0x55c58e241400 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dcd600700 10 client.27580 handle_command_reply: tid=0 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 client.27580 shutdown 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 msgr2=0x55c58e1d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c9e80 tx=0x7f7dd009d790 comp rx=0 tx=0).stop 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 shutdown_connections 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c58e241400 0x55c58e243820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c58e1d7720 0x55c58e1d3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c58e1d1890 0x55c58e1e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dcd600700 0 client.27580 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 --2- 172.21.15.46:0/270032570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c58e1d6e00 0x55c58e1d39b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 >> 172.21.15.46:0/270032570 conn(0x55c58e12f210 msgr2=0x55c58e1d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 shutdown_connections 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 1 -- 172.21.15.46:0/270032570 wait complete. 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.385+0000 7f7dfe50bb80 20 client.27580 trim_cache size 0 max 16384 2024-05-08T23:10:49.388 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 msgr2=0x7f7dc80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 0x7f7dc80a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00f870 tx=0x7f7ddc00fc30 comp rx=0 tx=0).stop 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 msgr2=0x7f7de41a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d980 tx=0x7f7dd000dc90 comp rx=0 tx=0).stop 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 shutdown_connections 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7de410cdb0 0x7f7de410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc80a1290 0x7f7dc80a3750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de4101080 0x7f7de41a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 --2- 172.21.15.46:0/1824776057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de41061a0 0x7f7de41a3c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 >> 172.21.15.46:0/1824776057 conn(0x7f7de40fc8e0 msgr2=0x7f7de40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 shutdown_connections 2024-05-08T23:10:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.386+0000 7f7deaa00700 1 -- 172.21.15.46:0/1824776057 wait complete. 2024-05-08T23:10:49.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 -- 172.21.15.46:0/3207930661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0106180 msgr2=0x7fe0a010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/3207930661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0106180 0x7fe0a010dd00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b0a0 tx=0x7fe09800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 -- 172.21.15.46:0/3207930661 shutdown_connections 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/3207930661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0106180 0x7fe0a010dd00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/3207930661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a0101060 0x7fe0a0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/3207930661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0101a10 0x7fe0a0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 -- 172.21.15.46:0/3207930661 >> 172.21.15.46:0/3207930661 conn(0x7fe0a00fc900 msgr2=0x7fe0a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.654+0000 7fe0a4e00700 1 -- 172.21.15.46:0/3207930661 shutdown_connections 2024-05-08T23:10:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.655+0000 7fe0a4e00700 1 -- 172.21.15.46:0/3207930661 wait complete. 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.655+0000 7fe0a4e00700 1 Processor -- start 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 -- start start 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0101060 0x7fe0a010d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 0x7fe0a010b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a010dfe0 con 0x7fe0a0101060 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a01aa240 con 0x7fe0a0106180 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe0a4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a01aa380 con 0x7fe0a010ae00 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe09d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe09e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 0x7fe0a010b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe09d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34088/0 (socket says 172.21.15.46:34088) 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe09d400700 1 -- 172.21.15.46:0/1877245421 learned_addr learned my addr 172.21.15.46:0/1877245421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.656+0000 7fe09de00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0101060 0x7fe0a010d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 -- 172.21.15.46:0/1877245421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 msgr2=0x7fe0a010b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 0x7fe0a010b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 -- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0101060 msgr2=0x7fe0a010d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0101060 0x7fe0a010d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 -- 172.21.15.46:0/1877245421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098009d00 con 0x7fe0a0106180 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09e800700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 0x7fe0a010b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe09d400700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e970 tx=0x7fe08c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe096a00700 1 -- 172.21.15.46:0/1877245421 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c009790 con 0x7fe0a0106180 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a01aa610 con 0x7fe0a0106180 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.657+0000 7fe096a00700 1 -- 172.21.15.46:0/1877245421 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08c00c7e0 con 0x7fe0a0106180 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.658+0000 7fe096a00700 1 -- 172.21.15.46:0/1877245421 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0054e0 con 0x7fe0a0106180 2024-05-08T23:10:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.658+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a01aab20 con 0x7fe0a0106180 2024-05-08T23:10:49.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.661+0000 7fe096a00700 1 -- 172.21.15.46:0/1877245421 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe08c00c950 con 0x7fe0a0106180 2024-05-08T23:10:49.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.662+0000 7fe0b89beb80 1 Processor -- start 2024-05-08T23:10:49.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.662+0000 7fe096a00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 0x7fe0840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.662+0000 7fe0b89beb80 1 -- start start 2024-05-08T23:10:49.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.662+0000 7fe096a00700 1 -- 172.21.15.46:0/1877245421 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe08c014070 con 0x7fe0a0106180 2024-05-08T23:10:49.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09e800700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 0x7fe0840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561558921150 con 0x5615589ca890 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155891eb10 con 0x5615589cfe00 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155891a400 con 0x5615589d0720 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44410/0 (socket says 172.21.15.46:44410) 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34092/0 (socket says 172.21.15.46:34092) 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 -- 172.21.15.46:0/3272605007 learned_addr learned my addr 172.21.15.46:0/3272605007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09e800700 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 -- 172.21.15.46:0/3272605007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 msgr2=0x5615589d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 -- 172.21.15.46:0/3272605007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 msgr2=0x5615589cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09d400700 1 -- 172.21.15.46:0/3272605007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155879b3b0 con 0x5615589d0720 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.663+0000 7fe09de00700 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.664+0000 7fe09d400700 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe08c096a30 tx=0x7fe08c00ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebcbe0227ca4357a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.664+0000 7fe09e800700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 0x7fe0840a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b780 tx=0x7fe098007bf0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.664+0000 7fe08b400700 1 -- 172.21.15.46:0/3272605007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c09ddf0 con 0x5615589d0720 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.664+0000 7fe08b400700 1 -- 172.21.15.46:0/3272605007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08c0ade90 con 0x5615589d0720 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.664+0000 7fe08b400700 1 -- 172.21.15.46:0/3272605007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c016330 con 0x5615589d0720 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- 172.21.15.46:0/3272605007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 msgr2=0x5615589ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe08c096a30 tx=0x7fe08c00ccd0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- 172.21.15.46:0/3272605007 shutdown_connections 2024-05-08T23:10:49.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/3272605007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589d0720 0x5615589ca350 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- 172.21.15.46:0/3272605007 >> 172.21.15.46:0/3272605007 conn(0x561558928210 msgr2=0x5615589c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- 172.21.15.46:0/3272605007 shutdown_connections 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- 172.21.15.46:0/3272605007 wait complete. 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.665+0000 7fe0b89beb80 1 -- start start 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 Processor -- start 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589d0720 0x5615589d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615588b6560 con 0x5615589d0720 2024-05-08T23:10:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615588b6200 con 0x5615589cfe00 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155879d6a0 con 0x5615589ca890 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe0b89beb80 10 client.0 _ll_register_callbacks cb 0x5615588bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe09de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe09de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44414/0 (socket says 172.21.15.46:44414) 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe09de00700 1 -- 172.21.15.46:0/2128697586 learned_addr learned my addr 172.21.15.46:0/2128697586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.666+0000 7fe09d400700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09e800700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589d0720 0x5615589d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 -- 172.21.15.46:0/2128697586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 msgr2=0x5615589d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 -- 172.21.15.46:0/2128697586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589d0720 msgr2=0x5615589d4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589d0720 0x5615589d4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 -- 172.21.15.46:0/2128697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0900098e0 con 0x5615589ca890 2024-05-08T23:10:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe09de00700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe090012980 tx=0x7fe090012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed645a1c79226761 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe07fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090017070 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561558873270 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.667+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe09000f7b0 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0900122e0 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 10 client.27592 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe0a01aab20 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561558873270 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe090010aa0 con 0x5615589ca890 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 10 client.27592 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 10 client.27592 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 20 client.27592 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5615589e4230 0x5615589e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 4 client.27592 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:49.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.668+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561558999810 con 0x5615589e4230 2024-05-08T23:10:49.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.669+0000 7fe09d400700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5615589e4230 0x5615589e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.669+0000 7fe09d400700 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5615589e4230 0x5615589e6650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28ba68de298c973f server_cookie=ef583dced726927b in_seq=0 out_seq=0 2024-05-08T23:10:49.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.669+0000 7fe07fe00700 10 client.27592 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe07fe00700 1 -- 172.21.15.46:0/2128697586 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561558999810 con 0x5615589e4230 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe07fe00700 10 client.27592 handle_command_reply: tid=0 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 client.27592 shutdown 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 msgr2=0x5615589d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe090012980 tx=0x7fe090012c90 comp rx=0 tx=0).stop 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 shutdown_connections 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5615589e4230 0x5615589e6650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589d0720 0x5615589d4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe07fe00700 0 client.27592 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x5615589d3bc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/2128697586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589cfe00 0x5615589d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 >> 172.21.15.46:0/2128697586 conn(0x561558928210 msgr2=0x5615589c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 shutdown_connections 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.670+0000 7fe0b89beb80 1 -- 172.21.15.46:0/2128697586 wait complete. 2024-05-08T23:10:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.671+0000 7fe0b89beb80 20 client.27592 trim_cache size 0 max 16384 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 Processor -- start 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 -- start start 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x5615589c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615589c6a30 con 0x5615589ca890 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615589bd040 con 0x5615589d0720 2024-05-08T23:10:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615589eb820 con 0x5615589cfe00 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe09d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe09d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44416/0 (socket says 172.21.15.46:44416) 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe09d400700 1 -- 172.21.15.46:0/668830450 learned_addr learned my addr 172.21.15.46:0/668830450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.686+0000 7fe09e800700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x5615589c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09de00700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 -- 172.21.15.46:0/668830450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 msgr2=0x5615589c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x5615589c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 -- 172.21.15.46:0/668830450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 msgr2=0x5615589ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 -- 172.21.15.46:0/668830450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615589caf40 con 0x5615589cfe00 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09de00700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:10:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe09d400700 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00efd0 tx=0x7fe08c099c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7114c97f53203998 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe08be00700 1 -- 172.21.15.46:0/668830450 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c063070 con 0x5615589cfe00 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe08be00700 1 -- 172.21.15.46:0/668830450 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08c005660 con 0x5615589cfe00 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.687+0000 7fe08be00700 1 -- 172.21.15.46:0/668830450 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0b8040 con 0x5615589cfe00 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 -- 172.21.15.46:0/668830450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 msgr2=0x5615589cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00efd0 tx=0x7fe08c099c30 comp rx=0 tx=0).stop 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 -- 172.21.15.46:0/668830450 shutdown_connections 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x5615589c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589ca890 0x5615589ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/668830450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589cfe00 0x5615589cf3c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 -- 172.21.15.46:0/668830450 >> 172.21.15.46:0/668830450 conn(0x561558928210 msgr2=0x5615589cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 -- 172.21.15.46:0/668830450 shutdown_connections 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.688+0000 7fe0b89beb80 1 -- 172.21.15.46:0/668830450 wait complete. 2024-05-08T23:10:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 -- start start 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 Processor -- start 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589cfe00 0x561558a2ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x561558a39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561558a2f370 con 0x5615589cfe00 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561558a2f4b0 con 0x5615589d0720 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.689+0000 7fe0b89beb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561558a39c40 con 0x5615589ca890 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe0b89beb80 10 client.0 _ll_register_callbacks cb 0x5615588bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x561558a39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589cfe00 0x561558a2ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44428/0 (socket says 172.21.15.46:44428) 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 -- 172.21.15.46:0/332620355 learned_addr learned my addr 172.21.15.46:0/332620355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 -- 172.21.15.46:0/332620355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 msgr2=0x561558a39700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x561558a39700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 -- 172.21.15.46:0/332620355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589cfe00 msgr2=0x561558a2ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589cfe00 0x561558a2ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 -- 172.21.15.46:0/332620355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c05fe70 con 0x5615589ca890 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe09de00700 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe090010dd0 tx=0x7fe09000ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ee08ba5147b492 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.690+0000 7fe07fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090017040 con 0x5615589ca890 2024-05-08T23:10:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fe090000e40 con 0x5615589ca890 2024-05-08T23:10:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe08c0ade90 con 0x5615589ca890 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0900122e0 con 0x5615589ca890 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe0b89beb80 10 client.27598 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561558747b60 con 0x5615589ca890 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08c0ade90 con 0x5615589ca890 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.691+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe090033070 con 0x5615589ca890 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 10 client.27598 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 10 client.27598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 20 client.27598 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x561558a3b3b0 0x561558a3d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 4 client.27598 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561558912ec0 con 0x561558a3b3b0 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe09d400700 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x561558a3b3b0 0x561558a3d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe07fe00700 10 client.27598 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe09d400700 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x561558a3b3b0 0x561558a3d7d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95a4bbe18dc518b9 server_cookie=c31850d7d668fd66 in_seq=0 out_seq=0 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe07fe00700 1 -- 172.21.15.46:0/332620355 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561558912ec0 con 0x561558a3b3b0 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe07fe00700 10 client.27598 handle_command_reply: tid=0 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.692+0000 7fe0b89beb80 1 client.27598 shutdown 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 msgr2=0x561558a2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe090010dd0 tx=0x7fe09000ff70 comp rx=0 tx=0).stop 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 shutdown_connections 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x561558a3b3b0 0x561558a3d7d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615589d0720 0x561558a39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615589ca890 0x561558a2e680 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe07fe00700 0 client.27598 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 --2- 172.21.15.46:0/332620355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615589cfe00 0x561558a2ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 >> 172.21.15.46:0/332620355 conn(0x561558928210 msgr2=0x5615589d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 shutdown_connections 2024-05-08T23:10:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 1 -- 172.21.15.46:0/332620355 wait complete. 2024-05-08T23:10:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.693+0000 7fe0b89beb80 20 client.27598 trim_cache size 0 max 16384 2024-05-08T23:10:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 msgr2=0x7fe0840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 0x7fe0840a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b780 tx=0x7fe098007bf0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 msgr2=0x7fe0a010a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e970 tx=0x7fe08c00ec80 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 shutdown_connections 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a010ae00 0x7fe0a010b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe0840a1290 0x7fe0840a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a0101060 0x7fe0a010d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 --2- 172.21.15.46:0/1877245421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106180 0x7fe0a010a8c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 >> 172.21.15.46:0/1877245421 conn(0x7fe0a00fc900 msgr2=0x7fe0a00fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 shutdown_connections 2024-05-08T23:10:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.694+0000 7fe0a4e00700 1 -- 172.21.15.46:0/1877245421 wait complete. 2024-05-08T23:10:49.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:49 smithi046 ceph-mon[36898]: pgmap v213: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:49.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:49 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1999764908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:49.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:49 smithi086 ceph-mon[78610]: pgmap v213: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:49.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:49 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1999764908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:49.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:10:49.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:49 smithi053 ceph-mon[77587]: pgmap v213: 129 pgs: 129 active+clean; 367 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:49 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1999764908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:10:49.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 569863 10.21 MB/sec execute 191 sec latency 15.530 ms 2024-05-08T23:10:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 -- 172.21.15.46:0/79633021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8073c90 msgr2=0x7fc6e810d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 --2- 172.21.15.46:0/79633021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8073c90 0x7fc6e810d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009990 tx=0x7fc6d4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 -- 172.21.15.46:0/79633021 shutdown_connections 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 --2- 172.21.15.46:0/79633021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e810d6d0 0x7fc6e810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 --2- 172.21.15.46:0/79633021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 --2- 172.21.15.46:0/79633021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8073c90 0x7fc6e810d190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 -- 172.21.15.46:0/79633021 >> 172.21.15.46:0/79633021 conn(0x7fc6e80fc940 msgr2=0x7fc6e80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 -- 172.21.15.46:0/79633021 shutdown_connections 2024-05-08T23:10:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.943+0000 7fc6f0600700 1 -- 172.21.15.46:0/79633021 wait complete. 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 Processor -- start 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 -- start start 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e8073c90 0x7fc6e8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e819e4e0 con 0x7fc6e8073370 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e819e650 con 0x7fc6e8073c90 2024-05-08T23:10:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.944+0000 7fc6f0600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e819e810 con 0x7fc6e810d6d0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ed200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e8073c90 0x7fc6e8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6edc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44448/0 (socket says 172.21.15.46:44448) 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 -- 172.21.15.46:0/2102081992 learned_addr learned my addr 172.21.15.46:0/2102081992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 -- 172.21.15.46:0/2102081992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e8073c90 msgr2=0x7fc6e8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e8073c90 0x7fc6e8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 -- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 msgr2=0x7fc6e81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 -- 172.21.15.46:0/2102081992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40095f0 con 0x7fc6e810d6d0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6edc00700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.945+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00094f0 tx=0x7fc6e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.946+0000 7fc6e6000700 1 -- 172.21.15.46:0/2102081992 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0004db0 con 0x7fc6e810d6d0 2024-05-08T23:10:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.946+0000 7fc6e6000700 1 -- 172.21.15.46:0/2102081992 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e000f040 con 0x7fc6e810d6d0 2024-05-08T23:10:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.946+0000 7fc6e6000700 1 -- 172.21.15.46:0/2102081992 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0005980 con 0x7fc6e810d6d0 2024-05-08T23:10:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.946+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e819eaf0 con 0x7fc6e810d6d0 2024-05-08T23:10:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.946+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8100f00 con 0x7fc6e810d6d0 2024-05-08T23:10:49.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.949+0000 7fc6e6000700 1 -- 172.21.15.46:0/2102081992 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc6e0005020 con 0x7fc6e810d6d0 2024-05-08T23:10:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.950+0000 7fc6e6000700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 0x7fc6d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.950+0000 7fc7040f7b80 1 Processor -- start 2024-05-08T23:10:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.950+0000 7fc7040f7b80 1 -- start start 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.950+0000 7fc6e6000700 1 -- 172.21.15.46:0/2102081992 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc6e00cc8f0 con 0x7fc6e810d6d0 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.950+0000 7fc6edc00700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 0x7fc6d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d6720 0x55eb646d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb64627150 con 0x55eb646d5e00 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb64624b10 con 0x55eb646d6720 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb64620400 con 0x55eb646d0890 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6edc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44462/0 (socket says 172.21.15.46:44462) 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 -- 172.21.15.46:0/2172474263 learned_addr learned my addr 172.21.15.46:0/2172474263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ed200700 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d6720 0x55eb646d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 -- 172.21.15.46:0/2172474263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d6720 msgr2=0x55eb646d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d6720 0x55eb646d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 -- 172.21.15.46:0/2172474263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 msgr2=0x55eb646d61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646d61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.951+0000 7fc6ee600700 1 -- 172.21.15.46:0/2172474263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb644a13b0 con 0x55eb646d0890 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.952+0000 7fc6ee600700 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e00959b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e89b9840dbf3140 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.952+0000 7fc6edc00700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 0x7fc6d00a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009fd0 tx=0x7fc6dc009f40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.952+0000 7fc6daa00700 1 -- 172.21.15.46:0/2172474263 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e009a400 con 0x55eb646d0890 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.952+0000 7fc6daa00700 1 -- 172.21.15.46:0/2172474263 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e00b9070 con 0x55eb646d0890 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.952+0000 7fc6daa00700 1 -- 172.21.15.46:0/2172474263 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e009d420 con 0x55eb646d0890 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- 172.21.15.46:0/2172474263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 msgr2=0x55eb646d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e00959b0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- 172.21.15.46:0/2172474263 shutdown_connections 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646d0cd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646d61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/2172474263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d6720 0x55eb646d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- 172.21.15.46:0/2172474263 >> 172.21.15.46:0/2172474263 conn(0x55eb6462e210 msgr2=0x55eb646cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- 172.21.15.46:0/2172474263 shutdown_connections 2024-05-08T23:10:49.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- 172.21.15.46:0/2172474263 wait complete. 2024-05-08T23:10:49.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 -- start start 2024-05-08T23:10:49.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.953+0000 7fc7040f7b80 1 Processor -- start 2024-05-08T23:10:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d5e00 0x55eb646d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb645bc560 con 0x55eb646d6720 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb645bc200 con 0x55eb646d5e00 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb644a36a0 con 0x55eb646d0890 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc7040f7b80 10 client.0 _ll_register_callbacks cb 0x55eb645c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc6edc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc6edc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44464/0 (socket says 172.21.15.46:44464) 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc6edc00700 1 -- 172.21.15.46:0/354968527 learned_addr learned my addr 172.21.15.46:0/354968527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc6ed200700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d5e00 0x55eb646d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.954+0000 7fc6ee600700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.955+0000 7fc6edc00700 1 -- 172.21.15.46:0/354968527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d5e00 msgr2=0x55eb646d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.955+0000 7fc6edc00700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d5e00 0x55eb646d9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.955+0000 7fc6edc00700 1 -- 172.21.15.46:0/354968527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 msgr2=0x55eb646da2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.955+0000 7fc6edc00700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646da2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.955+0000 7fc6edc00700 1 -- 172.21.15.46:0/354968527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8100f00 con 0x55eb646d0890 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.956+0000 7fc6edc00700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0138d0 tx=0x7fc6dc01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57c04874c2d1af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.956+0000 7fc6cf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.956+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc003680 con 0x55eb646d0890 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.956+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc6dc025030 con 0x55eb646d0890 2024-05-08T23:10:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.956+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb64579270 con 0x55eb646d0890 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6dc0037e0 con 0x55eb646d0890 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 10 client.27610 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eb6444db60 con 0x55eb646d0890 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb64579270 con 0x55eb646d0890 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc6dc010880 con 0x55eb646d0890 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 10 client.27610 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 10 client.27610 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 20 client.27610 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb646ea150 0x55eb646ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 4 client.27610 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.957+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb6469f810 con 0x55eb646ea150 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.958+0000 7fc6ed200700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb646ea150 0x55eb646ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.958+0000 7fc6ed200700 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb646ea150 0x55eb646ec570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7dd6cd394b1f95b server_cookie=acdac7016b16096e in_seq=0 out_seq=0 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.958+0000 7fc6cf400700 10 client.27610 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.959+0000 7fc6cf400700 1 -- 172.21.15.46:0/354968527 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb6469f810 con 0x55eb646ea150 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.959+0000 7fc6cf400700 10 client.27610 handle_command_reply: tid=0 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.959+0000 7fc7040f7b80 1 client.27610 shutdown 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.959+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 msgr2=0x55eb646dc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0138d0 tx=0x7fc6dc01d000 comp rx=0 tx=0).stop 2024-05-08T23:10:49.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 shutdown_connections 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb646ea150 0x55eb646ec570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d0890 0x55eb646dc800 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/354968527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d5e00 0x55eb646d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 >> 172.21.15.46:0/354968527 conn(0x55eb6462e210 msgr2=0x55eb646ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc6cf400700 0 client.27610 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 shutdown_connections 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 1 -- 172.21.15.46:0/354968527 wait complete. 2024-05-08T23:10:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.960+0000 7fc7040f7b80 20 client.27610 trim_cache size 0 max 16384 2024-05-08T23:10:49.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.976+0000 7fc7040f7b80 1 Processor -- start 2024-05-08T23:10:49.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 -- start start 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646d6560 con 0x55eb646d6720 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646c5800 con 0x55eb646d0890 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646da8a0 con 0x55eb646d5e00 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc6ed200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc6ed200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44478/0 (socket says 172.21.15.46:44478) 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc6ed200700 1 -- 172.21.15.46:0/885801480 learned_addr learned my addr 172.21.15.46:0/885801480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc6edc00700 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.977+0000 7fc6ee600700 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 -- 172.21.15.46:0/885801480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 msgr2=0x55eb646d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 -- 172.21.15.46:0/885801480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 msgr2=0x55eb646d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 -- 172.21.15.46:0/885801480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb645e6560 con 0x55eb646d5e00 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6ed200700 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25a693951fea1da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6db400700 1 -- 172.21.15.46:0/885801480 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4032040 con 0x55eb646d5e00 2024-05-08T23:10:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6db400700 1 -- 172.21.15.46:0/885801480 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6d4026040 con 0x55eb646d5e00 2024-05-08T23:10:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.978+0000 7fc6db400700 1 -- 172.21.15.46:0/885801480 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4025040 con 0x55eb646d5e00 2024-05-08T23:10:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 -- 172.21.15.46:0/885801480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 msgr2=0x55eb646cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 -- 172.21.15.46:0/885801480 shutdown_connections 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d6720 0x55eb646d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/885801480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d5e00 0x55eb646cad20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 -- 172.21.15.46:0/885801480 >> 172.21.15.46:0/885801480 conn(0x55eb6462e210 msgr2=0x55eb646e5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 -- 172.21.15.46:0/885801480 shutdown_connections 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.979+0000 7fc7040f7b80 1 -- 172.21.15.46:0/885801480 wait complete. 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 -- start start 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 Processor -- start 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646e0210 con 0x55eb646d5e00 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646e0350 con 0x55eb646d0890 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.980+0000 7fc7040f7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb646e0510 con 0x55eb646d6720 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc7040f7b80 10 client.0 _ll_register_callbacks cb 0x55eb645c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44482/0 (socket says 172.21.15.46:44482) 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 -- 172.21.15.46:0/1875543711 learned_addr learned my addr 172.21.15.46:0/1875543711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6edc00700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ed200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 -- 172.21.15.46:0/1875543711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 msgr2=0x55eb646e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 -- 172.21.15.46:0/1875543711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 msgr2=0x55eb646df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 -- 172.21.15.46:0/1875543711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8110810 con 0x55eb646d6720 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6ee600700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00992f0 tx=0x7fc6e0009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844ee99fa05e79dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.981+0000 7fc6cf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0099850 con 0x55eb646d6720 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fc6e0099cf0 con 0x55eb646d6720 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6e00b9070 con 0x55eb646d6720 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6e00a1a70 con 0x55eb646d6720 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 10 client.27616 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eb644a5cd0 con 0x55eb646d6720 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e00b9070 con 0x55eb646d6720 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc6e00b7be0 con 0x55eb646d6720 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 10 client.27616 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 10 client.27616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 20 client.27616 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb647417b0 0x55eb64743bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 4 client.27616 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.982+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb64618ec0 con 0x55eb647417b0 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc6edc00700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb647417b0 0x55eb64743bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc6edc00700 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb647417b0 0x55eb64743bd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b62649d83eddea1d server_cookie=5ab5051f59f3fd3f in_seq=0 out_seq=0 2024-05-08T23:10:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc6cf400700 10 client.27616 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc6cf400700 1 -- 172.21.15.46:0/1875543711 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb64618ec0 con 0x55eb647417b0 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc6cf400700 10 client.27616 handle_command_reply: tid=0 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.983+0000 7fc7040f7b80 1 client.27616 shutdown 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 msgr2=0x55eb646dfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00992f0 tx=0x7fc6e0009240 comp rx=0 tx=0).stop 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 shutdown_connections 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb646d6720 0x55eb646dfcd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb646d0890 0x55eb646e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eb647417b0 0x55eb64743bd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 --2- 172.21.15.46:0/1875543711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb646d5e00 0x55eb646df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 >> 172.21.15.46:0/1875543711 conn(0x55eb6462e210 msgr2=0x55eb646d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc6cf400700 0 client.27616 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:10:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 shutdown_connections 2024-05-08T23:10:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 1 -- 172.21.15.46:0/1875543711 wait complete. 2024-05-08T23:10:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.984+0000 7fc7040f7b80 20 client.27616 trim_cache size 0 max 16384 2024-05-08T23:10:49.993 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 msgr2=0x7fc6d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 0x7fc6d00a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009fd0 tx=0x7fc6dc009f40 comp rx=0 tx=0).stop 2024-05-08T23:10:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 msgr2=0x7fc6e819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00094f0 tx=0x7fc6e000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 shutdown_connections 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e810d6d0 0x7fc6e819dea0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc6d00a1310 0x7fc6d00a37d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8073370 0x7fc6e81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 --2- 172.21.15.46:0/2102081992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6e8073c90 0x7fc6e8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 >> 172.21.15.46:0/2102081992 conn(0x7fc6e80fc940 msgr2=0x7fc6e80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 shutdown_connections 2024-05-08T23:10:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:49.985+0000 7fc6f0600700 1 -- 172.21.15.46:0/2102081992 wait complete. 2024-05-08T23:10:50.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:10:50.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.232+0000 7fca91a00700 1 -- 172.21.15.46:0/2006624700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101a00 msgr2=0x7fca8c105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.232+0000 7fca91a00700 1 --2- 172.21.15.46:0/2006624700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101a00 0x7fca8c105a40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fca78009990 tx=0x7fca78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 -- 172.21.15.46:0/2006624700 shutdown_connections 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 --2- 172.21.15.46:0/2006624700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 0x7fca8c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 --2- 172.21.15.46:0/2006624700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c101050 0x7fca8c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 --2- 172.21.15.46:0/2006624700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101a00 0x7fca8c105a40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 -- 172.21.15.46:0/2006624700 >> 172.21.15.46:0/2006624700 conn(0x7fca8c0fc8b0 msgr2=0x7fca8c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 -- 172.21.15.46:0/2006624700 shutdown_connections 2024-05-08T23:10:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.233+0000 7fca91a00700 1 -- 172.21.15.46:0/2006624700 wait complete. 2024-05-08T23:10:50.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.235+0000 7fca91a00700 1 Processor -- start 2024-05-08T23:10:50.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.235+0000 7fca91a00700 1 -- start start 2024-05-08T23:10:50.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.235+0000 7fca91a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca91a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 0x7fca8c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca91a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 0x7fca8c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca91a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19e930 con 0x7fca8c19deb0 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca91a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19eaa0 con 0x7fca8c106170 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca91a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19ec60 con 0x7fca8c101050 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44510/0 (socket says 172.21.15.46:44510) 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 -- 172.21.15.46:0/3605141081 learned_addr learned my addr 172.21.15.46:0/3605141081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8a000700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 0x7fca8c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8b400700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 0x7fca8c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 -- 172.21.15.46:0/3605141081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 msgr2=0x7fca8c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 0x7fca8c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 -- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 msgr2=0x7fca8c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 0x7fca8c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8aa00700 1 -- 172.21.15.46:0/3605141081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780095f0 con 0x7fca8c101050 2024-05-08T23:10:50.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.236+0000 7fca8b400700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 0x7fca8c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:10:50.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00e9a0 tx=0x7fca7c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca83400700 1 -- 172.21.15.46:0/3605141081 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c009790 con 0x7fca8c101050 2024-05-08T23:10:50.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca83400700 1 -- 172.21.15.46:0/3605141081 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7c00cbd0 con 0x7fca8c101050 2024-05-08T23:10:50.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c1a61c0 con 0x7fca8c101050 2024-05-08T23:10:50.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca83400700 1 -- 172.21.15.46:0/3605141081 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c018680 con 0x7fca8c101050 2024-05-08T23:10:50.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.237+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1a66f0 con 0x7fca8c101050 2024-05-08T23:10:50.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.240+0000 7fca83400700 1 -- 172.21.15.46:0/3605141081 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fca7c00cd40 con 0x7fca8c101050 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.242+0000 7fcaa54d2b80 1 Processor -- start 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 -- start start 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca83400700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 0x7fca700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca83400700 1 -- 172.21.15.46:0/3605141081 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca7c014070 con 0x7fca8c101050 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca8b400700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 0x7fca700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 0x562ba21b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 0x562ba21b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba210e150 con 0x562ba21bd720 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba210bb10 con 0x562ba21b7890 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba2107400 con 0x562ba21bce00 2024-05-08T23:10:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44518/0 (socket says 172.21.15.46:44518) 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.243+0000 7fca8aa00700 1 -- 172.21.15.46:0/2369443638 learned_addr learned my addr 172.21.15.46:0/2369443638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 0x562ba21b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8b400700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 0x562ba21b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 -- 172.21.15.46:0/2369443638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 msgr2=0x562ba21bd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 -- 172.21.15.46:0/2369443638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 msgr2=0x562ba21b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 0x562ba21b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 -- 172.21.15.46:0/2369443638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba1f883b0 con 0x562ba21bd720 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8aa00700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8b400700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 0x562ba21b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8b400700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 0x7fca700a37d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca840076f0 tx=0x7fca84007ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.244+0000 7fca8a000700 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 0x562ba21b7350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fca78000c00 tx=0x7fca78005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447b2e7f5467ba8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fca77e00700 1 -- 172.21.15.46:0/2369443638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780056a0 con 0x562ba21bd720 2024-05-08T23:10:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fca77e00700 1 -- 172.21.15.46:0/2369443638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7801e040 con 0x562ba21bd720 2024-05-08T23:10:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fca77e00700 1 -- 172.21.15.46:0/2369443638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78004850 con 0x562ba21bd720 2024-05-08T23:10:50.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/2369443638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 msgr2=0x562ba21b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 0x562ba21b7350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fca78000c00 tx=0x7fca78005ea0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/2369443638 shutdown_connections 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21b7890 0x562ba21b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bce00 0x562ba21bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.245+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/2369443638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bd720 0x562ba21b7350 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.246+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/2369443638 >> 172.21.15.46:0/2369443638 conn(0x562ba2115210 msgr2=0x562ba21b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.246+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/2369443638 shutdown_connections 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.246+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/2369443638 wait complete. 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.246+0000 7fcaa54d2b80 1 -- start start 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.246+0000 7fcaa54d2b80 1 Processor -- start 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba20a3560 con 0x562ba21b7890 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba20a3200 con 0x562ba21bce00 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba1f8a6a0 con 0x562ba21bd720 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fcaa54d2b80 10 client.0 _ll_register_callbacks cb 0x562ba20acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33516/0 (socket says 172.21.15.46:33516) 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fca8aa00700 1 -- 172.21.15.46:0/386293317 learned_addr learned my addr 172.21.15.46:0/386293317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fca8a000700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.247+0000 7fca8b400700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 -- 172.21.15.46:0/386293317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 msgr2=0x562ba21c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 -- 172.21.15.46:0/386293317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 msgr2=0x562ba21c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 -- 172.21.15.46:0/386293317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1a66f0 con 0x562ba21b7890 2024-05-08T23:10:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8b400700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca8aa00700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00c5b0 tx=0x7fca7c09a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc200cad47e7aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.248+0000 7fca74c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b1070 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca7c0a2570 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ba2060270 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c0995e0 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 10 client.17565 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fca8c10e560 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ba2060270 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca7c0b8220 con 0x562ba21b7890 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 10 client.17565 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 10 client.17565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 20 client.17565 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba21d1140 0x562ba21d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 4 client.17565 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:50.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.249+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ba2186810 con 0x562ba21d1140 2024-05-08T23:10:50.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.250+0000 7fca8a000700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba21d1140 0x562ba21d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.250+0000 7fca74c00700 10 client.17565 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:50.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.250+0000 7fca8a000700 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba21d1140 0x562ba21d3560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7dacbc1fb85e572 server_cookie=bbc676845606dd9a in_seq=0 out_seq=0 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fca74c00700 1 -- 172.21.15.46:0/386293317 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ba2186810 con 0x562ba21d1140 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fca74c00700 10 client.17565 handle_command_reply: tid=0 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 client.17565 shutdown 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 msgr2=0x562ba21c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00c5b0 tx=0x7fca7c09a230 comp rx=0 tx=0).stop 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 shutdown_connections 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba21d1140 0x562ba21d3560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21b7890 0x562ba21c3720 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/386293317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fca74c00700 0 client.17565 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.251+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 >> 172.21.15.46:0/386293317 conn(0x562ba2115210 msgr2=0x562ba21b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.252+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 shutdown_connections 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.252+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/386293317 wait complete. 2024-05-08T23:10:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.252+0000 7fcaa54d2b80 20 client.17565 trim_cache size 0 max 16384 2024-05-08T23:10:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 Processor -- start 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 -- start start 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 0x562ba21ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21aedc0 con 0x562ba21bb330 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21b3e50 con 0x562ba21bce00 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21bd560 con 0x562ba21bd720 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fca8a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.267+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34202/0 (socket says 172.21.15.46:34202) 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 -- 172.21.15.46:0/3741694904 learned_addr learned my addr 172.21.15.46:0/3741694904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8b400700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 0x562ba21ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 -- 172.21.15.46:0/3741694904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 msgr2=0x562ba21b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 -- 172.21.15.46:0/3741694904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 msgr2=0x562ba21ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 0x562ba21ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 -- 172.21.15.46:0/3741694904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba1f34b60 con 0x562ba21bce00 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8a000700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8b400700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 0x562ba21ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca8aa00700 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0cd360 tx=0x7fca7c099d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd4243b4ddd9e02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca80c00700 1 -- 172.21.15.46:0/3741694904 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b1020 con 0x562ba21bce00 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca80c00700 1 -- 172.21.15.46:0/3741694904 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7c0a2570 con 0x562ba21bce00 2024-05-08T23:10:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.268+0000 7fca80c00700 1 -- 172.21.15.46:0/3741694904 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0a2320 con 0x562ba21bce00 2024-05-08T23:10:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/3741694904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 msgr2=0x562ba21b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0cd360 tx=0x7fca7c099d60 comp rx=0 tx=0).stop 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/3741694904 shutdown_connections 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bb330 0x562ba21ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21bce00 0x562ba21b7350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/3741694904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/3741694904 >> 172.21.15.46:0/3741694904 conn(0x562ba2115210 msgr2=0x562ba21bac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/3741694904 shutdown_connections 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.269+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/3741694904 wait complete. 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.270+0000 7fcaa54d2b80 1 -- start start 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.270+0000 7fcaa54d2b80 1 Processor -- start 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21c87b0 0x562ba21b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21b9010 con 0x562ba21bce00 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21b9180 con 0x562ba21c87b0 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba21b9340 con 0x562ba21bd720 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fcaa54d2b80 10 client.0 _ll_register_callbacks cb 0x562ba20acb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33544/0 (socket says 172.21.15.46:33544) 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8aa00700 1 -- 172.21.15.46:0/4158345899 learned_addr learned my addr 172.21.15.46:0/4158345899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8b400700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21c87b0 0x562ba21b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8aa00700 1 -- 172.21.15.46:0/4158345899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 msgr2=0x562ba21c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.271+0000 7fca8aa00700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca8aa00700 1 -- 172.21.15.46:0/4158345899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21c87b0 msgr2=0x562ba21b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca8aa00700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21c87b0 0x562ba21b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca8aa00700 1 -- 172.21.15.46:0/4158345899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba1f883b0 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca8aa00700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0a2040 tx=0x7fca7c099810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360bb0769aae607d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca74c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b1030 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fca7c0b8330 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca7801e040 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c0990a0 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fcaa54d2b80 10 client.17571 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.272+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fca8c10e560 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7801e040 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca7c09ee40 con 0x562ba21bce00 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 10 client.17571 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 10 client.17571 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 20 client.17571 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba2228330 0x562ba222a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 4 client.17571 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ba20ffec0 con 0x562ba2228330 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.273+0000 7fca8a000700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba2228330 0x562ba222a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fca8a000700 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba2228330 0x562ba222a720 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc49f142c3d7d4b2 server_cookie=cfaea2f165e013d4 in_seq=0 out_seq=0 2024-05-08T23:10:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fca74c00700 10 client.17571 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fca74c00700 1 -- 172.21.15.46:0/4158345899 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ba20ffec0 con 0x562ba2228330 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fca74c00700 10 client.17571 handle_command_reply: tid=0 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 client.17571 shutdown 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 msgr2=0x562ba21ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0a2040 tx=0x7fca7c099810 comp rx=0 tx=0).stop 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 shutdown_connections 2024-05-08T23:10:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ba21c87b0 0x562ba21b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562ba2228330 0x562ba222a720 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba21bce00 0x562ba21ba3a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 --2- 172.21.15.46:0/4158345899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ba21bd720 0x562ba21c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fca74c00700 0 client.17571 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 >> 172.21.15.46:0/4158345899 conn(0x562ba2115210 msgr2=0x562ba21c0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 shutdown_connections 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.274+0000 7fcaa54d2b80 1 -- 172.21.15.46:0/4158345899 wait complete. 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fcaa54d2b80 20 client.17571 trim_cache size 0 max 16384 2024-05-08T23:10:50.278 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 msgr2=0x7fca700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 0x7fca700a37d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca840076f0 tx=0x7fca84007ab0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 msgr2=0x7fca8c1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00e9a0 tx=0x7fca7c00ed60 comp rx=0 tx=0).stop 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 shutdown_connections 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fca700a1310 0x7fca700a37d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c19deb0 0x7fca8c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8c101050 0x7fca8c1995d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 --2- 172.21.15.46:0/3605141081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca8c106170 0x7fca8c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 >> 172.21.15.46:0/3605141081 conn(0x7fca8c0fc8b0 msgr2=0x7fca8c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 shutdown_connections 2024-05-08T23:10:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.275+0000 7fca91a00700 1 -- 172.21.15.46:0/3605141081 wait complete. 2024-05-08T23:10:50.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:10:50.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.525+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1930219313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc78101fe0 msgr2=0x7fdc781023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.525+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1930219313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc78101fe0 0x7fdc781023c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009990 tx=0x7fdc68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1930219313 shutdown_connections 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1930219313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc7810b400 0x7fdc7810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1930219313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc78101fe0 0x7fdc781023c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1930219313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78102900 0x7fdc7810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1930219313 >> 172.21.15.46:0/1930219313 conn(0x7fdc780fb760 msgr2=0x7fdc780fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1930219313 shutdown_connections 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.526+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1930219313 wait complete. 2024-05-08T23:10:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.527+0000 7fdc7ec00700 1 Processor -- start 2024-05-08T23:10:50.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.527+0000 7fdc7ec00700 1 -- start start 2024-05-08T23:10:50.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78101fe0 0x7fdc78199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 0x7fdc78199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 0x7fdc7819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7819e450 con 0x7fdc78101fe0 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7819e5c0 con 0x7fdc78102900 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7819e780 con 0x7fdc7810b400 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 0x7fdc78199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 0x7fdc7819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78101fe0 0x7fdc78199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 0x7fdc78199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34236/0 (socket says 172.21.15.46:34236) 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc77400700 1 -- 172.21.15.46:0/1229135583 learned_addr learned my addr 172.21.15.46:0/1229135583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7cc00700 1 -- 172.21.15.46:0/1229135583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 msgr2=0x7fdc78199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.528+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 0x7fdc78199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7cc00700 1 -- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78101fe0 msgr2=0x7fdc78199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78101fe0 0x7fdc78199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7cc00700 1 -- 172.21.15.46:0/1229135583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680095f0 con 0x7fdc7810b400 2024-05-08T23:10:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 0x7fdc7819de10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdc70007f10 tx=0x7fdc7000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc74c00700 1 -- 172.21.15.46:0/1229135583 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70009ea0 con 0x7fdc7810b400 2024-05-08T23:10:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc74c00700 1 -- 172.21.15.46:0/1229135583 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7000f040 con 0x7fdc7810b400 2024-05-08T23:10:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7819ea60 con 0x7fdc7810b400 2024-05-08T23:10:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc74c00700 1 -- 172.21.15.46:0/1229135583 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70004350 con 0x7fdc7810b400 2024-05-08T23:10:50.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.529+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc781a62a0 con 0x7fdc7810b400 2024-05-08T23:10:50.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.533+0000 7fdc74c00700 1 -- 172.21.15.46:0/1229135583 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc700044b0 con 0x7fdc7810b400 2024-05-08T23:10:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.534+0000 7fdc92803b80 1 Processor -- start 2024-05-08T23:10:50.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.534+0000 7fdc92803b80 1 -- start start 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.534+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 0x55d94fa3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa37cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc92803b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f98e150 con 0x55d94fa3ce00 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc92803b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f98bb10 con 0x55d94fa3d720 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc92803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f987400 con 0x55d94fa37890 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34244/0 (socket says 172.21.15.46:34244) 2024-05-08T23:10:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 -- 172.21.15.46:0/161563059 learned_addr learned my addr 172.21.15.46:0/161563059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 -- 172.21.15.46:0/161563059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 msgr2=0x55d94fa37cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa37cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.535+0000 7fdc77e00700 1 -- 172.21.15.46:0/161563059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 msgr2=0x55d94fa3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77400700 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 0x55d94fa3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77e00700 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 0x55d94fa3d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77e00700 1 -- 172.21.15.46:0/161563059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d94f8083b0 con 0x55d94fa3d720 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77400700 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 0x55d94fa3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:10:50.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc74c00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 0x7fdc5c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc74c00700 1 -- 172.21.15.46:0/1229135583 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdc700cc880 con 0x7fdc7810b400 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77400700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 0x7fdc5c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc77e00700 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800b610 tx=0x7fdc68005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cde1e9fc1e17bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.536+0000 7fdc6d200700 1 -- 172.21.15.46:0/161563059 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6801d070 con 0x55d94fa3d720 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc6d200700 1 -- 172.21.15.46:0/161563059 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6801e040 con 0x55d94fa3d720 2024-05-08T23:10:50.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc6d200700 1 -- 172.21.15.46:0/161563059 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004940 con 0x55d94fa3d720 2024-05-08T23:10:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 -- 172.21.15.46:0/161563059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 msgr2=0x55d94fa37350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc77400700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 0x7fdc5c0a37d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc7819aa80 tx=0x7fdc64009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800b610 tx=0x7fdc68005bc0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 -- 172.21.15.46:0/161563059 shutdown_connections 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa37cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3ce00 0x55d94fa3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 --2- 172.21.15.46:0/161563059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa37350 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 -- 172.21.15.46:0/161563059 >> 172.21.15.46:0/161563059 conn(0x55d94f995210 msgr2=0x55d94fa31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.537+0000 7fdc92803b80 1 -- 172.21.15.46:0/161563059 shutdown_connections 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.538+0000 7fdc92803b80 1 -- 172.21.15.46:0/161563059 wait complete. 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.538+0000 7fdc92803b80 1 -- start start 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.538+0000 7fdc92803b80 1 Processor -- start 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3ce00 0x55d94fa410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa41610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f923560 con 0x55d94fa37890 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f923200 con 0x55d94fa3d720 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94f80a6a0 con 0x55d94fa3ce00 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc92803b80 10 client.0 _ll_register_callbacks cb 0x55d94f92cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33582/0 (socket says 172.21.15.46:33582) 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc77e00700 1 -- 172.21.15.46:0/3735117288 learned_addr learned my addr 172.21.15.46:0/3735117288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc77400700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3ce00 0x55d94fa410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.539+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa41610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 -- 172.21.15.46:0/3735117288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3ce00 msgr2=0x55d94fa410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3ce00 0x55d94fa410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 -- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 msgr2=0x55d94fa41610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa41610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 -- 172.21.15.46:0/3735117288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7810e530 con 0x55d94fa37890 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c00 tx=0x7fdc68004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b6e0e791f6e419 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc61600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6801d030 con 0x55d94fa37890 2024-05-08T23:10:50.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdc68004840 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.540+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d94f8e0270 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc68004a00 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 10 client.17580 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d94f7b4b60 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d94f8e0270 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc6800fc50 con 0x55d94fa37890 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 10 client.17580 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 20 client.17580 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94fa50d30 0x55d94fa53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 4 client.17580 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.541+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d94fa06810 con 0x55d94fa50d30 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.542+0000 7fdc77400700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94fa50d30 0x55d94fa53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.542+0000 7fdc77400700 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94fa50d30 0x55d94fa53150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2ee8df7cbc6be3 server_cookie=2c6660d33924a6d5 in_seq=0 out_seq=0 2024-05-08T23:10:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.542+0000 7fdc61600700 10 client.17580 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.543+0000 7fdc61600700 1 -- 172.21.15.46:0/3735117288 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d94fa06810 con 0x55d94fa50d30 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.543+0000 7fdc61600700 10 client.17580 handle_command_reply: tid=0 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.543+0000 7fdc92803b80 1 client.17580 shutdown 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 msgr2=0x55d94fa40b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c00 tx=0x7fdc68004500 comp rx=0 tx=0).stop 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 shutdown_connections 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3d720 0x55d94fa41610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94fa50d30 0x55d94fa53150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa40b90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3735117288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3ce00 0x55d94fa410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc61600700 0 client.17580 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 >> 172.21.15.46:0/3735117288 conn(0x55d94f995210 msgr2=0x55d94fa315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 shutdown_connections 2024-05-08T23:10:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 1 -- 172.21.15.46:0/3735117288 wait complete. 2024-05-08T23:10:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.544+0000 7fdc92803b80 20 client.17580 trim_cache size 0 max 16384 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 Processor -- start 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 -- start start 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3d720 0x55d94fa486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94fa32e80 con 0x55d94fa3d720 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94fa33070 con 0x55d94fa3ce00 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc92803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94fa32ca0 con 0x55d94fa37890 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.561+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44584/0 (socket says 172.21.15.46:44584) 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 -- 172.21.15.46:0/3488335947 learned_addr learned my addr 172.21.15.46:0/3488335947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3d720 0x55d94fa486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 -- 172.21.15.46:0/3488335947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 msgr2=0x55d94fa48170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa48170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 -- 172.21.15.46:0/3488335947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3d720 msgr2=0x55d94fa486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3d720 0x55d94fa486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 -- 172.21.15.46:0/3488335947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d94f94d560 con 0x55d94fa37890 2024-05-08T23:10:50.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc77e00700 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004570 tx=0x7fdc6800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c87831ad7447ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc6dc00700 1 -- 172.21.15.46:0/3488335947 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6801d070 con 0x55d94fa37890 2024-05-08T23:10:50.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc6dc00700 1 -- 172.21.15.46:0/3488335947 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6800e3e0 con 0x55d94fa37890 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.562+0000 7fdc6dc00700 1 -- 172.21.15.46:0/3488335947 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800eb30 con 0x55d94fa37890 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 -- 172.21.15.46:0/3488335947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 msgr2=0x55d94fa31d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004570 tx=0x7fdc6800b260 comp rx=0 tx=0).stop 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 -- 172.21.15.46:0/3488335947 shutdown_connections 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa3d720 0x55d94fa486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa37890 0x55d94fa31d20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 --2- 172.21.15.46:0/3488335947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa48170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 -- 172.21.15.46:0/3488335947 >> 172.21.15.46:0/3488335947 conn(0x55d94f995210 msgr2=0x55d94fa41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 -- 172.21.15.46:0/3488335947 shutdown_connections 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.563+0000 7fdc92803b80 1 -- 172.21.15.46:0/3488335947 wait complete. 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 -- start start 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 Processor -- start 2024-05-08T23:10:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94fa464b0 con 0x55d94fa37890 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94fa46620 con 0x55d94fa3ce00 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.564+0000 7fdc92803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94faa6230 con 0x55d94fa3d720 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc92803b80 10 client.0 _ll_register_callbacks cb 0x55d94f92cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc77e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33598/0 (socket says 172.21.15.46:33598) 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:44592/0 (socket says 172.21.15.46:44592) 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc77e00700 1 -- 172.21.15.46:0/2555530933 learned_addr learned my addr 172.21.15.46:0/2555530933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc77400700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 -- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 msgr2=0x55d94fa45880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa45880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 -- 172.21.15.46:0/2555530933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 msgr2=0x55d94fa484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 -- 172.21.15.46:0/2555530933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d94f7b4b60 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc7cc00700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdc700c9bc0 tx=0x7fdc70099140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49692fd38be25f89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.565+0000 7fdc61600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700b3060 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fdc70095040 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d94f97fec0 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc70099c80 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 10 client.27649 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d94f80ccd0 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d94f97fec0 con 0x55d94fa3d720 2024-05-08T23:10:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc70095c90 con 0x55d94fa3d720 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 10 client.27649 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 10 client.27649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 20 client.27649 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94faa78a0 0x55d94faa9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 4 client.27649 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:10:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.566+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d94f7d5cf0 con 0x55d94faa78a0 2024-05-08T23:10:50.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.567+0000 7fdc77e00700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94faa78a0 0x55d94faa9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:50.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.567+0000 7fdc77e00700 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94faa78a0 0x55d94faa9cc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177added291508e4 server_cookie=eb7059ea0c807f47 in_seq=0 out_seq=0 2024-05-08T23:10:50.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.567+0000 7fdc61600700 10 client.27649 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.567+0000 7fdc61600700 1 -- 172.21.15.46:0/2555530933 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d94f7d5cf0 con 0x55d94faa78a0 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.567+0000 7fdc61600700 10 client.27649 handle_command_reply: tid=0 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 client.27649 shutdown 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 msgr2=0x55d94fa45dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdc700c9bc0 tx=0x7fdc70099140 comp rx=0 tx=0).stop 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 shutdown_connections 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d94fa3d720 0x55d94fa45dc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d94faa78a0 0x55d94faa9cc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94fa37890 0x55d94fa484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 --2- 172.21.15.46:0/2555530933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d94fa3ce00 0x55d94fa45880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc61600700 0 client.27649 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 >> 172.21.15.46:0/2555530933 conn(0x55d94f995210 msgr2=0x55d94fa41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 shutdown_connections 2024-05-08T23:10:50.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 1 -- 172.21.15.46:0/2555530933 wait complete. 2024-05-08T23:10:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.568+0000 7fdc92803b80 20 client.27649 trim_cache size 0 max 16384 2024-05-08T23:10:50.572 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:10:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 msgr2=0x7fdc5c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 0x7fdc5c0a37d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc7819aa80 tx=0x7fdc64009260 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 msgr2=0x7fdc7819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 0x7fdc7819de10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdc70007f10 tx=0x7fdc7000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 shutdown_connections 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7810b400 0x7fdc7819de10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc5c0a1310 0x7fdc5c0a37d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc78101fe0 0x7fdc78199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 --2- 172.21.15.46:0/1229135583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102900 0x7fdc78199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 >> 172.21.15.46:0/1229135583 conn(0x7fdc780fb760 msgr2=0x7fdc7810c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 shutdown_connections 2024-05-08T23:10:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:50.569+0000 7fdc7ec00700 1 -- 172.21.15.46:0/1229135583 wait complete. 2024-05-08T23:10:50.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 571971 10.22 MB/sec execute 192 sec latency 16.456 ms 2024-05-08T23:10:51.591 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:10:51.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:10:51.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:51 smithi086 ceph-mon[78610]: pgmap v214: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-08T23:10:51.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:51 smithi053 ceph-mon[77587]: pgmap v214: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-08T23:10:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:51 smithi046 ceph-mon[36898]: pgmap v214: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-08T23:10:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.817+0000 7ff923e00700 1 -- 172.21.15.46:0/1030324169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c101080 msgr2=0x7ff91c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.817+0000 7ff923e00700 1 --2- 172.21.15.46:0/1030324169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c101080 0x7ff91c101460 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff910009990 tx=0x7ff910009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 -- 172.21.15.46:0/1030324169 shutdown_connections 2024-05-08T23:10:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 --2- 172.21.15.46:0/1030324169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c1061a0 0x7ff91c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 --2- 172.21.15.46:0/1030324169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c101080 0x7ff91c101460 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 --2- 172.21.15.46:0/1030324169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101a30 0x7ff91c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 -- 172.21.15.46:0/1030324169 >> 172.21.15.46:0/1030324169 conn(0x7ff91c0fc8e0 msgr2=0x7ff91c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 -- 172.21.15.46:0/1030324169 shutdown_connections 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.818+0000 7ff923e00700 1 -- 172.21.15.46:0/1030324169 wait complete. 2024-05-08T23:10:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.819+0000 7ff923e00700 1 Processor -- start 2024-05-08T23:10:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.819+0000 7ff923e00700 1 -- start start 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 0x7ff91c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 0x7ff91c1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a5b50 con 0x7ff91c073d20 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a5cc0 con 0x7ff91c101080 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff923e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1a5e80 con 0x7ff91c1061a0 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff921400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 0x7ff91c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff921400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47930/0 (socket says 172.21.15.46:47930) 2024-05-08T23:10:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff921400700 1 -- 172.21.15.46:0/2062691421 learned_addr learned my addr 172.21.15.46:0/2062691421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.820+0000 7ff921e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 0x7ff91c1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 -- 172.21.15.46:0/2062691421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 msgr2=0x7ff91c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 0x7ff91c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 -- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 msgr2=0x7ff91c1a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 0x7ff91c1a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 -- 172.21.15.46:0/2062691421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100095f0 con 0x7ff91c101080 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 0x7ff91c1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff920a00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 0x7ff91c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff921400700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff910005910 tx=0x7ff910004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff91a000700 1 -- 172.21.15.46:0/2062691421 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91001d070 con 0x7ff91c101080 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c1a6100 con 0x7ff91c101080 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff91a000700 1 -- 172.21.15.46:0/2062691421 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff910022470 con 0x7ff91c101080 2024-05-08T23:10:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff91a000700 1 -- 172.21.15.46:0/2062691421 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910017650 con 0x7ff91c101080 2024-05-08T23:10:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.821+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c1a6580 con 0x7ff91c101080 2024-05-08T23:10:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.825+0000 7ff91a000700 1 -- 172.21.15.46:0/2062691421 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff9100225e0 con 0x7ff91c101080 2024-05-08T23:10:51.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.826+0000 7ff937952b80 1 Processor -- start 2024-05-08T23:10:51.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.826+0000 7ff937952b80 1 -- start start 2024-05-08T23:10:51.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.826+0000 7ff91a000700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 0x7ff9040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff91a000700 1 -- 172.21.15.46:0/2062691421 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff9100ce2e0 con 0x7ff91c101080 2024-05-08T23:10:51.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847be80 0x562a1847c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a18476d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a183cd1d0 con 0x562a18476910 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a183cab90 con 0x562a1847be80 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff937952b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a183c6480 con 0x562a1847c7a0 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff921e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 0x7ff9040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39442/0 (socket says 172.21.15.46:39442) 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 -- 172.21.15.46:0/1335593559 learned_addr learned my addr 172.21.15.46:0/1335593559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff921400700 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847be80 0x562a1847c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff921e00700 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a18476d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 -- 172.21.15.46:0/1335593559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847be80 msgr2=0x562a1847c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847be80 0x562a1847c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 -- 172.21.15.46:0/1335593559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 msgr2=0x562a18476d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a18476d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.827+0000 7ff920a00700 1 -- 172.21.15.46:0/1335593559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a18247430 con 0x562a1847c7a0 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.828+0000 7ff920a00700 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9080114f0 tx=0x7ff908011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ff68db0c7f1ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.828+0000 7ff921e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 0x7ff9040a37d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff914009fd0 tx=0x7ff914009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.828+0000 7ff90e600700 1 -- 172.21.15.46:0/1335593559 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908004310 con 0x562a1847c7a0 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.828+0000 7ff90e600700 1 -- 172.21.15.46:0/1335593559 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff908004470 con 0x562a1847c7a0 2024-05-08T23:10:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.828+0000 7ff90e600700 1 -- 172.21.15.46:0/1335593559 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908019910 con 0x562a1847c7a0 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- 172.21.15.46:0/1335593559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 msgr2=0x562a184763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9080114f0 tx=0x7ff908011800 comp rx=0 tx=0).stop 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- 172.21.15.46:0/1335593559 shutdown_connections 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a18476d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847be80 0x562a1847c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 --2- 172.21.15.46:0/1335593559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847c7a0 0x562a184763d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- 172.21.15.46:0/1335593559 >> 172.21.15.46:0/1335593559 conn(0x562a183d4290 msgr2=0x562a18470e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- 172.21.15.46:0/1335593559 shutdown_connections 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- 172.21.15.46:0/1335593559 wait complete. 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.829+0000 7ff937952b80 1 -- start start 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 Processor -- start 2024-05-08T23:10:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a1847fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847c7a0 0x562a18480600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a183625e0 con 0x562a18476910 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18362280 con 0x562a1847c7a0 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18249720 con 0x562a1847be80 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff937952b80 10 client.0 _ll_register_callbacks cb 0x562a1836bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff921400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a1847fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff921e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847c7a0 0x562a18480600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff921400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a1847fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42422/0 (socket says 172.21.15.46:42422) 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff920a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39458/0 (socket says 172.21.15.46:39458) 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.830+0000 7ff921400700 1 -- 172.21.15.46:0/291231702 learned_addr learned my addr 172.21.15.46:0/291231702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 -- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847c7a0 msgr2=0x562a18480600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847c7a0 0x562a18480600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 -- 172.21.15.46:0/291231702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 msgr2=0x562a1847fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a1847fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 -- 172.21.15.46:0/291231702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908009cd0 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff920a00700 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff908014040 tx=0x7ff908005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4512bffa8fa9fb5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff902a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908003a10 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff908003b70 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.831+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a1831f2f0 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff908003d50 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 10 client.27661 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562a181f3be0 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a1831f2f0 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff90801a6b0 con 0x562a1847be80 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 10 client.27661 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 10 client.27661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 20 client.27661 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a18490140 0x562a18492530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 4 client.27661 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.832+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a18445890 con 0x562a18490140 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.833+0000 7ff921400700 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a18490140 0x562a18492530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.833+0000 7ff921400700 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a18490140 0x562a18492530 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace6eb582530cdfb server_cookie=9ac3c20268dfc684 in_seq=0 out_seq=0 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.833+0000 7ff902a00700 10 client.27661 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff902a00700 1 -- 172.21.15.46:0/291231702 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a18445890 con 0x562a18490140 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff902a00700 10 client.27661 handle_command_reply: tid=0 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 client.27661 shutdown 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 msgr2=0x562a184800c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff908014040 tx=0x7ff908005df0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 shutdown_connections 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a1847c7a0 0x562a18480600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a18490140 0x562a18492530 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a18476910 0x562a1847fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 --2- 172.21.15.46:0/291231702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a184800c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff902a00700 0 client.27661 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 >> 172.21.15.46:0/291231702 conn(0x562a183d4290 msgr2=0x562a18470620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 shutdown_connections 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 1 -- 172.21.15.46:0/291231702 wait complete. 2024-05-08T23:10:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.834+0000 7ff937952b80 20 client.27661 trim_cache size 0 max 16384 2024-05-08T23:10:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 Processor -- start 2024-05-08T23:10:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 -- start start 2024-05-08T23:10:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a183d4290 0x562a1847eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a1847b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18482cf0 con 0x562a183d4290 2024-05-08T23:10:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18482ed0 con 0x562a18476910 2024-05-08T23:10:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff937952b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a184826f0 con 0x562a1847be80 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.855+0000 7ff921e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a183d4290 0x562a1847eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39468/0 (socket says 172.21.15.46:39468) 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 -- 172.21.15.46:0/3559557083 learned_addr learned my addr 172.21.15.46:0/3559557083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff920a00700 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a1847b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 -- 172.21.15.46:0/3559557083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 msgr2=0x562a1847b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a1847b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 -- 172.21.15.46:0/3559557083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a183d4290 msgr2=0x562a1847eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a183d4290 0x562a1847eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 -- 172.21.15.46:0/3559557083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1838c5e0 con 0x562a1847be80 2024-05-08T23:10:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.856+0000 7ff921e00700 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff9140090c0 tx=0x7ff91400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f8f567c98ef788 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff90f000700 1 -- 172.21.15.46:0/3559557083 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91401a700 con 0x562a1847be80 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff90f000700 1 -- 172.21.15.46:0/3559557083 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91400f040 con 0x562a1847be80 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff90f000700 1 -- 172.21.15.46:0/3559557083 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914024bf0 con 0x562a1847be80 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 -- 172.21.15.46:0/3559557083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 msgr2=0x562a18470da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff9140090c0 tx=0x7ff91400b040 comp rx=0 tx=0).stop 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 -- 172.21.15.46:0/3559557083 shutdown_connections 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a1847be80 0x562a18470da0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a183d4290 0x562a1847eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 --2- 172.21.15.46:0/3559557083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a1847b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.857+0000 7ff937952b80 1 -- 172.21.15.46:0/3559557083 >> 172.21.15.46:0/3559557083 conn(0x562a18477050 msgr2=0x562a18481420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.858+0000 7ff937952b80 1 -- 172.21.15.46:0/3559557083 shutdown_connections 2024-05-08T23:10:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.858+0000 7ff937952b80 1 -- 172.21.15.46:0/3559557083 wait complete. 2024-05-08T23:10:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.858+0000 7ff937952b80 1 -- start start 2024-05-08T23:10:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.858+0000 7ff937952b80 1 Processor -- start 2024-05-08T23:10:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a1847be80 0x562a184865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18486cb0 con 0x562a1847be80 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18486e20 con 0x562a18476910 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a18486fe0 con 0x562a183d4290 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff937952b80 10 client.0 _ll_register_callbacks cb 0x562a1836bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff921400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff921400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39470/0 (socket says 172.21.15.46:39470) 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47968/0 (socket says 172.21.15.46:47968) 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff921400700 1 -- 172.21.15.46:0/2703019729 learned_addr learned my addr 172.21.15.46:0/2703019729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff921e00700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a1847be80 0x562a184865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 -- 172.21.15.46:0/2703019729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 msgr2=0x562a184df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 -- 172.21.15.46:0/2703019729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a1847be80 msgr2=0x562a184865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.859+0000 7ff920a00700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a1847be80 0x562a184865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff920a00700 1 -- 172.21.15.46:0/2703019729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff914008e30 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff921400700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff920a00700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff908011b50 tx=0x7ff908011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c0eea6cfe788e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff902a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908011b80 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff908004470 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7ff908000e40 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff908003c20 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff937952b80 10 client.27599 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.860+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562a1824bd50 con 0x562a18476910 2024-05-08T23:10:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff908004470 con 0x562a18476910 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff908019940 con 0x562a18476910 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 10 client.27599 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 10 client.27599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 20 client.27599 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a1847c7a0 0x562a184ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 4 client.27599 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562a183bef40 con 0x562a1847c7a0 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.861+0000 7ff921400700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a1847c7a0 0x562a184ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.862+0000 7ff921400700 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a1847c7a0 0x562a184ea6f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d114a88c0240612 server_cookie=dd8665714ec5db07 in_seq=0 out_seq=0 2024-05-08T23:10:51.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.862+0000 7ff902a00700 10 client.27599 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.863+0000 7ff902a00700 1 -- 172.21.15.46:0/2703019729 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562a183bef40 con 0x562a1847c7a0 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.863+0000 7ff902a00700 10 client.27599 handle_command_reply: tid=0 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.863+0000 7ff937952b80 1 client.27599 shutdown 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 msgr2=0x562a184dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff908011b50 tx=0x7ff908011800 comp rx=0 tx=0).stop 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 shutdown_connections 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a1847c7a0 0x562a184ea6f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a1847be80 0x562a184865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a183d4290 0x562a184df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 --2- 172.21.15.46:0/2703019729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a18476910 0x562a184dfee0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff902a00700 0 client.27599 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 >> 172.21.15.46:0/2703019729 conn(0x562a18477050 msgr2=0x562a1847d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 shutdown_connections 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 1 -- 172.21.15.46:0/2703019729 wait complete. 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.864+0000 7ff937952b80 20 client.27599 trim_cache size 0 max 16384 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "f6206d85-8813-4a8a-9a9f-92069e84da1e", 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:10:51.868 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 msgr2=0x7ff9040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 0x7ff9040a37d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff914009fd0 tx=0x7ff914009920 comp rx=0 tx=0).stop 2024-05-08T23:10:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 msgr2=0x7ff91c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff910005910 tx=0x7ff910004730 comp rx=0 tx=0).stop 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 shutdown_connections 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9040a1310 0x7ff9040a37d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c073d20 0x7ff91c1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff91c101080 0x7ff91c0732a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 --2- 172.21.15.46:0/2062691421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff91c1061a0 0x7ff91c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 >> 172.21.15.46:0/2062691421 conn(0x7ff91c0fc8e0 msgr2=0x7ff91c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 shutdown_connections 2024-05-08T23:10:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:51.865+0000 7ff923e00700 1 -- 172.21.15.46:0/2062691421 wait complete. 2024-05-08T23:10:51.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:10:51.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 573380 10.21 MB/sec execute 193 sec latency 16.878 ms 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 -- 172.21.15.46:0/143045800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38073c60 msgr2=0x7fea381118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 --2- 172.21.15.46:0/143045800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38073c60 0x7fea381118e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fea24009990 tx=0x7fea24009ca0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 -- 172.21.15.46:0/143045800 shutdown_connections 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 --2- 172.21.15.46:0/143045800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38111e20 0x7fea38114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 --2- 172.21.15.46:0/143045800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073340 0x7fea38073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 --2- 172.21.15.46:0/143045800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38073c60 0x7fea381118e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 -- 172.21.15.46:0/143045800 >> 172.21.15.46:0/143045800 conn(0x7fea380fc8e0 msgr2=0x7fea380fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.108+0000 7fea3fe00700 1 -- 172.21.15.46:0/143045800 shutdown_connections 2024-05-08T23:10:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.109+0000 7fea3fe00700 1 -- 172.21.15.46:0/143045800 wait complete. 2024-05-08T23:10:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 Processor -- start 2024-05-08T23:10:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 -- start start 2024-05-08T23:10:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073c60 0x7fea381a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38111e20 0x7fea3819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3819de10 con 0x7fea38111e20 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3819df80 con 0x7fea38073340 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.110+0000 7fea3fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3819e140 con 0x7fea38073c60 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47980/0 (socket says 172.21.15.46:47980) 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 -- 172.21.15.46:0/2762599501 learned_addr learned my addr 172.21.15.46:0/2762599501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3ca00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073c60 0x7fea381a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3de00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38111e20 0x7fea3819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 -- 172.21.15.46:0/2762599501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073c60 msgr2=0x7fea381a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073c60 0x7fea381a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 -- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38111e20 msgr2=0x7fea3819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38111e20 0x7fea3819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 -- 172.21.15.46:0/2762599501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240095f0 con 0x7fea38073340 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.111+0000 7fea3d400700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9c0 tx=0x7fea2c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.112+0000 7fea36000700 1 -- 172.21.15.46:0/2762599501 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0097b0 con 0x7fea38073340 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.112+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3819e420 con 0x7fea38073340 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.112+0000 7fea36000700 1 -- 172.21.15.46:0/2762599501 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c00ee60 con 0x7fea38073340 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.112+0000 7fea36000700 1 -- 172.21.15.46:0/2762599501 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0054f0 con 0x7fea38073340 2024-05-08T23:10:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.112+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea381aa660 con 0x7fea38073340 2024-05-08T23:10:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.115+0000 7fea36000700 1 -- 172.21.15.46:0/2762599501 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fea2c004750 con 0x7fea38073340 2024-05-08T23:10:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.117+0000 7fea5392eb80 1 Processor -- start 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 -- start start 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea36000700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 0x7fea200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea36000700 1 -- 172.21.15.46:0/2762599501 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fea2c014070 con 0x7fea38073340 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a206350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a206cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a15d150 con 0x55925a20c720 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a15ab10 con 0x55925a20be00 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea5392eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a156400 con 0x55925a206890 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea3de00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 0x7fea200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.118+0000 7fea3d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47994/0 (socket says 172.21.15.46:47994) 2024-05-08T23:10:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 -- 172.21.15.46:0/1992253243 learned_addr learned my addr 172.21.15.46:0/1992253243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3ca00700 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a206350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3de00700 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a206cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 -- 172.21.15.46:0/1992253243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 msgr2=0x55925a206cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a206cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 -- 172.21.15.46:0/1992253243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 msgr2=0x55925a206350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a206350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 -- 172.21.15.46:0/1992253243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559259fd73b0 con 0x55925a20be00 2024-05-08T23:10:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3de00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 0x7fea200a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea381a4370 tx=0x7fea3000fc00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea3d400700 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fea2c096ad0 tx=0x7fea2c09a830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7acbb327bbbbdb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea2a600700 1 -- 172.21.15.46:0/1992253243 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0cd160 con 0x55925a20be00 2024-05-08T23:10:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea2a600700 1 -- 172.21.15.46:0/1992253243 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c0a24e0 con 0x55925a20be00 2024-05-08T23:10:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.119+0000 7fea2a600700 1 -- 172.21.15.46:0/1992253243 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c016330 con 0x55925a20be00 2024-05-08T23:10:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 -- 172.21.15.46:0/1992253243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 msgr2=0x55925a20c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fea2c096ad0 tx=0x7fea2c09a830 comp rx=0 tx=0).stop 2024-05-08T23:10:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 -- 172.21.15.46:0/1992253243 shutdown_connections 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a206cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20be00 0x55925a20c1e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 --2- 172.21.15.46:0/1992253243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a206350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 -- 172.21.15.46:0/1992253243 >> 172.21.15.46:0/1992253243 conn(0x55925a164210 msgr2=0x55925a200df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 -- 172.21.15.46:0/1992253243 shutdown_connections 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.120+0000 7fea5392eb80 1 -- 172.21.15.46:0/1992253243 wait complete. 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.121+0000 7fea5392eb80 1 -- start start 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.121+0000 7fea5392eb80 1 Processor -- start 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 0x55925a20fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a210510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a0f2560 con 0x55925a20c720 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a0f2200 con 0x55925a206890 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559259fd96a0 con 0x55925a20be00 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea3ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea5392eb80 10 client.0 _ll_register_callbacks cb 0x55925a0fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea3ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39510/0 (socket says 172.21.15.46:39510) 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea3ca00700 1 -- 172.21.15.46:0/557178135 learned_addr learned my addr 172.21.15.46:0/557178135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea3de00700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a210510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.122+0000 7fea3d400700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 0x55925a20fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 -- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 msgr2=0x55925a20fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 0x55925a20fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 -- 172.21.15.46:0/557178135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 msgr2=0x55925a210510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a210510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 -- 172.21.15.46:0/557178135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea381aa660 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3d400700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 0x55925a20fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea3ca00700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fea24009990 tx=0x7fea24005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65752488458fd7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea1ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.123+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2401d070 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55925a0af270 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fea24004500 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2402e8b0 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 10 client.27679 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559259f83b60 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55925a0af270 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea24004c40 con 0x55925a20be00 2024-05-08T23:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 10 client.27679 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 20 client.27679 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a2201f0 0x55925a222610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 4 client.27679 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.124+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55925a1d5810 con 0x55925a2201f0 2024-05-08T23:10:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.125+0000 7fea3d400700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a2201f0 0x55925a222610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.125+0000 7fea3d400700 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a2201f0 0x55925a222610 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1ec96d3fff89b server_cookie=722fccd277a2a9b8 in_seq=0 out_seq=0 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.125+0000 7fea1ea00700 10 client.27679 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.126+0000 7fea1ea00700 1 -- 172.21.15.46:0/557178135 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55925a1d5810 con 0x55925a2201f0 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.126+0000 7fea1ea00700 10 client.27679 handle_command_reply: tid=0 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.126+0000 7fea5392eb80 1 client.27679 shutdown 2024-05-08T23:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 msgr2=0x55925a20ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fea24009990 tx=0x7fea24005800 comp rx=0 tx=0).stop 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 shutdown_connections 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a2201f0 0x55925a222610 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20c720 0x55925a210510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a206890 0x55925a20fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 --2- 172.21.15.46:0/557178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a20ffd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 >> 172.21.15.46:0/557178135 conn(0x55925a164210 msgr2=0x55925a2005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea1ea00700 0 client.27679 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 shutdown_connections 2024-05-08T23:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.127+0000 7fea5392eb80 1 -- 172.21.15.46:0/557178135 wait complete. 2024-05-08T23:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.128+0000 7fea5392eb80 20 client.27679 trim_cache size 0 max 16384 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.150+0000 7fea5392eb80 1 Processor -- start 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.150+0000 7fea5392eb80 1 -- start start 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a206890 0x55925a206350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a217160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a1fddc0 con 0x55925a206890 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a227840 con 0x55925a20c720 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea5392eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a1fba90 con 0x55925a20be00 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a206890 0x55925a206350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39524/0 (socket says 172.21.15.46:39524) 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 -- 172.21.15.46:0/3518068300 learned_addr learned my addr 172.21.15.46:0/3518068300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3de00700 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a217160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 -- 172.21.15.46:0/3518068300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 msgr2=0x55925a217160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a217160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 -- 172.21.15.46:0/3518068300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a206890 msgr2=0x55925a206350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a206890 0x55925a206350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3ca00700 1 -- 172.21.15.46:0/3518068300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925a11c560 con 0x55925a20be00 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.151+0000 7fea3de00700 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a217160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:10:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.152+0000 7fea3ca00700 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea24009290 tx=0x7fea24004fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c188c192ec9ea58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.152+0000 7fea2b000700 1 -- 172.21.15.46:0/3518068300 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2401d070 con 0x55925a20be00 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.152+0000 7fea2b000700 1 -- 172.21.15.46:0/3518068300 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2401e040 con 0x55925a20be00 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.152+0000 7fea2b000700 1 -- 172.21.15.46:0/3518068300 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2402c3a0 con 0x55925a20be00 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- 172.21.15.46:0/3518068300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 msgr2=0x55925a200d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea24009290 tx=0x7fea24004fd0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- 172.21.15.46:0/3518068300 shutdown_connections 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a217160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a206890 0x55925a206350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 --2- 172.21.15.46:0/3518068300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a20be00 0x55925a200d20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- 172.21.15.46:0/3518068300 >> 172.21.15.46:0/3518068300 conn(0x55925a164210 msgr2=0x55925a206cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- 172.21.15.46:0/3518068300 shutdown_connections 2024-05-08T23:10:52.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- 172.21.15.46:0/3518068300 wait complete. 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 -- start start 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.153+0000 7fea5392eb80 1 Processor -- start 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 0x55925a208b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a2092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a207970 con 0x55925a20be00 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a207ae0 con 0x55925a20c720 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925a207ca0 con 0x55925a206890 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea5392eb80 10 client.0 _ll_register_callbacks cb 0x55925a0fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39532/0 (socket says 172.21.15.46:39532) 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 -- 172.21.15.46:0/768556394 learned_addr learned my addr 172.21.15.46:0/768556394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:10:52.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3ca00700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 0x55925a208b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3de00700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a2092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 -- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 msgr2=0x55925a2092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a2092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 -- 172.21.15.46:0/768556394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 msgr2=0x55925a208b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.154+0000 7fea3d400700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 0x55925a208b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea3d400700 1 -- 172.21.15.46:0/768556394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559259f83b60 con 0x55925a206890 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea3ca00700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 0x55925a208b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea3d400700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea2c099d60 tx=0x7fea2c099cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3084582bbf989d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea1ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c096770 con 0x55925a206890 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9466+0+0 (secure 0 0 0) 0x7fea2c0968d0 con 0x55925a206890 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2c0a24e0 con 0x55925a206890 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2c016330 con 0x55925a206890 2024-05-08T23:10:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea5392eb80 10 client.27685 fetch_fsmap learned FSMap version 30 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559259fdbcd0 con 0x55925a206890 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.155+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c0a24e0 con 0x55925a206890 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea2c099550 con 0x55925a206890 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 10 client.27685 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 10 client.27685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 20 client.27685 populate_metadata read hostname 'smithi046' 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a276cf0 0x55925a279110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 4 client.27685 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55925a14eec0 con 0x55925a276cf0 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.156+0000 7fea3ca00700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a276cf0 0x55925a279110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:10:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea3ca00700 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a276cf0 0x55925a279110 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0f425d8b651ab server_cookie=caae70af5a19c19f in_seq=0 out_seq=0 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea1ea00700 10 client.27685 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea1ea00700 1 -- 172.21.15.46:0/768556394 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55925a14eec0 con 0x55925a276cf0 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea1ea00700 10 client.27685 handle_command_reply: tid=0 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 client.27685 shutdown 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 msgr2=0x55925a217650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea2c099d60 tx=0x7fea2c099cd0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 shutdown_connections 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55925a20c720 0x55925a2092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55925a206890 0x55925a217650 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55925a276cf0 0x55925a279110 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 --2- 172.21.15.46:0/768556394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55925a20be00 0x55925a208b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 >> 172.21.15.46:0/768556394 conn(0x55925a164210 msgr2=0x55925a20e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea1ea00700 0 client.27685 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 shutdown_connections 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 1 -- 172.21.15.46:0/768556394 wait complete. 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.157+0000 7fea5392eb80 20 client.27685 trim_cache size 0 max 16384 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:10:52.162 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: "f6206d85-8813-4a8a-9a9f-92069e84da1e": { 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "f6206d85-8813-4a8a-9a9f-92069e84da1e", 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:10:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 msgr2=0x7fea200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 0x7fea200a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea381a4370 tx=0x7fea3000fc00 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 msgr2=0x7fea381a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9c0 tx=0x7fea2c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 shutdown_connections 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fea200a1290 0x7fea200a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea38111e20 0x7fea3819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea38073340 0x7fea381a3670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 --2- 172.21.15.46:0/2762599501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea38073c60 0x7fea381a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:10:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 >> 172.21.15.46:0/2762599501 conn(0x7fea380fc8e0 msgr2=0x7fea380fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:10:52.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 shutdown_connections 2024-05-08T23:10:52.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:10:52.158+0000 7fea3fe00700 1 -- 172.21.15.46:0/2762599501 wait complete. 2024-05-08T23:10:52.179 INFO:tasks.cephfs.filesystem:scrub status for tag:f6206d85-8813-4a8a-9a9f-92069e84da1e - {'path': '/', 'tag': 'f6206d85-8813-4a8a-9a9f-92069e84da1e', 'options': 'recursive,force'} 2024-05-08T23:10:52.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:10:52.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:10:52.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:52.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:52.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:52 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:10:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:52 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:52.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:52.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:52 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:52.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 575539 10.22 MB/sec execute 194 sec latency 18.490 ms 2024-05-08T23:10:53.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:53 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:10:53.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:53 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:53.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:53 smithi086 ceph-mon[78610]: pgmap v215: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:53.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:53 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:10:53.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:53 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:53.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:53 smithi053 ceph-mon[77587]: pgmap v215: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:53 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:10:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:53 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:10:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:53 smithi046 ceph-mon[36898]: pgmap v215: 129 pgs: 129 active+clean; 375 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-08T23:10:53.892 INFO:tasks.workunit.client.0.smithi046.stdout: 1 576953 10.20 MB/sec execute 195 sec latency 15.472 ms 2024-05-08T23:10:54.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 579096 10.21 MB/sec execute 196 sec latency 15.783 ms 2024-05-08T23:10:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:55 smithi086 ceph-mon[78610]: pgmap v216: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-08T23:10:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:55 smithi053 ceph-mon[77587]: pgmap v216: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-08T23:10:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:55 smithi046 ceph-mon[36898]: pgmap v216: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-08T23:10:55.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 580617 10.19 MB/sec execute 197 sec latency 17.082 ms 2024-05-08T23:10:56.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 582801 10.21 MB/sec execute 198 sec latency 16.296 ms 2024-05-08T23:10:57.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:57 smithi086 ceph-mon[78610]: pgmap v217: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:57 smithi053 ceph-mon[77587]: pgmap v217: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:57.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:57 smithi046 ceph-mon[36898]: pgmap v217: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:57.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 584391 10.19 MB/sec execute 199 sec latency 16.250 ms 2024-05-08T23:10:58.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 586467 10.20 MB/sec execute 200 sec latency 12.509 ms 2024-05-08T23:10:59.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:10:59 smithi086 ceph-mon[78610]: pgmap v218: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:59.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:10:59 smithi053 ceph-mon[77587]: pgmap v218: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:59.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:10:59 smithi046 ceph-mon[36898]: pgmap v218: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-08T23:10:59.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 588346 10.19 MB/sec execute 201 sec latency 27.501 ms 2024-05-08T23:11:00.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 589798 10.19 MB/sec execute 202 sec latency 59.973 ms 2024-05-08T23:11:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:01 smithi086 ceph-mon[78610]: pgmap v219: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-05-08T23:11:01.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:01 smithi053 ceph-mon[77587]: pgmap v219: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-05-08T23:11:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:01 smithi046 ceph-mon[36898]: pgmap v219: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-05-08T23:11:01.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 590379 10.15 MB/sec execute 203 sec latency 52.155 ms 2024-05-08T23:11:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:11:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:11:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:11:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:11:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:11:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:11:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:11:02.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:11:02.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 591312 10.12 MB/sec execute 204 sec latency 368.421 ms 2024-05-08T23:11:02.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:11:02.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:11:02.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:11:02.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:11:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:03 smithi053 ceph-mon[77587]: pgmap v220: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-05-08T23:11:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:03 smithi046 ceph-mon[36898]: pgmap v220: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-05-08T23:11:03.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 593186 10.13 MB/sec execute 205 sec latency 50.185 ms 2024-05-08T23:11:03.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:03 smithi086 ceph-mon[78610]: pgmap v220: 129 pgs: 129 active+clean; 390 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-05-08T23:11:04.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 594574 10.12 MB/sec execute 206 sec latency 17.416 ms 2024-05-08T23:11:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:05 smithi053 ceph-mon[77587]: pgmap v221: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2024-05-08T23:11:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:05 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:05 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:05 smithi046 ceph-mon[36898]: pgmap v221: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2024-05-08T23:11:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:05 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:05 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:05.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 596782 10.13 MB/sec execute 207 sec latency 17.454 ms 2024-05-08T23:11:05.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:05 smithi086 ceph-mon[78610]: pgmap v221: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2024-05-08T23:11:05.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:05 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:05.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:05 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:06 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:06 smithi053 ceph-mon[77587]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:11:06.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:06 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:06.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:06 smithi046 ceph-mon[36898]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:11:06.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 598269 10.11 MB/sec execute 208 sec latency 14.631 ms 2024-05-08T23:11:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:06 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:06 smithi086 ceph-mon[78610]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:11:07.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:07 smithi053 ceph-mon[77587]: pgmap v223: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:07.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:07 smithi046 ceph-mon[36898]: pgmap v223: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:07.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 600014 10.11 MB/sec execute 209 sec latency 153.279 ms 2024-05-08T23:11:07.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:07 smithi086 ceph-mon[78610]: pgmap v223: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:08.893 INFO:tasks.workunit.client.0.smithi046.stdout: 1 601122 10.09 MB/sec execute 210 sec latency 29.339 ms 2024-05-08T23:11:09.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:09 smithi053 ceph-mon[77587]: pgmap v224: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:09 smithi046 ceph-mon[36898]: pgmap v224: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:09.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 603224 10.09 MB/sec execute 211 sec latency 27.468 ms 2024-05-08T23:11:09.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:09 smithi086 ceph-mon[78610]: pgmap v224: 129 pgs: 129 active+clean; 406 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 299 op/s 2024-05-08T23:11:10.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 604677 10.08 MB/sec execute 212 sec latency 14.143 ms 2024-05-08T23:11:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:11 smithi053 ceph-mon[77587]: pgmap v225: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:11.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:11 smithi046 ceph-mon[36898]: pgmap v225: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:11.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 606813 10.09 MB/sec execute 213 sec latency 15.967 ms 2024-05-08T23:11:11.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:11 smithi086 ceph-mon[78610]: pgmap v225: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:12.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 608318 10.08 MB/sec execute 214 sec latency 13.512 ms 2024-05-08T23:11:13.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:13 smithi053 ceph-mon[77587]: pgmap v226: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:13.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:13 smithi046 ceph-mon[36898]: pgmap v226: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:13.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 610612 10.09 MB/sec execute 215 sec latency 16.496 ms 2024-05-08T23:11:13.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:13 smithi086 ceph-mon[78610]: pgmap v226: 129 pgs: 129 active+clean; 401 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 364 op/s 2024-05-08T23:11:14.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 612073 10.08 MB/sec execute 216 sec latency 19.647 ms 2024-05-08T23:11:15.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 614400 10.10 MB/sec execute 217 sec latency 16.111 ms 2024-05-08T23:11:16.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:15 smithi053 ceph-mon[77587]: pgmap v227: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:16.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:15 smithi053 ceph-mon[77587]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:11:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:15 smithi046 ceph-mon[36898]: pgmap v227: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:16.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:15 smithi046 ceph-mon[36898]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:11:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:15 smithi086 ceph-mon[78610]: pgmap v227: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:15 smithi086 ceph-mon[78610]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:11:16.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 615822 10.09 MB/sec execute 218 sec latency 18.719 ms 2024-05-08T23:11:17.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 618070 10.10 MB/sec execute 219 sec latency 16.284 ms 2024-05-08T23:11:18.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:17 smithi053 ceph-mon[77587]: pgmap v229: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:18.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:17 smithi046 ceph-mon[36898]: pgmap v229: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:17 smithi086 ceph-mon[78610]: pgmap v229: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:18.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 619513 10.08 MB/sec execute 220 sec latency 15.833 ms 2024-05-08T23:11:19.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 621772 10.10 MB/sec execute 221 sec latency 15.311 ms 2024-05-08T23:11:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:19 smithi053 ceph-mon[77587]: pgmap v230: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:19 smithi046 ceph-mon[36898]: pgmap v230: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:19 smithi086 ceph-mon[78610]: pgmap v230: 129 pgs: 129 active+clean; 410 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-05-08T23:11:20.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 623303 10.08 MB/sec execute 222 sec latency 17.744 ms 2024-05-08T23:11:21.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 625367 10.09 MB/sec execute 223 sec latency 26.420 ms 2024-05-08T23:11:22.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:21 smithi053 ceph-mon[77587]: pgmap v231: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:22.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:21 smithi046 ceph-mon[36898]: pgmap v231: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:22.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:11:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:21 smithi086 ceph-mon[78610]: pgmap v231: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:22.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.414+0000 7f7caee00700 1 -- 172.21.15.46:0/359783862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 msgr2=0x7f7ca80737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.414+0000 7f7caee00700 1 --2- 172.21.15.46:0/359783862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca80737b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009990 tx=0x7f7c98009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 -- 172.21.15.46:0/359783862 shutdown_connections 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 --2- 172.21.15.46:0/359783862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca810d650 0x7f7ca810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 --2- 172.21.15.46:0/359783862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca80737b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 --2- 172.21.15.46:0/359783862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca8073cf0 0x7f7ca810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 -- 172.21.15.46:0/359783862 >> 172.21.15.46:0/359783862 conn(0x7f7ca80fc8e0 msgr2=0x7f7ca80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 -- 172.21.15.46:0/359783862 shutdown_connections 2024-05-08T23:11:22.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.415+0000 7f7caee00700 1 -- 172.21.15.46:0/359783862 wait complete. 2024-05-08T23:11:22.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.416+0000 7f7caee00700 1 Processor -- start 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.416+0000 7f7caee00700 1 -- start start 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 0x7f7ca8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca810d650 0x7f7ca819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819e480 con 0x7f7ca8073cf0 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819e5f0 con 0x7f7ca810d650 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7caee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819e7b0 con 0x7f7ca80733d0 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60058/0 (socket says 172.21.15.46:60058) 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 -- 172.21.15.46:0/2601629315 learned_addr learned my addr 172.21.15.46:0/2601629315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7400700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 0x7f7ca8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7cace00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca810d650 0x7f7ca819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 -- 172.21.15.46:0/2601629315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca810d650 msgr2=0x7f7ca819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca810d650 0x7f7ca819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 -- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 msgr2=0x7f7ca8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 0x7f7ca8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7e00700 1 -- 172.21.15.46:0/2601629315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980095f0 con 0x7f7ca80733d0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.417+0000 7f7ca7400700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 0x7f7ca8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c98005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7ca4c00700 1 -- 172.21.15.46:0/2601629315 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9801d070 con 0x7f7ca80733d0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7ca4c00700 1 -- 172.21.15.46:0/2601629315 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c98022470 con 0x7f7ca80733d0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819ea30 con 0x7f7ca80733d0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7ca4c00700 1 -- 172.21.15.46:0/2601629315 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98017650 con 0x7f7ca80733d0 2024-05-08T23:11:22.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.418+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a62d0 con 0x7f7ca80733d0 2024-05-08T23:11:22.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.421+0000 7f7ca4c00700 1 -- 172.21.15.46:0/2601629315 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7c980225e0 con 0x7f7ca80733d0 2024-05-08T23:11:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.423+0000 7f7ca4c00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 0x7f7c8c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.423+0000 7f7ca4c00700 1 -- 172.21.15.46:0/2601629315 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7c980ce370 con 0x7f7ca80733d0 2024-05-08T23:11:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.423+0000 7f7cc2840b80 1 Processor -- start 2024-05-08T23:11:22.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.423+0000 7f7ca7400700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 0x7f7c8c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.423+0000 7f7cc2840b80 1 -- start start 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7ca7400700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 0x7f7c8c0a37d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ca819aab0 tx=0x7f7c94009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969d80a0 0x5595969d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596926150 con 0x5595969ddf10 2024-05-08T23:11:22.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596923b10 con 0x5595969d80a0 2024-05-08T23:11:22.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959691f400 con 0x5595969dd5f0 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58502/0 (socket says 172.21.15.46:58502) 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7ca7400700 1 -- 172.21.15.46:0/3441214812 learned_addr learned my addr 172.21.15.46:0/3441214812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.424+0000 7f7cace00700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969d80a0 0x5595969d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 -- 172.21.15.46:0/3441214812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969d80a0 msgr2=0x5595969d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969d80a0 0x5595969d8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 -- 172.21.15.46:0/3441214812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 msgr2=0x5595969d7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 -- 172.21.15.46:0/3441214812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595967a03b0 con 0x5595969dd5f0 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7400700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969dd9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cb650 tx=0x7f7c98005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6686da34a4a6257 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7c9d600700 1 -- 172.21.15.46:0/3441214812 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9809b7a0 con 0x5595969dd5f0 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7c9d600700 1 -- 172.21.15.46:0/3441214812 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9809b0a0 con 0x5595969dd5f0 2024-05-08T23:11:22.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.425+0000 7f7c9d600700 1 -- 172.21.15.46:0/3441214812 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98022a00 con 0x5595969dd5f0 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 -- 172.21.15.46:0/3441214812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 msgr2=0x5595969dd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969dd9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cb650 tx=0x7f7c98005fd0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 -- 172.21.15.46:0/3441214812 shutdown_connections 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969d80a0 0x5595969d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969dd9d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/3441214812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.426+0000 7f7cc2840b80 1 -- 172.21.15.46:0/3441214812 >> 172.21.15.46:0/3441214812 conn(0x55959692d210 msgr2=0x5595969c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.427+0000 7f7cc2840b80 1 -- 172.21.15.46:0/3441214812 shutdown_connections 2024-05-08T23:11:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.427+0000 7f7cc2840b80 1 -- 172.21.15.46:0/3441214812 wait complete. 2024-05-08T23:11:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.427+0000 7f7cc2840b80 1 -- start start 2024-05-08T23:11:22.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 Processor -- start 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 0x5595969e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595968bb560 con 0x5595969ddf10 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595968bb200 con 0x5595969dd5f0 2024-05-08T23:11:22.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.428+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595967a26a0 con 0x5595969d80a0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7cc2840b80 10 client.0 _ll_register_callbacks cb 0x5595968c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7cace00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7cace00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58510/0 (socket says 172.21.15.46:58510) 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 0x5595969e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7cace00700 1 -- 172.21.15.46:0/1495329008 learned_addr learned my addr 172.21.15.46:0/1495329008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7400700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 msgr2=0x5595969e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1495329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 msgr2=0x5595969e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1495329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c94011c60 con 0x5595969d80a0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.429+0000 7f7cace00700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 0x5595969e1280 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5595969e45d0 tx=0x7f7c9800b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642f128d8c1b7469 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980a3490 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7c9809eed0 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559596878270 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c980a10b0 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7cc2840b80 10 client.27697 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f7ca81a62d0 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559596878270 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.430+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7c980a3000 con 0x5595969d80a0 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 10 client.27697 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 20 client.27697 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5595969f1210 0x5595969f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 4 client.27697 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:22.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55959699e810 con 0x5595969f1210 2024-05-08T23:11:22.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.431+0000 7f7ca7400700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5595969f1210 0x5595969f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7ca7400700 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5595969f1210 0x5595969f3600 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a9870a5b1cc2e6 server_cookie=dbc3fa994fe66ec9 in_seq=0 out_seq=0 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7c92000700 10 client.27697 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7c92000700 1 -- 172.21.15.46:0/1495329008 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55959699e810 con 0x5595969f1210 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7c92000700 10 client.27697 handle_command_reply: tid=0 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7cc2840b80 1 client.27697 shutdown 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 msgr2=0x5595969e1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.432+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 0x5595969e1280 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5595969e45d0 tx=0x7f7c9800b690 comp rx=0 tx=0).stop 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 shutdown_connections 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5595969f1210 0x5595969f3600 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969ddf10 0x5595969e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969d80a0 0x5595969e1280 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7c92000700 0 client.27697 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1495329008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 >> 172.21.15.46:0/1495329008 conn(0x55959692d210 msgr2=0x5595969c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 shutdown_connections 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1495329008 wait complete. 2024-05-08T23:11:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.433+0000 7f7cc2840b80 20 client.27697 trim_cache size 0 max 16384 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 Processor -- start 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 -- start start 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969ddf10 0x5595969e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596878270 con 0x5595969d80a0 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595969e28f0 con 0x5595969dd5f0 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.455+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595969c6df0 con 0x5595969ddf10 2024-05-08T23:11:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7cace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969ddf10 0x5595969e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36910/0 (socket says 172.21.15.46:36910) 2024-05-08T23:11:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58512/0 (socket says 172.21.15.46:58512) 2024-05-08T23:11:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7400700 1 -- 172.21.15.46:0/1263111578 learned_addr learned my addr 172.21.15.46:0/1263111578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1263111578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969ddf10 msgr2=0x5595969e1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969ddf10 0x5595969e1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1263111578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 msgr2=0x5595969c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 -- 172.21.15.46:0/1263111578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55959674cb60 con 0x5595969d80a0 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7400700 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.456+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7c9809ada0 tx=0x7f7c98099650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe4ca004b810803 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.457+0000 7f7c9e000700 1 -- 172.21.15.46:0/1263111578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9809abe0 con 0x5595969d80a0 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.457+0000 7f7c9e000700 1 -- 172.21.15.46:0/1263111578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c98098790 con 0x5595969d80a0 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.457+0000 7f7c9e000700 1 -- 172.21.15.46:0/1263111578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980229f0 con 0x5595969d80a0 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1263111578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 msgr2=0x5595969dca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7c9809ada0 tx=0x7f7c98099650 comp rx=0 tx=0).stop 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1263111578 shutdown_connections 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969ddf10 0x5595969e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969dca70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/1263111578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969dd5f0 0x5595969c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1263111578 >> 172.21.15.46:0/1263111578 conn(0x55959692d210 msgr2=0x5595969dc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1263111578 shutdown_connections 2024-05-08T23:11:22.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- 172.21.15.46:0/1263111578 wait complete. 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 -- start start 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.458+0000 7f7cc2840b80 1 Processor -- start 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969e6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596a46850 con 0x5595969d80a0 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596a469c0 con 0x5595969ddf10 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559596a46b80 con 0x5595969dd5f0 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cc2840b80 10 client.0 _ll_register_callbacks cb 0x5595968c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cace00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cace00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36922/0 (socket says 172.21.15.46:36922) 2024-05-08T23:11:22.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969e6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60084/0 (socket says 172.21.15.46:60084) 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7cace00700 1 -- 172.21.15.46:0/755390690 learned_addr learned my addr 172.21.15.46:0/755390690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.459+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 -- 172.21.15.46:0/755390690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 msgr2=0x5595969e6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969e6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 -- 172.21.15.46:0/755390690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 msgr2=0x5595969e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969e6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 -- 172.21.15.46:0/755390690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c940125d0 con 0x5595969ddf10 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cace00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0017430 tx=0x7f7ca0017740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc7eb8cdf53637cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7c92000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0010040 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c9809b0a0 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7ca0006e40 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca0020640 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cc2840b80 10 client.27632 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.460+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f7ca81a62d0 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9809b0a0 con 0x5595969ddf10 2024-05-08T23:11:22.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7ca0008030 con 0x5595969ddf10 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 10 client.27632 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 10 client.27632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 20 client.27632 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559596a48440 0x559596a4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 4 client.27632 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559596917ec0 con 0x559596a48440 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.461+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559596a48440 0x559596a4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7ca7e00700 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559596a48440 0x559596a4a860 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea63f84ce8fbc20 server_cookie=72ddc543b00e4e14 in_seq=0 out_seq=0 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7c92000700 10 client.27632 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7c92000700 1 -- 172.21.15.46:0/755390690 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559596917ec0 con 0x559596a48440 2024-05-08T23:11:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7c92000700 10 client.27632 handle_command_reply: tid=0 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 client.27632 shutdown 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 msgr2=0x5595969e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0017430 tx=0x7f7ca0017740 comp rx=0 tx=0).stop 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 shutdown_connections 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5595969ddf10 0x5595969e6fb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559596a48440 0x559596a4a860 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595969d80a0 0x5595969e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 --2- 172.21.15.46:0/755390690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595969dd5f0 0x5595969e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 >> 172.21.15.46:0/755390690 conn(0x55959692d210 msgr2=0x5595969cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.462+0000 7f7c92000700 0 client.27632 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 shutdown_connections 2024-05-08T23:11:22.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7cc2840b80 1 -- 172.21.15.46:0/755390690 wait complete. 2024-05-08T23:11:22.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7cc2840b80 20 client.27632 trim_cache size 0 max 16384 2024-05-08T23:11:22.469 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:11:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:11:22.469 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 msgr2=0x7f7c8c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 0x7f7c8c0a37d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ca819aab0 tx=0x7f7c94009260 comp rx=0 tx=0).stop 2024-05-08T23:11:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 msgr2=0x7f7ca81995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.463+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c98005740 comp rx=0 tx=0).stop 2024-05-08T23:11:22.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 shutdown_connections 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ca810d650 0x7f7ca819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca80733d0 0x7f7ca81995b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7c8c0a1310 0x7f7c8c0a37d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 --2- 172.21.15.46:0/2601629315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ca8073cf0 0x7f7ca8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 >> 172.21.15.46:0/2601629315 conn(0x7f7ca80fc8e0 msgr2=0x7f7ca80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 shutdown_connections 2024-05-08T23:11:22.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.464+0000 7f7caee00700 1 -- 172.21.15.46:0/2601629315 wait complete. 2024-05-08T23:11:22.482 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:11:22.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 -- 172.21.15.46:0/3911363727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073340 msgr2=0x7f9200073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 --2- 172.21.15.46:0/3911363727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073340 0x7f9200073720 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009990 tx=0x7f91f0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 -- 172.21.15.46:0/3911363727 shutdown_connections 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 --2- 172.21.15.46:0/3911363727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f9200114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 --2- 172.21.15.46:0/3911363727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073340 0x7f9200073720 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 --2- 172.21.15.46:0/3911363727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073c60 0x7f92001118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 -- 172.21.15.46:0/3911363727 >> 172.21.15.46:0/3911363727 conn(0x7f92000fc930 msgr2=0x7f92000fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 -- 172.21.15.46:0/3911363727 shutdown_connections 2024-05-08T23:11:22.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.717+0000 7f9206200700 1 -- 172.21.15.46:0/3911363727 wait complete. 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.718+0000 7f9206200700 1 Processor -- start 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 -- start start 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f920019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920019de60 con 0x7f9200111e10 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920019dfd0 con 0x7f9200073340 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f9206200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920019e190 con 0x7f9200073c60 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91ff400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91ff400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36938/0 (socket says 172.21.15.46:36938) 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60108/0 (socket says 172.21.15.46:60108) 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91ff400700 1 -- 172.21.15.46:0/2223977358 learned_addr learned my addr 172.21.15.46:0/2223977358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:22.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.719+0000 7f91ffe00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f920019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 -- 172.21.15.46:0/2223977358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 msgr2=0x7f92001a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 -- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 msgr2=0x7f920019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f920019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 -- 172.21.15.46:0/2223977358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00095f0 con 0x7f9200073340 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91fea00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ffe00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f920019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91ff400700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f91f0000c00 tx=0x7f91f0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f001d070 con 0x7f9200073340 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920019e410 con 0x7f9200073340 2024-05-08T23:11:22.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f0022470 con 0x7f9200073340 2024-05-08T23:11:22.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.720+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0017650 con 0x7f9200073340 2024-05-08T23:11:22.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.721+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001aa640 con 0x7f9200073340 2024-05-08T23:11:22.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.721+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200104330 con 0x7f9200073340 2024-05-08T23:11:22.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.724+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f91f00225e0 con 0x7f9200073340 2024-05-08T23:11:22.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.725+0000 7f91f7e00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 0x7f91e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:22.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.725+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f91f00cebb0 con 0x7f9200073340 2024-05-08T23:11:22.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.726+0000 7f91fea00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 0x7f91e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:22.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.726+0000 7f91fea00700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 0x7f91e40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f920019ee90 tx=0x7f91ec0073d0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:22.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:22.728+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f91f0097c20 con 0x7f9200073340 2024-05-08T23:11:22.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 626903 10.07 MB/sec execute 224 sec latency 18.813 ms 2024-05-08T23:11:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.037+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9200074a80 con 0x7f9200073340 2024-05-08T23:11:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.038+0000 7f91f7e00700 1 -- 172.21.15.46:0/2223977358 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f91f009b240 con 0x7f9200073340 2024-05-08T23:11:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:11:23.039 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 msgr2=0x7f91e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 0x7f91e40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f920019ee90 tx=0x7f91ec0073d0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 msgr2=0x7f92001a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f91f0000c00 tx=0x7f91f0005740 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 shutdown_connections 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f91e40a1290 0x7f91e40a3750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9200111e10 0x7f920019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9200073340 0x7f92001a3610 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 --2- 172.21.15.46:0/2223977358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9200073c60 0x7f92001a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.041+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 >> 172.21.15.46:0/2223977358 conn(0x7f92000fc930 msgr2=0x7f92000fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.042+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 shutdown_connections 2024-05-08T23:11:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.042+0000 7f9206200700 1 -- 172.21.15.46:0/2223977358 wait complete. 2024-05-08T23:11:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:11:23.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.276+0000 7f1111c00700 1 -- 172.21.15.46:0/4055545233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 msgr2=0x7f110c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.276+0000 7f1111c00700 1 --2- 172.21.15.46:0/4055545233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1012a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009990 tx=0x7f10fc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 -- 172.21.15.46:0/4055545233 shutdown_connections 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 --2- 172.21.15.46:0/4055545233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 0x7f110c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 --2- 172.21.15.46:0/4055545233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1012a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 --2- 172.21.15.46:0/4055545233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c101870 0x7f110c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 -- 172.21.15.46:0/4055545233 >> 172.21.15.46:0/4055545233 conn(0x7f110c0fc740 msgr2=0x7f110c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 -- 172.21.15.46:0/4055545233 shutdown_connections 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.277+0000 7f1111c00700 1 -- 172.21.15.46:0/4055545233 wait complete. 2024-05-08T23:11:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 Processor -- start 2024-05-08T23:11:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 -- start start 2024-05-08T23:11:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 0x7f110c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c19bbf0 0x7f110c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c19c670 con 0x7f110c19bbf0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c19c7e0 con 0x7f110c105fe0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.278+0000 7f1111c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c19c9a0 con 0x7f110c100ec0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60130/0 (socket says 172.21.15.46:60130) 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 -- 172.21.15.46:0/313183774 learned_addr learned my addr 172.21.15.46:0/313183774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f1111200700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c19bbf0 0x7f110c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110b400700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 0x7f110c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 -- 172.21.15.46:0/313183774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 msgr2=0x7f110c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 0x7f110c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 -- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c19bbf0 msgr2=0x7f110c19c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c19bbf0 0x7f110c19c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 -- 172.21.15.46:0/313183774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0095f0 con 0x7f110c100ec0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.279+0000 7f110be00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000c00 tx=0x7f10fc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.280+0000 7f1108c00700 1 -- 172.21.15.46:0/313183774 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc01d070 con 0x7f110c100ec0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.280+0000 7f1108c00700 1 -- 172.21.15.46:0/313183774 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10fc022470 con 0x7f110c100ec0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.280+0000 7f1108c00700 1 -- 172.21.15.46:0/313183774 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc017650 con 0x7f110c100ec0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.280+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c1a3ea0 con 0x7f110c100ec0 2024-05-08T23:11:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.280+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c1a4260 con 0x7f110c100ec0 2024-05-08T23:11:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.284+0000 7f1108c00700 1 -- 172.21.15.46:0/313183774 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f10fc0225e0 con 0x7f110c100ec0 2024-05-08T23:11:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.285+0000 7f1108c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 0x7f10f00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.285+0000 7f1126de2b80 1 Processor -- start 2024-05-08T23:11:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.285+0000 7f1108c00700 1 -- 172.21.15.46:0/313183774 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f10fc0ce340 con 0x7f110c100ec0 2024-05-08T23:11:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 -- start start 2024-05-08T23:11:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f110b400700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 0x7f10f00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7b77150 con 0x564af7c2a3d0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7b74b10 con 0x564af7c2acf0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.286+0000 7f1126de2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7b70400 con 0x564af7c24ca0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110b400700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 0x7f10f00a3870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f110c107470 tx=0x7f10f8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58550/0 (socket says 172.21.15.46:58550) 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110be00700 1 -- 172.21.15.46:0/173687895 learned_addr learned my addr 172.21.15.46:0/173687895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 -- 172.21.15.46:0/173687895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 msgr2=0x564af7c24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c24760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 -- 172.21.15.46:0/173687895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 msgr2=0x564af7c2a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 -- 172.21.15.46:0/173687895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564af79f13b0 con 0x564af7c24ca0 2024-05-08T23:11:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f110b400700 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.287+0000 7f1111200700 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c250e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1104011580 tx=0x7f1104009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e688b3d097a8230 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1101600700 1 -- 172.21.15.46:0/173687895 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400e040 con 0x564af7c24ca0 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1101600700 1 -- 172.21.15.46:0/173687895 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110400f040 con 0x564af7c24ca0 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1101600700 1 -- 172.21.15.46:0/173687895 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040044f0 con 0x564af7c24ca0 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 -- 172.21.15.46:0/173687895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 msgr2=0x564af7c250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c250e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1104011580 tx=0x7f1104009920 comp rx=0 tx=0).stop 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 -- 172.21.15.46:0/173687895 shutdown_connections 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c250e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 --2- 172.21.15.46:0/173687895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.288+0000 7f1126de2b80 1 -- 172.21.15.46:0/173687895 >> 172.21.15.46:0/173687895 conn(0x564af7b7e210 msgr2=0x564af7c1b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.289+0000 7f1126de2b80 1 -- 172.21.15.46:0/173687895 shutdown_connections 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.289+0000 7f1126de2b80 1 -- 172.21.15.46:0/173687895 wait complete. 2024-05-08T23:11:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.289+0000 7f1126de2b80 1 -- start start 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.289+0000 7f1126de2b80 1 Processor -- start 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c2e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c2eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7b0c560 con 0x564af7c2a3d0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7b0c200 con 0x564af7c2acf0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af79f36a0 con 0x564af7c24ca0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1126de2b80 10 client.0 _ll_register_callbacks cb 0x564af7b15b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c2e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1111200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c2eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c2e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60150/0 (socket says 172.21.15.46:60150) 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f110be00700 1 -- 172.21.15.46:0/658845585 learned_addr learned my addr 172.21.15.46:0/658845585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f110b400700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1111200700 1 -- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 msgr2=0x564af7c2e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1111200700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c2e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1111200700 1 -- 172.21.15.46:0/658845585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 msgr2=0x564af7c2e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.290+0000 7f1111200700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f1111200700 1 -- 172.21.15.46:0/658845585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f8017070 con 0x564af7c2acf0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f1111200700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c2eba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f110400b010 tx=0x7f1104011e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676eed69298f0db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f10f6000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400e040 con 0x564af7c2acf0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1104003c80 con 0x564af7c2acf0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564af7ac9270 con 0x564af7c2acf0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1104003e40 con 0x564af7c2acf0 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f1126de2b80 10 client.27650 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.291+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1104011230 con 0x564af7c2acf0 2024-05-08T23:11:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564af7ac9270 con 0x564af7c2acf0 2024-05-08T23:11:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f110401b020 con 0x564af7c2acf0 2024-05-08T23:11:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 10 client.27650 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 10 client.27650 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:11:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 20 client.27650 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c3e7c0 0x564af7c40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 4 client.27650 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.292+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564af7bef810 con 0x564af7c3e7c0 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.293+0000 7f110be00700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c3e7c0 0x564af7c40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.293+0000 7f110be00700 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c3e7c0 0x564af7c40be0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d5ba2cac0cd0a93 server_cookie=277ac0f3ba390cae in_seq=0 out_seq=0 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.293+0000 7f10f6000700 10 client.27650 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f10f6000700 1 -- 172.21.15.46:0/658845585 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564af7bef810 con 0x564af7c3e7c0 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f10f6000700 10 client.27650 handle_command_reply: tid=0 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 client.27650 shutdown 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 msgr2=0x564af7c2eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c2eba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f110400b010 tx=0x7f1104011e80 comp rx=0 tx=0).stop 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 shutdown_connections 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c3e7c0 0x564af7c40be0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2acf0 0x564af7c2eba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c2e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 --2- 172.21.15.46:0/658845585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2a3d0 0x564af7c2e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f10f6000700 0 client.27650 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 >> 172.21.15.46:0/658845585 conn(0x564af7b7e210 msgr2=0x564af7c1b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 shutdown_connections 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 1 -- 172.21.15.46:0/658845585 wait complete. 2024-05-08T23:11:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.294+0000 7f1126de2b80 20 client.27650 trim_cache size 0 max 16384 2024-05-08T23:11:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 Processor -- start 2024-05-08T23:11:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 -- start start 2024-05-08T23:11:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 0x564af7c29730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c17dc0 con 0x564af7c2acf0 2024-05-08T23:11:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c2ab30 con 0x564af7c2a3d0 2024-05-08T23:11:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.310+0000 7f1126de2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c45df0 con 0x564af7c24ca0 2024-05-08T23:11:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 0x564af7c29730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60164/0 (socket says 172.21.15.46:60164) 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 -- 172.21.15.46:0/4240521995 learned_addr learned my addr 172.21.15.46:0/4240521995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 0x564af7c29730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36998/0 (socket says 172.21.15.46:36998) 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f1111200700 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 -- 172.21.15.46:0/4240521995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 msgr2=0x564af7c29730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 0x564af7c29730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 -- 172.21.15.46:0/4240521995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 msgr2=0x564af7c2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c2c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 -- 172.21.15.46:0/4240521995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564af799db60 con 0x564af7c24ca0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f110be00700 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f10fc09b120 tx=0x7f10fc02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe148bc1f59fbf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.311+0000 7f1102000700 1 -- 172.21.15.46:0/4240521995 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0974d0 con 0x564af7c24ca0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1102000700 1 -- 172.21.15.46:0/4240521995 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10fc09b890 con 0x564af7c24ca0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1102000700 1 -- 172.21.15.46:0/4240521995 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc099360 con 0x564af7c24ca0 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 -- 172.21.15.46:0/4240521995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 msgr2=0x564af7c291f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f10fc09b120 tx=0x7f10fc02e040 comp rx=0 tx=0).stop 2024-05-08T23:11:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 -- 172.21.15.46:0/4240521995 shutdown_connections 2024-05-08T23:11:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c24ca0 0x564af7c291f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 --2- 172.21.15.46:0/4240521995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c2a3d0 0x564af7c29730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.312+0000 7f1126de2b80 1 -- 172.21.15.46:0/4240521995 >> 172.21.15.46:0/4240521995 conn(0x564af7b7e210 msgr2=0x564af7c19d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.313+0000 7f1126de2b80 1 -- 172.21.15.46:0/4240521995 shutdown_connections 2024-05-08T23:11:23.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.313+0000 7f1126de2b80 1 -- 172.21.15.46:0/4240521995 wait complete. 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.313+0000 7f1126de2b80 1 -- start start 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.313+0000 7f1126de2b80 1 Processor -- start 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c93cc0 con 0x564af7c2acf0 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c93e00 con 0x564af7c24ca0 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7c93fc0 con 0x564af7c2a3d0 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1126de2b80 10 client.0 _ll_register_callbacks cb 0x564af7b15b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37012/0 (socket says 172.21.15.46:37012) 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60180/0 (socket says 172.21.15.46:60180) 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 -- 172.21.15.46:0/38360134 learned_addr learned my addr 172.21.15.46:0/38360134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f1111200700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 -- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 msgr2=0x564af7c33cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 -- 172.21.15.46:0/38360134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 msgr2=0x564af7c341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110be00700 1 -- 172.21.15.46:0/38360134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c1a4260 con 0x564af7c24ca0 2024-05-08T23:11:23.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.314+0000 7f110b400700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f110be00700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10fc09ae90 tx=0x7f10fc022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af70913351004a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f10f6000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc09efc0 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f10fc084070 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110400f040 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10fc0a3580 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f1126de2b80 10 client.27656 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564af7b36560 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.315+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110400f040 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10fc0b9300 con 0x564af7c24ca0 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 10 client.27656 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 10 client.27656 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 20 client.27656 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c953e0 0x564af7c97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 4 client.27656 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564af7b68ec0 con 0x564af7c953e0 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f110b400700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c953e0 0x564af7c97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f110b400700 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c953e0 0x564af7c97800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0fb8b2b40f0f457 server_cookie=6d4e0cc48919f6aa in_seq=0 out_seq=0 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.316+0000 7f10f6000700 10 client.27656 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f10f6000700 1 -- 172.21.15.46:0/38360134 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564af7b68ec0 con 0x564af7c953e0 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f10f6000700 10 client.27656 handle_command_reply: tid=0 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 client.27656 shutdown 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 msgr2=0x564af7c33770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10fc09ae90 tx=0x7f10fc022c90 comp rx=0 tx=0).stop 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 shutdown_connections 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564af7c953e0 0x564af7c97800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af7c2acf0 0x564af7c341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564af7c24ca0 0x564af7c33770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f10f6000700 0 client.27656 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 --2- 172.21.15.46:0/38360134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af7c2a3d0 0x564af7c33cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 >> 172.21.15.46:0/38360134 conn(0x564af7b7e210 msgr2=0x564af7c291f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 shutdown_connections 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 1 -- 172.21.15.46:0/38360134 wait complete. 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.317+0000 7f1126de2b80 20 client.27656 trim_cache size 0 max 16384 2024-05-08T23:11:23.322 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 msgr2=0x7f10f00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 0x7f10f00a3870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f110c107470 tx=0x7f10f8009260 comp rx=0 tx=0).stop 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 msgr2=0x7f110c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000c00 tx=0x7f10fc005740 comp rx=0 tx=0).stop 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 shutdown_connections 2024-05-08T23:11:23.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10f00a13b0 0x7f10f00a3870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f110c19bbf0 0x7f110c19c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f110c100ec0 0x7f110c1973d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 --2- 172.21.15.46:0/313183774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110c105fe0 0x7f110c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 >> 172.21.15.46:0/313183774 conn(0x7f110c0fc740 msgr2=0x7f110c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 shutdown_connections 2024-05-08T23:11:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.318+0000 7f1111c00700 1 -- 172.21.15.46:0/313183774 wait complete. 2024-05-08T23:11:23.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 -- 172.21.15.46:0/1759881843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 msgr2=0x7fe3e8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1759881843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e8101460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009990 tx=0x7fe3dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 -- 172.21.15.46:0/1759881843 shutdown_connections 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1759881843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e81061a0 0x7fe3e810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1759881843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e8101460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1759881843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8101a30 0x7fe3e8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.559+0000 7fe3ef400700 1 -- 172.21.15.46:0/1759881843 >> 172.21.15.46:0/1759881843 conn(0x7fe3e80fc8e0 msgr2=0x7fe3e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.560+0000 7fe3ef400700 1 -- 172.21.15.46:0/1759881843 shutdown_connections 2024-05-08T23:11:23.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.560+0000 7fe3ef400700 1 -- 172.21.15.46:0/1759881843 wait complete. 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 Processor -- start 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 -- start start 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e81061a0 0x7fe3e8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819c650 con 0x7fe3e81061a0 2024-05-08T23:11:23.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819c7c0 con 0x7fe3e8101080 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ef400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819c980 con 0x7fe3e819bbd0 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60206/0 (socket says 172.21.15.46:60206) 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.561+0000 7fe3ed400700 1 -- 172.21.15.46:0/1441227933 learned_addr learned my addr 172.21.15.46:0/1441227933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3e7e00700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e81061a0 0x7fe3e8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 -- 172.21.15.46:0/1441227933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 msgr2=0x7fe3e81972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e81972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 -- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e81061a0 msgr2=0x7fe3e8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e81061a0 0x7fe3e8197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 -- 172.21.15.46:0/1441227933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0009c40 con 0x7fe3e819bbd0 2024-05-08T23:11:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ed400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c370 tx=0x7fe3e000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3e5600700 1 -- 172.21.15.46:0/1441227933 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0017070 con 0x7fe3e819bbd0 2024-05-08T23:11:23.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc0095f0 con 0x7fe3e819bbd0 2024-05-08T23:11:23.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3e5600700 1 -- 172.21.15.46:0/1441227933 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e000f040 con 0x7fe3e819bbd0 2024-05-08T23:11:23.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3e5600700 1 -- 172.21.15.46:0/1441227933 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000e050 con 0x7fe3e819bbd0 2024-05-08T23:11:23.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.562+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81a41d0 con 0x7fe3e819bbd0 2024-05-08T23:11:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.565+0000 7fe3e5600700 1 -- 172.21.15.46:0/1441227933 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe3e000cb60 con 0x7fe3e819bbd0 2024-05-08T23:11:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.567+0000 7fe3e5600700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 0x7fe3d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.567+0000 7fe3e5600700 1 -- 172.21.15.46:0/1441227933 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe3e00cc9e0 con 0x7fe3e819bbd0 2024-05-08T23:11:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.567+0000 7fe3eca00700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 0x7fe3d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 Processor -- start 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 -- start start 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe3eca00700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 0x7fe3d00a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006010 tx=0x7fe3dc00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce613b10 0x5637ce60d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 0x5637ce60e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce560150 con 0x5637ce613b10 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce55db10 con 0x5637ce6131f0 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.568+0000 7fe402fccb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce559400 con 0x5637ce60dca0 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37048/0 (socket says 172.21.15.46:37048) 2024-05-08T23:11:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3eca00700 1 -- 172.21.15.46:0/4155992653 learned_addr learned my addr 172.21.15.46:0/4155992653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce613b10 0x5637ce60d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 0x5637ce60e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 -- 172.21.15.46:0/4155992653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 msgr2=0x5637ce6135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 -- 172.21.15.46:0/4155992653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce613b10 msgr2=0x5637ce60d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce613b10 0x5637ce60d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 -- 172.21.15.46:0/4155992653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ce3da3b0 con 0x5637ce60dca0 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3eca00700 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3ed400700 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 0x5637ce60e100 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00c9cc0 tx=0x7fe3e0095ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9a27f0c503b6fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.569+0000 7fe3d9c00700 1 -- 172.21.15.46:0/4155992653 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0099d80 con 0x5637ce60dca0 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe3d9c00700 1 -- 172.21.15.46:0/4155992653 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e0099680 con 0x5637ce60dca0 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe3d9c00700 1 -- 172.21.15.46:0/4155992653 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0097b10 con 0x5637ce60dca0 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 -- 172.21.15.46:0/4155992653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 msgr2=0x5637ce60e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 0x5637ce60e100 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00c9cc0 tx=0x7fe3e0095ad0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 -- 172.21.15.46:0/4155992653 shutdown_connections 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce60dca0 0x5637ce60e100 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce6135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 --2- 172.21.15.46:0/4155992653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce613b10 0x5637ce60d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 -- 172.21.15.46:0/4155992653 >> 172.21.15.46:0/4155992653 conn(0x5637ce567210 msgr2=0x5637ce603df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.570+0000 7fe402fccb80 1 -- 172.21.15.46:0/4155992653 shutdown_connections 2024-05-08T23:11:23.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.571+0000 7fe402fccb80 1 -- 172.21.15.46:0/4155992653 wait complete. 2024-05-08T23:11:23.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.571+0000 7fe402fccb80 1 -- start start 2024-05-08T23:11:23.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.571+0000 7fe402fccb80 1 Processor -- start 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce617450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce617990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce4f5560 con 0x5637ce60dca0 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce4f5200 con 0x5637ce6131f0 2024-05-08T23:11:23.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce3dc6a0 con 0x5637ce613b10 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe402fccb80 10 client.0 _ll_register_callbacks cb 0x5637ce4feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe3ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce617990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce617450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58600/0 (socket says 172.21.15.46:58600) 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.572+0000 7fe3eca00700 1 -- 172.21.15.46:0/109253585 learned_addr learned my addr 172.21.15.46:0/109253585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 -- 172.21.15.46:0/109253585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 msgr2=0x5637ce617450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce617450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 -- 172.21.15.46:0/109253585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 msgr2=0x5637ce616f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 -- 172.21.15.46:0/109253585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e810e590 con 0x5637ce613b10 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3eca00700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ed400700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce617990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3e009d670 tx=0x7fe3e00990a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a850b661f4ab982b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ce000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0099d80 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe3e009d4b0 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637ce4b2270 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e009d170 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe402fccb80 10 client.27739 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.573+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe3e81a41d0 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637ce4b2270 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3e0098c90 con 0x5637ce613b10 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 10 client.27739 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 10 client.27739 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:11:23.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 20 client.27739 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce627230 0x5637ce629650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 4 client.27739 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637ce5d8810 con 0x5637ce627230 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.574+0000 7fe3eca00700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce627230 0x5637ce629650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.575+0000 7fe3eca00700 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce627230 0x5637ce629650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c0004f5fa8dac42 server_cookie=86fa61c1a4689c6e in_seq=0 out_seq=0 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.575+0000 7fe3ce000700 10 client.27739 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe3ce000700 1 -- 172.21.15.46:0/109253585 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637ce5d8810 con 0x5637ce627230 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe3ce000700 10 client.27739 handle_command_reply: tid=0 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 client.27739 shutdown 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 msgr2=0x5637ce617990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce617990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3e009d670 tx=0x7fe3e00990a0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 shutdown_connections 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce627230 0x5637ce629650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce617990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce616f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe3ce000700 0 client.27739 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 --2- 172.21.15.46:0/109253585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce6131f0 0x5637ce617450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 >> 172.21.15.46:0/109253585 conn(0x5637ce567210 msgr2=0x5637ce6035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 shutdown_connections 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 1 -- 172.21.15.46:0/109253585 wait complete. 2024-05-08T23:11:23.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.576+0000 7fe402fccb80 20 client.27739 trim_cache size 0 max 16384 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 Processor -- start 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 -- start start 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce622f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 0x5637ce6057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce619a30 con 0x5637ce60dca0 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce600dc0 con 0x5637ce613b10 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.591+0000 7fe402fccb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce604e80 con 0x5637ce6131f0 2024-05-08T23:11:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60224/0 (socket says 172.21.15.46:60224) 2024-05-08T23:11:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3e7e00700 1 -- 172.21.15.46:0/3305583210 learned_addr learned my addr 172.21.15.46:0/3305583210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3eca00700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce622f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 0x5637ce6057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 -- 172.21.15.46:0/3305583210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 msgr2=0x5637ce603d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 -- 172.21.15.46:0/3305583210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 msgr2=0x5637ce622f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce622f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3ed400700 1 -- 172.21.15.46:0/3305583210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ce386b60 con 0x5637ce613b10 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3e7e00700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.592+0000 7fe3eca00700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce622f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe3ed400700 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 0x5637ce6057e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00995c0 tx=0x7fe3e0099110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0070ed3df06f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe3da600700 1 -- 172.21.15.46:0/3305583210 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0099d80 con 0x5637ce613b10 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe3da600700 1 -- 172.21.15.46:0/3305583210 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e00a1b70 con 0x5637ce613b10 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe3da600700 1 -- 172.21.15.46:0/3305583210 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0084310 con 0x5637ce613b10 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe402fccb80 1 -- 172.21.15.46:0/3305583210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 msgr2=0x5637ce6057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.593+0000 7fe402fccb80 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 0x5637ce6057e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00995c0 tx=0x7fe3e0099110 comp rx=0 tx=0).stop 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 -- 172.21.15.46:0/3305583210 shutdown_connections 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce613b10 0x5637ce6057e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce60dca0 0x5637ce622f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 --2- 172.21.15.46:0/3305583210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce6131f0 0x5637ce603d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 -- 172.21.15.46:0/3305583210 >> 172.21.15.46:0/3305583210 conn(0x5637ce567210 msgr2=0x5637ce60e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 -- 172.21.15.46:0/3305583210 shutdown_connections 2024-05-08T23:11:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 -- 172.21.15.46:0/3305583210 wait complete. 2024-05-08T23:11:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 -- start start 2024-05-08T23:11:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.594+0000 7fe402fccb80 1 Processor -- start 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce60dca0 0x5637ce61e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce6131f0 0x5637ce61c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce61ede0 con 0x5637ce6131f0 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce61cf20 con 0x5637ce60dca0 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce61d0e0 con 0x5637ce613b10 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe402fccb80 10 client.0 _ll_register_callbacks cb 0x5637ce4feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3eca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce60dca0 0x5637ce61e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60226/0 (socket says 172.21.15.46:60226) 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 -- 172.21.15.46:0/792441025 learned_addr learned my addr 172.21.15.46:0/792441025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3e7e00700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce6131f0 0x5637ce61c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 -- 172.21.15.46:0/792441025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce60dca0 msgr2=0x5637ce61e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce60dca0 0x5637ce61e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 -- 172.21.15.46:0/792441025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce6131f0 msgr2=0x5637ce61c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce6131f0 0x5637ce61c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.595+0000 7fe3ed400700 1 -- 172.21.15.46:0/792441025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc005630 con 0x5637ce613b10 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ed400700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0003ff0 tx=0x7fe3e000e1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3423f2faf7bb6656 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0099d80 con 0x5637ce613b10 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe3e00977c0 con 0x5637ce613b10 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637ce551ec0 con 0x5637ce613b10 2024-05-08T23:11:23.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e00a1b70 con 0x5637ce613b10 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe402fccb80 10 client.27745 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe3d4009b20 con 0x5637ce613b10 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637ce551ec0 con 0x5637ce613b10 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.596+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3e0004420 con 0x5637ce613b10 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 10 client.27745 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 10 client.27745 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 20 client.27745 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce67d650 0x5637ce67fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 4 client.27745 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637ce3a7cf0 con 0x5637ce67d650 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe3eca00700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce67d650 0x5637ce67fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe3eca00700 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce67d650 0x5637ce67fa40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd99adedb273152 server_cookie=cdbdb5880cbe74d7 in_seq=0 out_seq=0 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.597+0000 7fe3ce000700 10 client.27745 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe3ce000700 1 -- 172.21.15.46:0/792441025 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637ce3a7cf0 con 0x5637ce67d650 2024-05-08T23:11:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe3ce000700 10 client.27745 handle_command_reply: tid=0 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 client.27745 shutdown 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 msgr2=0x5637ce61c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0003ff0 tx=0x7fe3e000e1b0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 shutdown_connections 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x5637ce67d650 0x5637ce67fa40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637ce613b10 0x5637ce61c9e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637ce60dca0 0x5637ce61e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 --2- 172.21.15.46:0/792441025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637ce6131f0 0x5637ce61c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe3ce000700 0 client.27745 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 >> 172.21.15.46:0/792441025 conn(0x5637ce567210 msgr2=0x5637ce62dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 shutdown_connections 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 1 -- 172.21.15.46:0/792441025 wait complete. 2024-05-08T23:11:23.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.598+0000 7fe402fccb80 20 client.27745 trim_cache size 0 max 16384 2024-05-08T23:11:23.603 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 msgr2=0x7fe3d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 0x7fe3d00a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006010 tx=0x7fe3dc00b260 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 msgr2=0x7fe3e819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c370 tx=0x7fe3e000c680 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 shutdown_connections 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e819bbd0 0x7fe3e819c050 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e8101080 0x7fe3e81972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe3d00a1290 0x7fe3d00a3750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 --2- 172.21.15.46:0/1441227933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e81061a0 0x7fe3e8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 >> 172.21.15.46:0/1441227933 conn(0x7fe3e80fc8e0 msgr2=0x7fe3e80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 shutdown_connections 2024-05-08T23:11:23.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.599+0000 7fe3ef400700 1 -- 172.21.15.46:0/1441227933 wait complete. 2024-05-08T23:11:23.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 -- 172.21.15.46:0/2926463294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101a30 msgr2=0x7f7ddc105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 --2- 172.21.15.46:0/2926463294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101a30 0x7f7ddc105a70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009990 tx=0x7f7dc8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 -- 172.21.15.46:0/2926463294 shutdown_connections 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 --2- 172.21.15.46:0/2926463294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc1061a0 0x7f7ddc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 --2- 172.21.15.46:0/2926463294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc101080 0x7f7ddc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 --2- 172.21.15.46:0/2926463294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101a30 0x7f7ddc105a70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 -- 172.21.15.46:0/2926463294 >> 172.21.15.46:0/2926463294 conn(0x7f7ddc0fc8e0 msgr2=0x7f7ddc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 -- 172.21.15.46:0/2926463294 shutdown_connections 2024-05-08T23:11:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.830+0000 7f7de2c00700 1 -- 172.21.15.46:0/2926463294 wait complete. 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 Processor -- start 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 -- start start 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc1061a0 0x7f7ddc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc19de40 0x7f7ddc19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc19e8c0 con 0x7f7ddc1061a0 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc19ea30 con 0x7f7ddc19de40 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7de2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc19ebf0 con 0x7f7ddc101080 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60246/0 (socket says 172.21.15.46:60246) 2024-05-08T23:11:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.832+0000 7f7ddbe00700 1 -- 172.21.15.46:0/439857054 learned_addr learned my addr 172.21.15.46:0/439857054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddb400700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc1061a0 0x7f7ddc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7de0c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc19de40 0x7f7ddc19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 -- 172.21.15.46:0/439857054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc19de40 msgr2=0x7f7ddc19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc19de40 0x7f7ddc19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 -- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc1061a0 msgr2=0x7f7ddc199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc1061a0 0x7f7ddc199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 -- 172.21.15.46:0/439857054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc80095f0 con 0x7f7ddc101080 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e9a0 tx=0x7f7dcc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7dd8c00700 1 -- 172.21.15.46:0/439857054 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc009790 con 0x7f7ddc101080 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7dd8c00700 1 -- 172.21.15.46:0/439857054 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dcc010430 con 0x7f7ddc101080 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc1a6150 con 0x7f7ddc101080 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.833+0000 7f7dd8c00700 1 -- 172.21.15.46:0/439857054 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc018680 con 0x7f7ddc101080 2024-05-08T23:11:23.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.834+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1a6630 con 0x7f7ddc101080 2024-05-08T23:11:23.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.837+0000 7f7dd8c00700 1 -- 172.21.15.46:0/439857054 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7dcc0105a0 con 0x7f7ddc101080 2024-05-08T23:11:23.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.838+0000 7f7df662bb80 1 Processor -- start 2024-05-08T23:11:23.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 -- start start 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba51eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba5145150 con 0x55eba51f4720 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba5142b10 con 0x55eba51f3e00 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7df662bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba513e400 con 0x55eba51ee890 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.839+0000 7f7dd8c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 0x7f7dc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7dd8c00700 1 -- 172.21.15.46:0/439857054 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7dcc014070 con 0x7f7ddc101080 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37100/0 (socket says 172.21.15.46:37100) 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 -- 172.21.15.46:0/809494152 learned_addr learned my addr 172.21.15.46:0/809494152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7ddb400700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba51eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 0x7f7dc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 -- 172.21.15.46:0/809494152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 msgr2=0x55eba51eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba51eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 -- 172.21.15.46:0/809494152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 msgr2=0x55eba51ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7de0c00700 1 -- 172.21.15.46:0/809494152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba4fbf3b0 con 0x55eba51f3e00 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7ddb400700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba51eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.840+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.841+0000 7f7de0c00700 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4008fa0 tx=0x7f7dd401a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6ba71e61a451c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.841+0000 7f7de0c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 0x7f7dc00a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009fd0 tx=0x7f7dd401e000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.841+0000 7f7dd1200700 1 -- 172.21.15.46:0/809494152 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4027070 con 0x55eba51f3e00 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.841+0000 7f7dd1200700 1 -- 172.21.15.46:0/809494152 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd400f040 con 0x55eba51f3e00 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.841+0000 7f7dd1200700 1 -- 172.21.15.46:0/809494152 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400e030 con 0x55eba51f3e00 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- 172.21.15.46:0/809494152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 msgr2=0x55eba51f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4008fa0 tx=0x7f7dd401a6c0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- 172.21.15.46:0/809494152 shutdown_connections 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba51eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51f41e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 --2- 172.21.15.46:0/809494152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- 172.21.15.46:0/809494152 >> 172.21.15.46:0/809494152 conn(0x55eba514c210 msgr2=0x55eba51e8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- 172.21.15.46:0/809494152 shutdown_connections 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- 172.21.15.46:0/809494152 wait complete. 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.842+0000 7f7df662bb80 1 -- start start 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 Processor -- start 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51ee890 0x55eba51f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba50da560 con 0x55eba51f3e00 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba50da200 con 0x55eba51ee890 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba4fc16a0 con 0x55eba51f4720 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7df662bb80 10 client.0 _ll_register_callbacks cb 0x55eba50e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7de0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58664/0 (socket says 172.21.15.46:58664) 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7ddb400700 1 -- 172.21.15.46:0/1533488037 learned_addr learned my addr 172.21.15.46:0/1533488037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.843+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51ee890 0x55eba51f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 -- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 msgr2=0x55eba51f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 -- 172.21.15.46:0/1533488037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51ee890 msgr2=0x55eba51f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51ee890 0x55eba51f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 -- 172.21.15.46:0/1533488037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1a6630 con 0x55eba51f3e00 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7ddb400700 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8015000 tx=0x7f7dc8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81a942eb1855a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7dc5600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc801d030 con 0x55eba51f3e00 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.844+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7dc8004500 con 0x55eba51f3e00 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eba5097270 con 0x55eba51f3e00 2024-05-08T23:11:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc80046c0 con 0x55eba51f3e00 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 10 client.17691 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f7dd4008b50 con 0x55eba51f3e00 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eba5097270 con 0x55eba51f3e00 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dc800f700 con 0x55eba51f3e00 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 10 client.17691 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 10 client.17691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 20 client.17691 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.845+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5208200 0x55eba520a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.846+0000 7f7df662bb80 4 client.17691 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.846+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eba51bd810 con 0x55eba5208200 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.846+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5208200 0x55eba520a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.846+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5208200 0x55eba520a620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cbe6d4480f76b80 server_cookie=4260488774f356b4 in_seq=0 out_seq=0 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.846+0000 7f7dc5600700 10 client.17691 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7dc5600700 1 -- 172.21.15.46:0/1533488037 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eba51bd810 con 0x55eba5208200 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7dc5600700 10 client.17691 handle_command_reply: tid=0 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 client.17691 shutdown 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 msgr2=0x55eba51f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8015000 tx=0x7f7dc8005800 comp rx=0 tx=0).stop 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 shutdown_connections 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51ee890 0x55eba51f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5208200 0x55eba520a620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 --2- 172.21.15.46:0/1533488037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f3e00 0x55eba51f7ff0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7dc5600700 0 client.17691 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 >> 172.21.15.46:0/1533488037 conn(0x55eba514c210 msgr2=0x55eba51e85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.848+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 shutdown_connections 2024-05-08T23:11:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.849+0000 7f7df662bb80 1 -- 172.21.15.46:0/1533488037 wait complete. 2024-05-08T23:11:23.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.849+0000 7f7df662bb80 20 client.17691 trim_cache size 0 max 16384 2024-05-08T23:11:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.863+0000 7f7df662bb80 1 Processor -- start 2024-05-08T23:11:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.863+0000 7f7df662bb80 1 -- start start 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba51f9120 con 0x55eba51f4720 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba51e5dc0 con 0x55eba51f3e00 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7df662bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba51ee6d0 con 0x55eba51ee890 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60280/0 (socket says 172.21.15.46:60280) 2024-05-08T23:11:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 -- 172.21.15.46:0/2174967304 learned_addr learned my addr 172.21.15.46:0/2174967304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7de0c00700 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 -- 172.21.15.46:0/2174967304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 msgr2=0x55eba51eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 -- 172.21.15.46:0/2174967304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 msgr2=0x55eba51e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.864+0000 7f7ddbe00700 1 -- 172.21.15.46:0/2174967304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba4f6bb60 con 0x55eba51ee890 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.865+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e970 tx=0x7f7dcc0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cce2b83b3dd2d4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.865+0000 7f7dd1c00700 1 -- 172.21.15.46:0/2174967304 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc099fd0 con 0x55eba51ee890 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.865+0000 7f7dd1c00700 1 -- 172.21.15.46:0/2174967304 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dcc09de00 con 0x55eba51ee890 2024-05-08T23:11:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.865+0000 7f7dd1c00700 1 -- 172.21.15.46:0/2174967304 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc09a230 con 0x55eba51ee890 2024-05-08T23:11:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- 172.21.15.46:0/2174967304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 msgr2=0x55eba5204260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e970 tx=0x7f7dcc0cc040 comp rx=0 tx=0).stop 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- 172.21.15.46:0/2174967304 shutdown_connections 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51f4720 0x55eba51e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51ee890 0x55eba5204260 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2174967304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- 172.21.15.46:0/2174967304 >> 172.21.15.46:0/2174967304 conn(0x55eba514c210 msgr2=0x55eba51f2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- 172.21.15.46:0/2174967304 shutdown_connections 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- 172.21.15.46:0/2174967304 wait complete. 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 -- start start 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.866+0000 7f7df662bb80 1 Processor -- start 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51ee890 0x55eba51ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba51fd580 con 0x55eba51ee890 2024-05-08T23:11:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba51fd6f0 con 0x55eba51f3e00 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba5265f10 con 0x55eba51f4720 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7df662bb80 10 client.0 _ll_register_callbacks cb 0x55eba50e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51ee890 0x55eba51ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37130/0 (socket says 172.21.15.46:37130) 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7ddb400700 1 -- 172.21.15.46:0/2195512261 learned_addr learned my addr 172.21.15.46:0/2195512261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.867+0000 7f7de0c00700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 -- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 msgr2=0x55eba51fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 -- 172.21.15.46:0/2195512261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51ee890 msgr2=0x55eba51ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51ee890 0x55eba51ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 -- 172.21.15.46:0/2195512261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1a6150 con 0x55eba51f3e00 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7ddb400700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc800ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f14d3aee4588197 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:23.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7dc5600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:23.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc801d040 con 0x55eba51f3e00 2024-05-08T23:11:23.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dd400f040 con 0x55eba51f3e00 2024-05-08T23:11:23.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.868+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7dc8000e40 con 0x55eba51f3e00 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc800fd50 con 0x55eba51f3e00 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 10 client.27692 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eba5104560 con 0x55eba51f3e00 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd400f040 con 0x55eba51f3e00 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dc8004cc0 con 0x55eba51f3e00 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 10 client.27692 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 10 client.27692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:11:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 20 client.27692 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5267120 0x55eba5269540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 4 client.27692 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.869+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eba5136ec0 con 0x55eba5267120 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.870+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5267120 0x55eba5269540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.870+0000 7f7ddbe00700 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5267120 0x55eba5269540 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a06302f90738ec2 server_cookie=206ffdd0af41b457 in_seq=0 out_seq=0 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.870+0000 7f7dc5600700 10 client.27692 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7dc5600700 1 -- 172.21.15.46:0/2195512261 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eba5136ec0 con 0x55eba5267120 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7dc5600700 10 client.27692 handle_command_reply: tid=0 2024-05-08T23:11:23.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 client.27692 shutdown 2024-05-08T23:11:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 msgr2=0x55eba51fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8006010 tx=0x7f7dc800ba50 comp rx=0 tx=0).stop 2024-05-08T23:11:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 shutdown_connections 2024-05-08T23:11:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eba51f4720 0x55eba51fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55eba5267120 0x55eba5269540 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eba51ee890 0x55eba51ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 --2- 172.21.15.46:0/2195512261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eba51f3e00 0x55eba51fc960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7dc5600700 0 client.27692 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.872+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 >> 172.21.15.46:0/2195512261 conn(0x55eba514c210 msgr2=0x55eba51f3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 shutdown_connections 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7df662bb80 1 -- 172.21.15.46:0/2195512261 wait complete. 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7df662bb80 20 client.27692 trim_cache size 0 max 16384 2024-05-08T23:11:23.875 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 msgr2=0x7f7dc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 0x7f7dc00a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009fd0 tx=0x7f7dd401e000 comp rx=0 tx=0).stop 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 msgr2=0x7f7ddc199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e9a0 tx=0x7f7dcc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 shutdown_connections 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ddc19de40 0x7f7ddc19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ddc101080 0x7f7ddc199560 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7dc00a1290 0x7f7dc00a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 --2- 172.21.15.46:0/439857054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ddc1061a0 0x7f7ddc199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.873+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 >> 172.21.15.46:0/439857054 conn(0x7f7ddc0fc8e0 msgr2=0x7f7ddc0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.874+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 shutdown_connections 2024-05-08T23:11:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:23.874+0000 7f7de2c00700 1 -- 172.21.15.46:0/439857054 wait complete. 2024-05-08T23:11:23.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:23 smithi046 ceph-mon[36898]: pgmap v232: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:23.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:23 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2223977358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:23.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:11:23.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 629005 10.09 MB/sec execute 225 sec latency 13.416 ms 2024-05-08T23:11:24.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:23 smithi053 ceph-mon[77587]: pgmap v232: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:24.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:23 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2223977358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 -- 172.21.15.46:0/1779686682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b4101080 msgr2=0x7f06b4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 --2- 172.21.15.46:0/1779686682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b4101080 0x7f06b4101460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009990 tx=0x7f06a4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 -- 172.21.15.46:0/1779686682 shutdown_connections 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 --2- 172.21.15.46:0/1779686682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b41061a0 0x7f06b410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 --2- 172.21.15.46:0/1779686682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b4101080 0x7f06b4101460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 --2- 172.21.15.46:0/1779686682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101a30 0x7f06b4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.111+0000 7f06ba600700 1 -- 172.21.15.46:0/1779686682 >> 172.21.15.46:0/1779686682 conn(0x7f06b40fc8e0 msgr2=0x7f06b40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.112+0000 7f06ba600700 1 -- 172.21.15.46:0/1779686682 shutdown_connections 2024-05-08T23:11:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.112+0000 7f06ba600700 1 -- 172.21.15.46:0/1779686682 wait complete. 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 Processor -- start 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 -- start start 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 0x7f06b4199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 0x7f06b4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 0x7f06b419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419e870 con 0x7f06b4101080 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419e9e0 con 0x7f06b41061a0 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06ba600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419eba0 con 0x7f06b419ddf0 2024-05-08T23:11:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 0x7f06b4199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 0x7f06b419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06b2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 0x7f06b4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 0x7f06b419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60294/0 (socket says 172.21.15.46:60294) 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.113+0000 7f06b3e00700 1 -- 172.21.15.46:0/960271153 learned_addr learned my addr 172.21.15.46:0/960271153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 -- 172.21.15.46:0/960271153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 msgr2=0x7f06b419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 0x7f06b419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 -- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 msgr2=0x7f06b4199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 0x7f06b4199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 -- 172.21.15.46:0/960271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40095f0 con 0x7f06b41061a0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b3400700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 0x7f06b4199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06b2a00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 0x7f06b4199aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06a000e9c0 tx=0x7f06a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06abe00700 1 -- 172.21.15.46:0/960271153 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000c9c0 con 0x7f06b41061a0 2024-05-08T23:11:24.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.114+0000 7f06abe00700 1 -- 172.21.15.46:0/960271153 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06a0010430 con 0x7f06b41061a0 2024-05-08T23:11:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.115+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b41a6100 con 0x7f06b41061a0 2024-05-08T23:11:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.115+0000 7f06abe00700 1 -- 172.21.15.46:0/960271153 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0018680 con 0x7f06b41061a0 2024-05-08T23:11:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.115+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41a6630 con 0x7f06b41061a0 2024-05-08T23:11:24.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.118+0000 7f06abe00700 1 -- 172.21.15.46:0/960271153 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f06a000cb20 con 0x7f06b41061a0 2024-05-08T23:11:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.119+0000 7f06ce03bb80 1 Processor -- start 2024-05-08T23:11:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.119+0000 7f06ce03bb80 1 -- start start 2024-05-08T23:11:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbd720 0x562a8cfb7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfb7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cf0e150 con 0x562a8cfb7890 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cf0bb10 con 0x562a8cfbce00 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cf07400 con 0x562a8cfbd720 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06b3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06b2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfb7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06b3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37166/0 (socket says 172.21.15.46:37166) 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06b3e00700 1 -- 172.21.15.46:0/1312127344 learned_addr learned my addr 172.21.15.46:0/1312127344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.120+0000 7f06abe00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 0x7f06980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 0x7f06980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06abe00700 1 -- 172.21.15.46:0/960271153 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06a0014070 con 0x7f06b41061a0 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3400700 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbd720 0x562a8cfb7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 -- 172.21.15.46:0/1312127344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbd720 msgr2=0x562a8cfb7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbd720 0x562a8cfb7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 -- 172.21.15.46:0/1312127344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 msgr2=0x562a8cfb7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfb7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 -- 172.21.15.46:0/1312127344 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a8cd883b0 con 0x562a8cfbce00 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b2a00700 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.121+0000 7f06b3e00700 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00ed20 tx=0x7f06ac018360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f35bbd0c4a60825 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06b3e00700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 0x7f06980a37d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009000 tx=0x7f06ac01e000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06a8c00700 1 -- 172.21.15.46:0/1312127344 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00e040 con 0x562a8cfbce00 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06a8c00700 1 -- 172.21.15.46:0/1312127344 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ac00f040 con 0x562a8cfbce00 2024-05-08T23:11:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06a8c00700 1 -- 172.21.15.46:0/1312127344 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac01a920 con 0x562a8cfbce00 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1312127344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 msgr2=0x562a8cfbd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.122+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00ed20 tx=0x7f06ac018360 comp rx=0 tx=0).stop 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1312127344 shutdown_connections 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfbd1e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1312127344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbd720 0x562a8cfb7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1312127344 >> 172.21.15.46:0/1312127344 conn(0x562a8cf15210 msgr2=0x562a8cfb1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1312127344 shutdown_connections 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1312127344 wait complete. 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.123+0000 7f06ce03bb80 1 -- start start 2024-05-08T23:11:24.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 Processor -- start 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfc0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbd720 0x562a8cfc15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cea3560 con 0x562a8cfb7890 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cea3200 con 0x562a8cfbd720 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cd8a6a0 con 0x562a8cfbce00 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06ce03bb80 10 client.0 _ll_register_callbacks cb 0x562a8ceacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06b3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbd720 0x562a8cfc15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06b2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.124+0000 7f06b3400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfc0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60314/0 (socket says 172.21.15.46:60314) 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 -- 172.21.15.46:0/504220450 learned_addr learned my addr 172.21.15.46:0/504220450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 -- 172.21.15.46:0/504220450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbd720 msgr2=0x562a8cfc15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbd720 0x562a8cfc15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 -- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 msgr2=0x562a8cfc0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfc0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 -- 172.21.15.46:0/504220450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41a6630 con 0x562a8cfbce00 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b3400700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfc0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06b2a00700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c5b0 tx=0x7f06a00caa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d506d67a1626d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f069d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a009a570 con 0x562a8cfbce00 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06a00a22f0 con 0x562a8cfbce00 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a8ce60270 con 0x562a8cfbce00 2024-05-08T23:11:24.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.125+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a0099e70 con 0x562a8cfbce00 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 10 client.27775 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f06ac00e8d0 con 0x562a8cfbce00 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a8ce60270 con 0x562a8cfbce00 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06a009a160 con 0x562a8cfbce00 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 10 client.27775 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 10 client.27775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 20 client.27775 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8cfd10f0 0x562a8cfd3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 4 client.27775 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a8cf86810 con 0x562a8cfd10f0 2024-05-08T23:11:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.126+0000 7f06b3400700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8cfd10f0 0x562a8cfd3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.127+0000 7f06b3400700 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8cfd10f0 0x562a8cfd3510 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a20e223413ba058 server_cookie=6ba83f4937065cc6 in_seq=0 out_seq=0 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.127+0000 7f069d600700 10 client.27775 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f069d600700 1 -- 172.21.15.46:0/504220450 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a8cf86810 con 0x562a8cfd10f0 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f069d600700 10 client.27775 handle_command_reply: tid=0 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 client.27775 shutdown 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 msgr2=0x562a8cfc1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c5b0 tx=0x7f06a00caa10 comp rx=0 tx=0).stop 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 shutdown_connections 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbd720 0x562a8cfc15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8cfd10f0 0x562a8cfd3510 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfb7890 0x562a8cfc0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/504220450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfbce00 0x562a8cfc1080 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f069d600700 0 client.27775 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 >> 172.21.15.46:0/504220450 conn(0x562a8cf15210 msgr2=0x562a8cfb15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.128+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 shutdown_connections 2024-05-08T23:11:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.129+0000 7f06ce03bb80 1 -- 172.21.15.46:0/504220450 wait complete. 2024-05-08T23:11:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.129+0000 7f06ce03bb80 20 client.27775 trim_cache size 0 max 16384 2024-05-08T23:11:24.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 Processor -- start 2024-05-08T23:11:24.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 -- start start 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfcd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfb1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cfb3870 con 0x562a8cfbd720 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cfc21b0 con 0x562a8cfbce00 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.144+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cfaedc0 con 0x562a8cfb7890 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60318/0 (socket says 172.21.15.46:60318) 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 -- 172.21.15.46:0/2421687953 learned_addr learned my addr 172.21.15.46:0/2421687953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b2a00700 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfcd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3e00700 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfb1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 -- 172.21.15.46:0/2421687953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 msgr2=0x562a8cfcd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfcd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 -- 172.21.15.46:0/2421687953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 msgr2=0x562a8cfb1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfb1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 -- 172.21.15.46:0/2421687953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a8cd34b60 con 0x562a8cfb7890 2024-05-08T23:11:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.145+0000 7f06b3400700 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06a40098e0 tx=0x7f06a4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc97e3e17da94e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.146+0000 7f06a9600700 1 -- 172.21.15.46:0/2421687953 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401d040 con 0x562a8cfb7890 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.146+0000 7f06a9600700 1 -- 172.21.15.46:0/2421687953 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06a402c070 con 0x562a8cfb7890 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.146+0000 7f06a9600700 1 -- 172.21.15.46:0/2421687953 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4004500 con 0x562a8cfb7890 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- 172.21.15.46:0/2421687953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 msgr2=0x562a8cfb7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06a40098e0 tx=0x7f06a4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- 172.21.15.46:0/2421687953 shutdown_connections 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfb1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfb7350 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/2421687953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfcd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- 172.21.15.46:0/2421687953 >> 172.21.15.46:0/2421687953 conn(0x562a8cf15210 msgr2=0x562a8cfbf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- 172.21.15.46:0/2421687953 shutdown_connections 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- 172.21.15.46:0/2421687953 wait complete. 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.147+0000 7f06ce03bb80 1 -- start start 2024-05-08T23:11:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.148+0000 7f06ce03bb80 1 Processor -- start 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.148+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfc60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfc6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8cfc6e60 con 0x562a8cfbd720 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8d0265f0 con 0x562a8cfbce00 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8d026730 con 0x562a8cfb7890 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06ce03bb80 10 client.0 _ll_register_callbacks cb 0x562a8ceacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60320/0 (socket says 172.21.15.46:60320) 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 -- 172.21.15.46:0/1440396617 learned_addr learned my addr 172.21.15.46:0/1440396617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b2a00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfc60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3e00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfc6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 -- 172.21.15.46:0/1440396617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 msgr2=0x562a8cfc60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfc60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 -- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 msgr2=0x562a8cfc6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfc6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3400700 1 -- 172.21.15.46:0/1440396617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b410e590 con 0x562a8cfb7890 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b3e00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfc6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.149+0000 7f06b2a00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f06b3400700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000c00 tx=0x7f06a4028240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9983159073ef9647 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f069d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401d040 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06ac00f040 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06a40286f0 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.150+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a40288b0 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 10 client.27781 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562a8cecd560 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ac00f040 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06a40048d0 con 0x562a8cfb7890 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 10 client.27781 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 10 client.27781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 20 client.27781 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8d027950 0x562a8d029d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 4 client.27781 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a8ceffec0 con 0x562a8d027950 2024-05-08T23:11:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.151+0000 7f06b2a00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8d027950 0x562a8d029d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06b2a00700 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8d027950 0x562a8d029d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=847034dcb37e70a1 server_cookie=ec758c5c80d1acec in_seq=0 out_seq=0 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f069d600700 10 client.27781 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f069d600700 1 -- 172.21.15.46:0/1440396617 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a8ceffec0 con 0x562a8d027950 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f069d600700 10 client.27781 handle_command_reply: tid=0 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 client.27781 shutdown 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 msgr2=0x562a8cfc85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000c00 tx=0x7f06a4028240 comp rx=0 tx=0).stop 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 shutdown_connections 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x562a8d027950 0x562a8d029d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a8cfbd720 0x562a8cfc6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a8cfb7890 0x562a8cfc85a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f069d600700 0 client.27781 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 --2- 172.21.15.46:0/1440396617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a8cfbce00 0x562a8cfc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.152+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 >> 172.21.15.46:0/1440396617 conn(0x562a8cf15210 msgr2=0x562a8cfbf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 shutdown_connections 2024-05-08T23:11:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ce03bb80 1 -- 172.21.15.46:0/1440396617 wait complete. 2024-05-08T23:11:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ce03bb80 20 client.27781 trim_cache size 0 max 16384 2024-05-08T23:11:24.157 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 msgr2=0x7f06980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 0x7f06980a37d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009000 tx=0x7f06ac01e000 comp rx=0 tx=0).stop 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.153+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 msgr2=0x7f06b4199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 0x7f06b4199aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06a000e9c0 tx=0x7f06a000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 shutdown_connections 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419ddf0 0x7f06b419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06980a1310 0x7f06980a37d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4101080 0x7f06b4199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 --2- 172.21.15.46:0/960271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06b41061a0 0x7f06b4199aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 >> 172.21.15.46:0/960271153 conn(0x7f06b40fc8e0 msgr2=0x7f06b40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 shutdown_connections 2024-05-08T23:11:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.154+0000 7f06ba600700 1 -- 172.21.15.46:0/960271153 wait complete. 2024-05-08T23:11:24.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:11:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:23 smithi086 ceph-mon[78610]: pgmap v232: 129 pgs: 129 active+clean; 418 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-05-08T23:11:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:23 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2223977358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 -- 172.21.15.46:0/2870904655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff954101080 msgr2=0x7ff954101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 --2- 172.21.15.46:0/2870904655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff954101080 0x7ff954101460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 -- 172.21.15.46:0/2870904655 shutdown_connections 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 --2- 172.21.15.46:0/2870904655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9541061a0 0x7ff95410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 --2- 172.21.15.46:0/2870904655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff954101080 0x7ff954101460 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 --2- 172.21.15.46:0/2870904655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101a30 0x7ff954105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 -- 172.21.15.46:0/2870904655 >> 172.21.15.46:0/2870904655 conn(0x7ff9540fc8e0 msgr2=0x7ff9540fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.401+0000 7ff95b200700 1 -- 172.21.15.46:0/2870904655 shutdown_connections 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.402+0000 7ff95b200700 1 -- 172.21.15.46:0/2870904655 wait complete. 2024-05-08T23:11:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.403+0000 7ff95b200700 1 Processor -- start 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.403+0000 7ff95b200700 1 -- start start 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9541061a0 0x7ff954197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 0x7ff95419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419c680 con 0x7ff9541061a0 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419c7f0 con 0x7ff954101080 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff95b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419c9b0 con 0x7ff95419bc00 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff953e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff953e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37222/0 (socket says 172.21.15.46:37222) 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff953e00700 1 -- 172.21.15.46:0/3838955876 learned_addr learned my addr 172.21.15.46:0/3838955876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 0x7ff95419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff953400700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9541061a0 0x7ff954197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 -- 172.21.15.46:0/3838955876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 msgr2=0x7ff954197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 -- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9541061a0 msgr2=0x7ff954197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9541061a0 0x7ff954197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff959200700 1 -- 172.21.15.46:0/3838955876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940009520 con 0x7ff95419bc00 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.404+0000 7ff953e00700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff959200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 0x7ff95419c080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00efc0 tx=0x7ff94c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff950c00700 1 -- 172.21.15.46:0/3838955876 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c018070 con 0x7ff95419bc00 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff950c00700 1 -- 172.21.15.46:0/3838955876 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94c00f040 con 0x7ff95419bc00 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9440095f0 con 0x7ff95419bc00 2024-05-08T23:11:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff950c00700 1 -- 172.21.15.46:0/3838955876 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00cd40 con 0x7ff95419bc00 2024-05-08T23:11:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.405+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9541a41b0 con 0x7ff95419bc00 2024-05-08T23:11:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.408+0000 7ff950c00700 1 -- 172.21.15.46:0/3838955876 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff94c003a00 con 0x7ff95419bc00 2024-05-08T23:11:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.410+0000 7ff950c00700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 0x7ff9380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.410+0000 7ff950c00700 1 -- 172.21.15.46:0/3838955876 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff94c0cccd0 con 0x7ff95419bc00 2024-05-08T23:11:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.410+0000 7ff96edc3b80 1 Processor -- start 2024-05-08T23:11:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.410+0000 7ff953e00700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 0x7ff9380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.410+0000 7ff96edc3b80 1 -- start start 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5ebca0 0x56312b5ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b53e150 con 0x56312b5ebca0 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b53bb10 con 0x56312b5f1b30 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b537400 con 0x56312b5f1210 2024-05-08T23:11:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 0x7ff9380a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff94400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60356/0 (socket says 172.21.15.46:60356) 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 -- 172.21.15.46:0/2881031979 learned_addr learned my addr 172.21.15.46:0/2881031979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff959200700 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5ebca0 0x56312b5ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 -- 172.21.15.46:0/2881031979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 msgr2=0x56312b5eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 -- 172.21.15.46:0/2881031979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5ebca0 msgr2=0x56312b5ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5ebca0 0x56312b5ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.411+0000 7ff953e00700 1 -- 172.21.15.46:0/2881031979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312b3b83b0 con 0x56312b5f1210 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.412+0000 7ff953e00700 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff944005990 tx=0x7ff944020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad65b352aa2bcc66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.412+0000 7ff949600700 1 -- 172.21.15.46:0/2881031979 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944004880 con 0x56312b5f1210 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.412+0000 7ff949600700 1 -- 172.21.15.46:0/2881031979 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94401e040 con 0x56312b5f1210 2024-05-08T23:11:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.412+0000 7ff949600700 1 -- 172.21.15.46:0/2881031979 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94400fe30 con 0x56312b5f1210 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- 172.21.15.46:0/2881031979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 msgr2=0x56312b5f15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff944005990 tx=0x7ff944020000 comp rx=0 tx=0).stop 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- 172.21.15.46:0/2881031979 shutdown_connections 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5ebca0 0x56312b5ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f15f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/2881031979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- 172.21.15.46:0/2881031979 >> 172.21.15.46:0/2881031979 conn(0x56312b545210 msgr2=0x56312b5e1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- 172.21.15.46:0/2881031979 shutdown_connections 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- 172.21.15.46:0/2881031979 wait complete. 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.413+0000 7ff96edc3b80 1 -- start start 2024-05-08T23:11:24.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.414+0000 7ff96edc3b80 1 Processor -- start 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.414+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 0x56312b5f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.414+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b4d3560 con 0x56312b5f1b30 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b4d3200 con 0x56312b5ebca0 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b3ba6a0 con 0x56312b5f1210 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff96edc3b80 10 client.0 _ll_register_callbacks cb 0x56312b4dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 0x56312b5f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 0x56312b5f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37242/0 (socket says 172.21.15.46:37242) 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60368/0 (socket says 172.21.15.46:60368) 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953e00700 1 -- 172.21.15.46:0/981941089 learned_addr learned my addr 172.21.15.46:0/981941089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff959200700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 -- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 msgr2=0x56312b5f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 0x56312b5f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 -- 172.21.15.46:0/981941089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 msgr2=0x56312b5f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.415+0000 7ff953400700 1 -- 172.21.15.46:0/981941089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95410e590 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff953400700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56312b5f1f90 tx=0x7ff940012c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898ba20fb64d1f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff93e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940017070 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56312b490270 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff94000f4a0 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94000f660 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff96edc3b80 10 client.27793 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56312b364b60 con 0x56312b5f1210 2024-05-08T23:11:24.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.416+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56312b490270 con 0x56312b5f1210 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9400108a0 con 0x56312b5f1210 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 10 client.27793 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 10 client.27793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 20 client.27793 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b605790 0x56312b607bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 4 client.27793 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56312b5b6810 con 0x56312b605790 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.417+0000 7ff953e00700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b605790 0x56312b607bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.418+0000 7ff953e00700 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b605790 0x56312b607bb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22a4cec859afbfd server_cookie=343ba5eb9f263173 in_seq=0 out_seq=0 2024-05-08T23:11:24.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.418+0000 7ff93e000700 10 client.27793 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.418+0000 7ff93e000700 1 -- 172.21.15.46:0/981941089 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56312b5b6810 con 0x56312b605790 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.418+0000 7ff93e000700 10 client.27793 handle_command_reply: tid=0 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.418+0000 7ff96edc3b80 1 client.27793 shutdown 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 msgr2=0x56312b5f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56312b5f1f90 tx=0x7ff940012c30 comp rx=0 tx=0).stop 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 shutdown_connections 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b605790 0x56312b607bb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5ebca0 0x56312b5f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff93e000700 0 client.27793 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/981941089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5f1210 0x56312b5f54b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 >> 172.21.15.46:0/981941089 conn(0x56312b545210 msgr2=0x56312b5e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 shutdown_connections 2024-05-08T23:11:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 1 -- 172.21.15.46:0/981941089 wait complete. 2024-05-08T23:11:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.419+0000 7ff96edc3b80 20 client.27793 trim_cache size 0 max 16384 2024-05-08T23:11:24.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.434+0000 7ff96edc3b80 1 Processor -- start 2024-05-08T23:11:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 -- start start 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 0x56312b5f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b5f4d90 con 0x56312b5f1210 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b5e3870 con 0x56312b5f1b30 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b6021c0 con 0x56312b5ebca0 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60380/0 (socket says 172.21.15.46:60380) 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 -- 172.21.15.46:0/803875234 learned_addr learned my addr 172.21.15.46:0/803875234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff959200700 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.435+0000 7ff953400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 0x56312b5f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 -- 172.21.15.46:0/803875234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 msgr2=0x56312b5e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5e1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 -- 172.21.15.46:0/803875234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 msgr2=0x56312b5f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 0x56312b5f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 -- 172.21.15.46:0/803875234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312b4fd560 con 0x56312b5ebca0 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953400700 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 0x56312b5f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff953e00700 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff944009ce0 tx=0x7ff944026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc38e6ecc6bb1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.436+0000 7ff94a000700 1 -- 172.21.15.46:0/803875234 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944034040 con 0x56312b5ebca0 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff94a000700 1 -- 172.21.15.46:0/803875234 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94400f6e0 con 0x56312b5ebca0 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff94a000700 1 -- 172.21.15.46:0/803875234 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440048e0 con 0x56312b5ebca0 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 -- 172.21.15.46:0/803875234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 msgr2=0x56312b5f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff944009ce0 tx=0x7ff944026040 comp rx=0 tx=0).stop 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 -- 172.21.15.46:0/803875234 shutdown_connections 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1b30 0x56312b5e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5f08e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/803875234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1210 0x56312b5f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 -- 172.21.15.46:0/803875234 >> 172.21.15.46:0/803875234 conn(0x56312b545210 msgr2=0x56312b5f4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.437+0000 7ff96edc3b80 1 -- 172.21.15.46:0/803875234 shutdown_connections 2024-05-08T23:11:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.438+0000 7ff96edc3b80 1 -- 172.21.15.46:0/803875234 wait complete. 2024-05-08T23:11:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.438+0000 7ff96edc3b80 1 -- start start 2024-05-08T23:11:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.438+0000 7ff96edc3b80 1 Processor -- start 2024-05-08T23:11:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1210 0x56312b5e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b5e6c90 con 0x56312b5f1b30 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b5e6e00 con 0x56312b5f1210 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312b5fb580 con 0x56312b5ebca0 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff96edc3b80 10 client.0 _ll_register_callbacks cb 0x56312b4dcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60394/0 (socket says 172.21.15.46:60394) 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 -- 172.21.15.46:0/1238468025 learned_addr learned my addr 172.21.15.46:0/1238468025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953400700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1210 0x56312b5e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff959200700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 -- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1210 msgr2=0x56312b5e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1210 0x56312b5e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 -- 172.21.15.46:0/1238468025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 msgr2=0x56312b5e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff953e00700 1 -- 172.21.15.46:0/1238468025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56312b364b60 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.439+0000 7ff959200700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff953e00700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b690 tx=0x7ff94400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c5979515b2c8c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff93e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944034070 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff94400fbd0 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff94401e040 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94403a650 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff96edc3b80 10 client.27799 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56312b3b83b0 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.440+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94401e040 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff94404a070 con 0x56312b5ebca0 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 10 client.27799 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:24.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 10 client.27799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 20 client.27799 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b657f20 0x56312b65a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 4 client.27799 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56312b52fec0 con 0x56312b657f20 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff953400700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b657f20 0x56312b65a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff953400700 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b657f20 0x56312b65a340 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c50ab807740cee server_cookie=cd91bbb37f307fbd in_seq=0 out_seq=0 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.441+0000 7ff93e000700 10 client.27799 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff93e000700 1 -- 172.21.15.46:0/1238468025 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56312b52fec0 con 0x56312b657f20 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff93e000700 10 client.27799 handle_command_reply: tid=0 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 client.27799 shutdown 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 msgr2=0x56312b5fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b690 tx=0x7ff94400f610 comp rx=0 tx=0).stop 2024-05-08T23:11:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 shutdown_connections 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56312b657f20 0x56312b65a340 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56312b5f1b30 0x56312b5e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56312b5ebca0 0x56312b5fc970 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 --2- 172.21.15.46:0/1238468025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56312b5f1210 0x56312b5e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff93e000700 0 client.27799 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 >> 172.21.15.46:0/1238468025 conn(0x56312b545210 msgr2=0x56312b5f2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 shutdown_connections 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 1 -- 172.21.15.46:0/1238468025 wait complete. 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.442+0000 7ff96edc3b80 20 client.27799 trim_cache size 0 max 16384 2024-05-08T23:11:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 msgr2=0x7ff9380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 0x7ff9380a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff94400b260 comp rx=0 tx=0).stop 2024-05-08T23:11:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 msgr2=0x7ff95419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 0x7ff95419c080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00efc0 tx=0x7ff94c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 shutdown_connections 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff95419bc00 0x7ff95419c080 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954101080 0x7ff954197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9380a1290 0x7ff9380a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 --2- 172.21.15.46:0/3838955876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9541061a0 0x7ff954197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 >> 172.21.15.46:0/3838955876 conn(0x7ff9540fc8e0 msgr2=0x7ff9540fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 shutdown_connections 2024-05-08T23:11:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:24.443+0000 7ff95b200700 1 -- 172.21.15.46:0/3838955876 wait complete. 2024-05-08T23:11:24.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 630914 10.08 MB/sec execute 226 sec latency 23.356 ms 2024-05-08T23:11:25.463 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:11:25.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:11:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 -- 172.21.15.46:0/549375194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 msgr2=0x7fdff410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 --2- 172.21.15.46:0/549375194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff410dd20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b0a0 tx=0x7fdfec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 -- 172.21.15.46:0/549375194 shutdown_connections 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 --2- 172.21.15.46:0/549375194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff410dd20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 --2- 172.21.15.46:0/549375194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff4101080 0x7fdff4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 --2- 172.21.15.46:0/549375194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101a30 0x7fdff4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 -- 172.21.15.46:0/549375194 >> 172.21.15.46:0/549375194 conn(0x7fdff40fc8e0 msgr2=0x7fdff40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 -- 172.21.15.46:0/549375194 shutdown_connections 2024-05-08T23:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.689+0000 7fdffc400700 1 -- 172.21.15.46:0/549375194 wait complete. 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.690+0000 7fdffc400700 1 Processor -- start 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.690+0000 7fdffc400700 1 -- start start 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 0x7fdff4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 0x7fdff419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff419e970 con 0x7fdff419def0 2024-05-08T23:11:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff419eae0 con 0x7fdff4101080 2024-05-08T23:11:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff419eca0 con 0x7fdff41061a0 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 0x7fdff4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdffa400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 0x7fdff419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 0x7fdff4199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37272/0 (socket says 172.21.15.46:37272) 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdff9a00700 1 -- 172.21.15.46:0/1530562207 learned_addr learned my addr 172.21.15.46:0/1530562207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.691+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60414/0 (socket says 172.21.15.46:60414) 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 -- 172.21.15.46:0/1530562207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 msgr2=0x7fdff4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 -- 172.21.15.46:0/1530562207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 msgr2=0x7fdff4199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 0x7fdff4199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 -- 172.21.15.46:0/1530562207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009d00 con 0x7fdff419def0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffa400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 0x7fdff419e370 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0062a0 tx=0x7fdfec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdff2000700 1 -- 172.21.15.46:0/1530562207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00e040 con 0x7fdff419def0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdff2000700 1 -- 172.21.15.46:0/1530562207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfec01d480 con 0x7fdff419def0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdff2000700 1 -- 172.21.15.46:0/1530562207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec01b6d0 con 0x7fdff419def0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.692+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff41a61a0 con 0x7fdff419def0 2024-05-08T23:11:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.693+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41a65d0 con 0x7fdff419def0 2024-05-08T23:11:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.696+0000 7fdff2000700 1 -- 172.21.15.46:0/1530562207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdfec01d5f0 con 0x7fdff419def0 2024-05-08T23:11:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 Processor -- start 2024-05-08T23:11:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 -- start start 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fdff2000700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 0x7fdfdc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea61c1d0 con 0x55a6ea6cb7a0 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea619b90 con 0x55a6ea6c5910 2024-05-08T23:11:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.697+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea615480 con 0x55a6ea6cae80 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff2000700 1 -- 172.21.15.46:0/1530562207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdfec0ce450 con 0x7fdff419def0 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60420/0 (socket says 172.21.15.46:60420) 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 -- 172.21.15.46:0/620434992 learned_addr learned my addr 172.21.15.46:0/620434992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdffa400700 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 0x7fdfdc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 -- 172.21.15.46:0/620434992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6c5910 msgr2=0x55a6ea6c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 -- 172.21.15.46:0/620434992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 msgr2=0x55a6ea6c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.698+0000 7fdff9000700 1 -- 172.21.15.46:0/620434992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ea496430 con 0x55a6ea6cae80 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.699+0000 7fdff9000700 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0013b20 tx=0x7fdfe0013e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8770d703e770b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.699+0000 7fdff9000700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 0x7fdfdc0a37d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0018000 tx=0x7fdfe001e000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.699+0000 7fdfe6a00700 1 -- 172.21.15.46:0/620434992 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00099f0 con 0x55a6ea6cae80 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.699+0000 7fdfe6a00700 1 -- 172.21.15.46:0/620434992 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe00044f0 con 0x55a6ea6cae80 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.699+0000 7fdfe6a00700 1 -- 172.21.15.46:0/620434992 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003680 con 0x55a6ea6cae80 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 -- 172.21.15.46:0/620434992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 msgr2=0x55a6ea6cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0013b20 tx=0x7fdfe0013e30 comp rx=0 tx=0).stop 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 -- 172.21.15.46:0/620434992 shutdown_connections 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cb260 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/620434992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 -- 172.21.15.46:0/620434992 >> 172.21.15.46:0/620434992 conn(0x55a6ea623290 msgr2=0x55a6ea6bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 -- 172.21.15.46:0/620434992 shutdown_connections 2024-05-08T23:11:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.700+0000 7fe00fec3b80 1 -- 172.21.15.46:0/620434992 wait complete. 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.701+0000 7fe00fec3b80 1 -- start start 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.701+0000 7fe00fec3b80 1 Processor -- start 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea5b15e0 con 0x55a6ea6c5910 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea5b1280 con 0x55a6ea6cb7a0 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea498720 con 0x55a6ea6cae80 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fe00fec3b80 10 client.0 _ll_register_callbacks cb 0x55a6ea5babb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60436/0 (socket says 172.21.15.46:60436) 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdffa400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 -- 172.21.15.46:0/2643090592 learned_addr learned my addr 172.21.15.46:0/2643090592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 -- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 msgr2=0x55a6ea6cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 -- 172.21.15.46:0/2643090592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 msgr2=0x55a6ea6ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.702+0000 7fdff9000700 1 -- 172.21.15.46:0/2643090592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe001b040 con 0x55a6ea6cae80 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fdff9000700 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0021040 tx=0x7fdfe0005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e14949d1a01a9aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fdfdb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0013310 con 0x55a6ea6cae80 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ea56e2f0 con 0x55a6ea6cae80 2024-05-08T23:11:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdfe0004240 con 0x55a6ea6cae80 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe0003680 con 0x55a6ea6cae80 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fe00fec3b80 10 client.27811 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.703+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a6ea442be0 con 0x55a6ea6cae80 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6ea56e2f0 con 0x55a6ea6cae80 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfe0030020 con 0x55a6ea6cae80 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 10 client.27811 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 10 client.27811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 20 client.27811 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea6df190 0x55a6ea6e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 4 client.27811 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6ea694890 con 0x55a6ea6df190 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.704+0000 7fdff9a00700 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea6df190 0x55a6ea6e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.705+0000 7fdfdb400700 10 client.27811 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.705+0000 7fdff9a00700 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea6df190 0x55a6ea6e15b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3c6a4a47dc925b server_cookie=a6dfde1dcb620563 in_seq=0 out_seq=0 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.705+0000 7fdfdb400700 1 -- 172.21.15.46:0/2643090592 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6ea694890 con 0x55a6ea6df190 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.705+0000 7fdfdb400700 10 client.27811 handle_command_reply: tid=0 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 client.27811 shutdown 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 msgr2=0x55a6ea6cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0021040 tx=0x7fdfe0005df0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 shutdown_connections 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea6df190 0x55a6ea6e15b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2643090592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cf030 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fdfdb400700 0 client.27811 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 >> 172.21.15.46:0/2643090592 conn(0x55a6ea623290 msgr2=0x55a6ea6bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 shutdown_connections 2024-05-08T23:11:25.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2643090592 wait complete. 2024-05-08T23:11:25.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.706+0000 7fe00fec3b80 20 client.27811 trim_cache size 0 max 16384 2024-05-08T23:11:25.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.726+0000 7fe00fec3b80 1 Processor -- start 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.726+0000 7fe00fec3b80 1 -- start start 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea6b8190 con 0x55a6ea6cb7a0 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea6c0bb0 con 0x55a6ea6cae80 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea6d1c70 con 0x55a6ea6c5910 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60450/0 (socket says 172.21.15.46:60450) 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 -- 172.21.15.46:0/101280853 learned_addr learned my addr 172.21.15.46:0/101280853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdffa400700 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9000700 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 -- 172.21.15.46:0/101280853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cae80 msgr2=0x55a6ea6cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 -- 172.21.15.46:0/101280853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 msgr2=0x55a6ea6bfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6bfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 -- 172.21.15.46:0/101280853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ea6bc560 con 0x55a6ea6c5910 2024-05-08T23:11:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.727+0000 7fdff9a00700 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55a6ea6cbc00 tx=0x7fdfe8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a9bca28a9b1813 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fdfe7400700 1 -- 172.21.15.46:0/101280853 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8017070 con 0x55a6ea6c5910 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fdfe7400700 1 -- 172.21.15.46:0/101280853 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe8018040 con 0x55a6ea6c5910 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fdfe7400700 1 -- 172.21.15.46:0/101280853 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8004d90 con 0x55a6ea6c5910 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 -- 172.21.15.46:0/101280853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 msgr2=0x55a6ea6c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55a6ea6cbc00 tx=0x7fdfe8009260 comp rx=0 tx=0).stop 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 -- 172.21.15.46:0/101280853 shutdown_connections 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea6bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c53d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/101280853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cae80 0x55a6ea6cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.728+0000 7fe00fec3b80 1 -- 172.21.15.46:0/101280853 >> 172.21.15.46:0/101280853 conn(0x55a6ea623290 msgr2=0x55a6ea6c9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.729+0000 7fe00fec3b80 1 -- 172.21.15.46:0/101280853 shutdown_connections 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.729+0000 7fe00fec3b80 1 -- 172.21.15.46:0/101280853 wait complete. 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.729+0000 7fe00fec3b80 1 -- start start 2024-05-08T23:11:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.729+0000 7fe00fec3b80 1 Processor -- start 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea735690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea6c86f0 con 0x55a6ea6c5910 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea735bd0 con 0x55a6ea6cb7a0 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ea735d60 con 0x55a6ea6cae80 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fe00fec3b80 10 client.0 _ll_register_callbacks cb 0x55a6ea5babb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60466/0 (socket says 172.21.15.46:60466) 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 -- 172.21.15.46:0/2162944628 learned_addr learned my addr 172.21.15.46:0/2162944628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdffa400700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea735690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 -- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 msgr2=0x55a6ea735690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea735690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 -- 172.21.15.46:0/2162944628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 msgr2=0x55a6ea6c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.730+0000 7fdff9000700 1 -- 172.21.15.46:0/2162944628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe8009d00 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdff9000700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009900 tx=0x7fdfe00117f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a26795fbf563db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002a980 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfe00044f0 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdfe0013610 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe0004240 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fe00fec3b80 10 client.27817 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a6ea5db5e0 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe0048080 con 0x55a6ea6cae80 2024-05-08T23:11:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.731+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfe00113e0 con 0x55a6ea6cae80 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 10 client.27817 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 20 client.27817 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea737110 0x55a6ea739500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 4 client.27817 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a6ea60df40 con 0x55a6ea737110 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fdff9a00700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea737110 0x55a6ea739500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fdff9a00700 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea737110 0x55a6ea739500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90915a9fab663070 server_cookie=f50ba85f4c8b5799 in_seq=0 out_seq=0 2024-05-08T23:11:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.732+0000 7fdfdb400700 10 client.27817 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fdfdb400700 1 -- 172.21.15.46:0/2162944628 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a6ea60df40 con 0x55a6ea737110 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fdfdb400700 10 client.27817 handle_command_reply: tid=0 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 client.27817 shutdown 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 msgr2=0x55a6ea6c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009900 tx=0x7fdfe00117f0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 shutdown_connections 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6ea6cb7a0 0x55a6ea735690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a6ea737110 0x55a6ea739500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6ea6c5910 0x55a6ea6c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 --2- 172.21.15.46:0/2162944628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6ea6cae80 0x55a6ea6c7fb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fdfdb400700 0 client.27817 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 >> 172.21.15.46:0/2162944628 conn(0x55a6ea623290 msgr2=0x55a6ea6cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 shutdown_connections 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 1 -- 172.21.15.46:0/2162944628 wait complete. 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.734+0000 7fe00fec3b80 20 client.27817 trim_cache size 0 max 16384 2024-05-08T23:11:25.738 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "918718e9-f43d-47b9-bcff-7826bd78c201", 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 msgr2=0x7fdfdc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 0x7fdfdc0a37d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0018000 tx=0x7fdfe001e000 comp rx=0 tx=0).stop 2024-05-08T23:11:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 msgr2=0x7fdff419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 0x7fdff419e370 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0062a0 tx=0x7fdfec00bd70 comp rx=0 tx=0).stop 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 shutdown_connections 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdfdc0a1310 0x7fdfdc0a37d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdff419def0 0x7fdff419e370 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff4101080 0x7fdff4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 --2- 172.21.15.46:0/1530562207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff41061a0 0x7fdff4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 >> 172.21.15.46:0/1530562207 conn(0x7fdff40fc8e0 msgr2=0x7fdff40fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 shutdown_connections 2024-05-08T23:11:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.735+0000 7fdffc400700 1 -- 172.21.15.46:0/1530562207 wait complete. 2024-05-08T23:11:25.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:11:25.819 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:25 smithi046 ceph-mon[36898]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-05-08T23:11:25.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 632641 10.09 MB/sec execute 227 sec latency 11.751 ms 2024-05-08T23:11:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 -- 172.21.15.46:0/11372559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041083d0 msgr2=0x7f270410ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 --2- 172.21.15.46:0/11372559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041083d0 0x7f270410ff50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00b0a0 tx=0x7f26fc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 -- 172.21.15.46:0/11372559 shutdown_connections 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 --2- 172.21.15.46:0/11372559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041083d0 0x7f270410ff50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 --2- 172.21.15.46:0/11372559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041032b0 0x7f2704103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 --2- 172.21.15.46:0/11372559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2704103c60 0x7f2704107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 -- 172.21.15.46:0/11372559 >> 172.21.15.46:0/11372559 conn(0x7f27040feb10 msgr2=0x7f2704100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 -- 172.21.15.46:0/11372559 shutdown_connections 2024-05-08T23:11:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.982+0000 7f270a800700 1 -- 172.21.15.46:0/11372559 wait complete. 2024-05-08T23:11:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.983+0000 7f270a800700 1 Processor -- start 2024-05-08T23:11:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.983+0000 7f270a800700 1 -- start start 2024-05-08T23:11:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041083d0 0x7f2704199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270419df40 0x7f270419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270419e9c0 con 0x7f270419df40 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270419eb30 con 0x7f27041083d0 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f270a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270419ecf0 con 0x7f27041032b0 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60482/0 (socket says 172.21.15.46:60482) 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 -- 172.21.15.46:0/1232108209 learned_addr learned my addr 172.21.15.46:0/1232108209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703e00700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270419df40 0x7f270419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2702a00700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041083d0 0x7f2704199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 -- 172.21.15.46:0/1232108209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041083d0 msgr2=0x7f2704199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041083d0 0x7f2704199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 -- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270419df40 msgr2=0x7f270419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270419df40 0x7f270419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.984+0000 7f2703400700 1 -- 172.21.15.46:0/1232108209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc009d00 con 0x7f27041032b0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f2703400700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e9c0 tx=0x7f26f400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f26fbe00700 1 -- 172.21.15.46:0/1232108209 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40097b0 con 0x7f27041032b0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27041a6250 con 0x7f27041032b0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f26fbe00700 1 -- 172.21.15.46:0/1232108209 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f400ee60 con 0x7f27041032b0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f26fbe00700 1 -- 172.21.15.46:0/1232108209 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40054f0 con 0x7f27041032b0 2024-05-08T23:11:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.985+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041a6730 con 0x7f27041032b0 2024-05-08T23:11:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.988+0000 7f26fbe00700 1 -- 172.21.15.46:0/1232108209 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f26f4004750 con 0x7f27041032b0 2024-05-08T23:11:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.990+0000 7f26fbe00700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 0x7f26e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.990+0000 7f26fbe00700 1 -- 172.21.15.46:0/1232108209 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f26f4014070 con 0x7f27041032b0 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.990+0000 7f2702a00700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 0x7f26e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.990+0000 7f271e2a8b80 1 Processor -- start 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.990+0000 7f271e2a8b80 1 -- start start 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4454350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa4454cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa43ab150 con 0x556aa445a720 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa43a8b10 con 0x556aa4454890 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa43a4400 con 0x556aa4459e00 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60484/0 (socket says 172.21.15.46:60484) 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2702a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4454350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 -- 172.21.15.46:0/1908346494 learned_addr learned my addr 172.21.15.46:0/1908346494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703e00700 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa4454cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2702a00700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 0x7f26e80a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f270419abb0 tx=0x7f26f0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 -- 172.21.15.46:0/1908346494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 msgr2=0x556aa4454cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa4454cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 -- 172.21.15.46:0/1908346494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 msgr2=0x556aa4454350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4454350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.991+0000 7f2703400700 1 -- 172.21.15.46:0/1908346494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa42253b0 con 0x556aa4459e00 2024-05-08T23:11:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.992+0000 7f2703400700 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f26f4096a80 tx=0x7f26f409a7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548b76834df6e9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.992+0000 7f26f8c00700 1 -- 172.21.15.46:0/1908346494 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40cd160 con 0x556aa4459e00 2024-05-08T23:11:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.992+0000 7f26f8c00700 1 -- 172.21.15.46:0/1908346494 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f40a2490 con 0x556aa4459e00 2024-05-08T23:11:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.992+0000 7f26f8c00700 1 -- 172.21.15.46:0/1908346494 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4016590 con 0x556aa4459e00 2024-05-08T23:11:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- 172.21.15.46:0/1908346494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 msgr2=0x556aa445a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f26f4096a80 tx=0x7f26f409a7e0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- 172.21.15.46:0/1908346494 shutdown_connections 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa4454cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445a1e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/1908346494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4454350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- 172.21.15.46:0/1908346494 >> 172.21.15.46:0/1908346494 conn(0x556aa43b2210 msgr2=0x556aa444edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- 172.21.15.46:0/1908346494 shutdown_connections 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- 172.21.15.46:0/1908346494 wait complete. 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.993+0000 7f271e2a8b80 1 -- start start 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 Processor -- start 2024-05-08T23:11:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa445dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa445e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa4340560 con 0x556aa445a720 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa4340200 con 0x556aa4454890 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.994+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa42276a0 con 0x556aa4459e00 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f271e2a8b80 10 client.0 _ll_register_callbacks cb 0x556aa4349b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60498/0 (socket says 172.21.15.46:60498) 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 -- 172.21.15.46:0/2414957484 learned_addr learned my addr 172.21.15.46:0/2414957484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2703400700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa445dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2703e00700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa445e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 -- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 msgr2=0x556aa445dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa445dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 -- 172.21.15.46:0/2414957484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 msgr2=0x556aa445e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa445e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 -- 172.21.15.46:0/2414957484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041a6730 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2703e00700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa445e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f2702a00700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2704068d20 tx=0x7f26f00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7dc2a4f2f4e67cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.995+0000 7f26ed600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0022070 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f26f0004030 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556aa42fd270 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f00041f0 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f271e2a8b80 10 client.27829 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556aa41d1b60 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556aa42fd270 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26f00037a0 con 0x556aa4459e00 2024-05-08T23:11:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f271e2a8b80 10 client.27829 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.996+0000 7f271e2a8b80 10 client.27829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.997+0000 7f271e2a8b80 20 client.27829 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.997+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa446e380 0x556aa44707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.997+0000 7f271e2a8b80 4 client.27829 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.997+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aa4423810 con 0x556aa446e380 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.997+0000 7f2703400700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa446e380 0x556aa44707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f2703400700 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa446e380 0x556aa44707a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c386503e3fe644f8 server_cookie=560f97badb2975e in_seq=0 out_seq=0 2024-05-08T23:11:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f26ed600700 10 client.27829 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f26ed600700 1 -- 172.21.15.46:0/2414957484 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556aa4423810 con 0x556aa446e380 2024-05-08T23:11:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f26ed600700 10 client.27829 handle_command_reply: tid=0 2024-05-08T23:11:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 client.27829 shutdown 2024-05-08T23:11:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 msgr2=0x556aa445e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:26.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2704068d20 tx=0x7f26f00047c0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 shutdown_connections 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa446e380 0x556aa44707a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa445e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4454890 0x556aa445dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f26ed600700 0 client.27829 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/2414957484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4459e00 0x556aa445e020 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.998+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 >> 172.21.15.46:0/2414957484 conn(0x556aa43b2210 msgr2=0x556aa444e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.999+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 shutdown_connections 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.999+0000 7f271e2a8b80 1 -- 172.21.15.46:0/2414957484 wait complete. 2024-05-08T23:11:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:25.999+0000 7f271e2a8b80 20 client.27829 trim_cache size 0 max 16384 2024-05-08T23:11:26.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:25 smithi053 ceph-mon[77587]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.020+0000 7f271e2a8b80 1 Processor -- start 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.020+0000 7f271e2a8b80 1 -- start start 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa444ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa44592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa445f150 con 0x556aa445a720 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa44507e0 con 0x556aa4459e00 2024-05-08T23:11:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa4449b30 con 0x556aa4454890 2024-05-08T23:11:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60500/0 (socket says 172.21.15.46:60500) 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f2703400700 1 -- 172.21.15.46:0/4112413549 learned_addr learned my addr 172.21.15.46:0/4112413549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f2703e00700 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa44592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.021+0000 7f2702a00700 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa444ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 -- 172.21.15.46:0/4112413549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 msgr2=0x556aa444ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa444ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 -- 172.21.15.46:0/4112413549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 msgr2=0x556aa44592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa44592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 -- 172.21.15.46:0/4112413549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa4454f40 con 0x556aa4454890 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f2703400700 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f26f40cd2e0 tx=0x7f26f400cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a99c60c60ee5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f26f9600700 1 -- 172.21.15.46:0/4112413549 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4098150 con 0x556aa4454890 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f26f9600700 1 -- 172.21.15.46:0/4112413549 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f40983b0 con 0x556aa4454890 2024-05-08T23:11:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.022+0000 7f26f9600700 1 -- 172.21.15.46:0/4112413549 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f409a270 con 0x556aa4454890 2024-05-08T23:11:26.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 -- 172.21.15.46:0/4112413549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 msgr2=0x556aa4454350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f26f40cd2e0 tx=0x7f26f400cf30 comp rx=0 tx=0).stop 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 -- 172.21.15.46:0/4112413549 shutdown_connections 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa44592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4454350 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/4112413549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa444ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 -- 172.21.15.46:0/4112413549 >> 172.21.15.46:0/4112413549 conn(0x556aa43b2210 msgr2=0x556aa4458bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 -- 172.21.15.46:0/4112413549 shutdown_connections 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.023+0000 7f271e2a8b80 1 -- 172.21.15.46:0/4112413549 wait complete. 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 -- start start 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 Processor -- start 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa4461680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4461bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa44c3a20 con 0x556aa445a720 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa44c3b90 con 0x556aa4459e00 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa44c3d50 con 0x556aa4454890 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.024+0000 7f271e2a8b80 10 client.0 _ll_register_callbacks cb 0x556aa4349b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:60502/0 (socket says 172.21.15.46:60502) 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 -- 172.21.15.46:0/3059195893 learned_addr learned my addr 172.21.15.46:0/3059195893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703e00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4461bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2702a00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa4461680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 -- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 msgr2=0x556aa4461680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa4461680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 -- 172.21.15.46:0/3059195893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 msgr2=0x556aa4461bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4461bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 -- 172.21.15.46:0/3059195893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aa42253b0 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2702a00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa4461680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703e00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4461bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f2703400700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f26f4096510 tx=0x7f26f4099c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26b61e386247a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f26ed600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4099de0 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f26f4098150 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.025+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26f40a2490 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f4016330 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 10 client.27835 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556aa436a560 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f40a2490 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26f40a2000 con 0x556aa4454890 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 10 client.27835 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:26.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 10 client.27835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 20 client.27835 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa44c5010 0x556aa44c7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 4 client.27835 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:11:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556aa439cec0 con 0x556aa44c5010 2024-05-08T23:11:26.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.026+0000 7f2702a00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa44c5010 0x556aa44c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f2702a00700 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa44c5010 0x556aa44c7400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecc9adf7a6887b8 server_cookie=52873d1e8f5fa191 in_seq=0 out_seq=0 2024-05-08T23:11:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f26ed600700 10 client.27835 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f26ed600700 1 -- 172.21.15.46:0/3059195893 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556aa439cec0 con 0x556aa44c5010 2024-05-08T23:11:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f26ed600700 10 client.27835 handle_command_reply: tid=0 2024-05-08T23:11:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f271e2a8b80 1 client.27835 shutdown 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 msgr2=0x556aa4461140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.027+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f26f4096510 tx=0x7f26f4099c80 comp rx=0 tx=0).stop 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 shutdown_connections 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556aa44c5010 0x556aa44c7400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa445a720 0x556aa4461bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aa4454890 0x556aa4461140 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 --2- 172.21.15.46:0/3059195893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556aa4459e00 0x556aa4461680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f26ed600700 0 client.27835 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 >> 172.21.15.46:0/3059195893 conn(0x556aa43b2210 msgr2=0x556aa4459460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 shutdown_connections 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 1 -- 172.21.15.46:0/3059195893 wait complete. 2024-05-08T23:11:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.028+0000 7f271e2a8b80 20 client.27835 trim_cache size 0 max 16384 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "918718e9-f43d-47b9-bcff-7826bd78c201": { 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "918718e9-f43d-47b9-bcff-7826bd78c201", 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:11:26.033 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 msgr2=0x7f26e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 0x7f26e80a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f270419abb0 tx=0x7f26f0009260 comp rx=0 tx=0).stop 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 msgr2=0x7f2704199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e9c0 tx=0x7f26f400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 shutdown_connections 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26e80a1290 0x7f26e80a3750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270419df40 0x7f270419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27041032b0 0x7f2704199660 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 --2- 172.21.15.46:0/1232108209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27041083d0 0x7f2704199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 >> 172.21.15.46:0/1232108209 conn(0x7f27040feb10 msgr2=0x7f2704100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 shutdown_connections 2024-05-08T23:11:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:26.029+0000 7f270a800700 1 -- 172.21.15.46:0/1232108209 wait complete. 2024-05-08T23:11:26.053 INFO:tasks.cephfs.filesystem:scrub status for tag:918718e9-f43d-47b9-bcff-7826bd78c201 - {'path': '/', 'tag': '918718e9-f43d-47b9-bcff-7826bd78c201', 'options': 'recursive,force'} 2024-05-08T23:11:26.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:25 smithi086 ceph-mon[78610]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-05-08T23:11:26.894 INFO:tasks.workunit.client.0.smithi046.stdout: 1 634490 10.07 MB/sec execute 228 sec latency 57.992 ms 2024-05-08T23:11:27.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:26 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:26 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:11:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:11:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:11:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:11:27.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:26 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:11:27.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 636179 10.08 MB/sec execute 229 sec latency 11.730 ms 2024-05-08T23:11:28.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:27 smithi053 ceph-mon[77587]: pgmap v234: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-05-08T23:11:28.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:27 smithi046 ceph-mon[36898]: pgmap v234: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-05-08T23:11:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:27 smithi086 ceph-mon[78610]: pgmap v234: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-05-08T23:11:28.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 638164 10.07 MB/sec execute 230 sec latency 22.876 ms 2024-05-08T23:11:29.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 639831 10.08 MB/sec execute 231 sec latency 11.523 ms 2024-05-08T23:11:30.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:29 smithi046 ceph-mon[36898]: pgmap v235: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-08T23:11:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:29 smithi086 ceph-mon[78610]: pgmap v235: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-08T23:11:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:29 smithi053 ceph-mon[77587]: pgmap v235: 129 pgs: 129 active+clean; 427 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-08T23:11:30.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 641815 10.07 MB/sec execute 232 sec latency 17.878 ms 2024-05-08T23:11:31.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 643510 10.07 MB/sec execute 233 sec latency 14.809 ms 2024-05-08T23:11:32.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:31 smithi046 ceph-mon[36898]: pgmap v236: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-05-08T23:11:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:31 smithi086 ceph-mon[78610]: pgmap v236: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-05-08T23:11:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:31 smithi053 ceph-mon[77587]: pgmap v236: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-05-08T23:11:32.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 645577 10.08 MB/sec execute 234 sec latency 28.051 ms 2024-05-08T23:11:33.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 646424 10.06 MB/sec execute 235 sec latency 263.354 ms 2024-05-08T23:11:34.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:33 smithi046 ceph-mon[36898]: pgmap v237: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:11:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:33 smithi086 ceph-mon[78610]: pgmap v237: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:11:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:33 smithi053 ceph-mon[77587]: pgmap v237: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-08T23:11:34.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 647633 10.03 MB/sec execute 236 sec latency 24.947 ms 2024-05-08T23:11:35.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 649621 10.04 MB/sec execute 237 sec latency 25.022 ms 2024-05-08T23:11:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:35 smithi046 ceph-mon[36898]: pgmap v238: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 276 op/s 2024-05-08T23:11:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:35 smithi086 ceph-mon[78610]: pgmap v238: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 276 op/s 2024-05-08T23:11:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:35 smithi053 ceph-mon[77587]: pgmap v238: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 276 op/s 2024-05-08T23:11:36.895 INFO:tasks.workunit.client.0.smithi046.stdout: 1 651310 10.04 MB/sec execute 238 sec latency 14.889 ms 2024-05-08T23:11:37.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 653510 10.05 MB/sec execute 239 sec latency 17.057 ms 2024-05-08T23:11:38.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:37 smithi046 ceph-mon[36898]: pgmap v239: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:37 smithi086 ceph-mon[78610]: pgmap v239: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:37 smithi053 ceph-mon[77587]: pgmap v239: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:38.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 654430 10.02 MB/sec execute 240 sec latency 50.723 ms 2024-05-08T23:11:39.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 656607 10.03 MB/sec execute 241 sec latency 21.587 ms 2024-05-08T23:11:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:39 smithi086 ceph-mon[78610]: pgmap v240: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:39 smithi053 ceph-mon[77587]: pgmap v240: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:39 smithi046 ceph-mon[36898]: pgmap v240: 129 pgs: 129 active+clean; 471 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-05-08T23:11:40.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 657906 10.02 MB/sec execute 242 sec latency 15.245 ms 2024-05-08T23:11:41.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:41.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:41.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 659964 10.02 MB/sec execute 243 sec latency 16.525 ms 2024-05-08T23:11:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:41 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:11:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:41 smithi086 ceph-mon[78610]: pgmap v241: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-05-08T23:11:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:41 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:11:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:41 smithi053 ceph-mon[77587]: pgmap v241: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-05-08T23:11:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:41 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:11:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:41 smithi046 ceph-mon[36898]: pgmap v241: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-05-08T23:11:42.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 661413 10.01 MB/sec execute 244 sec latency 17.768 ms 2024-05-08T23:11:43.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 663529 10.02 MB/sec execute 245 sec latency 15.927 ms 2024-05-08T23:11:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:43 smithi086 ceph-mon[78610]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-05-08T23:11:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:43 smithi053 ceph-mon[77587]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-05-08T23:11:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:43 smithi046 ceph-mon[36898]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 268 op/s 2024-05-08T23:11:44.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 665112 10.01 MB/sec execute 246 sec latency 14.868 ms 2024-05-08T23:11:45.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 667278 10.02 MB/sec execute 247 sec latency 32.069 ms 2024-05-08T23:11:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:45 smithi086 ceph-mon[78610]: pgmap v243: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 336 op/s 2024-05-08T23:11:46.346 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:45 smithi046 ceph-mon[36898]: pgmap v243: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 336 op/s 2024-05-08T23:11:46.347 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:45 smithi053 ceph-mon[77587]: pgmap v243: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.2 MiB/s wr, 336 op/s 2024-05-08T23:11:46.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 669032 10.02 MB/sec execute 248 sec latency 13.862 ms 2024-05-08T23:11:47.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 671238 10.03 MB/sec execute 249 sec latency 19.899 ms 2024-05-08T23:11:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:47 smithi086 ceph-mon[78610]: pgmap v244: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:47 smithi053 ceph-mon[77587]: pgmap v244: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:47 smithi046 ceph-mon[36898]: pgmap v244: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:48.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 672589 10.02 MB/sec execute 250 sec latency 17.766 ms 2024-05-08T23:11:49.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 674871 10.03 MB/sec execute 251 sec latency 16.281 ms 2024-05-08T23:11:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:49 smithi086 ceph-mon[78610]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:49 smithi053 ceph-mon[77587]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:49 smithi046 ceph-mon[36898]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-05-08T23:11:50.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 676272 10.02 MB/sec execute 252 sec latency 15.486 ms 2024-05-08T23:11:51.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 678533 10.03 MB/sec execute 253 sec latency 16.282 ms 2024-05-08T23:11:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:51 smithi086 ceph-mon[78610]: pgmap v246: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T23:11:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:51 smithi053 ceph-mon[77587]: pgmap v246: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T23:11:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:51 smithi046 ceph-mon[36898]: pgmap v246: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T23:11:52.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 680013 10.01 MB/sec execute 254 sec latency 30.001 ms 2024-05-08T23:11:53.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 682158 10.03 MB/sec execute 255 sec latency 18.414 ms 2024-05-08T23:11:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:53 smithi086 ceph-mon[78610]: pgmap v247: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:11:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:53 smithi053 ceph-mon[77587]: pgmap v247: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:11:54.276 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:53 smithi046 ceph-mon[36898]: pgmap v247: 129 pgs: 129 active+clean; 445 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-05-08T23:11:54.896 INFO:tasks.workunit.client.0.smithi046.stdout: 1 683836 10.01 MB/sec execute 256 sec latency 18.200 ms 2024-05-08T23:11:55.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 685810 10.02 MB/sec execute 257 sec latency 14.222 ms 2024-05-08T23:11:56.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:11:56.116 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:55 smithi046 ceph-mon[36898]: pgmap v248: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-05-08T23:11:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:55 smithi086 ceph-mon[78610]: pgmap v248: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-05-08T23:11:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:55 smithi053 ceph-mon[77587]: pgmap v248: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 -- 172.21.15.46:0/3947459641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13241061c0 msgr2=0x7f132410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 --2- 172.21.15.46:0/3947459641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13241061c0 0x7f132410dd40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f131c009f90 tx=0x7f131c00b330 comp rx=0 tx=0).stop 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 -- 172.21.15.46:0/3947459641 shutdown_connections 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 --2- 172.21.15.46:0/3947459641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13241061c0 0x7f132410dd40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 --2- 172.21.15.46:0/3947459641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241010a0 0x7f1324101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 --2- 172.21.15.46:0/3947459641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1324101a50 0x7f1324105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 -- 172.21.15.46:0/3947459641 >> 172.21.15.46:0/3947459641 conn(0x7f13240fc960 msgr2=0x7f13240fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 -- 172.21.15.46:0/3947459641 shutdown_connections 2024-05-08T23:11:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.283+0000 7f132a200700 1 -- 172.21.15.46:0/3947459641 wait complete. 2024-05-08T23:11:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.284+0000 7f132a200700 1 Processor -- start 2024-05-08T23:11:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 -- start start 2024-05-08T23:11:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 0x7f13240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241061c0 0x7f13240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241aa0a0 con 0x7f13241061c0 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241aa210 con 0x7f13241010a0 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f132a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13241aa3d0 con 0x7f1324074f60 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1323400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 0x7f13240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1323e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1322a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241061c0 0x7f13240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1323e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47980/0 (socket says 172.21.15.46:47980) 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1323e00700 1 -- 172.21.15.46:0/3961790839 learned_addr learned my addr 172.21.15.46:0/3961790839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.285+0000 7f1323400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 0x7f13240732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42128/0 (socket says 172.21.15.46:42128) 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 -- 172.21.15.46:0/3961790839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 msgr2=0x7f13240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 0x7f13240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 -- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241061c0 msgr2=0x7f13240737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241061c0 0x7f13240737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 -- 172.21.15.46:0/3961790839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100095f0 con 0x7f1324074f60 2024-05-08T23:11:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f1323e00700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f131c008a10 tx=0x7f131c008a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f131be00700 1 -- 172.21.15.46:0/3961790839 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00e040 con 0x7f1324074f60 2024-05-08T23:11:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f131be00700 1 -- 172.21.15.46:0/3961790839 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131c00be50 con 0x7f1324074f60 2024-05-08T23:11:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c009c40 con 0x7f1324074f60 2024-05-08T23:11:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.286+0000 7f131be00700 1 -- 172.21.15.46:0/3961790839 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c005040 con 0x7f1324074f60 2024-05-08T23:11:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.287+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13241aa990 con 0x7f1324074f60 2024-05-08T23:11:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.290+0000 7f131be00700 1 -- 172.21.15.46:0/3961790839 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f131c00b990 con 0x7f1324074f60 2024-05-08T23:11:56.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.291+0000 7f131be00700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 0x7f13080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.291+0000 7f131be00700 1 -- 172.21.15.46:0/3961790839 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f131c0ce400 con 0x7f1324074f60 2024-05-08T23:11:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.291+0000 7f133ddc2b80 1 Processor -- start 2024-05-08T23:11:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.291+0000 7f1323400700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 0x7f13080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 -- start start 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d01670a720 0x55d016704350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d016704cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01665b150 con 0x55d01670a720 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d016658b10 con 0x55d016709e00 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d016654400 con 0x55d016704890 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1323400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1322a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d01670a720 0x55d016704350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1323400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42138/0 (socket says 172.21.15.46:42138) 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1323400700 1 -- 172.21.15.46:0/2482799125 learned_addr learned my addr 172.21.15.46:0/2482799125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1323e00700 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d016704cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.292+0000 7f1323400700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 0x7f13080a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1314009c60 tx=0x7f1314009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 -- 172.21.15.46:0/2482799125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 msgr2=0x55d016704cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d016704cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 -- 172.21.15.46:0/2482799125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d01670a720 msgr2=0x55d016704350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d01670a720 0x55d016704350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 -- 172.21.15.46:0/2482799125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0164d53b0 con 0x55d016709e00 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f1323400700 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f131400e430 tx=0x7f1314014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bc48950b94f8e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.293+0000 7f130fe00700 1 -- 172.21.15.46:0/2482799125 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400edd0 con 0x55d016709e00 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f130fe00700 1 -- 172.21.15.46:0/2482799125 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1314021070 con 0x55d016709e00 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f130fe00700 1 -- 172.21.15.46:0/2482799125 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131401d410 con 0x55d016709e00 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 -- 172.21.15.46:0/2482799125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 msgr2=0x55d01670a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f131400e430 tx=0x7f1314014000 comp rx=0 tx=0).stop 2024-05-08T23:11:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 -- 172.21.15.46:0/2482799125 shutdown_connections 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d016704cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d01670a1e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/2482799125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d01670a720 0x55d016704350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 -- 172.21.15.46:0/2482799125 >> 172.21.15.46:0/2482799125 conn(0x55d016662210 msgr2=0x55d0166fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.294+0000 7f133ddc2b80 1 -- 172.21.15.46:0/2482799125 shutdown_connections 2024-05-08T23:11:56.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.295+0000 7f133ddc2b80 1 -- 172.21.15.46:0/2482799125 wait complete. 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.295+0000 7f133ddc2b80 1 -- start start 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.295+0000 7f133ddc2b80 1 Processor -- start 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 0x55d01670e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d01670a720 0x55d01670e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0165f0560 con 0x55d016709e00 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0165f0200 con 0x55d01670a720 2024-05-08T23:11:56.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0164d76a0 con 0x55d016704890 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f133ddc2b80 10 client.0 _ll_register_callbacks cb 0x55d0165f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47998/0 (socket says 172.21.15.46:47998) 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 -- 172.21.15.46:0/553062184 learned_addr learned my addr 172.21.15.46:0/553062184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323e00700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d01670a720 0x55d01670e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1322a00700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 0x55d01670e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 -- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d01670a720 msgr2=0x55d01670e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d01670a720 0x55d01670e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 -- 172.21.15.46:0/553062184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 msgr2=0x55d01670e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 0x55d01670e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1323400700 1 -- 172.21.15.46:0/553062184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1314017070 con 0x55d016704890 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.296+0000 7f1322a00700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 0x55d01670e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f1323400700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1314009e90 tx=0x7f13140037e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e510d4c3d3c1481c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f130cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314005250 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f13140053b0 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0165ad270 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1314005570 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f133ddc2b80 10 client.27847 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.297+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d016481b60 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0165ad270 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1314026890 con 0x55d016704890 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 10 client.27847 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 10 client.27847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 20 client.27847 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01671e110 0x55d016720500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 4 client.27847 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0166d3810 con 0x55d01671e110 2024-05-08T23:11:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.298+0000 7f1322a00700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01671e110 0x55d016720500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.299+0000 7f1322a00700 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01671e110 0x55d016720500 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9921345e41c854 server_cookie=324c012eee941b7a in_seq=0 out_seq=0 2024-05-08T23:11:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.299+0000 7f130cc00700 10 client.27847 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.299+0000 7f130cc00700 1 -- 172.21.15.46:0/553062184 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0166d3810 con 0x55d01671e110 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.299+0000 7f130cc00700 10 client.27847 handle_command_reply: tid=0 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 client.27847 shutdown 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 msgr2=0x55d01670dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1314009e90 tx=0x7f13140037e0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 shutdown_connections 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d01670a720 0x55d01670e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016704890 0x55d01670dbd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01671e110 0x55d016720500 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/553062184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016709e00 0x55d01670e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f130cc00700 0 client.27847 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 >> 172.21.15.46:0/553062184 conn(0x55d016662210 msgr2=0x55d0166fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 shutdown_connections 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 1 -- 172.21.15.46:0/553062184 wait complete. 2024-05-08T23:11:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.300+0000 7f133ddc2b80 20 client.27847 trim_cache size 0 max 16384 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 Processor -- start 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 -- start start 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016704890 0x55d01670e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d016708cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01670bbc0 con 0x55d016704890 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0166ff260 con 0x55d016709e00 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.319+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d016708590 con 0x55d016662210 2024-05-08T23:11:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1322a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016704890 0x55d01670e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48010/0 (socket says 172.21.15.46:48010) 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 -- 172.21.15.46:0/1959234293 learned_addr learned my addr 172.21.15.46:0/1959234293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323e00700 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d016708cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 -- 172.21.15.46:0/1959234293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 msgr2=0x55d016708cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d016708cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 -- 172.21.15.46:0/1959234293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016704890 msgr2=0x55d01670e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016704890 0x55d01670e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 -- 172.21.15.46:0/1959234293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d01661a560 con 0x55d016662210 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323e00700 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d016708cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.320+0000 7f1323400700 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f131400e430 tx=0x7f1314003ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53759dd35002032 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f1319200700 1 -- 172.21.15.46:0/1959234293 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314003900 con 0x55d016662210 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f1319200700 1 -- 172.21.15.46:0/1959234293 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131401b8e0 con 0x55d016662210 2024-05-08T23:11:56.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f1319200700 1 -- 172.21.15.46:0/1959234293 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140266e0 con 0x55d016662210 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 -- 172.21.15.46:0/1959234293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 msgr2=0x55d0166fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f131400e430 tx=0x7f1314003ae0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 -- 172.21.15.46:0/1959234293 shutdown_connections 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016709e00 0x55d016708cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016662210 0x55d0166fed20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/1959234293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016704890 0x55d01670e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.321+0000 7f133ddc2b80 1 -- 172.21.15.46:0/1959234293 >> 172.21.15.46:0/1959234293 conn(0x55d016719ef0 msgr2=0x55d01670c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.322+0000 7f133ddc2b80 1 -- 172.21.15.46:0/1959234293 shutdown_connections 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.322+0000 7f133ddc2b80 1 -- 172.21.15.46:0/1959234293 wait complete. 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.322+0000 7f133ddc2b80 1 -- start start 2024-05-08T23:11:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.322+0000 7f133ddc2b80 1 Processor -- start 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016662210 0x55d016713420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 0x55d01676d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01676dac0 con 0x55d016662210 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01676dc00 con 0x55d016704890 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01676ddc0 con 0x55d016709e00 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f133ddc2b80 10 client.0 _ll_register_callbacks cb 0x55d0165f9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1323e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 0x55d01676d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42160/0 (socket says 172.21.15.46:42160) 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 -- 172.21.15.46:0/4189755992 learned_addr learned my addr 172.21.15.46:0/4189755992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1323400700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016662210 0x55d016713420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 -- 172.21.15.46:0/4189755992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 msgr2=0x55d01676d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 0x55d01676d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 -- 172.21.15.46:0/4189755992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016662210 msgr2=0x55d016713420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016662210 0x55d016713420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1322a00700 1 -- 172.21.15.46:0/4189755992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d016481b60 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.323+0000 7f1323e00700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 0x55d01676d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f1322a00700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1324068d20 tx=0x7f131001f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4b09fdc9cf41aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f130cc00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310010c80 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1314021070 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1310010de0 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1310028420 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f133ddc2b80 10 client.27782 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.324+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d0164d9cd0 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1314021070 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1310027760 con 0x55d016704890 2024-05-08T23:11:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 10 client.27782 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 10 client.27782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 20 client.27782 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01670a720 0x55d016777700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 4 client.27782 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d01664cec0 con 0x55d01670a720 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.325+0000 7f1323400700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01670a720 0x55d016777700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f1323400700 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01670a720 0x55d016777700 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9accc4b707dcec3a server_cookie=728a6f25f2e650b8 in_seq=0 out_seq=0 2024-05-08T23:11:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f130cc00700 10 client.27782 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f130cc00700 1 -- 172.21.15.46:0/4189755992 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d01664cec0 con 0x55d01670a720 2024-05-08T23:11:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f130cc00700 10 client.27782 handle_command_reply: tid=0 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 client.27782 shutdown 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 msgr2=0x55d016713960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1324068d20 tx=0x7f131001f840 comp rx=0 tx=0).stop 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 shutdown_connections 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d016709e00 0x55d01676d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d01670a720 0x55d016777700 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d016662210 0x55d016713420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 --2- 172.21.15.46:0/4189755992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d016704890 0x55d016713960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 >> 172.21.15.46:0/4189755992 conn(0x55d016719ef0 msgr2=0x55d016708cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f130cc00700 0 client.27782 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.326+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 shutdown_connections 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f133ddc2b80 1 -- 172.21.15.46:0/4189755992 wait complete. 2024-05-08T23:11:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f133ddc2b80 20 client.27782 trim_cache size 0 max 16384 2024-05-08T23:11:56.332 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:56.332 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:11:56.332 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:11:56.332 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 msgr2=0x7f13080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 0x7f13080a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1314009c60 tx=0x7f1314009260 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 msgr2=0x7f1324073d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f131c008a10 tx=0x7f131c008a40 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 shutdown_connections 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324074f60 0x7f1324073d60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13241010a0 0x7f13240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f13080a1310 0x7f13080a37d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 --2- 172.21.15.46:0/3961790839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13241061c0 0x7f13240737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 >> 172.21.15.46:0/3961790839 conn(0x7f13240fc960 msgr2=0x7f13240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 shutdown_connections 2024-05-08T23:11:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.327+0000 7f132a200700 1 -- 172.21.15.46:0/3961790839 wait complete. 2024-05-08T23:11:56.346 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:11:56.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:11:56.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 -- 172.21.15.46:0/1120859456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4100e90 msgr2=0x7fa7c4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1120859456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4100e90 0x7fa7c4101270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009990 tx=0x7fa7b8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 -- 172.21.15.46:0/1120859456 shutdown_connections 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1120859456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1120859456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4100e90 0x7fa7c4101270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1120859456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4101840 0x7fa7c4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 -- 172.21.15.46:0/1120859456 >> 172.21.15.46:0/1120859456 conn(0x7fa7c40fc730 msgr2=0x7fa7c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 -- 172.21.15.46:0/1120859456 shutdown_connections 2024-05-08T23:11:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.559+0000 7fa7cac00700 1 -- 172.21.15.46:0/1120859456 wait complete. 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 Processor -- start 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 -- start start 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 0x7fa7c419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4199190 0x7fa7c4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4199c40 con 0x7fa7c4105fb0 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4199db0 con 0x7fa7c4199190 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.560+0000 7fa7cac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4199f70 con 0x7fa7c4100e90 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50974/0 (socket says 172.21.15.46:50974) 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c8e00700 1 -- 172.21.15.46:0/1734640996 learned_addr learned my addr 172.21.15.46:0/1734640996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7ca200700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4199190 0x7fa7c4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 0x7fa7c419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 -- 172.21.15.46:0/1734640996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4199190 msgr2=0x7fa7c4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4199190 0x7fa7c4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 -- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 msgr2=0x7fa7c419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 -- 172.21.15.46:0/1734640996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b0009520 con 0x7fa7c4100e90 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.561+0000 7fa7c9800700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 0x7fa7c419deb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80059d0 tx=0x7fa7b8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.562+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b801d070 con 0x7fa7c4100e90 2024-05-08T23:11:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.562+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7b8022470 con 0x7fa7c4100e90 2024-05-08T23:11:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.562+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b80095f0 con 0x7fa7c4100e90 2024-05-08T23:11:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.562+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8017650 con 0x7fa7c4100e90 2024-05-08T23:11:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.562+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c41a6170 con 0x7fa7c4100e90 2024-05-08T23:11:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.563+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4102360 con 0x7fa7c4100e90 2024-05-08T23:11:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.565+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa7b8022870 con 0x7fa7c4100e90 2024-05-08T23:11:56.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.566+0000 7fa7c2000700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 0x7fa7ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.566+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa7b80ce2b0 con 0x7fa7c4100e90 2024-05-08T23:11:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.567+0000 7fa7c8e00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 0x7fa7ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.567+0000 7fa7c8e00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 0x7fa7ac0a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4105c60 tx=0x7fa7b0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.571+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa7b8097440 con 0x7fa7c4100e90 2024-05-08T23:11:56.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.890+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7c4101840 con 0x7fa7c4100e90 2024-05-08T23:11:56.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.891+0000 7fa7c2000700 1 -- 172.21.15.46:0/1734640996 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fa7b809aa60 con 0x7fa7c4100e90 2024-05-08T23:11:56.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:11:56.891 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:11:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 msgr2=0x7fa7ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 0x7fa7ac0a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4105c60 tx=0x7fa7b0009260 comp rx=0 tx=0).stop 2024-05-08T23:11:56.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 msgr2=0x7fa7c419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 0x7fa7c419deb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80059d0 tx=0x7fa7b8004730 comp rx=0 tx=0).stop 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 shutdown_connections 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4199190 0x7fa7c4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c4100e90 0x7fa7c419deb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa7ac0a1290 0x7fa7ac0a3750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 --2- 172.21.15.46:0/1734640996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c4105fb0 0x7fa7c419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 >> 172.21.15.46:0/1734640996 conn(0x7fa7c40fc730 msgr2=0x7fa7c40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 shutdown_connections 2024-05-08T23:11:56.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:56.894+0000 7fa7cac00700 1 -- 172.21.15.46:0/1734640996 wait complete. 2024-05-08T23:11:56.897 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:11:56.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 687777 10.01 MB/sec execute 258 sec latency 27.565 ms 2024-05-08T23:11:56.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:11:57.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 -- 172.21.15.46:0/2556805444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01061a0 msgr2=0x7efca010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 --2- 172.21.15.46:0/2556805444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01061a0 0x7efca010dd20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efc9800b0a0 tx=0x7efc9800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 -- 172.21.15.46:0/2556805444 shutdown_connections 2024-05-08T23:11:57.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 --2- 172.21.15.46:0/2556805444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01061a0 0x7efca010dd20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 --2- 172.21.15.46:0/2556805444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca0101080 0x7efca0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 --2- 172.21.15.46:0/2556805444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101a30 0x7efca0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 -- 172.21.15.46:0/2556805444 >> 172.21.15.46:0/2556805444 conn(0x7efca00fc8e0 msgr2=0x7efca00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 -- 172.21.15.46:0/2556805444 shutdown_connections 2024-05-08T23:11:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.135+0000 7efca4e00700 1 -- 172.21.15.46:0/2556805444 wait complete. 2024-05-08T23:11:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 Processor -- start 2024-05-08T23:11:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 -- start start 2024-05-08T23:11:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101080 0x7efca00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 0x7efca00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 0x7efca0073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca01aa080 con 0x7efca0101080 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca01aa1f0 con 0x7efca01061a0 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.136+0000 7efca4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca01aa3b0 con 0x7efca0074f60 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 0x7efca00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 0x7efca0073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 0x7efca00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42188/0 (socket says 172.21.15.46:42188) 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9d400700 1 -- 172.21.15.46:0/4277736839 learned_addr learned my addr 172.21.15.46:0/4277736839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9de00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101080 0x7efca00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 -- 172.21.15.46:0/4277736839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 msgr2=0x7efca00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 0x7efca00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 -- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101080 msgr2=0x7efca00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101080 0x7efca00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 -- 172.21.15.46:0/4277736839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc98009d00 con 0x7efca0074f60 2024-05-08T23:11:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.137+0000 7efc9e800700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 0x7efca0073d60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efc9800b070 tx=0x7efc9800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.138+0000 7efc96a00700 1 -- 172.21.15.46:0/4277736839 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9800e040 con 0x7efca0074f60 2024-05-08T23:11:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.138+0000 7efc96a00700 1 -- 172.21.15.46:0/4277736839 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc9801d480 con 0x7efca0074f60 2024-05-08T23:11:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.138+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca01aa630 con 0x7efca0074f60 2024-05-08T23:11:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.138+0000 7efc96a00700 1 -- 172.21.15.46:0/4277736839 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc980052a0 con 0x7efca0074f60 2024-05-08T23:11:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.138+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca01aaa50 con 0x7efca0074f60 2024-05-08T23:11:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.141+0000 7efc96a00700 1 -- 172.21.15.46:0/4277736839 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc9801d5f0 con 0x7efca0074f60 2024-05-08T23:11:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.142+0000 7efc96a00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 0x7efc840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.142+0000 7efc96a00700 1 -- 172.21.15.46:0/4277736839 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efc980cdfe0 con 0x7efca0074f60 2024-05-08T23:11:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.143+0000 7efc9de00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 0x7efc840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.143+0000 7efcb89b6b80 1 Processor -- start 2024-05-08T23:11:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.143+0000 7efc9de00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 0x7efc840a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc90009c60 tx=0x7efc90009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.143+0000 7efcb89b6b80 1 -- start start 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e3720 0x5599c94dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9434150 con 0x5599c94e3720 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9431b10 con 0x5599c94dd890 2024-05-08T23:11:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c942d400 con 0x5599c94e2e00 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48070/0 (socket says 172.21.15.46:48070) 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9de00700 1 -- 172.21.15.46:0/2299325377 learned_addr learned my addr 172.21.15.46:0/2299325377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9d400700 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e3720 0x5599c94dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9e800700 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.144+0000 7efc9de00700 1 -- 172.21.15.46:0/2299325377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 msgr2=0x5599c94ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc9de00700 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc9de00700 1 -- 172.21.15.46:0/2299325377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e3720 msgr2=0x5599c94dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc9de00700 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e3720 0x5599c94dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc9de00700 1 -- 172.21.15.46:0/2299325377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c92ae3b0 con 0x5599c94e2e00 2024-05-08T23:11:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc9de00700 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc90004140 tx=0x7efc90014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757bdf0558f008d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc8b400700 1 -- 172.21.15.46:0/2299325377 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90003b10 con 0x5599c94e2e00 2024-05-08T23:11:57.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc8b400700 1 -- 172.21.15.46:0/2299325377 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc90004cf0 con 0x5599c94e2e00 2024-05-08T23:11:57.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.145+0000 7efc8b400700 1 -- 172.21.15.46:0/2299325377 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900056e0 con 0x5599c94e2e00 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2299325377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 msgr2=0x5599c94e31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efc90004140 tx=0x7efc90014000 comp rx=0 tx=0).stop 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2299325377 shutdown_connections 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e2e00 0x5599c94e31e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2299325377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e3720 0x5599c94dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2299325377 >> 172.21.15.46:0/2299325377 conn(0x5599c943b210 msgr2=0x5599c94d7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2299325377 shutdown_connections 2024-05-08T23:11:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.146+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2299325377 wait complete. 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.147+0000 7efcb89b6b80 1 -- start start 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.147+0000 7efcb89b6b80 1 Processor -- start 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e3720 0x5599c94e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c93c9560 con 0x5599c94e2e00 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c93c9200 con 0x5599c94e3720 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c92b06a0 con 0x5599c94dd890 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efcb89b6b80 10 client.0 _ll_register_callbacks cb 0x5599c93d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48076/0 (socket says 172.21.15.46:48076) 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 -- 172.21.15.46:0/2115746868 learned_addr learned my addr 172.21.15.46:0/2115746868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9d400700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9e800700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e3720 0x5599c94e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 -- 172.21.15.46:0/2115746868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e3720 msgr2=0x5599c94e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e3720 0x5599c94e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 -- 172.21.15.46:0/2115746868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 msgr2=0x5599c94e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 -- 172.21.15.46:0/2115746868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc90017070 con 0x5599c94dd890 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9d400700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc9de00700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc90003da0 tx=0x7efc90009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=389c71a56b6147b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.148+0000 7efc7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9002c070 con 0x5599c94dd890 2024-05-08T23:11:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efc90004450 con 0x5599c94dd890 2024-05-08T23:11:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c9386270 con 0x5599c94dd890 2024-05-08T23:11:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc90004610 con 0x5599c94dd890 2024-05-08T23:11:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efcb89b6b80 10 client.27871 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5599c925ab60 con 0x5599c94dd890 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.149+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599c9386270 con 0x5599c94dd890 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc90024030 con 0x5599c94dd890 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 10 client.27871 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 10 client.27871 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 20 client.27871 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c94f7130 0x5599c94f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 4 client.27871 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c94ac810 con 0x5599c94f7130 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.150+0000 7efc9d400700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c94f7130 0x5599c94f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.151+0000 7efc9d400700 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c94f7130 0x5599c94f9520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=543d5758bd6f27bc server_cookie=2c96eb741e1d1c91 in_seq=0 out_seq=0 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.151+0000 7efc7fe00700 10 client.27871 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.151+0000 7efc7fe00700 1 -- 172.21.15.46:0/2115746868 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599c94ac810 con 0x5599c94f7130 2024-05-08T23:11:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.151+0000 7efc7fe00700 10 client.27871 handle_command_reply: tid=0 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.151+0000 7efcb89b6b80 1 client.27871 shutdown 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 msgr2=0x5599c94e6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efc90003da0 tx=0x7efc90009e50 comp rx=0 tx=0).stop 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 shutdown_connections 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c94f7130 0x5599c94f9520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e3720 0x5599c94e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94dd890 0x5599c94e6b90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2115746868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efc7fe00700 0 client.27871 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 >> 172.21.15.46:0/2115746868 conn(0x5599c943b210 msgr2=0x5599c94d75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 shutdown_connections 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2115746868 wait complete. 2024-05-08T23:11:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.152+0000 7efcb89b6b80 20 client.27871 trim_cache size 0 max 16384 2024-05-08T23:11:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 Processor -- start 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 -- start start 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c94d2800 con 0x5599c94e2e00 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c94e8160 con 0x5599c94dd890 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c94d8ac0 con 0x5599c94e3720 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efc9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.170+0000 7efc9de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48090/0 (socket says 172.21.15.46:48090) 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 -- 172.21.15.46:0/2935742848 learned_addr learned my addr 172.21.15.46:0/2935742848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94d7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42226/0 (socket says 172.21.15.46:42226) 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 -- 172.21.15.46:0/2935742848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 msgr2=0x5599c94d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 -- 172.21.15.46:0/2935742848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 msgr2=0x5599c94e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 -- 172.21.15.46:0/2935742848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c94ddf40 con 0x5599c94e3720 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc9e800700 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc980977e0 tx=0x7efc98009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b177b4058dd062e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc8be00700 1 -- 172.21.15.46:0/2935742848 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9809b470 con 0x5599c94e3720 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc8be00700 1 -- 172.21.15.46:0/2935742848 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc9809ad70 con 0x5599c94e3720 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.171+0000 7efc8be00700 1 -- 172.21.15.46:0/2935742848 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98097610 con 0x5599c94e3720 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2935742848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 msgr2=0x5599c94e1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc980977e0 tx=0x7efc98009040 comp rx=0 tx=0).stop 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2935742848 shutdown_connections 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c94e1c80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94dd890 0x5599c94d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/2935742848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94e2e00 0x5599c94e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2935742848 >> 172.21.15.46:0/2935742848 conn(0x5599c943b210 msgr2=0x5599c94e5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2935742848 shutdown_connections 2024-05-08T23:11:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.172+0000 7efcb89b6b80 1 -- 172.21.15.46:0/2935742848 wait complete. 2024-05-08T23:11:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.173+0000 7efcb89b6b80 1 -- start start 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.173+0000 7efcb89b6b80 1 Processor -- start 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e2e00 0x5599c94ec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9546b50 con 0x5599c94dd890 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9546cc0 con 0x5599c94e2e00 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9546e80 con 0x5599c94e3720 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efcb89b6b80 10 client.0 _ll_register_callbacks cb 0x5599c93d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48106/0 (socket says 172.21.15.46:48106) 2024-05-08T23:11:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51022/0 (socket says 172.21.15.46:51022) 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 -- 172.21.15.46:0/548826393 learned_addr learned my addr 172.21.15.46:0/548826393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9d400700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e2e00 0x5599c94ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 -- 172.21.15.46:0/548826393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e2e00 msgr2=0x5599c94ec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e2e00 0x5599c94ec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 -- 172.21.15.46:0/548826393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 msgr2=0x5599c94ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.174+0000 7efc9e800700 1 -- 172.21.15.46:0/548826393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca01aaa50 con 0x5599c94e3720 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc9de00700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc9e800700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc9809ad00 tx=0x7efc98099320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf4e866c7b9fc3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9809b1d0 con 0x5599c94e3720 2024-05-08T23:11:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efc9809eba0 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc90004cf0 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc9801db20 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efcb89b6b80 10 client.27877 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5599c93f3560 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.175+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc90004cf0 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc980a2cd0 con 0x5599c94e3720 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 10 client.27877 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 10 client.27877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 20 client.27877 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c954e180 0x5599c9550570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 4 client.27877 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599c9425ec0 con 0x5599c954e180 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efc9de00700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c954e180 0x5599c9550570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efc9de00700 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c954e180 0x5599c9550570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c04f7a504293a0e0 server_cookie=44745ebdd61fa817 in_seq=0 out_seq=0 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.176+0000 7efc7fe00700 10 client.27877 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efc7fe00700 1 -- 172.21.15.46:0/548826393 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599c9425ec0 con 0x5599c954e180 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efc7fe00700 10 client.27877 handle_command_reply: tid=0 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 client.27877 shutdown 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 msgr2=0x5599c9546610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc9809ad00 tx=0x7efc98099320 comp rx=0 tx=0).stop 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 shutdown_connections 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5599c954e180 0x5599c9550570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599c94e3720 0x5599c9546610 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599c94dd890 0x5599c94ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efc7fe00700 0 client.27877 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 --2- 172.21.15.46:0/548826393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599c94e2e00 0x5599c94ec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 >> 172.21.15.46:0/548826393 conn(0x5599c943b210 msgr2=0x5599c94e1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 shutdown_connections 2024-05-08T23:11:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 1 -- 172.21.15.46:0/548826393 wait complete. 2024-05-08T23:11:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.177+0000 7efcb89b6b80 20 client.27877 trim_cache size 0 max 16384 2024-05-08T23:11:57.183 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 msgr2=0x7efc840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 0x7efc840a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc90009c60 tx=0x7efc90009260 comp rx=0 tx=0).stop 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 msgr2=0x7efca0073d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 0x7efca0073d60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efc9800b070 tx=0x7efc9800bcf0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 shutdown_connections 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca0074f60 0x7efca0073d60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efc840a1310 0x7efc840a37d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0101080 0x7efca00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 --2- 172.21.15.46:0/4277736839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efca01061a0 0x7efca00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 >> 172.21.15.46:0/4277736839 conn(0x7efca00fc8e0 msgr2=0x7efca00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 shutdown_connections 2024-05-08T23:11:57.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.178+0000 7efca4e00700 1 -- 172.21.15.46:0/4277736839 wait complete. 2024-05-08T23:11:57.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 -- 172.21.15.46:0/285169128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 msgr2=0x7ff5c410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 --2- 172.21.15.46:0/285169128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c410dd20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009f90 tx=0x7ff5bc00b330 comp rx=0 tx=0).stop 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 -- 172.21.15.46:0/285169128 shutdown_connections 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 --2- 172.21.15.46:0/285169128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c410dd20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 --2- 172.21.15.46:0/285169128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 --2- 172.21.15.46:0/285169128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c4101a30 0x7ff5c4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 -- 172.21.15.46:0/285169128 >> 172.21.15.46:0/285169128 conn(0x7ff5c40fc8e0 msgr2=0x7ff5c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 -- 172.21.15.46:0/285169128 shutdown_connections 2024-05-08T23:11:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.424+0000 7ff5cae00700 1 -- 172.21.15.46:0/285169128 wait complete. 2024-05-08T23:11:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.425+0000 7ff5cae00700 1 Processor -- start 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 -- start start 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c419bd50 0x7ff5c419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419c7d0 con 0x7ff5c419bd50 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419c940 con 0x7ff5c4101080 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5cae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c419cb00 con 0x7ff5c41061a0 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48126/0 (socket says 172.21.15.46:48126) 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3400700 1 -- 172.21.15.46:0/3281975020 learned_addr learned my addr 172.21.15.46:0/3281975020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c419bd50 0x7ff5c419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3400700 1 -- 172.21.15.46:0/3281975020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 msgr2=0x7ff5c4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.426+0000 7ff5c3400700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c3400700 1 -- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c419bd50 msgr2=0x7ff5c419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c3400700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c419bd50 0x7ff5c419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c3400700 1 -- 172.21.15.46:0/3281975020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009c40 con 0x7ff5c41061a0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c3400700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e970 tx=0x7ff5b000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c0c00700 1 -- 172.21.15.46:0/3281975020 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0009790 con 0x7ff5c41061a0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c0c00700 1 -- 172.21.15.46:0/3281975020 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b000eec0 con 0x7ff5c41061a0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c41a4060 con 0x7ff5c41061a0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5c0c00700 1 -- 172.21.15.46:0/3281975020 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0005590 con 0x7ff5c41061a0 2024-05-08T23:11:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.427+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c41a4590 con 0x7ff5c41061a0 2024-05-08T23:11:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.430+0000 7ff5c0c00700 1 -- 172.21.15.46:0/3281975020 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff5b000cbd0 con 0x7ff5c41061a0 2024-05-08T23:11:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.431+0000 7ff5c0c00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 0x7ff5a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.432+0000 7ff5c0c00700 1 -- 172.21.15.46:0/3281975020 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff5b0014070 con 0x7ff5c41061a0 2024-05-08T23:11:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.432+0000 7ff5de937b80 1 Processor -- start 2024-05-08T23:11:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.432+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 0x7ff5a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.432+0000 7ff5de937b80 1 -- start start 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 0x7ff5a80a37d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b4009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e59adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e5904150 con 0x55f6e59ad890 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e5901b10 con 0x55f6e59b3720 2024-05-08T23:11:57.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5de937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e58fd400 con 0x55f6e59b2e00 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e59adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48136/0 (socket says 172.21.15.46:48136) 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3169621811 learned_addr learned my addr 172.21.15.46:0/3169621811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3400700 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3169621811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 msgr2=0x55f6e59ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3169621811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 msgr2=0x55f6e59adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e59adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3169621811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e577e3b0 con 0x55f6e59b2e00 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.433+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e59adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.434+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4011f80 tx=0x7ff5b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7fb9ef4f11ab97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.434+0000 7ff5b9600700 1 -- 172.21.15.46:0/3169621811 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4009ea0 con 0x55f6e59b2e00 2024-05-08T23:11:57.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.434+0000 7ff5b9600700 1 -- 172.21.15.46:0/3169621811 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b4016070 con 0x55f6e59b2e00 2024-05-08T23:11:57.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.434+0000 7ff5b9600700 1 -- 172.21.15.46:0/3169621811 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b401d610 con 0x55f6e59b2e00 2024-05-08T23:11:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 -- 172.21.15.46:0/3169621811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 msgr2=0x55f6e59b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4011f80 tx=0x7ff5b4014000 comp rx=0 tx=0).stop 2024-05-08T23:11:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 -- 172.21.15.46:0/3169621811 shutdown_connections 2024-05-08T23:11:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e59adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e59b31e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3169621811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 -- 172.21.15.46:0/3169621811 >> 172.21.15.46:0/3169621811 conn(0x55f6e590b210 msgr2=0x55f6e59a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 -- 172.21.15.46:0/3169621811 shutdown_connections 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.435+0000 7ff5de937b80 1 -- 172.21.15.46:0/3169621811 wait complete. 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 -- start start 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 Processor -- start 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b2e00 0x55f6e59b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 0x55f6e59b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e5899560 con 0x55f6e59b3720 2024-05-08T23:11:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e5899200 con 0x55f6e59b2e00 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.436+0000 7ff5de937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e57806a0 con 0x55f6e59ad890 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5de937b80 10 client.0 _ll_register_callbacks cb 0x55f6e58a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48142/0 (socket says 172.21.15.46:48142) 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 -- 172.21.15.46:0/780719756 learned_addr learned my addr 172.21.15.46:0/780719756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b2e00 0x55f6e59b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 0x55f6e59b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 -- 172.21.15.46:0/780719756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b2e00 msgr2=0x55f6e59b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b2e00 0x55f6e59b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 -- 172.21.15.46:0/780719756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 msgr2=0x55f6e59b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 0x55f6e59b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 -- 172.21.15.46:0/780719756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c410e590 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 0x55f6e59b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40099a0 tx=0x7ff5b401f650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c50d2298b22d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.437+0000 7ff5ae000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b402d070 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff5b4018070 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6e5856270 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b401fea0 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5de937b80 10 client.27889 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff5b4011c60 con 0x55f6e59ad890 2024-05-08T23:11:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6e5856270 con 0x55f6e59ad890 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.438+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5b4004910 con 0x55f6e59ad890 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 10 client.27889 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 10 client.27889 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 20 client.27889 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e59c70a0 0x55f6e59c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 4 client.27889 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6e597c810 con 0x55f6e59c70a0 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5c3400700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e59c70a0 0x55f6e59c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5c3400700 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e59c70a0 0x55f6e59c94c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f4f76d136c7fc04 server_cookie=9f0a23b829382b83 in_seq=0 out_seq=0 2024-05-08T23:11:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.439+0000 7ff5ae000700 10 client.27889 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5ae000700 1 -- 172.21.15.46:0/780719756 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6e597c810 con 0x55f6e59c70a0 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5ae000700 10 client.27889 handle_command_reply: tid=0 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 client.27889 shutdown 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 msgr2=0x55f6e59b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40099a0 tx=0x7ff5b401f650 comp rx=0 tx=0).stop 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 shutdown_connections 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e59c70a0 0x55f6e59c94c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b3720 0x55f6e59b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59b6b90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5ae000700 0 client.27889 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 --2- 172.21.15.46:0/780719756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b2e00 0x55f6e59b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 >> 172.21.15.46:0/780719756 conn(0x55f6e590b210 msgr2=0x55f6e59a75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 shutdown_connections 2024-05-08T23:11:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.440+0000 7ff5de937b80 1 -- 172.21.15.46:0/780719756 wait complete. 2024-05-08T23:11:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.441+0000 7ff5de937b80 20 client.27889 trim_cache size 0 max 16384 2024-05-08T23:11:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 Processor -- start 2024-05-08T23:11:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 -- start start 2024-05-08T23:11:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b2e00 0x55f6e59a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59a2800 con 0x55f6e59b2e00 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59ce6b0 con 0x55f6e59b3720 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5de937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59b4b10 con 0x55f6e59ad890 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48152/0 (socket says 172.21.15.46:48152) 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.456+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3097375570 learned_addr learned my addr 172.21.15.46:0/3097375570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3400700 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b2e00 0x55f6e59a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3097375570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 msgr2=0x55f6e59a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3097375570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b2e00 msgr2=0x55f6e59a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b2e00 0x55f6e59a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 -- 172.21.15.46:0/3097375570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e572ab60 con 0x55f6e59ad890 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005c80 tx=0x7ff5b40059e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f85893d51d6830a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5ba000700 1 -- 172.21.15.46:0/3097375570 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b402d030 con 0x55f6e59ad890 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5ba000700 1 -- 172.21.15.46:0/3097375570 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b4012a10 con 0x55f6e59ad890 2024-05-08T23:11:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.457+0000 7ff5ba000700 1 -- 172.21.15.46:0/3097375570 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40055d0 con 0x55f6e59ad890 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 -- 172.21.15.46:0/3097375570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 msgr2=0x55f6e59c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005c80 tx=0x7ff5b40059e0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 -- 172.21.15.46:0/3097375570 shutdown_connections 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59ad890 0x55f6e59c9490 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 --2- 172.21.15.46:0/3097375570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59b2e00 0x55f6e59a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 -- 172.21.15.46:0/3097375570 >> 172.21.15.46:0/3097375570 conn(0x55f6e590b210 msgr2=0x55f6e59a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 -- 172.21.15.46:0/3097375570 shutdown_connections 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.458+0000 7ff5de937b80 1 -- 172.21.15.46:0/3097375570 wait complete. 2024-05-08T23:11:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.459+0000 7ff5de937b80 1 -- start start 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.459+0000 7ff5de937b80 1 Processor -- start 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e5a15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59bb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59bb960 con 0x55f6e59ad890 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59bbad0 con 0x55f6e59b3720 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e59bbc90 con 0x55f6e59b2e00 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5de937b80 10 client.0 _ll_register_callbacks cb 0x55f6e58a2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e5a15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48158/0 (socket says 172.21.15.46:48158) 2024-05-08T23:11:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 -- 172.21.15.46:0/514617493 learned_addr learned my addr 172.21.15.46:0/514617493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c8e00700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59bb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 -- 172.21.15.46:0/514617493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 msgr2=0x55f6e59bb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59bb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 -- 172.21.15.46:0/514617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 msgr2=0x55f6e5a15910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e5a15910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.460+0000 7ff5c3400700 1 -- 172.21.15.46:0/514617493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b4011c60 con 0x55f6e59b2e00 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5c3400700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000c5b0 tx=0x7ff5b0099a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be23918429c1ae07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5ae000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00a23d0 con 0x55f6e59b2e00 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff5b0083040 con 0x55f6e59b2e00 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5b4016070 con 0x55f6e59b2e00 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b00b8200 con 0x55f6e59b2e00 2024-05-08T23:11:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5de937b80 10 client.27895 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f6e58c3560 con 0x55f6e59b2e00 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.461+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b4016070 con 0x55f6e59b2e00 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5b0099440 con 0x55f6e59b2e00 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 10 client.27895 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 10 client.27895 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 20 client.27895 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e5a1d7e0 0x55f6e5a1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 4 client.27895 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6e58f5ec0 con 0x55f6e5a1d7e0 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e5a1d7e0 0x55f6e5a1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5c3e00700 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e5a1d7e0 0x55f6e5a1fc00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96e4ffcd4d1a803a server_cookie=f287d0269ac9a98a in_seq=0 out_seq=0 2024-05-08T23:11:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.462+0000 7ff5ae000700 10 client.27895 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5ae000700 1 -- 172.21.15.46:0/514617493 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6e58f5ec0 con 0x55f6e5a1d7e0 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5ae000700 10 client.27895 handle_command_reply: tid=0 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 client.27895 shutdown 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 msgr2=0x55f6e5a15e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000c5b0 tx=0x7ff5b0099a90 comp rx=0 tx=0).stop 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 shutdown_connections 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f6e5a1d7e0 0x55f6e5a1fc00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e59b3720 0x55f6e59bb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6e59ad890 0x55f6e5a15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 --2- 172.21.15.46:0/514617493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6e59b2e00 0x55f6e5a15e50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 >> 172.21.15.46:0/514617493 conn(0x55f6e590b210 msgr2=0x55f6e59b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5ae000700 0 client.27895 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:11:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 shutdown_connections 2024-05-08T23:11:57.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 1 -- 172.21.15.46:0/514617493 wait complete. 2024-05-08T23:11:57.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.463+0000 7ff5de937b80 20 client.27895 trim_cache size 0 max 16384 2024-05-08T23:11:57.469 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 msgr2=0x7ff5a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 0x7ff5a80a37d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b4009310 comp rx=0 tx=0).stop 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 msgr2=0x7ff5c41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e970 tx=0x7ff5b000ec80 comp rx=0 tx=0).stop 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 shutdown_connections 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5a80a1310 0x7ff5a80a37d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c419bd50 0x7ff5c419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c4101080 0x7ff5c4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 --2- 172.21.15.46:0/3281975020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c41061a0 0x7ff5c41979b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 >> 172.21.15.46:0/3281975020 conn(0x7ff5c40fc8e0 msgr2=0x7ff5c40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 shutdown_connections 2024-05-08T23:11:57.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.464+0000 7ff5cae00700 1 -- 172.21.15.46:0/3281975020 wait complete. 2024-05-08T23:11:57.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 -- 172.21.15.46:0/1125257199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 msgr2=0x7f4a2410fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1125257199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2410fb20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009f90 tx=0x7f4a1c00b330 comp rx=0 tx=0).stop 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 -- 172.21.15.46:0/1125257199 shutdown_connections 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1125257199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2410fb20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1125257199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a240ffb80 0x7f4a240fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1125257199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a24100530 0x7f4a2410d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 -- 172.21.15.46:0/1125257199 >> 172.21.15.46:0/1125257199 conn(0x7f4a240fb760 msgr2=0x7f4a240fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 -- 172.21.15.46:0/1125257199 shutdown_connections 2024-05-08T23:11:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.704+0000 7f4a29a00700 1 -- 172.21.15.46:0/1125257199 wait complete. 2024-05-08T23:11:57.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.705+0000 7f4a29a00700 1 Processor -- start 2024-05-08T23:11:57.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.705+0000 7f4a29a00700 1 -- start start 2024-05-08T23:11:57.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 0x7f4a24199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a24100530 0x7f4a24199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2419e550 con 0x7f4a240ffb80 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2419e6c0 con 0x7f4a24100530 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a29a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2419e880 con 0x7f4a2410d730 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 0x7f4a24199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48174/0 (socket says 172.21.15.46:48174) 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 0x7f4a24199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51112/0 (socket says 172.21.15.46:51112) 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a23400700 1 -- 172.21.15.46:0/1341423549 learned_addr learned my addr 172.21.15.46:0/1341423549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.706+0000 7f4a22000700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a24100530 0x7f4a24199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 -- 172.21.15.46:0/1341423549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a24100530 msgr2=0x7f4a24199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a24100530 0x7f4a24199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 -- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 msgr2=0x7f4a24199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 0x7f4a24199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 -- 172.21.15.46:0/1341423549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10009520 con 0x7f4a2410d730 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a23400700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009f60 tx=0x7f4a1c004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a1b400700 1 -- 172.21.15.46:0/1341423549 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00e040 con 0x7f4a2410d730 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a1b400700 1 -- 172.21.15.46:0/1341423549 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a1c01b450 con 0x7f4a2410d730 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c009c40 con 0x7f4a2410d730 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a1b400700 1 -- 172.21.15.46:0/1341423549 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c009300 con 0x7f4a2410d730 2024-05-08T23:11:57.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.707+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a241a61c0 con 0x7f4a2410d730 2024-05-08T23:11:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.710+0000 7f4a1b400700 1 -- 172.21.15.46:0/1341423549 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4a1c01b8c0 con 0x7f4a2410d730 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.711+0000 7f4a1b400700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 0x7f4a080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a1b400700 1 -- 172.21.15.46:0/1341423549 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4a1c0ce4f0 con 0x7f4a2410d730 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a3d5deb80 1 Processor -- start 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 0x7f4a080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a3d5deb80 1 -- start start 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.712+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d333350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d333cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d28a150 con 0x562c8d339720 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d287b10 con 0x562c8d333890 2024-05-08T23:11:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d283400 con 0x562c8d338e00 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 0x7f4a080a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a14005fd0 tx=0x7f4a14009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d333350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48190/0 (socket says 172.21.15.46:48190) 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 -- 172.21.15.46:0/4188358458 learned_addr learned my addr 172.21.15.46:0/4188358458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a23400700 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d333cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 -- 172.21.15.46:0/4188358458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 msgr2=0x562c8d333cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d333cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 -- 172.21.15.46:0/4188358458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 msgr2=0x562c8d333350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d333350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 -- 172.21.15.46:0/4188358458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c8d1043b0 con 0x562c8d338e00 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a23400700 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d333cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.713+0000 7f4a22a00700 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4a14004140 tx=0x7f4a14014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9ba19d72b11285 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.714+0000 7f4a0fe00700 1 -- 172.21.15.46:0/4188358458 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14003b10 con 0x562c8d338e00 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.714+0000 7f4a0fe00700 1 -- 172.21.15.46:0/4188358458 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14003c70 con 0x562c8d338e00 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.714+0000 7f4a0fe00700 1 -- 172.21.15.46:0/4188358458 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14026600 con 0x562c8d338e00 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.714+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/4188358458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 msgr2=0x562c8d3391e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.714+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4a14004140 tx=0x7f4a14014000 comp rx=0 tx=0).stop 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/4188358458 shutdown_connections 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d333cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d3391e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/4188358458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d333350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/4188358458 >> 172.21.15.46:0/4188358458 conn(0x562c8d291210 msgr2=0x562c8d32ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/4188358458 shutdown_connections 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/4188358458 wait complete. 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.715+0000 7f4a3d5deb80 1 -- start start 2024-05-08T23:11:57.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 Processor -- start 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d33cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d33d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d21f560 con 0x562c8d339720 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d21f200 con 0x562c8d333890 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d1066a0 con 0x562c8d338e00 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a3d5deb80 10 client.0 _ll_register_callbacks cb 0x562c8d228b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48192/0 (socket says 172.21.15.46:48192) 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a22000700 1 -- 172.21.15.46:0/2320441919 learned_addr learned my addr 172.21.15.46:0/2320441919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a23400700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d33d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.716+0000 7f4a22a00700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d33cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 -- 172.21.15.46:0/2320441919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 msgr2=0x562c8d33cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d33cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 -- 172.21.15.46:0/2320441919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 msgr2=0x562c8d33d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d33d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 -- 172.21.15.46:0/2320441919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a14017070 con 0x562c8d338e00 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a23400700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d33d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a22000700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a10012880 tx=0x7f4a10012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d8e02df968baa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10017070 con 0x562c8d338e00 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4a1000f560 con 0x562c8d338e00 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.717+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c8d1dc270 con 0x562c8d338e00 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1000f720 con 0x562c8d338e00 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 10 client.27907 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c8d0b0b60 con 0x562c8d338e00 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c8d1dc270 con 0x562c8d338e00 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a100109d0 con 0x562c8d338e00 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 10 client.27907 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 10 client.27907 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 20 client.27907 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d34d3b0 0x562c8d34f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 4 client.27907 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c8d302810 con 0x562c8d34d3b0 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.718+0000 7f4a22a00700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d34d3b0 0x562c8d34f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.719+0000 7f4a22a00700 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d34d3b0 0x562c8d34f7d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4b43b913698173b server_cookie=d6bbf2d63f9b6242 in_seq=0 out_seq=0 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.719+0000 7f4a0cc00700 10 client.27907 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a0cc00700 1 -- 172.21.15.46:0/2320441919 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c8d302810 con 0x562c8d34d3b0 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a0cc00700 10 client.27907 handle_command_reply: tid=0 2024-05-08T23:11:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 client.27907 shutdown 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 msgr2=0x562c8d33d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a10012880 tx=0x7f4a10012b90 comp rx=0 tx=0).stop 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 shutdown_connections 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d34d3b0 0x562c8d34f7d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d339720 0x562c8d33d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d33cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a0cc00700 0 client.27907 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/2320441919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d338e00 0x562c8d33d090 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 >> 172.21.15.46:0/2320441919 conn(0x562c8d291210 msgr2=0x562c8d32d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 shutdown_connections 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.720+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/2320441919 wait complete. 2024-05-08T23:11:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.721+0000 7f4a3d5deb80 20 client.27907 trim_cache size 0 max 16384 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.736+0000 7f4a3d5deb80 1 Processor -- start 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.736+0000 7f4a3d5deb80 1 -- start start 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d348c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d32ee80 con 0x562c8d338e00 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d32f070 con 0x562c8d333890 2024-05-08T23:11:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d328800 con 0x562c8d339720 2024-05-08T23:11:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2024-05-08T23:11:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 -- 172.21.15.46:0/649143979 learned_addr learned my addr 172.21.15.46:0/649143979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42328/0 (socket says 172.21.15.46:42328) 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a22000700 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d348c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 -- 172.21.15.46:0/649143979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 msgr2=0x562c8d349320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 -- 172.21.15.46:0/649143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 msgr2=0x562c8d348c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d348c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.737+0000 7f4a23400700 1 -- 172.21.15.46:0/649143979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c8d249560 con 0x562c8d339720 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.738+0000 7f4a22a00700 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.738+0000 7f4a23400700 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c09af20 tx=0x7f4a1c009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d255037e4d83483 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.738+0000 7f4a18c00700 1 -- 172.21.15.46:0/649143979 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c09b220 con 0x562c8d339720 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.738+0000 7f4a18c00700 1 -- 172.21.15.46:0/649143979 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a1c09f050 con 0x562c8d339720 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.738+0000 7f4a18c00700 1 -- 172.21.15.46:0/649143979 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c09b480 con 0x562c8d339720 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/649143979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 msgr2=0x562c8d32dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c09af20 tx=0x7f4a1c009040 comp rx=0 tx=0).stop 2024-05-08T23:11:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/649143979 shutdown_connections 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d32dd20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d349320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/649143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d348c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/649143979 >> 172.21.15.46:0/649143979 conn(0x562c8d291210 msgr2=0x562c8d336d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/649143979 shutdown_connections 2024-05-08T23:11:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/649143979 wait complete. 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.739+0000 7f4a3d5deb80 1 -- start start 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 Processor -- start 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d3426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d342db0 con 0x562c8d338e00 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.740+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d342f20 con 0x562c8d333890 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a3d5deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c8d399810 con 0x562c8d339720 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a3d5deb80 10 client.0 _ll_register_callbacks cb 0x562c8d228b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d3426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a22000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a22a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a22000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51162/0 (socket says 172.21.15.46:51162) 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a22000700 1 -- 172.21.15.46:0/1819364612 learned_addr learned my addr 172.21.15.46:0/1819364612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.741+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42332/0 (socket says 172.21.15.46:42332) 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 -- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 msgr2=0x562c8d3426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d3426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 -- 172.21.15.46:0/1819364612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 msgr2=0x562c8d342180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 -- 172.21.15.46:0/1819364612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c8d1043b0 con 0x562c8d333890 2024-05-08T23:11:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a23400700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d3426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22000700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a22a00700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a010 tx=0x7f4a1401ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7174ee9d21e90d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.742+0000 7f4a0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402b040 con 0x562c8d333890 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4a1401e420 con 0x562c8d333890 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a14003c70 con 0x562c8d333890 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1401e5e0 con 0x562c8d333890 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a3d5deb80 10 client.27842 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.743+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f4a241a61c0 con 0x562c8d333890 2024-05-08T23:11:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a14003c70 con 0x562c8d333890 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a14004170 con 0x562c8d333890 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 10 client.27842 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 10 client.27842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 20 client.27842 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d39aee0 0x562c8d39d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 4 client.27842 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.744+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c8d27bec0 con 0x562c8d39aee0 2024-05-08T23:11:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.745+0000 7f4a22000700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d39aee0 0x562c8d39d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.746+0000 7f4a0cc00700 10 client.27842 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.746+0000 7f4a22000700 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d39aee0 0x562c8d39d300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcf889d74e63edd2 server_cookie=59a5eee0533550f3 in_seq=0 out_seq=0 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.746+0000 7f4a0cc00700 1 -- 172.21.15.46:0/1819364612 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c8d27bec0 con 0x562c8d39aee0 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.746+0000 7f4a0cc00700 10 client.27842 handle_command_reply: tid=0 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.746+0000 7f4a3d5deb80 1 client.27842 shutdown 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 msgr2=0x562c8d3444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a010 tx=0x7f4a1401ec10 comp rx=0 tx=0).stop 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 shutdown_connections 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c8d339720 0x562c8d3426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c8d333890 0x562c8d3444f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x562c8d39aee0 0x562c8d39d300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 --2- 172.21.15.46:0/1819364612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8d338e00 0x562c8d342180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 >> 172.21.15.46:0/1819364612 conn(0x562c8d291210 msgr2=0x562c8d33b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a0cc00700 0 client.27842 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:11:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 shutdown_connections 2024-05-08T23:11:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 1 -- 172.21.15.46:0/1819364612 wait complete. 2024-05-08T23:11:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.747+0000 7f4a3d5deb80 20 client.27842 trim_cache size 0 max 16384 2024-05-08T23:11:57.749 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 msgr2=0x7f4a080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 0x7f4a080a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a14005fd0 tx=0x7f4a14009310 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 msgr2=0x7f4a2419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009f60 tx=0x7f4a1c004f80 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 shutdown_connections 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a2410d730 0x7f4a2419df10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4a080a1290 0x7f4a080a3750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a240ffb80 0x7f4a24199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 --2- 172.21.15.46:0/1341423549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a24100530 0x7f4a24199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 >> 172.21.15.46:0/1341423549 conn(0x7f4a240fb760 msgr2=0x7f4a240fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 shutdown_connections 2024-05-08T23:11:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.748+0000 7f4a29a00700 1 -- 172.21.15.46:0/1341423549 wait complete. 2024-05-08T23:11:57.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:11:57.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 689422 10.02 MB/sec execute 259 sec latency 13.561 ms 2024-05-08T23:11:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.985+0000 7fed66c00700 1 -- 172.21.15.46:0/120863047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed601061c0 msgr2=0x7fed6010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.985+0000 7fed66c00700 1 --2- 172.21.15.46:0/120863047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed601061c0 0x7fed6010dd40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b0a0 tx=0x7fed5800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 -- 172.21.15.46:0/120863047 shutdown_connections 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 --2- 172.21.15.46:0/120863047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed601061c0 0x7fed6010dd40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 --2- 172.21.15.46:0/120863047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601010a0 0x7fed60101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 --2- 172.21.15.46:0/120863047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed60101a50 0x7fed60105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 -- 172.21.15.46:0/120863047 >> 172.21.15.46:0/120863047 conn(0x7fed600fc920 msgr2=0x7fed600fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 -- 172.21.15.46:0/120863047 shutdown_connections 2024-05-08T23:11:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.986+0000 7fed66c00700 1 -- 172.21.15.46:0/120863047 wait complete. 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.987+0000 7fed66c00700 1 Processor -- start 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.987+0000 7fed66c00700 1 -- start start 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.987+0000 7fed66c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.987+0000 7fed66c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601061c0 0x7fed600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.987+0000 7fed66c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60073d20 0x7fed601a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed66c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a5ca0 con 0x7fed601010a0 2024-05-08T23:11:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed66c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a5de0 con 0x7fed601061c0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed66c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a5fa0 con 0x7fed60073d20 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51184/0 (socket says 172.21.15.46:51184) 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 -- 172.21.15.46:0/3799887594 learned_addr learned my addr 172.21.15.46:0/3799887594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5f400700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601061c0 0x7fed600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed64c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60073d20 0x7fed601a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 -- 172.21.15.46:0/3799887594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60073d20 msgr2=0x7fed601a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60073d20 0x7fed601a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 -- 172.21.15.46:0/3799887594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601061c0 msgr2=0x7fed600737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601061c0 0x7fed600737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.988+0000 7fed5fe00700 1 -- 172.21.15.46:0/3799887594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58009d00 con 0x7fed601010a0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed5fe00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9c0 tx=0x7fed5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed5cc00700 1 -- 172.21.15.46:0/3799887594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500097b0 con 0x7fed601010a0 2024-05-08T23:11:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed601a6280 con 0x7fed601010a0 2024-05-08T23:11:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed5cc00700 1 -- 172.21.15.46:0/3799887594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5000ee60 con 0x7fed601010a0 2024-05-08T23:11:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed5cc00700 1 -- 172.21.15.46:0/3799887594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500054f0 con 0x7fed601010a0 2024-05-08T23:11:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.989+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed601a6740 con 0x7fed601010a0 2024-05-08T23:11:57.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.992+0000 7fed5cc00700 1 -- 172.21.15.46:0/3799887594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fed50004750 con 0x7fed601010a0 2024-05-08T23:11:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.994+0000 7fed5cc00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 0x7fed440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.994+0000 7fed5cc00700 1 -- 172.21.15.46:0/3799887594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fed50014070 con 0x7fed601010a0 2024-05-08T23:11:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.994+0000 7fed7a766b80 1 Processor -- start 2024-05-08T23:11:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.994+0000 7fed5f400700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 0x7fed440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.994+0000 7fed7a766b80 1 -- start start 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5f400700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 0x7fed440a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed60074980 tx=0x7fed4c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fc6890 0x5640b6fc6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6f1d150 con 0x5640b6fcc720 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6f1ab10 con 0x5640b6fc6890 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed7a766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6f16400 con 0x5640b6fcbe00 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48226/0 (socket says 172.21.15.46:48226) 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5fe00700 1 -- 172.21.15.46:0/567597684 learned_addr learned my addr 172.21.15.46:0/567597684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:57.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed64c00700 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fc6890 0x5640b6fc6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5f400700 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5fe00700 1 -- 172.21.15.46:0/567597684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fc6890 msgr2=0x5640b6fc6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.995+0000 7fed5fe00700 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fc6890 0x5640b6fc6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed5fe00700 1 -- 172.21.15.46:0/567597684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 msgr2=0x5640b6fc6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed5fe00700 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fc6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed5fe00700 1 -- 172.21.15.46:0/567597684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b6d973b0 con 0x5640b6fcbe00 2024-05-08T23:11:57.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed5fe00700 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed50096ad0 tx=0x7fed5009a830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f4d4cdadae67dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed55200700 1 -- 172.21.15.46:0/567597684 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500a24e0 con 0x5640b6fcbe00 2024-05-08T23:11:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed55200700 1 -- 172.21.15.46:0/567597684 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5000cbd0 con 0x5640b6fcbe00 2024-05-08T23:11:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.996+0000 7fed55200700 1 -- 172.21.15.46:0/567597684 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50016330 con 0x5640b6fcbe00 2024-05-08T23:11:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 -- 172.21.15.46:0/567597684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 msgr2=0x5640b6fcc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fed50096ad0 tx=0x7fed5009a830 comp rx=0 tx=0).stop 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 -- 172.21.15.46:0/567597684 shutdown_connections 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fc6890 0x5640b6fc6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcbe00 0x5640b6fcc1e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 --2- 172.21.15.46:0/567597684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fc6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 -- 172.21.15.46:0/567597684 >> 172.21.15.46:0/567597684 conn(0x5640b6f24210 msgr2=0x5640b6fc0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 -- 172.21.15.46:0/567597684 shutdown_connections 2024-05-08T23:11:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.997+0000 7fed7a766b80 1 -- 172.21.15.46:0/567597684 wait complete. 2024-05-08T23:11:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.998+0000 7fed7a766b80 1 -- start start 2024-05-08T23:11:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 Processor -- start 2024-05-08T23:11:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fd0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fd0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6eb2560 con 0x5640b6fcc720 2024-05-08T23:11:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6eb2200 con 0x5640b6fcbe00 2024-05-08T23:11:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6d996a0 con 0x5640b6fc6890 2024-05-08T23:11:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:57.999+0000 7fed7a766b80 10 client.0 _ll_register_callbacks cb 0x5640b6ebbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48232/0 (socket says 172.21.15.46:48232) 2024-05-08T23:11:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fd0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 -- 172.21.15.46:0/156992810 learned_addr learned my addr 172.21.15.46:0/156992810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 -- 172.21.15.46:0/156992810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 msgr2=0x5640b6fd0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fd0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 -- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 msgr2=0x5640b6fd0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fd0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 -- 172.21.15.46:0/156992810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed601a6740 con 0x5640b6fc6890 2024-05-08T23:11:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed5fe00700 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed500cac20 tx=0x7fed5000efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175db680b57b1de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed49600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500cd160 con 0x5640b6fc6890 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fed500b82c0 con 0x5640b6fc6890 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.000+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b6e6f270 con 0x5640b6fc6890 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed50016330 con 0x5640b6fc6890 2024-05-08T23:11:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 10 client.27925 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5640b6d43b60 con 0x5640b6fc6890 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640b6e6f270 con 0x5640b6fc6890 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed50099550 con 0x5640b6fc6890 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 10 client.27925 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 10 client.27925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:11:58.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 20 client.27925 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b6fdfe60 0x5640b6fe2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 4 client.27925 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.001+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640b6f95810 con 0x5640b6fdfe60 2024-05-08T23:11:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.002+0000 7fed5f400700 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b6fdfe60 0x5640b6fe2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.002+0000 7fed49600700 10 client.27925 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.002+0000 7fed5f400700 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b6fdfe60 0x5640b6fe2250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21c1fc368623712e server_cookie=76af4fb7047971e2 in_seq=0 out_seq=0 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.003+0000 7fed49600700 1 -- 172.21.15.46:0/156992810 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640b6f95810 con 0x5640b6fdfe60 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.003+0000 7fed49600700 10 client.27925 handle_command_reply: tid=0 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.003+0000 7fed7a766b80 1 client.27925 shutdown 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 msgr2=0x5640b6fcfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed500cac20 tx=0x7fed5000efd0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 shutdown_connections 2024-05-08T23:11:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b6fdfe60 0x5640b6fe2250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcc720 0x5640b6fd0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fcfad0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 --2- 172.21.15.46:0/156992810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fd0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed49600700 0 client.27925 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 >> 172.21.15.46:0/156992810 conn(0x5640b6f24210 msgr2=0x5640b6fc05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 shutdown_connections 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 1 -- 172.21.15.46:0/156992810 wait complete. 2024-05-08T23:11:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.004+0000 7fed7a766b80 20 client.27925 trim_cache size 0 max 16384 2024-05-08T23:11:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.019+0000 7fed7a766b80 1 Processor -- start 2024-05-08T23:11:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 -- start start 2024-05-08T23:11:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 0x5640b6fc6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fdbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 0x5640b6fc0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fc2e50 con 0x5640b6fc6890 2024-05-08T23:11:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fe7460 con 0x5640b6fcbe00 2024-05-08T23:11:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed7a766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fbb800 con 0x5640b6fcc720 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 0x5640b6fc6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed64c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 0x5640b6fc0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed64c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 0x5640b6fc0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48246/0 (socket says 172.21.15.46:48246) 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed64c00700 1 -- 172.21.15.46:0/2585310738 learned_addr learned my addr 172.21.15.46:0/2585310738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.020+0000 7fed5f400700 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fdbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 -- 172.21.15.46:0/2585310738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 msgr2=0x5640b6fc0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 0x5640b6fc0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 -- 172.21.15.46:0/2585310738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 msgr2=0x5640b6fdbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fdbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 -- 172.21.15.46:0/2585310738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b6edc560 con 0x5640b6fc6890 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5f400700 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fdbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed5fe00700 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 0x5640b6fc6350 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fed500998b0 tx=0x7fed50098610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d19e8150b412ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed55c00700 1 -- 172.21.15.46:0/2585310738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500a2050 con 0x5640b6fc6890 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed55c00700 1 -- 172.21.15.46:0/2585310738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5007a030 con 0x5640b6fc6890 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.021+0000 7fed55c00700 1 -- 172.21.15.46:0/2585310738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500a2240 con 0x5640b6fc6890 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- 172.21.15.46:0/2585310738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 msgr2=0x5640b6fc6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 0x5640b6fc6350 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fed500998b0 tx=0x7fed50098610 comp rx=0 tx=0).stop 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- 172.21.15.46:0/2585310738 shutdown_connections 2024-05-08T23:11:58.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fcc720 0x5640b6fc0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fc6890 0x5640b6fc6350 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 --2- 172.21.15.46:0/2585310738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcbe00 0x5640b6fdbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- 172.21.15.46:0/2585310738 >> 172.21.15.46:0/2585310738 conn(0x5640b6f24210 msgr2=0x5640b6fc6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- 172.21.15.46:0/2585310738 shutdown_connections 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- 172.21.15.46:0/2585310738 wait complete. 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.022+0000 7fed7a766b80 1 -- start start 2024-05-08T23:11:58.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.023+0000 7fed7a766b80 1 Processor -- start 2024-05-08T23:11:58.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.023+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.023+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 0x5640b6fd5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.023+0000 7fed7a766b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcc720 0x5640b6fd5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.023+0000 7fed7a766b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fd5bd0 con 0x5640b6fcbe00 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed7a766b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fd5d40 con 0x5640b6fcc720 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed7a766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b6fd5eb0 con 0x5640b6fc6890 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed7a766b80 10 client.0 _ll_register_callbacks cb 0x5640b6ebbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48250/0 (socket says 172.21.15.46:48250) 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 -- 172.21.15.46:0/4019084109 learned_addr learned my addr 172.21.15.46:0/4019084109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5f400700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 0x5640b6fd5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed64c00700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcc720 0x5640b6fd5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 -- 172.21.15.46:0/4019084109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcc720 msgr2=0x5640b6fd5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcc720 0x5640b6fd5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 -- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 msgr2=0x5640b6fd5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 0x5640b6fd5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 -- 172.21.15.46:0/4019084109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c017070 con 0x5640b6fc6890 2024-05-08T23:11:58.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5f400700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 0x5640b6fd5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed5fe00700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed50096560 tx=0x7fed500ca780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914cb3b7c0257bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.024+0000 7fed49600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50099e30 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fed500981a0 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5000cbd0 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed500cd160 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 10 client.27931 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5640b6d43b60 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5000cbd0 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed50098fc0 con 0x5640b6fc6890 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 10 client.27931 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:58.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 10 client.27931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 20 client.27931 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b7036370 0x5640b7038760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 4 client.27931 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.025+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5640b6f0eec0 con 0x5640b7036370 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.026+0000 7fed5f400700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b7036370 0x5640b7038760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.026+0000 7fed5f400700 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b7036370 0x5640b7038760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7b02b30ff09fce5 server_cookie=1ec0d2508f5c9f9d in_seq=0 out_seq=0 2024-05-08T23:11:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.026+0000 7fed49600700 10 client.27931 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:58.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:57 smithi046 ceph-mon[36898]: pgmap v249: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:11:58.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1734640996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed49600700 1 -- 172.21.15.46:0/4019084109 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5640b6f0eec0 con 0x5640b7036370 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed49600700 10 client.27931 handle_command_reply: tid=0 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 client.27931 shutdown 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 msgr2=0x5640b6fd7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed50096560 tx=0x7fed500ca780 comp rx=0 tx=0).stop 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 shutdown_connections 2024-05-08T23:11:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640b6fcc720 0x5640b6fd5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640b6fc6890 0x5640b6fd7360 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5640b7036370 0x5640b7038760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 --2- 172.21.15.46:0/4019084109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640b6fcbe00 0x5640b6fd5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed49600700 0 client.27931 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 >> 172.21.15.46:0/4019084109 conn(0x5640b6f24210 msgr2=0x5640b6fc9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 shutdown_connections 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.027+0000 7fed7a766b80 1 -- 172.21.15.46:0/4019084109 wait complete. 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed7a766b80 20 client.27931 trim_cache size 0 max 16384 2024-05-08T23:11:58.034 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 msgr2=0x7fed440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 0x7fed440a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed60074980 tx=0x7fed4c009260 comp rx=0 tx=0).stop 2024-05-08T23:11:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 msgr2=0x7fed600732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9c0 tx=0x7fed5000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 shutdown_connections 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60073d20 0x7fed601a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fed440a1290 0x7fed440a3750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed601010a0 0x7fed600732a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 --2- 172.21.15.46:0/3799887594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed601061c0 0x7fed600737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.028+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 >> 172.21.15.46:0/3799887594 conn(0x7fed600fc920 msgr2=0x7fed600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.029+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 shutdown_connections 2024-05-08T23:11:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.029+0000 7fed66c00700 1 -- 172.21.15.46:0/3799887594 wait complete. 2024-05-08T23:11:58.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:11:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:57 smithi086 ceph-mon[78610]: pgmap v249: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:11:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1734640996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:57 smithi053 ceph-mon[77587]: pgmap v249: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:11:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1734640996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:11:58.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.277+0000 7ff149600700 1 -- 172.21.15.46:0/1703050021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 msgr2=0x7ff14410ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.277+0000 7ff149600700 1 --2- 172.21.15.46:0/1703050021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff14410ff20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009f90 tx=0x7ff13c00b330 comp rx=0 tx=0).stop 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 -- 172.21.15.46:0/1703050021 shutdown_connections 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 --2- 172.21.15.46:0/1703050021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff14410ff20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 --2- 172.21.15.46:0/1703050021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 0x7ff144103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 --2- 172.21.15.46:0/1703050021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff144103c30 0x7ff144107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 -- 172.21.15.46:0/1703050021 >> 172.21.15.46:0/1703050021 conn(0x7ff1440fea80 msgr2=0x7ff144100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 -- 172.21.15.46:0/1703050021 shutdown_connections 2024-05-08T23:11:58.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.278+0000 7ff149600700 1 -- 172.21.15.46:0/1703050021 wait complete. 2024-05-08T23:11:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 Processor -- start 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 -- start start 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 0x7ff144199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14419dcf0 0x7ff14419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419e770 con 0x7ff144103280 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419e8e0 con 0x7ff1441083a0 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.279+0000 7ff149600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14419eaa0 con 0x7ff14419dcf0 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42402/0 (socket says 172.21.15.46:42402) 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 -- 172.21.15.46:0/2788144971 learned_addr learned my addr 172.21.15.46:0/2788144971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142a00700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 0x7ff144199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff143400700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14419dcf0 0x7ff14419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 -- 172.21.15.46:0/2788144971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14419dcf0 msgr2=0x7ff14419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14419dcf0 0x7ff14419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 -- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 msgr2=0x7ff144199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 0x7ff144199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 -- 172.21.15.46:0/2788144971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340095f0 con 0x7ff1441083a0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.280+0000 7ff142000700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff13000e9f0 tx=0x7ff13000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.281+0000 7ff13b400700 1 -- 172.21.15.46:0/2788144971 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130009750 con 0x7ff1441083a0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.281+0000 7ff13b400700 1 -- 172.21.15.46:0/2788144971 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff130004cf0 con 0x7ff1441083a0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.281+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c009c40 con 0x7ff1441083a0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.281+0000 7ff13b400700 1 -- 172.21.15.46:0/2788144971 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13000eee0 con 0x7ff1441083a0 2024-05-08T23:11:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.281+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1441a62f0 con 0x7ff1441083a0 2024-05-08T23:11:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.285+0000 7ff13b400700 1 -- 172.21.15.46:0/2788144971 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff130004750 con 0x7ff1441083a0 2024-05-08T23:11:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 Processor -- start 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 -- start start 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff13b400700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 0x7ff1280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff13b400700 1 -- 172.21.15.46:0/2788144971 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff130014070 con 0x7ff1441083a0 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f4128150 con 0x55b7f41d7720 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f4125b10 con 0x55b7f41d1890 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f4121400 con 0x55b7f41d6e00 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.286+0000 7ff143400700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 0x7ff1280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48280/0 (socket says 172.21.15.46:48280) 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 -- 172.21.15.46:0/3159968139 learned_addr learned my addr 172.21.15.46:0/3159968139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142000700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff143400700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 -- 172.21.15.46:0/3159968139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 msgr2=0x55b7f41d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 -- 172.21.15.46:0/3159968139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 msgr2=0x55b7f41d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 -- 172.21.15.46:0/3159968139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f3fa23b0 con 0x55b7f41d6e00 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff143400700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142000700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.287+0000 7ff142a00700 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff134009fc0 tx=0x7ff134011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5847c21c406940 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff143400700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 0x7ff1280a37d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009550 tx=0x7ff13c007ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff12f400700 1 -- 172.21.15.46:0/3159968139 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400e670 con 0x55b7f41d6e00 2024-05-08T23:11:58.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff12f400700 1 -- 172.21.15.46:0/3159968139 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff13400ec90 con 0x55b7f41d6e00 2024-05-08T23:11:58.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff12f400700 1 -- 172.21.15.46:0/3159968139 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134019d30 con 0x55b7f41d6e00 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff15d15eb80 1 -- 172.21.15.46:0/3159968139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 msgr2=0x55b7f41d71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.288+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff134009fc0 tx=0x7ff134011720 comp rx=0 tx=0).stop 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 -- 172.21.15.46:0/3159968139 shutdown_connections 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41d71e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/3159968139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 -- 172.21.15.46:0/3159968139 >> 172.21.15.46:0/3159968139 conn(0x55b7f412f210 msgr2=0x55b7f41cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 -- 172.21.15.46:0/3159968139 shutdown_connections 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 -- 172.21.15.46:0/3159968139 wait complete. 2024-05-08T23:11:58.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 -- start start 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.289+0000 7ff15d15eb80 1 Processor -- start 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f40bd560 con 0x55b7f41d6e00 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f40bd200 con 0x55b7f41d7720 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3fa46a0 con 0x55b7f41d1890 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff15d15eb80 10 client.0 _ll_register_callbacks cb 0x55b7f40c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48292/0 (socket says 172.21.15.46:48292) 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff142a00700 1 -- 172.21.15.46:0/1134555515 learned_addr learned my addr 172.21.15.46:0/1134555515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff142000700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.290+0000 7ff143400700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 -- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 msgr2=0x55b7f41db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 -- 172.21.15.46:0/1134555515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 msgr2=0x55b7f41daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 -- 172.21.15.46:0/1134555515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff134009c20 con 0x55b7f41d1890 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff142a00700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff134006010 tx=0x7ff13401abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe28acb4482f5be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff123e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400e680 con 0x55b7f41d1890 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7f407a270 con 0x55b7f41d1890 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff13400e7e0 con 0x55b7f41d1890 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.291+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff13400e9a0 con 0x55b7f41d1890 2024-05-08T23:11:58.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 10 client.27943 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b7f3f4eb60 con 0x55b7f41d1890 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7f407a270 con 0x55b7f41d1890 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff134004870 con 0x55b7f41d1890 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 10 client.27943 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 10 client.27943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 20 client.27943 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f41eafa0 0x55b7f41ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 4 client.27943 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7f41a0810 con 0x55b7f41eafa0 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.292+0000 7ff142000700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f41eafa0 0x55b7f41ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.293+0000 7ff142000700 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f41eafa0 0x55b7f41ed3c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938b47756bd62b5a server_cookie=2588ee526b89c526 in_seq=0 out_seq=0 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.293+0000 7ff123e00700 10 client.27943 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.293+0000 7ff123e00700 1 -- 172.21.15.46:0/1134555515 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7f41a0810 con 0x55b7f41eafa0 2024-05-08T23:11:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.293+0000 7ff123e00700 10 client.27943 handle_command_reply: tid=0 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 client.27943 shutdown 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 msgr2=0x55b7f41daa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff134006010 tx=0x7ff13401abf0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 shutdown_connections 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41daa50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f41eafa0 0x55b7f41ed3c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/1134555515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff123e00700 0 client.27943 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 >> 172.21.15.46:0/1134555515 conn(0x55b7f412f210 msgr2=0x55b7f41cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 shutdown_connections 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 1 -- 172.21.15.46:0/1134555515 wait complete. 2024-05-08T23:11:58.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.294+0000 7ff15d15eb80 20 client.27943 trim_cache size 0 max 16384 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.310+0000 7ff15d15eb80 1 Processor -- start 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 -- start start 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f41dd5c0 con 0x55b7f41d6e00 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f41d7560 con 0x55b7f41d7720 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f41ddb10 con 0x55b7f41d1890 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48294/0 (socket says 172.21.15.46:48294) 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 -- 172.21.15.46:0/4039958650 learned_addr learned my addr 172.21.15.46:0/4039958650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff143400700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142000700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 -- 172.21.15.46:0/4039958650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 msgr2=0x55b7f41e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 -- 172.21.15.46:0/4039958650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 msgr2=0x55b7f41cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142a00700 1 -- 172.21.15.46:0/4039958650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f40e7560 con 0x55b7f41d1890 2024-05-08T23:11:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.311+0000 7ff142000700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:11:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.312+0000 7ff142a00700 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff1340058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76174bae8e77ed51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.312+0000 7ff12fe00700 1 -- 172.21.15.46:0/4039958650 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400e680 con 0x55b7f41d1890 2024-05-08T23:11:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.312+0000 7ff12fe00700 1 -- 172.21.15.46:0/4039958650 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff134019cb0 con 0x55b7f41d1890 2024-05-08T23:11:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.312+0000 7ff12fe00700 1 -- 172.21.15.46:0/4039958650 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13401a480 con 0x55b7f41d1890 2024-05-08T23:11:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- 172.21.15.46:0/4039958650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 msgr2=0x55b7f41d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff1340058e0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- 172.21.15.46:0/4039958650 shutdown_connections 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d7720 0x55b7f41e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d1890 0x55b7f41d1350 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/4039958650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d6e00 0x55b7f41cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- 172.21.15.46:0/4039958650 >> 172.21.15.46:0/4039958650 conn(0x55b7f412f210 msgr2=0x55b7f41d9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- 172.21.15.46:0/4039958650 shutdown_connections 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- 172.21.15.46:0/4039958650 wait complete. 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.313+0000 7ff15d15eb80 1 -- start start 2024-05-08T23:11:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 Processor -- start 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f4240630 con 0x55b7f41d7720 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f42407a0 con 0x55b7f41d1890 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f4240960 con 0x55b7f41d6e00 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.314+0000 7ff15d15eb80 10 client.0 _ll_register_callbacks cb 0x55b7f40c6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff143400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48310/0 (socket says 172.21.15.46:48310) 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 -- 172.21.15.46:0/736627475 learned_addr learned my addr 172.21.15.46:0/736627475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142a00700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 -- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 msgr2=0x55b7f41de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 -- 172.21.15.46:0/736627475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 msgr2=0x55b7f41debc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41debc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 -- 172.21.15.46:0/736627475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7f3f4eb60 con 0x55b7f41d6e00 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff142000700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff130096490 tx=0x7ff13000cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d13edc5fa9bb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff123e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.315+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130099f90 con 0x55b7f41d6e00 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff13009ddc0 con 0x55b7f41d6e00 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff13400ec90 con 0x55b7f41d6e00 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1300a2410 con 0x55b7f41d6e00 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 10 client.27949 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b7f3fa6cd0 con 0x55b7f41d6e00 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff13400ec90 con 0x55b7f41d6e00 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1300966a0 con 0x55b7f41d6e00 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 10 client.27949 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 10 client.27949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 20 client.27949 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f4242070 0x55b7f4244460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 4 client.27949 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7f4119ec0 con 0x55b7f4242070 2024-05-08T23:11:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.316+0000 7ff142a00700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f4242070 0x55b7f4244460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff142a00700 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f4242070 0x55b7f4244460 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a947c218a9681c server_cookie=5b3f2df7b4ca95e2 in_seq=0 out_seq=0 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff123e00700 10 client.27949 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff123e00700 1 -- 172.21.15.46:0/736627475 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7f4119ec0 con 0x55b7f4242070 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff123e00700 10 client.27949 handle_command_reply: tid=0 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff15d15eb80 1 client.27949 shutdown 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 msgr2=0x55b7f41de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.317+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff130096490 tx=0x7ff13000cc50 comp rx=0 tx=0).stop 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 shutdown_connections 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b7f4242070 0x55b7f4244460 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff123e00700 0 client.27949 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f41d7720 0x55b7f41debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7f41d1890 0x55b7f41de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 --2- 172.21.15.46:0/736627475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7f41d6e00 0x55b7f41de680 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 >> 172.21.15.46:0/736627475 conn(0x55b7f412f210 msgr2=0x55b7f41dc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 shutdown_connections 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 1 -- 172.21.15.46:0/736627475 wait complete. 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.318+0000 7ff15d15eb80 20 client.27949 trim_cache size 0 max 16384 2024-05-08T23:11:58.324 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 msgr2=0x7ff1280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 0x7ff1280a37d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009550 tx=0x7ff13c007ab0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 msgr2=0x7ff1441999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff13000e9f0 tx=0x7ff13000ed00 comp rx=0 tx=0).stop 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 shutdown_connections 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14419dcf0 0x7ff14419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1280a1310 0x7ff1280a37d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff144103280 0x7ff144199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 --2- 172.21.15.46:0/2788144971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1441083a0 0x7ff1441999a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 >> 172.21.15.46:0/2788144971 conn(0x7ff1440fea80 msgr2=0x7ff144100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 shutdown_connections 2024-05-08T23:11:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:58.319+0000 7ff149600700 1 -- 172.21.15.46:0/2788144971 wait complete. 2024-05-08T23:11:58.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 691384 10.01 MB/sec execute 260 sec latency 17.932 ms 2024-05-08T23:11:59.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:11:59.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:11:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 -- 172.21.15.46:0/4095844065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 msgr2=0x7efe90102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 --2- 172.21.15.46:0/4095844065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90102380 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efe80009990 tx=0x7efe80009ca0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 -- 172.21.15.46:0/4095844065 shutdown_connections 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 --2- 172.21.15.46:0/4095844065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe9010b3c0 0x7efe9010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 --2- 172.21.15.46:0/4095844065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90102380 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 --2- 172.21.15.46:0/4095844065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe901028c0 0x7efe9010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 -- 172.21.15.46:0/4095844065 >> 172.21.15.46:0/4095844065 conn(0x7efe900fb760 msgr2=0x7efe900fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 -- 172.21.15.46:0/4095844065 shutdown_connections 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.568+0000 7efe95e00700 1 -- 172.21.15.46:0/4095844065 wait complete. 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 Processor -- start 2024-05-08T23:11:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 -- start start 2024-05-08T23:11:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 0x7efe90199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 0x7efe9019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9019e420 con 0x7efe901028c0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9019e590 con 0x7efe9010b3c0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.569+0000 7efe95e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9019e750 con 0x7efe90101fa0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 0x7efe90199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48328/0 (socket says 172.21.15.46:48328) 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8ea00700 1 -- 172.21.15.46:0/2374477750 learned_addr learned my addr 172.21.15.46:0/2374477750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8f400700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 0x7efe9019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 -- 172.21.15.46:0/2374477750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 msgr2=0x7efe90199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 -- 172.21.15.46:0/2374477750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 msgr2=0x7efe9019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 0x7efe9019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 -- 172.21.15.46:0/2374477750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800095f0 con 0x7efe901028c0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8f400700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 0x7efe9019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe8e000700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 0x7efe90199a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00e9f0 tx=0x7efe7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.570+0000 7efe94a00700 1 -- 172.21.15.46:0/2374477750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c004220 con 0x7efe901028c0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.571+0000 7efe94a00700 1 -- 172.21.15.46:0/2374477750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe7c0095c0 con 0x7efe901028c0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.571+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9019ea30 con 0x7efe901028c0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.571+0000 7efe94a00700 1 -- 172.21.15.46:0/2374477750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00cdc0 con 0x7efe901028c0 2024-05-08T23:11:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.571+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe901a6270 con 0x7efe901028c0 2024-05-08T23:11:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.574+0000 7efe94a00700 1 -- 172.21.15.46:0/2374477750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efe7c005230 con 0x7efe901028c0 2024-05-08T23:11:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.575+0000 7efe94a00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 0x7efe740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.575+0000 7efea99c8b80 1 Processor -- start 2024-05-08T23:11:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.575+0000 7efe94a00700 1 -- 172.21.15.46:0/2374477750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe7c014070 con 0x7efe901028c0 2024-05-08T23:11:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 0x7efe740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 -- start start 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bf23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332b491d0 con 0x564332bf87a0 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332b46b90 con 0x564332bf2910 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efea99c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332b42480 con 0x564332bf7e80 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48340/0 (socket says 172.21.15.46:48340) 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8ea00700 1 -- 172.21.15.46:0/2882304289 learned_addr learned my addr 172.21.15.46:0/2882304289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8e000700 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bf23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8f400700 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.576+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 0x7efe740a37d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe80006010 tx=0x7efe8000b280 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 -- 172.21.15.46:0/2882304289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 msgr2=0x564332bf2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 -- 172.21.15.46:0/2882304289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 msgr2=0x564332bf23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bf23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 -- 172.21.15.46:0/2882304289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643329c3430 con 0x564332bf7e80 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe8ea00700 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe800051c0 tx=0x7efe80020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5955de0fc375c2dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe7be00700 1 -- 172.21.15.46:0/2882304289 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe800048c0 con 0x564332bf7e80 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe7be00700 1 -- 172.21.15.46:0/2882304289 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8001e040 con 0x564332bf7e80 2024-05-08T23:11:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.577+0000 7efe7be00700 1 -- 172.21.15.46:0/2882304289 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8000fdf0 con 0x564332bf7e80 2024-05-08T23:11:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 -- 172.21.15.46:0/2882304289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 msgr2=0x564332bf8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe800051c0 tx=0x7efe80020000 comp rx=0 tx=0).stop 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 -- 172.21.15.46:0/2882304289 shutdown_connections 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bf8260 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 --2- 172.21.15.46:0/2882304289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bf23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 -- 172.21.15.46:0/2882304289 >> 172.21.15.46:0/2882304289 conn(0x564332b50290 msgr2=0x564332bece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 -- 172.21.15.46:0/2882304289 shutdown_connections 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.578+0000 7efea99c8b80 1 -- 172.21.15.46:0/2882304289 wait complete. 2024-05-08T23:11:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.579+0000 7efea99c8b80 1 -- start start 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.579+0000 7efea99c8b80 1 Processor -- start 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bfc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bfc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332ade5e0 con 0x564332bf87a0 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332ade280 con 0x564332bf2910 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643329c5720 con 0x564332bf7e80 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efea99c8b80 10 client.0 _ll_register_callbacks cb 0x564332ae7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bfc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42480/0 (socket says 172.21.15.46:42480) 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 -- 172.21.15.46:0/3655888934 learned_addr learned my addr 172.21.15.46:0/3655888934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8f400700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bfc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 -- 172.21.15.46:0/3655888934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 msgr2=0x564332bfc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bfc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 -- 172.21.15.46:0/3655888934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 msgr2=0x564332bfc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bfc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.580+0000 7efe8ea00700 1 -- 172.21.15.46:0/3655888934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9010e540 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe8e000700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bfc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe8ea00700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe80009ce0 tx=0x7efe8000fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c083a89705a132 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe78c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80004ad0 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe80004c30 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564332a9b2f0 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe80004df0 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efea99c8b80 10 client.27887 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.581+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56433296fbe0 con 0x564332bf2910 2024-05-08T23:11:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564332a9b2f0 con 0x564332bf2910 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe800394d0 con 0x564332bf2910 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 10 client.27887 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 20 client.27887 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c0bd10 0x564332c0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 4 client.27887 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.582+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564332bc1890 con 0x564332c0bd10 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.583+0000 7efe8e000700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c0bd10 0x564332c0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.583+0000 7efe8e000700 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c0bd10 0x564332c0e100 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabf417d59cffa1e server_cookie=6e1a42ee43d8dbfa in_seq=0 out_seq=0 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.583+0000 7efe78c00700 10 client.27887 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efe78c00700 1 -- 172.21.15.46:0/3655888934 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564332bc1890 con 0x564332c0bd10 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efe78c00700 10 client.27887 handle_command_reply: tid=0 2024-05-08T23:11:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 client.27887 shutdown 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 msgr2=0x564332bfbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe80009ce0 tx=0x7efe8000fd40 comp rx=0 tx=0).stop 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 shutdown_connections 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c0bd10 0x564332c0e100 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf87a0 0x564332bfc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bfbad0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 --2- 172.21.15.46:0/3655888934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf7e80 0x564332bfc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 >> 172.21.15.46:0/3655888934 conn(0x564332b50290 msgr2=0x564332bec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efe78c00700 0 client.27887 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 shutdown_connections 2024-05-08T23:11:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.584+0000 7efea99c8b80 1 -- 172.21.15.46:0/3655888934 wait complete. 2024-05-08T23:11:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.585+0000 7efea99c8b80 20 client.27887 trim_cache size 0 max 16384 2024-05-08T23:11:59.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 Processor -- start 2024-05-08T23:11:59.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 -- start start 2024-05-08T23:11:59.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332becda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332bfe640 con 0x564332bf7e80 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332be9e40 con 0x564332bf2910 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efea99c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332bee6c0 con 0x564332bf87a0 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48362/0 (socket says 172.21.15.46:48362) 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42486/0 (socket says 172.21.15.46:42486) 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8f400700 1 -- 172.21.15.46:0/704961650 learned_addr learned my addr 172.21.15.46:0/704961650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.604+0000 7efe8e000700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332becda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 -- 172.21.15.46:0/704961650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 msgr2=0x564332bf23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 -- 172.21.15.46:0/704961650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 msgr2=0x564332becda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332becda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 -- 172.21.15.46:0/704961650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564332bf2fc0 con 0x564332bf87a0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8ea00700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8e000700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332becda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe8f400700 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe88007630 tx=0x7efe8800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ed84b443937e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe85200700 1 -- 172.21.15.46:0/704961650 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800e040 con 0x564332bf87a0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe85200700 1 -- 172.21.15.46:0/704961650 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8800f040 con 0x564332bf87a0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.605+0000 7efe85200700 1 -- 172.21.15.46:0/704961650 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88016b90 con 0x564332bf87a0 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 -- 172.21.15.46:0/704961650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 msgr2=0x564332bf72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe88007630 tx=0x7efe8800b040 comp rx=0 tx=0).stop 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 -- 172.21.15.46:0/704961650 shutdown_connections 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332bf72d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332bf23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 --2- 172.21.15.46:0/704961650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332becda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 -- 172.21.15.46:0/704961650 >> 172.21.15.46:0/704961650 conn(0x564332b50290 msgr2=0x564332bf6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 -- 172.21.15.46:0/704961650 shutdown_connections 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.606+0000 7efea99c8b80 1 -- 172.21.15.46:0/704961650 wait complete. 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 -- start start 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 Processor -- start 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332c02500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332c02a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.607+0000 7efea99c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332c5c660 con 0x564332bf7e80 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efea99c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332c5c7a0 con 0x564332bf2910 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efea99c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564332c5c960 con 0x564332bf87a0 2024-05-08T23:11:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efea99c8b80 10 client.0 _ll_register_callbacks cb 0x564332ae7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332c02a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48366/0 (socket says 172.21.15.46:48366) 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332c02500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 -- 172.21.15.46:0/225309126 learned_addr learned my addr 172.21.15.46:0/225309126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 -- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 msgr2=0x564332c02500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332c02500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 -- 172.21.15.46:0/225309126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 msgr2=0x564332c02a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332c02a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8f400700 1 -- 172.21.15.46:0/225309126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56433296fbe0 con 0x564332bf87a0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.608+0000 7efe8e000700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332c02a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe8f400700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe88000f80 tx=0x7efe88009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe15571b5dc21736 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe78c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800e040 con 0x564332bf87a0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe8001e040 con 0x564332bf87a0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe880164c0 con 0x564332bf87a0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe88017620 con 0x564332bf87a0 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efea99c8b80 10 client.27967 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efe901a6270 con 0x564332bf87a0 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.609+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8001e040 con 0x564332bf87a0 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe88020720 con 0x564332bf87a0 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 10 client.27967 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 10 client.27967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 20 client.27967 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c64180 0x564332c665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 4 client.27967 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564332b3af40 con 0x564332c64180 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.610+0000 7efe8ea00700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c64180 0x564332c665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.611+0000 7efe78c00700 10 client.27967 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.611+0000 7efe8ea00700 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c64180 0x564332c665a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ba3af296006ca4 server_cookie=965984464492b5a5 in_seq=0 out_seq=0 2024-05-08T23:11:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efe78c00700 1 -- 172.21.15.46:0/225309126 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564332b3af40 con 0x564332c64180 2024-05-08T23:11:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efe78c00700 10 client.27967 handle_command_reply: tid=0 2024-05-08T23:11:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 client.27967 shutdown 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 msgr2=0x564332c5c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe88000f80 tx=0x7efe88009240 comp rx=0 tx=0).stop 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 shutdown_connections 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564332bf87a0 0x564332c5c120 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564332bf2910 0x564332c02500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564332c64180 0x564332c665a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 --2- 172.21.15.46:0/225309126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564332bf7e80 0x564332c02a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 >> 172.21.15.46:0/225309126 conn(0x564332b50290 msgr2=0x564332bf7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efe78c00700 0 client.27967 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 shutdown_connections 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.612+0000 7efea99c8b80 1 -- 172.21.15.46:0/225309126 wait complete. 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efea99c8b80 20 client.27967 trim_cache size 0 max 16384 2024-05-08T23:11:59.617 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:11:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4af4fc1c-bd2c-4bf0-9b5a-4f1af0c8baf3", 2024-05-08T23:11:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:11:59.618 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 msgr2=0x7efe740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 0x7efe740a37d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe80006010 tx=0x7efe8000b280 comp rx=0 tx=0).stop 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 msgr2=0x7efe90199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 0x7efe90199a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00e9f0 tx=0x7efe7c00ed00 comp rx=0 tx=0).stop 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 shutdown_connections 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe9010b3c0 0x7efe9019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90101fa0 0x7efe90199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe740a1310 0x7efe740a37d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 --2- 172.21.15.46:0/2374477750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe901028c0 0x7efe90199a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.613+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 >> 172.21.15.46:0/2374477750 conn(0x7efe900fb760 msgr2=0x7efe900fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.614+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 shutdown_connections 2024-05-08T23:11:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.614+0000 7efe95e00700 1 -- 172.21.15.46:0/2374477750 wait complete. 2024-05-08T23:11:59.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 -- 172.21.15.46:0/1990289962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c10b410 msgr2=0x7fdc2c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 --2- 172.21.15.46:0/1990289962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c10b410 0x7fdc2c10d880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc2400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 -- 172.21.15.46:0/1990289962 shutdown_connections 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 --2- 172.21.15.46:0/1990289962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c10b410 0x7fdc2c10d880 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 --2- 172.21.15.46:0/1990289962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102040 0x7fdc2c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 --2- 172.21.15.46:0/1990289962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c102960 0x7fdc2c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 -- 172.21.15.46:0/1990289962 >> 172.21.15.46:0/1990289962 conn(0x7fdc2c0fb760 msgr2=0x7fdc2c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 -- 172.21.15.46:0/1990289962 shutdown_connections 2024-05-08T23:11:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.877+0000 7fdc32200700 1 -- 172.21.15.46:0/1990289962 wait complete. 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.878+0000 7fdc32200700 1 Processor -- start 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 -- start start 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102960 0x7fdc2c1033c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 0x7fdc2c103900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a9c40 con 0x7fdc2c102960 2024-05-08T23:11:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a9d80 con 0x7fdc2c10b410 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc32200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a9ef0 con 0x7fdc2c102040 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48378/0 (socket says 172.21.15.46:48378) 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc2b400700 1 -- 172.21.15.46:0/1475089121 learned_addr learned my addr 172.21.15.46:0/1475089121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102960 0x7fdc2c1033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.879+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 0x7fdc2c103900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 -- 172.21.15.46:0/1475089121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 msgr2=0x7fdc2c103900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 0x7fdc2c103900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 -- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102960 msgr2=0x7fdc2c1033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102960 0x7fdc2c1033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 -- 172.21.15.46:0/1475089121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009d00 con 0x7fdc2c102040 2024-05-08T23:11:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 0x7fdc2c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00e9a0 tx=0x7fdc1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc28c00700 1 -- 172.21.15.46:0/1475089121 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c009790 con 0x7fdc2c102040 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc28c00700 1 -- 172.21.15.46:0/1475089121 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1c00cbd0 con 0x7fdc2c102040 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c1aa1d0 con 0x7fdc2c102040 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc28c00700 1 -- 172.21.15.46:0/1475089121 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c018680 con 0x7fdc2c102040 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.880+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c1aa6e0 con 0x7fdc2c102040 2024-05-08T23:11:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.883+0000 7fdc28c00700 1 -- 172.21.15.46:0/1475089121 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc1c00cd40 con 0x7fdc2c102040 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc28c00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 0x7fdc100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 Processor -- start 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc28c00700 1 -- 172.21.15.46:0/1475089121 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdc1c014070 con 0x7fdc2c102040 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 -- start start 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 0x7fdc100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b51720 0x560295b4b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b4bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295aa2150 con 0x560295b51720 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295a9fb10 con 0x560295b4b890 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295a9b400 con 0x560295b50e00 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.885+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48394/0 (socket says 172.21.15.46:48394) 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 -- 172.21.15.46:0/1597699150 learned_addr learned my addr 172.21.15.46:0/1597699150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b51720 0x560295b4b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b4bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 -- 172.21.15.46:0/1597699150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 msgr2=0x560295b4bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b4bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 -- 172.21.15.46:0/1597699150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b51720 msgr2=0x560295b4b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b51720 0x560295b4b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 -- 172.21.15.46:0/1597699150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56029591c3b0 con 0x560295b50e00 2024-05-08T23:11:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b4bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:11:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c096ae0 tx=0x7fdc1c099a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab9bfe7e6e7ed60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 0x7fdc100a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c1045b0 tx=0x7fdc14013840 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc1be00700 1 -- 172.21.15.46:0/1597699150 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c09dea0 con 0x560295b50e00 2024-05-08T23:11:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc1be00700 1 -- 172.21.15.46:0/1597699150 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1c0a24f0 con 0x560295b50e00 2024-05-08T23:11:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.886+0000 7fdc1be00700 1 -- 172.21.15.46:0/1597699150 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c099560 con 0x560295b50e00 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1597699150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 msgr2=0x560295b511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c096ae0 tx=0x7fdc1c099a50 comp rx=0 tx=0).stop 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1597699150 shutdown_connections 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b4bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b511e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1597699150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b51720 0x560295b4b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1597699150 >> 172.21.15.46:0/1597699150 conn(0x560295aa9210 msgr2=0x560295b45df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1597699150 shutdown_connections 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.887+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1597699150 wait complete. 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.888+0000 7fdc45cadb80 1 -- start start 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.888+0000 7fdc45cadb80 1 Processor -- start 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b54ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b55530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295a37560 con 0x560295b50e00 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295a37200 con 0x560295b4b890 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029591e6a0 con 0x560295b51720 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc45cadb80 10 client.0 _ll_register_callbacks cb 0x560295a40b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc23e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b54ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b55530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42522/0 (socket says 172.21.15.46:42522) 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 -- 172.21.15.46:0/1492119269 learned_addr learned my addr 172.21.15.46:0/1492119269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 -- 172.21.15.46:0/1492119269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 msgr2=0x560295b55530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b55530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 -- 172.21.15.46:0/1492119269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 msgr2=0x560295b54ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b54ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2b400700 1 -- 172.21.15.46:0/1492119269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c1aa6e0 con 0x560295b4b890 2024-05-08T23:11:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.889+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b55530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0a1fc0 tx=0x7fdc1c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=376aedfbbcb304f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc18c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0b8070 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602959f4270 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdc1c09dea0 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1c09a4e0 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc45cadb80 10 client.27905 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.890+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5602958c8b60 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602959f4270 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc1c098fd0 con 0x560295b4b890 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 10 client.27905 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 10 client.27905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 20 client.27905 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295b64d90 0x560295b67180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 4 client.27905 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:11:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560295b1a810 con 0x560295b64d90 2024-05-08T23:11:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.891+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295b64d90 0x560295b67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.892+0000 7fdc23e00700 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295b64d90 0x560295b67180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3485088941d38d6b server_cookie=9943bceff745d621 in_seq=0 out_seq=0 2024-05-08T23:11:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.892+0000 7fdc18c00700 10 client.27905 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc18c00700 1 -- 172.21.15.46:0/1492119269 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560295b1a810 con 0x560295b64d90 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc18c00700 10 client.27905 handle_command_reply: tid=0 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 client.27905 shutdown 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 msgr2=0x560295b54ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0a1fc0 tx=0x7fdc1c00eef0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 shutdown_connections 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b55530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b54ab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295b64d90 0x560295b67180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1492119269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b54ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 >> 172.21.15.46:0/1492119269 conn(0x560295aa9210 msgr2=0x560295b455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc18c00700 0 client.27905 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 shutdown_connections 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.893+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1492119269 wait complete. 2024-05-08T23:11:59.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.894+0000 7fdc45cadb80 20 client.27905 trim_cache size 0 max 16384 2024-05-08T23:11:59.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 693023 10.02 MB/sec execute 261 sec latency 11.966 ms 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 Processor -- start 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 -- start start 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b45d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b4eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602959f42a0 con 0x560295b50e00 2024-05-08T23:11:59.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295b57600 con 0x560295b4b890 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.913+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295b42e10 con 0x560295b51720 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48416/0 (socket says 172.21.15.46:48416) 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 -- 172.21.15.46:0/3162235218 learned_addr learned my addr 172.21.15.46:0/3162235218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc23e00700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b4eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2b400700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b45d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 -- 172.21.15.46:0/3162235218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 msgr2=0x560295b45d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b45d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 -- 172.21.15.46:0/3162235218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 msgr2=0x560295b4eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b4eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc2be00700 1 -- 172.21.15.46:0/3162235218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560295a61560 con 0x560295b51720 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.914+0000 7fdc23e00700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b4eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.915+0000 7fdc2be00700 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc24007b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1974a9cd0e9d5805 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.915+0000 7fdc21200700 1 -- 172.21.15.46:0/3162235218 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400e040 con 0x560295b51720 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.915+0000 7fdc21200700 1 -- 172.21.15.46:0/3162235218 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc24021070 con 0x560295b51720 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.915+0000 7fdc21200700 1 -- 172.21.15.46:0/3162235218 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24005040 con 0x560295b51720 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- 172.21.15.46:0/3162235218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 msgr2=0x560295b4f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc24007b90 comp rx=0 tx=0).stop 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- 172.21.15.46:0/3162235218 shutdown_connections 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b51720 0x560295b4f1e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b4b890 0x560295b45d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/3162235218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b50e00 0x560295b4eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- 172.21.15.46:0/3162235218 >> 172.21.15.46:0/3162235218 conn(0x560295aa9210 msgr2=0x560295b6bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- 172.21.15.46:0/3162235218 shutdown_connections 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- 172.21.15.46:0/3162235218 wait complete. 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 -- start start 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.916+0000 7fdc45cadb80 1 Processor -- start 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 0x560295b59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 0x560295bba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295b5ad30 con 0x560295b4b890 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295bba7a0 con 0x560295b51720 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295bba8e0 con 0x560295b50e00 2024-05-08T23:11:59.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc45cadb80 10 client.0 _ll_register_callbacks cb 0x560295a40b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:11:59.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc2be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 0x560295bba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc2b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 0x560295b59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b5a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48430/0 (socket says 172.21.15.46:48430) 2024-05-08T23:11:59.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.917+0000 7fdc23e00700 1 -- 172.21.15.46:0/1569702846 learned_addr learned my addr 172.21.15.46:0/1569702846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 -- 172.21.15.46:0/1569702846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 msgr2=0x560295b5a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b5a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 -- 172.21.15.46:0/1569702846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 msgr2=0x560295b59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 0x560295b59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 -- 172.21.15.46:0/1569702846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc140099f0 con 0x560295b51720 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 0x560295b59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc2be00700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 0x560295bba260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b070 tx=0x7fdc2400bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad4222cced04bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc18c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400e040 con 0x560295b51720 2024-05-08T23:11:59.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc1c0a24f0 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.918+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdc24023070 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc24007e40 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 10 client.27911 fetch_fsmap learned FSMap version 30 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fdc2c10e540 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1c0a24f0 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc2400b0d0 con 0x560295b51720 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 10 client.27911 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 10 client.27911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 20 client.27911 populate_metadata read hostname 'smithi046' 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295bbbd90 0x560295bbe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 4 client.27911 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.919+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560295a93ec0 con 0x560295bbbd90 2024-05-08T23:11:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295bbbd90 0x560295bbe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc2b400700 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295bbbd90 0x560295bbe1b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332536b453c0e570 server_cookie=833dffe9a9e2c829 in_seq=0 out_seq=0 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc18c00700 10 client.27911 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc18c00700 1 -- 172.21.15.46:0/1569702846 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560295a93ec0 con 0x560295bbbd90 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc18c00700 10 client.27911 handle_command_reply: tid=0 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.920+0000 7fdc45cadb80 1 client.27911 shutdown 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 msgr2=0x560295bba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 0x560295bba260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b070 tx=0x7fdc2400bd40 comp rx=0 tx=0).stop 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 shutdown_connections 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560295b51720 0x560295bba260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560295bbbd90 0x560295bbe1b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560295b4b890 0x560295b59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc18c00700 0 client.27911 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 --2- 172.21.15.46:0/1569702846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560295b50e00 0x560295b5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 >> 172.21.15.46:0/1569702846 conn(0x560295aa9210 msgr2=0x560295b4eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 shutdown_connections 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 1 -- 172.21.15.46:0/1569702846 wait complete. 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.921+0000 7fdc45cadb80 20 client.27911 trim_cache size 0 max 16384 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "4af4fc1c-bd2c-4bf0-9b5a-4f1af0c8baf3": { 2024-05-08T23:11:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4af4fc1c-bd2c-4bf0-9b5a-4f1af0c8baf3", 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:11:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 msgr2=0x7fdc100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 0x7fdc100a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c1045b0 tx=0x7fdc14013840 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 msgr2=0x7fdc2c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00e9a0 tx=0x7fdc1c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 shutdown_connections 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c10b410 0x7fdc2c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2c102040 0x7fdc2c102e80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdc100a1290 0x7fdc100a3750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 --2- 172.21.15.46:0/1475089121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2c102960 0x7fdc2c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 >> 172.21.15.46:0/1475089121 conn(0x7fdc2c0fb760 msgr2=0x7fdc2c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 shutdown_connections 2024-05-08T23:11:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:11:59.922+0000 7fdc32200700 1 -- 172.21.15.46:0/1475089121 wait complete. 2024-05-08T23:11:59.941 INFO:tasks.cephfs.filesystem:scrub status for tag:4af4fc1c-bd2c-4bf0-9b5a-4f1af0c8baf3 - {'path': '/', 'tag': '4af4fc1c-bd2c-4bf0-9b5a-4f1af0c8baf3', 'options': 'recursive,force'} 2024-05-08T23:12:00.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:11:59 smithi046 ceph-mon[36898]: pgmap v250: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:11:59 smithi086 ceph-mon[78610]: pgmap v250: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:11:59 smithi053 ceph-mon[77587]: pgmap v250: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:00.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 694755 10.00 MB/sec execute 262 sec latency 45.147 ms 2024-05-08T23:12:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:12:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:00 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:12:01.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:00 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:00 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:01.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 695254 9.98 MB/sec execute 263 sec latency 69.212 ms 2024-05-08T23:12:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:01 smithi086 ceph-mon[78610]: pgmap v251: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-08T23:12:02.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:01 smithi053 ceph-mon[77587]: pgmap v251: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-08T23:12:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:01 smithi046 ceph-mon[36898]: pgmap v251: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-08T23:12:02.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 696719 9.98 MB/sec execute 264 sec latency 16.320 ms 2024-05-08T23:12:03.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:02 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:02 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:02 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:03.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 698621 9.97 MB/sec execute 265 sec latency 16.879 ms 2024-05-08T23:12:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:03 smithi086 ceph-mon[78610]: pgmap v252: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:03 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:12:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:03 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:12:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:03 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:12:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:03 smithi053 ceph-mon[77587]: pgmap v252: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:04.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:03 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:12:04.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:03 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:12:04.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:03 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:12:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:03 smithi046 ceph-mon[36898]: pgmap v252: 129 pgs: 129 active+clean; 454 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-05-08T23:12:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:03 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:12:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:03 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:12:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:03 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:12:04.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 700266 9.97 MB/sec execute 266 sec latency 14.800 ms 2024-05-08T23:12:05.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 702476 9.98 MB/sec execute 267 sec latency 19.427 ms 2024-05-08T23:12:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:05 smithi086 ceph-mon[78610]: pgmap v253: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-05-08T23:12:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:05 smithi053 ceph-mon[77587]: pgmap v253: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-05-08T23:12:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:05 smithi046 ceph-mon[36898]: pgmap v253: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-05-08T23:12:06.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 703938 9.97 MB/sec execute 268 sec latency 14.091 ms 2024-05-08T23:12:07.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 706119 9.98 MB/sec execute 269 sec latency 15.875 ms 2024-05-08T23:12:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:07 smithi086 ceph-mon[78610]: pgmap v254: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:07 smithi053 ceph-mon[77587]: pgmap v254: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:07 smithi046 ceph-mon[36898]: pgmap v254: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:08.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 707588 9.97 MB/sec execute 270 sec latency 18.773 ms 2024-05-08T23:12:09.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 708166 9.95 MB/sec execute 271 sec latency 251.811 ms 2024-05-08T23:12:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:09 smithi086 ceph-mon[78610]: pgmap v255: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:09 smithi053 ceph-mon[77587]: pgmap v255: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:09 smithi046 ceph-mon[36898]: pgmap v255: 129 pgs: 129 active+clean; 461 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-05-08T23:12:10.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 709839 9.94 MB/sec execute 272 sec latency 27.727 ms 2024-05-08T23:12:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:10 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:10 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:10 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:10 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:10 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:11.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:10 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:10 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:10 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:10 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:11.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 711167 9.93 MB/sec execute 273 sec latency 32.369 ms 2024-05-08T23:12:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:11 smithi086 ceph-mon[78610]: pgmap v256: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-05-08T23:12:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:11 smithi086 ceph-mon[78610]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:12:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:11 smithi053 ceph-mon[77587]: pgmap v256: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-05-08T23:12:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:11 smithi053 ceph-mon[77587]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:12:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:11 smithi046 ceph-mon[36898]: pgmap v256: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-05-08T23:12:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:11 smithi046 ceph-mon[36898]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:12:12.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 713396 9.94 MB/sec execute 274 sec latency 16.699 ms 2024-05-08T23:12:13.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 714767 9.93 MB/sec execute 275 sec latency 17.353 ms 2024-05-08T23:12:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:13 smithi086 ceph-mon[78610]: pgmap v258: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-08T23:12:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:13 smithi053 ceph-mon[77587]: pgmap v258: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-08T23:12:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:13 smithi046 ceph-mon[36898]: pgmap v258: 129 pgs: 129 active+clean; 488 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-08T23:12:14.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 716318 9.92 MB/sec execute 276 sec latency 142.050 ms 2024-05-08T23:12:15.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 717907 9.92 MB/sec execute 277 sec latency 11.021 ms 2024-05-08T23:12:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:15 smithi086 ceph-mon[78610]: pgmap v259: 129 pgs: 129 active+clean; 471 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-05-08T23:12:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:15 smithi053 ceph-mon[77587]: pgmap v259: 129 pgs: 129 active+clean; 471 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-05-08T23:12:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:15 smithi046 ceph-mon[36898]: pgmap v259: 129 pgs: 129 active+clean; 471 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-05-08T23:12:16.897 INFO:tasks.workunit.client.0.smithi046.stdout: 1 720031 9.93 MB/sec execute 278 sec latency 17.902 ms 2024-05-08T23:12:17.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 721591 9.92 MB/sec execute 279 sec latency 16.106 ms 2024-05-08T23:12:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:17 smithi086 ceph-mon[78610]: pgmap v260: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:17 smithi053 ceph-mon[77587]: pgmap v260: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:17 smithi046 ceph-mon[36898]: pgmap v260: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:18.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 723787 9.93 MB/sec execute 280 sec latency 17.495 ms 2024-05-08T23:12:19.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 725312 9.92 MB/sec execute 281 sec latency 16.567 ms 2024-05-08T23:12:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:19 smithi086 ceph-mon[78610]: pgmap v261: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:19 smithi086 ceph-mon[78610]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:12:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:19 smithi053 ceph-mon[77587]: pgmap v261: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:19 smithi053 ceph-mon[77587]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:12:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:19 smithi046 ceph-mon[36898]: pgmap v261: 129 pgs: 129 active+clean; 466 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-05-08T23:12:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:19 smithi046 ceph-mon[36898]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:12:20.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 727579 9.93 MB/sec execute 282 sec latency 18.509 ms 2024-05-08T23:12:21.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 729006 9.92 MB/sec execute 283 sec latency 17.046 ms 2024-05-08T23:12:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:21 smithi086 ceph-mon[78610]: pgmap v263: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-05-08T23:12:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:21 smithi053 ceph-mon[77587]: pgmap v263: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-05-08T23:12:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:21 smithi046 ceph-mon[36898]: pgmap v263: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-05-08T23:12:22.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 731131 9.93 MB/sec execute 284 sec latency 30.448 ms 2024-05-08T23:12:23.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 732554 9.92 MB/sec execute 285 sec latency 15.647 ms 2024-05-08T23:12:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:23 smithi086 ceph-mon[78610]: pgmap v264: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-05-08T23:12:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:23 smithi053 ceph-mon[77587]: pgmap v264: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-05-08T23:12:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:23 smithi046 ceph-mon[36898]: pgmap v264: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-05-08T23:12:24.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 734807 9.93 MB/sec execute 286 sec latency 17.645 ms 2024-05-08T23:12:25.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 736295 9.92 MB/sec execute 287 sec latency 16.380 ms 2024-05-08T23:12:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:25 smithi086 ceph-mon[78610]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-05-08T23:12:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:25 smithi053 ceph-mon[77587]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-05-08T23:12:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:25 smithi046 ceph-mon[36898]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 212 op/s 2024-05-08T23:12:26.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 738588 9.94 MB/sec execute 288 sec latency 15.880 ms 2024-05-08T23:12:27.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 740013 9.92 MB/sec execute 289 sec latency 17.935 ms 2024-05-08T23:12:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:27 smithi086 ceph-mon[78610]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:27 smithi053 ceph-mon[77587]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:27 smithi046 ceph-mon[36898]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:28.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 742126 9.93 MB/sec execute 290 sec latency 18.443 ms 2024-05-08T23:12:29.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 743553 9.92 MB/sec execute 291 sec latency 17.814 ms 2024-05-08T23:12:29.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:12:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.164+0000 7f1290600700 1 -- 172.21.15.46:0/1075101307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 msgr2=0x7f1288103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.164+0000 7f1290600700 1 --2- 172.21.15.46:0/1075101307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288103660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f127c009990 tx=0x7f127c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 -- 172.21.15.46:0/1075101307 shutdown_connections 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 --2- 172.21.15.46:0/1075101307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12881083a0 0x7f128810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 --2- 172.21.15.46:0/1075101307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288103660 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 --2- 172.21.15.46:0/1075101307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1288103c30 0x7f1288107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 -- 172.21.15.46:0/1075101307 >> 172.21.15.46:0/1075101307 conn(0x7f12880fea80 msgr2=0x7f1288100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 -- 172.21.15.46:0/1075101307 shutdown_connections 2024-05-08T23:12:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.165+0000 7f1290600700 1 -- 172.21.15.46:0/1075101307 wait complete. 2024-05-08T23:12:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 Processor -- start 2024-05-08T23:12:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 -- start start 2024-05-08T23:12:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12881083a0 0x7f1288199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128819de70 0x7f128819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819e8f0 con 0x7f12881083a0 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819ea60 con 0x7f1288103280 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.166+0000 7f1290600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819ec20 con 0x7f128819de70 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59240/0 (socket says 172.21.15.46:59240) 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 -- 172.21.15.46:0/526551185 learned_addr learned my addr 172.21.15.46:0/526551185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128e600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128819de70 0x7f128819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128d200700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12881083a0 0x7f1288199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 -- 172.21.15.46:0/526551185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128819de70 msgr2=0x7f128819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128819de70 0x7f128819e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 -- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12881083a0 msgr2=0x7f1288199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12881083a0 0x7f1288199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 -- 172.21.15.46:0/526551185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280009c40 con 0x7f1288103280 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.167+0000 7f128dc00700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f127c005910 tx=0x7f127c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.168+0000 7f1286000700 1 -- 172.21.15.46:0/526551185 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c01d070 con 0x7f1288103280 2024-05-08T23:12:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.168+0000 7f1286000700 1 -- 172.21.15.46:0/526551185 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127c022470 con 0x7f1288103280 2024-05-08T23:12:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.168+0000 7f1286000700 1 -- 172.21.15.46:0/526551185 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c017650 con 0x7f1288103280 2024-05-08T23:12:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.168+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0095f0 con 0x7f1288103280 2024-05-08T23:12:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.168+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12881a63c0 con 0x7f1288103280 2024-05-08T23:12:30.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.171+0000 7f1286000700 1 -- 172.21.15.46:0/526551185 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f127c0225e0 con 0x7f1288103280 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.172+0000 7f12a41c8b80 1 Processor -- start 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.172+0000 7f12a41c8b80 1 -- start start 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f1286000700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 0x7f12700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f1286000700 1 -- 172.21.15.46:0/526551185 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f127c0ce1c0 con 0x7f1288103280 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da80698cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128e600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 0x7f12700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da805ef150 con 0x55da8069de00 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da805ecb10 con 0x55da80698890 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da805e8400 con 0x55da8069e720 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40612/0 (socket says 172.21.15.46:40612) 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128e600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da80698cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47962/0 (socket says 172.21.15.46:47962) 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.173+0000 7f128d200700 1 -- 172.21.15.46:0/311737881 learned_addr learned my addr 172.21.15.46:0/311737881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 -- 172.21.15.46:0/311737881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 msgr2=0x55da80698350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 -- 172.21.15.46:0/311737881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 msgr2=0x55da80698cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da80698cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 -- 172.21.15.46:0/311737881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da804693b0 con 0x55da8069de00 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128d200700 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128e600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 0x7f12700a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1280009fd0 tx=0x7f128000f5a0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f128dc00700 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f127c0cb4a0 tx=0x7f127c09acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667cb3396fc3ec7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f127aa00700 1 -- 172.21.15.46:0/311737881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c09a850 con 0x55da8069de00 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f127aa00700 1 -- 172.21.15.46:0/311737881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127c09b5f0 con 0x55da8069de00 2024-05-08T23:12:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.174+0000 7f127aa00700 1 -- 172.21.15.46:0/311737881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c09b150 con 0x55da8069de00 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 -- 172.21.15.46:0/311737881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 msgr2=0x55da8069e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f127c0cb4a0 tx=0x7f127c09acc0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 -- 172.21.15.46:0/311737881 shutdown_connections 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da80698cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da8069e1e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/311737881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da80698350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 -- 172.21.15.46:0/311737881 >> 172.21.15.46:0/311737881 conn(0x55da805f6210 msgr2=0x55da80692df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 -- 172.21.15.46:0/311737881 shutdown_connections 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.175+0000 7f12a41c8b80 1 -- 172.21.15.46:0/311737881 wait complete. 2024-05-08T23:12:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.176+0000 7f12a41c8b80 1 -- start start 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.176+0000 7f12a41c8b80 1 Processor -- start 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da806a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da806a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da80584560 con 0x55da8069de00 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da80584200 con 0x55da80698890 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da8046b6a0 con 0x55da8069e720 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f12a41c8b80 10 client.0 _ll_register_callbacks cb 0x55da8058db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f128dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da806a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f128e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f128dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da806a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59264/0 (socket says 172.21.15.46:59264) 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f128d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da806a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.177+0000 7f128dc00700 1 -- 172.21.15.46:0/3222763042 learned_addr learned my addr 172.21.15.46:0/3222763042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 -- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 msgr2=0x55da806a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da806a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 -- 172.21.15.46:0/3222763042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 msgr2=0x55da806a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da806a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 -- 172.21.15.46:0/3222763042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1274009920 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f128e600700 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a24a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1280003c80 tx=0x7f1280004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26be39b38a84336d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f126f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000e040 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1280004860 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da80541270 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1280025450 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f12a41c8b80 10 client.27997 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.178+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f12881a63c0 con 0x55da8069e720 2024-05-08T23:12:30.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da80541270 con 0x55da8069e720 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1280023030 con 0x55da8069e720 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 10 client.27997 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 10 client.27997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 20 client.27997 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da806b1eb0 0x55da806b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 4 client.27997 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da80667810 con 0x55da806b1eb0 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.179+0000 7f128dc00700 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da806b1eb0 0x55da806b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.180+0000 7f128dc00700 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da806b1eb0 0x55da806b42d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be5e31533aa102c server_cookie=dd16ffae2d7fa737 in_seq=0 out_seq=0 2024-05-08T23:12:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.180+0000 7f126f400700 10 client.27997 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.180+0000 7f126f400700 1 -- 172.21.15.46:0/3222763042 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da80667810 con 0x55da806b1eb0 2024-05-08T23:12:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.180+0000 7f126f400700 10 client.27997 handle_command_reply: tid=0 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 client.27997 shutdown 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 msgr2=0x55da806a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a24a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1280003c80 tx=0x7f1280004b40 comp rx=0 tx=0).stop 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 shutdown_connections 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a24a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da80698890 0x55da806a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da806b1eb0 0x55da806b42d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/3222763042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da8069de00 0x55da806a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f126f400700 0 client.27997 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 >> 172.21.15.46:0/3222763042 conn(0x55da805f6210 msgr2=0x55da806925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 shutdown_connections 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 1 -- 172.21.15.46:0/3222763042 wait complete. 2024-05-08T23:12:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.181+0000 7f12a41c8b80 20 client.27997 trim_cache size 0 max 16384 2024-05-08T23:12:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.200+0000 7f12a41c8b80 1 Processor -- start 2024-05-08T23:12:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 -- start start 2024-05-08T23:12:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da80698350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da80692d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da806a30c0 con 0x55da80698890 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da80694a50 con 0x55da8069de00 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da806b94e0 con 0x55da8069e720 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f128dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da80698350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f128e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f128e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40632/0 (socket says 172.21.15.46:40632) 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f128e600700 1 -- 172.21.15.46:0/2128173534 learned_addr learned my addr 172.21.15.46:0/2128173534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.201+0000 7f128d200700 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da80692d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 -- 172.21.15.46:0/2128173534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 msgr2=0x55da80692d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da80692d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 -- 172.21.15.46:0/2128173534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 msgr2=0x55da80698350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da80698350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 -- 172.21.15.46:0/2128173534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da80415b60 con 0x55da8069e720 2024-05-08T23:12:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128dc00700 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da80698350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f128e600700 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1280006040 tx=0x7f128000f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c883bb235ef6115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f127b400700 1 -- 172.21.15.46:0/2128173534 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280009920 con 0x55da8069e720 2024-05-08T23:12:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f127b400700 1 -- 172.21.15.46:0/2128173534 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128000f040 con 0x55da8069e720 2024-05-08T23:12:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.202+0000 7f127b400700 1 -- 172.21.15.46:0/2128173534 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800256b0 con 0x55da8069e720 2024-05-08T23:12:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- 172.21.15.46:0/2128173534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 msgr2=0x55da8069caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1280006040 tx=0x7f128000f1b0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- 172.21.15.46:0/2128173534 shutdown_connections 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da8069caf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da80698350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/2128173534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da80692d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- 172.21.15.46:0/2128173534 >> 172.21.15.46:0/2128173534 conn(0x55da805f6210 msgr2=0x55da80698c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- 172.21.15.46:0/2128173534 shutdown_connections 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- 172.21.15.46:0/2128173534 wait complete. 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.203+0000 7f12a41c8b80 1 -- start start 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 Processor -- start 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da806a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da806a7d10 con 0x55da80698890 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da806a7e80 con 0x55da8069de00 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da806a8040 con 0x55da8069e720 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f12a41c8b80 10 client.0 _ll_register_callbacks cb 0x55da8058db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.204+0000 7f128dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47988/0 (socket says 172.21.15.46:47988) 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 -- 172.21.15.46:0/620535334 learned_addr learned my addr 172.21.15.46:0/620535334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128d200700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da806a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 -- 172.21.15.46:0/620535334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 msgr2=0x55da806a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 -- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 msgr2=0x55da806a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da806a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 -- 172.21.15.46:0/620535334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da804693b0 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f128dc00700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f127c09ae80 tx=0x7f127c09b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845351e6e3364a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f126f400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c09ed20 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f127c0a32e0 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.205+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127c09b5f0 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127c0b9150 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 10 client.17925 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f12881a63c0 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127c09b5f0 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f127c09b150 con 0x55da80698890 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 10 client.17925 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 20 client.17925 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da807093b0 0x55da8070b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 4 client.17925 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:12:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.206+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da805e0ec0 con 0x55da807093b0 2024-05-08T23:12:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f128d200700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da807093b0 0x55da8070b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f128d200700 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da807093b0 0x55da8070b7a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8972274129090b server_cookie=dcaa5a57f57c9846 in_seq=0 out_seq=0 2024-05-08T23:12:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f126f400700 10 client.17925 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f126f400700 1 -- 172.21.15.46:0/620535334 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da805e0ec0 con 0x55da807093b0 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f126f400700 10 client.17925 handle_command_reply: tid=0 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f12a41c8b80 1 client.17925 shutdown 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 msgr2=0x55da806a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.207+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f127c09ae80 tx=0x7f127c09b760 comp rx=0 tx=0).stop 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 shutdown_connections 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da8069e720 0x55da806a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55da807093b0 0x55da8070b7a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da80698890 0x55da806a8eb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 --2- 172.21.15.46:0/620535334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da8069de00 0x55da806a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f126f400700 0 client.17925 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 >> 172.21.15.46:0/620535334 conn(0x55da805f6210 msgr2=0x55da806a1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 shutdown_connections 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 1 -- 172.21.15.46:0/620535334 wait complete. 2024-05-08T23:12:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.208+0000 7f12a41c8b80 20 client.17925 trim_cache size 0 max 16384 2024-05-08T23:12:30.212 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:12:30.212 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:12:30.212 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:12:30.212 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 msgr2=0x7f12700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 0x7f12700a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1280009fd0 tx=0x7f128000f5a0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 msgr2=0x7f1288199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f127c005910 tx=0x7f127c004730 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 shutdown_connections 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f128819de70 0x7f128819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1288103280 0x7f1288199590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f12700a1290 0x7f12700a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 --2- 172.21.15.46:0/526551185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12881083a0 0x7f1288199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 >> 172.21.15.46:0/526551185 conn(0x7f12880fea80 msgr2=0x7f12880ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 shutdown_connections 2024-05-08T23:12:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.209+0000 7f1290600700 1 -- 172.21.15.46:0/526551185 wait complete. 2024-05-08T23:12:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:29 smithi086 ceph-mon[78610]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:30.224 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:29 smithi046 ceph-mon[36898]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:30.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:12:30.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:12:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:29 smithi053 ceph-mon[77587]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 -- 172.21.15.46:0/1958396796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c101070 msgr2=0x7f786c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 --2- 172.21.15.46:0/1958396796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c101070 0x7f786c101450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f785c009990 tx=0x7f785c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 -- 172.21.15.46:0/1958396796 shutdown_connections 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 --2- 172.21.15.46:0/1958396796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c106190 0x7f786c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 --2- 172.21.15.46:0/1958396796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c101070 0x7f786c101450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 --2- 172.21.15.46:0/1958396796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c101a20 0x7f786c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 -- 172.21.15.46:0/1958396796 >> 172.21.15.46:0/1958396796 conn(0x7f786c0fc930 msgr2=0x7f786c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.437+0000 7f7872e00700 1 -- 172.21.15.46:0/1958396796 shutdown_connections 2024-05-08T23:12:30.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.438+0000 7f7872e00700 1 -- 172.21.15.46:0/1958396796 wait complete. 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.438+0000 7f7872e00700 1 Processor -- start 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 -- start start 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c106190 0x7f786c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 0x7f786c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c19c690 con 0x7f786c19bc10 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c19c800 con 0x7f786c106190 2024-05-08T23:12:30.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7872e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c19c9c0 con 0x7f786c101070 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f786be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f786be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40656/0 (socket says 172.21.15.46:40656) 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f786be00700 1 -- 172.21.15.46:0/1536829956 learned_addr learned my addr 172.21.15.46:0/1536829956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f786b400700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c106190 0x7f786c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.439+0000 7f7870e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 0x7f786c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 -- 172.21.15.46:0/1536829956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c106190 msgr2=0x7f786c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c106190 0x7f786c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 -- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 msgr2=0x7f786c19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 0x7f786c19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 -- 172.21.15.46:0/1536829956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009520 con 0x7f786c101070 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7870e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 0x7f786c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f786be00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f785c000c00 tx=0x7f785c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c01d070 con 0x7f786c101070 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c022470 con 0x7f786c101070 2024-05-08T23:12:30.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7864009c40 con 0x7f786c101070 2024-05-08T23:12:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c017650 con 0x7f786c101070 2024-05-08T23:12:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.440+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c0095f0 con 0x7f786c101070 2024-05-08T23:12:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.441+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c1093b0 con 0x7f786c101070 2024-05-08T23:12:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.443+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f785c0225e0 con 0x7f786c101070 2024-05-08T23:12:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.445+0000 7f7868c00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 0x7f78500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.445+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f785c0cea70 con 0x7f786c101070 2024-05-08T23:12:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.446+0000 7f786b400700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 0x7f78500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.446+0000 7f786b400700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 0x7f78500a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f786c198880 tx=0x7f7858009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.447+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f785c097ae0 con 0x7f786c101070 2024-05-08T23:12:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.729+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f786c102540 con 0x7f786c101070 2024-05-08T23:12:30.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.730+0000 7f7868c00700 1 -- 172.21.15.46:0/1536829956 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f785c09b100 con 0x7f786c101070 2024-05-08T23:12:30.729 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:12:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:12:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.732+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 msgr2=0x7f78500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.732+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 0x7f78500a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f786c198880 tx=0x7f7858009260 comp rx=0 tx=0).stop 2024-05-08T23:12:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.732+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 msgr2=0x7f786c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.732+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f785c000c00 tx=0x7f785c005740 comp rx=0 tx=0).stop 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 shutdown_connections 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78500a1290 0x7f78500a3750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f786c19bc10 0x7f786c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f786c101070 0x7f786c197330 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 --2- 172.21.15.46:0/1536829956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f786c106190 0x7f786c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 >> 172.21.15.46:0/1536829956 conn(0x7f786c0fc930 msgr2=0x7f786c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 shutdown_connections 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.733+0000 7f7872e00700 1 -- 172.21.15.46:0/1536829956 wait complete. 2024-05-08T23:12:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:12:30.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:12:30.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 745664 9.93 MB/sec execute 292 sec latency 15.340 ms 2024-05-08T23:12:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 -- 172.21.15.46:0/532524274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3490101fa0 msgr2=0x7f3490102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 --2- 172.21.15.46:0/532524274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3490101fa0 0x7f3490102380 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3480009990 tx=0x7f3480009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 -- 172.21.15.46:0/532524274 shutdown_connections 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 --2- 172.21.15.46:0/532524274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f349010b3c0 0x7f349010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 --2- 172.21.15.46:0/532524274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3490101fa0 0x7f3490102380 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 --2- 172.21.15.46:0/532524274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 0x7f349010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 -- 172.21.15.46:0/532524274 >> 172.21.15.46:0/532524274 conn(0x7f34900fb760 msgr2=0x7f34900fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 -- 172.21.15.46:0/532524274 shutdown_connections 2024-05-08T23:12:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.969+0000 7f3495600700 1 -- 172.21.15.46:0/532524274 wait complete. 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 Processor -- start 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 -- start start 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101fa0 0x7f34900ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 0x7f3490100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901a57d0 con 0x7f34901028c0 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901a5940 con 0x7f3490101fa0 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.970+0000 7f3495600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901a5b00 con 0x7f349010b3c0 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 0x7f3490100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40680/0 (socket says 172.21.15.46:40680) 2024-05-08T23:12:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101fa0 0x7f34900ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 -- 172.21.15.46:0/2359685585 learned_addr learned my addr 172.21.15.46:0/2359685585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 -- 172.21.15.46:0/2359685585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101fa0 msgr2=0x7f34900ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101fa0 0x7f34900ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 -- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 msgr2=0x7f3490100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 0x7f3490100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 -- 172.21.15.46:0/2359685585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800095f0 con 0x7f349010b3c0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.971+0000 7f348f400700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f348800ef30 tx=0x7f348800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.972+0000 7f3487400700 1 -- 172.21.15.46:0/2359685585 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488018070 con 0x7f349010b3c0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.972+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34901a5d80 con 0x7f349010b3c0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.972+0000 7f3487400700 1 -- 172.21.15.46:0/2359685585 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348800f040 con 0x7f349010b3c0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.972+0000 7f3487400700 1 -- 172.21.15.46:0/2359685585 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348800cd00 con 0x7f349010b3c0 2024-05-08T23:12:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.972+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34901a6240 con 0x7f349010b3c0 2024-05-08T23:12:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.975+0000 7f3487400700 1 -- 172.21.15.46:0/2359685585 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3488003a00 con 0x7f349010b3c0 2024-05-08T23:12:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.977+0000 7f3487400700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 0x7f34740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.977+0000 7f34a91f1b80 1 Processor -- start 2024-05-08T23:12:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.977+0000 7f3487400700 1 -- 172.21.15.46:0/2359685585 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f34880ccc50 con 0x7f349010b3c0 2024-05-08T23:12:30.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.977+0000 7f34a91f1b80 1 -- start start 2024-05-08T23:12:30.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.977+0000 7f348ea00700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 0x7f34740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d973c150 con 0x55e9d97eae00 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d9739b10 con 0x55e9d97eb720 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d9735400 con 0x55e9d97e5890 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40686/0 (socket says 172.21.15.46:40686) 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f348f400700 1 -- 172.21.15.46:0/4157346394 learned_addr learned my addr 172.21.15.46:0/4157346394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.978+0000 7f348e000700 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348ea00700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 0x7f34740a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3480006010 tx=0x7f348000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 -- 172.21.15.46:0/4157346394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 msgr2=0x55e9d97e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 -- 172.21.15.46:0/4157346394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 msgr2=0x55e9d97eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 -- 172.21.15.46:0/4157346394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9d95b63b0 con 0x55e9d97e5890 2024-05-08T23:12:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f348f400700 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f34880a1840 tx=0x7f34880994c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e7d81770987f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f347b400700 1 -- 172.21.15.46:0/4157346394 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488004ad0 con 0x55e9d97e5890 2024-05-08T23:12:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f347b400700 1 -- 172.21.15.46:0/4157346394 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348801b320 con 0x55e9d97e5890 2024-05-08T23:12:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.979+0000 7f347b400700 1 -- 172.21.15.46:0/4157346394 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488099be0 con 0x55e9d97e5890 2024-05-08T23:12:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 -- 172.21.15.46:0/4157346394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 msgr2=0x55e9d97e5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f34880a1840 tx=0x7f34880994c0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 -- 172.21.15.46:0/4157346394 shutdown_connections 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97e5cd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/4157346394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 -- 172.21.15.46:0/4157346394 >> 172.21.15.46:0/4157346394 conn(0x55e9d9743210 msgr2=0x55e9d97dfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.980+0000 7f34a91f1b80 1 -- 172.21.15.46:0/4157346394 shutdown_connections 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.981+0000 7f34a91f1b80 1 -- 172.21.15.46:0/4157346394 wait complete. 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.981+0000 7f34a91f1b80 1 -- start start 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 Processor -- start 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d96d1560 con 0x55e9d97eae00 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d96d1200 con 0x55e9d97e5890 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d95b86a0 con 0x55e9d97eb720 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f34a91f1b80 10 client.0 _ll_register_callbacks cb 0x55e9d96dab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f348ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f348ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59356/0 (socket says 172.21.15.46:59356) 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f348ea00700 1 -- 172.21.15.46:0/912775898 learned_addr learned my addr 172.21.15.46:0/912775898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.982+0000 7f348e000700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 -- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 msgr2=0x55e9d97ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 -- 172.21.15.46:0/912775898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 msgr2=0x55e9d97ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:30.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 -- 172.21.15.46:0/912775898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34901a6240 con 0x55e9d97e5890 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f348ea00700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f348000bad0 tx=0x7f3480009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2423f934dc79c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f346fe00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.983+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480004b10 con 0x55e9d97e5890 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3480004c70 con 0x55e9d97e5890 2024-05-08T23:12:30.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9d968e270 con 0x55e9d97e5890 2024-05-08T23:12:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3480004e30 con 0x55e9d97e5890 2024-05-08T23:12:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 10 client.27947 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e9d9562b60 con 0x55e9d97e5890 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9d968e270 con 0x55e9d97e5890 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34800245a0 con 0x55e9d97e5890 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 10 client.27947 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 10 client.27947 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 20 client.27947 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d97ff260 0x55e9d9801680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.984+0000 7f34a91f1b80 4 client.27947 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.985+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9d97b4810 con 0x55e9d97ff260 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.985+0000 7f348e000700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d97ff260 0x55e9d9801680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.985+0000 7f348e000700 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d97ff260 0x55e9d9801680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f11ca5e35b2f27b8 server_cookie=34eafc320afb3608 in_seq=0 out_seq=0 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.985+0000 7f346fe00700 10 client.27947 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f346fe00700 1 -- 172.21.15.46:0/912775898 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9d97b4810 con 0x55e9d97ff260 2024-05-08T23:12:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f346fe00700 10 client.27947 handle_command_reply: tid=0 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 client.27947 shutdown 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 msgr2=0x55e9d97eebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f348000bad0 tx=0x7f3480009310 comp rx=0 tx=0).stop 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 shutdown_connections 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d97ff260 0x55e9d9801680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97e5890 0x55e9d97eebd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f346fe00700 0 client.27947 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/912775898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 >> 172.21.15.46:0/912775898 conn(0x55e9d9743210 msgr2=0x55e9d97df5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 shutdown_connections 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.986+0000 7f34a91f1b80 1 -- 172.21.15.46:0/912775898 wait complete. 2024-05-08T23:12:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:30.987+0000 7f34a91f1b80 20 client.27947 trim_cache size 0 max 16384 2024-05-08T23:12:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.001+0000 7f34a91f1b80 1 Processor -- start 2024-05-08T23:12:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.001+0000 7f34a91f1b80 1 -- start start 2024-05-08T23:12:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eae00 0x55e9d97e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97eb430 con 0x55e9d97e5890 2024-05-08T23:12:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97e0e80 con 0x55e9d97eae00 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97e1800 con 0x55e9d97eb720 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f348ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f348f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f348e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eae00 0x55e9d97e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f348ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.002+0000 7f348ea00700 1 -- 172.21.15.46:0/3918086701 learned_addr learned my addr 172.21.15.46:0/3918086701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 -- 172.21.15.46:0/3918086701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 msgr2=0x55e9d97dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97dfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 -- 172.21.15.46:0/3918086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eae00 msgr2=0x55e9d97e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eae00 0x55e9d97e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 -- 172.21.15.46:0/3918086701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9d96fb560 con 0x55e9d97e5890 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348f400700 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f348ea00700 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480024f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16239f52a0d62e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f347be00700 1 -- 172.21.15.46:0/3918086701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480024410 con 0x55e9d97e5890 2024-05-08T23:12:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f347be00700 1 -- 172.21.15.46:0/3918086701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3480028070 con 0x55e9d97e5890 2024-05-08T23:12:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.003+0000 7f347be00700 1 -- 172.21.15.46:0/3918086701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000f7e0 con 0x55e9d97e5890 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 -- 172.21.15.46:0/3918086701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 msgr2=0x55e9d97e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480024f60 comp rx=0 tx=0).stop 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 -- 172.21.15.46:0/3918086701 shutdown_connections 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97eb720 0x55e9d97dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97e5890 0x55e9d97e5350 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/3918086701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eae00 0x55e9d97e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 -- 172.21.15.46:0/3918086701 >> 172.21.15.46:0/3918086701 conn(0x55e9d9743210 msgr2=0x55e9d97ed4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 -- 172.21.15.46:0/3918086701 shutdown_connections 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.004+0000 7f34a91f1b80 1 -- 172.21.15.46:0/3918086701 wait complete. 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 -- start start 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 Processor -- start 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97f4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97f4c10 con 0x55e9d97eae00 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97f4d80 con 0x55e9d97eb720 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d97f4f40 con 0x55e9d97e5890 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.005+0000 7f34a91f1b80 10 client.0 _ll_register_callbacks cb 0x55e9d96dab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40710/0 (socket says 172.21.15.46:40710) 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 -- 172.21.15.46:0/1735043443 learned_addr learned my addr 172.21.15.46:0/1735043443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 -- 172.21.15.46:0/1735043443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 msgr2=0x55e9d97f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 -- 172.21.15.46:0/1735043443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 msgr2=0x55e9d97f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 -- 172.21.15.46:0/1735043443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9d95b63b0 con 0x55e9d97e5890 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f348ea00700 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f348000bad0 tx=0x7f3480024410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260b1bdc2e784bea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f346fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.006+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480004bf0 con 0x55e9d97e5890 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f348801b320 con 0x55e9d97e5890 2024-05-08T23:12:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f34800246c0 con 0x55e9d97e5890 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3480004d50 con 0x55e9d97e5890 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 10 client.28027 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e9d9562b60 con 0x55e9d97e5890 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3480050080 con 0x55e9d97e5890 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3480046070 con 0x55e9d97e5890 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 10 client.28027 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 10 client.28027 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 20 client.28027 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d9855810 0x55e9d9857c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 4 client.28027 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.007+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9d972dec0 con 0x55e9d9855810 2024-05-08T23:12:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f348e000700 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d9855810 0x55e9d9857c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f346fe00700 10 client.28027 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f348e000700 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d9855810 0x55e9d9857c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42e987dc580824d3 server_cookie=4d1337c91d1b50c2 in_seq=0 out_seq=0 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f346fe00700 1 -- 172.21.15.46:0/1735043443 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9d972dec0 con 0x55e9d9855810 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f346fe00700 10 client.28027 handle_command_reply: tid=0 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.008+0000 7f34a91f1b80 1 client.28027 shutdown 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 msgr2=0x55e9d97f63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f348000bad0 tx=0x7f3480024410 comp rx=0 tx=0).stop 2024-05-08T23:12:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 shutdown_connections 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55e9d9855810 0x55e9d9857c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9d97eb720 0x55e9d97f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9d97e5890 0x55e9d97f63d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 --2- 172.21.15.46:0/1735043443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9d97eae00 0x55e9d97f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f346fe00700 0 client.28027 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 >> 172.21.15.46:0/1735043443 conn(0x55e9d9743210 msgr2=0x55e9d97e9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 shutdown_connections 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 1 -- 172.21.15.46:0/1735043443 wait complete. 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.009+0000 7f34a91f1b80 20 client.28027 trim_cache size 0 max 16384 2024-05-08T23:12:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:12:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 msgr2=0x7f34740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 0x7f34740a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3480006010 tx=0x7f348000b260 comp rx=0 tx=0).stop 2024-05-08T23:12:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 msgr2=0x7f3490100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f348800ef30 tx=0x7f348800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 shutdown_connections 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349010b3c0 0x7f3490100670 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101fa0 0x7f34900ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f34740a1290 0x7f34740a3750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 --2- 172.21.15.46:0/2359685585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34901028c0 0x7f3490100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 >> 172.21.15.46:0/2359685585 conn(0x7f34900fb760 msgr2=0x7f34900fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 shutdown_connections 2024-05-08T23:12:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.010+0000 7f3495600700 1 -- 172.21.15.46:0/2359685585 wait complete. 2024-05-08T23:12:31.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:30 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1536829956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:12:31.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:12:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:30 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1536829956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 -- 172.21.15.46:0/3171167621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff888101080 msgr2=0x7ff888101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 --2- 172.21.15.46:0/3171167621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff888101080 0x7ff888101460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff878009990 tx=0x7ff878009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 -- 172.21.15.46:0/3171167621 shutdown_connections 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 --2- 172.21.15.46:0/3171167621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8881061a0 0x7ff88810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 --2- 172.21.15.46:0/3171167621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff888101080 0x7ff888101460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 --2- 172.21.15.46:0/3171167621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101a30 0x7ff888105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.238+0000 7ff88cc00700 1 -- 172.21.15.46:0/3171167621 >> 172.21.15.46:0/3171167621 conn(0x7ff8880fc8e0 msgr2=0x7ff8880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.239+0000 7ff88cc00700 1 -- 172.21.15.46:0/3171167621 shutdown_connections 2024-05-08T23:12:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.239+0000 7ff88cc00700 1 -- 172.21.15.46:0/3171167621 wait complete. 2024-05-08T23:12:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.239+0000 7ff88cc00700 1 Processor -- start 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 -- start start 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101080 0x7ff8880732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 0x7ff8880737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 0x7ff8881a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5b30 con 0x7ff8881061a0 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5ca0 con 0x7ff888101080 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff88cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881a5e60 con 0x7ff888073d20 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff886800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 0x7ff8881a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff885400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 0x7ff8880737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff885e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101080 0x7ff8880732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff886800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 0x7ff8881a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40734/0 (socket says 172.21.15.46:40734) 2024-05-08T23:12:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.240+0000 7ff886800700 1 -- 172.21.15.46:0/1001865737 learned_addr learned my addr 172.21.15.46:0/1001865737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 -- 172.21.15.46:0/1001865737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 msgr2=0x7ff8881a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 0x7ff8881a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 -- 172.21.15.46:0/1001865737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101080 msgr2=0x7ff8880732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101080 0x7ff8880732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 -- 172.21.15.46:0/1001865737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780095f0 con 0x7ff8881061a0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff885400700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 0x7ff8880737e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e9c0 tx=0x7ff87400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff87ea00700 1 -- 172.21.15.46:0/1001865737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400c9c0 con 0x7ff8881061a0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff87ea00700 1 -- 172.21.15.46:0/1001865737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff874010430 con 0x7ff8881061a0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8881a6140 con 0x7ff8881061a0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff87ea00700 1 -- 172.21.15.46:0/1001865737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874018680 con 0x7ff8881061a0 2024-05-08T23:12:31.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.241+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8881a66a0 con 0x7ff8881061a0 2024-05-08T23:12:31.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.245+0000 7ff87ea00700 1 -- 172.21.15.46:0/1001865737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff87400cb20 con 0x7ff8881061a0 2024-05-08T23:12:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 Processor -- start 2024-05-08T23:12:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 -- start start 2024-05-08T23:12:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff87ea00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 0x7ff86c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a319350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a270150 con 0x56048a31f720 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff87ea00700 1 -- 172.21.15.46:0/1001865737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff874014070 con 0x7ff8881061a0 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a26db10 con 0x56048a31ee00 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.247+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a269400 con 0x56048a319890 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59400/0 (socket says 172.21.15.46:59400) 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 -- 172.21.15.46:0/549300263 learned_addr learned my addr 172.21.15.46:0/549300263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff886800700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 0x7ff86c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885400700 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a319350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff886800700 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 -- 172.21.15.46:0/549300263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 msgr2=0x56048a319cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 -- 172.21.15.46:0/549300263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 msgr2=0x56048a319350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a319350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 -- 172.21.15.46:0/549300263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56048a0ea3b0 con 0x56048a31ee00 2024-05-08T23:12:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff885e00700 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ec0 tx=0x7ff878005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c35bc5466e7d311c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.248+0000 7ff886800700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 0x7ff86c0a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff880009fd0 tx=0x7ff880009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff873000700 1 -- 172.21.15.46:0/549300263 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87801d070 con 0x56048a31ee00 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff873000700 1 -- 172.21.15.46:0/549300263 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87801e040 con 0x56048a31ee00 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff873000700 1 -- 172.21.15.46:0/549300263 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780047f0 con 0x56048a31ee00 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 -- 172.21.15.46:0/549300263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 msgr2=0x56048a31f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ec0 tx=0x7ff878005ef0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 -- 172.21.15.46:0/549300263 shutdown_connections 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a31f1e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.249+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/549300263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a319350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.250+0000 7ff8a0687b80 1 -- 172.21.15.46:0/549300263 >> 172.21.15.46:0/549300263 conn(0x56048a277210 msgr2=0x56048a313df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.250+0000 7ff8a0687b80 1 -- 172.21.15.46:0/549300263 shutdown_connections 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.250+0000 7ff8a0687b80 1 -- 172.21.15.46:0/549300263 wait complete. 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.250+0000 7ff8a0687b80 1 -- start start 2024-05-08T23:12:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.250+0000 7ff8a0687b80 1 Processor -- start 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a319890 0x56048a322a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a323490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a205560 con 0x56048a31f720 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a205200 con 0x56048a319890 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a0ec6a0 con 0x56048a31ee00 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff8a0687b80 10 client.0 _ll_register_callbacks cb 0x56048a20eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40748/0 (socket says 172.21.15.46:40748) 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 -- 172.21.15.46:0/1920436527 learned_addr learned my addr 172.21.15.46:0/1920436527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885e00700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a319890 0x56048a322a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff886800700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a323490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 -- 172.21.15.46:0/1920436527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a319890 msgr2=0x56048a322a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a319890 0x56048a322a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 -- 172.21.15.46:0/1920436527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 msgr2=0x56048a323490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.251+0000 7ff885400700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a323490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff885400700 1 -- 172.21.15.46:0/1920436527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88810e590 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff886800700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a323490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff885400700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c5b0 tx=0x7ff87409a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4eae99b21753248 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff867400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87409dc90 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff8740a22e0 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56048a1c2270 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff874010860 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff8a0687b80 10 client.28039 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56048a096b60 con 0x56048a31ee00 2024-05-08T23:12:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.252+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56048a1c2270 con 0x56048a31ee00 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8740a1f70 con 0x56048a31ee00 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 10 client.28039 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 10 client.28039 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 20 client.28039 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a3330c0 0x56048a3354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 4 client.28039 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56048a2e8810 con 0x56048a3330c0 2024-05-08T23:12:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.253+0000 7ff885e00700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a3330c0 0x56048a3354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff885e00700 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a3330c0 0x56048a3354e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=843c89f334198c6d server_cookie=1c54f2d728d2d782 in_seq=0 out_seq=0 2024-05-08T23:12:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff867400700 10 client.28039 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:12:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff867400700 1 -- 172.21.15.46:0/1920436527 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56048a2e8810 con 0x56048a3330c0 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff867400700 10 client.28039 handle_command_reply: tid=0 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff8a0687b80 1 client.28039 shutdown 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 msgr2=0x56048a322f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.254+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c5b0 tx=0x7ff87409a5d0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 shutdown_connections 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a3330c0 0x56048a3354e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31f720 0x56048a323490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff867400700 0 client.28039 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a319890 0x56048a322a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/1920436527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31ee00 0x56048a322f50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 >> 172.21.15.46:0/1920436527 conn(0x56048a277210 msgr2=0x56048a3135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 shutdown_connections 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 1 -- 172.21.15.46:0/1920436527 wait complete. 2024-05-08T23:12:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.255+0000 7ff8a0687b80 20 client.28039 trim_cache size 0 max 16384 2024-05-08T23:12:31.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:30 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1536829956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:12:31.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.269+0000 7ff8a0687b80 1 Processor -- start 2024-05-08T23:12:31.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 -- start start 2024-05-08T23:12:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31ee00 0x56048a313d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31f720 0x56048a32a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a324080 con 0x56048a31ee00 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a310dc0 con 0x56048a31f720 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a33a6f0 con 0x56048a319890 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff885e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff885400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31ee00 0x56048a313d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff885e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40750/0 (socket says 172.21.15.46:40750) 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff885e00700 1 -- 172.21.15.46:0/106678946 learned_addr learned my addr 172.21.15.46:0/106678946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.270+0000 7ff886800700 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31f720 0x56048a32a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 -- 172.21.15.46:0/106678946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31f720 msgr2=0x56048a32a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31f720 0x56048a32a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 -- 172.21.15.46:0/106678946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31ee00 msgr2=0x56048a313d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31ee00 0x56048a313d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 -- 172.21.15.46:0/106678946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56048a22f560 con 0x56048a319890 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff885e00700 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ec0 tx=0x7ff87801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b83049b1f3c5c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff873a00700 1 -- 172.21.15.46:0/106678946 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878032070 con 0x56048a319890 2024-05-08T23:12:31.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff873a00700 1 -- 172.21.15.46:0/106678946 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff878005350 con 0x56048a319890 2024-05-08T23:12:31.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.271+0000 7ff873a00700 1 -- 172.21.15.46:0/106678946 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87801d040 con 0x56048a319890 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- 172.21.15.46:0/106678946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 msgr2=0x56048a319350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ec0 tx=0x7ff87801a040 comp rx=0 tx=0).stop 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- 172.21.15.46:0/106678946 shutdown_connections 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31f720 0x56048a32a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a319890 0x56048a319350 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/106678946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a31ee00 0x56048a313d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- 172.21.15.46:0/106678946 >> 172.21.15.46:0/106678946 conn(0x56048a277210 msgr2=0x56048a3212e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- 172.21.15.46:0/106678946 shutdown_connections 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- 172.21.15.46:0/106678946 wait complete. 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 -- start start 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.272+0000 7ff8a0687b80 1 Processor -- start 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a32a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a32a9d0 con 0x56048a319890 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a31ae70 con 0x56048a31ee00 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048a31b030 con 0x56048a31f720 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff8a0687b80 10 client.0 _ll_register_callbacks cb 0x56048a20eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff885e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff886800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff885400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a32a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff886800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40764/0 (socket says 172.21.15.46:40764) 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff885e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48106/0 (socket says 172.21.15.46:48106) 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.273+0000 7ff886800700 1 -- 172.21.15.46:0/3882013341 learned_addr learned my addr 172.21.15.46:0/3882013341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 -- 172.21.15.46:0/3882013341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 msgr2=0x56048a32a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a32a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 -- 172.21.15.46:0/3882013341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 msgr2=0x56048a31c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 -- 172.21.15.46:0/3882013341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56048a0ea3b0 con 0x56048a31f720 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff885400700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a32a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff885e00700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff886800700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff880009c10 tx=0x7ff880004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf05d075515dc74f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff867400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000e070 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff88001ab10 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.274+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56048a261ec0 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff88001ad10 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 10 client.28045 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56048a0eecd0 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56048a261ec0 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8800045d0 con 0x56048a31f720 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 10 client.28045 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 10 client.28045 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 20 client.28045 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a38a470 0x56048a38c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 4 client.28045 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:12:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56048a0b7cf0 con 0x56048a38a470 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.275+0000 7ff885e00700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a38a470 0x56048a38c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff885e00700 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a38a470 0x56048a38c890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd8524ee6a668ea0 server_cookie=c4fb56e90e23b4e5 in_seq=0 out_seq=0 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff867400700 10 client.28045 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff867400700 1 -- 172.21.15.46:0/3882013341 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56048a0b7cf0 con 0x56048a38a470 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff867400700 10 client.28045 handle_command_reply: tid=0 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 client.28045 shutdown 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 msgr2=0x56048a31a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff880009c10 tx=0x7ff880004b40 comp rx=0 tx=0).stop 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 shutdown_connections 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56048a38a470 0x56048a38c890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56048a31f720 0x56048a31a930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56048a319890 0x56048a31c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 --2- 172.21.15.46:0/3882013341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56048a31ee00 0x56048a32a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff867400700 0 client.28045 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:12:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 >> 172.21.15.46:0/3882013341 conn(0x56048a277210 msgr2=0x56048a31db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 shutdown_connections 2024-05-08T23:12:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.276+0000 7ff8a0687b80 1 -- 172.21.15.46:0/3882013341 wait complete. 2024-05-08T23:12:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff8a0687b80 20 client.28045 trim_cache size 0 max 16384 2024-05-08T23:12:31.281 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 msgr2=0x7ff86c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 0x7ff86c0a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff880009fd0 tx=0x7ff880009920 comp rx=0 tx=0).stop 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 msgr2=0x7ff8880737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 0x7ff8880737e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e9c0 tx=0x7ff87400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 shutdown_connections 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888073d20 0x7ff8881a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff888101080 0x7ff8880732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff86c0a1310 0x7ff86c0a37d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 --2- 172.21.15.46:0/1001865737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8881061a0 0x7ff8880737e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.277+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 >> 172.21.15.46:0/1001865737 conn(0x7ff8880fc8e0 msgr2=0x7ff8880fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.278+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 shutdown_connections 2024-05-08T23:12:31.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.278+0000 7ff88cc00700 1 -- 172.21.15.46:0/1001865737 wait complete. 2024-05-08T23:12:31.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2833481932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8101860 msgr2=0x7f3bb81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2833481932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8101860 0x7f3bb81058a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00098f0 tx=0x7f3ba0009c00 comp rx=0 tx=0).stop 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2833481932 shutdown_connections 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2833481932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2833481932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8100eb0 0x7f3bb8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2833481932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8101860 0x7f3bb81058a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.506+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2833481932 >> 172.21.15.46:0/2833481932 conn(0x7f3bb80fc770 msgr2=0x7f3bb80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.507+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2833481932 shutdown_connections 2024-05-08T23:12:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.507+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2833481932 wait complete. 2024-05-08T23:12:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.508+0000 7f3bb7e00700 1 Processor -- start 2024-05-08T23:12:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.508+0000 7f3bb7e00700 1 -- start start 2024-05-08T23:12:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 0x7f3bb80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8073d20 0x7f3bb81a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81a5840 con 0x7f3bb8105fd0 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81a59b0 con 0x7f3bb8100eb0 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81a5b70 con 0x7f3bb8073d20 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 0x7f3bb80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 0x7f3bb80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59456/0 (socket says 172.21.15.46:59456) 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bb6a00700 1 -- 172.21.15.46:0/2448655219 learned_addr learned my addr 172.21.15.46:0/2448655219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.509+0000 7f3bafe00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bb7400700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8073d20 0x7f3bb81a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 -- 172.21.15.46:0/2448655219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8073d20 msgr2=0x7f3bb81a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8073d20 0x7f3bb81a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 -- 172.21.15.46:0/2448655219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 msgr2=0x7f3bb80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 0x7f3bb80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 -- 172.21.15.46:0/2448655219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8009d00 con 0x7f3bb8105fd0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.510+0000 7f3bafe00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb80737e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000c00 tx=0x7f3ba000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.511+0000 7f3baf400700 1 -- 172.21.15.46:0/2448655219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001c070 con 0x7f3bb8105fd0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.511+0000 7f3baf400700 1 -- 172.21.15.46:0/2448655219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba000ba20 con 0x7f3bb8105fd0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.511+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba0009520 con 0x7f3bb8105fd0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.511+0000 7f3baf400700 1 -- 172.21.15.46:0/2448655219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00038e0 con 0x7f3bb8105fd0 2024-05-08T23:12:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.511+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb81a60e0 con 0x7f3bb8105fd0 2024-05-08T23:12:31.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.515+0000 7f3baf400700 1 -- 172.21.15.46:0/2448655219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3ba0020030 con 0x7f3bb8105fd0 2024-05-08T23:12:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.516+0000 7f3bd1077b80 1 Processor -- start 2024-05-08T23:12:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.516+0000 7f3bd1077b80 1 -- start start 2024-05-08T23:12:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.516+0000 7f3baf400700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 0x7f3b9c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3baf400700 1 -- 172.21.15.46:0/2448655219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3ba00cdee0 con 0x7f3bb8105fd0 2024-05-08T23:12:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8880b150 con 0x55be888ba970 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88808b10 con 0x55be888b4890 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88804400 con 0x55be888ba050 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb7400700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 0x7f3b9c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40806/0 (socket says 172.21.15.46:40806) 2024-05-08T23:12:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 -- 172.21.15.46:0/1809470718 learned_addr learned my addr 172.21.15.46:0/1809470718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bafe00700 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 -- 172.21.15.46:0/1809470718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 msgr2=0x55be888b4cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 -- 172.21.15.46:0/1809470718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 msgr2=0x55be888b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.517+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3bb6a00700 1 -- 172.21.15.46:0/1809470718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be886853b0 con 0x55be888ba050 2024-05-08T23:12:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3bafe00700 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:12:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3bb7400700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 0x7f3b9c0a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c30 tx=0x7f3bb0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c720 tx=0x7f3ba80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd03eaf04486a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3ba7400700 1 -- 172.21.15.46:0/1809470718 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x55be888ba050 2024-05-08T23:12:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3ba7400700 1 -- 172.21.15.46:0/1809470718 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba800f040 con 0x55be888ba050 2024-05-08T23:12:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.518+0000 7f3ba7400700 1 -- 172.21.15.46:0/1809470718 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8007ea0 con 0x55be888ba050 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 -- 172.21.15.46:0/1809470718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 msgr2=0x55be888ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c720 tx=0x7f3ba80099e0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 -- 172.21.15.46:0/1809470718 shutdown_connections 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba050 0x55be888ba430 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/1809470718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.519+0000 7f3bd1077b80 1 -- 172.21.15.46:0/1809470718 >> 172.21.15.46:0/1809470718 conn(0x55be88812210 msgr2=0x55be888af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.520+0000 7f3bd1077b80 1 -- 172.21.15.46:0/1809470718 shutdown_connections 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.520+0000 7f3bd1077b80 1 -- 172.21.15.46:0/1809470718 wait complete. 2024-05-08T23:12:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.520+0000 7f3bd1077b80 1 -- start start 2024-05-08T23:12:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.520+0000 7f3bd1077b80 1 Processor -- start 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be887a0560 con 0x55be888ba050 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be887a0200 con 0x55be888b4890 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be886876a0 con 0x55be888ba970 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bd1077b80 10 client.0 _ll_register_callbacks cb 0x55be887a9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59472/0 (socket says 172.21.15.46:59472) 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bb6a00700 1 -- 172.21.15.46:0/3409411305 learned_addr learned my addr 172.21.15.46:0/3409411305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.521+0000 7f3bafe00700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 -- 172.21.15.46:0/3409411305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 msgr2=0x55be888c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 -- 172.21.15.46:0/3409411305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 msgr2=0x55be888bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888bdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 -- 172.21.15.46:0/3409411305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba800c3d0 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888be330 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00116c0 tx=0x7f3bb00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce447c36ab5f2f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.522+0000 7f3b9be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011ea0 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3bb0017070 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be8875d270 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb0004440 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3bd1077b80 10 client.28057 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be88631b60 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.523+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be8875d270 con 0x55be888ba970 2024-05-08T23:12:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3bb0027740 con 0x55be888ba970 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 10 client.28057 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 10 client.28057 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 20 client.28057 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be888ce390 0x55be888d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 4 client.28057 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be88883810 con 0x55be888ce390 2024-05-08T23:12:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.524+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be888ce390 0x55be888d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.525+0000 7f3b9be00700 10 client.28057 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:12:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.525+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be888ce390 0x55be888d07b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1e38b263c00721 server_cookie=b1ce436df2238b5f in_seq=0 out_seq=0 2024-05-08T23:12:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.525+0000 7f3b9be00700 1 -- 172.21.15.46:0/3409411305 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be88883810 con 0x55be888ce390 2024-05-08T23:12:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.525+0000 7f3b9be00700 10 client.28057 handle_command_reply: tid=0 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 client.28057 shutdown 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 msgr2=0x55be888be330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888be330 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00116c0 tx=0x7f3bb00047c0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 shutdown_connections 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888be330 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be888ce390 0x55be888d07b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3409411305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888bdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3b9be00700 0 client.28057 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 >> 172.21.15.46:0/3409411305 conn(0x55be88812210 msgr2=0x55be888af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 shutdown_connections 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.526+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3409411305 wait complete. 2024-05-08T23:12:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.527+0000 7f3bd1077b80 20 client.28057 trim_cache size 0 max 16384 2024-05-08T23:12:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.542+0000 7f3bd1077b80 1 Processor -- start 2024-05-08T23:12:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 -- start start 2024-05-08T23:12:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888b51b0 con 0x55be888ba050 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888b5320 con 0x55be888b4890 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888be240 con 0x55be888ba970 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bb7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40820/0 (socket says 172.21.15.46:40820) 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bb7400700 1 -- 172.21.15.46:0/3360964653 learned_addr learned my addr 172.21.15.46:0/3360964653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.543+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 -- 172.21.15.46:0/3360964653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 msgr2=0x55be888b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 -- 172.21.15.46:0/3360964653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 msgr2=0x55be888c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 -- 172.21.15.46:0/3360964653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be887ca560 con 0x55be888ba970 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0011cb0 tx=0x7f3bb0011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba91f49c9e5fd8f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3ba7e00700 1 -- 172.21.15.46:0/3360964653 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011ea0 con 0x55be888ba970 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3ba7e00700 1 -- 172.21.15.46:0/3360964653 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb0015070 con 0x55be888ba970 2024-05-08T23:12:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.544+0000 7f3ba7e00700 1 -- 172.21.15.46:0/3360964653 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0003aa0 con 0x55be888ba970 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3360964653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 msgr2=0x55be888b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0011cb0 tx=0x7f3bb0011ce0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3360964653 shutdown_connections 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888ba970 0x55be888b4c70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888b4890 0x55be888b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3360964653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba050 0x55be888c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3360964653 >> 172.21.15.46:0/3360964653 conn(0x55be88812210 msgr2=0x55be888b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3360964653 shutdown_connections 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3360964653 wait complete. 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.545+0000 7f3bd1077b80 1 -- start start 2024-05-08T23:12:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 Processor -- start 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888ba050 0x55be888b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888b6480 con 0x55be888ba970 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888b65f0 con 0x55be888ba050 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be888b67b0 con 0x55be888b4890 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bd1077b80 10 client.0 _ll_register_callbacks cb 0x55be887a9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888ba050 0x55be888b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bb6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40826/0 (socket says 172.21.15.46:40826) 2024-05-08T23:12:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.546+0000 7f3bb6a00700 1 -- 172.21.15.46:0/3111530265 learned_addr learned my addr 172.21.15.46:0/3111530265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 -- 172.21.15.46:0/3111530265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888ba050 msgr2=0x55be888b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888ba050 0x55be888b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 -- 172.21.15.46:0/3111530265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 msgr2=0x55be888b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 -- 172.21.15.46:0/3111530265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be88631b60 con 0x55be888b4890 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb7400700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bb6a00700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c6c0 tx=0x7f3ba80075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51ae9965f52f416 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3b9be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x55be888b4890 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3ba8027d70 con 0x55be888b4890 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.547+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ba800f040 con 0x55be888b4890 2024-05-08T23:12:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba8003a00 con 0x55be888b4890 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 10 client.28063 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be88689cd0 con 0x55be888b4890 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba800f040 con 0x55be888b4890 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ba801c070 con 0x55be888b4890 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 10 client.28063 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 10 client.28063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 20 client.28063 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be88924d70 0x55be88927190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 4 client.28063 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be887fcec0 con 0x55be88924d70 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.548+0000 7f3bafe00700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be88924d70 0x55be88927190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bafe00700 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be88924d70 0x55be88927190 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82962bd297d7b3d7 server_cookie=cc816191d2574c77 in_seq=0 out_seq=0 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3b9be00700 10 client.28063 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:12:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3b9be00700 1 -- 172.21.15.46:0/3111530265 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be887fcec0 con 0x55be88924d70 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3b9be00700 10 client.28063 handle_command_reply: tid=0 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 client.28063 shutdown 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 msgr2=0x55be888c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c6c0 tx=0x7f3ba80075c0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 shutdown_connections 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55be88924d70 0x55be88927190 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be888ba970 0x55be888b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be888b4890 0x55be888c5520 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3b9be00700 0 client.28063 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 --2- 172.21.15.46:0/3111530265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be888ba050 0x55be888b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.549+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 >> 172.21.15.46:0/3111530265 conn(0x55be88812210 msgr2=0x55be888af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 shutdown_connections 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bd1077b80 1 -- 172.21.15.46:0/3111530265 wait complete. 2024-05-08T23:12:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bd1077b80 20 client.28063 trim_cache size 0 max 16384 2024-05-08T23:12:31.554 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:12:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 msgr2=0x7f3b9c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 0x7f3b9c0a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c30 tx=0x7f3bb0009260 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 msgr2=0x7f3bb80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb80737e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000c00 tx=0x7f3ba000f9e0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 shutdown_connections 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8073d20 0x7f3bb81a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bb8100eb0 0x7f3bb80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3b9c0a1330 0x7f3b9c0a37f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 --2- 172.21.15.46:0/2448655219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb8105fd0 0x7f3bb80737e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.550+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 >> 172.21.15.46:0/2448655219 conn(0x7f3bb80fc770 msgr2=0x7f3bb80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.551+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 shutdown_connections 2024-05-08T23:12:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.551+0000 7f3bb7e00700 1 -- 172.21.15.46:0/2448655219 wait complete. 2024-05-08T23:12:31.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:12:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.801+0000 7f8e92e00700 1 -- 172.21.15.46:0/2387824631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c101a30 msgr2=0x7f8e8c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.801+0000 7f8e92e00700 1 --2- 172.21.15.46:0/2387824631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c101a30 0x7f8e8c105a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009990 tx=0x7f8e78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 -- 172.21.15.46:0/2387824631 shutdown_connections 2024-05-08T23:12:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 --2- 172.21.15.46:0/2387824631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 --2- 172.21.15.46:0/2387824631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c101080 0x7f8e8c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 --2- 172.21.15.46:0/2387824631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c101a30 0x7f8e8c105a70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 -- 172.21.15.46:0/2387824631 >> 172.21.15.46:0/2387824631 conn(0x7f8e8c0fc8e0 msgr2=0x7f8e8c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 -- 172.21.15.46:0/2387824631 shutdown_connections 2024-05-08T23:12:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.802+0000 7f8e92e00700 1 -- 172.21.15.46:0/2387824631 wait complete. 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 Processor -- start 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 -- start start 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c101080 0x7f8e8c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c19de90 0x7f8e8c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c19e910 con 0x7f8e8c101080 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c19ea80 con 0x7f8e8c19de90 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.803+0000 7f8e92e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c19ec40 con 0x7f8e8c1061a0 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c101080 0x7f8e8c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40844/0 (socket says 172.21.15.46:40844) 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 -- 172.21.15.46:0/1303842010 learned_addr learned my addr 172.21.15.46:0/1303842010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e90e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c19de90 0x7f8e8c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 -- 172.21.15.46:0/1303842010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c19de90 msgr2=0x7f8e8c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c19de90 0x7f8e8c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 -- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c101080 msgr2=0x7f8e8c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c101080 0x7f8e8c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 -- 172.21.15.46:0/1303842010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009520 con 0x7f8e8c1061a0 2024-05-08T23:12:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.804+0000 7f8e8b400700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8e780059d0 tx=0x7f8e78004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.805+0000 7f8e88c00700 1 -- 172.21.15.46:0/1303842010 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7801d070 con 0x7f8e8c1061a0 2024-05-08T23:12:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.805+0000 7f8e88c00700 1 -- 172.21.15.46:0/1303842010 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e78022470 con 0x7f8e8c1061a0 2024-05-08T23:12:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.805+0000 7f8e88c00700 1 -- 172.21.15.46:0/1303842010 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78017650 con 0x7f8e8c1061a0 2024-05-08T23:12:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.805+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e84009c40 con 0x7f8e8c1061a0 2024-05-08T23:12:31.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.805+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e780095f0 con 0x7f8e8c1061a0 2024-05-08T23:12:31.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.808+0000 7f8e88c00700 1 -- 172.21.15.46:0/1303842010 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8e780225e0 con 0x7f8e8c1061a0 2024-05-08T23:12:31.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.809+0000 7f8e88c00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 0x7f8e700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.809+0000 7f8e88c00700 1 -- 172.21.15.46:0/1303842010 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8e780ce190 con 0x7f8e8c1061a0 2024-05-08T23:12:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.809+0000 7f8ea699db80 1 Processor -- start 2024-05-08T23:12:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.809+0000 7f8e8be00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 0x7f8e700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.809+0000 7f8ea699db80 1 -- start start 2024-05-08T23:12:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46ba5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46afc150 con 0x564f46baae00 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46af9b10 con 0x564f46ba5890 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8ea699db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46af5400 con 0x564f46bab720 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e90e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48174/0 (socket says 172.21.15.46:48174) 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e8be00700 1 -- 172.21.15.46:0/2662298056 learned_addr learned my addr 172.21.15.46:0/2662298056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e8b400700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46ba5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e8be00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 0x7f8e700a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009df0 tx=0x7f8e7c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.810+0000 7f8e90e00700 1 -- 172.21.15.46:0/2662298056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 msgr2=0x564f46ba5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e90e00700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46ba5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e90e00700 1 -- 172.21.15.46:0/2662298056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 msgr2=0x564f46bab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e90e00700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e90e00700 1 -- 172.21.15.46:0/2662298056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f469763b0 con 0x564f46ba5890 2024-05-08T23:12:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e8b400700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46ba5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e8be00700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e90e00700 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400e650 tx=0x7f8e84009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ca9aab39b9817 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e81600700 1 -- 172.21.15.46:0/2662298056 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400e040 con 0x564f46ba5890 2024-05-08T23:12:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e81600700 1 -- 172.21.15.46:0/2662298056 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e8400f040 con 0x564f46ba5890 2024-05-08T23:12:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.811+0000 7f8e81600700 1 -- 172.21.15.46:0/2662298056 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84007500 con 0x564f46ba5890 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 -- 172.21.15.46:0/2662298056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 msgr2=0x564f46ba5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400e650 tx=0x7f8e84009920 comp rx=0 tx=0).stop 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 -- 172.21.15.46:0/2662298056 shutdown_connections 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5cd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 --2- 172.21.15.46:0/2662298056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46ba5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 -- 172.21.15.46:0/2662298056 >> 172.21.15.46:0/2662298056 conn(0x564f46b03210 msgr2=0x564f46b9fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 -- 172.21.15.46:0/2662298056 shutdown_connections 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.812+0000 7f8ea699db80 1 -- 172.21.15.46:0/2662298056 wait complete. 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 -- start start 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 Processor -- start 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46baeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46baf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46a91560 con 0x564f46bab720 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46a91200 con 0x564f46ba5890 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f469786a0 con 0x564f46baae00 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8ea699db80 10 client.0 _ll_register_callbacks cb 0x564f46a9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8e8b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8e8b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39298/0 (socket says 172.21.15.46:39298) 2024-05-08T23:12:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8e8b400700 1 -- 172.21.15.46:0/3971682279 learned_addr learned my addr 172.21.15.46:0/3971682279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.813+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46baeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46baf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 -- 172.21.15.46:0/3971682279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 msgr2=0x564f46baeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46baeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 -- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 msgr2=0x564f46baf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46baf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 -- 172.21.15.46:0/3971682279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e780095f0 con 0x564f46baae00 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8e78022c60 tx=0x7f8e78097320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd06cc8e5ee132d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e76000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7809a5b0 con 0x564f46baae00 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8e7809b5c0 con 0x564f46baae00 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.814+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f46a4e270 con 0x564f46baae00 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7809a820 con 0x564f46baae00 2024-05-08T23:12:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 10 client.28075 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8e8400e330 con 0x564f46baae00 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7807d080 con 0x564f46baae00 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e780a2e20 con 0x564f46baae00 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 10 client.28075 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 10 client.28075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 20 client.28075 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46bbee60 0x564f46bc1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 4 client.28075 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f46b74810 con 0x564f46bbee60 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.815+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46bbee60 0x564f46bc1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.816+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46bbee60 0x564f46bc1250 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e24d9e842291186c server_cookie=5e404803a370446f in_seq=0 out_seq=0 2024-05-08T23:12:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.816+0000 7f8e76000700 10 client.28075 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8e76000700 1 -- 172.21.15.46:0/3971682279 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f46b74810 con 0x564f46bbee60 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8e76000700 10 client.28075 handle_command_reply: tid=0 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 client.28075 shutdown 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 msgr2=0x564f46baf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8e78022c60 tx=0x7f8e78097320 comp rx=0 tx=0).stop 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 shutdown_connections 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46bbee60 0x564f46bc1250 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46baf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46baeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8e76000700 0 client.28075 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3971682279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46baae00 0x564f46baf0c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 >> 172.21.15.46:0/3971682279 conn(0x564f46b03210 msgr2=0x564f46b9f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 shutdown_connections 2024-05-08T23:12:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 1 -- 172.21.15.46:0/3971682279 wait complete. 2024-05-08T23:12:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.817+0000 7f8ea699db80 20 client.28075 trim_cache size 0 max 16384 2024-05-08T23:12:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 Processor -- start 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 -- start start 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bbb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46b9fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46ba1640 con 0x564f46baae00 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46bb01a0 con 0x564f46ba5890 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8ea699db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46b9cdf0 con 0x564f46bab720 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8e90e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46b9fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8e8b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bbb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51812/0 (socket says 172.21.15.46:51812) 2024-05-08T23:12:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.833+0000 7f8e8be00700 1 -- 172.21.15.46:0/3845846005 learned_addr learned my addr 172.21.15.46:0/3845846005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 -- 172.21.15.46:0/3845846005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 msgr2=0x564f46ba5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 -- 172.21.15.46:0/3845846005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 msgr2=0x564f46bbb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bbb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 -- 172.21.15.46:0/3845846005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f46922b60 con 0x564f46bab720 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46b9fd20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400ec40 tx=0x7f8e84009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cfbff1314770fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e82000700 1 -- 172.21.15.46:0/3845846005 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400e040 con 0x564f46bab720 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e82000700 1 -- 172.21.15.46:0/3845846005 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e84004bc0 con 0x564f46bab720 2024-05-08T23:12:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.834+0000 7f8e82000700 1 -- 172.21.15.46:0/3845846005 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84007cf0 con 0x564f46bab720 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 -- 172.21.15.46:0/3845846005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 msgr2=0x564f46b9fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46b9fd20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400ec40 tx=0x7f8e84009620 comp rx=0 tx=0).stop 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 -- 172.21.15.46:0/3845846005 shutdown_connections 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46bab720 0x564f46b9fd20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46ba5890 0x564f46ba5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3845846005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46baae00 0x564f46bbb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 -- 172.21.15.46:0/3845846005 >> 172.21.15.46:0/3845846005 conn(0x564f46b03210 msgr2=0x564f46b9c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 -- 172.21.15.46:0/3845846005 shutdown_connections 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.835+0000 7f8ea699db80 1 -- 172.21.15.46:0/3845846005 wait complete. 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 -- start start 2024-05-08T23:12:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 Processor -- start 2024-05-08T23:12:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46baae00 0x564f46bb4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46bb4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46c143c0 con 0x564f46bab720 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46c14530 con 0x564f46baae00 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f46c146f0 con 0x564f46ba5890 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8ea699db80 10 client.0 _ll_register_callbacks cb 0x564f46a9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39324/0 (socket says 172.21.15.46:39324) 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8e8be00700 1 -- 172.21.15.46:0/3502161900 learned_addr learned my addr 172.21.15.46:0/3502161900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.836+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46bb4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46baae00 0x564f46bb4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 -- 172.21.15.46:0/3502161900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46baae00 msgr2=0x564f46bb4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46baae00 0x564f46bb4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 -- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 msgr2=0x564f46bb4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46bb4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 -- 172.21.15.46:0/3502161900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f469763b0 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e90e00700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46bb4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e8be00700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004680 tx=0x7f8e7c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c526f719edd54b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e76000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c022040 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8e7c003b40 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.837+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e8400f040 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c003d00 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 10 client.28081 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564f46abb560 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e8400f040 con 0x564f46ba5890 2024-05-08T23:12:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e7c01c400 con 0x564f46ba5890 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 10 client.28081 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 10 client.28081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 20 client.28081 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46c15ee0 0x564f46c18300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 4 client.28081 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f46aedec0 con 0x564f46c15ee0 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.838+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46c15ee0 0x564f46c18300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8e8b400700 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46c15ee0 0x564f46c18300 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a15be208a95f014 server_cookie=d8628a41793ee9e1 in_seq=0 out_seq=0 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8e76000700 10 client.28081 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8e76000700 1 -- 172.21.15.46:0/3502161900 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f46aedec0 con 0x564f46c15ee0 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8e76000700 10 client.28081 handle_command_reply: tid=0 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 client.28081 shutdown 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 msgr2=0x564f46bb4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004680 tx=0x7f8e7c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 shutdown_connections 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f46c15ee0 0x564f46c18300 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f46bab720 0x564f46bb4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f46ba5890 0x564f46bb4150 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8e76000700 0 client.28081 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 --2- 172.21.15.46:0/3502161900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f46baae00 0x564f46bb4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.839+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 >> 172.21.15.46:0/3502161900 conn(0x564f46b03210 msgr2=0x564f46bad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 shutdown_connections 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8ea699db80 1 -- 172.21.15.46:0/3502161900 wait complete. 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8ea699db80 20 client.28081 trim_cache size 0 max 16384 2024-05-08T23:12:31.843 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:12:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 msgr2=0x7f8e700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 0x7f8e700a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009df0 tx=0x7f8e7c009260 comp rx=0 tx=0).stop 2024-05-08T23:12:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 msgr2=0x7f8e8c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8e780059d0 tx=0x7f8e78004730 comp rx=0 tx=0).stop 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 shutdown_connections 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c19de90 0x7f8e8c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8e700a1310 0x7f8e700a37d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c101080 0x7f8e8c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 --2- 172.21.15.46:0/1303842010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e8c1061a0 0x7f8e8c199af0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.840+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 >> 172.21.15.46:0/1303842010 conn(0x7f8e8c0fc8e0 msgr2=0x7f8e8c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.841+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 shutdown_connections 2024-05-08T23:12:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:31.841+0000 7f8e92e00700 1 -- 172.21.15.46:0/1303842010 wait complete. 2024-05-08T23:12:31.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:12:31.898 INFO:tasks.workunit.client.0.smithi046.stdout: 1 747061 9.92 MB/sec execute 293 sec latency 13.864 ms 2024-05-08T23:12:32.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.070+0000 7fb2a9200700 1 -- 172.21.15.46:0/1704897791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a4103c60 msgr2=0x7fb2a4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.070+0000 7fb2a9200700 1 --2- 172.21.15.46:0/1704897791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a4103c60 0x7fb2a4107ca0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb290009990 tx=0x7fb290009ca0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 -- 172.21.15.46:0/1704897791 shutdown_connections 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 --2- 172.21.15.46:0/1704897791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a41083d0 0x7fb2a410ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 --2- 172.21.15.46:0/1704897791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41032b0 0x7fb2a4103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 --2- 172.21.15.46:0/1704897791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a4103c60 0x7fb2a4107ca0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 -- 172.21.15.46:0/1704897791 >> 172.21.15.46:0/1704897791 conn(0x7fb2a40feb10 msgr2=0x7fb2a4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 -- 172.21.15.46:0/1704897791 shutdown_connections 2024-05-08T23:12:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.071+0000 7fb2a9200700 1 -- 172.21.15.46:0/1704897791 wait complete. 2024-05-08T23:12:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 Processor -- start 2024-05-08T23:12:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 -- start start 2024-05-08T23:12:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 0x7fb2a4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 0x7fb2a419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419e8f0 con 0x7fb2a41083d0 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419ea60 con 0x7fb2a419de70 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.072+0000 7fb2a9200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419ec20 con 0x7fb2a41032b0 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 0x7fb2a419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39344/0 (socket says 172.21.15.46:39344) 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1e00700 1 -- 172.21.15.46:0/738896712 learned_addr learned my addr 172.21.15.46:0/738896712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 0x7fb2a4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 -- 172.21.15.46:0/738896712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 msgr2=0x7fb2a4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 -- 172.21.15.46:0/738896712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 msgr2=0x7fb2a419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 0x7fb2a419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 -- 172.21.15.46:0/738896712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009520 con 0x7fb2a41083d0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a2800700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 0x7fb2a419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.073+0000 7fb2a1400700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 0x7fb2a4199ad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb290000c00 tx=0x7fb290004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.074+0000 7fb29aa00700 1 -- 172.21.15.46:0/738896712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29001d070 con 0x7fb2a41083d0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.074+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2900095f0 con 0x7fb2a41083d0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.074+0000 7fb29aa00700 1 -- 172.21.15.46:0/738896712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb290021410 con 0x7fb2a41083d0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.074+0000 7fb29aa00700 1 -- 172.21.15.46:0/738896712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290017650 con 0x7fb2a41083d0 2024-05-08T23:12:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.074+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a41a63c0 con 0x7fb2a41083d0 2024-05-08T23:12:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.077+0000 7fb29aa00700 1 -- 172.21.15.46:0/738896712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb2900217f0 con 0x7fb2a41083d0 2024-05-08T23:12:32.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.078+0000 7fb29aa00700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 0x7fb2880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.078+0000 7fb29aa00700 1 -- 172.21.15.46:0/738896712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb2900ce3f0 con 0x7fb2a41083d0 2024-05-08T23:12:32.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.078+0000 7fb2bcd9bb80 1 Processor -- start 2024-05-08T23:12:32.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 -- start start 2024-05-08T23:12:32.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 0x7fb2880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6052720 0x5645f604c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5fa3150 con 0x5645f6052720 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5fa0b10 con 0x5645f6051e00 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5f9c400 con 0x5645f604c890 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51846/0 (socket says 172.21.15.46:51846) 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3332931788 learned_addr learned my addr 172.21.15.46:0/3332931788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6052720 0x5645f604c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.079+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 0x7fb2880a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb294009fd0 tx=0x7fb294009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3332931788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 msgr2=0x5645f604ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3332931788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6052720 msgr2=0x5645f604c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6052720 0x5645f604c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3332931788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f5e1d3b0 con 0x5645f6051e00 2024-05-08T23:12:32.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.080+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb29400f7a0 tx=0x7fb29401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314648dd359adcb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb28f400700 1 -- 172.21.15.46:0/3332931788 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29401c600 con 0x5645f6051e00 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb28f400700 1 -- 172.21.15.46:0/3332931788 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29401cc20 con 0x5645f6051e00 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb28f400700 1 -- 172.21.15.46:0/3332931788 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294024c30 con 0x5645f6051e00 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3332931788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 msgr2=0x5645f60521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb29400f7a0 tx=0x7fb29401c000 comp rx=0 tx=0).stop 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3332931788 shutdown_connections 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6051e00 0x5645f60521e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3332931788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6052720 0x5645f604c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.081+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3332931788 >> 172.21.15.46:0/3332931788 conn(0x5645f5faa210 msgr2=0x5645f6046df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.082+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3332931788 shutdown_connections 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.082+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3332931788 wait complete. 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.082+0000 7fb2bcd9bb80 1 -- start start 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.082+0000 7fb2bcd9bb80 1 Processor -- start 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6055ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f6056210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5f38560 con 0x5645f6051e00 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5f38200 con 0x5645f604c890 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f5e1f6a0 con 0x5645f6052720 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2bcd9bb80 10 client.0 _ll_register_callbacks cb 0x5645f5f41b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f6056210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6055ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51848/0 (socket says 172.21.15.46:51848) 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f6056210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39356/0 (socket says 172.21.15.46:39356) 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.083+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3132200061 learned_addr learned my addr 172.21.15.46:0/3132200061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3132200061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 msgr2=0x5645f6056210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f6056210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3132200061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 msgr2=0x5645f6055ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6055ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3132200061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294017050 con 0x5645f604c890 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb294025d00 tx=0x7fb294025d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f3fce3e0f0295c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb283e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294005040 con 0x5645f604c890 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645f5ef5270 con 0x5645f604c890 2024-05-08T23:12:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.084+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb2940051a0 con 0x5645f604c890 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb294005360 con 0x5645f604c890 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 10 client.28016 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5645f5dc9b60 con 0x5645f604c890 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645f5ef5270 con 0x5645f604c890 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2940046d0 con 0x5645f604c890 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 10 client.28016 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 10 client.28016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 20 client.28016 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f6066080 0x5645f60684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 4 client.28016 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.085+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645f601b810 con 0x5645f6066080 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.086+0000 7fb2a1400700 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f6066080 0x5645f60684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.086+0000 7fb2a1400700 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f6066080 0x5645f60684a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1f4a8b6433bcfb server_cookie=20547e2314f90394 in_seq=0 out_seq=0 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.086+0000 7fb283e00700 10 client.28016 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb283e00700 1 -- 172.21.15.46:0/3132200061 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645f601b810 con 0x5645f6066080 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb283e00700 10 client.28016 handle_command_reply: tid=0 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 client.28016 shutdown 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 msgr2=0x5645f60587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb294025d00 tx=0x7fb294025d30 comp rx=0 tx=0).stop 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 shutdown_connections 2024-05-08T23:12:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f6056210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f60587e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f6066080 0x5645f60684a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3132200061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6055ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb283e00700 0 client.28016 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 >> 172.21.15.46:0/3132200061 conn(0x5645f5faa210 msgr2=0x5645f60465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 shutdown_connections 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3132200061 wait complete. 2024-05-08T23:12:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.087+0000 7fb2bcd9bb80 20 client.28016 trim_cache size 0 max 16384 2024-05-08T23:12:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 Processor -- start 2024-05-08T23:12:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 -- start start 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6056210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6052720 0x5645f604cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f6041ad0 con 0x5645f6051e00 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f6058ca0 con 0x5645f6052720 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f6041830 con 0x5645f604c890 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6056210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6052720 0x5645f604cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39358/0 (socket says 172.21.15.46:39358) 2024-05-08T23:12:32.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.103+0000 7fb2a1e00700 1 -- 172.21.15.46:0/3779564963 learned_addr learned my addr 172.21.15.46:0/3779564963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 -- 172.21.15.46:0/3779564963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 msgr2=0x5645f604c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 -- 172.21.15.46:0/3779564963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6052720 msgr2=0x5645f604cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6052720 0x5645f604cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 -- 172.21.15.46:0/3779564963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f5f62560 con 0x5645f6051e00 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb2a1400700 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6056210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cb6d0 tx=0x7fb2900a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87e5e66fed9cfcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb28fe00700 1 -- 172.21.15.46:0/3779564963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2070 con 0x5645f6051e00 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb28fe00700 1 -- 172.21.15.46:0/3779564963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb290021e90 con 0x5645f6051e00 2024-05-08T23:12:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.104+0000 7fb28fe00700 1 -- 172.21.15.46:0/3779564963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29009d070 con 0x5645f6051e00 2024-05-08T23:12:32.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3779564963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 msgr2=0x5645f6056210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6056210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cb6d0 tx=0x7fb2900a1040 comp rx=0 tx=0).stop 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3779564963 shutdown_connections 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f6052720 0x5645f604cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f604c890 0x5645f604c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/3779564963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f6056210 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3779564963 >> 172.21.15.46:0/3779564963 conn(0x5645f5faa210 msgr2=0x5645f60487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3779564963 shutdown_connections 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.105+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/3779564963 wait complete. 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 -- start start 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 Processor -- start 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f605b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f605b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f605bbd0 con 0x5645f6051e00 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f605bd10 con 0x5645f604c890 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.106+0000 7fb2bcd9bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f605be80 con 0x5645f6052720 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2bcd9bb80 10 client.0 _ll_register_callbacks cb 0x5645f5f41b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f605b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51876/0 (socket says 172.21.15.46:51876) 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 -- 172.21.15.46:0/720344591 learned_addr learned my addr 172.21.15.46:0/720344591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a2800700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f605b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 -- 172.21.15.46:0/720344591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 msgr2=0x5645f605b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f605b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 -- 172.21.15.46:0/720344591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 msgr2=0x5645f605b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:32.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f605b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 -- 172.21.15.46:0/720344591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645f5dc9b60 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1400700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f605b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb2a1e00700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb294004a30 tx=0x7fb29400fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51912c915dc6b0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.107+0000 7fb283e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294011ea0 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb29401cc20 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb294004c50 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb294025440 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb2bcd9bb80 10 client.28022 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb29009fe70 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.108+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29401cc20 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb2940050e0 con 0x5645f604c890 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 10 client.28022 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:32.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 10 client.28022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 20 client.28022 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f60bc730 0x5645f60beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 4 client.28022 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645f5f94ec0 con 0x5645f60bc730 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2a1400700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f60bc730 0x5645f60beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb2a1400700 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f60bc730 0x5645f60beb50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca161d0dbb45ae2e server_cookie=3088df9291f94cee in_seq=0 out_seq=0 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.109+0000 7fb283e00700 10 client.28022 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb283e00700 1 -- 172.21.15.46:0/720344591 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645f5f94ec0 con 0x5645f60bc730 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb283e00700 10 client.28022 handle_command_reply: tid=0 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 client.28022 shutdown 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 msgr2=0x5645f605d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb294004a30 tx=0x7fb29400fe40 comp rx=0 tx=0).stop 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 shutdown_connections 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645f6052720 0x5645f605b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645f604c890 0x5645f605d360 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5645f60bc730 0x5645f60beb50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 --2- 172.21.15.46:0/720344591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645f6051e00 0x5645f605b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 >> 172.21.15.46:0/720344591 conn(0x5645f5faa210 msgr2=0x5645f604cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb283e00700 0 client.28022 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 shutdown_connections 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 1 -- 172.21.15.46:0/720344591 wait complete. 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.110+0000 7fb2bcd9bb80 20 client.28022 trim_cache size 0 max 16384 2024-05-08T23:12:32.114 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:12:32.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 msgr2=0x7fb2880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 0x7fb2880a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb294009fd0 tx=0x7fb294009260 comp rx=0 tx=0).stop 2024-05-08T23:12:32.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 msgr2=0x7fb2a4199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 0x7fb2a4199ad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb290000c00 tx=0x7fb290004730 comp rx=0 tx=0).stop 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 shutdown_connections 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2a419de70 0x7fb2a419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a41032b0 0x7fb2a4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb2880a1290 0x7fb2880a3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 --2- 172.21.15.46:0/738896712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a41083d0 0x7fb2a4199ad0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 >> 172.21.15.46:0/738896712 conn(0x7fb2a40feb10 msgr2=0x7fb2a40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 shutdown_connections 2024-05-08T23:12:32.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:32.111+0000 7fb2a9200700 1 -- 172.21.15.46:0/738896712 wait complete. 2024-05-08T23:12:32.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:31 smithi046 ceph-mon[36898]: pgmap v268: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:31 smithi086 ceph-mon[78610]: pgmap v268: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:31 smithi053 ceph-mon[77587]: pgmap v268: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:32.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 748066 9.89 MB/sec execute 294 sec latency 23.709 ms 2024-05-08T23:12:33.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:12:33.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 -- 172.21.15.46:0/2197352437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 msgr2=0x7f1acc10ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 --2- 172.21.15.46:0/2197352437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc10ff50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009f90 tx=0x7f1ac400b330 comp rx=0 tx=0).stop 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 -- 172.21.15.46:0/2197352437 shutdown_connections 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 --2- 172.21.15.46:0/2197352437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc10ff50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 --2- 172.21.15.46:0/2197352437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc1032b0 0x7f1acc103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 --2- 172.21.15.46:0/2197352437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc103c60 0x7f1acc107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 -- 172.21.15.46:0/2197352437 >> 172.21.15.46:0/2197352437 conn(0x7f1acc0feb10 msgr2=0x7f1acc100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.362+0000 7f1ad4600700 1 -- 172.21.15.46:0/2197352437 shutdown_connections 2024-05-08T23:12:33.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.363+0000 7f1ad4600700 1 -- 172.21.15.46:0/2197352437 wait complete. 2024-05-08T23:12:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 Processor -- start 2024-05-08T23:12:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 -- start start 2024-05-08T23:12:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 0x7f1acc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc19ea30 con 0x7f1acc1083d0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc19eba0 con 0x7f1acc1032b0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad4600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc19ed60 con 0x7f1acc19dfb0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51900/0 (socket says 172.21.15.46:51900) 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3391678189 learned_addr learned my addr 172.21.15.46:0/3391678189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad2600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 0x7f1acc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.364+0000 7f1ad1200700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3391678189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 msgr2=0x7f1acc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 0x7f1acc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 msgr2=0x7f1acc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3391678189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00095f0 con 0x7f1acc1032b0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad2600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 0x7f1acc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.365+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000ec10 tx=0x7f1ac000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.366+0000 7f1aca000700 1 -- 172.21.15.46:0/3391678189 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000cc90 con 0x7f1acc1032b0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.366+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac4009c40 con 0x7f1acc1032b0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.366+0000 7f1aca000700 1 -- 172.21.15.46:0/3391678189 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac00044e0 con 0x7f1acc1032b0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.366+0000 7f1aca000700 1 -- 172.21.15.46:0/3391678189 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0010430 con 0x7f1acc1032b0 2024-05-08T23:12:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.366+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc1a6660 con 0x7f1acc1032b0 2024-05-08T23:12:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.369+0000 7f1aca000700 1 -- 172.21.15.46:0/3391678189 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1ac0003fb0 con 0x7f1acc1032b0 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 Processor -- start 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1aca000700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 0x7f1ab40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1aca000700 1 -- 172.21.15.46:0/3391678189 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ac0014070 con 0x7f1acc1032b0 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 -- start start 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ad2600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 0x7f1ab40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 0x55f50789e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f507898910 0x55f507898d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5077ef1d0 con 0x55f50789de80 2024-05-08T23:12:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5077ecb90 con 0x55f507898910 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.371+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5077e8480 con 0x55f50789e7a0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 0x55f50789e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39392/0 (socket says 172.21.15.46:39392) 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 -- 172.21.15.46:0/1601180489 learned_addr learned my addr 172.21.15.46:0/1601180489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad2600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 0x7f1ab40a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009790 tx=0x7f1ac4009700 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad2600700 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f507898910 0x55f507898d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 -- 172.21.15.46:0/1601180489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f507898910 msgr2=0x55f507898d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f507898910 0x55f507898d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 -- 172.21.15.46:0/1601180489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 msgr2=0x55f50789e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 0x55f50789e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 -- 172.21.15.46:0/1601180489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f507669430 con 0x55f50789e7a0 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 0x55f50789e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.372+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009cb0 tx=0x7f1ab8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf9169882935e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.373+0000 7f1abea00700 1 -- 172.21.15.46:0/1601180489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8011d20 con 0x55f50789e7a0 2024-05-08T23:12:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.373+0000 7f1abea00700 1 -- 172.21.15.46:0/1601180489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab8011e80 con 0x55f50789e7a0 2024-05-08T23:12:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.373+0000 7f1abea00700 1 -- 172.21.15.46:0/1601180489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8005280 con 0x55f50789e7a0 2024-05-08T23:12:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.373+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1601180489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 msgr2=0x55f5078983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.373+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009cb0 tx=0x7f1ab8011440 comp rx=0 tx=0).stop 2024-05-08T23:12:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1601180489 shutdown_connections 2024-05-08T23:12:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f507898910 0x55f507898d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789de80 0x55f50789e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1601180489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078983d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1601180489 >> 172.21.15.46:0/1601180489 conn(0x55f5077f6290 msgr2=0x55f507892e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1601180489 shutdown_connections 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1601180489 wait complete. 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.374+0000 7f1ae8144b80 1 -- start start 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 Processor -- start 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5077845e0 con 0x55f507898910 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f507784280 con 0x55f50789de80 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50766b720 con 0x55f50789e7a0 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ae8144b80 10 client.0 _ll_register_callbacks cb 0x55f50778dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56524/0 (socket says 172.21.15.46:56524) 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ad2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3999380350 learned_addr learned my addr 172.21.15.46:0/3999380350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.375+0000 7f1ad1200700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3999380350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 msgr2=0x55f5078a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 msgr2=0x55f5078a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3999380350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc1a6660 con 0x55f507898910 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c5b0 tx=0x7f1ac0096620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c7f93efe5bb178 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ab3400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009a9e0 con 0x55f507898910 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ac009a2e0 con 0x55f507898910 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f507867890 con 0x55f507898910 2024-05-08T23:12:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.376+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac009e110 con 0x55f507898910 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 10 client.18027 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1ab80098e0 con 0x55f507898910 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f507867890 con 0x55f507898910 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ac00998f0 con 0x55f507898910 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 10 client.18027 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 20 client.18027 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5078b23f0 0x55f5078b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 4 client.18027 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5077e0f40 con 0x55f5078b23f0 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.377+0000 7f1ad1200700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5078b23f0 0x55f5078b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.378+0000 7f1ad1200700 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5078b23f0 0x55f5078b47e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50dc5b31a25e0a59 server_cookie=61596aed6591db9b in_seq=0 out_seq=0 2024-05-08T23:12:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.378+0000 7f1ab3400700 10 client.18027 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ab3400700 1 -- 172.21.15.46:0/3999380350 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5077e0f40 con 0x55f5078b23f0 2024-05-08T23:12:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ab3400700 10 client.18027 handle_command_reply: tid=0 2024-05-08T23:12:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 client.18027 shutdown 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 msgr2=0x55f5078a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c5b0 tx=0x7f1ac0096620 comp rx=0 tx=0).stop 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 shutdown_connections 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f50789e7a0 0x55f5078a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5078b23f0 0x55f5078b47e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f507898910 0x55f5078a1b70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3999380350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ab3400700 0 client.18027 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 >> 172.21.15.46:0/3999380350 conn(0x55f5077f6290 msgr2=0x55f507892620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 shutdown_connections 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.379+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3999380350 wait complete. 2024-05-08T23:12:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.380+0000 7f1ae8144b80 20 client.18027 trim_cache size 0 max 16384 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 Processor -- start 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 -- start start 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5078a46e0 con 0x55f50789e7a0 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50788fe40 con 0x55f50789de80 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.400+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f507894ed0 con 0x55f507898910 2024-05-08T23:12:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad2600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51920/0 (socket says 172.21.15.46:51920) 2024-05-08T23:12:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39412/0 (socket says 172.21.15.46:39412) 2024-05-08T23:12:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1200700 1 -- 172.21.15.46:0/3978319977 learned_addr learned my addr 172.21.15.46:0/3978319977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3978319977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 msgr2=0x55f5078a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3978319977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 msgr2=0x55f5078a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 -- 172.21.15.46:0/3978319977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f507898fc0 con 0x55f507898910 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1200700 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.401+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0099fe0 tx=0x7f1ac009a170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1560e3a4c27f9447 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1abf400700 1 -- 172.21.15.46:0/3978319977 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a7070 con 0x55f507898910 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1abf400700 1 -- 172.21.15.46:0/3978319977 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac00b2e90 con 0x55f507898910 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1abf400700 1 -- 172.21.15.46:0/3978319977 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009a370 con 0x55f507898910 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3978319977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 msgr2=0x55f507892da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0099fe0 tx=0x7f1ac009a170 comp rx=0 tx=0).stop 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3978319977 shutdown_connections 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f507892da0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/3978319977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3978319977 >> 172.21.15.46:0/3978319977 conn(0x55f5077f6290 msgr2=0x55f5078a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.402+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3978319977 shutdown_connections 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.403+0000 7f1ae8144b80 1 -- 172.21.15.46:0/3978319977 wait complete. 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.403+0000 7f1ae8144b80 1 -- start start 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.403+0000 7f1ae8144b80 1 Processor -- start 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5078a7ef0 con 0x55f50789e7a0 2024-05-08T23:12:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5078a8060 con 0x55f50789de80 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5078a8220 con 0x55f507898910 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ae8144b80 10 client.0 _ll_register_callbacks cb 0x55f50778dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39416/0 (socket says 172.21.15.46:39416) 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 -- 172.21.15.46:0/1608908089 learned_addr learned my addr 172.21.15.46:0/1608908089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad2600700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 -- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 msgr2=0x55f5078a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 -- 172.21.15.46:0/1608908089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 msgr2=0x55f5078a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 -- 172.21.15.46:0/1608908089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab801d040 con 0x55f507898910 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad2600700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:12:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ad1c00700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00038b0 tx=0x7f1ac0095f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b189a59cc43491 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.404+0000 7f1ab3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a7070 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ac009a750 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5077412f0 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac0009a10 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 10 client.28117 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1ab80098e0 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5077412f0 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ac0099840 con 0x55f507898910 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 10 client.28117 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 10 client.28117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 20 client.28117 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5079097d0 0x55f50790bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 4 client.28117 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:12:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.405+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f507636d70 con 0x55f5079097d0 2024-05-08T23:12:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.406+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5079097d0 0x55f50790bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.406+0000 7f1ad1200700 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5079097d0 0x55f50790bbc0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743a806e88a36ae8 server_cookie=6367d080dbb5b4da in_seq=0 out_seq=0 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.406+0000 7f1ab3400700 10 client.28117 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.407+0000 7f1ab3400700 1 -- 172.21.15.46:0/1608908089 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f507636d70 con 0x55f5079097d0 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.407+0000 7f1ab3400700 10 client.28117 handle_command_reply: tid=0 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.407+0000 7f1ae8144b80 1 client.28117 shutdown 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.407+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 msgr2=0x55f5078a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.407+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00038b0 tx=0x7f1ac0095f80 comp rx=0 tx=0).stop 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 shutdown_connections 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f5079097d0 0x55f50790bbc0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f50789e7a0 0x55f5078a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f507898910 0x55f5078a91a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ab3400700 0 client.28117 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 --2- 172.21.15.46:0/1608908089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f50789de80 0x55f5078a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 >> 172.21.15.46:0/1608908089 conn(0x55f5077f6290 msgr2=0x55f50789cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 shutdown_connections 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 1 -- 172.21.15.46:0/1608908089 wait complete. 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.408+0000 7f1ae8144b80 20 client.28117 trim_cache size 0 max 16384 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:12:33.411 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a0c049ee-f78c-4516-bb16-fe23239e1ca5", 2024-05-08T23:12:33.412 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:12:33.412 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:12:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 msgr2=0x7f1ab40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 0x7f1ab40a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009790 tx=0x7f1ac4009700 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 msgr2=0x7f1acc199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000ec10 tx=0x7f1ac000ef20 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 shutdown_connections 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1acc19dfb0 0x7f1acc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1acc1032b0 0x7f1acc199720 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ab40a1310 0x7f1ab40a37d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 --2- 172.21.15.46:0/3391678189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1acc1083d0 0x7f1acc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 >> 172.21.15.46:0/3391678189 conn(0x7f1acc0feb10 msgr2=0x7f1acc100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 shutdown_connections 2024-05-08T23:12:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.409+0000 7f1ad4600700 1 -- 172.21.15.46:0/3391678189 wait complete. 2024-05-08T23:12:33.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 -- 172.21.15.46:0/4172553673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 msgr2=0x7fcd58101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 --2- 172.21.15.46:0/4172553673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58101460 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcd480098f0 tx=0x7fcd48009c00 comp rx=0 tx=0).stop 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 -- 172.21.15.46:0/4172553673 shutdown_connections 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 --2- 172.21.15.46:0/4172553673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 0x7fcd5810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 --2- 172.21.15.46:0/4172553673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58101460 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 --2- 172.21.15.46:0/4172553673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd58101a30 0x7fcd58105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 -- 172.21.15.46:0/4172553673 >> 172.21.15.46:0/4172553673 conn(0x7fcd580fc8e0 msgr2=0x7fcd580fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 -- 172.21.15.46:0/4172553673 shutdown_connections 2024-05-08T23:12:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.645+0000 7fcd5e000700 1 -- 172.21.15.46:0/4172553673 wait complete. 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.646+0000 7fcd5e000700 1 Processor -- start 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.646+0000 7fcd5e000700 1 -- start start 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.646+0000 7fcd5e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.646+0000 7fcd5e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 0x7fcd58199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd5e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5819de50 0x7fcd5819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd5e000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819e8d0 con 0x7fcd581061a0 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd5e000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819ea40 con 0x7fcd5819de50 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd5e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819ec00 con 0x7fcd58101080 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39430/0 (socket says 172.21.15.46:39430) 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 -- 172.21.15.46:0/2997823720 learned_addr learned my addr 172.21.15.46:0/2997823720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd56a00700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 0x7fcd58199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57e00700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5819de50 0x7fcd5819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 -- 172.21.15.46:0/2997823720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5819de50 msgr2=0x7fcd5819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5819de50 0x7fcd5819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 -- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 msgr2=0x7fcd58199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 0x7fcd58199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.647+0000 7fcd57400700 1 -- 172.21.15.46:0/2997823720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440095f0 con 0x7fcd58101080 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd57400700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005950 tx=0x7fcd4800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd4fe00700 1 -- 172.21.15.46:0/2997823720 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801c070 con 0x7fcd58101080 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd4fe00700 1 -- 172.21.15.46:0/2997823720 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4800b9c0 con 0x7fcd58101080 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd48009520 con 0x7fcd58101080 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd4fe00700 1 -- 172.21.15.46:0/2997823720 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480038e0 con 0x7fcd58101080 2024-05-08T23:12:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.648+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd581a6390 con 0x7fcd58101080 2024-05-08T23:12:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.651+0000 7fcd4fe00700 1 -- 172.21.15.46:0/2997823720 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcd48020030 con 0x7fcd58101080 2024-05-08T23:12:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.652+0000 7fcd4fe00700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 0x7fcd3c0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.652+0000 7fcd4fe00700 1 -- 172.21.15.46:0/2997823720 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcd480cde00 con 0x7fcd58101080 2024-05-08T23:12:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd56a00700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 0x7fcd3c0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 Processor -- start 2024-05-08T23:12:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 -- start start 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd56a00700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 0x7fcd3c0a38a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd5819aac0 tx=0x7fcd44009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cea5c150 con 0x55d8ceb05890 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cea59b10 con 0x55d8ceb0ae00 2024-05-08T23:12:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.653+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cea55400 con 0x55d8ceb0b720 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd57400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39436/0 (socket says 172.21.15.46:39436) 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 -- 172.21.15.46:0/3143960324 learned_addr learned my addr 172.21.15.46:0/3143960324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd57400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51940/0 (socket says 172.21.15.46:51940) 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd57e00700 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 -- 172.21.15.46:0/3143960324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 msgr2=0x55d8ceb0b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 -- 172.21.15.46:0/3143960324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 msgr2=0x55d8ceb05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 -- 172.21.15.46:0/3143960324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ce8d63b0 con 0x55d8ceb0b720 2024-05-08T23:12:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e640 tx=0x7fcd44014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8114b290e787ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.654+0000 7fcd43e00700 1 -- 172.21.15.46:0/3143960324 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44026430 con 0x55d8ceb0b720 2024-05-08T23:12:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd43e00700 1 -- 172.21.15.46:0/3143960324 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44021070 con 0x55d8ceb0b720 2024-05-08T23:12:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd43e00700 1 -- 172.21.15.46:0/3143960324 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401d7e0 con 0x55d8ceb0b720 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3143960324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 msgr2=0x55d8ceb05350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e640 tx=0x7fcd44014000 comp rx=0 tx=0).stop 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3143960324 shutdown_connections 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3143960324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb05350 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3143960324 >> 172.21.15.46:0/3143960324 conn(0x55d8cea63210 msgr2=0x55d8ceaffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.655+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3143960324 shutdown_connections 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.656+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3143960324 wait complete. 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.656+0000 7fcd71a8ab80 1 -- start start 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.656+0000 7fcd71a8ab80 1 Processor -- start 2024-05-08T23:12:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce9f1560 con 0x55d8ceb05890 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce9f1200 con 0x55d8ceb0ae00 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ce8d86a0 con 0x55d8ceb0b720 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd71a8ab80 10 client.0 _ll_register_callbacks cb 0x55d8ce9fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39452/0 (socket says 172.21.15.46:39452) 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56572/0 (socket says 172.21.15.46:56572) 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 -- 172.21.15.46:0/1334828181 learned_addr learned my addr 172.21.15.46:0/1334828181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd56a00700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 -- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 msgr2=0x55d8ceb0f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 -- 172.21.15.46:0/1334828181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 msgr2=0x55d8ceb0ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.657+0000 7fcd57e00700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd57e00700 1 -- 172.21.15.46:0/1334828181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44017070 con 0x55d8ceb0b720 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd57400700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd57e00700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcd50011f60 tx=0x7fcd50014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5956ce2d3ed1e9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd40c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000e040 con 0x55d8ceb0b720 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8ce9ae270 con 0x55d8ceb0b720 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcd50014b40 con 0x55d8ceb0b720 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd50014d00 con 0x55d8ceb0b720 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd71a8ab80 10 client.28129 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.658+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8ce882b60 con 0x55d8ceb0b720 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8ce9ae270 con 0x55d8ceb0b720 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd5001c020 con 0x55d8ceb0b720 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 10 client.28129 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 10 client.28129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 20 client.28129 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb1f3a0 0x55d8ceb217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 4 client.28129 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8cead4810 con 0x55d8ceb1f3a0 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.659+0000 7fcd57400700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb1f3a0 0x55d8ceb217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.660+0000 7fcd57400700 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb1f3a0 0x55d8ceb217c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1b981edf03c4 server_cookie=bbd944107aa4ec25 in_seq=0 out_seq=0 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.660+0000 7fcd40c00700 10 client.28129 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.660+0000 7fcd40c00700 1 -- 172.21.15.46:0/1334828181 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8cead4810 con 0x55d8ceb1f3a0 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.660+0000 7fcd40c00700 10 client.28129 handle_command_reply: tid=0 2024-05-08T23:12:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.660+0000 7fcd71a8ab80 1 client.28129 shutdown 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 msgr2=0x55d8ceb0f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcd50011f60 tx=0x7fcd50014600 comp rx=0 tx=0).stop 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 shutdown_connections 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceb0f7b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb1f3a0 0x55d8ceb217c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb0ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/1334828181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb0f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd40c00700 0 client.28129 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 >> 172.21.15.46:0/1334828181 conn(0x55d8cea63210 msgr2=0x55d8ceaff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 shutdown_connections 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/1334828181 wait complete. 2024-05-08T23:12:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.661+0000 7fcd71a8ab80 20 client.28129 trim_cache size 0 max 16384 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 Processor -- start 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 -- start start 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceaffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceaf80b0 con 0x55d8ceb05890 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceb0d260 con 0x55d8ceb0ae00 2024-05-08T23:12:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.681+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceb061b0 con 0x55d8ceb0b720 2024-05-08T23:12:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd57400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceaffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51956/0 (socket says 172.21.15.46:51956) 2024-05-08T23:12:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 -- 172.21.15.46:0/3048938080 learned_addr learned my addr 172.21.15.46:0/3048938080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 -- 172.21.15.46:0/3048938080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 msgr2=0x55d8ceaffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceaffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 -- 172.21.15.46:0/3048938080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 msgr2=0x55d8ceb05350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 -- 172.21.15.46:0/3048938080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cea1b560 con 0x55d8ceb0ae00 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.682+0000 7fcd56a00700 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e640 tx=0x7fcd44005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50144313042f753b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.683+0000 7fcd4d200700 1 -- 172.21.15.46:0/3048938080 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400ede0 con 0x55d8ceb0ae00 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.683+0000 7fcd4d200700 1 -- 172.21.15.46:0/3048938080 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44005500 con 0x55d8ceb0ae00 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.683+0000 7fcd4d200700 1 -- 172.21.15.46:0/3048938080 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004df0 con 0x55d8ceb0ae00 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.683+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3048938080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 msgr2=0x55d8ceb05c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.683+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e640 tx=0x7fcd44005a90 comp rx=0 tx=0).stop 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3048938080 shutdown_connections 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0b720 0x55d8ceaffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb05890 0x55d8ceb05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/3048938080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb05c70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3048938080 >> 172.21.15.46:0/3048938080 conn(0x55d8cea63210 msgr2=0x55d8ceafc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3048938080 shutdown_connections 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/3048938080 wait complete. 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 -- start start 2024-05-08T23:12:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.684+0000 7fcd71a8ab80 1 Processor -- start 2024-05-08T23:12:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 0x55d8ceb12140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 0x55d8ceb12bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceb16a70 con 0x55d8ceb0b720 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceb74850 con 0x55d8ceb05890 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ceb74990 con 0x55d8ceb0ae00 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd71a8ab80 10 client.0 _ll_register_callbacks cb 0x55d8ce9fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd56a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39476/0 (socket says 172.21.15.46:39476) 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 0x55d8ceb12bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd56a00700 1 -- 172.21.15.46:0/4250831030 learned_addr learned my addr 172.21.15.46:0/4250831030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:12:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd57400700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 0x55d8ceb12140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.685+0000 7fcd56a00700 1 -- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 msgr2=0x55d8ceb12140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd56a00700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 0x55d8ceb12140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd56a00700 1 -- 172.21.15.46:0/4250831030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 msgr2=0x55d8ceb12bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd56a00700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 0x55d8ceb12bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd56a00700 1 -- 172.21.15.46:0/4250831030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4402a040 con 0x55d8ceb0ae00 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd57e00700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 0x55d8ceb12bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd57400700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 0x55d8ceb12140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd56a00700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009440 tx=0x7fcd44005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09c33637b75879c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd40c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401c410 con 0x55d8ceb0ae00 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcd4401c570 con 0x55d8ceb0ae00 2024-05-08T23:12:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd44021070 con 0x55d8ceb0ae00 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4401c730 con 0x55d8ceb0ae00 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd71a8ab80 10 client.28135 fetch_fsmap learned FSMap version 30 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.686+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8ce8dacd0 con 0x55d8ceb0ae00 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44021070 con 0x55d8ceb0ae00 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd4402d070 con 0x55d8ceb0ae00 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 10 client.28135 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 10 client.28135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 20 client.28135 populate_metadata read hostname 'smithi046' 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb75cc0 0x55d8ceb780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 4 client.28135 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8cea4dec0 con 0x55d8ceb75cc0 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.687+0000 7fcd57400700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb75cc0 0x55d8ceb780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd57400700 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb75cc0 0x55d8ceb780b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917baebbd82d171e server_cookie=9d807cb5c6f79bb6 in_seq=0 out_seq=0 2024-05-08T23:12:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd40c00700 10 client.28135 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd40c00700 1 -- 172.21.15.46:0/4250831030 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d8cea4dec0 con 0x55d8ceb75cc0 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd40c00700 10 client.28135 handle_command_reply: tid=0 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 client.28135 shutdown 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 msgr2=0x55d8ceb12680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009440 tx=0x7fcd44005700 comp rx=0 tx=0).stop 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 shutdown_connections 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d8ceb75cc0 0x55d8ceb780b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8ceb0b720 0x55d8ceb12bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8ceb05890 0x55d8ceb12140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 --2- 172.21.15.46:0/4250831030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8ceb0ae00 0x55d8ceb12680 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd40c00700 0 client.28135 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 >> 172.21.15.46:0/4250831030 conn(0x55d8cea63210 msgr2=0x55d8ceb0d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 shutdown_connections 2024-05-08T23:12:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.688+0000 7fcd71a8ab80 1 -- 172.21.15.46:0/4250831030 wait complete. 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd71a8ab80 20 client.28135 trim_cache size 0 max 16384 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "a0c049ee-f78c-4516-bb16-fe23239e1ca5": { 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a0c049ee-f78c-4516-bb16-fe23239e1ca5", 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:12:33.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:12:33.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:12:33.695 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:12:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 msgr2=0x7fcd3c0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 0x7fcd3c0a38a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd5819aac0 tx=0x7fcd44009310 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 msgr2=0x7fcd58199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005950 tx=0x7fcd4800fa00 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 shutdown_connections 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5819de50 0x7fcd5819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58101080 0x7fcd58199570 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.689+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcd3c0a13e0 0x7fcd3c0a38a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.690+0000 7fcd5e000700 1 --2- 172.21.15.46:0/2997823720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd581061a0 0x7fcd58199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.690+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 >> 172.21.15.46:0/2997823720 conn(0x7fcd580fc8e0 msgr2=0x7fcd580fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.690+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 shutdown_connections 2024-05-08T23:12:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:12:33.690+0000 7fcd5e000700 1 -- 172.21.15.46:0/2997823720 wait complete. 2024-05-08T23:12:33.708 INFO:tasks.cephfs.filesystem:scrub status for tag:a0c049ee-f78c-4516-bb16-fe23239e1ca5 - {'path': '/', 'tag': 'a0c049ee-f78c-4516-bb16-fe23239e1ca5', 'options': 'recursive,force'} 2024-05-08T23:12:33.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 749720 9.90 MB/sec execute 295 sec latency 13.232 ms 2024-05-08T23:12:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:33 smithi086 ceph-mon[78610]: pgmap v269: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-08T23:12:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:33 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:34.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:33 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:33 smithi053 ceph-mon[77587]: pgmap v269: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-08T23:12:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:33 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:33 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:33 smithi046 ceph-mon[36898]: pgmap v269: 129 pgs: 129 active+clean; 473 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-08T23:12:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:33 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:33 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:34.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 751697 9.89 MB/sec execute 296 sec latency 14.756 ms 2024-05-08T23:12:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:12:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:35.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:12:35.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:34 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:12:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:12:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:34 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:34 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:12:35.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 753387 9.90 MB/sec execute 297 sec latency 16.183 ms 2024-05-08T23:12:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:35 smithi086 ceph-mon[78610]: pgmap v270: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-05-08T23:12:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:35 smithi053 ceph-mon[77587]: pgmap v270: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-05-08T23:12:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:35 smithi046 ceph-mon[36898]: pgmap v270: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-05-08T23:12:36.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 755385 9.89 MB/sec execute 298 sec latency 15.660 ms 2024-05-08T23:12:37.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 756922 9.90 MB/sec execute 299 sec latency 16.752 ms 2024-05-08T23:12:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:37 smithi086 ceph-mon[78610]: pgmap v271: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:37 smithi053 ceph-mon[77587]: pgmap v271: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:37 smithi046 ceph-mon[36898]: pgmap v271: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-08T23:12:38.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 758703 9.89 MB/sec execute 300 sec latency 142.581 ms 2024-05-08T23:12:39.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 760191 9.89 MB/sec execute 301 sec latency 135.847 ms 2024-05-08T23:12:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:39 smithi086 ceph-mon[78610]: pgmap v272: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-05-08T23:12:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:39 smithi053 ceph-mon[77587]: pgmap v272: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-05-08T23:12:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:39 smithi046 ceph-mon[36898]: pgmap v272: 129 pgs: 129 active+clean; 481 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-05-08T23:12:40.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 760908 9.87 MB/sec execute 302 sec latency 193.391 ms 2024-05-08T23:12:41.900 INFO:tasks.workunit.client.0.smithi046.stdout: 1 762743 9.87 MB/sec execute 303 sec latency 28.615 ms 2024-05-08T23:12:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:41 smithi086 ceph-mon[78610]: pgmap v273: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:12:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:41 smithi053 ceph-mon[77587]: pgmap v273: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:12:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:41 smithi046 ceph-mon[36898]: pgmap v273: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 198 op/s 2024-05-08T23:12:42.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 764147 9.86 MB/sec execute 304 sec latency 14.368 ms 2024-05-08T23:12:43.899 INFO:tasks.workunit.client.0.smithi046.stdout: 1 766286 9.86 MB/sec execute 305 sec latency 20.246 ms 2024-05-08T23:12:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:43 smithi086 ceph-mon[78610]: pgmap v274: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-08T23:12:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:43 smithi053 ceph-mon[77587]: pgmap v274: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-08T23:12:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:43 smithi046 ceph-mon[36898]: pgmap v274: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-05-08T23:12:44.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 767667 9.86 MB/sec execute 306 sec latency 46.638 ms 2024-05-08T23:12:45.901 INFO:tasks.workunit.client.0.smithi046.stdout: 1 769436 9.85 MB/sec execute 307 sec latency 83.026 ms 2024-05-08T23:12:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:45 smithi086 ceph-mon[78610]: pgmap v275: 129 pgs: 129 active+clean; 489 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 253 op/s 2024-05-08T23:12:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:12:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:45 smithi053 ceph-mon[77587]: pgmap v275: 129 pgs: 129 active+clean; 489 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 253 op/s 2024-05-08T23:12:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:12:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:45 smithi046 ceph-mon[36898]: pgmap v275: 129 pgs: 129 active+clean; 489 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 253 op/s 2024-05-08T23:12:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:12:46.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 771046 9.85 MB/sec execute 308 sec latency 90.678 ms 2024-05-08T23:12:47.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 773057 9.85 MB/sec execute 309 sec latency 16.297 ms 2024-05-08T23:12:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:47 smithi086 ceph-mon[78610]: pgmap v276: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 231 op/s 2024-05-08T23:12:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:47 smithi053 ceph-mon[77587]: pgmap v276: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 231 op/s 2024-05-08T23:12:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:47 smithi046 ceph-mon[36898]: pgmap v276: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 231 op/s 2024-05-08T23:12:48.901 INFO:tasks.workunit.client.0.smithi046.stdout: 1 774650 9.85 MB/sec execute 310 sec latency 14.132 ms 2024-05-08T23:12:49.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 775736 9.84 MB/sec execute 311 sec latency 24.877 ms 2024-05-08T23:12:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:49 smithi086 ceph-mon[78610]: pgmap v277: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-05-08T23:12:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:49 smithi053 ceph-mon[77587]: pgmap v277: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-05-08T23:12:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:49 smithi046 ceph-mon[36898]: pgmap v277: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-05-08T23:12:50.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 777822 9.85 MB/sec execute 312 sec latency 17.582 ms 2024-05-08T23:12:51.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 779294 9.84 MB/sec execute 313 sec latency 24.254 ms 2024-05-08T23:12:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:51 smithi086 ceph-mon[78610]: pgmap v278: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 320 op/s 2024-05-08T23:12:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:51 smithi053 ceph-mon[77587]: pgmap v278: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 320 op/s 2024-05-08T23:12:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:51 smithi046 ceph-mon[36898]: pgmap v278: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 320 op/s 2024-05-08T23:12:52.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 781431 9.85 MB/sec execute 314 sec latency 14.723 ms 2024-05-08T23:12:53.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 783033 9.84 MB/sec execute 315 sec latency 21.134 ms 2024-05-08T23:12:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:53 smithi086 ceph-mon[78610]: pgmap v279: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-05-08T23:12:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:53 smithi053 ceph-mon[77587]: pgmap v279: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-05-08T23:12:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:53 smithi046 ceph-mon[36898]: pgmap v279: 129 pgs: 129 active+clean; 486 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-05-08T23:12:54.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 784925 9.85 MB/sec execute 316 sec latency 90.039 ms 2024-05-08T23:12:55.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 786303 9.83 MB/sec execute 317 sec latency 67.136 ms 2024-05-08T23:12:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:55 smithi086 ceph-mon[78610]: pgmap v280: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 299 op/s 2024-05-08T23:12:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:55 smithi053 ceph-mon[77587]: pgmap v280: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 299 op/s 2024-05-08T23:12:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:55 smithi046 ceph-mon[36898]: pgmap v280: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 299 op/s 2024-05-08T23:12:56.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 788450 9.84 MB/sec execute 318 sec latency 15.452 ms 2024-05-08T23:12:57.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 789878 9.83 MB/sec execute 319 sec latency 24.102 ms 2024-05-08T23:12:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:57 smithi086 ceph-mon[78610]: pgmap v281: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 194 op/s 2024-05-08T23:12:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:57 smithi053 ceph-mon[77587]: pgmap v281: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 194 op/s 2024-05-08T23:12:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:57 smithi046 ceph-mon[36898]: pgmap v281: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 194 op/s 2024-05-08T23:12:58.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 791976 9.84 MB/sec execute 320 sec latency 15.600 ms 2024-05-08T23:12:59.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:12:58 smithi053 ceph-mon[77587]: pgmap v282: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-05-08T23:12:59.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:12:58 smithi046 ceph-mon[36898]: pgmap v282: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-05-08T23:12:59.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:12:58 smithi086 ceph-mon[78610]: pgmap v282: 129 pgs: 129 active+clean; 491 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-05-08T23:12:59.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 793291 9.83 MB/sec execute 321 sec latency 30.142 ms 2024-05-08T23:13:00.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 795405 9.84 MB/sec execute 322 sec latency 16.260 ms 2024-05-08T23:13:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:01 smithi086 ceph-mon[78610]: pgmap v283: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-08T23:13:01.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:01 smithi053 ceph-mon[77587]: pgmap v283: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-08T23:13:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:01 smithi046 ceph-mon[36898]: pgmap v283: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-08T23:13:01.902 INFO:tasks.workunit.client.0.smithi046.stdout: 1 795659 9.81 MB/sec execute 323 sec latency 63.512 ms 2024-05-08T23:13:02.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 797148 9.80 MB/sec execute 324 sec latency 29.432 ms 2024-05-08T23:13:03.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:13:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:03 smithi086 ceph-mon[78610]: pgmap v284: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:13:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:03 smithi053 ceph-mon[77587]: pgmap v284: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:13:03.770 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:03 smithi046 ceph-mon[36898]: pgmap v284: 129 pgs: 129 active+clean; 495 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-08T23:13:03.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 799157 9.81 MB/sec execute 325 sec latency 14.618 ms 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 -- 172.21.15.46:0/326015450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101a30 msgr2=0x7f23d8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 --2- 172.21.15.46:0/326015450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101a30 0x7f23d8105a70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f23c40098f0 tx=0x7f23c4009c00 comp rx=0 tx=0).stop 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 -- 172.21.15.46:0/326015450 shutdown_connections 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 --2- 172.21.15.46:0/326015450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 0x7f23d810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 --2- 172.21.15.46:0/326015450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8101080 0x7f23d8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 --2- 172.21.15.46:0/326015450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101a30 0x7f23d8105a70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 -- 172.21.15.46:0/326015450 >> 172.21.15.46:0/326015450 conn(0x7f23d80fc8e0 msgr2=0x7f23d80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 -- 172.21.15.46:0/326015450 shutdown_connections 2024-05-08T23:13:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.958+0000 7f23df000700 1 -- 172.21.15.46:0/326015450 wait complete. 2024-05-08T23:13:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.959+0000 7f23df000700 1 Processor -- start 2024-05-08T23:13:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.959+0000 7f23df000700 1 -- start start 2024-05-08T23:13:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.959+0000 7f23df000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.959+0000 7f23df000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 0x7f23d8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23df000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 0x7f23d819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23df000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d819e8b0 con 0x7f23d81061a0 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23df000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d819ea20 con 0x7f23d8101080 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23df000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d819ebe0 con 0x7f23d819de30 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56110/0 (socket says 172.21.15.46:56110) 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23d7e00700 1 -- 172.21.15.46:0/1249310483 learned_addr learned my addr 172.21.15.46:0/1249310483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 0x7f23d819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23d7400700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 0x7f23d8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 -- 172.21.15.46:0/1249310483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 msgr2=0x7f23d8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 -- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 msgr2=0x7f23d8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 0x7f23d8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.960+0000 7f23dd000700 1 -- 172.21.15.46:0/1249310483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80095f0 con 0x7f23d819de30 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23dd000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 0x7f23d819e2b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f23d0007f10 tx=0x7f23d000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23d4c00700 1 -- 172.21.15.46:0/1249310483 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0009ea0 con 0x7f23d819de30 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4009520 con 0x7f23d819de30 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23d4c00700 1 -- 172.21.15.46:0/1249310483 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d000f040 con 0x7f23d819de30 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23d4c00700 1 -- 172.21.15.46:0/1249310483 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0004350 con 0x7f23d819de30 2024-05-08T23:13:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.961+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81a64e0 con 0x7f23d819de30 2024-05-08T23:13:03.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.965+0000 7f23d4c00700 1 -- 172.21.15.46:0/1249310483 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f23d000db20 con 0x7f23d819de30 2024-05-08T23:13:03.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.965+0000 7f23f2ac1b80 1 Processor -- start 2024-05-08T23:13:03.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 -- start start 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca7720 0x555ff8ca1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8ca1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8bf8150 con 0x555ff8ca7720 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8bf5b10 con 0x555ff8ca1890 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8bf1400 con 0x555ff8ca6e00 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23d7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23d7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55164/0 (socket says 172.21.15.46:55164) 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23d7400700 1 -- 172.21.15.46:0/313741631 learned_addr learned my addr 172.21.15.46:0/313741631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23d7e00700 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca7720 0x555ff8ca1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.966+0000 7f23d7400700 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8ca1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 -- 172.21.15.46:0/313741631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 msgr2=0x555ff8ca1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8ca1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 -- 172.21.15.46:0/313741631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca7720 msgr2=0x555ff8ca1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca7720 0x555ff8ca1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 -- 172.21.15.46:0/313741631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff8a723b0 con 0x555ff8ca6e00 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d4c00700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 0x7f23bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7400700 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009810 tx=0x7f23c4019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd6d0eb0d024d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d4c00700 1 -- 172.21.15.46:0/1249310483 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f23d00cc9c0 con 0x7f23d819de30 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 0x7f23bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23cd600700 1 -- 172.21.15.46:0/313741631 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c400f510 con 0x555ff8ca6e00 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23cd600700 1 -- 172.21.15.46:0/313741631 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c400ba20 con 0x555ff8ca6e00 2024-05-08T23:13:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.967+0000 7f23cd600700 1 -- 172.21.15.46:0/313741631 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40053b0 con 0x555ff8ca6e00 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/313741631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 msgr2=0x555ff8ca71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 0x7f23bc0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x555ff8c9dbd0 tx=0x7f23c8009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009810 tx=0x7f23c4019040 comp rx=0 tx=0).stop 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/313741631 shutdown_connections 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8ca1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8ca71e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/313741631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca7720 0x555ff8ca1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/313741631 >> 172.21.15.46:0/313741631 conn(0x555ff8bff210 msgr2=0x555ff8c9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.968+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/313741631 shutdown_connections 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.969+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/313741631 wait complete. 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.969+0000 7f23f2ac1b80 1 -- start start 2024-05-08T23:13:03.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.969+0000 7f23f2ac1b80 1 Processor -- start 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8caaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8caafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8b8d560 con 0x555ff8ca6e00 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8b8d200 con 0x555ff8ca1890 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8a746a0 con 0x555ff8ca7720 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23f2ac1b80 10 client.0 _ll_register_callbacks cb 0x555ff8b96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8caaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8caaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56122/0 (socket says 172.21.15.46:56122) 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23d7e00700 1 -- 172.21.15.46:0/906549277 learned_addr learned my addr 172.21.15.46:0/906549277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.970+0000 7f23d7400700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8caafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 -- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 msgr2=0x555ff8caaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8caaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 -- 172.21.15.46:0/906549277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 msgr2=0x555ff8caafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8caafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 -- 172.21.15.46:0/906549277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c401f040 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23dd000700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cab4f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f23d00c9ca0 tx=0x7f23d0099260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de712a603569160e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23c2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00b90d0 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f23d009d490 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ff8b4a270 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d00a1ae0 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23f2ac1b80 10 client.28147 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.971+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ff8a1eb60 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ff8b4a270 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23d00840c0 con 0x555ff8ca7720 2024-05-08T23:13:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 10 client.28147 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 10 client.28147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 20 client.28147 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8cbb270 0x555ff8cbd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 4 client.28147 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ff8c70810 con 0x555ff8cbb270 2024-05-08T23:13:03.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.972+0000 7f23d7e00700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8cbb270 0x555ff8cbd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.973+0000 7f23d7e00700 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8cbb270 0x555ff8cbd690 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f688d8794a6e9c7 server_cookie=45533b292156628f in_seq=0 out_seq=0 2024-05-08T23:13:03.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.973+0000 7f23c2000700 10 client.28147 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:03.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23c2000700 1 -- 172.21.15.46:0/906549277 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ff8c70810 con 0x555ff8cbb270 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23c2000700 10 client.28147 handle_command_reply: tid=0 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 client.28147 shutdown 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 msgr2=0x555ff8cab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cab4f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f23d00c9ca0 tx=0x7f23d0099260 comp rx=0 tx=0).stop 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 shutdown_connections 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cab4f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8caaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8cbb270 0x555ff8cbd690 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/906549277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8caafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 >> 172.21.15.46:0/906549277 conn(0x555ff8bff210 msgr2=0x555ff8c9b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23c2000700 0 client.28147 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 shutdown_connections 2024-05-08T23:13:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.974+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/906549277 wait complete. 2024-05-08T23:13:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.975+0000 7f23f2ac1b80 20 client.28147 trim_cache size 0 max 16384 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.995+0000 7f23f2ac1b80 1 Processor -- start 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.995+0000 7f23f2ac1b80 1 -- start start 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.995+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca1890 0x555ff8ca1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.995+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.995+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 0x555ff8ca6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8c9c9c0 con 0x555ff8ca1890 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8cb8920 con 0x555ff8ca7720 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8cad600 con 0x555ff8ca6e00 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55178/0 (socket says 172.21.15.46:55178) 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 -- 172.21.15.46:0/47349406 learned_addr learned my addr 172.21.15.46:0/47349406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23dd000700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 0x555ff8ca6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7e00700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca1890 0x555ff8ca1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 -- 172.21.15.46:0/47349406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 msgr2=0x555ff8ca6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 0x555ff8ca6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 -- 172.21.15.46:0/47349406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca1890 msgr2=0x555ff8ca1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca1890 0x555ff8ca1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23d7400700 1 -- 172.21.15.46:0/47349406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff8ca1f40 con 0x555ff8ca6e00 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.996+0000 7f23dd000700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 0x555ff8ca6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.997+0000 7f23d7400700 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009810 tx=0x7f23c4009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3c820a760bb35f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.997+0000 7f23ce000700 1 -- 172.21.15.46:0/47349406 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c402d070 con 0x555ff8ca6e00 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.997+0000 7f23ce000700 1 -- 172.21.15.46:0/47349406 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c400bb90 con 0x555ff8ca6e00 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.997+0000 7f23ce000700 1 -- 172.21.15.46:0/47349406 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4005070 con 0x555ff8ca6e00 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/47349406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 msgr2=0x555ff8c9bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009810 tx=0x7f23c4009400 comp rx=0 tx=0).stop 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/47349406 shutdown_connections 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca7720 0x555ff8ca6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca1890 0x555ff8ca1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/47349406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca6e00 0x555ff8c9bd20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/47349406 >> 172.21.15.46:0/47349406 conn(0x555ff8bff210 msgr2=0x555ff8ca5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/47349406 shutdown_connections 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/47349406 wait complete. 2024-05-08T23:13:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.998+0000 7f23f2ac1b80 1 -- start start 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 Processor -- start 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8cb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cb1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8cb21a0 con 0x555ff8ca6e00 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8cb2310 con 0x555ff8ca1890 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff8cb24d0 con 0x555ff8ca7720 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23f2ac1b80 10 client.0 _ll_register_callbacks cb 0x555ff8b96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56136/0 (socket says 172.21.15.46:56136) 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23dd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cb1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23d7e00700 1 -- 172.21.15.46:0/1332127537 learned_addr learned my addr 172.21.15.46:0/1332127537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:03.999+0000 7f23d7400700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8cb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 -- 172.21.15.46:0/1332127537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 msgr2=0x555ff8cb1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cb1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 -- 172.21.15.46:0/1332127537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 msgr2=0x555ff8cb0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8cb0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 -- 172.21.15.46:0/1332127537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ff8a1eb60 con 0x555ff8ca1890 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23dd000700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cb1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23d7e00700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x555ff8cbdf40 tx=0x7f23c8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3290d7d33c5054ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23c2000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8025040 con 0x555ff8ca1890 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.000+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23c400ba20 con 0x555ff8ca1890 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f23c802a460 con 0x555ff8ca1890 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c8017070 con 0x555ff8ca1890 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 10 client.28076 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ff8a723b0 con 0x555ff8ca1890 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c400ba20 con 0x555ff8ca1890 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23c801f420 con 0x555ff8ca1890 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 10 client.28076 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 10 client.28076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 20 client.28076 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.001+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8d12b20 0x555ff8d14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23f2ac1b80 4 client.28076 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ff8be9ec0 con 0x555ff8d12b20 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23d7400700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8d12b20 0x555ff8d14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23d7400700 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8d12b20 0x555ff8d14f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55db5c4991a3a869 server_cookie=4f6cf26d358d3293 in_seq=0 out_seq=0 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23c2000700 10 client.28076 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23c2000700 1 -- 172.21.15.46:0/1332127537 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555ff8be9ec0 con 0x555ff8d12b20 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.002+0000 7f23c2000700 10 client.28076 handle_command_reply: tid=0 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 client.28076 shutdown 2024-05-08T23:13:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 msgr2=0x555ff8cb08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x555ff8cbdf40 tx=0x7f23c8017000 comp rx=0 tx=0).stop 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 shutdown_connections 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ff8ca7720 0x555ff8cb1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ff8ca1890 0x555ff8cb08d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555ff8d12b20 0x555ff8d14f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 --2- 172.21.15.46:0/1332127537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ff8ca6e00 0x555ff8cb0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23c2000700 0 client.28076 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 >> 172.21.15.46:0/1332127537 conn(0x555ff8bff210 msgr2=0x555ff8cacba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 shutdown_connections 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 1 -- 172.21.15.46:0/1332127537 wait complete. 2024-05-08T23:13:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.003+0000 7f23f2ac1b80 20 client.28076 trim_cache size 0 max 16384 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 msgr2=0x7f23bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 0x7f23bc0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x555ff8c9dbd0 tx=0x7f23c8009310 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 msgr2=0x7f23d819e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 0x7f23d819e2b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f23d0007f10 tx=0x7f23d000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 shutdown_connections 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d819de30 0x7f23d819e2b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23d8101080 0x7f23d8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23bc0a1290 0x7f23bc0a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 --2- 172.21.15.46:0/1249310483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d81061a0 0x7f23d8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 >> 172.21.15.46:0/1249310483 conn(0x7f23d80fc8e0 msgr2=0x7f23d80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 shutdown_connections 2024-05-08T23:13:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.004+0000 7f23df000700 1 -- 172.21.15.46:0/1249310483 wait complete. 2024-05-08T23:13:04.025 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:13:04.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:13:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 -- 172.21.15.46:0/2832107185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6240fff60 msgr2=0x7fd624100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 --2- 172.21.15.46:0/2832107185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6240fff60 0x7fd624100340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd61800dd20 tx=0x7fd61800f330 comp rx=0 tx=0).stop 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 -- 172.21.15.46:0/2832107185 shutdown_connections 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 --2- 172.21.15.46:0/2832107185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd624105080 0x7fd62410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 --2- 172.21.15.46:0/2832107185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6240fff60 0x7fd624100340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 --2- 172.21.15.46:0/2832107185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd624100910 0x7fd624104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.257+0000 7fd62bc00700 1 -- 172.21.15.46:0/2832107185 >> 172.21.15.46:0/2832107185 conn(0x7fd6240fb820 msgr2=0x7fd6240fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.258+0000 7fd62bc00700 1 -- 172.21.15.46:0/2832107185 shutdown_connections 2024-05-08T23:13:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.258+0000 7fd62bc00700 1 -- 172.21.15.46:0/2832107185 wait complete. 2024-05-08T23:13:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 Processor -- start 2024-05-08T23:13:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 -- start start 2024-05-08T23:13:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240fff60 0x7fd624072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 0x7fd624072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 0x7fd62406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62406c3e0 con 0x7fd6240fff60 2024-05-08T23:13:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62406c550 con 0x7fd624105080 2024-05-08T23:13:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.260+0000 7fd62bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240701e0 con 0x7fd62406b790 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.261+0000 7fd623e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 0x7fd624072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.261+0000 7fd623e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 0x7fd624072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56164/0 (socket says 172.21.15.46:56164) 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.261+0000 7fd623e00700 1 -- 172.21.15.46:0/3653159664 learned_addr learned my addr 172.21.15.46:0/3653159664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.261+0000 7fd629c00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 0x7fd62406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.261+0000 7fd629200700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240fff60 0x7fd624072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 -- 172.21.15.46:0/3653159664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 msgr2=0x7fd624072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 0x7fd624072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 -- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240fff60 msgr2=0x7fd624072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240fff60 0x7fd624072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 -- 172.21.15.46:0/3653159664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61800da00 con 0x7fd62406b790 2024-05-08T23:13:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd629c00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 0x7fd62406bc10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00efc0 tx=0x7fd61c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c018070 con 0x7fd62406b790 2024-05-08T23:13:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd61c00f040 con 0x7fd62406b790 2024-05-08T23:13:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00cdf0 con 0x7fd62406b790 2024-05-08T23:13:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.262+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd624070460 con 0x7fd62406b790 2024-05-08T23:13:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.263+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624070920 con 0x7fd62406b790 2024-05-08T23:13:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.264+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624108260 con 0x7fd62406b790 2024-05-08T23:13:04.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.268+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd61c003a00 con 0x7fd62406b790 2024-05-08T23:13:04.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.270+0000 7fd621600700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 0x7fd60c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.271+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd61c0ccdd0 con 0x7fd62406b790 2024-05-08T23:13:04.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.271+0000 7fd629200700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 0x7fd60c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.271+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd61c0fad60 con 0x7fd62406b790 2024-05-08T23:13:04.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.272+0000 7fd629200700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 0x7fd60c0a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd618006010 tx=0x7fd618005040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.611+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62406cc70 con 0x7fd62406b790 2024-05-08T23:13:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.612+0000 7fd621600700 1 -- 172.21.15.46:0/3653159664 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fd61c095e40 con 0x7fd62406b790 2024-05-08T23:13:04.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:13:04.613 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:13:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 msgr2=0x7fd60c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 0x7fd60c0a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd618006010 tx=0x7fd618005040 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 msgr2=0x7fd62406bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 0x7fd62406bc10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00efc0 tx=0x7fd61c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 shutdown_connections 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd62406b790 0x7fd62406bc10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd60c0a1290 0x7fd60c0a3750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240fff60 0x7fd624072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 --2- 172.21.15.46:0/3653159664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624105080 0x7fd624072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 >> 172.21.15.46:0/3653159664 conn(0x7fd6240fb820 msgr2=0x7fd6241041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 shutdown_connections 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.615+0000 7fd62bc00700 1 -- 172.21.15.46:0/3653159664 wait complete. 2024-05-08T23:13:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:13:04.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:04 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:04.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:13:04.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:04 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:04.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:04 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 -- 172.21.15.46:0/813278565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 msgr2=0x7f92f4103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 --2- 172.21.15.46:0/813278565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4103660 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009990 tx=0x7f92e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 -- 172.21.15.46:0/813278565 shutdown_connections 2024-05-08T23:13:04.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 --2- 172.21.15.46:0/813278565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f41083a0 0x7f92f410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 --2- 172.21.15.46:0/813278565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4103660 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 --2- 172.21.15.46:0/813278565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f4103c30 0x7f92f4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.865+0000 7f92fb400700 1 -- 172.21.15.46:0/813278565 >> 172.21.15.46:0/813278565 conn(0x7f92f40fea80 msgr2=0x7f92f4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.866+0000 7f92fb400700 1 -- 172.21.15.46:0/813278565 shutdown_connections 2024-05-08T23:13:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.866+0000 7f92fb400700 1 -- 172.21.15.46:0/813278565 wait complete. 2024-05-08T23:13:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 Processor -- start 2024-05-08T23:13:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 -- start start 2024-05-08T23:13:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f41083a0 0x7f92f4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.867+0000 7f92fb400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f419e8d0 con 0x7f92f4103280 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92fb400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f419ea40 con 0x7f92f41083a0 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92fb400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f419ec00 con 0x7f92f419de50 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55240/0 (socket says 172.21.15.46:55240) 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 -- 172.21.15.46:0/3452095727 learned_addr learned my addr 172.21.15.46:0/3452095727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f41083a0 0x7f92f4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 -- 172.21.15.46:0/3452095727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f41083a0 msgr2=0x7f92f4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f41083a0 0x7f92f4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 -- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 msgr2=0x7f92f4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f9400700 1 -- 172.21.15.46:0/3452095727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80095f0 con 0x7f92f419de50 2024-05-08T23:13:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.868+0000 7f92f8a00700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92f9400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00ef30 tx=0x7f92ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92f1600700 1 -- 172.21.15.46:0/3452095727 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec018070 con 0x7f92f419de50 2024-05-08T23:13:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f41a6100 con 0x7f92f419de50 2024-05-08T23:13:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92f1600700 1 -- 172.21.15.46:0/3452095727 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ec00f040 con 0x7f92f419de50 2024-05-08T23:13:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92f1600700 1 -- 172.21.15.46:0/3452095727 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00cd00 con 0x7f92f419de50 2024-05-08T23:13:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.869+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f41a65e0 con 0x7f92f419de50 2024-05-08T23:13:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.874+0000 7f92f1600700 1 -- 172.21.15.46:0/3452095727 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f92ec003a00 con 0x7f92f419de50 2024-05-08T23:13:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 Processor -- start 2024-05-08T23:13:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 -- start start 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fd8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fd8890 0x5639d9fd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f92f1600700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 0x7f92dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9f2f150 con 0x5639d9fde720 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9f2cb10 con 0x5639d9fdde00 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f930ef04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9f28400 con 0x5639d9fd8890 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.875+0000 7f92f1600700 1 -- 172.21.15.46:0/3452095727 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f92ec0ccbd0 con 0x7f92f419de50 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f9400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 0x7f92dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fd8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56208/0 (socket says 172.21.15.46:56208) 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fd8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43536/0 (socket says 172.21.15.46:43536) 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 -- 172.21.15.46:0/247707415 learned_addr learned my addr 172.21.15.46:0/247707415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fd8890 0x5639d9fd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 -- 172.21.15.46:0/247707415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fd8890 msgr2=0x5639d9fd8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fd8890 0x5639d9fd8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 -- 172.21.15.46:0/247707415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 msgr2=0x5639d9fd8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fd8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 -- 172.21.15.46:0/247707415 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d9da93b0 con 0x5639d9fdde00 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f3e00700 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92e00095f0 tx=0x7f92e00058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b182b893343f56c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.876+0000 7f92f9400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 0x7f92dc0a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0c9f50 tx=0x7f92ec009ed0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.877+0000 7f92e5c00700 1 -- 172.21.15.46:0/247707415 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0004030 con 0x5639d9fdde00 2024-05-08T23:13:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.877+0000 7f92e5c00700 1 -- 172.21.15.46:0/247707415 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e0004830 con 0x5639d9fdde00 2024-05-08T23:13:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.877+0000 7f92e5c00700 1 -- 172.21.15.46:0/247707415 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e00052d0 con 0x5639d9fdde00 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.877+0000 7f930ef04b80 1 -- 172.21.15.46:0/247707415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 msgr2=0x5639d9fde1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.877+0000 7f930ef04b80 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92e00095f0 tx=0x7f92e00058e0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 -- 172.21.15.46:0/247707415 shutdown_connections 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fd8890 0x5639d9fd8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fdde00 0x5639d9fde1e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 --2- 172.21.15.46:0/247707415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fd8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 -- 172.21.15.46:0/247707415 >> 172.21.15.46:0/247707415 conn(0x5639d9f36210 msgr2=0x5639d9fd2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 -- 172.21.15.46:0/247707415 shutdown_connections 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 -- 172.21.15.46:0/247707415 wait complete. 2024-05-08T23:13:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 -- start start 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.878+0000 7f930ef04b80 1 Processor -- start 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fe4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fe2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9ec4560 con 0x5639d9fde720 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9ec4200 con 0x5639d9fd8890 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9dab6a0 con 0x5639d9fdde00 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f930ef04b80 10 client.0 _ll_register_callbacks cb 0x5639d9ecdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55250/0 (socket says 172.21.15.46:55250) 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 -- 172.21.15.46:0/3457369578 learned_addr learned my addr 172.21.15.46:0/3457369578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fe4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fe2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 -- 172.21.15.46:0/3457369578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 msgr2=0x5639d9fe4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fe4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 -- 172.21.15.46:0/3457369578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 msgr2=0x5639d9fe2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fe2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f8a00700 1 -- 172.21.15.46:0/3457369578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e0019040 con 0x5639d9fdde00 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.879+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fe2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f92f8a00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009990 tx=0x7f92e8005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db30148ef66e8fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f92da000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e801d070 con 0x5639d9fdde00 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f92e8004910 con 0x5639d9fdde00 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639d9e81270 con 0x5639d9fdde00 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e8004ad0 con 0x5639d9fdde00 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f930ef04b80 10 client.28171 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.880+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5639d9d55b60 con 0x5639d9fdde00 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639d9e81270 con 0x5639d9fdde00 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92e800fb90 con 0x5639d9fdde00 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 10 client.28171 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 10 client.28171 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 20 client.28171 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639d9ff2290 0x5639d9ff46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 4 client.28171 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639d9fa7810 con 0x5639d9ff2290 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.881+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639d9ff2290 0x5639d9ff46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.882+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639d9ff2290 0x5639d9ff46b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65e5af067cf69c25 server_cookie=3b20b27852c80048 in_seq=0 out_seq=0 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.882+0000 7f92da000700 10 client.28171 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.882+0000 7f92da000700 1 -- 172.21.15.46:0/3457369578 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639d9fa7810 con 0x5639d9ff2290 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.882+0000 7f92da000700 10 client.28171 handle_command_reply: tid=0 2024-05-08T23:13:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.882+0000 7f930ef04b80 1 client.28171 shutdown 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 msgr2=0x5639d9fe1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009990 tx=0x7f92e8005320 comp rx=0 tx=0).stop 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 shutdown_connections 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639d9ff2290 0x5639d9ff46b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fde720 0x5639d9fe2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fe4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3457369578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fe1a30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f92da000700 0 client.28171 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 >> 172.21.15.46:0/3457369578 conn(0x5639d9f36210 msgr2=0x5639d9fd25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 shutdown_connections 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 1 -- 172.21.15.46:0/3457369578 wait complete. 2024-05-08T23:13:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.883+0000 7f930ef04b80 20 client.28171 trim_cache size 0 max 16384 2024-05-08T23:13:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.897+0000 7f930ef04b80 1 Processor -- start 2024-05-08T23:13:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 -- start start 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 0x5639d9ff4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fde720 0x5639d9fd2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9fcd800 con 0x5639d9fdde00 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9fe29a0 con 0x5639d9fd8890 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f930ef04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9ff9860 con 0x5639d9fde720 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 0x5639d9ff4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56216/0 (socket says 172.21.15.46:56216) 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f3e00700 1 -- 172.21.15.46:0/3431498983 learned_addr learned my addr 172.21.15.46:0/3431498983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 0x5639d9ff4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43556/0 (socket says 172.21.15.46:43556) 2024-05-08T23:13:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.898+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fde720 0x5639d9fd2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 -- 172.21.15.46:0/3431498983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fde720 msgr2=0x5639d9fd2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fde720 0x5639d9fd2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 -- 172.21.15.46:0/3431498983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 msgr2=0x5639d9ff4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 0x5639d9ff4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 -- 172.21.15.46:0/3431498983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d9eee560 con 0x5639d9fd8890 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92f3e00700 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009340 tx=0x7f92e0021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7025f9f3d9e49cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92e6600700 1 -- 172.21.15.46:0/3431498983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0024040 con 0x5639d9fd8890 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92e6600700 1 -- 172.21.15.46:0/3431498983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e0035030 con 0x5639d9fd8890 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.899+0000 7f92e6600700 1 -- 172.21.15.46:0/3431498983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0005540 con 0x5639d9fd8890 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 -- 172.21.15.46:0/3431498983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 msgr2=0x5639d9fd8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009340 tx=0x7f92e0021040 comp rx=0 tx=0).stop 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 -- 172.21.15.46:0/3431498983 shutdown_connections 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fde720 0x5639d9fd2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fd8890 0x5639d9fd8350 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 --2- 172.21.15.46:0/3431498983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fdde00 0x5639d9ff4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 -- 172.21.15.46:0/3431498983 >> 172.21.15.46:0/3431498983 conn(0x5639d9f36210 msgr2=0x5639d9fd8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 -- 172.21.15.46:0/3431498983 shutdown_connections 2024-05-08T23:13:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.900+0000 7f930ef04b80 1 -- 172.21.15.46:0/3431498983 wait complete. 2024-05-08T23:13:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.901+0000 7f930ef04b80 1 -- start start 2024-05-08T23:13:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.901+0000 7f930ef04b80 1 Processor -- start 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 0x5639d9fe9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fde720 0x5639d9fda020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9fda730 con 0x5639d9fd8890 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9fda8a0 con 0x5639d9fde720 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d9fdaa60 con 0x5639d9fdde00 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f930ef04b80 10 client.0 _ll_register_callbacks cb 0x5639d9ecdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 0x5639d9fe9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55272/0 (socket says 172.21.15.46:55272) 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 -- 172.21.15.46:0/2899071368 learned_addr learned my addr 172.21.15.46:0/2899071368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f3e00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fde720 0x5639d9fda020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 800557 9.80 MB/sec execute 326 sec latency 17.862 ms 2024-05-08T23:13:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 -- 172.21.15.46:0/2899071368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fde720 msgr2=0x5639d9fda020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fde720 0x5639d9fda020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 -- 172.21.15.46:0/2899071368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 msgr2=0x5639d9fe9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 0x5639d9fe9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f8a00700 1 -- 172.21.15.46:0/2899071368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639d9da93b0 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.902+0000 7f92f3e00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 0x5639d9fe9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f92f8a00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b690 tx=0x7f92e8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86bd4ac1cffaf62f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f92da000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e801d070 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92e0004830 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f92e8024640 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e8024a90 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f930ef04b80 10 client.28177 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.903+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5639d9dadcd0 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e0004830 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92e80047a0 con 0x5639d9fdde00 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 10 client.28177 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 10 client.28177 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:13:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 20 client.28177 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639da048bc0 0x5639da04afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 4 client.28177 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639d9f20ec0 con 0x5639da048bc0 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.904+0000 7f92f3e00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639da048bc0 0x5639da04afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f92f3e00700 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639da048bc0 0x5639da04afe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f08e6c96eac07189 server_cookie=a828c96afadeb39c in_seq=0 out_seq=0 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f92da000700 10 client.28177 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f92da000700 1 -- 172.21.15.46:0/2899071368 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639d9f20ec0 con 0x5639da048bc0 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f92da000700 10 client.28177 handle_command_reply: tid=0 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 client.28177 shutdown 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 msgr2=0x5639d9fd9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b690 tx=0x7f92e8005ef0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 shutdown_connections 2024-05-08T23:13:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639da048bc0 0x5639da04afe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639d9fde720 0x5639d9fda020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639d9fd8890 0x5639d9fe9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f92da000700 0 client.28177 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 --2- 172.21.15.46:0/2899071368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639d9fdde00 0x5639d9fd9900 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 >> 172.21.15.46:0/2899071368 conn(0x5639d9f36210 msgr2=0x5639d9fdbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 shutdown_connections 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.905+0000 7f930ef04b80 1 -- 172.21.15.46:0/2899071368 wait complete. 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f930ef04b80 20 client.28177 trim_cache size 0 max 16384 2024-05-08T23:13:04.911 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 msgr2=0x7f92dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 0x7f92dc0a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0c9f50 tx=0x7f92ec009ed0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 msgr2=0x7f92f419e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00ef30 tx=0x7f92ec00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 shutdown_connections 2024-05-08T23:13:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f419de50 0x7f92f419e2d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f92dc0a1290 0x7f92dc0a3750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f4103280 0x7f92f4199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 --2- 172.21.15.46:0/3452095727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f41083a0 0x7f92f4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.906+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 >> 172.21.15.46:0/3452095727 conn(0x7f92f40fea80 msgr2=0x7f92f40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.907+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 shutdown_connections 2024-05-08T23:13:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:04.907+0000 7f92fb400700 1 -- 172.21.15.46:0/3452095727 wait complete. 2024-05-08T23:13:04.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 -- 172.21.15.46:0/799357558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6928101a10 msgr2=0x7f6928105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 --2- 172.21.15.46:0/799357558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6928101a10 0x7f6928105a50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6914009990 tx=0x7f6914009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 -- 172.21.15.46:0/799357558 shutdown_connections 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 --2- 172.21.15.46:0/799357558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 0x7f692810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 --2- 172.21.15.46:0/799357558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 0x7f6928101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 --2- 172.21.15.46:0/799357558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6928101a10 0x7f6928105a50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 -- 172.21.15.46:0/799357558 >> 172.21.15.46:0/799357558 conn(0x7f69280fc900 msgr2=0x7f69280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 -- 172.21.15.46:0/799357558 shutdown_connections 2024-05-08T23:13:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.151+0000 7f692ea00700 1 -- 172.21.15.46:0/799357558 wait complete. 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 Processor -- start 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 -- start start 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 0x7f69281973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 0x7f6928197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819c6d0 con 0x7f6928106180 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819c840 con 0x7f6928101060 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819ca00 con 0x7f692819bc50 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f6927e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 0x7f69281973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55296/0 (socket says 172.21.15.46:55296) 2024-05-08T23:13:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f6927400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 0x7f6928197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.153+0000 7f692ca00700 1 -- 172.21.15.46:0/1252306061 learned_addr learned my addr 172.21.15.46:0/1252306061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 -- 172.21.15.46:0/1252306061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 msgr2=0x7f69281973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 0x7f69281973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 -- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 msgr2=0x7f6928197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 0x7f6928197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 -- 172.21.15.46:0/1252306061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140095f0 con 0x7f692819bc50 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ca00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f692000ef30 tx=0x7f692000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f6924c00700 1 -- 172.21.15.46:0/1252306061 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920018070 con 0x7f692819bc50 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f6924c00700 1 -- 172.21.15.46:0/1252306061 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692000f040 con 0x7f692819bc50 2024-05-08T23:13:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69281a3f00 con 0x7f692819bc50 2024-05-08T23:13:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.154+0000 7f6924c00700 1 -- 172.21.15.46:0/1252306061 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000cd00 con 0x7f692819bc50 2024-05-08T23:13:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.155+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69281a4430 con 0x7f692819bc50 2024-05-08T23:13:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.158+0000 7f6924c00700 1 -- 172.21.15.46:0/1252306061 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6920003a00 con 0x7f692819bc50 2024-05-08T23:13:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.159+0000 7f69425deb80 1 Processor -- start 2024-05-08T23:13:05.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.159+0000 7f69425deb80 1 -- start start 2024-05-08T23:13:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653ae991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653ae93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653adea150 con 0x55653ae93890 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ade7b10 con 0x55653ae98e00 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f69425deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ade3400 con 0x55653ae99720 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55302/0 (socket says 172.21.15.46:55302) 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 -- 172.21.15.46:0/1271463069 learned_addr learned my addr 172.21.15.46:0/1271463069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927400700 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653ae991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927400700 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653ae93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 -- 172.21.15.46:0/1271463069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 msgr2=0x55653ae991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6924c00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 0x7f690c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653ae991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 -- 172.21.15.46:0/1271463069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 msgr2=0x55653ae93cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653ae93cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.160+0000 7f6927e00700 1 -- 172.21.15.46:0/1271463069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653ac643b0 con 0x55653ae99720 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f6924c00700 1 -- 172.21.15.46:0/1252306061 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f69200ccb90 con 0x7f692819bc50 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f6927e00700 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6918009db0 tx=0x7f6918011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3cf08b34b4ed1f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f692ca00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 0x7f690c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f691d200700 1 -- 172.21.15.46:0/1271463069 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918019630 con 0x55653ae99720 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f691d200700 1 -- 172.21.15.46:0/1271463069 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918019c50 con 0x55653ae99720 2024-05-08T23:13:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.161+0000 7f691d200700 1 -- 172.21.15.46:0/1271463069 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800ec50 con 0x55653ae99720 2024-05-08T23:13:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 -- 172.21.15.46:0/1271463069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 msgr2=0x55653ae93350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f692ca00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 0x7f690c0a37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f69200c9ed0 tx=0x7f6920009ed0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6918009db0 tx=0x7f6918011720 comp rx=0 tx=0).stop 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 -- 172.21.15.46:0/1271463069 shutdown_connections 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653ae93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653ae991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 --2- 172.21.15.46:0/1271463069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653ae93350 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 -- 172.21.15.46:0/1271463069 >> 172.21.15.46:0/1271463069 conn(0x55653adf1210 msgr2=0x55653ae8ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 -- 172.21.15.46:0/1271463069 shutdown_connections 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.162+0000 7f69425deb80 1 -- 172.21.15.46:0/1271463069 wait complete. 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 -- start start 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 Processor -- start 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653aea7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653aea14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ad7f560 con 0x55653ae93890 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ad7f200 con 0x55653ae98e00 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.163+0000 7f69425deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ac666a0 con 0x55653ae99720 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f69425deb80 10 client.0 _ll_register_callbacks cb 0x55653ad88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43586/0 (socket says 172.21.15.46:43586) 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 -- 172.21.15.46:0/1383011255 learned_addr learned my addr 172.21.15.46:0/1383011255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927e00700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653aea7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653aea14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 -- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 msgr2=0x55653aea14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653aea14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 -- 172.21.15.46:0/1383011255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 msgr2=0x55653aea7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653aea7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.164+0000 7f6927400700 1 -- 172.21.15.46:0/1383011255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69180099e0 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6927400700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f691400b690 tx=0x7f6914009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d874aed647eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6911600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914004710 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6914004870 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55653ad3c270 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6914004a30 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f69425deb80 10 client.18105 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55653ac10b60 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.165+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55653ad3c270 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f691400fb30 con 0x55653ae93890 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 10 client.18105 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 10 client.18105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:13:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 20 client.18105 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653aeb1620 0x55653aeb3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 4 client.18105 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55653ae62810 con 0x55653aeb1620 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.166+0000 7f6927e00700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653aeb1620 0x55653aeb3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f6927e00700 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653aeb1620 0x55653aeb3a40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=caf0acd552065539 server_cookie=106dc246ea24d007 in_seq=0 out_seq=0 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f6911600700 10 client.18105 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f6911600700 1 -- 172.21.15.46:0/1383011255 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55653ae62810 con 0x55653aeb1620 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f6911600700 10 client.18105 handle_command_reply: tid=0 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f69425deb80 1 client.18105 shutdown 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 msgr2=0x55653aea7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.167+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f691400b690 tx=0x7f6914009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 shutdown_connections 2024-05-08T23:13:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653aeb1620 0x55653aeb3a40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae99720 0x55653aea14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae93890 0x55653aea7130 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 --2- 172.21.15.46:0/1383011255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae98e00 0x55653aea7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 >> 172.21.15.46:0/1383011255 conn(0x55653adf1210 msgr2=0x55653ae8d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f6911600700 0 client.18105 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 shutdown_connections 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 1 -- 172.21.15.46:0/1383011255 wait complete. 2024-05-08T23:13:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.168+0000 7f69425deb80 20 client.18105 trim_cache size 0 max 16384 2024-05-08T23:13:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 Processor -- start 2024-05-08T23:13:05.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 -- start start 2024-05-08T23:13:05.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae93890 0x55653ae93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae99720 0x55653ae9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ae8eb30 con 0x55653ae99720 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ae8f070 con 0x55653ae93890 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f69425deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ae88800 con 0x55653ae98e00 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f6927400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae93890 0x55653ae93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f6927e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f6927400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae99720 0x55653ae9eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f6927e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55310/0 (socket says 172.21.15.46:55310) 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.183+0000 7f6927e00700 1 -- 172.21.15.46:0/518281530 learned_addr learned my addr 172.21.15.46:0/518281530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 -- 172.21.15.46:0/518281530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae93890 msgr2=0x55653ae93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae93890 0x55653ae93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 -- 172.21.15.46:0/518281530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae99720 msgr2=0x55653ae9eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae99720 0x55653ae9eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 -- 172.21.15.46:0/518281530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653ada9560 con 0x55653ae98e00 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f6927e00700 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6918009db0 tx=0x7f691801aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c432463e70eebb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f691dc00700 1 -- 172.21.15.46:0/518281530 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918019630 con 0x55653ae98e00 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f691dc00700 1 -- 172.21.15.46:0/518281530 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918009260 con 0x55653ae98e00 2024-05-08T23:13:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.184+0000 7f691dc00700 1 -- 172.21.15.46:0/518281530 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918004970 con 0x55653ae98e00 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 -- 172.21.15.46:0/518281530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 msgr2=0x55653ae952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6918009db0 tx=0x7f691801aef0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 -- 172.21.15.46:0/518281530 shutdown_connections 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae99720 0x55653ae9eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae93890 0x55653ae93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 --2- 172.21.15.46:0/518281530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae98e00 0x55653ae952b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 -- 172.21.15.46:0/518281530 >> 172.21.15.46:0/518281530 conn(0x55653adf1210 msgr2=0x55653aea0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 -- 172.21.15.46:0/518281530 shutdown_connections 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.185+0000 7f69425deb80 1 -- 172.21.15.46:0/518281530 wait complete. 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 -- start start 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 Processor -- start 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae98e00 0x55653aea98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae99720 0x55653aea9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653af06c40 con 0x55653ae98e00 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653af06db0 con 0x55653ae99720 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653af06f70 con 0x55653ae93890 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f69425deb80 10 client.0 _ll_register_callbacks cb 0x55653ad88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.186+0000 7f6927400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55318/0 (socket says 172.21.15.46:55318) 2024-05-08T23:13:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 -- 172.21.15.46:0/1337586097 learned_addr learned my addr 172.21.15.46:0/1337586097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927e00700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae98e00 0x55653aea98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae99720 0x55653aea9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 -- 172.21.15.46:0/1337586097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae99720 msgr2=0x55653aea9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae99720 0x55653aea9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 -- 172.21.15.46:0/1337586097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae98e00 msgr2=0x55653aea98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae98e00 0x55653aea98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 -- 172.21.15.46:0/1337586097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653ac643b0 con 0x55653ae93890 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6927400700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f691400b800 tx=0x7f691402c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be7f574c3aa3e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6911600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914026e60 con 0x55653ae93890 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6914000e40 con 0x55653ae93890 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.187+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6918019c50 con 0x55653ae93890 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6914026380 con 0x55653ae93890 2024-05-08T23:13:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 10 client.28195 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55653ac68cd0 con 0x55653ae93890 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918019c50 con 0x55653ae93890 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6914048060 con 0x55653ae93890 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 10 client.28195 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 10 client.28195 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 20 client.28195 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653af08270 0x55653af0a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 4 client.28195 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55653addbec0 con 0x55653af08270 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.188+0000 7f6927e00700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653af08270 0x55653af0a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f6927e00700 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653af08270 0x55653af0a660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7533385716cedea server_cookie=c40df590d0b546dc in_seq=0 out_seq=0 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f6911600700 10 client.28195 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f6911600700 1 -- 172.21.15.46:0/1337586097 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55653addbec0 con 0x55653af08270 2024-05-08T23:13:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f6911600700 10 client.28195 handle_command_reply: tid=0 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 client.28195 shutdown 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 msgr2=0x55653aea9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f691400b800 tx=0x7f691402c040 comp rx=0 tx=0).stop 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 shutdown_connections 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55653af08270 0x55653af0a660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653ae99720 0x55653aea9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653ae93890 0x55653aea9380 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f6911600700 0 client.28195 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 --2- 172.21.15.46:0/1337586097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55653ae98e00 0x55653aea98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.189+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 >> 172.21.15.46:0/1337586097 conn(0x55653adf1210 msgr2=0x55653ae952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 shutdown_connections 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f69425deb80 1 -- 172.21.15.46:0/1337586097 wait complete. 2024-05-08T23:13:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f69425deb80 20 client.28195 trim_cache size 0 max 16384 2024-05-08T23:13:05.196 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 msgr2=0x7f690c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 0x7f690c0a37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f69200c9ed0 tx=0x7f6920009ed0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 msgr2=0x7f692819c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.190+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f692000ef30 tx=0x7f692000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 shutdown_connections 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692819bc50 0x7f692819c0d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6928101060 0x7f69281973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f690c0a1310 0x7f690c0a37d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 --2- 172.21.15.46:0/1252306061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6928106180 0x7f6928197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 >> 172.21.15.46:0/1252306061 conn(0x7f69280fc900 msgr2=0x7f69280fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 shutdown_connections 2024-05-08T23:13:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.191+0000 7f692ea00700 1 -- 172.21.15.46:0/1252306061 wait complete. 2024-05-08T23:13:05.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 -- 172.21.15.46:0/4142567705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 msgr2=0x7f6e98103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 --2- 172.21.15.46:0/4142567705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e98103690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009990 tx=0x7f6e8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 -- 172.21.15.46:0/4142567705 shutdown_connections 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 --2- 172.21.15.46:0/4142567705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e981083d0 0x7f6e9810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 --2- 172.21.15.46:0/4142567705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e98103690 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 --2- 172.21.15.46:0/4142567705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e98103c60 0x7f6e98107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.456+0000 7f6ea0600700 1 -- 172.21.15.46:0/4142567705 >> 172.21.15.46:0/4142567705 conn(0x7f6e980feb10 msgr2=0x7f6e98100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.457+0000 7f6ea0600700 1 -- 172.21.15.46:0/4142567705 shutdown_connections 2024-05-08T23:13:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.457+0000 7f6ea0600700 1 -- 172.21.15.46:0/4142567705 wait complete. 2024-05-08T23:13:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.458+0000 7f6ea0600700 1 Processor -- start 2024-05-08T23:13:05.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.458+0000 7f6ea0600700 1 -- start start 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.458+0000 7f6ea0600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.458+0000 7f6ea0600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e981083d0 0x7f6e980759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6ea0600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e98075f30 0x7f6e981a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6ea0600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980796b0 con 0x7f6e98075f30 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6ea0600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981aa040 con 0x7f6e981083d0 2024-05-08T23:13:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6ea0600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981aa200 con 0x7f6e981032b0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55332/0 (socket says 172.21.15.46:55332) 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 -- 172.21.15.46:0/2697989641 learned_addr learned my addr 172.21.15.46:0/2697989641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9e600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e98075f30 0x7f6e981a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e981083d0 0x7f6e980759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 -- 172.21.15.46:0/2697989641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e981083d0 msgr2=0x7f6e980759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e981083d0 0x7f6e980759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 -- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e98075f30 msgr2=0x7f6e981a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e98075f30 0x7f6e981a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.459+0000 7f6e9dc00700 1 -- 172.21.15.46:0/2697989641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c0095f0 con 0x7f6e981032b0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005770 tx=0x7f6e8c0057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6e96000700 1 -- 172.21.15.46:0/2697989641 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c01d070 con 0x7f6e981032b0 2024-05-08T23:13:05.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6e96000700 1 -- 172.21.15.46:0/2697989641 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e8c022470 con 0x7f6e981032b0 2024-05-08T23:13:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e981aa480 con 0x7f6e981032b0 2024-05-08T23:13:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6e96000700 1 -- 172.21.15.46:0/2697989641 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c017650 con 0x7f6e981032b0 2024-05-08T23:13:05.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.460+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e981aa900 con 0x7f6e981032b0 2024-05-08T23:13:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.463+0000 7f6e96000700 1 -- 172.21.15.46:0/2697989641 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6e8c00fb00 con 0x7f6e981032b0 2024-05-08T23:13:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.465+0000 7f6e96000700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 0x7f6e800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.465+0000 7f6e96000700 1 -- 172.21.15.46:0/2697989641 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6e8c0ce360 con 0x7f6e981032b0 2024-05-08T23:13:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.465+0000 7f6eb4193b80 1 Processor -- start 2024-05-08T23:13:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.465+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 0x7f6e800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.465+0000 7f6eb4193b80 1 -- start start 2024-05-08T23:13:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9c68150 con 0x5652b9d11890 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9c65b10 con 0x5652b9d16e00 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9c61400 con 0x5652b9d17720 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 0x7f6e800a37d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e98076be0 tx=0x7f6e84009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55340/0 (socket says 172.21.15.46:55340) 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9d200700 1 -- 172.21.15.46:0/187259262 learned_addr learned my addr 172.21.15.46:0/187259262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.466+0000 7f6e9e600700 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 -- 172.21.15.46:0/187259262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 msgr2=0x5652b9d171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 -- 172.21.15.46:0/187259262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 msgr2=0x5652b9d11cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 -- 172.21.15.46:0/187259262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b9ae23b0 con 0x5652b9d17720 2024-05-08T23:13:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e9d200700 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6e84011d20 tx=0x7f6e84014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57186294277fdc5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e8aa00700 1 -- 172.21.15.46:0/187259262 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84003880 con 0x5652b9d17720 2024-05-08T23:13:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e8aa00700 1 -- 172.21.15.46:0/187259262 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e84011e50 con 0x5652b9d17720 2024-05-08T23:13:05.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.467+0000 7f6e8aa00700 1 -- 172.21.15.46:0/187259262 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8401edb0 con 0x5652b9d17720 2024-05-08T23:13:05.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 -- 172.21.15.46:0/187259262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 msgr2=0x5652b9d11350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6e84011d20 tx=0x7f6e84014000 comp rx=0 tx=0).stop 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 -- 172.21.15.46:0/187259262 shutdown_connections 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/187259262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d11350 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 -- 172.21.15.46:0/187259262 >> 172.21.15.46:0/187259262 conn(0x5652b9c6f210 msgr2=0x5652b9d0bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 -- 172.21.15.46:0/187259262 shutdown_connections 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.468+0000 7f6eb4193b80 1 -- 172.21.15.46:0/187259262 wait complete. 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.469+0000 7f6eb4193b80 1 -- start start 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.469+0000 7f6eb4193b80 1 Processor -- start 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d1aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9bfd560 con 0x5652b9d11890 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9bfd200 con 0x5652b9d17720 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9ae46a0 con 0x5652b9d16e00 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6eb4193b80 10 client.0 _ll_register_callbacks cb 0x5652b9c06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55342/0 (socket says 172.21.15.46:55342) 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 -- 172.21.15.46:0/2522530837 learned_addr learned my addr 172.21.15.46:0/2522530837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d1aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9e600700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 -- 172.21.15.46:0/2522530837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 msgr2=0x5652b9d1b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d1b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 -- 172.21.15.46:0/2522530837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 msgr2=0x5652b9d1aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d1aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.470+0000 7f6e9d200700 1 -- 172.21.15.46:0/2522530837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e84017070 con 0x5652b9d16e00 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6e9d200700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6e84011cf0 tx=0x7f6e8401ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79a2e20582f74f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6e7f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8402a040 con 0x5652b9d16e00 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652b9bba270 con 0x5652b9d16e00 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6e84003a90 con 0x5652b9d16e00 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e84003c50 con 0x5652b9d16e00 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6eb4193b80 10 client.28207 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.471+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5652b9a8eb60 con 0x5652b9d16e00 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652b9bba270 con 0x5652b9d16e00 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6e84024020 con 0x5652b9d16e00 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 10 client.28207 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 10 client.28207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 20 client.28207 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d2b080 0x5652b9d2d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 4 client.28207 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652b9ce0810 con 0x5652b9d2b080 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.472+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d2b080 0x5652b9d2d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.473+0000 7f6e7f400700 10 client.28207 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.473+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d2b080 0x5652b9d2d4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef5eeeb2c59a606a server_cookie=cd51aa38a3b980ed in_seq=0 out_seq=0 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.473+0000 7f6e7f400700 1 -- 172.21.15.46:0/2522530837 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652b9ce0810 con 0x5652b9d2b080 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.473+0000 7f6e7f400700 10 client.28207 handle_command_reply: tid=0 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.473+0000 7f6eb4193b80 1 client.28207 shutdown 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 msgr2=0x5652b9d1b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6e84011cf0 tx=0x7f6e8401ed40 comp rx=0 tx=0).stop 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 shutdown_connections 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d1b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d2b080 0x5652b9d2d4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d1aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/2522530837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d1b030 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 >> 172.21.15.46:0/2522530837 conn(0x5652b9c6f210 msgr2=0x5652b9d0b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6e7f400700 0 client.28207 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 shutdown_connections 2024-05-08T23:13:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 1 -- 172.21.15.46:0/2522530837 wait complete. 2024-05-08T23:13:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.474+0000 7f6eb4193b80 20 client.28207 trim_cache size 0 max 16384 2024-05-08T23:13:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.489+0000 7f6eb4193b80 1 Processor -- start 2024-05-08T23:13:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.489+0000 7f6eb4193b80 1 -- start start 2024-05-08T23:13:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d06800 con 0x5652b9d11890 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d0cac0 con 0x5652b9d17720 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d19040 con 0x5652b9d16e00 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6e9e600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6e9d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6e9dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6e9e600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56308/0 (socket says 172.21.15.46:56308) 2024-05-08T23:13:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.490+0000 7f6e9e600700 1 -- 172.21.15.46:0/1703919335 learned_addr learned my addr 172.21.15.46:0/1703919335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 -- 172.21.15.46:0/1703919335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 msgr2=0x5652b9d162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 -- 172.21.15.46:0/1703919335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 msgr2=0x5652b9d11350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 -- 172.21.15.46:0/1703919335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b9d11f40 con 0x5652b9d16e00 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e9d200700 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d0bd20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e84004600 tx=0x7f6e84009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d571d760f4494b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e8b400700 1 -- 172.21.15.46:0/1703919335 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8402a040 con 0x5652b9d16e00 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e8b400700 1 -- 172.21.15.46:0/1703919335 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e8401edb0 con 0x5652b9d16e00 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.491+0000 7f6e8b400700 1 -- 172.21.15.46:0/1703919335 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84003890 con 0x5652b9d16e00 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1703919335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 msgr2=0x5652b9d0bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d0bd20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e84004600 tx=0x7f6e84009990 comp rx=0 tx=0).stop 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1703919335 shutdown_connections 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d17720 0x5652b9d162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d11350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1703919335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d16e00 0x5652b9d0bd20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1703919335 >> 172.21.15.46:0/1703919335 conn(0x5652b9c6f210 msgr2=0x5652b9d15bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1703919335 shutdown_connections 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.492+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1703919335 wait complete. 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 -- start start 2024-05-08T23:13:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 Processor -- start 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d20340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d20880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d21010 con 0x5652b9d11890 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d80ad0 con 0x5652b9d16e00 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b9d80c90 con 0x5652b9d17720 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.493+0000 7f6eb4193b80 10 client.0 _ll_register_callbacks cb 0x5652b9c06b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d20340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55352/0 (socket says 172.21.15.46:55352) 2024-05-08T23:13:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 -- 172.21.15.46:0/1729504343 learned_addr learned my addr 172.21.15.46:0/1729504343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9d200700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d20880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 -- 172.21.15.46:0/1729504343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 msgr2=0x5652b9d20880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d20880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 -- 172.21.15.46:0/1729504343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 msgr2=0x5652b9d20340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d20340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 -- 172.21.15.46:0/1729504343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652b9a8eb60 con 0x5652b9d17720 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9d200700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d20880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d20340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e9e600700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6e90007f00 tx=0x7f6e90007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e8b32fa6f66b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.494+0000 7f6e7f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000e040 con 0x5652b9d17720 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652b9c59ec0 con 0x5652b9d17720 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6e90004bb0 con 0x5652b9d17720 2024-05-08T23:13:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e90004d70 con 0x5652b9d17720 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6eb4193b80 10 client.28213 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5652b9c27560 con 0x5652b9d17720 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652b9c59ec0 con 0x5652b9d17720 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6e900045c0 con 0x5652b9d17720 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.495+0000 7f6eb4193b80 10 client.28213 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6eb4193b80 10 client.28213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6eb4193b80 20 client.28213 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d824d0 0x5652b9d848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6eb4193b80 4 client.28213 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652b9aafcf0 con 0x5652b9d824d0 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d824d0 0x5652b9d848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6e9dc00700 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d824d0 0x5652b9d848f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e42e29c194c07d8 server_cookie=45b4ffd1c53b7528 in_seq=0 out_seq=0 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.496+0000 7f6e7f400700 10 client.28213 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6e7f400700 1 -- 172.21.15.46:0/1729504343 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652b9aafcf0 con 0x5652b9d824d0 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6e7f400700 10 client.28213 handle_command_reply: tid=0 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 client.28213 shutdown 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 msgr2=0x5652b9d80560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6e90007f00 tx=0x7f6e90007f30 comp rx=0 tx=0).stop 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 shutdown_connections 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652b9d17720 0x5652b9d80560 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5652b9d824d0 0x5652b9d848f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652b9d11890 0x5652b9d20340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 --2- 172.21.15.46:0/1729504343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652b9d16e00 0x5652b9d20880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6e7f400700 0 client.28213 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 >> 172.21.15.46:0/1729504343 conn(0x5652b9c6f210 msgr2=0x5652b9d193c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.497+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 shutdown_connections 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6eb4193b80 1 -- 172.21.15.46:0/1729504343 wait complete. 2024-05-08T23:13:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6eb4193b80 20 client.28213 trim_cache size 0 max 16384 2024-05-08T23:13:05.502 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 msgr2=0x7f6e800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 0x7f6e800a37d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e98076be0 tx=0x7f6e84009260 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 msgr2=0x7f6e980754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005770 tx=0x7f6e8c0057a0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 shutdown_connections 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6e800a1310 0x7f6e800a37d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e98075f30 0x7f6e981a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e981032b0 0x7f6e980754b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 --2- 172.21.15.46:0/2697989641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e981083d0 0x7f6e980759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.498+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 >> 172.21.15.46:0/2697989641 conn(0x7f6e980feb10 msgr2=0x7f6e980ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.499+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 shutdown_connections 2024-05-08T23:13:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.499+0000 7f6ea0600700 1 -- 172.21.15.46:0/2697989641 wait complete. 2024-05-08T23:13:05.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:05 smithi046 ceph-mon[36898]: pgmap v285: 129 pgs: 129 active+clean; 498 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:13:05.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:05 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3653159664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:05.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:05 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:05.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:05 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:05.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:05 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:13:05.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:13:05.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:05 smithi086 ceph-mon[78610]: pgmap v285: 129 pgs: 129 active+clean; 498 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:13:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:05 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3653159664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:05 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:05 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:05 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 -- 172.21.15.46:0/555390826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 msgr2=0x7f176410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 --2- 172.21.15.46:0/555390826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176410db70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b0a0 tx=0x7f175c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 -- 172.21.15.46:0/555390826 shutdown_connections 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 --2- 172.21.15.46:0/555390826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176410db70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 --2- 172.21.15.46:0/555390826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f17641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 --2- 172.21.15.46:0/555390826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1764101880 0x7f17641058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 -- 172.21.15.46:0/555390826 >> 172.21.15.46:0/555390826 conn(0x7f17640fc770 msgr2=0x7f17640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 -- 172.21.15.46:0/555390826 shutdown_connections 2024-05-08T23:13:05.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.741+0000 7f1768a00700 1 -- 172.21.15.46:0/555390826 wait complete. 2024-05-08T23:13:05.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.742+0000 7f1768a00700 1 Processor -- start 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.742+0000 7f1768a00700 1 -- start start 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17641992a0 0x7f1764199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764199d50 con 0x7f1764100ed0 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764199ec0 con 0x7f17641992a0 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1768a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419a080 con 0x7f1764105ff0 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1761e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1761e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55372/0 (socket says 172.21.15.46:55372) 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1761e00700 1 -- 172.21.15.46:0/956534596 learned_addr learned my addr 172.21.15.46:0/956534596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43676/0 (socket says 172.21.15.46:43676) 2024-05-08T23:13:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1763200700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17641992a0 0x7f1764199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.743+0000 7f1761e00700 1 -- 172.21.15.46:0/956534596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17641992a0 msgr2=0x7f1764199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1761e00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17641992a0 0x7f1764199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1761e00700 1 -- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 msgr2=0x7f176419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1761e00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1761e00700 1 -- 172.21.15.46:0/956534596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009d00 con 0x7f1764105ff0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1762800700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1761e00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f175000e9f0 tx=0x7f175000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f175b400700 1 -- 172.21.15.46:0/956534596 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750009750 con 0x7f1764105ff0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f175b400700 1 -- 172.21.15.46:0/956534596 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175000c7c0 con 0x7f1764105ff0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17641a5f90 con 0x7f1764105ff0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f175b400700 1 -- 172.21.15.46:0/956534596 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500054d0 con 0x7f1764105ff0 2024-05-08T23:13:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.744+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17641a63b0 con 0x7f1764105ff0 2024-05-08T23:13:05.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.747+0000 7f175b400700 1 -- 172.21.15.46:0/956534596 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f175000c930 con 0x7f1764105ff0 2024-05-08T23:13:05.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f175b400700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 0x7f17480a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f177cf4fb80 1 Processor -- start 2024-05-08T23:13:05.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f175b400700 1 -- 172.21.15.46:0/956534596 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1750014070 con 0x7f1764105ff0 2024-05-08T23:13:05.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f1762800700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 0x7f17480a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f177cf4fb80 1 -- start start 2024-05-08T23:13:05.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.749+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a685bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6860f0 0x56094a67fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a5ce150 con 0x56094a6860f0 2024-05-08T23:13:05.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a5cbb10 con 0x56094a6800a0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a5c7400 con 0x56094a6857d0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56336/0 (socket says 172.21.15.46:56336) 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 -- 172.21.15.46:0/3465192846 learned_addr learned my addr 172.21.15.46:0/3465192846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1761e00700 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6860f0 0x56094a67fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1762800700 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a685bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1762800700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 0x7f17480a3870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f176419aa50 tx=0x7f1754009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 -- 172.21.15.46:0/3465192846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 msgr2=0x56094a685bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a685bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 -- 172.21.15.46:0/3465192846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6860f0 msgr2=0x56094a67fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6860f0 0x56094a67fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.750+0000 7f1763200700 1 -- 172.21.15.46:0/3465192846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56094a4483b0 con 0x56094a6800a0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.751+0000 7f1763200700 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b0a0 tx=0x7f175c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=995fabd64a98d85f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.751+0000 7f174f400700 1 -- 172.21.15.46:0/3465192846 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e040 con 0x56094a6800a0 2024-05-08T23:13:05.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.751+0000 7f174f400700 1 -- 172.21.15.46:0/3465192846 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c0094c0 con 0x56094a6800a0 2024-05-08T23:13:05.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.751+0000 7f174f400700 1 -- 172.21.15.46:0/3465192846 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c003c40 con 0x56094a6800a0 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3465192846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 msgr2=0x56094a6804e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b0a0 tx=0x7f175c009c60 comp rx=0 tx=0).stop 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3465192846 shutdown_connections 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6800a0 0x56094a6804e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a685bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3465192846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6860f0 0x56094a67fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3465192846 >> 172.21.15.46:0/3465192846 conn(0x56094a5d5210 msgr2=0x56094a6722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3465192846 shutdown_connections 2024-05-08T23:13:05.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.752+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3465192846 wait complete. 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 -- start start 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 Processor -- start 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6857d0 0x56094a689360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a563560 con 0x56094a6857d0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a563200 con 0x56094a6860f0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a44a6a0 con 0x56094a6800a0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f177cf4fb80 10 client.0 _ll_register_callbacks cb 0x56094a56cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.753+0000 7f1762800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55380/0 (socket says 172.21.15.46:55380) 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 -- 172.21.15.46:0/1198501208 learned_addr learned my addr 172.21.15.46:0/1198501208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1761e00700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6857d0 0x56094a689360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1763200700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 -- 172.21.15.46:0/1198501208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 msgr2=0x56094a6899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 -- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6857d0 msgr2=0x56094a689360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6857d0 0x56094a689360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 -- 172.21.15.46:0/1198501208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176410e3e0 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1762800700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1754003a00 tx=0x7f17540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e164528f7d448f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.754+0000 7f1743e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175401f8a0 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f175401fa00 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56094a646810 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175401fbc0 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 10 client.28225 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56094a3f4b60 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56094a646810 con 0x56094a6800a0 2024-05-08T23:13:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f175400e3b0 con 0x56094a6800a0 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 10 client.28225 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 10 client.28225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.755+0000 7f177cf4fb80 20 client.28225 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a699b00 0x56094a69bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f177cf4fb80 4 client.28225 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56094a5bfec0 con 0x56094a699b00 2024-05-08T23:13:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f1761e00700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a699b00 0x56094a69bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f1761e00700 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a699b00 0x56094a69bf20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f397ceb298aae319 server_cookie=8c547c894c287616 in_seq=0 out_seq=0 2024-05-08T23:13:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.756+0000 7f1743e00700 10 client.28225 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f1743e00700 1 -- 172.21.15.46:0/1198501208 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56094a5bfec0 con 0x56094a699b00 2024-05-08T23:13:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f1743e00700 10 client.28225 handle_command_reply: tid=0 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 client.28225 shutdown 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 msgr2=0x56094a68c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1754003a00 tx=0x7f17540047c0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 shutdown_connections 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6800a0 0x56094a68c070 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a699b00 0x56094a69bf20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/1198501208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6857d0 0x56094a689360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 >> 172.21.15.46:0/1198501208 conn(0x56094a5d5210 msgr2=0x56094a6720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f1743e00700 0 client.28225 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 shutdown_connections 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.757+0000 7f177cf4fb80 1 -- 172.21.15.46:0/1198501208 wait complete. 2024-05-08T23:13:05.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.758+0000 7f177cf4fb80 20 client.28225 trim_cache size 0 max 16384 2024-05-08T23:13:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:05 smithi053 ceph-mon[77587]: pgmap v285: 129 pgs: 129 active+clean; 498 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-05-08T23:13:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:05 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3653159664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:05 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:05 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:05 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:13:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.772+0000 7f177cf4fb80 1 Processor -- start 2024-05-08T23:13:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.772+0000 7f177cf4fb80 1 -- start start 2024-05-08T23:13:05.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 0x56094a690af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 0x56094a691030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a672850 con 0x56094a6800a0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a68a930 con 0x56094a6857d0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a66a040 con 0x56094a6860f0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43692/0 (socket says 172.21.15.46:43692) 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1762800700 1 -- 172.21.15.46:0/3482206241 learned_addr learned my addr 172.21.15.46:0/3482206241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 0x56094a691030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1761e00700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 0x56094a690af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 -- 172.21.15.46:0/3482206241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 msgr2=0x56094a690af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 0x56094a690af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 -- 172.21.15.46:0/3482206241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 msgr2=0x56094a6a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.773+0000 7f1763200700 1 -- 172.21.15.46:0/3482206241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56094a58d560 con 0x56094a6860f0 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f1762800700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f1761e00700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 0x56094a690af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f1763200700 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 0x56094a691030 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f175c0062a0 tx=0x7f175c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b4abbf5820751a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f174fe00700 1 -- 172.21.15.46:0/3482206241 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e070 con 0x56094a6860f0 2024-05-08T23:13:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f174fe00700 1 -- 172.21.15.46:0/3482206241 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c029070 con 0x56094a6860f0 2024-05-08T23:13:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.774+0000 7f174fe00700 1 -- 172.21.15.46:0/3482206241 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c003c60 con 0x56094a6860f0 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3482206241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 msgr2=0x56094a691030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 0x56094a691030 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f175c0062a0 tx=0x7f175c000f40 comp rx=0 tx=0).stop 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3482206241 shutdown_connections 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6860f0 0x56094a691030 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a6a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/3482206241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6857d0 0x56094a690af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3482206241 >> 172.21.15.46:0/3482206241 conn(0x56094a5d5210 msgr2=0x56094a66e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3482206241 shutdown_connections 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- 172.21.15.46:0/3482206241 wait complete. 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.775+0000 7f177cf4fb80 1 -- start start 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 Processor -- start 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a690f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a681ee0 con 0x56094a6800a0 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a682050 con 0x56094a6860f0 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094a682210 con 0x56094a6857d0 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.776+0000 7f177cf4fb80 10 client.0 _ll_register_callbacks cb 0x56094a56cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a690f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1762800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a690f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43700/0 (socket says 172.21.15.46:43700) 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55408/0 (socket says 172.21.15.46:55408) 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1762800700 1 -- 172.21.15.46:0/743632120 learned_addr learned my addr 172.21.15.46:0/743632120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1763200700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 -- 172.21.15.46:0/743632120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 msgr2=0x56094a6817d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6817d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 -- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 msgr2=0x56094a690f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:05.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a690f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.777+0000 7f1761e00700 1 -- 172.21.15.46:0/743632120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175402d040 con 0x56094a6857d0 2024-05-08T23:13:05.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.778+0000 7f1761e00700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f17500ca510 tx=0x7f17500cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c248706b3a7a97b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.778+0000 7f1743e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:05.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.783+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500a2350 con 0x56094a6857d0 2024-05-08T23:13:05.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.783+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f175000cc10 con 0x56094a6857d0 2024-05-08T23:13:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.785+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56094a520270 con 0x56094a6857d0 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.785+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17500b8170 con 0x56094a6857d0 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.785+0000 7f177cf4fb80 10 client.28231 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.785+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56094a4483b0 con 0x56094a6857d0 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56094a520270 con 0x56094a6857d0 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f175009a200 con 0x56094a6857d0 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 10 client.28231 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 10 client.28231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:13:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 20 client.28231 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a6f0850 0x56094a6f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 4 client.28231 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56094a415cf0 con 0x56094a6f0850 2024-05-08T23:13:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.786+0000 7f1762800700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a6f0850 0x56094a6f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.787+0000 7f1762800700 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a6f0850 0x56094a6f2c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fd2ed2c2258c3d9 server_cookie=9b97928689cd9985 in_seq=0 out_seq=0 2024-05-08T23:13:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.787+0000 7f1743e00700 10 client.28231 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.788+0000 7f1743e00700 1 -- 172.21.15.46:0/743632120 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56094a415cf0 con 0x56094a6f0850 2024-05-08T23:13:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.788+0000 7f1743e00700 10 client.28231 handle_command_reply: tid=0 2024-05-08T23:13:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.788+0000 7f177cf4fb80 1 client.28231 shutdown 2024-05-08T23:13:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.789+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 msgr2=0x56094a681230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.789+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f17500ca510 tx=0x7f17500cd040 comp rx=0 tx=0).stop 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 shutdown_connections 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56094a6860f0 0x56094a6817d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x56094a6f0850 0x56094a6f2c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f1743e00700 0 client.28231 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56094a6800a0 0x56094a690f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 --2- 172.21.15.46:0/743632120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56094a6857d0 0x56094a681230 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 >> 172.21.15.46:0/743632120 conn(0x56094a5d5210 msgr2=0x56094a6a0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 shutdown_connections 2024-05-08T23:13:05.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.790+0000 7f177cf4fb80 1 -- 172.21.15.46:0/743632120 wait complete. 2024-05-08T23:13:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.793+0000 7f177cf4fb80 20 client.28231 trim_cache size 0 max 16384 2024-05-08T23:13:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 msgr2=0x7f17480a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 0x7f17480a3870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f176419aa50 tx=0x7f1754009260 comp rx=0 tx=0).stop 2024-05-08T23:13:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 msgr2=0x7f176419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f175000e9f0 tx=0x7f175000ed00 comp rx=0 tx=0).stop 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 shutdown_connections 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17641992a0 0x7f1764199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17480a13b0 0x7f17480a3870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1764100ed0 0x7f176419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 --2- 172.21.15.46:0/956534596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1764105ff0 0x7f176419e500 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.797+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 >> 172.21.15.46:0/956534596 conn(0x7f17640fc770 msgr2=0x7f17640fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.798+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 shutdown_connections 2024-05-08T23:13:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:05.798+0000 7f1768a00700 1 -- 172.21.15.46:0/956534596 wait complete. 2024-05-08T23:13:05.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:13:05.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 802620 9.81 MB/sec execute 327 sec latency 22.087 ms 2024-05-08T23:13:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 -- 172.21.15.46:0/212694255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd800101a30 msgr2=0x7fd800105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 --2- 172.21.15.46:0/212694255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd800101a30 0x7fd800105a70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009990 tx=0x7fd7ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 -- 172.21.15.46:0/212694255 shutdown_connections 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 --2- 172.21.15.46:0/212694255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8001061a0 0x7fd80010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 --2- 172.21.15.46:0/212694255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800101080 0x7fd800101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 --2- 172.21.15.46:0/212694255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd800101a30 0x7fd800105a70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 -- 172.21.15.46:0/212694255 >> 172.21.15.46:0/212694255 conn(0x7fd8000fc8e0 msgr2=0x7fd8000fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 -- 172.21.15.46:0/212694255 shutdown_connections 2024-05-08T23:13:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.087+0000 7fd806000700 1 -- 172.21.15.46:0/212694255 wait complete. 2024-05-08T23:13:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 Processor -- start 2024-05-08T23:13:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 -- start start 2024-05-08T23:13:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 0x7fd800199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 0x7fd80019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80019e890 con 0x7fd8001061a0 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80019ea00 con 0x7fd800101080 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.088+0000 7fd806000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80019ebc0 con 0x7fd80019de10 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 0x7fd80019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 0x7fd800199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 0x7fd80019e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55432/0 (socket says 172.21.15.46:55432) 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43724/0 (socket says 172.21.15.46:43724) 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7ffe00700 1 -- 172.21.15.46:0/2117318534 learned_addr learned my addr 172.21.15.46:0/2117318534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 0x7fd800199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56378/0 (socket says 172.21.15.46:56378) 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 -- 172.21.15.46:0/2117318534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 msgr2=0x7fd80019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 0x7fd80019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 -- 172.21.15.46:0/2117318534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 msgr2=0x7fd800199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 0x7fd800199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 -- 172.21.15.46:0/2117318534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec0095f0 con 0x7fd8001061a0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.089+0000 7fd7fea00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0098e0 tx=0x7fd7ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.090+0000 7fd7f7e00700 1 -- 172.21.15.46:0/2117318534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec01d070 con 0x7fd8001061a0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.090+0000 7fd7f7e00700 1 -- 172.21.15.46:0/2117318534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ec022470 con 0x7fd8001061a0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.090+0000 7fd7f7e00700 1 -- 172.21.15.46:0/2117318534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec017650 con 0x7fd8001061a0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.090+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8001a60c0 con 0x7fd8001061a0 2024-05-08T23:13:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.090+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8001a6490 con 0x7fd8001061a0 2024-05-08T23:13:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.093+0000 7fd7f7e00700 1 -- 172.21.15.46:0/2117318534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd7ec0225e0 con 0x7fd8001061a0 2024-05-08T23:13:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.094+0000 7fd819aafb80 1 Processor -- start 2024-05-08T23:13:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 -- start start 2024-05-08T23:13:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd7f7e00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 0x7fd7e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd7f7e00700 1 -- 172.21.15.46:0/2117318534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fd7ec0ce410 con 0x7fd8001061a0 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd7ffe00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 0x7fd7e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 0x5577ff8e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e2890 0x5577ff8e2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff839150 con 0x5577ff8e8720 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff836b10 con 0x5577ff8e2890 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd819aafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff832400 con 0x5577ff8e7e00 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.095+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55446/0 (socket says 172.21.15.46:55446) 2024-05-08T23:13:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 -- 172.21.15.46:0/990399924 learned_addr learned my addr 172.21.15.46:0/990399924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ffe00700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e2890 0x5577ff8e2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 -- 172.21.15.46:0/990399924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e2890 msgr2=0x5577ff8e2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e2890 0x5577ff8e2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 -- 172.21.15.46:0/990399924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 msgr2=0x5577ff8e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7fea00700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 0x5577ff8e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 0x5577ff8e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 -- 172.21.15.46:0/990399924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ff6b33b0 con 0x5577ff8e7e00 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7fea00700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 0x5577ff8e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ffe00700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 0x7fd7e40a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009c10 tx=0x7fd7f800fd50 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.096+0000 7fd7ff400700 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009d00 tx=0x7fd7f0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5c3a362e731706 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd7ebe00700 1 -- 172.21.15.46:0/990399924 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0004230 con 0x5577ff8e7e00 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd7ebe00700 1 -- 172.21.15.46:0/990399924 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f0009e50 con 0x5577ff8e7e00 2024-05-08T23:13:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd7ebe00700 1 -- 172.21.15.46:0/990399924 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0019800 con 0x5577ff8e7e00 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 -- 172.21.15.46:0/990399924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 msgr2=0x5577ff8e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009d00 tx=0x7fd7f0011720 comp rx=0 tx=0).stop 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 -- 172.21.15.46:0/990399924 shutdown_connections 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e2890 0x5577ff8e2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8e81e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 --2- 172.21.15.46:0/990399924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e8720 0x5577ff8e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.097+0000 7fd819aafb80 1 -- 172.21.15.46:0/990399924 >> 172.21.15.46:0/990399924 conn(0x5577ff840210 msgr2=0x5577ff8dcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.098+0000 7fd819aafb80 1 -- 172.21.15.46:0/990399924 shutdown_connections 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.098+0000 7fd819aafb80 1 -- 172.21.15.46:0/990399924 wait complete. 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.098+0000 7fd819aafb80 1 -- start start 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.098+0000 7fd819aafb80 1 Processor -- start 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff7ce560 con 0x5577ff8e7e00 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff7ce200 con 0x5577ff8e8720 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff6b56a0 con 0x5577ff8e2890 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd819aafb80 10 client.0 _ll_register_callbacks cb 0x5577ff7d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd7fea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55454/0 (socket says 172.21.15.46:55454) 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd7ff400700 1 -- 172.21.15.46:0/830473099 learned_addr learned my addr 172.21.15.46:0/830473099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.099+0000 7fd7ffe00700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 -- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 msgr2=0x5577ff8ec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8ec5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 -- 172.21.15.46:0/830473099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 msgr2=0x5577ff8ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 -- 172.21.15.46:0/830473099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00099e0 con 0x5577ff8e2890 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7fea00700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7ff400700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd7f001b040 tx=0x7fd7f00041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e158102eb3c03239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0005240 con 0x5577ff8e2890 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fd7f0005ce0 con 0x5577ff8e2890 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ff78b270 con 0x5577ff8e2890 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7f0003dd0 con 0x5577ff8e2890 2024-05-08T23:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd819aafb80 10 client.28243 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.100+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5577ff65fb60 con 0x5577ff8e2890 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577ff78b270 con 0x5577ff8e2890 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7f0021040 con 0x5577ff8e2890 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 10 client.28243 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 10 client.28243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 20 client.28243 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff8fc0d0 0x5577ff8fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 4 client.28243 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ff8b1810 con 0x5577ff8fc0d0 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.101+0000 7fd7fea00700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff8fc0d0 0x5577ff8fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.102+0000 7fd7fea00700 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff8fc0d0 0x5577ff8fe4f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fefba9be7f891e server_cookie=6f98c2eaede853a5 in_seq=0 out_seq=0 2024-05-08T23:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.102+0000 7fd7e8c00700 10 client.28243 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd7e8c00700 1 -- 172.21.15.46:0/830473099 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577ff8b1810 con 0x5577ff8fc0d0 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd7e8c00700 10 client.28243 handle_command_reply: tid=0 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 client.28243 shutdown 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 msgr2=0x5577ff8ebb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd7f001b040 tx=0x7fd7f00041a0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 shutdown_connections 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8ec5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8ebb50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff8fc0d0 0x5577ff8fe4f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 --2- 172.21.15.46:0/830473099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd7e8c00700 0 client.28243 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 >> 172.21.15.46:0/830473099 conn(0x5577ff840210 msgr2=0x5577ff8dc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 shutdown_connections 2024-05-08T23:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 1 -- 172.21.15.46:0/830473099 wait complete. 2024-05-08T23:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.103+0000 7fd819aafb80 20 client.28243 trim_cache size 0 max 16384 2024-05-08T23:13:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 Processor -- start 2024-05-08T23:13:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 -- start start 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff903720 con 0x5577ff8e7e00 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff8e6550 con 0x5577ff8e8720 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd819aafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff8dd260 con 0x5577ff8e2890 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55466/0 (socket says 172.21.15.46:55466) 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd7ff400700 1 -- 172.21.15.46:0/2073885554 learned_addr learned my addr 172.21.15.46:0/2073885554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.119+0000 7fd7fea00700 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 -- 172.21.15.46:0/2073885554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 msgr2=0x5577ff8dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 -- 172.21.15.46:0/2073885554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 msgr2=0x5577ff8f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 -- 172.21.15.46:0/2073885554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ff7f8560 con 0x5577ff8e2890 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7ff400700 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0011760 tx=0x7fd7f0005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d284575e92fc542 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7f5200700 1 -- 172.21.15.46:0/2073885554 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0003b60 con 0x5577ff8e2890 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7f5200700 1 -- 172.21.15.46:0/2073885554 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f0004210 con 0x5577ff8e2890 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.120+0000 7fd7f5200700 1 -- 172.21.15.46:0/2073885554 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0023a20 con 0x5577ff8e2890 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 -- 172.21.15.46:0/2073885554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 msgr2=0x5577ff8f82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0011760 tx=0x7fd7f0005a00 comp rx=0 tx=0).stop 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 -- 172.21.15.46:0/2073885554 shutdown_connections 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e2890 0x5577ff8f82f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 --2- 172.21.15.46:0/2073885554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e7e00 0x5577ff8f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 -- 172.21.15.46:0/2073885554 >> 172.21.15.46:0/2073885554 conn(0x5577ff840210 msgr2=0x5577ff8d9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 -- 172.21.15.46:0/2073885554 shutdown_connections 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.121+0000 7fd819aafb80 1 -- 172.21.15.46:0/2073885554 wait complete. 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 -- start start 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 Processor -- start 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 0x5577ff8f3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff8f1950 con 0x5577ff8e2890 2024-05-08T23:13:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff8f1ac0 con 0x5577ff8e8720 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.122+0000 7fd819aafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff8f1c80 con 0x5577ff8e7e00 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd819aafb80 10 client.0 _ll_register_callbacks cb 0x5577ff7d7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 0x5577ff8f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56414/0 (socket says 172.21.15.46:56414) 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7ffe00700 1 -- 172.21.15.46:0/1536061511 learned_addr learned my addr 172.21.15.46:0/1536061511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55472/0 (socket says 172.21.15.46:55472) 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7ff400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 0x5577ff8f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43762/0 (socket says 172.21.15.46:43762) 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 -- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 msgr2=0x5577ff8f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 -- 172.21.15.46:0/1536061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 msgr2=0x5577ff8f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 0x5577ff8f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 -- 172.21.15.46:0/1536061511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577ff65fb60 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7fea00700 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cb6a0 tx=0x7fd7ec0a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e845c398db4923 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.123+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0b2040 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fd7ec09b910 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7f0009e50 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7ec09b210 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 10 client.28249 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5577ff6b7cd0 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ec083030 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7ec0a10b0 con 0x5577ff8e7e00 2024-05-08T23:13:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 10 client.28249 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 10 client.28249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 20 client.28249 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff9525c0 0x5577ff9549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 4 client.28249 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.124+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577ff82aec0 con 0x5577ff9525c0 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd7ff400700 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff9525c0 0x5577ff9549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd7ff400700 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff9525c0 0x5577ff9549e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73552fce6e183bbd server_cookie=378e029bba635161 in_seq=0 out_seq=0 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd7e8c00700 10 client.28249 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd7e8c00700 1 -- 172.21.15.46:0/1536061511 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577ff82aec0 con 0x5577ff9525c0 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd7e8c00700 10 client.28249 handle_command_reply: tid=0 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd819aafb80 1 client.28249 shutdown 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 msgr2=0x5577ff8f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.125+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cb6a0 tx=0x7fd7ec0a1040 comp rx=0 tx=0).stop 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 shutdown_connections 2024-05-08T23:13:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577ff8e8720 0x5577ff8f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5577ff9525c0 0x5577ff9549e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577ff8e2890 0x5577ff8f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 --2- 172.21.15.46:0/1536061511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577ff8e7e00 0x5577ff8f36b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd7e8c00700 0 client.28249 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 >> 172.21.15.46:0/1536061511 conn(0x5577ff840210 msgr2=0x5577ff8ea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 shutdown_connections 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 1 -- 172.21.15.46:0/1536061511 wait complete. 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.126+0000 7fd819aafb80 20 client.28249 trim_cache size 0 max 16384 2024-05-08T23:13:06.130 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 msgr2=0x7fd7e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 0x7fd7e40a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009c10 tx=0x7fd7f800fd50 comp rx=0 tx=0).stop 2024-05-08T23:13:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 msgr2=0x7fd800199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0098e0 tx=0x7fd7ec005740 comp rx=0 tx=0).stop 2024-05-08T23:13:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 shutdown_connections 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80019de10 0x7fd80019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd800101080 0x7fd800199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd7e40a1290 0x7fd7e40a3750 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 --2- 172.21.15.46:0/2117318534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8001061a0 0x7fd800199a70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 >> 172.21.15.46:0/2117318534 conn(0x7fd8000fc8e0 msgr2=0x7fd8000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 shutdown_connections 2024-05-08T23:13:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:06.127+0000 7fd806000700 1 -- 172.21.15.46:0/2117318534 wait complete. 2024-05-08T23:13:06.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 804070 9.79 MB/sec execute 328 sec latency 31.075 ms 2024-05-08T23:13:07.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:13:07.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:13:07.208 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:06 smithi046 ceph-mon[36898]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:13:07.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:06 smithi086 ceph-mon[78610]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:13:07.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:06 smithi053 ceph-mon[77587]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:13:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 -- 172.21.15.46:0/1388366765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c80ffb80 msgr2=0x7ff1c80fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 --2- 172.21.15.46:0/1388366765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c80ffb80 0x7ff1c80fff60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009990 tx=0x7ff1bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 -- 172.21.15.46:0/1388366765 shutdown_connections 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 --2- 172.21.15.46:0/1388366765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c810d730 0x7ff1c810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 --2- 172.21.15.46:0/1388366765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c80ffb80 0x7ff1c80fff60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 --2- 172.21.15.46:0/1388366765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c8100530 0x7ff1c810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 -- 172.21.15.46:0/1388366765 >> 172.21.15.46:0/1388366765 conn(0x7ff1c80fb760 msgr2=0x7ff1c80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 -- 172.21.15.46:0/1388366765 shutdown_connections 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.362+0000 7ff1cf400700 1 -- 172.21.15.46:0/1388366765 wait complete. 2024-05-08T23:13:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.363+0000 7ff1cf400700 1 Processor -- start 2024-05-08T23:13:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 -- start start 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c8100530 0x7ff1c8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c810d730 0x7ff1c819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c819e440 con 0x7ff1c8100530 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c819e5b0 con 0x7ff1c810d730 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cf400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c819e770 con 0x7ff1c80ffb80 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c8100530 0x7ff1c8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55494/0 (socket says 172.21.15.46:55494) 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cca00700 1 -- 172.21.15.46:0/3103498337 learned_addr learned my addr 172.21.15.46:0/3103498337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.364+0000 7ff1cd400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c810d730 0x7ff1c819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 -- 172.21.15.46:0/3103498337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c810d730 msgr2=0x7ff1c819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c810d730 0x7ff1c819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 -- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c8100530 msgr2=0x7ff1c8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c8100530 0x7ff1c8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 -- 172.21.15.46:0/3103498337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0095f0 con 0x7ff1c80ffb80 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cca00700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1c5600700 1 -- 172.21.15.46:0/3103498337 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc01d070 con 0x7ff1c80ffb80 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1c5600700 1 -- 172.21.15.46:0/3103498337 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1bc022470 con 0x7ff1c80ffb80 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c819e9f0 con 0x7ff1c80ffb80 2024-05-08T23:13:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.365+0000 7ff1c5600700 1 -- 172.21.15.46:0/3103498337 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc017650 con 0x7ff1c80ffb80 2024-05-08T23:13:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.366+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c81a6290 con 0x7ff1c80ffb80 2024-05-08T23:13:07.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.369+0000 7ff1c5600700 1 -- 172.21.15.46:0/3103498337 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1bc0225e0 con 0x7ff1c80ffb80 2024-05-08T23:13:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.370+0000 7ff1c5600700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 0x7ff1b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.371+0000 7ff1c5600700 1 -- 172.21.15.46:0/3103498337 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff1bc0ce450 con 0x7ff1c80ffb80 2024-05-08T23:13:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.371+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 0x7ff1b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.371+0000 7ff1e2e5bb80 1 Processor -- start 2024-05-08T23:13:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.371+0000 7ff1e2e5bb80 1 -- start start 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 0x7ff1b00a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1c819aa70 tx=0x7ff1b4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2b07a0 0x557e6b2aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2aa910 0x557e6b2aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2011d0 con 0x557e6b2b07a0 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b1feb90 con 0x557e6b2aa910 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b1fa480 con 0x557e6b2afe80 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55498/0 (socket says 172.21.15.46:55498) 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2b07a0 0x557e6b2aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 -- 172.21.15.46:0/1134424351 learned_addr learned my addr 172.21.15.46:0/1134424351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cd400700 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2aa910 0x557e6b2aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 -- 172.21.15.46:0/1134424351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2aa910 msgr2=0x557e6b2aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2aa910 0x557e6b2aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 -- 172.21.15.46:0/1134424351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2b07a0 msgr2=0x557e6b2aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2b07a0 0x557e6b2aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.372+0000 7ff1cca00700 1 -- 172.21.15.46:0/1134424351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6b07b430 con 0x557e6b2afe80 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.373+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc097690 tx=0x7ff1bc09adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc724d4f96c7366 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.373+0000 7ff1b9c00700 1 -- 172.21.15.46:0/1134424351 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc09b190 con 0x557e6b2afe80 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.373+0000 7ff1b9c00700 1 -- 172.21.15.46:0/1134424351 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1bc0cbe90 con 0x557e6b2afe80 2024-05-08T23:13:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.373+0000 7ff1b9c00700 1 -- 172.21.15.46:0/1134424351 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0228e0 con 0x557e6b2afe80 2024-05-08T23:13:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1134424351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 msgr2=0x557e6b2b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc097690 tx=0x7ff1bc09adc0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1134424351 shutdown_connections 2024-05-08T23:13:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2aa910 0x557e6b2aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b0260 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1134424351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2b07a0 0x557e6b2aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1134424351 >> 172.21.15.46:0/1134424351 conn(0x557e6b208290 msgr2=0x557e6b2a4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1134424351 shutdown_connections 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1134424351 wait complete. 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.374+0000 7ff1e2e5bb80 1 -- start start 2024-05-08T23:13:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.375+0000 7ff1e2e5bb80 1 Processor -- start 2024-05-08T23:13:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b1965e0 con 0x557e6b2aa910 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b196280 con 0x557e6b2b07a0 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b07d720 con 0x557e6b2afe80 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1e2e5bb80 10 client.0 _ll_register_callbacks cb 0x557e6b19fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56464/0 (socket says 172.21.15.46:56464) 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55510/0 (socket says 172.21.15.46:55510) 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 -- 172.21.15.46:0/3575070553 learned_addr learned my addr 172.21.15.46:0/3575070553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 -- 172.21.15.46:0/3575070553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 msgr2=0x557e6b2b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 -- 172.21.15.46:0/3575070553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 msgr2=0x557e6b2b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1cd400700 1 -- 172.21.15.46:0/3575070553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c81a6290 con 0x557e6b2b07a0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.376+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1cd400700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009550 tx=0x7ff1c0007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebdc7ee202ae1766 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1ae000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000e070 con 0x557e6b2b07a0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff1c0003c00 con 0x557e6b2b07a0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e6b1532f0 con 0x557e6b2b07a0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1c0004bb0 con 0x557e6b2b07a0 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1e2e5bb80 10 client.28184 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.377+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e6b027be0 con 0x557e6b2b07a0 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e6b1532f0 con 0x557e6b2b07a0 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1c0014040 con 0x557e6b2b07a0 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 10 client.28184 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 10 client.28184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 20 client.28184 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b2c4010 0x557e6b2c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 4 client.28184 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.378+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e6b279890 con 0x557e6b2c4010 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.379+0000 7ff1cca00700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b2c4010 0x557e6b2c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.379+0000 7ff1cca00700 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b2c4010 0x557e6b2c6430 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53613f94dda6fc32 server_cookie=ba1941f8b813d30 in_seq=0 out_seq=0 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.379+0000 7ff1ae000700 10 client.28184 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.380+0000 7ff1ae000700 1 -- 172.21.15.46:0/3575070553 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e6b279890 con 0x557e6b2c4010 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.380+0000 7ff1ae000700 10 client.28184 handle_command_reply: tid=0 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.380+0000 7ff1e2e5bb80 1 client.28184 shutdown 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 msgr2=0x557e6b2b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009550 tx=0x7ff1c0007e30 comp rx=0 tx=0).stop 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 shutdown_connections 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2b41f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b2c4010 0x557e6b2c6430 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/3575070553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 >> 172.21.15.46:0/3575070553 conn(0x557e6b208290 msgr2=0x557e6b2a4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1ae000700 0 client.28184 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 shutdown_connections 2024-05-08T23:13:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/3575070553 wait complete. 2024-05-08T23:13:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.381+0000 7ff1e2e5bb80 20 client.28184 trim_cache size 0 max 16384 2024-05-08T23:13:07.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.404+0000 7ff1e2e5bb80 1 Processor -- start 2024-05-08T23:13:07.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.404+0000 7ff1e2e5bb80 1 -- start start 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2afe80 0x557e6b2a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 0x557e6b2b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b29fb60 con 0x557e6b2aa910 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2a6f00 con 0x557e6b2afe80 2024-05-08T23:13:07.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2b4e80 con 0x557e6b2b07a0 2024-05-08T23:13:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1cca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43812/0 (socket says 172.21.15.46:43812) 2024-05-08T23:13:07.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1cca00700 1 -- 172.21.15.46:0/1675688703 learned_addr learned my addr 172.21.15.46:0/1675688703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.405+0000 7ff1cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 0x557e6b2b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2afe80 0x557e6b2a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 -- 172.21.15.46:0/1675688703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 msgr2=0x557e6b2b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 0x557e6b2b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 -- 172.21.15.46:0/1675688703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2afe80 msgr2=0x557e6b2a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2afe80 0x557e6b2a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 -- 172.21.15.46:0/1675688703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6b2aafc0 con 0x557e6b2aa910 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cd400700 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 0x557e6b2b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0cb7f0 tx=0x7ff1bc022c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ec00f18eb52aae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1ba600700 1 -- 172.21.15.46:0/1675688703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc09b190 con 0x557e6b2aa910 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1ba600700 1 -- 172.21.15.46:0/1675688703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1bc0a3580 con 0x557e6b2aa910 2024-05-08T23:13:07.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.406+0000 7ff1ba600700 1 -- 172.21.15.46:0/1675688703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0a10c0 con 0x557e6b2aa910 2024-05-08T23:13:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1675688703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 msgr2=0x557e6b2aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0cb7f0 tx=0x7ff1bc022c80 comp rx=0 tx=0).stop 2024-05-08T23:13:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1675688703 shutdown_connections 2024-05-08T23:13:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2b07a0 0x557e6b2b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2aa3d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1675688703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2afe80 0x557e6b2a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1675688703 >> 172.21.15.46:0/1675688703 conn(0x557e6b208290 msgr2=0x557e6b2adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1675688703 shutdown_connections 2024-05-08T23:13:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.407+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1675688703 wait complete. 2024-05-08T23:13:07.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.408+0000 7ff1e2e5bb80 1 -- start start 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.408+0000 7ff1e2e5bb80 1 Processor -- start 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2baf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2ac2b0 con 0x557e6b2aa910 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2ac420 con 0x557e6b2b07a0 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6b2ac5e0 con 0x557e6b2afe80 2024-05-08T23:13:07.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1e2e5bb80 10 client.0 _ll_register_callbacks cb 0x557e6b19fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55528/0 (socket says 172.21.15.46:55528) 2024-05-08T23:13:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1c7e00700 1 -- 172.21.15.46:0/1755958075 learned_addr learned my addr 172.21.15.46:0/1755958075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.409+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2baf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cd400700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cca00700 1 -- 172.21.15.46:0/1755958075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 msgr2=0x557e6b2bb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cca00700 1 -- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 msgr2=0x557e6b2abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1cca00700 1 -- 172.21.15.46:0/1755958075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6b027be0 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.410+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1cca00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2baf60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0994e0 tx=0x7ff1bc09acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd64390188444a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1ae000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0a10e0 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff1bc0a1240 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1bc0cbe90 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1bc0228e0 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.411+0000 7ff1e2e5bb80 10 client.18180 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e6b07b430 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1bc0cbe90 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1bc0a30f0 con 0x557e6b2aa910 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 10 client.18180 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 20 client.18180 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b3239d0 0x557e6b325dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 4 client.18180 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:13:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e6b1f2f40 con 0x557e6b3239d0 2024-05-08T23:13:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.412+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b3239d0 0x557e6b325dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.413+0000 7ff1c7e00700 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b3239d0 0x557e6b325dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e08ee6380929631 server_cookie=9725eeea1a3b3439 in_seq=0 out_seq=0 2024-05-08T23:13:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.413+0000 7ff1ae000700 10 client.18180 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.414+0000 7ff1ae000700 1 -- 172.21.15.46:0/1755958075 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e6b1f2f40 con 0x557e6b3239d0 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.414+0000 7ff1ae000700 10 client.18180 handle_command_reply: tid=0 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.414+0000 7ff1e2e5bb80 1 client.18180 shutdown 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 msgr2=0x557e6b2baf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2baf60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0994e0 tx=0x7ff1bc09acd0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 shutdown_connections 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6b2b07a0 0x557e6b2abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x557e6b3239d0 0x557e6b325dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e6b2aa910 0x557e6b2baf60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 --2- 172.21.15.46:0/1755958075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e6b2afe80 0x557e6b2bb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 >> 172.21.15.46:0/1755958075 conn(0x557e6b208290 msgr2=0x557e6b2b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1ae000700 0 client.18180 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 shutdown_connections 2024-05-08T23:13:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 1 -- 172.21.15.46:0/1755958075 wait complete. 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.415+0000 7ff1e2e5bb80 20 client.18180 trim_cache size 0 max 16384 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "de2dc871-1648-4bd1-bb73-fd6eef13d8f5", 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:13:07.417 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 msgr2=0x7ff1b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 0x7ff1b00a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1c819aa70 tx=0x7ff1b4009260 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 msgr2=0x7ff1c8199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc004730 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 shutdown_connections 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c810d730 0x7ff1c819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c80ffb80 0x7ff1c8199520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff1b00a1290 0x7ff1b00a3750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 --2- 172.21.15.46:0/3103498337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c8100530 0x7ff1c8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 >> 172.21.15.46:0/3103498337 conn(0x7ff1c80fb760 msgr2=0x7ff1c80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 shutdown_connections 2024-05-08T23:13:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.416+0000 7ff1cf400700 1 -- 172.21.15.46:0/3103498337 wait complete. 2024-05-08T23:13:07.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.672+0000 7f5a59600700 1 -- 172.21.15.46:0/3574818101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54100ec0 msgr2=0x7f5a541012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.672+0000 7f5a59600700 1 --2- 172.21.15.46:0/3574818101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54100ec0 0x7f5a541012a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009990 tx=0x7f5a44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 -- 172.21.15.46:0/3574818101 shutdown_connections 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 --2- 172.21.15.46:0/3574818101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54105fe0 0x7f5a5410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 --2- 172.21.15.46:0/3574818101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54100ec0 0x7f5a541012a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 --2- 172.21.15.46:0/3574818101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54101870 0x7f5a541058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 -- 172.21.15.46:0/3574818101 >> 172.21.15.46:0/3574818101 conn(0x7f5a540fc740 msgr2=0x7f5a540feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 -- 172.21.15.46:0/3574818101 shutdown_connections 2024-05-08T23:13:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.673+0000 7f5a59600700 1 -- 172.21.15.46:0/3574818101 wait complete. 2024-05-08T23:13:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 Processor -- start 2024-05-08T23:13:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 -- start start 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54105fe0 0x7f5a54199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419df00 0x7f5a5419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5419e980 con 0x7f5a54105fe0 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5419eaf0 con 0x7f5a54100ec0 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.674+0000 7f5a59600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5419ecb0 con 0x7f5a5419df00 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56510/0 (socket says 172.21.15.46:56510) 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54105fe0 0x7f5a54199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 -- 172.21.15.46:0/1563208900 learned_addr learned my addr 172.21.15.46:0/1563208900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a58c00700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419df00 0x7f5a5419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 -- 172.21.15.46:0/1563208900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419df00 msgr2=0x7f5a5419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419df00 0x7f5a5419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 -- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54105fe0 msgr2=0x7f5a54199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54105fe0 0x7f5a54199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 -- 172.21.15.46:0/1563208900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440095f0 con 0x7f5a54100ec0 2024-05-08T23:13:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.675+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a44005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.676+0000 7f5a50c00700 1 -- 172.21.15.46:0/1563208900 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4401d070 con 0x7f5a54100ec0 2024-05-08T23:13:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.676+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a541a61b0 con 0x7f5a54100ec0 2024-05-08T23:13:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.676+0000 7f5a50c00700 1 -- 172.21.15.46:0/1563208900 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a44022470 con 0x7f5a54100ec0 2024-05-08T23:13:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.676+0000 7f5a50c00700 1 -- 172.21.15.46:0/1563208900 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44017650 con 0x7f5a54100ec0 2024-05-08T23:13:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.676+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a541a65e0 con 0x7f5a54100ec0 2024-05-08T23:13:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.679+0000 7f5a50c00700 1 -- 172.21.15.46:0/1563208900 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5a440225e0 con 0x7f5a54100ec0 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.680+0000 7f5a6e6d9b80 1 Processor -- start 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.680+0000 7f5a50c00700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 0x7f5a380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 -- start start 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a50c00700 1 -- 172.21.15.46:0/1563208900 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5a440ce300 con 0x7f5a54100ec0 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a53400700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 0x7f5a380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a970 0x55a24c564350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c564cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c4bb150 con 0x55a24c564890 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c4b8b10 con 0x55a24c56a970 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c4b4400 con 0x55a24c56a050 2024-05-08T23:13:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55548/0 (socket says 172.21.15.46:55548) 2024-05-08T23:13:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a53400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a970 0x55a24c564350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.681+0000 7f5a53e00700 1 -- 172.21.15.46:0/1343523515 learned_addr learned my addr 172.21.15.46:0/1343523515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a58c00700 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c564cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 -- 172.21.15.46:0/1343523515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a970 msgr2=0x55a24c564350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a970 0x55a24c564350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 -- 172.21.15.46:0/1343523515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 msgr2=0x55a24c564cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c564cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 -- 172.21.15.46:0/1343523515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a24c3353b0 con 0x55a24c56a050 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53400700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 0x7f5a380a37f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a5419ab70 tx=0x7f5a400118e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.682+0000 7f5a53e00700 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a440a2ea0 tx=0x7f5a44009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59752b9c51311319 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a49200700 1 -- 172.21.15.46:0/1343523515 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4409b6e0 con 0x55a24c56a050 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a49200700 1 -- 172.21.15.46:0/1343523515 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a4409afe0 con 0x55a24c56a050 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a49200700 1 -- 172.21.15.46:0/1343523515 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440228e0 con 0x55a24c56a050 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/1343523515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 msgr2=0x55a24c56a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5a440a2ea0 tx=0x7f5a44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/1343523515 shutdown_connections 2024-05-08T23:13:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c564cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c56a430 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/1343523515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a970 0x55a24c564350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.683+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/1343523515 >> 172.21.15.46:0/1343523515 conn(0x55a24c4c2210 msgr2=0x55a24c55f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.684+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/1343523515 shutdown_connections 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.684+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/1343523515 wait complete. 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.684+0000 7f5a6e6d9b80 1 -- start start 2024-05-08T23:13:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.684+0000 7f5a6e6d9b80 1 Processor -- start 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c56e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c450560 con 0x55a24c564890 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c450200 con 0x55a24c56a050 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c3376a0 con 0x55a24c56a970 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a6e6d9b80 10 client.0 _ll_register_callbacks cb 0x55a24c459b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a53400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a53400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56534/0 (socket says 172.21.15.46:56534) 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43862/0 (socket says 172.21.15.46:43862) 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a53400700 1 -- 172.21.15.46:0/2494007178 learned_addr learned my addr 172.21.15.46:0/2494007178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.685+0000 7f5a58c00700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c56e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 -- 172.21.15.46:0/2494007178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 msgr2=0x55a24c56e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c56e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 -- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 msgr2=0x55a24c56dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 -- 172.21.15.46:0/2494007178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a541a65e0 con 0x55a24c564890 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53400700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5a44099330 tx=0x7f5a44009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fc78c134c74f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a3d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:13:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4409b6e0 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5a4409ee10 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a24c40d270 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a440a33d0 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.686+0000 7f5a6e6d9b80 10 client.18192 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a24c2e1b60 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a24c40d270 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a4409a4d0 con 0x55a24c564890 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 10 client.18192 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 20 client.18192 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c57db90 0x55a24c57ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 4 client.18192 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.687+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a24c533810 con 0x55a24c57db90 2024-05-08T23:13:07.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.688+0000 7f5a53400700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c57db90 0x55a24c57ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.688+0000 7f5a53400700 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c57db90 0x55a24c57ff80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33e7a4d28a87c7d server_cookie=29aaf0da04a9db0e in_seq=0 out_seq=0 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.688+0000 7f5a3d600700 10 client.18192 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a3d600700 1 -- 172.21.15.46:0/2494007178 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a24c533810 con 0x55a24c57db90 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a3d600700 10 client.18192 handle_command_reply: tid=0 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 client.18192 shutdown 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 msgr2=0x55a24c570960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5a44099330 tx=0x7f5a44009530 comp rx=0 tx=0).stop 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 shutdown_connections 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c56e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c57db90 0x55a24c57ff80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c564890 0x55a24c570960 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2494007178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c56a050 0x55a24c56dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a3d600700 0 client.18192 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 >> 172.21.15.46:0/2494007178 conn(0x55a24c4c2210 msgr2=0x55a24c55f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 shutdown_connections 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2494007178 wait complete. 2024-05-08T23:13:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.689+0000 7f5a6e6d9b80 20 client.18192 trim_cache size 0 max 16384 2024-05-08T23:13:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 Processor -- start 2024-05-08T23:13:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 -- start start 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c564350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a970 0x55a24c584810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c55e3d0 con 0x55a24c56a970 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c559830 con 0x55a24c564890 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c560260 con 0x55a24c56a050 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c564350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55568/0 (socket says 172.21.15.46:55568) 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53400700 1 -- 172.21.15.46:0/2147866608 learned_addr learned my addr 172.21.15.46:0/2147866608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.709+0000 7f5a58c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a970 0x55a24c584810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 -- 172.21.15.46:0/2147866608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 msgr2=0x55a24c570570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 -- 172.21.15.46:0/2147866608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a970 msgr2=0x55a24c584810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a970 0x55a24c584810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 -- 172.21.15.46:0/2147866608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a24c47a560 con 0x55a24c564890 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53400700 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c564350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a4409aab0 tx=0x7f5a440cb670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65ddfc480a7a8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a49c00700 1 -- 172.21.15.46:0/2147866608 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4409b6e0 con 0x55a24c564890 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a49c00700 1 -- 172.21.15.46:0/2147866608 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a440a33d0 con 0x55a24c564890 2024-05-08T23:13:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.710+0000 7f5a49c00700 1 -- 172.21.15.46:0/2147866608 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4409ee10 con 0x55a24c564890 2024-05-08T23:13:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2147866608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 msgr2=0x55a24c564350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c564350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a4409aab0 tx=0x7f5a440cb670 comp rx=0 tx=0).stop 2024-05-08T23:13:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2147866608 shutdown_connections 2024-05-08T23:13:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a970 0x55a24c584810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c564350 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2147866608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a050 0x55a24c570570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2147866608 >> 172.21.15.46:0/2147866608 conn(0x55a24c4c2210 msgr2=0x55a24c564c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2147866608 shutdown_connections 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.711+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2147866608 wait complete. 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 -- start start 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 Processor -- start 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a050 0x55a24c566b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c5670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c5677b0 con 0x55a24c56a050 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c567920 con 0x55a24c564890 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24c565b10 con 0x55a24c56a970 2024-05-08T23:13:07.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.712+0000 7f5a6e6d9b80 10 client.0 _ll_register_callbacks cb 0x55a24c459b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c5670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56548/0 (socket says 172.21.15.46:56548) 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a53e00700 1 -- 172.21.15.46:0/2165677413 learned_addr learned my addr 172.21.15.46:0/2165677413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a53400700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a050 0x55a24c566b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 -- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 msgr2=0x55a24c5753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 -- 172.21.15.46:0/2165677413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a050 msgr2=0x55a24c566b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a050 0x55a24c566b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 -- 172.21.15.46:0/2165677413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a400099e0 con 0x55a24c56a970 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a58c00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c5670a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009fd0 tx=0x7f5a4c007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a684f3302b0a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.713+0000 7f5a3d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00e040 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a4409afe0 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5a4c003c00 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a4c009400 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 10 client.28285 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a24c2e1b60 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a4409afe0 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a4c020bb0 con 0x55a24c56a970 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 10 client.28285 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 10 client.28285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.714+0000 7f5a6e6d9b80 20 client.28285 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c5d4870 0x55a24c5d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a6e6d9b80 4 client.28285 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:13:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a24c4acec0 con 0x55a24c5d4870 2024-05-08T23:13:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c5d4870 0x55a24c5d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a53e00700 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c5d4870 0x55a24c5d6c90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2191702ca071d818 server_cookie=17c24cef4b8925c1 in_seq=0 out_seq=0 2024-05-08T23:13:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a3d600700 10 client.28285 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a3d600700 1 -- 172.21.15.46:0/2165677413 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a24c4acec0 con 0x55a24c5d4870 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.715+0000 7f5a3d600700 10 client.28285 handle_command_reply: tid=0 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 client.28285 shutdown 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 msgr2=0x55a24c5670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c5670a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009fd0 tx=0x7f5a4c007eb0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 shutdown_connections 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a24c56a970 0x55a24c5670a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a24c564890 0x55a24c5753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a24c5d4870 0x55a24c5d6c90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 --2- 172.21.15.46:0/2165677413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a24c56a050 0x55a24c566b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 >> 172.21.15.46:0/2165677413 conn(0x55a24c4c2210 msgr2=0x55a24c584810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a3d600700 0 client.28285 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 shutdown_connections 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 1 -- 172.21.15.46:0/2165677413 wait complete. 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.716+0000 7f5a6e6d9b80 20 client.28285 trim_cache size 0 max 16384 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "de2dc871-1648-4bd1-bb73-fd6eef13d8f5": { 2024-05-08T23:13:07.721 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "de2dc871-1648-4bd1-bb73-fd6eef13d8f5", 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:07.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 msgr2=0x7f5a380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 0x7f5a380a37f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a5419ab70 tx=0x7f5a400118e0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 msgr2=0x7f5a54199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a44005740 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 shutdown_connections 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419df00 0x7f5a5419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a54100ec0 0x7f5a54199670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5a380a1330 0x7f5a380a37f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 --2- 172.21.15.46:0/1563208900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a54105fe0 0x7f5a54199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 >> 172.21.15.46:0/1563208900 conn(0x7f5a540fc740 msgr2=0x7f5a540fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 shutdown_connections 2024-05-08T23:13:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:07.717+0000 7f5a59600700 1 -- 172.21.15.46:0/1563208900 wait complete. 2024-05-08T23:13:07.737 INFO:tasks.cephfs.filesystem:scrub status for tag:de2dc871-1648-4bd1-bb73-fd6eef13d8f5 - {'path': '/', 'tag': 'de2dc871-1648-4bd1-bb73-fd6eef13d8f5', 'options': 'recursive,force'} 2024-05-08T23:13:07.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 805965 9.80 MB/sec execute 329 sec latency 16.505 ms 2024-05-08T23:13:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:07 smithi086 ceph-mon[78610]: pgmap v287: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:07 smithi053 ceph-mon[77587]: pgmap v287: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:07 smithi046 ceph-mon[36898]: pgmap v287: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:08.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 806935 9.78 MB/sec execute 330 sec latency 15.622 ms 2024-05-08T23:13:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:08 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:08 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:08 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:09.903 INFO:tasks.workunit.client.0.smithi046.stdout: 1 808970 9.79 MB/sec execute 331 sec latency 18.218 ms 2024-05-08T23:13:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:09 smithi086 ceph-mon[78610]: pgmap v288: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:09 smithi053 ceph-mon[77587]: pgmap v288: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:09 smithi046 ceph-mon[36898]: pgmap v288: 129 pgs: 129 active+clean; 503 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-05-08T23:13:10.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 810049 9.78 MB/sec execute 332 sec latency 22.576 ms 2024-05-08T23:13:11.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 811938 9.77 MB/sec execute 333 sec latency 19.672 ms 2024-05-08T23:13:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:11 smithi086 ceph-mon[78610]: pgmap v289: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:11 smithi053 ceph-mon[77587]: pgmap v289: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:11 smithi046 ceph-mon[36898]: pgmap v289: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:12.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 813300 9.77 MB/sec execute 334 sec latency 38.208 ms 2024-05-08T23:13:13.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 814820 9.76 MB/sec execute 335 sec latency 17.230 ms 2024-05-08T23:13:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:13 smithi086 ceph-mon[78610]: pgmap v290: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:13 smithi053 ceph-mon[77587]: pgmap v290: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:13 smithi046 ceph-mon[36898]: pgmap v290: 129 pgs: 129 active+clean; 506 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-08T23:13:14.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 816697 9.77 MB/sec execute 336 sec latency 17.619 ms 2024-05-08T23:13:15.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 817192 9.74 MB/sec execute 337 sec latency 160.859 ms 2024-05-08T23:13:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:15 smithi086 ceph-mon[78610]: pgmap v291: 129 pgs: 129 active+clean; 511 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-05-08T23:13:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:15 smithi086 ceph-mon[78610]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:13:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:15 smithi053 ceph-mon[77587]: pgmap v291: 129 pgs: 129 active+clean; 511 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-05-08T23:13:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:15 smithi053 ceph-mon[77587]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:13:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:15 smithi046 ceph-mon[36898]: pgmap v291: 129 pgs: 129 active+clean; 511 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-05-08T23:13:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:15 smithi046 ceph-mon[36898]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:13:16.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 818288 9.73 MB/sec execute 338 sec latency 172.975 ms 2024-05-08T23:13:17.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 819961 9.73 MB/sec execute 339 sec latency 53.155 ms 2024-05-08T23:13:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:17 smithi086 ceph-mon[78610]: pgmap v293: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:17 smithi053 ceph-mon[77587]: pgmap v293: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:17 smithi046 ceph-mon[36898]: pgmap v293: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:18.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 820927 9.72 MB/sec execute 340 sec latency 22.680 ms 2024-05-08T23:13:19.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 823050 9.72 MB/sec execute 341 sec latency 19.918 ms 2024-05-08T23:13:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:19 smithi086 ceph-mon[78610]: pgmap v294: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:20.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:19 smithi053 ceph-mon[77587]: pgmap v294: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:19 smithi046 ceph-mon[36898]: pgmap v294: 129 pgs: 129 active+clean; 512 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-08T23:13:20.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 824130 9.71 MB/sec execute 342 sec latency 14.796 ms 2024-05-08T23:13:21.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:20 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:21.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:20 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:20 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:20 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:20 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:20 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:21.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 825545 9.70 MB/sec execute 343 sec latency 87.071 ms 2024-05-08T23:13:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:21 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:21 smithi086 ceph-mon[78610]: pgmap v295: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:21 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:21 smithi053 ceph-mon[77587]: pgmap v295: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:21 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:21 smithi046 ceph-mon[36898]: pgmap v295: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:22.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 827306 9.71 MB/sec execute 344 sec latency 38.467 ms 2024-05-08T23:13:23.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 828296 9.69 MB/sec execute 345 sec latency 31.269 ms 2024-05-08T23:13:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:23 smithi086 ceph-mon[78610]: pgmap v296: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:23 smithi053 ceph-mon[77587]: pgmap v296: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:23 smithi046 ceph-mon[36898]: pgmap v296: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.9 MiB/s wr, 272 op/s 2024-05-08T23:13:24.904 INFO:tasks.workunit.client.0.smithi046.stdout: 1 830566 9.70 MB/sec execute 346 sec latency 17.176 ms 2024-05-08T23:13:25.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 831492 9.68 MB/sec execute 347 sec latency 15.339 ms 2024-05-08T23:13:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:25 smithi086 ceph-mon[78610]: pgmap v297: 129 pgs: 129 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:13:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:25 smithi053 ceph-mon[77587]: pgmap v297: 129 pgs: 129 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:13:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:25 smithi046 ceph-mon[36898]: pgmap v297: 129 pgs: 129 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:13:26.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 833690 9.69 MB/sec execute 348 sec latency 16.554 ms 2024-05-08T23:13:27.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 834745 9.68 MB/sec execute 349 sec latency 17.283 ms 2024-05-08T23:13:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:27 smithi086 ceph-mon[78610]: pgmap v298: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 252 op/s 2024-05-08T23:13:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:27 smithi053 ceph-mon[77587]: pgmap v298: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 252 op/s 2024-05-08T23:13:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:27 smithi046 ceph-mon[36898]: pgmap v298: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 252 op/s 2024-05-08T23:13:28.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 836618 9.67 MB/sec execute 350 sec latency 17.320 ms 2024-05-08T23:13:29.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 838053 9.68 MB/sec execute 351 sec latency 31.063 ms 2024-05-08T23:13:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:29 smithi086 ceph-mon[78610]: pgmap v299: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 241 op/s 2024-05-08T23:13:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:29 smithi053 ceph-mon[77587]: pgmap v299: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 241 op/s 2024-05-08T23:13:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:29 smithi046 ceph-mon[36898]: pgmap v299: 129 pgs: 129 active+clean; 513 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 241 op/s 2024-05-08T23:13:30.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 839426 9.67 MB/sec execute 352 sec latency 26.256 ms 2024-05-08T23:13:31.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 841449 9.68 MB/sec execute 353 sec latency 16.808 ms 2024-05-08T23:13:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:31 smithi086 ceph-mon[78610]: pgmap v300: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 304 op/s 2024-05-08T23:13:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:31 smithi053 ceph-mon[77587]: pgmap v300: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 304 op/s 2024-05-08T23:13:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:31 smithi046 ceph-mon[36898]: pgmap v300: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 304 op/s 2024-05-08T23:13:32.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 842362 9.66 MB/sec execute 354 sec latency 22.742 ms 2024-05-08T23:13:33.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 844232 9.66 MB/sec execute 355 sec latency 18.010 ms 2024-05-08T23:13:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:33 smithi086 ceph-mon[78610]: pgmap v301: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-05-08T23:13:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:33 smithi053 ceph-mon[77587]: pgmap v301: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-05-08T23:13:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:33 smithi046 ceph-mon[36898]: pgmap v301: 129 pgs: 129 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-05-08T23:13:34.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 845409 9.65 MB/sec execute 356 sec latency 28.101 ms 2024-05-08T23:13:35.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 846914 9.64 MB/sec execute 357 sec latency 25.403 ms 2024-05-08T23:13:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:35 smithi086 ceph-mon[78610]: pgmap v302: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 196 op/s 2024-05-08T23:13:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:35 smithi053 ceph-mon[77587]: pgmap v302: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 196 op/s 2024-05-08T23:13:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:35 smithi046 ceph-mon[36898]: pgmap v302: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 196 op/s 2024-05-08T23:13:36.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 848685 9.65 MB/sec execute 358 sec latency 15.568 ms 2024-05-08T23:13:37.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:13:37.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 850056 9.64 MB/sec execute 359 sec latency 17.633 ms 2024-05-08T23:13:37.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.971+0000 7f64ef800700 1 -- 172.21.15.46:0/1575366083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8073340 msgr2=0x7f64e8073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.971+0000 7f64ef800700 1 --2- 172.21.15.46:0/1575366083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8073340 0x7f64e8073720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009990 tx=0x7f64dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 -- 172.21.15.46:0/1575366083 shutdown_connections 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 --2- 172.21.15.46:0/1575366083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8111e20 0x7f64e8114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 --2- 172.21.15.46:0/1575366083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8073340 0x7f64e8073720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 --2- 172.21.15.46:0/1575366083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073c60 0x7f64e81118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 -- 172.21.15.46:0/1575366083 >> 172.21.15.46:0/1575366083 conn(0x7f64e80fc8e0 msgr2=0x7f64e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 -- 172.21.15.46:0/1575366083 shutdown_connections 2024-05-08T23:13:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.972+0000 7f64ef800700 1 -- 172.21.15.46:0/1575366083 wait complete. 2024-05-08T23:13:37.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.973+0000 7f64ef800700 1 Processor -- start 2024-05-08T23:13:37.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.973+0000 7f64ef800700 1 -- start start 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073340 0x7f64e81a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8111e20 0x7f64e819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819de70 con 0x7f64e8073340 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819dfe0 con 0x7f64e8111e20 2024-05-08T23:13:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ef800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819e1a0 con 0x7f64e8073c60 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ece00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073340 0x7f64e81a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33708/0 (socket says 172.21.15.46:33708) 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64e7e00700 1 -- 172.21.15.46:0/400957806 learned_addr learned my addr 172.21.15.46:0/400957806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.974+0000 7f64ed800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8111e20 0x7f64e819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 -- 172.21.15.46:0/400957806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8111e20 msgr2=0x7f64e819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8111e20 0x7f64e819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 -- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073340 msgr2=0x7f64e81a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073340 0x7f64e81a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 -- 172.21.15.46:0/400957806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0009c40 con 0x7f64e8073c60 2024-05-08T23:13:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e7e00700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9a0 tx=0x7f64d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e5600700 1 -- 172.21.15.46:0/400957806 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4009790 con 0x7f64e8073c60 2024-05-08T23:13:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc0095f0 con 0x7f64e8073c60 2024-05-08T23:13:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e5600700 1 -- 172.21.15.46:0/400957806 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d400ce80 con 0x7f64e8073c60 2024-05-08T23:13:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64e5600700 1 -- 172.21.15.46:0/400957806 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4018680 con 0x7f64e8073c60 2024-05-08T23:13:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.975+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e819e670 con 0x7f64e8073c60 2024-05-08T23:13:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.978+0000 7f64e5600700 1 -- 172.21.15.46:0/400957806 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f64d4010430 con 0x7f64e8073c60 2024-05-08T23:13:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.979+0000 7f6503374b80 1 Processor -- start 2024-05-08T23:13:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 -- start start 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e1094fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64e5600700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 0x7f64d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e109452150 con 0x55e109501720 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10944fb10 con 0x55e109500e00 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f6503374b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10944b400 con 0x55e1094fb890 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48260/0 (socket says 172.21.15.46:48260) 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64ed800700 1 -- 172.21.15.46:0/3947760769 learned_addr learned my addr 172.21.15.46:0/3947760769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64ece00700 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e1094fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64ed800700 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.980+0000 7f64e5600700 1 -- 172.21.15.46:0/400957806 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f64d4014070 con 0x7f64e8073c60 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64e7e00700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 0x7f64d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 -- 172.21.15.46:0/3947760769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 msgr2=0x55e1094fbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 -- 172.21.15.46:0/3947760769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 msgr2=0x55e1094fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e1094fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 -- 172.21.15.46:0/3947760769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1092cc3b0 con 0x55e109500e00 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64ed800700 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009b50 tx=0x7f64e0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48b4c5b7859ab55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64e7e00700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 0x7f64d00a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64d4096560 tx=0x7f64d40963a0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64da000700 1 -- 172.21.15.46:0/3947760769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000e070 con 0x55e109500e00 2024-05-08T23:13:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64da000700 1 -- 172.21.15.46:0/3947760769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e000f040 con 0x55e109500e00 2024-05-08T23:13:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.981+0000 7f64da000700 1 -- 172.21.15.46:0/3947760769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004340 con 0x55e109500e00 2024-05-08T23:13:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 -- 172.21.15.46:0/3947760769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 msgr2=0x55e1095011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009b50 tx=0x7f64e0009920 comp rx=0 tx=0).stop 2024-05-08T23:13:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 -- 172.21.15.46:0/3947760769 shutdown_connections 2024-05-08T23:13:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109500e00 0x55e1095011e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 --2- 172.21.15.46:0/3947760769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e1094fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 -- 172.21.15.46:0/3947760769 >> 172.21.15.46:0/3947760769 conn(0x55e109459210 msgr2=0x55e1094f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 -- 172.21.15.46:0/3947760769 shutdown_connections 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.982+0000 7f6503374b80 1 -- 172.21.15.46:0/3947760769 wait complete. 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.983+0000 7f6503374b80 1 -- start start 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.983+0000 7f6503374b80 1 Processor -- start 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e109505030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109501720 0x55e109505570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1093e7560 con 0x55e109500e00 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1093e7200 con 0x55e1094fb890 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1092ce6a0 con 0x55e109501720 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f6503374b80 10 client.0 _ll_register_callbacks cb 0x55e1093f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48274/0 (socket says 172.21.15.46:48274) 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 -- 172.21.15.46:0/1593203689 learned_addr learned my addr 172.21.15.46:0/1593203689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109501720 0x55e109505570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ece00700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e109505030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 -- 172.21.15.46:0/1593203689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109501720 msgr2=0x55e109505570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109501720 0x55e109505570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 -- 172.21.15.46:0/1593203689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 msgr2=0x55e109505030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e109505030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.984+0000 7f64ed800700 1 -- 172.21.15.46:0/1593203689 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e0008a10 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f64ed800700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64e0018c20 tx=0x7f64e0018ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c59fe563f620ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f64cea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000e070 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f64e001b650 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1093a4270 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e001b810 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f6503374b80 10 client.28217 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.985+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e109278b60 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1093a4270 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f64e0018640 con 0x55e1094fb890 2024-05-08T23:13:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 10 client.28217 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 10 client.28217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 20 client.28217 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e109514fd0 0x55e1095173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 4 client.28217 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1094ca810 con 0x55e109514fd0 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.986+0000 7f64ece00700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e109514fd0 0x55e1095173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.987+0000 7f64ece00700 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e109514fd0 0x55e1095173c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb73bc3648d14a4 server_cookie=7c43152d3f3c2f5c in_seq=0 out_seq=0 2024-05-08T23:13:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.987+0000 7f64cea00700 10 client.28217 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f64cea00700 1 -- 172.21.15.46:0/1593203689 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1094ca810 con 0x55e109514fd0 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f64cea00700 10 client.28217 handle_command_reply: tid=0 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 client.28217 shutdown 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 msgr2=0x55e109504af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64e0018c20 tx=0x7f64e0018ec0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 shutdown_connections 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109501720 0x55e109505570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e109504af0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e109514fd0 0x55e1095173c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 --2- 172.21.15.46:0/1593203689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e109505030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 >> 172.21.15.46:0/1593203689 conn(0x55e109459210 msgr2=0x55e1094f55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f64cea00700 0 client.28217 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.988+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 shutdown_connections 2024-05-08T23:13:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.989+0000 7f6503374b80 1 -- 172.21.15.46:0/1593203689 wait complete. 2024-05-08T23:13:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:37.989+0000 7f6503374b80 20 client.28217 trim_cache size 0 max 16384 2024-05-08T23:13:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 Processor -- start 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 -- start start 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109501720 0x55e109500280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10951c590 con 0x55e109500e00 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1094f0800 con 0x55e109501720 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.008+0000 7f6503374b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e109503040 con 0x55e1094fb890 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ece00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ece00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57670/0 (socket says 172.21.15.46:57670) 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33732/0 (socket says 172.21.15.46:33732) 2024-05-08T23:13:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ece00700 1 -- 172.21.15.46:0/366545408 learned_addr learned my addr 172.21.15.46:0/366545408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109501720 0x55e109500280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 -- 172.21.15.46:0/366545408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109501720 msgr2=0x55e109500280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109501720 0x55e109500280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 -- 172.21.15.46:0/366545408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 msgr2=0x55e1094f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 -- 172.21.15.46:0/366545408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1094fbf40 con 0x55e1094fb890 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ece00700 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64ed800700 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f64e0006130 tx=0x7f64e0005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0a1fd7df3b169a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.009+0000 7f64daa00700 1 -- 172.21.15.46:0/366545408 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000e070 con 0x55e1094fb890 2024-05-08T23:13:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f64daa00700 1 -- 172.21.15.46:0/366545408 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e00087a0 con 0x55e1094fb890 2024-05-08T23:13:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f64daa00700 1 -- 172.21.15.46:0/366545408 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0027e40 con 0x55e1094fb890 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 -- 172.21.15.46:0/366545408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 msgr2=0x55e1094fb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f64e0006130 tx=0x7f64e0005ba0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 -- 172.21.15.46:0/366545408 shutdown_connections 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e109501720 0x55e109500280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1094fb890 0x55e1094fb350 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 --2- 172.21.15.46:0/366545408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109500e00 0x55e1094f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.010+0000 7f6503374b80 1 -- 172.21.15.46:0/366545408 >> 172.21.15.46:0/366545408 conn(0x55e109459210 msgr2=0x55e1094ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.011+0000 7f6503374b80 1 -- 172.21.15.46:0/366545408 shutdown_connections 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.011+0000 7f6503374b80 1 -- 172.21.15.46:0/366545408 wait complete. 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.011+0000 7f6503374b80 1 -- start start 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.011+0000 7f6503374b80 1 Processor -- start 2024-05-08T23:13:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e10950c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e10950b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10950b8b0 con 0x55e109501720 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10956b470 con 0x55e1094fb890 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10956b5b0 con 0x55e109500e00 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f6503374b80 10 client.0 _ll_register_callbacks cb 0x55e1093f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f64ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e10950c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.012+0000 7f64ece00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ed800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e10950b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33738/0 (socket says 172.21.15.46:33738) 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 -- 172.21.15.46:0/714947342 learned_addr learned my addr 172.21.15.46:0/714947342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ed800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e10950b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57680/0 (socket says 172.21.15.46:57680) 2024-05-08T23:13:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 -- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 msgr2=0x55e10950c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e10950c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 -- 172.21.15.46:0/714947342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 msgr2=0x55e10950b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e10950b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 -- 172.21.15.46:0/714947342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8114bc0 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64ece00700 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc0052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770d7b15cfe398b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.013+0000 7f64cea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc01d040 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64e000f040 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f64dc005690 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64dc00b950 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f6503374b80 10 client.28303 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e109278b60 con 0x55e109500e00 2024-05-08T23:13:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e000f040 con 0x55e109500e00 2024-05-08T23:13:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.014+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f64dc032470 con 0x55e109500e00 2024-05-08T23:13:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 10 client.28303 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 10 client.28303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 20 client.28303 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e10956cd80 0x55e10956f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 4 client.28303 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e109443ec0 con 0x55e10956cd80 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f64ed800700 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e10956cd80 0x55e10956f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f64ed800700 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e10956cd80 0x55e10956f1a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84d18e2fa08c8fa server_cookie=9e99f97123299d8a in_seq=0 out_seq=0 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.015+0000 7f64cea00700 10 client.28303 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f64cea00700 1 -- 172.21.15.46:0/714947342 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e109443ec0 con 0x55e10956cd80 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f64cea00700 10 client.28303 handle_command_reply: tid=0 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 client.28303 shutdown 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 msgr2=0x55e10950aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b2e0 tx=0x7f64dc0052d0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 shutdown_connections 2024-05-08T23:13:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e10956cd80 0x55e10956f1a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e109501720 0x55e10950b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1094fb890 0x55e10950c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f64cea00700 0 client.28303 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 --2- 172.21.15.46:0/714947342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e109500e00 0x55e10950aab0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 >> 172.21.15.46:0/714947342 conn(0x55e109459210 msgr2=0x55e109506bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 shutdown_connections 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 1 -- 172.21.15.46:0/714947342 wait complete. 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.016+0000 7f6503374b80 20 client.28303 trim_cache size 0 max 16384 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:13:38.022 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:13:38.023 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 msgr2=0x7f64d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 0x7f64d00a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64d4096560 tx=0x7f64d40963a0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 msgr2=0x7f64e81a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9a0 tx=0x7f64d400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 shutdown_connections 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8111e20 0x7f64e819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64d00a1290 0x7f64d00a3750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64e8073340 0x7f64e81a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 --2- 172.21.15.46:0/400957806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e8073c60 0x7f64e81a3bb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 >> 172.21.15.46:0/400957806 conn(0x7f64e80fc8e0 msgr2=0x7f64e80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 shutdown_connections 2024-05-08T23:13:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.017+0000 7f64ef800700 1 -- 172.21.15.46:0/400957806 wait complete. 2024-05-08T23:13:38.036 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:13:38.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:13:38.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:37 smithi046 ceph-mon[36898]: pgmap v303: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-05-08T23:13:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:37 smithi086 ceph-mon[78610]: pgmap v303: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 -- 172.21.15.46:0/3142986710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8106190 msgr2=0x7f9fe810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 --2- 172.21.15.46:0/3142986710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8106190 0x7f9fe810dd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b0a0 tx=0x7f9fe000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 -- 172.21.15.46:0/3142986710 shutdown_connections 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 --2- 172.21.15.46:0/3142986710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8106190 0x7f9fe810dd10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 --2- 172.21.15.46:0/3142986710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 --2- 172.21.15.46:0/3142986710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8101a20 0x7f9fe8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 -- 172.21.15.46:0/3142986710 >> 172.21.15.46:0/3142986710 conn(0x7f9fe80fc930 msgr2=0x7f9fe80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 -- 172.21.15.46:0/3142986710 shutdown_connections 2024-05-08T23:13:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.247+0000 7f9feda00700 1 -- 172.21.15.46:0/3142986710 wait complete. 2024-05-08T23:13:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.248+0000 7f9feda00700 1 Processor -- start 2024-05-08T23:13:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 -- start start 2024-05-08T23:13:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 0x7f9fe8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe819e9e0 con 0x7f9fe8106190 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe819eb50 con 0x7f9fe8101070 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.249+0000 7f9feda00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe819ed10 con 0x7f9fe819df60 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe6000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 0x7f9fe8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48316/0 (socket says 172.21.15.46:48316) 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33748/0 (socket says 172.21.15.46:33748) 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe6a00700 1 -- 172.21.15.46:0/3000982768 learned_addr learned my addr 172.21.15.46:0/3000982768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 -- 172.21.15.46:0/3000982768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 msgr2=0x7f9fe81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 -- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 msgr2=0x7f9fe8199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 0x7f9fe8199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 -- 172.21.15.46:0/3000982768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4009520 con 0x7f9fe819df60 2024-05-08T23:13:38.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe6000700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 0x7f9fe8199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:38.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fe7400700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b070 tx=0x7f9fe000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000e040 con 0x7f9fe819df60 2024-05-08T23:13:38.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.250+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe001d480 con 0x7f9fe819df60 2024-05-08T23:13:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.251+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0009d00 con 0x7f9fe819df60 2024-05-08T23:13:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.251+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00052a0 con 0x7f9fe819df60 2024-05-08T23:13:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.251+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81a6450 con 0x7f9fe819df60 2024-05-08T23:13:38.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.252+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe8102540 con 0x7f9fe819df60 2024-05-08T23:13:38.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.254+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9fe001d5f0 con 0x7f9fe819df60 2024-05-08T23:13:38.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.255+0000 7f9fdf400700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 0x7f9fcc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.255+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9fe00ce100 con 0x7f9fe819df60 2024-05-08T23:13:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.255+0000 7f9fe6a00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 0x7f9fcc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.256+0000 7f9fe6a00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 0x7f9fcc0a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d60 tx=0x7f9fd8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.259+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9fe0097170 con 0x7f9fe819df60 2024-05-08T23:13:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:37 smithi053 ceph-mon[77587]: pgmap v303: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-05-08T23:13:38.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.560+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fe8101a20 con 0x7f9fe819df60 2024-05-08T23:13:38.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.561+0000 7f9fdf400700 1 -- 172.21.15.46:0/3000982768 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f9fe009a790 con 0x7f9fe819df60 2024-05-08T23:13:38.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:13:38.562 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:13:38.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.563+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 msgr2=0x7f9fcc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.563+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 0x7f9fcc0a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d60 tx=0x7f9fd8009260 comp rx=0 tx=0).stop 2024-05-08T23:13:38.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.563+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 msgr2=0x7f9fe819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.563+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b070 tx=0x7f9fe000bcf0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 shutdown_connections 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe819df60 0x7f9fe819e3e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe8101070 0x7f9fe81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9fcc0a1290 0x7f9fcc0a3750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 --2- 172.21.15.46:0/3000982768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8106190 0x7f9fe8199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 >> 172.21.15.46:0/3000982768 conn(0x7f9fe80fc930 msgr2=0x7f9fe80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 shutdown_connections 2024-05-08T23:13:38.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.564+0000 7f9feda00700 1 -- 172.21.15.46:0/3000982768 wait complete. 2024-05-08T23:13:38.566 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:13:38.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:13:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.808+0000 7f453fe00700 1 -- 172.21.15.46:0/3231945471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810d750 msgr2=0x7f453810fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.808+0000 7f453fe00700 1 --2- 172.21.15.46:0/3231945471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810d750 0x7f453810fb40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4530009f90 tx=0x7f453000b330 comp rx=0 tx=0).stop 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 -- 172.21.15.46:0/3231945471 shutdown_connections 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 --2- 172.21.15.46:0/3231945471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810d750 0x7f453810fb40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 --2- 172.21.15.46:0/3231945471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 0x7f45380fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 --2- 172.21.15.46:0/3231945471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4538100550 0x7f453810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 -- 172.21.15.46:0/3231945471 >> 172.21.15.46:0/3231945471 conn(0x7f45380fb760 msgr2=0x7f45380fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 -- 172.21.15.46:0/3231945471 shutdown_connections 2024-05-08T23:13:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.809+0000 7f453fe00700 1 -- 172.21.15.46:0/3231945471 wait complete. 2024-05-08T23:13:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.810+0000 7f453fe00700 1 Processor -- start 2024-05-08T23:13:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.810+0000 7f453fe00700 1 -- start start 2024-05-08T23:13:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 0x7f4538199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f453810d750 0x7f453819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453819e570 con 0x7f453810d750 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453819e6e0 con 0x7f45380ffba0 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453819e8a0 con 0x7f4538100550 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33758/0 (socket says 172.21.15.46:33758) 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 -- 172.21.15.46:0/1695017816 learned_addr learned my addr 172.21.15.46:0/1695017816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453de00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f453810d750 0x7f453819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453d400700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 0x7f4538199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 -- 172.21.15.46:0/1695017816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 msgr2=0x7f4538199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 0x7f4538199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 -- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f453810d750 msgr2=0x7f453819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f453810d750 0x7f453819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.811+0000 7f453ca00700 1 -- 172.21.15.46:0/1695017816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4530009c40 con 0x7f4538100550 2024-05-08T23:13:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f453ca00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f452400e9a0 tx=0x7f452400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f4536000700 1 -- 172.21.15.46:0/1695017816 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524009790 con 0x7f4538100550 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f4536000700 1 -- 172.21.15.46:0/1695017816 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f452400ce80 con 0x7f4538100550 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453819eb80 con 0x7f4538100550 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f4536000700 1 -- 172.21.15.46:0/1695017816 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524018680 con 0x7f4538100550 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.812+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45381a63c0 con 0x7f4538100550 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.815+0000 7f4536000700 1 -- 172.21.15.46:0/1695017816 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4524010430 con 0x7f4538100550 2024-05-08T23:13:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.817+0000 7f4536000700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 0x7f45200a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.817+0000 7f4536000700 1 -- 172.21.15.46:0/1695017816 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4524014070 con 0x7f4538100550 2024-05-08T23:13:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.817+0000 7f453d400700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 0x7f45200a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.818+0000 7f453d400700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 0x7f45200a3810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.818+0000 7f4553876b80 1 Processor -- start 2024-05-08T23:13:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 -- start start 2024-05-08T23:13:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a45f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a459350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a3b0150 con 0x55f49a45ee00 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a3adb10 con 0x55f49a45f720 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f4553876b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a3a9400 con 0x55f49a459890 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f453ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a459350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f453d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a45f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33774/0 (socket says 172.21.15.46:33774) 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.819+0000 7f453de00700 1 -- 172.21.15.46:0/1995772787 learned_addr learned my addr 172.21.15.46:0/1995772787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 -- 172.21.15.46:0/1995772787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 msgr2=0x55f49a459350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a459350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 -- 172.21.15.46:0/1995772787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 msgr2=0x55f49a45f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a45f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 -- 172.21.15.46:0/1995772787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f49a22a3b0 con 0x55f49a459890 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453ca00700 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a459350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f453de00700 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f45300094d0 tx=0x7f4530009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7562af8b6e8f24a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f452a600700 1 -- 172.21.15.46:0/1995772787 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000e040 con 0x55f49a459890 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f452a600700 1 -- 172.21.15.46:0/1995772787 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4530007620 con 0x55f49a459890 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.820+0000 7f452a600700 1 -- 172.21.15.46:0/1995772787 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530007c10 con 0x55f49a459890 2024-05-08T23:13:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 -- 172.21.15.46:0/1995772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 msgr2=0x55f49a459cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f45300094d0 tx=0x7f4530009500 comp rx=0 tx=0).stop 2024-05-08T23:13:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 -- 172.21.15.46:0/1995772787 shutdown_connections 2024-05-08T23:13:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a459cd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a45f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 --2- 172.21.15.46:0/1995772787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a459350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 -- 172.21.15.46:0/1995772787 >> 172.21.15.46:0/1995772787 conn(0x55f49a3b7210 msgr2=0x55f49a453df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.821+0000 7f4553876b80 1 -- 172.21.15.46:0/1995772787 shutdown_connections 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.822+0000 7f4553876b80 1 -- 172.21.15.46:0/1995772787 wait complete. 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.822+0000 7f4553876b80 1 -- start start 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.822+0000 7f4553876b80 1 Processor -- start 2024-05-08T23:13:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 0x55f49a462a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 0x55f49a463140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a345560 con 0x55f49a45f720 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a345200 con 0x55f49a45ee00 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a22c6a0 con 0x55f49a459890 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f4553876b80 10 client.0 _ll_register_callbacks cb 0x55f49a34eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f453d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f453ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 0x55f49a462a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f453d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33780/0 (socket says 172.21.15.46:33780) 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f453d400700 1 -- 172.21.15.46:0/3183426443 learned_addr learned my addr 172.21.15.46:0/3183426443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.823+0000 7f453de00700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 0x55f49a463140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 -- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 msgr2=0x55f49a465760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 -- 172.21.15.46:0/3183426443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 msgr2=0x55f49a463140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 0x55f49a463140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 -- 172.21.15.46:0/3183426443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45381a63c0 con 0x55f49a45ee00 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453d400700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453de00700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 0x55f49a463140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f453ca00700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 0x55f49a462a50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f45240ca4e0 tx=0x7f452409a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748d5b4e0696ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f451ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524099590 con 0x55f49a45ee00 2024-05-08T23:13:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f49a302270 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f45240a2320 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45240a1e90 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f4553876b80 10 client.28241 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.824+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f45381107c0 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f49a302270 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4524094ab0 con 0x55f49a45ee00 2024-05-08T23:13:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 10 client.28241 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 10 client.28241 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 20 client.28241 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a473190 0x55f49a4755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 4 client.28241 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f49a428810 con 0x55f49a473190 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.825+0000 7f453d400700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a473190 0x55f49a4755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.826+0000 7f453d400700 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a473190 0x55f49a4755b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96e1c411249a07df server_cookie=b08a9f0e777927f8 in_seq=0 out_seq=0 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.826+0000 7f451ea00700 10 client.28241 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f451ea00700 1 -- 172.21.15.46:0/3183426443 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f49a428810 con 0x55f49a473190 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f451ea00700 10 client.28241 handle_command_reply: tid=0 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 client.28241 shutdown 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 msgr2=0x55f49a462a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 0x55f49a462a50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f45240ca4e0 tx=0x7f452409a6e0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 shutdown_connections 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a473190 0x55f49a4755b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45f720 0x55f49a463140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a465760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 --2- 172.21.15.46:0/3183426443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45ee00 0x55f49a462a50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f451ea00700 0 client.28241 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 >> 172.21.15.46:0/3183426443 conn(0x55f49a3b7210 msgr2=0x55f49a4535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 shutdown_connections 2024-05-08T23:13:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 1 -- 172.21.15.46:0/3183426443 wait complete. 2024-05-08T23:13:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.827+0000 7f4553876b80 20 client.28241 trim_cache size 0 max 16384 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 Processor -- start 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 -- start start 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a465290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a4622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a44e800 con 0x55f49a45ee00 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a463740 con 0x55f49a45f720 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f4553876b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a460fa0 con 0x55f49a459890 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f453d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f453d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33792/0 (socket says 172.21.15.46:33792) 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f453d400700 1 -- 172.21.15.46:0/4038101658 learned_addr learned my addr 172.21.15.46:0/4038101658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f453de00700 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a4622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.842+0000 7f453ca00700 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a465290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 -- 172.21.15.46:0/4038101658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 msgr2=0x55f49a4622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a4622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 -- 172.21.15.46:0/4038101658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 msgr2=0x55f49a465290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a465290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 -- 172.21.15.46:0/4038101658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f49a1d6b60 con 0x55f49a459890 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f453d400700 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55f49a475e50 tx=0x7f452c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1bf53f0c973414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f452b000700 1 -- 172.21.15.46:0/4038101658 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c022040 con 0x55f49a459890 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f452b000700 1 -- 172.21.15.46:0/4038101658 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f452c023040 con 0x55f49a459890 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.843+0000 7f452b000700 1 -- 172.21.15.46:0/4038101658 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c029bb0 con 0x55f49a459890 2024-05-08T23:13:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 -- 172.21.15.46:0/4038101658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 msgr2=0x55f49a453d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55f49a475e50 tx=0x7f452c014000 comp rx=0 tx=0).stop 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 -- 172.21.15.46:0/4038101658 shutdown_connections 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a4622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a459890 0x55f49a453d20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 --2- 172.21.15.46:0/4038101658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a45ee00 0x55f49a465290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 -- 172.21.15.46:0/4038101658 >> 172.21.15.46:0/4038101658 conn(0x55f49a3b7210 msgr2=0x55f49a46ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 -- 172.21.15.46:0/4038101658 shutdown_connections 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.844+0000 7f4553876b80 1 -- 172.21.15.46:0/4038101658 wait complete. 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 -- start start 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 Processor -- start 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a45ee00 0x55f49a466620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a466b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a4c8640 con 0x55f49a459890 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a4c8780 con 0x55f49a45f720 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.845+0000 7f4553876b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a4c88c0 con 0x55f49a45ee00 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f4553876b80 10 client.0 _ll_register_callbacks cb 0x55f49a34eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a45ee00 0x55f49a466620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57754/0 (socket says 172.21.15.46:57754) 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 -- 172.21.15.46:0/81023129 learned_addr learned my addr 172.21.15.46:0/81023129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453de00700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a466b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 -- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a45ee00 msgr2=0x55f49a466620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a45ee00 0x55f49a466620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 -- 172.21.15.46:0/81023129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 msgr2=0x55f49a466b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a466b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.846+0000 7f453d400700 1 -- 172.21.15.46:0/81023129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c017070 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f453d400700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f452c029480 tx=0x7f452c0294b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68207155b3613644 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f451ea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c029d40 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f452c005040 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4530007620 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f452c029ea0 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f4553876b80 10 client.18240 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.847+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f49a36f560 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4530007620 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f452c02b0b0 con 0x55f49a459890 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 10 client.18240 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:13:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 20 client.18240 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:38.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a4c9f90 0x55f49a4cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 4 client.18240 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:38.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f49a3a1ec0 con 0x55f49a4c9f90 2024-05-08T23:13:38.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.848+0000 7f453ca00700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a4c9f90 0x55f49a4cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f453ca00700 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a4c9f90 0x55f49a4cc3b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=350defe977cc8fb7 server_cookie=7400108f107fb591 in_seq=0 out_seq=0 2024-05-08T23:13:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f451ea00700 10 client.18240 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f451ea00700 1 -- 172.21.15.46:0/81023129 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f49a3a1ec0 con 0x55f49a4c9f90 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f451ea00700 10 client.18240 handle_command_reply: tid=0 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 client.18240 shutdown 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 msgr2=0x55f49a4660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f452c029480 tx=0x7f452c0294b0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 shutdown_connections 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55f49a4c9f90 0x55f49a4cc3b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f49a45f720 0x55f49a466b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f49a459890 0x55f49a4660e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 --2- 172.21.15.46:0/81023129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49a45ee00 0x55f49a466620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f451ea00700 0 client.18240 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 >> 172.21.15.46:0/81023129 conn(0x55f49a3b7210 msgr2=0x55f49a45db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 shutdown_connections 2024-05-08T23:13:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.849+0000 7f4553876b80 1 -- 172.21.15.46:0/81023129 wait complete. 2024-05-08T23:13:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f4553876b80 20 client.18240 trim_cache size 0 max 16384 2024-05-08T23:13:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 msgr2=0x7f45200a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 0x7f45200a3810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c009310 comp rx=0 tx=0).stop 2024-05-08T23:13:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 msgr2=0x7f4538199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f452400e9a0 tx=0x7f452400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 shutdown_connections 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f45200a1350 0x7f45200a3810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f453810d750 0x7f453819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45380ffba0 0x7f4538199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 --2- 172.21.15.46:0/1695017816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538100550 0x7f4538199b90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 >> 172.21.15.46:0/1695017816 conn(0x7f45380fb760 msgr2=0x7f45380fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 shutdown_connections 2024-05-08T23:13:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:38.850+0000 7f453fe00700 1 -- 172.21.15.46:0/1695017816 wait complete. 2024-05-08T23:13:38.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3000982768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:38.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:13:38.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 852106 9.65 MB/sec execute 360 sec latency 21.146 ms 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 -- 172.21.15.46:0/3438920611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640101870 msgr2=0x7f86401058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 --2- 172.21.15.46:0/3438920611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640101870 0x7f86401058b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f862c009990 tx=0x7f862c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 -- 172.21.15.46:0/3438920611 shutdown_connections 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 --2- 172.21.15.46:0/3438920611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640105fe0 0x7f864010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 --2- 172.21.15.46:0/3438920611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640100ec0 0x7f86401012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 --2- 172.21.15.46:0/3438920611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640101870 0x7f86401058b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 -- 172.21.15.46:0/3438920611 >> 172.21.15.46:0/3438920611 conn(0x7f86400fc740 msgr2=0x7f86400feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 -- 172.21.15.46:0/3438920611 shutdown_connections 2024-05-08T23:13:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.094+0000 7f863fe00700 1 -- 172.21.15.46:0/3438920611 wait complete. 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.095+0000 7f863fe00700 1 Processor -- start 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 -- start start 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 0x7f86400732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640074f60 0x7f8640073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401a9fa0 con 0x7f8640074f60 2024-05-08T23:13:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401aa110 con 0x7f8640100ec0 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401aa2d0 con 0x7f8640105fe0 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 0x7f86400732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33810/0 (socket says 172.21.15.46:33810) 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863e000700 1 -- 172.21.15.46:0/3740485785 learned_addr learned my addr 172.21.15.46:0/3740485785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 0x7f86400732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48402/0 (socket says 172.21.15.46:48402) 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.096+0000 7f863f400700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640074f60 0x7f8640073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 -- 172.21.15.46:0/3740485785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 msgr2=0x7f86400732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 0x7f86400732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 -- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640074f60 msgr2=0x7f8640073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640074f60 0x7f8640073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 -- 172.21.15.46:0/3740485785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009520 con 0x7f8640105fe0 2024-05-08T23:13:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f863e000700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.097+0000 7f8637400700 1 -- 172.21.15.46:0/3740485785 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c01d070 con 0x7f8640105fe0 2024-05-08T23:13:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.098+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c0095f0 con 0x7f8640105fe0 2024-05-08T23:13:39.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.098+0000 7f8637400700 1 -- 172.21.15.46:0/3740485785 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862c022470 con 0x7f8640105fe0 2024-05-08T23:13:39.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.098+0000 7f8637400700 1 -- 172.21.15.46:0/3740485785 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c017650 con 0x7f8640105fe0 2024-05-08T23:13:39.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.098+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86401aa890 con 0x7f8640105fe0 2024-05-08T23:13:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.102+0000 7f8637400700 1 -- 172.21.15.46:0/3740485785 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f862c00fad0 con 0x7f8640105fe0 2024-05-08T23:13:39.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.103+0000 7f865965fb80 1 Processor -- start 2024-05-08T23:13:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.103+0000 7f865965fb80 1 -- start start 2024-05-08T23:13:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f8637400700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 0x7f86240a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac042150 con 0x55c0ac0f1050 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac03fb10 con 0x55c0ac0eb890 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f865965fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac03b400 con 0x55c0ac0f1970 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f8637400700 1 -- 172.21.15.46:0/3740485785 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f862c0ce330 con 0x7f8640105fe0 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863f400700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 0x7f86240a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33816/0 (socket says 172.21.15.46:33816) 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863e000700 1 -- 172.21.15.46:0/763840594 learned_addr learned my addr 172.21.15.46:0/763840594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863ea00700 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.104+0000 7f863f400700 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 -- 172.21.15.46:0/763840594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 msgr2=0x55c0ac0ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 -- 172.21.15.46:0/763840594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1050 msgr2=0x55c0ac0f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 -- 172.21.15.46:0/763840594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0abebc3b0 con 0x55c0ac0f1970 2024-05-08T23:13:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863e000700 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f862c097680 tx=0x7f862c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ead47fc2a05910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f862be00700 1 -- 172.21.15.46:0/763840594 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c09aae0 con 0x55c0ac0f1970 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f862be00700 1 -- 172.21.15.46:0/763840594 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862c007a20 con 0x55c0ac0f1970 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f863f400700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 0x7f86240a3870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8638009ec0 tx=0x7f86380134a0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.105+0000 7f862be00700 1 -- 172.21.15.46:0/763840594 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0a3570 con 0x55c0ac0f1970 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 -- 172.21.15.46:0/763840594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 msgr2=0x55c0ac0eb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f862c097680 tx=0x7f862c009310 comp rx=0 tx=0).stop 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 -- 172.21.15.46:0/763840594 shutdown_connections 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 --2- 172.21.15.46:0/763840594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0eb350 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 -- 172.21.15.46:0/763840594 >> 172.21.15.46:0/763840594 conn(0x55c0ac049210 msgr2=0x55c0ac0e62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 -- 172.21.15.46:0/763840594 shutdown_connections 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.106+0000 7f865965fb80 1 -- 172.21.15.46:0/763840594 wait complete. 2024-05-08T23:13:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 -- start start 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 Processor -- start 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0abfd7560 con 0x55c0ac0f1970 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0abfd7200 con 0x55c0ac0eb890 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.107+0000 7f865965fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0abebe6a0 con 0x55c0ac0f1050 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f865965fb80 10 client.0 _ll_register_callbacks cb 0x55c0abfe0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48422/0 (socket says 172.21.15.46:48422) 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 -- 172.21.15.46:0/1291461527 learned_addr learned my addr 172.21.15.46:0/1291461527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863f400700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863e000700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 -- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 msgr2=0x55c0ac0f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 -- 172.21.15.46:0/1291461527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 msgr2=0x55c0ac0f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 -- 172.21.15.46:0/1291461527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86401aa890 con 0x55c0ac0eb890 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f863ea00700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86300119b0 tx=0x7f8630011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f19625afadeec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.108+0000 7f8628c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630016070 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0abf94270 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f863000fa40 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f863000fc00 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f865965fb80 10 client.28259 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c0abe68b60 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.109+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0abf94270 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8630019390 con 0x55c0ac0eb890 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 10 client.28259 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 10 client.28259 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 20 client.28259 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac104fd0 0x55c0ac1073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 4 client.28259 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0ac0ba810 con 0x55c0ac104fd0 2024-05-08T23:13:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f863e000700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac104fd0 0x55c0ac1073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f863e000700 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac104fd0 0x55c0ac1073f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9216420b9e3dde9 server_cookie=e9b5a5470218dd63 in_seq=0 out_seq=0 2024-05-08T23:13:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.110+0000 7f8628c00700 10 client.28259 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f8628c00700 1 -- 172.21.15.46:0/1291461527 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0ac0ba810 con 0x55c0ac104fd0 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f8628c00700 10 client.28259 handle_command_reply: tid=0 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 client.28259 shutdown 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 msgr2=0x55c0ac0f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86300119b0 tx=0x7f8630011cc0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 shutdown_connections 2024-05-08T23:13:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac104fd0 0x55c0ac1073f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f7890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f8628c00700 0 client.28259 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 --2- 172.21.15.46:0/1291461527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.111+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 >> 172.21.15.46:0/1291461527 conn(0x55c0ac049210 msgr2=0x55c0ac0e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.112+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 shutdown_connections 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.112+0000 7f865965fb80 1 -- 172.21.15.46:0/1291461527 wait complete. 2024-05-08T23:13:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.112+0000 7f865965fb80 20 client.28259 trim_cache size 0 max 16384 2024-05-08T23:13:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.127+0000 7f865965fb80 1 Processor -- start 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.127+0000 7f865965fb80 1 -- start start 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.127+0000 7f865965fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f865965fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f865965fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f865965fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac0f5890 con 0x55c0ac0f1970 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f865965fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac0e7820 con 0x55c0ac0eb890 2024-05-08T23:13:39.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f865965fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac10c5c0 con 0x55c0ac0f1050 2024-05-08T23:13:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33842/0 (socket says 172.21.15.46:33842) 2024-05-08T23:13:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 -- 172.21.15.46:0/1359035231 learned_addr learned my addr 172.21.15.46:0/1359035231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863f400700 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 -- 172.21.15.46:0/1359035231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 msgr2=0x55c0ac0f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 -- 172.21.15.46:0/1359035231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 msgr2=0x55c0ac0f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.128+0000 7f863e000700 1 -- 172.21.15.46:0/1359035231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0ac001560 con 0x55c0ac0f1050 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.129+0000 7f863e000700 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f862c006010 tx=0x7f862c0a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9486014381960bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.129+0000 7f8634c00700 1 -- 172.21.15.46:0/1359035231 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c09aae0 con 0x55c0ac0f1050 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.129+0000 7f8634c00700 1 -- 172.21.15.46:0/1359035231 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862c09b180 con 0x55c0ac0f1050 2024-05-08T23:13:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.129+0000 7f8634c00700 1 -- 172.21.15.46:0/1359035231 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c00fd80 con 0x55c0ac0f1050 2024-05-08T23:13:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- 172.21.15.46:0/1359035231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 msgr2=0x55c0ac0f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f862c006010 tx=0x7f862c0a1040 comp rx=0 tx=0).stop 2024-05-08T23:13:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- 172.21.15.46:0/1359035231 shutdown_connections 2024-05-08T23:13:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0f1970 0x55c0ac0f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0eb890 0x55c0ac0f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 --2- 172.21.15.46:0/1359035231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1050 0x55c0ac0f7900 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- 172.21.15.46:0/1359035231 >> 172.21.15.46:0/1359035231 conn(0x55c0ac049210 msgr2=0x55c0ac0eebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- 172.21.15.46:0/1359035231 shutdown_connections 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- 172.21.15.46:0/1359035231 wait complete. 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.130+0000 7f865965fb80 1 -- start start 2024-05-08T23:13:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.131+0000 7f865965fb80 1 Processor -- start 2024-05-08T23:13:39.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.131+0000 7f865965fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.131+0000 7f865965fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0f1050 0x55c0ac0edca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.131+0000 7f865965fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f865965fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac0ecac0 con 0x55c0ac0eb890 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f865965fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac0ecc30 con 0x55c0ac0f1050 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f865965fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0ac0ecdf0 con 0x55c0ac0f1970 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f865965fb80 10 client.0 _ll_register_callbacks cb 0x55c0abfe0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57812/0 (socket says 172.21.15.46:57812) 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 -- 172.21.15.46:0/3397518420 learned_addr learned my addr 172.21.15.46:0/3397518420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863e000700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0f1050 0x55c0ac0edca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 -- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 msgr2=0x55c0ac0ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 -- 172.21.15.46:0/3397518420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0f1050 msgr2=0x55c0ac0edca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0f1050 0x55c0ac0edca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 -- 172.21.15.46:0/3397518420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c0b2040 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f863ea00700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8630019770 tx=0x7f8630005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e263c2c47e55d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.132+0000 7f8628c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630016070 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f862c007a20 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8630000e40 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8630019ea0 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f865965fb80 10 client.18258 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.133+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c0abec0cd0 con 0x55c0ac0eb890 2024-05-08T23:13:39.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862c007a20 con 0x55c0ac0eb890 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8630032060 con 0x55c0ac0eb890 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 10 client.18258 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 10 client.18258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 20 client.18258 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac15c070 0x55c0ac15e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 4 client.18258 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0ac033ec0 con 0x55c0ac15c070 2024-05-08T23:13:39.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.134+0000 7f863e000700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac15c070 0x55c0ac15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f863e000700 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac15c070 0x55c0ac15e490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb08f9f5c8844237 server_cookie=304f9d1cb469b497 in_seq=0 out_seq=0 2024-05-08T23:13:39.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f8628c00700 10 client.18258 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f8628c00700 1 -- 172.21.15.46:0/3397518420 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0ac033ec0 con 0x55c0ac15c070 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f8628c00700 10 client.18258 handle_command_reply: tid=0 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 client.18258 shutdown 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 msgr2=0x55c0ac0fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8630019770 tx=0x7f8630005e50 comp rx=0 tx=0).stop 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 shutdown_connections 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55c0ac15c070 0x55c0ac15e490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0ac0f1970 0x55c0ac0ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0ac0eb890 0x55c0ac0fc4d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 --2- 172.21.15.46:0/3397518420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c0ac0f1050 0x55c0ac0edca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f8628c00700 0 client.18258 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 >> 172.21.15.46:0/3397518420 conn(0x55c0ac049210 msgr2=0x55c0ac0e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 shutdown_connections 2024-05-08T23:13:39.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.135+0000 7f865965fb80 1 -- 172.21.15.46:0/3397518420 wait complete. 2024-05-08T23:13:39.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f865965fb80 20 client.18258 trim_cache size 0 max 16384 2024-05-08T23:13:39.140 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:39.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 msgr2=0x7f86240a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 0x7f86240a3870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8638009ec0 tx=0x7f86380134a0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 msgr2=0x7f86400737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c005740 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 shutdown_connections 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f86240a13b0 0x7f86240a3870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8640074f60 0x7f8640073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8640100ec0 0x7f86400732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 --2- 172.21.15.46:0/3740485785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8640105fe0 0x7f86400737e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.136+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 >> 172.21.15.46:0/3740485785 conn(0x7f86400fc740 msgr2=0x7f86400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.137+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 shutdown_connections 2024-05-08T23:13:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.137+0000 7f863fe00700 1 -- 172.21.15.46:0/3740485785 wait complete. 2024-05-08T23:13:39.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:13:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3000982768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:39.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3000982768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:13:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 -- 172.21.15.46:0/2754919201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 msgr2=0x7f222c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 --2- 172.21.15.46:0/2754919201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c101460 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f221c009990 tx=0x7f221c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 -- 172.21.15.46:0/2754919201 shutdown_connections 2024-05-08T23:13:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 --2- 172.21.15.46:0/2754919201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 0x7f222c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 --2- 172.21.15.46:0/2754919201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c101460 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 --2- 172.21.15.46:0/2754919201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c101a30 0x7f222c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 -- 172.21.15.46:0/2754919201 >> 172.21.15.46:0/2754919201 conn(0x7f222c0fc8e0 msgr2=0x7f222c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 -- 172.21.15.46:0/2754919201 shutdown_connections 2024-05-08T23:13:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.410+0000 7f2231e00700 1 -- 172.21.15.46:0/2754919201 wait complete. 2024-05-08T23:13:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.412+0000 7f2231e00700 1 Processor -- start 2024-05-08T23:13:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.412+0000 7f2231e00700 1 -- start start 2024-05-08T23:13:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.412+0000 7f2231e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f2231e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 0x7f222c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f2231e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c19de60 0x7f222c19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f2231e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19e8e0 con 0x7f222c1061a0 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f2231e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19ea50 con 0x7f222c19de60 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f2231e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19ec10 con 0x7f222c101080 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33880/0 (socket says 172.21.15.46:33880) 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 -- 172.21.15.46:0/2108137165 learned_addr learned my addr 172.21.15.46:0/2108137165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222b400700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c19de60 0x7f222c19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222a000700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 0x7f222c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 -- 172.21.15.46:0/2108137165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c19de60 msgr2=0x7f222c19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c19de60 0x7f222c19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 -- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 msgr2=0x7f222c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 0x7f222c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.413+0000 7f222aa00700 1 -- 172.21.15.46:0/2108137165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c0095f0 con 0x7f222c101080 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f222aa00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f221c004760 tx=0x7f221c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f2230a00700 1 -- 172.21.15.46:0/2108137165 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c01d070 con 0x7f222c101080 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f2230a00700 1 -- 172.21.15.46:0/2108137165 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c022470 con 0x7f222c101080 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1a6110 con 0x7f222c101080 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f2230a00700 1 -- 172.21.15.46:0/2108137165 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c017650 con 0x7f222c101080 2024-05-08T23:13:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.414+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1a6590 con 0x7f222c101080 2024-05-08T23:13:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.417+0000 7f2230a00700 1 -- 172.21.15.46:0/2108137165 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f221c0225e0 con 0x7f222c101080 2024-05-08T23:13:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.418+0000 7f2245833b80 1 Processor -- start 2024-05-08T23:13:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.418+0000 7f2245833b80 1 -- start start 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.418+0000 7f2230a00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 0x7f22100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2230a00700 1 -- 172.21.15.46:0/2108137165 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f221c0ce1e0 con 0x7f222c101080 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdc31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222b400700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 0x7f22100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdbdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fd14150 con 0x56047fdbd890 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fd11b10 con 0x56047fdc2e00 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f2245833b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fd0d400 con 0x56047fdc3720 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdc31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33884/0 (socket says 172.21.15.46:33884) 2024-05-08T23:13:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222a000700 1 -- 172.21.15.46:0/2947406891 learned_addr learned my addr 172.21.15.46:0/2947406891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.419+0000 7f222b400700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdbdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 -- 172.21.15.46:0/2947406891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 msgr2=0x56047fdc31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdc31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 -- 172.21.15.46:0/2947406891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 msgr2=0x56047fdbdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdbdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 -- 172.21.15.46:0/2947406891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047fb8e3b0 con 0x56047fdc3720 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222aa00700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdc31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222b400700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdbdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222b400700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 0x7f22100a37d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f222400f870 tx=0x7f222400fc30 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f222a000700 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2218009cf0 tx=0x7f2218011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c7a2ea321426e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f2217e00700 1 -- 172.21.15.46:0/2947406891 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218011d80 con 0x56047fdc3720 2024-05-08T23:13:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f2217e00700 1 -- 172.21.15.46:0/2947406891 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22180044f0 con 0x56047fdc3720 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.420+0000 7f2217e00700 1 -- 172.21.15.46:0/2947406891 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22180193e0 con 0x56047fdc3720 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 -- 172.21.15.46:0/2947406891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 msgr2=0x56047fdbd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2218009cf0 tx=0x7f2218011440 comp rx=0 tx=0).stop 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 -- 172.21.15.46:0/2947406891 shutdown_connections 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdbdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdc31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 --2- 172.21.15.46:0/2947406891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fdbd350 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 -- 172.21.15.46:0/2947406891 >> 172.21.15.46:0/2947406891 conn(0x56047fd1b210 msgr2=0x56047fdb7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 -- 172.21.15.46:0/2947406891 shutdown_connections 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.421+0000 7f2245833b80 1 -- 172.21.15.46:0/2947406891 wait complete. 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 -- start start 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 Processor -- start 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdc2e00 0x56047fdc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdc7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fca9560 con 0x56047fdc2e00 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fca9200 con 0x56047fdc3720 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fb906a0 con 0x56047fdbd890 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.422+0000 7f2245833b80 10 client.0 _ll_register_callbacks cb 0x56047fcb2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33890/0 (socket says 172.21.15.46:33890) 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 -- 172.21.15.46:0/2629977971 learned_addr learned my addr 172.21.15.46:0/2629977971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222b400700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdc7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222a000700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdc2e00 0x56047fdc6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 -- 172.21.15.46:0/2629977971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 msgr2=0x56047fdc7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdc7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 -- 172.21.15.46:0/2629977971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdc2e00 msgr2=0x56047fdc6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdc2e00 0x56047fdc6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 -- 172.21.15.46:0/2629977971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1a6590 con 0x56047fdbd890 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f222aa00700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f221c004730 tx=0x7f221c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5e6f5fa64bc782 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f2214c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c09a600 con 0x56047fdbd890 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f221c09b610 con 0x56047fdbd890 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.423+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56047fc66270 con 0x56047fdbd890 2024-05-08T23:13:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221c09af10 con 0x56047fdbd890 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 10 client.28357 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2218009920 con 0x56047fdbd890 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c0b9030 con 0x56047fdbd890 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f221c0a2ee0 con 0x56047fdbd890 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 10 client.28357 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 10 client.28357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 20 client.28357 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fdd6e20 0x56047fdd9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 4 client.28357 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56047fd8c810 con 0x56047fdd6e20 2024-05-08T23:13:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.424+0000 7f222a000700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fdd6e20 0x56047fdd9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.425+0000 7f222a000700 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fdd6e20 0x56047fdd9210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70c9c325d93096ec server_cookie=5368d50641514389 in_seq=0 out_seq=0 2024-05-08T23:13:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.425+0000 7f2214c00700 10 client.28357 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2214c00700 1 -- 172.21.15.46:0/2629977971 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56047fd8c810 con 0x56047fdd6e20 2024-05-08T23:13:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2214c00700 10 client.28357 handle_command_reply: tid=0 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 client.28357 shutdown 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 msgr2=0x56047fdc9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f221c004730 tx=0x7f221c009310 comp rx=0 tx=0).stop 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 shutdown_connections 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdbd890 0x56047fdc9780 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fdd6e20 0x56047fdd9210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 --2- 172.21.15.46:0/2629977971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdc2e00 0x56047fdc6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 >> 172.21.15.46:0/2629977971 conn(0x56047fd1b210 msgr2=0x56047fdb75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2214c00700 0 client.28357 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 shutdown_connections 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 1 -- 172.21.15.46:0/2629977971 wait complete. 2024-05-08T23:13:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.426+0000 7f2245833b80 20 client.28357 trim_cache size 0 max 16384 2024-05-08T23:13:39.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.440+0000 7f2245833b80 1 Processor -- start 2024-05-08T23:13:39.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.440+0000 7f2245833b80 1 -- start start 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdd2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fdc9c80 con 0x56047fdbd890 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fdb9e50 con 0x56047fdc3720 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f2245833b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fdbd700 con 0x56047fdc2e00 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdd2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33896/0 (socket says 172.21.15.46:33896) 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222a000700 1 -- 172.21.15.46:0/2358908091 learned_addr learned my addr 172.21.15.46:0/2358908091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.441+0000 7f222a000700 1 -- 172.21.15.46:0/2358908091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 msgr2=0x56047fdce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f222a000700 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f222a000700 1 -- 172.21.15.46:0/2358908091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 msgr2=0x56047fdd2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f222a000700 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdd2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f222a000700 1 -- 172.21.15.46:0/2358908091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047fb3ab60 con 0x56047fdc2e00 2024-05-08T23:13:39.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f222a000700 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2218011790 tx=0x7f22180046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8776218d9b5371e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f2221200700 1 -- 172.21.15.46:0/2358908091 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218005d60 con 0x56047fdc2e00 2024-05-08T23:13:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f2221200700 1 -- 172.21.15.46:0/2358908091 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2218005500 con 0x56047fdc2e00 2024-05-08T23:13:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.442+0000 7f2221200700 1 -- 172.21.15.46:0/2358908091 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218003820 con 0x56047fdc2e00 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- 172.21.15.46:0/2358908091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 msgr2=0x56047fdb7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2218011790 tx=0x7f22180046a0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- 172.21.15.46:0/2358908091 shutdown_connections 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc3720 0x56047fdce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdd2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 --2- 172.21.15.46:0/2358908091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc2e00 0x56047fdb7d20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- 172.21.15.46:0/2358908091 >> 172.21.15.46:0/2358908091 conn(0x56047fd1b210 msgr2=0x56047fdbdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- 172.21.15.46:0/2358908091 shutdown_connections 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- 172.21.15.46:0/2358908091 wait complete. 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 -- start start 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.443+0000 7f2245833b80 1 Processor -- start 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdcc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdcc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fe2c790 con 0x56047fdbd890 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fe2c8d0 con 0x56047fdc2e00 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047fe2ca10 con 0x56047fdc3720 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f2245833b80 10 client.0 _ll_register_callbacks cb 0x56047fcb2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdcc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdcc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33904/0 (socket says 172.21.15.46:33904) 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222b400700 1 -- 172.21.15.46:0/1802358344 learned_addr learned my addr 172.21.15.46:0/1802358344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.444+0000 7f222aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdcc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57872/0 (socket says 172.21.15.46:57872) 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 -- 172.21.15.46:0/1802358344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 msgr2=0x56047fdcc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdcc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 -- 172.21.15.46:0/1802358344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 msgr2=0x56047fdcc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdcc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 -- 172.21.15.46:0/1802358344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221801e040 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f222b400700 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2224018b00 tx=0x7f2224004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405423fc26bda36d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2214c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400e040 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2224003c00 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22180044f0 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2224003dc0 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2245833b80 10 client.28363 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.445+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56047fcd3560 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22180044f0 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2224004520 con 0x56047fdc3720 2024-05-08T23:13:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 10 client.28363 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 10 client.28363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 20 client.28363 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fe2e210 0x56047fe30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 4 client.28363 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56047fd05ec0 con 0x56047fe2e210 2024-05-08T23:13:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.446+0000 7f222aa00700 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fe2e210 0x56047fe30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f222aa00700 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fe2e210 0x56047fe30630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a82f9a3a79b13e1e server_cookie=db181a772a863e45 in_seq=0 out_seq=0 2024-05-08T23:13:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2214c00700 10 client.28363 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2214c00700 1 -- 172.21.15.46:0/1802358344 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56047fd05ec0 con 0x56047fe2e210 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2214c00700 10 client.28363 handle_command_reply: tid=0 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 client.28363 shutdown 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 msgr2=0x56047fe2c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2224018b00 tx=0x7f2224004b40 comp rx=0 tx=0).stop 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 shutdown_connections 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56047fdc3720 0x56047fe2c250 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x56047fe2e210 0x56047fe30630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56047fdbd890 0x56047fdcc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 --2- 172.21.15.46:0/1802358344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56047fdc2e00 0x56047fdcc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 >> 172.21.15.46:0/1802358344 conn(0x56047fd1b210 msgr2=0x56047fdc0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2214c00700 0 client.28363 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:13:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 shutdown_connections 2024-05-08T23:13:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 1 -- 172.21.15.46:0/1802358344 wait complete. 2024-05-08T23:13:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.447+0000 7f2245833b80 20 client.28363 trim_cache size 0 max 16384 2024-05-08T23:13:39.453 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 msgr2=0x7f22100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 0x7f22100a37d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f222400f870 tx=0x7f222400fc30 comp rx=0 tx=0).stop 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 msgr2=0x7f222c1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f221c004760 tx=0x7f221c004790 comp rx=0 tx=0).stop 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 shutdown_connections 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c19de60 0x7f222c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c101080 0x7f222c1995d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f22100a1310 0x7f22100a37d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 --2- 172.21.15.46:0/2108137165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f222c1061a0 0x7f222c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 >> 172.21.15.46:0/2108137165 conn(0x7f222c0fc8e0 msgr2=0x7f222c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 shutdown_connections 2024-05-08T23:13:39.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.448+0000 7f2231e00700 1 -- 172.21.15.46:0/2108137165 wait complete. 2024-05-08T23:13:39.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:13:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 -- 172.21.15.46:0/2320187354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4478101a10 msgr2=0x7f4478105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 --2- 172.21.15.46:0/2320187354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4478101a10 0x7f4478105a50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f44640098f0 tx=0x7f4464009c00 comp rx=0 tx=0).stop 2024-05-08T23:13:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 -- 172.21.15.46:0/2320187354 shutdown_connections 2024-05-08T23:13:39.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 --2- 172.21.15.46:0/2320187354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 0x7f447810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 --2- 172.21.15.46:0/2320187354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 --2- 172.21.15.46:0/2320187354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4478101a10 0x7f4478105a50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 -- 172.21.15.46:0/2320187354 >> 172.21.15.46:0/2320187354 conn(0x7f44780fc900 msgr2=0x7f44780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.677+0000 7f447fa00700 1 -- 172.21.15.46:0/2320187354 shutdown_connections 2024-05-08T23:13:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.678+0000 7f447fa00700 1 -- 172.21.15.46:0/2320187354 wait complete. 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 Processor -- start 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 -- start start 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 0x7f4478199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 0x7f447819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819e980 con 0x7f4478101060 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819eaf0 con 0x7f4478106180 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819ecb0 con 0x7f447819df00 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 0x7f447819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57896/0 (socket says 172.21.15.46:57896) 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f447d000700 1 -- 172.21.15.46:0/1274611240 learned_addr learned my addr 172.21.15.46:0/1274611240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.679+0000 7f4477e00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 0x7f4478199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 -- 172.21.15.46:0/1274611240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 msgr2=0x7f4478199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 0x7f4478199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 -- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 msgr2=0x7f4478199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 -- 172.21.15.46:0/1274611240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0095f0 con 0x7f447819df00 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447d000700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447da00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 0x7f447819e380 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f447000efc0 tx=0x7f447000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f4475600700 1 -- 172.21.15.46:0/1274611240 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470018070 con 0x7f447819df00 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f4475600700 1 -- 172.21.15.46:0/1274611240 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447000f040 con 0x7f447819df00 2024-05-08T23:13:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4464009520 con 0x7f447819df00 2024-05-08T23:13:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f4475600700 1 -- 172.21.15.46:0/1274611240 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000cd40 con 0x7f447819df00 2024-05-08T23:13:39.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.680+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44781a6500 con 0x7f447819df00 2024-05-08T23:13:39.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.684+0000 7f4475600700 1 -- 172.21.15.46:0/1274611240 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4470003a00 con 0x7f447819df00 2024-05-08T23:13:39.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 Processor -- start 2024-05-08T23:13:39.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 -- start start 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f4475600700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 0x7f44600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aa1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383aa1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643839f8150 con 0x564383aa1890 2024-05-08T23:13:39.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643839f5b10 con 0x564383aa6e00 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f449341bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643839f1400 con 0x564383aa7720 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f4475600700 1 -- 172.21.15.46:0/1274611240 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f44700ccc60 con 0x7f447819df00 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f447d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.685+0000 7f447d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48506/0 (socket says 172.21.15.46:48506) 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447d000700 1 -- 172.21.15.46:0/3448875310 learned_addr learned my addr 172.21.15.46:0/3448875310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f4477e00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 0x7f44600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f4477e00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aa1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383aa1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 -- 172.21.15.46:0/3448875310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 msgr2=0x564383aa1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aa1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 -- 172.21.15.46:0/3448875310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 msgr2=0x564383aa71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447da00700 1 -- 172.21.15.46:0/3448875310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643838723b0 con 0x564383aa1890 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f4477e00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aa1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f447d000700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.686+0000 7f4477e00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 0x7f44600a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4464005e90 tx=0x7f446400bf90 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.687+0000 7f447da00700 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383aa1cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44700c9f40 tx=0x7f4470099760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660d4f0b4f4d2b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.687+0000 7f446a000700 1 -- 172.21.15.46:0/3448875310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447001b320 con 0x564383aa1890 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.687+0000 7f446a000700 1 -- 172.21.15.46:0/3448875310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470004ad0 con 0x564383aa1890 2024-05-08T23:13:39.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.687+0000 7f446a000700 1 -- 172.21.15.46:0/3448875310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470099bf0 con 0x564383aa1890 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- 172.21.15.46:0/3448875310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 msgr2=0x564383aa1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383aa1cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44700c9f40 tx=0x7f4470099760 comp rx=0 tx=0).stop 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- 172.21.15.46:0/3448875310 shutdown_connections 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383aa1cd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aa71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 --2- 172.21.15.46:0/3448875310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aa1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- 172.21.15.46:0/3448875310 >> 172.21.15.46:0/3448875310 conn(0x5643839ff210 msgr2=0x564383a9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- 172.21.15.46:0/3448875310 shutdown_connections 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- 172.21.15.46:0/3448875310 wait complete. 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.688+0000 7f449341bb80 1 -- start start 2024-05-08T23:13:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 Processor -- start 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 0x564383aaafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438398d560 con 0x564383aa6e00 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438398d200 con 0x564383aa1890 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643838746a0 con 0x564383aa7720 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f449341bb80 10 client.0 _ll_register_callbacks cb 0x564383996b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f447d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f447da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f4477e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 0x564383aaafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f447d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48512/0 (socket says 172.21.15.46:48512) 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f447da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33930/0 (socket says 172.21.15.46:33930) 2024-05-08T23:13:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.689+0000 7f447d000700 1 -- 172.21.15.46:0/3818660856 learned_addr learned my addr 172.21.15.46:0/3818660856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 -- 172.21.15.46:0/3818660856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 msgr2=0x564383aab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 -- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 msgr2=0x564383aaafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 0x564383aaafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 -- 172.21.15.46:0/3818660856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447810e570 con 0x564383aa1890 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447da00700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f4477e00700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 0x564383aaafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f447d000700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f446c012e50 tx=0x7f446c00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a628b5eedd03ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f445ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.690+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c017070 con 0x564383aa1890 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f446c00fc30 con 0x564383aa1890 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56438394a270 con 0x564383aa1890 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f446c0122e0 con 0x564383aa1890 2024-05-08T23:13:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f449341bb80 10 client.28295 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f44781a6500 con 0x564383aa1890 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56438394a270 con 0x564383aa1890 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f446c0104c0 con 0x564383aa1890 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f449341bb80 10 client.28295 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.691+0000 7f449341bb80 10 client.28295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f449341bb80 20 client.28295 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383abb260 0x564383abd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f449341bb80 4 client.28295 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564383a70810 con 0x564383abb260 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f4477e00700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383abb260 0x564383abd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f4477e00700 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383abb260 0x564383abd680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b731755d4372543 server_cookie=3026e1afd0bb0191 in_seq=0 out_seq=0 2024-05-08T23:13:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.692+0000 7f445ea00700 10 client.28295 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f445ea00700 1 -- 172.21.15.46:0/3818660856 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564383a70810 con 0x564383abb260 2024-05-08T23:13:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f445ea00700 10 client.28295 handle_command_reply: tid=0 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 client.28295 shutdown 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 msgr2=0x564383aaaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f446c012e50 tx=0x7f446c00f6a0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 shutdown_connections 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383aab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa1890 0x564383aaaa90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383abb260 0x564383abd680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 --2- 172.21.15.46:0/3818660856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa6e00 0x564383aaafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 >> 172.21.15.46:0/3818660856 conn(0x5643839ff210 msgr2=0x564383a9b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.693+0000 7f445ea00700 0 client.28295 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.694+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 shutdown_connections 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.694+0000 7f449341bb80 1 -- 172.21.15.46:0/3818660856 wait complete. 2024-05-08T23:13:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.694+0000 7f449341bb80 20 client.28295 trim_cache size 0 max 16384 2024-05-08T23:13:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.709+0000 7f449341bb80 1 Processor -- start 2024-05-08T23:13:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.709+0000 7f449341bb80 1 -- start start 2024-05-08T23:13:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.709+0000 7f449341bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.709+0000 7f449341bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383ab72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f449341bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa7720 0x564383aa62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f449341bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383ac2890 con 0x564383aa7720 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f449341bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383a96800 con 0x564383aa6e00 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f449341bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383aa8fe0 con 0x564383aa1890 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f4477e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383ab72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f4477e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383ab72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48526/0 (socket says 172.21.15.46:48526) 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f4477e00700 1 -- 172.21.15.46:0/3020152321 learned_addr learned my addr 172.21.15.46:0/3020152321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33946/0 (socket says 172.21.15.46:33946) 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447da00700 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa7720 0x564383aa62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 -- 172.21.15.46:0/3020152321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 msgr2=0x564383ab72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383ab72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 -- 172.21.15.46:0/3020152321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa7720 msgr2=0x564383aa62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa7720 0x564383aa62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 -- 172.21.15.46:0/3020152321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56438381eb60 con 0x564383aa1890 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.710+0000 7f447d000700 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f446c0108d0 tx=0x7f446c010a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704abd95349fc77f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f446aa00700 1 -- 172.21.15.46:0/3020152321 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c017030 con 0x564383aa1890 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f446aa00700 1 -- 172.21.15.46:0/3020152321 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446c018040 con 0x564383aa1890 2024-05-08T23:13:39.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f446aa00700 1 -- 172.21.15.46:0/3020152321 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00fe90 con 0x564383aa1890 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f449341bb80 1 -- 172.21.15.46:0/3020152321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 msgr2=0x564383aa1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f449341bb80 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f446c0108d0 tx=0x7f446c010a70 comp rx=0 tx=0).stop 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.711+0000 7f449341bb80 1 -- 172.21.15.46:0/3020152321 shutdown_connections 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa7720 0x564383aa62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa1890 0x564383aa1350 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 --2- 172.21.15.46:0/3020152321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383ab72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 -- 172.21.15.46:0/3020152321 >> 172.21.15.46:0/3020152321 conn(0x5643839ff210 msgr2=0x564383aa5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 -- 172.21.15.46:0/3020152321 shutdown_connections 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 -- 172.21.15.46:0/3020152321 wait complete. 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 -- start start 2024-05-08T23:13:39.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.712+0000 7f449341bb80 1 Processor -- start 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383ab2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aafad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383ab08f0 con 0x564383aa1890 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383b10760 con 0x564383aa6e00 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564383b108a0 con 0x564383aa7720 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f449341bb80 10 client.0 _ll_register_callbacks cb 0x564383996b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f447da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f447d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383ab2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f447da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33952/0 (socket says 172.21.15.46:33952) 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f4477e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aafad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.713+0000 7f447da00700 1 -- 172.21.15.46:0/2200782468 learned_addr learned my addr 172.21.15.46:0/2200782468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 -- 172.21.15.46:0/2200782468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 msgr2=0x564383aafad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aafad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 -- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 msgr2=0x564383ab2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383ab2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 -- 172.21.15.46:0/2200782468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44781a6500 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f447da00700 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f44700090a0 tx=0x7f44700ca4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a31033dc7065a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f445ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700b6070 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4470084040 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4470004ad0 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f447009d7c0 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.714+0000 7f449341bb80 10 client.28381 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5643839b7560 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447007c080 con 0x564383aa7720 2024-05-08T23:13:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4470065170 con 0x564383aa7720 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 10 client.28381 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 10 client.28381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 20 client.28381 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383b11aa0 0x564383b13e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 4 client.28381 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643839e9ec0 con 0x564383b11aa0 2024-05-08T23:13:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.715+0000 7f447d000700 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383b11aa0 0x564383b13e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f447d000700 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383b11aa0 0x564383b13e90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3bf4c45c153cd90a server_cookie=8a817ce64c5a2619 in_seq=0 out_seq=0 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f445ea00700 10 client.28381 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f445ea00700 1 -- 172.21.15.46:0/2200782468 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643839e9ec0 con 0x564383b11aa0 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f445ea00700 10 client.28381 handle_command_reply: tid=0 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f449341bb80 1 client.28381 shutdown 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 msgr2=0x564383ab0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.716+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f44700090a0 tx=0x7f44700ca4a0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 shutdown_connections 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564383aa7720 0x564383ab0210 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564383b11aa0 0x564383b13e90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564383aa1890 0x564383ab2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f445ea00700 0 client.28381 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 --2- 172.21.15.46:0/2200782468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564383aa6e00 0x564383aafad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 >> 172.21.15.46:0/2200782468 conn(0x5643839ff210 msgr2=0x564383aa6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 shutdown_connections 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 1 -- 172.21.15.46:0/2200782468 wait complete. 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.717+0000 7f449341bb80 20 client.28381 trim_cache size 0 max 16384 2024-05-08T23:13:39.722 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 msgr2=0x7f44600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 0x7f44600a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4464005e90 tx=0x7f446400bf90 comp rx=0 tx=0).stop 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 msgr2=0x7f447819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 0x7f447819e380 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f447000efc0 tx=0x7f447000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 shutdown_connections 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f447819df00 0x7f447819e380 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f44600a1290 0x7f44600a3750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4478101060 0x7f4478199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 --2- 172.21.15.46:0/1274611240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478106180 0x7f4478199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 >> 172.21.15.46:0/1274611240 conn(0x7f44780fc900 msgr2=0x7f44780fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 shutdown_connections 2024-05-08T23:13:39.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.718+0000 7f447fa00700 1 -- 172.21.15.46:0/1274611240 wait complete. 2024-05-08T23:13:39.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:13:39.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 852930 9.63 MB/sec execute 361 sec latency 40.317 ms 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 -- 172.21.15.46:0/2493741143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48103c60 msgr2=0x7f8d48107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2493741143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48103c60 0x7f8d48107ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009990 tx=0x7f8d34009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 -- 172.21.15.46:0/2493741143 shutdown_connections 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2493741143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d4810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2493741143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d481032b0 0x7f8d48103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2493741143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48103c60 0x7f8d48107ca0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 -- 172.21.15.46:0/2493741143 >> 172.21.15.46:0/2493741143 conn(0x7f8d480feb10 msgr2=0x7f8d48100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 -- 172.21.15.46:0/2493741143 shutdown_connections 2024-05-08T23:13:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.980+0000 7f8d4e800700 1 -- 172.21.15.46:0/2493741143 wait complete. 2024-05-08T23:13:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.981+0000 7f8d4e800700 1 Processor -- start 2024-05-08T23:13:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.981+0000 7f8d4e800700 1 -- start start 2024-05-08T23:13:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d480759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 0x7f8d481a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48079660 con 0x7f8d48075f30 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481aa080 con 0x7f8d481083d0 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d4e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481aa240 con 0x7f8d481032b0 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33970/0 (socket says 172.21.15.46:33970) 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d47400700 1 -- 172.21.15.46:0/2102114888 learned_addr learned my addr 172.21.15.46:0/2102114888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d480759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.982+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 0x7f8d481a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 -- 172.21.15.46:0/2102114888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 msgr2=0x7f8d480759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d480759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 -- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 msgr2=0x7f8d481a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 0x7f8d481a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 -- 172.21.15.46:0/2102114888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340095f0 con 0x7f8d481032b0 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d480759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 0x7f8d481a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d47400700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9a0 tx=0x7f8d3800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d3fe00700 1 -- 172.21.15.46:0/2102114888 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38009790 con 0x7f8d481032b0 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d3fe00700 1 -- 172.21.15.46:0/2102114888 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d3800cbd0 con 0x7f8d481032b0 2024-05-08T23:13:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d3fe00700 1 -- 172.21.15.46:0/2102114888 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38018680 con 0x7f8d481032b0 2024-05-08T23:13:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.983+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d481aa520 con 0x7f8d481032b0 2024-05-08T23:13:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.984+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481aa9e0 con 0x7f8d481032b0 2024-05-08T23:13:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.987+0000 7f8d3fe00700 1 -- 172.21.15.46:0/2102114888 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d3800cd40 con 0x7f8d481032b0 2024-05-08T23:13:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 Processor -- start 2024-05-08T23:13:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 -- start start 2024-05-08T23:13:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d3fe00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 0x7f8d2c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d192cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d0e9150 con 0x55d21d197e00 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d0e6b10 con 0x55d21d192890 2024-05-08T23:13:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d62318b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d0e2400 con 0x55d21d198720 2024-05-08T23:13:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.988+0000 7f8d3fe00700 1 -- 172.21.15.46:0/2102114888 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8d38014070 con 0x7f8d481032b0 2024-05-08T23:13:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33980/0 (socket says 172.21.15.46:33980) 2024-05-08T23:13:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 -- 172.21.15.46:0/2207980838 learned_addr learned my addr 172.21.15.46:0/2207980838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d192cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 0x7f8d2c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d47400700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 -- 172.21.15.46:0/2207980838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 msgr2=0x55d21d192cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d192cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 -- 172.21.15.46:0/2207980838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 msgr2=0x55d21d1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 -- 172.21.15.46:0/2207980838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d21cf633b0 con 0x55d21d198720 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d47400700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.989+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006010 tx=0x7f8d34005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94780b42d878ee34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.990+0000 7f8d3cc00700 1 -- 172.21.15.46:0/2207980838 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340056a0 con 0x55d21d198720 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.990+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 0x7f8d2c0a37d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009fd0 tx=0x7f8d4000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.990+0000 7f8d3cc00700 1 -- 172.21.15.46:0/2207980838 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d3401e040 con 0x55d21d198720 2024-05-08T23:13:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.990+0000 7f8d3cc00700 1 -- 172.21.15.46:0/2207980838 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34004990 con 0x55d21d198720 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- 172.21.15.46:0/2207980838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 msgr2=0x55d21d192350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006010 tx=0x7f8d34005c00 comp rx=0 tx=0).stop 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- 172.21.15.46:0/2207980838 shutdown_connections 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d192cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2207980838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d192350 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- 172.21.15.46:0/2207980838 >> 172.21.15.46:0/2207980838 conn(0x55d21d0f0210 msgr2=0x55d21d18cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- 172.21.15.46:0/2207980838 shutdown_connections 2024-05-08T23:13:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- 172.21.15.46:0/2207980838 wait complete. 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.991+0000 7f8d62318b80 1 -- start start 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 Processor -- start 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d19c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d07e560 con 0x55d21d197e00 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d07e200 con 0x55d21d192890 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cf656a0 con 0x55d21d198720 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.992+0000 7f8d62318b80 10 client.0 _ll_register_callbacks cb 0x55d21d087b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33990/0 (socket says 172.21.15.46:33990) 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 -- 172.21.15.46:0/2498317270 learned_addr learned my addr 172.21.15.46:0/2498317270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47400700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d46a00700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 -- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 msgr2=0x55d21d19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d19baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 -- 172.21.15.46:0/2498317270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 msgr2=0x55d21d19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.993+0000 7f8d47e00700 1 -- 172.21.15.46:0/2498317270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481107c0 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d47e00700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d40016ab0 tx=0x7f8d40004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5cf6eff13df722 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d31600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4000e040 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d21d03b270 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8d400077f0 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d40003a00 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d62318b80 10 client.28393 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.994+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d21cf0fb60 con 0x55d21d198720 2024-05-08T23:13:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.995+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4002b020 con 0x55d21d198720 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d400079b0 con 0x55d21d198720 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 10 client.28393 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 10 client.28393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 20 client.28393 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d1ac0b0 0x55d21d1ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 4 client.28393 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d21d161810 con 0x55d21d1ac0b0 2024-05-08T23:13:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.996+0000 7f8d47400700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d1ac0b0 0x55d21d1ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.997+0000 7f8d47400700 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d1ac0b0 0x55d21d1ae4d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c144b508868304b1 server_cookie=8e99218f94727fcb in_seq=0 out_seq=0 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.997+0000 7f8d31600700 10 client.28393 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.997+0000 7f8d31600700 1 -- 172.21.15.46:0/2498317270 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d21d161810 con 0x55d21d1ac0b0 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.997+0000 7f8d31600700 10 client.28393 handle_command_reply: tid=0 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 client.28393 shutdown 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 msgr2=0x55d21d19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d40016ab0 tx=0x7f8d40004b40 comp rx=0 tx=0).stop 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 shutdown_connections 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d19c570 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d1ac0b0 0x55d21d1ae4d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 --2- 172.21.15.46:0/2498317270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d197e00 0x55d21d19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d31600700 0 client.28393 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 >> 172.21.15.46:0/2498317270 conn(0x55d21d0f0210 msgr2=0x55d21d18c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 shutdown_connections 2024-05-08T23:13:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 1 -- 172.21.15.46:0/2498317270 wait complete. 2024-05-08T23:13:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:39.998+0000 7f8d62318b80 20 client.28393 trim_cache size 0 max 16384 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.013+0000 7f8d62318b80 1 Processor -- start 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.013+0000 7f8d62318b80 1 -- start start 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.013+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.013+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 0x55d21d197450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d18cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d62318b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d198560 con 0x55d21d192890 2024-05-08T23:13:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d62318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d1b36e0 con 0x55d21d197e00 2024-05-08T23:13:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d62318b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d187800 con 0x55d21d198720 2024-05-08T23:13:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d46a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 0x55d21d197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d46a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 0x55d21d197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48572/0 (socket says 172.21.15.46:48572) 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58006/0 (socket says 172.21.15.46:58006) 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d46a00700 1 -- 172.21.15.46:0/927943322 learned_addr learned my addr 172.21.15.46:0/927943322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47e00700 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d18cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 -- 172.21.15.46:0/927943322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 msgr2=0x55d21d18cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d18cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 -- 172.21.15.46:0/927943322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 msgr2=0x55d21d197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 0x55d21d197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.014+0000 7f8d47400700 1 -- 172.21.15.46:0/927943322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d21d192f40 con 0x55d21d192890 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.015+0000 7f8d47400700 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ca730 tx=0x7f8d3800efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d52af998d6fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.015+0000 7f8d3d600700 1 -- 172.21.15.46:0/927943322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3809df20 con 0x55d21d192890 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.015+0000 7f8d3d600700 1 -- 172.21.15.46:0/927943322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d380a2570 con 0x55d21d192890 2024-05-08T23:13:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.015+0000 7f8d3d600700 1 -- 172.21.15.46:0/927943322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3809a350 con 0x55d21d192890 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- 172.21.15.46:0/927943322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 msgr2=0x55d21d192350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ca730 tx=0x7f8d3800efa0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- 172.21.15.46:0/927943322 shutdown_connections 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d198720 0x55d21d18cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d192890 0x55d21d192350 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 --2- 172.21.15.46:0/927943322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d197e00 0x55d21d197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- 172.21.15.46:0/927943322 >> 172.21.15.46:0/927943322 conn(0x55d21d0f0210 msgr2=0x55d21d19a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- 172.21.15.46:0/927943322 shutdown_connections 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- 172.21.15.46:0/927943322 wait complete. 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 -- start start 2024-05-08T23:13:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.016+0000 7f8d62318b80 1 Processor -- start 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 0x55d21d19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 0x55d21d1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d1a07b0 con 0x55d21d198720 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d1a0920 con 0x55d21d192890 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21d202600 con 0x55d21d197e00 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d62318b80 10 client.0 _ll_register_callbacks cb 0x55d21d087b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d47400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 0x55d21d19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d47400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48582/0 (socket says 172.21.15.46:48582) 2024-05-08T23:13:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d47400700 1 -- 172.21.15.46:0/472585892 learned_addr learned my addr 172.21.15.46:0/472585892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.017+0000 7f8d47e00700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 0x55d21d1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 -- 172.21.15.46:0/472585892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 msgr2=0x55d21d19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 0x55d21d19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 -- 172.21.15.46:0/472585892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 msgr2=0x55d21d1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 0x55d21d1a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 -- 172.21.15.46:0/472585892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481aa9e0 con 0x55d21d192890 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d46a00700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 0x55d21d19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47e00700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 0x55d21d1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d47400700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d38099da0 tx=0x7f8d3800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eae60cd59f2ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d31600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380cd230 con 0x55d21d192890 2024-05-08T23:13:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d3401e040 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8d380995e0 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d3809dfa0 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d62318b80 10 client.28316 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.018+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d21d0a8560 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d3401e040 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d380a20e0 con 0x55d21d192890 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 10 client.28316 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 10 client.28316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 20 client.28316 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d203ad0 0x55d21d205ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 4 client.28316 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d21d0daec0 con 0x55d21d203ad0 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.019+0000 7f8d46a00700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d203ad0 0x55d21d205ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d46a00700 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d203ad0 0x55d21d205ec0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec4447a53c8e94f server_cookie=e30b247054e2dbb in_seq=0 out_seq=0 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d31600700 10 client.28316 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d31600700 1 -- 172.21.15.46:0/472585892 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d21d0daec0 con 0x55d21d203ad0 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d31600700 10 client.28316 handle_command_reply: tid=0 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 client.28316 shutdown 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 msgr2=0x55d21d1a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d38099da0 tx=0x7f8d3800c6e0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 shutdown_connections 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d21d203ad0 0x55d21d205ec0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d21d198720 0x55d21d1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d21d192890 0x55d21d1a35c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d31600700 0 client.28316 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 --2- 172.21.15.46:0/472585892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d21d197e00 0x55d21d19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.020+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 >> 172.21.15.46:0/472585892 conn(0x55d21d0f0210 msgr2=0x55d21d199440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 shutdown_connections 2024-05-08T23:13:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d62318b80 1 -- 172.21.15.46:0/472585892 wait complete. 2024-05-08T23:13:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d62318b80 20 client.28316 trim_cache size 0 max 16384 2024-05-08T23:13:40.026 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 msgr2=0x7f8d2c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 0x7f8d2c0a37d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009fd0 tx=0x7f8d4000b040 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 msgr2=0x7f8d480754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.021+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9a0 tx=0x7f8d3800ed60 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 shutdown_connections 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8d2c0a1310 0x7f8d2c0a37d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d48075f30 0x7f8d481a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481032b0 0x7f8d480754b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 --2- 172.21.15.46:0/2102114888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d481083d0 0x7f8d480759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 >> 172.21.15.46:0/2102114888 conn(0x7f8d480feb10 msgr2=0x7f8d48100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 shutdown_connections 2024-05-08T23:13:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:40.022+0000 7f8d4e800700 1 -- 172.21.15.46:0/2102114888 wait complete. 2024-05-08T23:13:40.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:39 smithi046 ceph-mon[36898]: pgmap v304: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-08T23:13:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:39 smithi086 ceph-mon[78610]: pgmap v304: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-08T23:13:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:39 smithi053 ceph-mon[77587]: pgmap v304: 129 pgs: 129 active+clean; 522 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-08T23:13:40.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 855102 9.63 MB/sec execute 362 sec latency 16.686 ms 2024-05-08T23:13:41.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:13:41.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:13:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 -- 172.21.15.46:0/568474976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 msgr2=0x7ff2f0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/568474976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0101460 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009990 tx=0x7ff2e0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 -- 172.21.15.46:0/568474976 shutdown_connections 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/568474976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f01061a0 0x7ff2f010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/568474976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0101460 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/568474976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f0101a30 0x7ff2f0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 -- 172.21.15.46:0/568474976 >> 172.21.15.46:0/568474976 conn(0x7ff2f00fc8e0 msgr2=0x7ff2f00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 -- 172.21.15.46:0/568474976 shutdown_connections 2024-05-08T23:13:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.258+0000 7ff2f4e00700 1 -- 172.21.15.46:0/568474976 wait complete. 2024-05-08T23:13:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.259+0000 7ff2f4e00700 1 Processor -- start 2024-05-08T23:13:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.259+0000 7ff2f4e00700 1 -- start start 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 0x7ff2f0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 0x7ff2f019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019e860 con 0x7ff2f019dde0 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019e9d0 con 0x7ff2f01061a0 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2f4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019eb90 con 0x7ff2f0101080 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34022/0 (socket says 172.21.15.46:34022) 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 0x7ff2f0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ede00700 1 -- 172.21.15.46:0/1384457933 learned_addr learned my addr 172.21.15.46:0/1384457933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 0x7ff2f019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 -- 172.21.15.46:0/1384457933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 msgr2=0x7ff2f0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 -- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 msgr2=0x7ff2f019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 0x7ff2f019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ed400700 1 -- 172.21.15.46:0/1384457933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009c40 con 0x7ff2f01061a0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ede00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.260+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 0x7ff2f019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2ed400700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 0x7ff2f0199a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00e9c0 tx=0x7ff2dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2e6a00700 1 -- 172.21.15.46:0/1384457933 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0096b0 con 0x7ff2f01061a0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e00095f0 con 0x7ff2f01061a0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2e6a00700 1 -- 172.21.15.46:0/1384457933 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2dc004cf0 con 0x7ff2f01061a0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f01a6440 con 0x7ff2f01061a0 2024-05-08T23:13:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.261+0000 7ff2e6a00700 1 -- 172.21.15.46:0/1384457933 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00cdb0 con 0x7ff2f01061a0 2024-05-08T23:13:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.265+0000 7ff2e6a00700 1 -- 172.21.15.46:0/1384457933 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff2dc004220 con 0x7ff2f01061a0 2024-05-08T23:13:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.266+0000 7ff308968b80 1 Processor -- start 2024-05-08T23:13:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.266+0000 7ff308968b80 1 -- start start 2024-05-08T23:13:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e7913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e791910 0x55de1e791d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e6e81d0 con 0x55de1e7977a0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e6e5b90 con 0x55de1e791910 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff308968b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e6e1480 con 0x55de1e796e80 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2ee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2ee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34032/0 (socket says 172.21.15.46:34032) 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2ee800700 1 -- 172.21.15.46:0/1390793502 learned_addr learned my addr 172.21.15.46:0/1390793502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2e6a00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2ede00700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e7913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.267+0000 7ff2ed400700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e791910 0x55de1e791d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2e6a00700 1 -- 172.21.15.46:0/1384457933 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff2dc014070 con 0x7ff2f01061a0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 -- 172.21.15.46:0/1390793502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e791910 msgr2=0x55de1e791d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e791910 0x55de1e791d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 -- 172.21.15.46:0/1390793502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 msgr2=0x55de1e7913d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e7913d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 -- 172.21.15.46:0/1390793502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1e562430 con 0x55de1e796e80 2024-05-08T23:13:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ede00700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e7913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8007c60 tx=0x7ff2e8007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff85e8e5e2e6dcb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.268+0000 7ff2db400700 1 -- 172.21.15.46:0/1390793502 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e801ab30 con 0x55de1e796e80 2024-05-08T23:13:41.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff2ee800700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 0x7ff2d40a37d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2e801d000 tx=0x7ff2e8020000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff2db400700 1 -- 172.21.15.46:0/1390793502 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e800f040 con 0x55de1e796e80 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff2db400700 1 -- 172.21.15.46:0/1390793502 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80233c0 con 0x55de1e796e80 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 -- 172.21.15.46:0/1390793502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 msgr2=0x55de1e797260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8007c60 tx=0x7ff2e8007f70 comp rx=0 tx=0).stop 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 -- 172.21.15.46:0/1390793502 shutdown_connections 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e791910 0x55de1e791d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e796e80 0x55de1e797260 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 --2- 172.21.15.46:0/1390793502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e7913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.269+0000 7ff308968b80 1 -- 172.21.15.46:0/1390793502 >> 172.21.15.46:0/1390793502 conn(0x55de1e6ef290 msgr2=0x55de1e78be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.270+0000 7ff308968b80 1 -- 172.21.15.46:0/1390793502 shutdown_connections 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.270+0000 7ff308968b80 1 -- 172.21.15.46:0/1390793502 wait complete. 2024-05-08T23:13:41.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.270+0000 7ff308968b80 1 -- start start 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.270+0000 7ff308968b80 1 Processor -- start 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 0x55de1e79aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e79b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e67d5e0 con 0x55de1e791910 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e67d280 con 0x55de1e796e80 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e564720 con 0x55de1e7977a0 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff308968b80 10 client.0 _ll_register_callbacks cb 0x55de1e686bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e79b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 0x55de1e79aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ee800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34048/0 (socket says 172.21.15.46:34048) 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 0x55de1e79aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58058/0 (socket says 172.21.15.46:58058) 2024-05-08T23:13:41.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.271+0000 7ff2ee800700 1 -- 172.21.15.46:0/2926159941 learned_addr learned my addr 172.21.15.46:0/2926159941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 -- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 msgr2=0x55de1e79b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e79b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 -- 172.21.15.46:0/2926159941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 msgr2=0x55de1e79aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 0x55de1e79aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 -- 172.21.15.46:0/2926159941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f010e590 con 0x55de1e7977a0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8024af0 tx=0x7ff2e8007570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86e7cef99e21a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004e80 con 0x55de1e7977a0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff2e8024420 con 0x55de1e7977a0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de1e63a2f0 con 0x55de1e7977a0 2024-05-08T23:13:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e80245e0 con 0x55de1e7977a0 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff308968b80 10 client.28411 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.272+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff2e8007840 con 0x55de1e7977a0 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de1e63a2f0 con 0x55de1e7977a0 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2e8005610 con 0x55de1e7977a0 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 10 client.28411 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 10 client.28411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 20 client.28411 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e7ab1c0 0x55de1e7ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 4 client.28411 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de1e760890 con 0x55de1e7ab1c0 2024-05-08T23:13:41.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.273+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e7ab1c0 0x55de1e7ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.274+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e7ab1c0 0x55de1e7ad5e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454c25c57f584945 server_cookie=8ae48b15193e4d15 in_seq=0 out_seq=0 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.274+0000 7ff2cfe00700 10 client.28411 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2926159941 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de1e760890 con 0x55de1e7ab1c0 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff2cfe00700 10 client.28411 handle_command_reply: tid=0 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 client.28411 shutdown 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 msgr2=0x55de1e79b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8024af0 tx=0x7ff2e8007570 comp rx=0 tx=0).stop 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 shutdown_connections 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e7977a0 0x55de1e79b560 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e7ab1c0 0x55de1e7ad5e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e791910 0x55de1e79aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 --2- 172.21.15.46:0/2926159941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e79b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 >> 172.21.15.46:0/2926159941 conn(0x55de1e6ef290 msgr2=0x55de1e78b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff2cfe00700 0 client.28411 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 shutdown_connections 2024-05-08T23:13:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 1 -- 172.21.15.46:0/2926159941 wait complete. 2024-05-08T23:13:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.275+0000 7ff308968b80 20 client.28411 trim_cache size 0 max 16384 2024-05-08T23:13:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.296+0000 7ff308968b80 1 Processor -- start 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.296+0000 7ff308968b80 1 -- start start 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.296+0000 7ff308968b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff308968b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e796340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff308968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e79aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff308968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e63a2f0 con 0x55de1e7977a0 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff308968b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e79a910 con 0x55de1e796e80 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff308968b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e788e90 con 0x55de1e791910 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e796340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34060/0 (socket says 172.21.15.46:34060) 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 -- 172.21.15.46:0/2431566675 learned_addr learned my addr 172.21.15.46:0/2431566675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e79aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 -- 172.21.15.46:0/2431566675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 msgr2=0x55de1e796340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e796340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 -- 172.21.15.46:0/2431566675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 msgr2=0x55de1e79aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e79aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.297+0000 7ff2ede00700 1 -- 172.21.15.46:0/2431566675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1e50ebe0 con 0x55de1e791910 2024-05-08T23:13:41.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.298+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55de1e78dc50 tx=0x7ff2e000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae622872a31508a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.298+0000 7ff2dbe00700 1 -- 172.21.15.46:0/2431566675 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001d040 con 0x55de1e791910 2024-05-08T23:13:41.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.298+0000 7ff2dbe00700 1 -- 172.21.15.46:0/2431566675 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e001e040 con 0x55de1e791910 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.298+0000 7ff2dbe00700 1 -- 172.21.15.46:0/2431566675 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0004500 con 0x55de1e791910 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- 172.21.15.46:0/2431566675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 msgr2=0x55de1e78bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55de1e78dc50 tx=0x7ff2e000b260 comp rx=0 tx=0).stop 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- 172.21.15.46:0/2431566675 shutdown_connections 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e79aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e78bda0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 --2- 172.21.15.46:0/2431566675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e796340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- 172.21.15.46:0/2431566675 >> 172.21.15.46:0/2431566675 conn(0x55de1e6ef290 msgr2=0x55de1e795c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- 172.21.15.46:0/2431566675 shutdown_connections 2024-05-08T23:13:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- 172.21.15.46:0/2431566675 wait complete. 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 -- start start 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.299+0000 7ff308968b80 1 Processor -- start 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e7a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e809ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e7a1e40 con 0x55de1e7977a0 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e80a420 con 0x55de1e796e80 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1e80a560 con 0x55de1e791910 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff308968b80 10 client.0 _ll_register_callbacks cb 0x55de1e686bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff2ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34072/0 (socket says 172.21.15.46:34072) 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff2ede00700 1 -- 172.21.15.46:0/2986870597 learned_addr learned my addr 172.21.15.46:0/2986870597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff2ed400700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e7a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.300+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e809ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 -- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 msgr2=0x55de1e7a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e7a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 -- 172.21.15.46:0/2986870597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 msgr2=0x55de1e809ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e809ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 -- 172.21.15.46:0/2986870597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e000baa0 con 0x55de1e791910 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ee800700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e809ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2ede00700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000bc50 tx=0x7ff2e0029290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6090bae0f97891b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001d050 con 0x55de1e791910 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff2e0005250 con 0x55de1e791910 2024-05-08T23:13:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2e800f040 con 0x55de1e791910 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e0029910 con 0x55de1e791910 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff308968b80 10 client.28417 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.301+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55de1e6a75e0 con 0x55de1e791910 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e800f040 con 0x55de1e791910 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2e000f460 con 0x55de1e791910 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 10 client.28417 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 10 client.28417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 20 client.28417 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e80b900 0x55de1e80dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 4 client.28417 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55de1e6d9f40 con 0x55de1e80b900 2024-05-08T23:13:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.302+0000 7ff2ed400700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e80b900 0x55de1e80dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.303+0000 7ff2ed400700 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e80b900 0x55de1e80dd20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3091a74212b1d432 server_cookie=d0a4945e8782a552 in_seq=0 out_seq=0 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.303+0000 7ff2cfe00700 10 client.28417 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff2cfe00700 1 -- 172.21.15.46:0/2986870597 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de1e6d9f40 con 0x55de1e80b900 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff2cfe00700 10 client.28417 handle_command_reply: tid=0 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 client.28417 shutdown 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 msgr2=0x55de1e7a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000bc50 tx=0x7ff2e0029290 comp rx=0 tx=0).stop 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 shutdown_connections 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55de1e80b900 0x55de1e80dd20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1e7977a0 0x55de1e809ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de1e791910 0x55de1e7a0ff0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 --2- 172.21.15.46:0/2986870597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de1e796e80 0x55de1e7a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff2cfe00700 0 client.28417 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 >> 172.21.15.46:0/2986870597 conn(0x55de1e6ef290 msgr2=0x55de1e7964c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.304+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 shutdown_connections 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff308968b80 1 -- 172.21.15.46:0/2986870597 wait complete. 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff308968b80 20 client.28417 trim_cache size 0 max 16384 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4fa8cb84-a82f-48fc-b397-68c68483687e", 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:13:41.309 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 msgr2=0x7ff2d40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 0x7ff2d40a37d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2e801d000 tx=0x7ff2e8020000 comp rx=0 tx=0).stop 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 msgr2=0x7ff2f0199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 0x7ff2f0199a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00e9c0 tx=0x7ff2dc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 shutdown_connections 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f019dde0 0x7ff2f019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2f0101080 0x7ff2f0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 --2- 172.21.15.46:0/1384457933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f01061a0 0x7ff2f0199a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.305+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 >> 172.21.15.46:0/1384457933 conn(0x7ff2f00fc8e0 msgr2=0x7ff2f00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.306+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 shutdown_connections 2024-05-08T23:13:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.306+0000 7ff2f4e00700 1 -- 172.21.15.46:0/1384457933 wait complete. 2024-05-08T23:13:41.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 -- 172.21.15.46:0/2467730077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c101a30 msgr2=0x7f9d9c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 --2- 172.21.15.46:0/2467730077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c101a30 0x7f9d9c105a70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 -- 172.21.15.46:0/2467730077 shutdown_connections 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 --2- 172.21.15.46:0/2467730077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 --2- 172.21.15.46:0/2467730077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c101080 0x7f9d9c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 --2- 172.21.15.46:0/2467730077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c101a30 0x7f9d9c105a70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 -- 172.21.15.46:0/2467730077 >> 172.21.15.46:0/2467730077 conn(0x7f9d9c0fc8e0 msgr2=0x7f9d9c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 -- 172.21.15.46:0/2467730077 shutdown_connections 2024-05-08T23:13:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.547+0000 7f9da1000700 1 -- 172.21.15.46:0/2467730077 wait complete. 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.548+0000 7f9da1000700 1 Processor -- start 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 -- start start 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19c6a0 con 0x7f9d9c101080 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19c810 con 0x7f9d9c1061a0 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9da1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19c9d0 con 0x7f9d9c19bc20 2024-05-08T23:13:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d99400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d9a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d99e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d9a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34082/0 (socket says 172.21.15.46:34082) 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d9a800700 1 -- 172.21.15.46:0/2885111306 learned_addr learned my addr 172.21.15.46:0/2885111306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d99400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48648/0 (socket says 172.21.15.46:48648) 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.549+0000 7f9d99e00700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58108/0 (socket says 172.21.15.46:58108) 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 -- 172.21.15.46:0/2885111306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 msgr2=0x7f9d9c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 -- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 msgr2=0x7f9d9c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 -- 172.21.15.46:0/2885111306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880095f0 con 0x7f9d9c19bc20 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d99e00700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d9a800700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400ef30 tx=0x7f9d9400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d92a00700 1 -- 172.21.15.46:0/2885111306 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94018070 con 0x7f9d9c19bc20 2024-05-08T23:13:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c1a3ed0 con 0x7f9d9c19bc20 2024-05-08T23:13:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d92a00700 1 -- 172.21.15.46:0/2885111306 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9400f040 con 0x7f9d9c19bc20 2024-05-08T23:13:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9d92a00700 1 -- 172.21.15.46:0/2885111306 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400cd00 con 0x7f9d9c19bc20 2024-05-08T23:13:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.550+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c1a4400 con 0x7f9d9c19bc20 2024-05-08T23:13:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.554+0000 7f9d92a00700 1 -- 172.21.15.46:0/2885111306 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d94003a00 con 0x7f9d9c19bc20 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 Processor -- start 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9d92a00700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 0x7f9d800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 -- start start 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9d92a00700 1 -- 172.21.15.46:0/2885111306 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9d940ccb90 con 0x7f9d9c19bc20 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9d99400700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 0x7f9d800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6fe00 0x55ce86d701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 0x55ce86d6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86cc1150 con 0x55ce86d6fe00 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.555+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86cbeb10 con 0x55ce86d6a890 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86cba400 con 0x55ce86d70720 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 0x55ce86d6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 0x55ce86d6a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34084/0 (socket says 172.21.15.46:34084) 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d99400700 1 -- 172.21.15.46:0/4224847510 learned_addr learned my addr 172.21.15.46:0/4224847510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48652/0 (socket says 172.21.15.46:48652) 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d99e00700 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6fe00 0x55ce86d701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 -- 172.21.15.46:0/4224847510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 msgr2=0x55ce86d6a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 0x55ce86d6a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 -- 172.21.15.46:0/4224847510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6fe00 msgr2=0x55ce86d701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6fe00 0x55ce86d701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d9a800700 1 -- 172.21.15.46:0/4224847510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce86b3b3b0 con 0x55ce86d6a890 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.556+0000 7f9d99400700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 0x7f9d800a37d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c06a4a0 tx=0x7f9d88005380 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9d9a800700 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d940c9e70 tx=0x7f9d940997b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7da159abe1b5995 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9d87400700 1 -- 172.21.15.46:0/4224847510 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94004ad0 con 0x55ce86d6a890 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9d87400700 1 -- 172.21.15.46:0/4224847510 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9401b320 con 0x55ce86d6a890 2024-05-08T23:13:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9d87400700 1 -- 172.21.15.46:0/4224847510 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940a1cb0 con 0x55ce86d6a890 2024-05-08T23:13:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/4224847510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 msgr2=0x55ce86d6acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.557+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d940c9e70 tx=0x7f9d940997b0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/4224847510 shutdown_connections 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6acd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6fe00 0x55ce86d701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/4224847510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d70720 0x55ce86d6a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/4224847510 >> 172.21.15.46:0/4224847510 conn(0x55ce86cc8210 msgr2=0x55ce86d64df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/4224847510 shutdown_connections 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/4224847510 wait complete. 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 -- start start 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.558+0000 7f9db4bfbb80 1 Processor -- start 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6a890 0x55ce86d73ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 0x55ce86d74530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86c56560 con 0x55ce86d6a890 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86c56200 con 0x55ce86d70720 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86b3d6a0 con 0x55ce86d6fe00 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9db4bfbb80 10 client.0 _ll_register_callbacks cb 0x55ce86c5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9d99e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6a890 0x55ce86d73ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34096/0 (socket says 172.21.15.46:34096) 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9d99400700 1 -- 172.21.15.46:0/1994472676 learned_addr learned my addr 172.21.15.46:0/1994472676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.559+0000 7f9d9a800700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 0x55ce86d74530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 -- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 msgr2=0x55ce86d74530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 0x55ce86d74530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 -- 172.21.15.46:0/1994472676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6a890 msgr2=0x55ce86d73ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6a890 0x55ce86d73ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 -- 172.21.15.46:0/1994472676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c1a4400 con 0x55ce86d6fe00 2024-05-08T23:13:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d9a800700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 0x55ce86d74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d99400700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098e0 tx=0x7f9d88009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3202120383ff9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d7be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d880049d0 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9d88004b30 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce86c13270 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d88004cf0 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9db4bfbb80 10 client.28429 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.560+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ce86ae7b60 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce86c13270 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d88018490 con 0x55ce86d6fe00 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 10 client.28429 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 10 client.28429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 20 client.28429 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86d884b0 0x55ce86d8a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 4 client.28429 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:13:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce86d39810 con 0x55ce86d884b0 2024-05-08T23:13:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.561+0000 7f9d99e00700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86d884b0 0x55ce86d8a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.562+0000 7f9d99e00700 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86d884b0 0x55ce86d8a8d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f394c283889d36c server_cookie=6ffbf6aace10898b in_seq=0 out_seq=0 2024-05-08T23:13:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.562+0000 7f9d7be00700 10 client.28429 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9d7be00700 1 -- 172.21.15.46:0/1994472676 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce86d39810 con 0x55ce86d884b0 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9d7be00700 10 client.28429 handle_command_reply: tid=0 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 client.28429 shutdown 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 msgr2=0x55ce86d73ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098e0 tx=0x7f9d88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 shutdown_connections 2024-05-08T23:13:41.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d70720 0x55ce86d74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86d884b0 0x55ce86d8a8d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d6a890 0x55ce86d73ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1994472676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d73ff0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9d7be00700 0 client.28429 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 >> 172.21.15.46:0/1994472676 conn(0x55ce86cc8210 msgr2=0x55ce86d645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 shutdown_connections 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.563+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1994472676 wait complete. 2024-05-08T23:13:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.564+0000 7f9db4bfbb80 20 client.28429 trim_cache size 0 max 16384 2024-05-08T23:13:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 Processor -- start 2024-05-08T23:13:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 -- start start 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86d64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86d5f800 con 0x55ce86d70720 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86d66a00 con 0x55ce86d6a890 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86d71b10 con 0x55ce86d6fe00 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:34100/0 (socket says 172.21.15.46:34100) 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9d99400700 1 -- 172.21.15.46:0/1388373028 learned_addr learned my addr 172.21.15.46:0/1388373028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9d9a800700 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86d64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.582+0000 7f9d99e00700 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 -- 172.21.15.46:0/1388373028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 msgr2=0x55ce86d6a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 -- 172.21.15.46:0/1388373028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 msgr2=0x55ce86d64d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86d64d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 -- 172.21.15.46:0/1388373028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce86c80560 con 0x55ce86d6fe00 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d99400700 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9d88000c00 tx=0x7f9d8800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1527fbc39364285d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d87e00700 1 -- 172.21.15.46:0/1388373028 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d880189c0 con 0x55ce86d6fe00 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d87e00700 1 -- 172.21.15.46:0/1388373028 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d8801e040 con 0x55ce86d6fe00 2024-05-08T23:13:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.583+0000 7f9d87e00700 1 -- 172.21.15.46:0/1388373028 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8803f6d0 con 0x55ce86d6fe00 2024-05-08T23:13:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1388373028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 msgr2=0x55ce86d6f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9d88000c00 tx=0x7f9d8800bea0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1388373028 shutdown_connections 2024-05-08T23:13:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86d64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d6a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/1388373028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d6f430 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1388373028 >> 172.21.15.46:0/1388373028 conn(0x55ce86cc8210 msgr2=0x55ce86d72380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1388373028 shutdown_connections 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/1388373028 wait complete. 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.584+0000 7f9db4bfbb80 1 -- start start 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.585+0000 7f9db4bfbb80 1 Processor -- start 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.585+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.585+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d7db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.585+0000 7f9db4bfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86ddd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86d7e2d0 con 0x55ce86d70720 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86d7e410 con 0x55ce86d6a890 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9db4bfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce86dddef0 con 0x55ce86d6fe00 2024-05-08T23:13:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9db4bfbb80 10 client.0 _ll_register_callbacks cb 0x55ce86c5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48676/0 (socket says 172.21.15.46:48676) 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 -- 172.21.15.46:0/2205093240 learned_addr learned my addr 172.21.15.46:0/2205093240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99400700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d7db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d9a800700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86ddd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 -- 172.21.15.46:0/2205093240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 msgr2=0x55ce86d7db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d7db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 -- 172.21.15.46:0/2205093240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 msgr2=0x55ce86ddd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86ddd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 -- 172.21.15.46:0/2205093240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce86ae7b60 con 0x55ce86d6a890 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99400700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d99e00700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55ce86d8b1e0 tx=0x7f9d8c012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea9a82eae1c1249 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.586+0000 7f9d7be00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:13:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c017040 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9d8c01f810 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d9401b320 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c028470 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9db4bfbb80 10 client.28352 fetch_fsmap learned FSMap version 30 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ce86b3fcd0 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.587+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9401b320 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d8c010690 con 0x55ce86d6a890 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 10 client.28352 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 10 client.28352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:13:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 20 client.28352 populate_metadata read hostname 'smithi046' 2024-05-08T23:13:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86ddf690 0x55ce86de1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 4 client.28352 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:13:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce86cb2ec0 con 0x55ce86ddf690 2024-05-08T23:13:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.588+0000 7f9d99400700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86ddf690 0x55ce86de1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:41.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9d7be00700 10 client.28352 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9d99400700 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86ddf690 0x55ce86de1ab0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedb7a365afdf7ce server_cookie=b0f4d581f00a0048 in_seq=0 out_seq=0 2024-05-08T23:13:41.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9d7be00700 1 -- 172.21.15.46:0/2205093240 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ce86cb2ec0 con 0x55ce86ddf690 2024-05-08T23:13:41.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9d7be00700 10 client.28352 handle_command_reply: tid=0 2024-05-08T23:13:41.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 client.28352 shutdown 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 msgr2=0x55ce86d7d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55ce86d8b1e0 tx=0x7f9d8c012aa0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 shutdown_connections 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ce86ddf690 0x55ce86de1ab0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce86d70720 0x55ce86ddd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce86d6a890 0x55ce86d7d5e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9d7be00700 0 client.28352 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 --2- 172.21.15.46:0/2205093240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce86d6fe00 0x55ce86d7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 >> 172.21.15.46:0/2205093240 conn(0x55ce86cc8210 msgr2=0x55ce86d72380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.589+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 shutdown_connections 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9db4bfbb80 1 -- 172.21.15.46:0/2205093240 wait complete. 2024-05-08T23:13:41.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9db4bfbb80 20 client.28352 trim_cache size 0 max 16384 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "4fa8cb84-a82f-48fc-b397-68c68483687e": { 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4fa8cb84-a82f-48fc-b397-68c68483687e", 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:13:41.595 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 msgr2=0x7f9d800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 0x7f9d800a37d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c06a4a0 tx=0x7f9d88005380 comp rx=0 tx=0).stop 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 msgr2=0x7f9d9c19c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400ef30 tx=0x7f9d9400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 shutdown_connections 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d9c19bc20 0x7f9d9c19c0a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d800a1310 0x7f9d800a37d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c101080 0x7f9d9c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 --2- 172.21.15.46:0/2885111306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9c1061a0 0x7f9d9c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.590+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 >> 172.21.15.46:0/2885111306 conn(0x7f9d9c0fc8e0 msgr2=0x7f9d9c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.591+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 shutdown_connections 2024-05-08T23:13:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:13:41.591+0000 7f9da1000700 1 -- 172.21.15.46:0/2885111306 wait complete. 2024-05-08T23:13:41.609 INFO:tasks.cephfs.filesystem:scrub status for tag:4fa8cb84-a82f-48fc-b397-68c68483687e - {'path': '/', 'tag': '4fa8cb84-a82f-48fc-b397-68c68483687e', 'options': 'recursive,force'} 2024-05-08T23:13:41.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 856247 9.63 MB/sec execute 363 sec latency 21.758 ms 2024-05-08T23:13:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:41 smithi086 ceph-mon[78610]: pgmap v305: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-08T23:13:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:41 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:42.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:41 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:41 smithi053 ceph-mon[77587]: pgmap v305: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-08T23:13:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:41 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:41 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:41 smithi046 ceph-mon[36898]: pgmap v305: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-05-08T23:13:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:41 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:41 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:42.905 INFO:tasks.workunit.client.0.smithi046.stdout: 1 858152 9.62 MB/sec execute 364 sec latency 24.038 ms 2024-05-08T23:13:43.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:13:43.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:43.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:13:43.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:42 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:42 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:42 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:13:43.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 859574 9.63 MB/sec execute 365 sec latency 19.152 ms 2024-05-08T23:13:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:43 smithi086 ceph-mon[78610]: pgmap v306: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-08T23:13:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:43 smithi053 ceph-mon[77587]: pgmap v306: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-08T23:13:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:43 smithi046 ceph-mon[36898]: pgmap v306: 129 pgs: 129 active+clean; 530 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-05-08T23:13:44.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 861182 9.61 MB/sec execute 366 sec latency 16.993 ms 2024-05-08T23:13:45.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 862884 9.62 MB/sec execute 367 sec latency 26.315 ms 2024-05-08T23:13:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:45 smithi086 ceph-mon[78610]: pgmap v307: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-05-08T23:13:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:45 smithi053 ceph-mon[77587]: pgmap v307: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-05-08T23:13:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:45 smithi046 ceph-mon[36898]: pgmap v307: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-05-08T23:13:46.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 864154 9.61 MB/sec execute 368 sec latency 102.041 ms 2024-05-08T23:13:47.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 865352 9.59 MB/sec execute 369 sec latency 400.413 ms 2024-05-08T23:13:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:47 smithi086 ceph-mon[78610]: pgmap v308: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-08T23:13:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:47 smithi053 ceph-mon[77587]: pgmap v308: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-08T23:13:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:47 smithi046 ceph-mon[36898]: pgmap v308: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-05-08T23:13:48.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 866409 9.60 MB/sec execute 370 sec latency 59.082 ms 2024-05-08T23:13:49.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 867786 9.59 MB/sec execute 371 sec latency 16.107 ms 2024-05-08T23:13:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:49 smithi086 ceph-mon[78610]: pgmap v309: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-08T23:13:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:49 smithi053 ceph-mon[77587]: pgmap v309: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-08T23:13:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:49 smithi046 ceph-mon[36898]: pgmap v309: 129 pgs: 129 active+clean; 531 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-05-08T23:13:50.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 869829 9.59 MB/sec execute 372 sec latency 28.153 ms 2024-05-08T23:13:51.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 870738 9.57 MB/sec execute 373 sec latency 15.851 ms 2024-05-08T23:13:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:51 smithi086 ceph-mon[78610]: pgmap v310: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-05-08T23:13:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:51 smithi053 ceph-mon[77587]: pgmap v310: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-05-08T23:13:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:51 smithi046 ceph-mon[36898]: pgmap v310: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-05-08T23:13:52.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 872587 9.58 MB/sec execute 374 sec latency 111.753 ms 2024-05-08T23:13:53.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 873791 9.57 MB/sec execute 375 sec latency 29.864 ms 2024-05-08T23:13:54.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:53 smithi086 ceph-mon[78610]: pgmap v311: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 197 op/s 2024-05-08T23:13:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:53 smithi053 ceph-mon[77587]: pgmap v311: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 197 op/s 2024-05-08T23:13:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:53 smithi046 ceph-mon[36898]: pgmap v311: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 197 op/s 2024-05-08T23:13:54.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 875584 9.56 MB/sec execute 376 sec latency 22.093 ms 2024-05-08T23:13:55.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 877068 9.57 MB/sec execute 377 sec latency 15.705 ms 2024-05-08T23:13:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:55 smithi086 ceph-mon[78610]: pgmap v312: 129 pgs: 129 active+clean; 523 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:13:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:55 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:55 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:55 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:13:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:55 smithi053 ceph-mon[77587]: pgmap v312: 129 pgs: 129 active+clean; 523 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:13:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:55 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:55 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:55 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:13:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:55 smithi046 ceph-mon[36898]: pgmap v312: 129 pgs: 129 active+clean; 523 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-05-08T23:13:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:55 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:55 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:55 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:13:56.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 878420 9.56 MB/sec execute 378 sec latency 25.750 ms 2024-05-08T23:13:57.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 880462 9.57 MB/sec execute 379 sec latency 17.833 ms 2024-05-08T23:13:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:57 smithi086 ceph-mon[78610]: pgmap v313: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 250 op/s 2024-05-08T23:13:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:57 smithi053 ceph-mon[77587]: pgmap v313: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 250 op/s 2024-05-08T23:13:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:57 smithi046 ceph-mon[36898]: pgmap v313: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 250 op/s 2024-05-08T23:13:58.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 881423 9.55 MB/sec execute 380 sec latency 26.159 ms 2024-05-08T23:13:59.906 INFO:tasks.workunit.client.0.smithi046.stdout: 1 883473 9.56 MB/sec execute 381 sec latency 16.611 ms 2024-05-08T23:14:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:13:59 smithi086 ceph-mon[78610]: pgmap v314: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 227 op/s 2024-05-08T23:14:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:13:59 smithi053 ceph-mon[77587]: pgmap v314: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 227 op/s 2024-05-08T23:14:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:13:59 smithi046 ceph-mon[36898]: pgmap v314: 129 pgs: 129 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 227 op/s 2024-05-08T23:14:00.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 884527 9.55 MB/sec execute 382 sec latency 21.208 ms 2024-05-08T23:14:01.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 885048 9.53 MB/sec execute 383 sec latency 75.069 ms 2024-05-08T23:14:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:01 smithi086 ceph-mon[78610]: pgmap v315: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-05-08T23:14:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:01 smithi053 ceph-mon[77587]: pgmap v315: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-05-08T23:14:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:01 smithi046 ceph-mon[36898]: pgmap v315: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-05-08T23:14:02.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 886577 9.52 MB/sec execute 384 sec latency 61.751 ms 2024-05-08T23:14:03.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 887885 9.52 MB/sec execute 385 sec latency 13.454 ms 2024-05-08T23:14:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:03 smithi086 ceph-mon[78610]: pgmap v316: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:14:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:03 smithi053 ceph-mon[77587]: pgmap v316: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:14:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:03 smithi046 ceph-mon[36898]: pgmap v316: 129 pgs: 129 active+clean; 532 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:14:04.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 889545 9.51 MB/sec execute 386 sec latency 15.662 ms 2024-05-08T23:14:05.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 891231 9.52 MB/sec execute 387 sec latency 14.877 ms 2024-05-08T23:14:06.222 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:05 smithi086 ceph-mon[78610]: pgmap v317: 129 pgs: 129 active+clean; 534 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-05-08T23:14:06.222 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:05 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:05 smithi053 ceph-mon[77587]: pgmap v317: 129 pgs: 129 active+clean; 534 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-05-08T23:14:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:05 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:05 smithi046 ceph-mon[36898]: pgmap v317: 129 pgs: 129 active+clean; 534 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-05-08T23:14:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:05 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:06.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 892494 9.51 MB/sec execute 388 sec latency 35.219 ms 2024-05-08T23:14:07.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: pgmap v318: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-08T23:14:07.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:07 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: pgmap v318: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:07 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: pgmap v318: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-08T23:14:07.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.716 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:07 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:07.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 894484 9.51 MB/sec execute 389 sec latency 25.277 ms 2024-05-08T23:14:08.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:08.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:08.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:08.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:08.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:08.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:14:08.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 895441 9.50 MB/sec execute 390 sec latency 20.274 ms 2024-05-08T23:14:09.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:09 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:14:09.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:09 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:14:09.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:09 smithi053 ceph-mon[77587]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-08T23:14:09.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:09 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:14:09.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:09 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:14:09.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:09 smithi046 ceph-mon[36898]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-08T23:14:09.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:09 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:14:09.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:09 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:14:09.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:09 smithi086 ceph-mon[78610]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-08T23:14:09.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 897535 9.50 MB/sec execute 391 sec latency 17.942 ms 2024-05-08T23:14:10.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 898575 9.50 MB/sec execute 392 sec latency 53.805 ms 2024-05-08T23:14:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:11 smithi046 ceph-mon[36898]: pgmap v320: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-05-08T23:14:11.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:11 smithi046 ceph-mon[36898]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:14:11.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:14:11.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:11 smithi053 ceph-mon[77587]: pgmap v320: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-05-08T23:14:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:11 smithi053 ceph-mon[77587]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:14:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 -- 172.21.15.46:0/1323286777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4101080 msgr2=0x7ff0d4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 --2- 172.21.15.46:0/1323286777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4101080 0x7ff0d4101460 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009990 tx=0x7ff0c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 -- 172.21.15.46:0/1323286777 shutdown_connections 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 --2- 172.21.15.46:0/1323286777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d41061a0 0x7ff0d410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 --2- 172.21.15.46:0/1323286777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4101080 0x7ff0d4101460 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 --2- 172.21.15.46:0/1323286777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101a30 0x7ff0d4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 -- 172.21.15.46:0/1323286777 >> 172.21.15.46:0/1323286777 conn(0x7ff0d40fc8e0 msgr2=0x7ff0d40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.844+0000 7ff0dc400700 1 -- 172.21.15.46:0/1323286777 shutdown_connections 2024-05-08T23:14:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.845+0000 7ff0dc400700 1 -- 172.21.15.46:0/1323286777 wait complete. 2024-05-08T23:14:11.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 Processor -- start 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 -- start start 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 0x7ff0d4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d419de30 0x7ff0d419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d419e8b0 con 0x7ff0d4101080 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d419ea20 con 0x7ff0d419de30 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0dc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d419ebe0 con 0x7ff0d41061a0 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.846+0000 7ff0d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 0x7ff0d4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50786/0 (socket says 172.21.15.46:50786) 2024-05-08T23:14:11.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 -- 172.21.15.46:0/4204515616 learned_addr learned my addr 172.21.15.46:0/4204515616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0da400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d419de30 0x7ff0d419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 -- 172.21.15.46:0/4204515616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d419de30 msgr2=0x7ff0d419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d419de30 0x7ff0d419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 -- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 msgr2=0x7ff0d4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 0x7ff0d4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 -- 172.21.15.46:0/4204515616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0009520 con 0x7ff0d41061a0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 0x7ff0d4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d9000700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ea30 tx=0x7ff0c000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.847+0000 7ff0d2000700 1 -- 172.21.15.46:0/4204515616 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000cbd0 con 0x7ff0d41061a0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.848+0000 7ff0d2000700 1 -- 172.21.15.46:0/4204515616 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c000eed0 con 0x7ff0d41061a0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.848+0000 7ff0d2000700 1 -- 172.21.15.46:0/4204515616 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003d20 con 0x7ff0d41061a0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.848+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80095f0 con 0x7ff0d41061a0 2024-05-08T23:14:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.848+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d41a6440 con 0x7ff0d41061a0 2024-05-08T23:14:11.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.852+0000 7ff0d2000700 1 -- 172.21.15.46:0/4204515616 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff0c0003ee0 con 0x7ff0d41061a0 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.852+0000 7ff0efef7b80 1 Processor -- start 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.852+0000 7ff0efef7b80 1 -- start start 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad6211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad572150 con 0x55b3ad61b890 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad56fb10 con 0x55b3ad621720 2024-05-08T23:14:11.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad56b400 con 0x55b3ad620e00 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0d2000700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 0x7ff0bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0d2000700 1 -- 172.21.15.46:0/4204515616 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff0c0014070 con 0x7ff0d41061a0 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0da400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad6211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54772/0 (socket says 172.21.15.46:54772) 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3821840406 learned_addr learned my addr 172.21.15.46:0/3821840406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.853+0000 7ff0da400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 0x7ff0bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9000700 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3821840406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 msgr2=0x55b3ad6211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad6211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3821840406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 msgr2=0x55b3ad61bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3821840406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ad3ec3b0 con 0x55b3ad621720 2024-05-08T23:14:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005f00 tx=0x7ff0c800b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256b43923495e7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0da400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 0x7ff0bc0a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009ec0 tx=0x7ff0cc00b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.854+0000 7ff0c6a00700 1 -- 172.21.15.46:0/3821840406 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801d070 con 0x55b3ad621720 2024-05-08T23:14:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0c6a00700 1 -- 172.21.15.46:0/3821840406 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c801e040 con 0x55b3ad621720 2024-05-08T23:14:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0c6a00700 1 -- 172.21.15.46:0/3821840406 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004ba0 con 0x55b3ad621720 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3821840406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 msgr2=0x55b3ad61b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005f00 tx=0x7ff0c800b5b0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3821840406 shutdown_connections 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad6211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3821840406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad61b350 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.855+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3821840406 >> 172.21.15.46:0/3821840406 conn(0x55b3ad579210 msgr2=0x55b3ad615df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.856+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3821840406 shutdown_connections 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.856+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3821840406 wait complete. 2024-05-08T23:14:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.856+0000 7ff0efef7b80 1 -- start start 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.856+0000 7ff0efef7b80 1 Processor -- start 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad627770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad625150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad507560 con 0x55b3ad61b890 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad507200 con 0x55b3ad621720 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad3ee6a0 con 0x55b3ad620e00 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0efef7b80 10 client.0 _ll_register_callbacks cb 0x55b3ad510b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad627770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50808/0 (socket says 172.21.15.46:50808) 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 -- 172.21.15.46:0/2858741647 learned_addr learned my addr 172.21.15.46:0/2858741647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0da400700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad625150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 -- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 msgr2=0x55b3ad625150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad625150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 -- 172.21.15.46:0/2858741647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 msgr2=0x55b3ad627770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad627770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.857+0000 7ff0d9000700 1 -- 172.21.15.46:0/2858741647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d410e590 con 0x55b3ad620e00 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0d9000700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c5b0 tx=0x7ff0c009a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9986c3bfd7223e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0bb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c009c030 con 0x55b3ad620e00 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff0c00b2020 con 0x55b3ad620e00 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3ad4c4270 con 0x55b3ad620e00 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c00a25a0 con 0x55b3ad620e00 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0efef7b80 10 client.28447 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b3ad398b60 con 0x55b3ad620e00 2024-05-08T23:14:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.858+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3ad4c4270 con 0x55b3ad620e00 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0c009a380 con 0x55b3ad620e00 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 10 client.28447 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 10 client.28447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 20 client.28447 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad635110 0x55b3ad637530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 4 client.28447 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3ad5ea810 con 0x55b3ad635110 2024-05-08T23:14:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad635110 0x55b3ad637530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad635110 0x55b3ad637530 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b33d10ea10b65e server_cookie=277e93b8b02ec8a4 in_seq=0 out_seq=0 2024-05-08T23:14:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.859+0000 7ff0bb400700 10 client.28447 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.860+0000 7ff0bb400700 1 -- 172.21.15.46:0/2858741647 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3ad5ea810 con 0x55b3ad635110 2024-05-08T23:14:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.860+0000 7ff0bb400700 10 client.28447 handle_command_reply: tid=0 2024-05-08T23:14:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.860+0000 7ff0efef7b80 1 client.28447 shutdown 2024-05-08T23:14:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.860+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 msgr2=0x55b3ad624ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.860+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c5b0 tx=0x7ff0c009a120 comp rx=0 tx=0).stop 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 shutdown_connections 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad625150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad635110 0x55b3ad637530 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad627770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/2858741647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad624ad0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0bb400700 0 client.28447 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 >> 172.21.15.46:0/2858741647 conn(0x55b3ad579210 msgr2=0x55b3ad6155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 shutdown_connections 2024-05-08T23:14:11.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 1 -- 172.21.15.46:0/2858741647 wait complete. 2024-05-08T23:14:11.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.861+0000 7ff0efef7b80 20 client.28447 trim_cache size 0 max 16384 2024-05-08T23:14:11.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 Processor -- start 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 -- start start 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad615d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad62c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad634ef0 con 0x55b3ad61b890 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad623040 con 0x55b3ad621720 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad61c230 con 0x55b3ad620e00 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58274/0 (socket says 172.21.15.46:58274) 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3645872590 learned_addr learned my addr 172.21.15.46:0/3645872590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:11.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0da400700 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad62c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.882+0000 7ff0d9000700 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad615d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3645872590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 msgr2=0x55b3ad615d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad615d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3645872590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 msgr2=0x55b3ad62c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad62c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 -- 172.21.15.46:0/3645872590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ad531560 con 0x55b3ad61b890 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005f50 tx=0x7ff0c801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acf95aa7997dff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0c7400700 1 -- 172.21.15.46:0/3645872590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801d040 con 0x55b3ad61b890 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0c7400700 1 -- 172.21.15.46:0/3645872590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c8034850 con 0x55b3ad61b890 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.883+0000 7ff0c7400700 1 -- 172.21.15.46:0/3645872590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8005580 con 0x55b3ad61b890 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3645872590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 msgr2=0x55b3ad61b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005f50 tx=0x7ff0c801a040 comp rx=0 tx=0).stop 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3645872590 shutdown_connections 2024-05-08T23:14:11.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad621720 0x55b3ad62c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad61b350 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3645872590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad620e00 0x55b3ad615d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3645872590 >> 172.21.15.46:0/3645872590 conn(0x55b3ad579210 msgr2=0x55b3ad630c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3645872590 shutdown_connections 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.884+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3645872590 wait complete. 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 -- start start 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 Processor -- start 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad62c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 0x55b3ad61db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad61c930 con 0x55b3ad61b890 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad61caa0 con 0x55b3ad620e00 2024-05-08T23:14:11.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad61cc60 con 0x55b3ad621720 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.885+0000 7ff0efef7b80 10 client.0 _ll_register_callbacks cb 0x55b3ad510b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0d9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad62c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50828/0 (socket says 172.21.15.46:50828) 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 -- 172.21.15.46:0/3779040848 learned_addr learned my addr 172.21.15.46:0/3779040848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0d9000700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 0x55b3ad61db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 -- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 msgr2=0x55b3ad61db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 0x55b3ad61db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 -- 172.21.15.46:0/3779040848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 msgr2=0x55b3ad62c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad62c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 -- 172.21.15.46:0/3779040848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3ad3ec3b0 con 0x55b3ad621720 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0d9000700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 0x55b3ad61db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:11.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0da400700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc01b010 tx=0x7ff0cc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9efe2cca0668075 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.886+0000 7ff0bb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc021040 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff0cc004bb0 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0c801e040 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0cc00e070 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0efef7b80 10 client.28453 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b3ad3f0cd0 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c801e040 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.887+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0cc028d00 con 0x55b3ad621720 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 10 client.28453 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 10 client.28453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 20 client.28453 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad68c140 0x55b3ad68e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 4 client.28453 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:14:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3ad563ec0 con 0x55b3ad68c140 2024-05-08T23:14:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.888+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad68c140 0x55b3ad68e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0bb400700 10 client.28453 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:11.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0d9a00700 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad68c140 0x55b3ad68e560 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38163de99847f22 server_cookie=c638af6f4e2bdee6 in_seq=0 out_seq=0 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0bb400700 1 -- 172.21.15.46:0/3779040848 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b3ad563ec0 con 0x55b3ad68c140 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0bb400700 10 client.28453 handle_command_reply: tid=0 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 client.28453 shutdown 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 msgr2=0x55b3ad61e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc01b010 tx=0x7ff0cc004b40 comp rx=0 tx=0).stop 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 shutdown_connections 2024-05-08T23:14:11.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3ad621720 0x55b3ad61e240 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b3ad68c140 0x55b3ad68e560 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3ad61b890 0x55b3ad62c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 --2- 172.21.15.46:0/3779040848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad620e00 0x55b3ad61db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0bb400700 0 client.28453 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 >> 172.21.15.46:0/3779040848 conn(0x55b3ad579210 msgr2=0x55b3ad623360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 shutdown_connections 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 1 -- 172.21.15.46:0/3779040848 wait complete. 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.889+0000 7ff0efef7b80 20 client.28453 trim_cache size 0 max 16384 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:14:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:14:11.896 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:11.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 msgr2=0x7ff0bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 0x7ff0bc0a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009ec0 tx=0x7ff0cc00b040 comp rx=0 tx=0).stop 2024-05-08T23:14:11.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 msgr2=0x7ff0d4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ea30 tx=0x7ff0c000ed40 comp rx=0 tx=0).stop 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 shutdown_connections 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0d419de30 0x7ff0d419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff0bc0a1290 0x7ff0bc0a3750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d4101080 0x7ff0d4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 --2- 172.21.15.46:0/4204515616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d41061a0 0x7ff0d4199a90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 >> 172.21.15.46:0/4204515616 conn(0x7ff0d40fc8e0 msgr2=0x7ff0d40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 shutdown_connections 2024-05-08T23:14:11.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:11.890+0000 7ff0dc400700 1 -- 172.21.15.46:0/4204515616 wait complete. 2024-05-08T23:14:11.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 900091 9.49 MB/sec execute 393 sec latency 18.821 ms 2024-05-08T23:14:11.909 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:14:11.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 -- 172.21.15.46:0/1723798494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294101070 msgr2=0x7f6294101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 --2- 172.21.15.46:0/1723798494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294101070 0x7f6294101450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6284009990 tx=0x7f6284009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 -- 172.21.15.46:0/1723798494 shutdown_connections 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 --2- 172.21.15.46:0/1723798494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294106190 0x7f629410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 --2- 172.21.15.46:0/1723798494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294101070 0x7f6294101450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 --2- 172.21.15.46:0/1723798494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6294101a20 0x7f6294105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 -- 172.21.15.46:0/1723798494 >> 172.21.15.46:0/1723798494 conn(0x7f62940fc930 msgr2=0x7f62940fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 -- 172.21.15.46:0/1723798494 shutdown_connections 2024-05-08T23:14:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.152+0000 7f6299c00700 1 -- 172.21.15.46:0/1723798494 wait complete. 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.153+0000 7f6299c00700 1 Processor -- start 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 -- start start 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294101070 0x7f62941973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6292000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 0x7f629419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419c6b0 con 0x7f629419bc30 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6292000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50848/0 (socket says 172.21.15.46:50848) 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419c820 con 0x7f6294101070 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629419c9e0 con 0x7f6294106190 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6292000700 1 -- 172.21.15.46:0/2082541936 learned_addr learned my addr 172.21.15.46:0/2082541936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.154+0000 7f6292a00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294101070 0x7f62941973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6293400700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 0x7f629419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 -- 172.21.15.46:0/2082541936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294101070 msgr2=0x7f62941973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294101070 0x7f62941973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 -- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 msgr2=0x7f629419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 0x7f629419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840095f0 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6293400700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 0x7f629419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6292000700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f628000e970 tx=0x7f628000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280009790 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f628000eec0 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62940731d0 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280005590 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.155+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6294073730 con 0x7f6294106190 2024-05-08T23:14:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.156+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62941093b0 con 0x7f6294106190 2024-05-08T23:14:12.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.158+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f628000cbd0 con 0x7f6294106190 2024-05-08T23:14:12.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.160+0000 7f628b400700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 0x7f62780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.160+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f6280014070 con 0x7f6294106190 2024-05-08T23:14:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.160+0000 7f6292a00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 0x7f62780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.161+0000 7f6292a00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 0x7f62780a37d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6284005b00 tx=0x7f6284005a90 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.164+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6280096350 con 0x7f6294106190 2024-05-08T23:14:12.283 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:11 smithi086 ceph-mon[78610]: pgmap v320: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-05-08T23:14:12.283 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:11 smithi086 ceph-mon[78610]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:14:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.490+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6294101a20 con 0x7f6294106190 2024-05-08T23:14:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.491+0000 7f628b400700 1 -- 172.21.15.46:0/2082541936 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f6280004ef0 con 0x7f6294106190 2024-05-08T23:14:12.490 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:14:12.490 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 msgr2=0x7f62780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 0x7f62780a37d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6284005b00 tx=0x7f6284005a90 comp rx=0 tx=0).stop 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 msgr2=0x7f62941978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f628000e970 tx=0x7f628000ec80 comp rx=0 tx=0).stop 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 shutdown_connections 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.493+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f62780a1310 0x7f62780a37d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629419bc30 0x7f629419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6294101070 0x7f62941973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 --2- 172.21.15.46:0/2082541936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6294106190 0x7f62941978e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 >> 172.21.15.46:0/2082541936 conn(0x7f62940fc930 msgr2=0x7f62940fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 shutdown_connections 2024-05-08T23:14:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.494+0000 7f6299c00700 1 -- 172.21.15.46:0/2082541936 wait complete. 2024-05-08T23:14:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:14:12.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:14:12.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.744+0000 7f94e7200700 1 -- 172.21.15.46:0/1960634447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 msgr2=0x7f94e0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.744+0000 7f94e7200700 1 --2- 172.21.15.46:0/1960634447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e0102380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009990 tx=0x7f94d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 -- 172.21.15.46:0/1960634447 shutdown_connections 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 --2- 172.21.15.46:0/1960634447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 0x7f94e010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 --2- 172.21.15.46:0/1960634447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e0102380 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 --2- 172.21.15.46:0/1960634447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 -- 172.21.15.46:0/1960634447 >> 172.21.15.46:0/1960634447 conn(0x7f94e00fb760 msgr2=0x7f94e00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 -- 172.21.15.46:0/1960634447 shutdown_connections 2024-05-08T23:14:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.745+0000 7f94e7200700 1 -- 172.21.15.46:0/1960634447 wait complete. 2024-05-08T23:14:12.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 Processor -- start 2024-05-08T23:14:12.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 -- start start 2024-05-08T23:14:12.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 0x7f94e019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e019e400 con 0x7f94e0101fa0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e019e570 con 0x7f94e010b3c0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.746+0000 7f94e7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e019e730 con 0x7f94e01028c0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50856/0 (socket says 172.21.15.46:50856) 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 -- 172.21.15.46:0/1118635305 learned_addr learned my addr 172.21.15.46:0/1118635305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e01994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58314/0 (socket says 172.21.15.46:58314) 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94e5200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 0x7f94e019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 -- 172.21.15.46:0/1118635305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 msgr2=0x7f94e019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 0x7f94e019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 -- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 msgr2=0x7f94e01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 -- 172.21.15.46:0/1118635305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8009c40 con 0x7f94e01028c0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.747+0000 7f94df400700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9f0 tx=0x7f94cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.748+0000 7f94dcc00700 1 -- 172.21.15.46:0/1118635305 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00eee0 con 0x7f94e01028c0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.748+0000 7f94dcc00700 1 -- 172.21.15.46:0/1118635305 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94cc00ce80 con 0x7f94e01028c0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.748+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d00095f0 con 0x7f94e01028c0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.748+0000 7f94dcc00700 1 -- 172.21.15.46:0/1118635305 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc018420 con 0x7f94e01028c0 2024-05-08T23:14:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.748+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e01a6070 con 0x7f94e01028c0 2024-05-08T23:14:12.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.752+0000 7f94dcc00700 1 -- 172.21.15.46:0/1118635305 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f94cc00c9c0 con 0x7f94e01028c0 2024-05-08T23:14:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.755+0000 7f94dcc00700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 0x7f94c40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.755+0000 7f94dcc00700 1 -- 172.21.15.46:0/1118635305 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f94cc014070 con 0x7f94e01028c0 2024-05-08T23:14:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.756+0000 7f94fad20b80 1 Processor -- start 2024-05-08T23:14:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.756+0000 7f94fad20b80 1 -- start start 2024-05-08T23:14:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0f80 0x558e5d6aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6ab0f0 0x558e5d6ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d5f9150 con 0x558e5d6b0f80 2024-05-08T23:14:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d5f6b10 con 0x558e5d6ab0f0 2024-05-08T23:14:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.757+0000 7f94fad20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d5f2400 con 0x558e5d6b0660 2024-05-08T23:14:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.758+0000 7f94dfe00700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 0x7f94c40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.759+0000 7f94e5200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6ab0f0 0x558e5d6ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.759+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.759+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50864/0 (socket says 172.21.15.46:50864) 2024-05-08T23:14:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.759+0000 7f94dfe00700 1 -- 172.21.15.46:0/4215642533 learned_addr learned my addr 172.21.15.46:0/4215642533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94df400700 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0f80 0x558e5d6aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94dfe00700 1 -- 172.21.15.46:0/4215642533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6ab0f0 msgr2=0x558e5d6ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94dfe00700 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6ab0f0 0x558e5d6ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94dfe00700 1 -- 172.21.15.46:0/4215642533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0f80 msgr2=0x558e5d6aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94dfe00700 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0f80 0x558e5d6aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.761+0000 7f94dfe00700 1 -- 172.21.15.46:0/4215642533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5d4733b0 con 0x558e5d6b0660 2024-05-08T23:14:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.762+0000 7f94dfe00700 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94d0015000 tx=0x7f94d00056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a1836f28f37880 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.764+0000 7f94dfe00700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 0x7f94c40a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94d0024000 tx=0x7f94d001f000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.765+0000 7f94d5600700 1 -- 172.21.15.46:0/4215642533 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d001d070 con 0x558e5d6b0660 2024-05-08T23:14:12.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.765+0000 7f94d5600700 1 -- 172.21.15.46:0/4215642533 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94d0005060 con 0x558e5d6b0660 2024-05-08T23:14:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.765+0000 7f94d5600700 1 -- 172.21.15.46:0/4215642533 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d00297f0 con 0x558e5d6b0660 2024-05-08T23:14:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 -- 172.21.15.46:0/4215642533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 msgr2=0x558e5d6b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94d0015000 tx=0x7f94d00056b0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 -- 172.21.15.46:0/4215642533 shutdown_connections 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6ab0f0 0x558e5d6ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0660 0x558e5d6b0a40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4215642533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0f80 0x558e5d6aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.766+0000 7f94fad20b80 1 -- 172.21.15.46:0/4215642533 >> 172.21.15.46:0/4215642533 conn(0x558e5d600210 msgr2=0x558e5d69cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.767+0000 7f94fad20b80 1 -- 172.21.15.46:0/4215642533 shutdown_connections 2024-05-08T23:14:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.767+0000 7f94fad20b80 1 -- 172.21.15.46:0/4215642533 wait complete. 2024-05-08T23:14:12.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.769+0000 7f94fad20b80 1 -- start start 2024-05-08T23:14:12.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.770+0000 7f94fad20b80 1 Processor -- start 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 0x558e5d6b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0f80 0x558e5d6b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d58e560 con 0x558e5d6b0660 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d58e200 con 0x558e5d6b0f80 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d4756a0 con 0x558e5d6ab0f0 2024-05-08T23:14:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.771+0000 7f94fad20b80 10 client.0 _ll_register_callbacks cb 0x558e5d597b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.773+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.773+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50866/0 (socket says 172.21.15.46:50866) 2024-05-08T23:14:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.773+0000 7f94dfe00700 1 -- 172.21.15.46:0/2972395917 learned_addr learned my addr 172.21.15.46:0/2972395917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.773+0000 7f94e5200700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0f80 0x558e5d6b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.773+0000 7f94df400700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 0x558e5d6b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.774+0000 7f94df400700 1 -- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 msgr2=0x558e5d6b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.774+0000 7f94df400700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.774+0000 7f94df400700 1 -- 172.21.15.46:0/2972395917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0f80 msgr2=0x558e5d6b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.774+0000 7f94df400700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0f80 0x558e5d6b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.774+0000 7f94df400700 1 -- 172.21.15.46:0/2972395917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d0020040 con 0x558e5d6b0660 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.775+0000 7f94df400700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 0x558e5d6b4830 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ca760 tx=0x7f94cc0cacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bc9c055a5d439 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.775+0000 7f94ca000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:14:12.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.775+0000 7f94dfe00700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:12.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.775+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc09c070 con 0x558e5d6b0660 2024-05-08T23:14:12.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.775+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f94cc0a25a0 con 0x558e5d6b0660 2024-05-08T23:14:12.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.776+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e5d54b270 con 0x558e5d6b0660 2024-05-08T23:14:12.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.776+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94cc0984c0 con 0x558e5d6b0660 2024-05-08T23:14:12.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.776+0000 7f94fad20b80 10 client.18378 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:12.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.776+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e5d41fb60 con 0x558e5d6b0660 2024-05-08T23:14:12.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.777+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e5d54b270 con 0x558e5d6b0660 2024-05-08T23:14:12.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.777+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94cc099610 con 0x558e5d6b0660 2024-05-08T23:14:12.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.777+0000 7f94fad20b80 10 client.18378 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:12.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.778+0000 7f94fad20b80 10 client.18378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:14:12.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.778+0000 7f94fad20b80 20 client.18378 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:12.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.778+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d6c49b0 0x558e5d6c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.778+0000 7f94fad20b80 4 client.18378 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:12.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.778+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e5d671810 con 0x558e5d6c49b0 2024-05-08T23:14:12.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.779+0000 7f94dfe00700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d6c49b0 0x558e5d6c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.780+0000 7f94dfe00700 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d6c49b0 0x558e5d6c6dd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=425be913b6d7a4cd server_cookie=c387037e9bd4171e in_seq=0 out_seq=0 2024-05-08T23:14:12.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.780+0000 7f94ca000700 10 client.18378 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:12.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.781+0000 7f94ca000700 1 -- 172.21.15.46:0/2972395917 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e5d671810 con 0x558e5d6c49b0 2024-05-08T23:14:12.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.781+0000 7f94ca000700 10 client.18378 handle_command_reply: tid=0 2024-05-08T23:14:12.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.781+0000 7f94fad20b80 1 client.18378 shutdown 2024-05-08T23:14:12.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 msgr2=0x558e5d6b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 0x558e5d6b4830 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ca760 tx=0x7f94cc0cacc0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 shutdown_connections 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d6c49b0 0x558e5d6c6dd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0f80 0x558e5d6b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6ab0f0 0x558e5d6b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 --2- 172.21.15.46:0/2972395917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6b0660 0x558e5d6b4830 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 >> 172.21.15.46:0/2972395917 conn(0x558e5d600210 msgr2=0x558e5d69c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94ca000700 0 client.18378 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 shutdown_connections 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 1 -- 172.21.15.46:0/2972395917 wait complete. 2024-05-08T23:14:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.782+0000 7f94fad20b80 20 client.18378 trim_cache size 0 max 16384 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.797+0000 7f94fad20b80 1 Processor -- start 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.797+0000 7f94fad20b80 1 -- start start 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d69cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d6cbfc0 con 0x558e5d6ab0f0 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d697a90 con 0x558e5d6b0660 2024-05-08T23:14:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94fad20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d69d260 con 0x558e5d6b0f80 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94e5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d69cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54876/0 (socket says 172.21.15.46:54876) 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 -- 172.21.15.46:0/4010702875 learned_addr learned my addr 172.21.15.46:0/4010702875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94e5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d69cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50878/0 (socket says 172.21.15.46:50878) 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 -- 172.21.15.46:0/4010702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 msgr2=0x558e5d69cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d69cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 -- 172.21.15.46:0/4010702875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 msgr2=0x558e5d6c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.798+0000 7f94df400700 1 -- 172.21.15.46:0/4010702875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5d5b8560 con 0x558e5d6b0660 2024-05-08T23:14:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.799+0000 7f94df400700 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9c0 tx=0x7f94cc09e0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b23c335856298f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.799+0000 7f94d6000700 1 -- 172.21.15.46:0/4010702875 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0a25a0 con 0x558e5d6b0660 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.799+0000 7f94d6000700 1 -- 172.21.15.46:0/4010702875 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94cc0b2070 con 0x558e5d6b0660 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.799+0000 7f94d6000700 1 -- 172.21.15.46:0/4010702875 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc09a380 con 0x558e5d6b0660 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- 172.21.15.46:0/4010702875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 msgr2=0x558e5d6c03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9c0 tx=0x7f94cc09e0d0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- 172.21.15.46:0/4010702875 shutdown_connections 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d69cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 --2- 172.21.15.46:0/4010702875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6c03c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- 172.21.15.46:0/4010702875 >> 172.21.15.46:0/4010702875 conn(0x558e5d600210 msgr2=0x558e5d6ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- 172.21.15.46:0/4010702875 shutdown_connections 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- 172.21.15.46:0/4010702875 wait complete. 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.800+0000 7f94fad20b80 1 -- start start 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 Processor -- start 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d6ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d6acd40 con 0x558e5d6ab0f0 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d6aceb0 con 0x558e5d6b0660 2024-05-08T23:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5d6ad070 con 0x558e5d6b0f80 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94fad20b80 10 client.0 _ll_register_callbacks cb 0x558e5d597b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94e5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d6ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94dfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58340/0 (socket says 172.21.15.46:58340) 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.801+0000 7f94dfe00700 1 -- 172.21.15.46:0/3921530869 learned_addr learned my addr 172.21.15.46:0/3921530869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 -- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 msgr2=0x558e5d6ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d6ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 -- 172.21.15.46:0/3921530869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 msgr2=0x558e5d6bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 -- 172.21.15.46:0/3921530869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e5d41fb60 con 0x558e5d6b0660 2024-05-08T23:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94dfe00700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94df400700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6ac060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558e5d69dde0 tx=0x7f94cc099cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13efdec5828e366 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94ca000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc09c020 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f94cc0a25a0 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94d0005060 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94cc0a23b0 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94fad20b80 10 client.28391 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.802+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e5d477cd0 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94cc07f080 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94cc07c070 con 0x558e5d6b0660 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 10 client.28391 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 10 client.28391 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 20 client.28391 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d71b200 0x558e5d71d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 4 client.28391 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.803+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e5d5eaec0 con 0x558e5d71b200 2024-05-08T23:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.804+0000 7f94dfe00700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d71b200 0x558e5d71d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.804+0000 7f94ca000700 10 client.28391 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.804+0000 7f94dfe00700 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d71b200 0x558e5d71d5f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb9ad82697618ff7 server_cookie=da8caa2427bd4cb8 in_seq=0 out_seq=0 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94ca000700 1 -- 172.21.15.46:0/3921530869 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e5d5eaec0 con 0x558e5d71b200 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94ca000700 10 client.28391 handle_command_reply: tid=0 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 client.28391 shutdown 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 msgr2=0x558e5d6ac060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6ac060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558e5d69dde0 tx=0x7f94cc099cc0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 shutdown_connections 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558e5d71b200 0x558e5d71d5f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e5d6b0f80 0x558e5d6ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e5d6ab0f0 0x558e5d6bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 --2- 172.21.15.46:0/3921530869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e5d6b0660 0x558e5d6ac060 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94ca000700 0 client.28391 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 >> 172.21.15.46:0/3921530869 conn(0x558e5d600210 msgr2=0x558e5d6b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 shutdown_connections 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 1 -- 172.21.15.46:0/3921530869 wait complete. 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.805+0000 7f94fad20b80 20 client.28391 trim_cache size 0 max 16384 2024-05-08T23:14:12.809 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 msgr2=0x7f94c40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 0x7f94c40a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94d0024000 tx=0x7f94d001f000 comp rx=0 tx=0).stop 2024-05-08T23:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 msgr2=0x7f94e0199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9f0 tx=0x7f94cc00ed00 comp rx=0 tx=0).stop 2024-05-08T23:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 shutdown_connections 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e010b3c0 0x7f94e019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94c40a1310 0x7f94c40a37d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94e0101fa0 0x7f94e01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 --2- 172.21.15.46:0/1118635305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e01028c0 0x7f94e0199a20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 >> 172.21.15.46:0/1118635305 conn(0x7f94e00fb760 msgr2=0x7f94e00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 shutdown_connections 2024-05-08T23:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:12.806+0000 7f94e7200700 1 -- 172.21.15.46:0/1118635305 wait complete. 2024-05-08T23:14:12.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:14:12.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 901885 9.50 MB/sec execute 394 sec latency 47.257 ms 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 -- 172.21.15.46:0/2143790225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98101f40 msgr2=0x7f5e98102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 --2- 172.21.15.46:0/2143790225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98101f40 0x7f5e98102320 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009990 tx=0x7f5e88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 -- 172.21.15.46:0/2143790225 shutdown_connections 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 --2- 172.21.15.46:0/2143790225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e9810b360 0x7f5e9810d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 --2- 172.21.15.46:0/2143790225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98101f40 0x7f5e98102320 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 --2- 172.21.15.46:0/2143790225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e98102860 0x7f5e9810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 -- 172.21.15.46:0/2143790225 >> 172.21.15.46:0/2143790225 conn(0x7f5e980fb760 msgr2=0x7f5e980fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 -- 172.21.15.46:0/2143790225 shutdown_connections 2024-05-08T23:14:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.069+0000 7f5e9e600700 1 -- 172.21.15.46:0/2143790225 wait complete. 2024-05-08T23:14:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.071+0000 7f5e9e600700 1 Processor -- start 2024-05-08T23:14:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.071+0000 7f5e9e600700 1 -- start start 2024-05-08T23:14:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.071+0000 7f5e9e600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e98101f40 0x7f5e98199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.071+0000 7f5e9e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e9e600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e9810b360 0x7f5e9819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e9e600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9819e3a0 con 0x7f5e9810b360 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e9e600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9819e510 con 0x7f5e98101f40 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e9e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9819e6d0 con 0x7f5e98102860 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e97400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e98101f40 0x7f5e98199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50904/0 (socket says 172.21.15.46:50904) 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 -- 172.21.15.46:0/886635465 learned_addr learned my addr 172.21.15.46:0/886635465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e97e00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e9810b360 0x7f5e9819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 -- 172.21.15.46:0/886635465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e98101f40 msgr2=0x7f5e98199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e98101f40 0x7f5e98199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 -- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e9810b360 msgr2=0x7f5e9819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e9810b360 0x7f5e9819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 -- 172.21.15.46:0/886635465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880095f0 con 0x7f5e98102860 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.072+0000 7f5e96a00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400e9f0 tx=0x7f5e8400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.073+0000 7f5e8fe00700 1 -- 172.21.15.46:0/886635465 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400c9c0 con 0x7f5e98102860 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.073+0000 7f5e8fe00700 1 -- 172.21.15.46:0/886635465 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e84010430 con 0x7f5e98102860 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.073+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9819e9b0 con 0x7f5e98102860 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.073+0000 7f5e8fe00700 1 -- 172.21.15.46:0/886635465 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84018680 con 0x7f5e98102860 2024-05-08T23:14:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.073+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e981a61f0 con 0x7f5e98102860 2024-05-08T23:14:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.076+0000 7f5e8fe00700 1 -- 172.21.15.46:0/886635465 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5e84010860 con 0x7f5e98102860 2024-05-08T23:14:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.078+0000 7f5eb21b5b80 1 Processor -- start 2024-05-08T23:14:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.078+0000 7f5e8fe00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 0x7f5e7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5e8fe00700 1 -- 172.21.15.46:0/886635465 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5e84014070 con 0x7f5e98102860 2024-05-08T23:14:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 -- start start 2024-05-08T23:14:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5e97e00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 0x7f5e7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597936a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b597931570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59787f150 con 0x55b5979310f0 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59787cb10 con 0x55b597936660 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b597878400 con 0x55b597936f80 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.079+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597936a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50914/0 (socket says 172.21.15.46:50914) 2024-05-08T23:14:13.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b597931570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e96a00700 1 -- 172.21.15.46:0/3983693108 learned_addr learned my addr 172.21.15.46:0/3983693108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97e00700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 0x7f5e7c0a37d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009550 tx=0x7f5e90009130 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 -- 172.21.15.46:0/3983693108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 msgr2=0x55b597930b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 -- 172.21.15.46:0/3983693108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 msgr2=0x55b597931570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b597931570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 -- 172.21.15.46:0/3983693108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5976f93b0 con 0x55b597936660 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e96a00700 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e97400700 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597936a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e8800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2318ab41c80556 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.080+0000 7f5e8cc00700 1 -- 172.21.15.46:0/3983693108 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8801d070 con 0x55b597936660 2024-05-08T23:14:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5e8cc00700 1 -- 172.21.15.46:0/3983693108 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e8801e040 con 0x55b597936660 2024-05-08T23:14:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5e8cc00700 1 -- 172.21.15.46:0/3983693108 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88004790 con 0x55b597936660 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/3983693108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 msgr2=0x55b597936a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597936a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e8800b610 comp rx=0 tx=0).stop 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/3983693108 shutdown_connections 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b597931570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597936a40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/3983693108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597930b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.081+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/3983693108 >> 172.21.15.46:0/3983693108 conn(0x55b597886210 msgr2=0x55b597922df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.082+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/3983693108 shutdown_connections 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.082+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/3983693108 wait complete. 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.082+0000 7f5eb21b5b80 1 -- start start 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 Processor -- start 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b59793a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b597814560 con 0x55b597936660 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b597814200 con 0x55b5979310f0 2024-05-08T23:14:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5976fb6a0 con 0x55b597936f80 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.083+0000 7f5eb21b5b80 10 client.0 _ll_register_callbacks cb 0x55b59781db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e97400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58354/0 (socket says 172.21.15.46:58354) 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 -- 172.21.15.46:0/2281472605 learned_addr learned my addr 172.21.15.46:0/2281472605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e97400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54928/0 (socket says 172.21.15.46:54928) 2024-05-08T23:14:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e97e00700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b59793a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 -- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 msgr2=0x55b59793a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b59793a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 -- 172.21.15.46:0/2281472605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 msgr2=0x55b59793cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 -- 172.21.15.46:0/2281472605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e981a61f0 con 0x55b597936660 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e97e00700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b59793a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e97400700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e96a00700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e8409a020 tx=0x7f5e8409a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a08654e7245645 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.084+0000 7f5e81600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840b1070 con 0x55b597936660 2024-05-08T23:14:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5e840a2510 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5977d1270 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e840b8350 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 10 client.18396 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5e9810e4e0 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5977d1270 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e840b8070 con 0x55b597936660 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 10 client.18396 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 10 client.18396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.085+0000 7f5eb21b5b80 20 client.18396 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b59794a8d0 0x55b59794ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5eb21b5b80 4 client.18396 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5978f7810 con 0x55b59794a8d0 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5e97400700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b59794a8d0 0x55b59794ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5e97400700 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b59794a8d0 0x55b59794ccf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e77bd32546dab746 server_cookie=34206eaeaa0402b5 in_seq=0 out_seq=0 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.086+0000 7f5e81600700 10 client.18396 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5e81600700 1 -- 172.21.15.46:0/2281472605 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5978f7810 con 0x55b59794a8d0 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5e81600700 10 client.18396 handle_command_reply: tid=0 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 client.18396 shutdown 2024-05-08T23:14:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 msgr2=0x55b59793a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e8409a020 tx=0x7f5e8409a800 comp rx=0 tx=0).stop 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 shutdown_connections 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b59794a8d0 0x55b59794ccf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b59793a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5979310f0 0x55b59793cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2281472605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793a2d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5e81600700 0 client.18396 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 >> 172.21.15.46:0/2281472605 conn(0x55b597886210 msgr2=0x55b5979225a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 shutdown_connections 2024-05-08T23:14:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.087+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2281472605 wait complete. 2024-05-08T23:14:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.088+0000 7f5eb21b5b80 20 client.18396 trim_cache size 0 max 16384 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.102+0000 7f5eb21b5b80 1 Processor -- start 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.102+0000 7f5eb21b5b80 1 -- start start 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597922d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597939810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59791da90 con 0x55b5979310f0 2024-05-08T23:14:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b597923260 con 0x55b597936660 2024-05-08T23:14:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59793a920 con 0x55b597936f80 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5e97400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5e97e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597939810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5e97400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58366/0 (socket says 172.21.15.46:58366) 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5e97400700 1 -- 172.21.15.46:0/481280193 learned_addr learned my addr 172.21.15.46:0/481280193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.103+0000 7f5e96a00700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597922d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 -- 172.21.15.46:0/481280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 msgr2=0x55b597922d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597922d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 -- 172.21.15.46:0/481280193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 msgr2=0x55b59793a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 -- 172.21.15.46:0/481280193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5976a5b60 con 0x55b597936f80 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e96a00700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597922d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97400700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e97e00700 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597939810 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5e90015540 tx=0x7f5e90013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3489a2211dc079ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e8d600700 1 -- 172.21.15.46:0/481280193 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90023430 con 0x55b597936f80 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e8d600700 1 -- 172.21.15.46:0/481280193 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9000f040 con 0x55b597936f80 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.104+0000 7f5e8d600700 1 -- 172.21.15.46:0/481280193 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9002b9b0 con 0x55b597936f80 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/481280193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 msgr2=0x55b597939810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597939810 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5e90015540 tx=0x7f5e90013000 comp rx=0 tx=0).stop 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/481280193 shutdown_connections 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b597936f80 0x55b597939810 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5979310f0 0x55b59793a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/481280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936660 0x55b597922d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/481280193 >> 172.21.15.46:0/481280193 conn(0x55b597886210 msgr2=0x55b59791f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/481280193 shutdown_connections 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.105+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/481280193 wait complete. 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 -- start start 2024-05-08T23:14:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 Processor -- start 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936f80 0x55b59793fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5979405a0 con 0x55b597936660 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b597940710 con 0x55b597936f80 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59799fdd0 con 0x55b5979310f0 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5eb21b5b80 10 client.0 _ll_register_callbacks cb 0x7f5e84099c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.106+0000 7f5e97400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50954/0 (socket says 172.21.15.46:50954) 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 -- 172.21.15.46:0/2678994262 learned_addr learned my addr 172.21.15.46:0/2678994262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 -- 172.21.15.46:0/2678994262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936f80 msgr2=0x55b59793fe90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936f80 0x55b59793fe90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 -- 172.21.15.46:0/2678994262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 msgr2=0x55b59793f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 -- 172.21.15.46:0/2678994262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5976f93b0 con 0x55b5979310f0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e97400700 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e88026000 tx=0x7f5e8801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c38e09c6379ce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e81600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.107+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88032040 con 0x55b5979310f0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f5e8800b280 con 0x55b5979310f0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e8801e040 con 0x55b5979310f0 2024-05-08T23:14:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e8801d040 con 0x55b5979310f0 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5eb21b5b80 10 client.28495 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5e9810e4e0 con 0x55b5979310f0 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e8801e040 con 0x55b5979310f0 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.108+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e88004620 con 0x55b5979310f0 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 10 client.28495 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 10 client.28495 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 20 client.28495 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b5979a11d0 0x55b5979a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 4 client.28495 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b597870ec0 con 0x55b5979a11d0 2024-05-08T23:14:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5e96a00700 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b5979a11d0 0x55b5979a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5e96a00700 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b5979a11d0 0x55b5979a35c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b25e95c8f5ffeb41 server_cookie=d0b35c831c332d6e in_seq=0 out_seq=0 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.109+0000 7f5e81600700 10 client.28495 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5e81600700 1 -- 172.21.15.46:0/2678994262 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b597870ec0 con 0x55b5979a11d0 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5e81600700 10 client.28495 handle_command_reply: tid=0 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 client.28495 shutdown 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 msgr2=0x55b597941d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e88026000 tx=0x7f5e8801a040 comp rx=0 tx=0).stop 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 shutdown_connections 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b5979a11d0 0x55b5979a35c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b597936f80 0x55b59793fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5979310f0 0x55b597941d10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5e81600700 0 client.28495 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 --2- 172.21.15.46:0/2678994262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b597936660 0x55b59793f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 >> 172.21.15.46:0/2678994262 conn(0x55b597886210 msgr2=0x55b5979343f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 shutdown_connections 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 1 -- 172.21.15.46:0/2678994262 wait complete. 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.110+0000 7f5eb21b5b80 20 client.28495 trim_cache size 0 max 16384 2024-05-08T23:14:13.115 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 msgr2=0x7f5e7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 0x7f5e7c0a37d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009550 tx=0x7f5e90009130 comp rx=0 tx=0).stop 2024-05-08T23:14:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 msgr2=0x7f5e981999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400e9f0 tx=0x7f5e8400ed00 comp rx=0 tx=0).stop 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 shutdown_connections 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5e7c0a1310 0x7f5e7c0a37d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e9810b360 0x7f5e9819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e98101f40 0x7f5e98199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 --2- 172.21.15.46:0/886635465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98102860 0x7f5e981999c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 >> 172.21.15.46:0/886635465 conn(0x7f5e980fb760 msgr2=0x7f5e9810c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 shutdown_connections 2024-05-08T23:14:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.111+0000 7f5e9e600700 1 -- 172.21.15.46:0/886635465 wait complete. 2024-05-08T23:14:13.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 -- 172.21.15.46:0/3718577167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 msgr2=0x7f9d3c073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 --2- 172.21.15.46:0/3718577167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c073720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009990 tx=0x7f9d2c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 -- 172.21.15.46:0/3718577167 shutdown_connections 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 --2- 172.21.15.46:0/3718577167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 --2- 172.21.15.46:0/3718577167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c073720 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 --2- 172.21.15.46:0/3718577167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 0x7f9d3c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 -- 172.21.15.46:0/3718577167 >> 172.21.15.46:0/3718577167 conn(0x7f9d3c0fc8e0 msgr2=0x7f9d3c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 -- 172.21.15.46:0/3718577167 shutdown_connections 2024-05-08T23:14:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.354+0000 7f9d42a00700 1 -- 172.21.15.46:0/3718577167 wait complete. 2024-05-08T23:14:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.355+0000 7f9d42a00700 1 Processor -- start 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.355+0000 7f9d42a00700 1 -- start start 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 0x7f9d3c1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19de20 con 0x7f9d3c073340 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19df90 con 0x7f9d3c073c60 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d42a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19e150 con 0x7f9d3c111e20 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50958/0 (socket says 172.21.15.46:50958) 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 -- 172.21.15.46:0/2512463973 learned_addr learned my addr 172.21.15.46:0/2512463973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 0x7f9d3c1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 -- 172.21.15.46:0/2512463973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 msgr2=0x7f9d3c1a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 0x7f9d3c1a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 -- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 msgr2=0x7f9d3c1a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c1a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.356+0000 7f9d40a00700 1 -- 172.21.15.46:0/2512463973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0095f0 con 0x7f9d3c111e20 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400ef90 tx=0x7f9d3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d38c00700 1 -- 172.21.15.46:0/2512463973 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34018070 con 0x7f9d3c111e20 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c19e3d0 con 0x7f9d3c111e20 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d38c00700 1 -- 172.21.15.46:0/2512463973 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3400f040 con 0x7f9d3c111e20 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d38c00700 1 -- 172.21.15.46:0/2512463973 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3400cda0 con 0x7f9d3c111e20 2024-05-08T23:14:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.357+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c1aa670 con 0x7f9d3c111e20 2024-05-08T23:14:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.361+0000 7f9d38c00700 1 -- 172.21.15.46:0/2512463973 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d34003a00 con 0x7f9d3c111e20 2024-05-08T23:14:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.362+0000 7f9d56447b80 1 Processor -- start 2024-05-08T23:14:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.362+0000 7f9d56447b80 1 -- start start 2024-05-08T23:14:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.362+0000 7f9d38c00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 0x7f9d200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d38c00700 1 -- 172.21.15.46:0/2512463973 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f9d340cccf0 con 0x7f9d3c111e20 2024-05-08T23:14:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf042350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 0x557ddf042cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddef99150 con 0x557ddf042890 2024-05-08T23:14:13.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddef96b10 con 0x557ddf048720 2024-05-08T23:14:13.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d56447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddef92400 con 0x557ddf047e00 2024-05-08T23:14:13.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 0x7f9d200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50968/0 (socket says 172.21.15.46:50968) 2024-05-08T23:14:13.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.363+0000 7f9d3be00700 1 -- 172.21.15.46:0/2085691190 learned_addr learned my addr 172.21.15.46:0/2085691190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 0x557ddf042cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 -- 172.21.15.46:0/2085691190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 msgr2=0x557ddf042350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf042350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf042350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 -- 172.21.15.46:0/2085691190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 msgr2=0x557ddf042cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 0x557ddf042cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 -- 172.21.15.46:0/2085691190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ddee133b0 con 0x557ddf047e00 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 0x557ddf042cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 0x7f9d200a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c19ebd0 tx=0x7f9d28009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.364+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005ec0 tx=0x7f9d2c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687ca6f07d770671 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.365+0000 7f9d31200700 1 -- 172.21.15.46:0/2085691190 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01d070 con 0x557ddf047e00 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.365+0000 7f9d31200700 1 -- 172.21.15.46:0/2085691190 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d2c01e040 con 0x557ddf047e00 2024-05-08T23:14:13.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.365+0000 7f9d31200700 1 -- 172.21.15.46:0/2085691190 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004860 con 0x557ddf047e00 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.365+0000 7f9d56447b80 1 -- 172.21.15.46:0/2085691190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 msgr2=0x557ddf0481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.365+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005ec0 tx=0x7f9d2c005ef0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 -- 172.21.15.46:0/2085691190 shutdown_connections 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf042890 0x557ddf042cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf0481e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2085691190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf042350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 -- 172.21.15.46:0/2085691190 >> 172.21.15.46:0/2085691190 conn(0x557ddefa0210 msgr2=0x557ddf03cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 -- 172.21.15.46:0/2085691190 shutdown_connections 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 -- 172.21.15.46:0/2085691190 wait complete. 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 -- start start 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.366+0000 7f9d56447b80 1 Processor -- start 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf04c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf04c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddef2e560 con 0x557ddf047e00 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddef2e200 con 0x557ddf048720 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddee156a0 con 0x557ddf042890 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d56447b80 10 client.0 _ll_register_callbacks cb 0x557ddef37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d3b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf04c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50970/0 (socket says 172.21.15.46:50970) 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.367+0000 7f9d3be00700 1 -- 172.21.15.46:0/2759972688 learned_addr learned my addr 172.21.15.46:0/2759972688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf04c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 -- 172.21.15.46:0/2759972688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 msgr2=0x557ddf04c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf04c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 -- 172.21.15.46:0/2759972688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 msgr2=0x557ddf04c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf04c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 -- 172.21.15.46:0/2759972688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c1aa670 con 0x557ddf042890 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d40a00700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf04c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d3be00700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005320 tx=0x7f9d2c00ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c2038c79e612d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d25600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01d030 con 0x557ddf042890 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f9d2c004910 con 0x557ddf042890 2024-05-08T23:14:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.368+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ddeeeb270 con 0x557ddf042890 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d2c00bd90 con 0x557ddf042890 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d56447b80 10 client.28507 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f9d3c114bc0 con 0x557ddf042890 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ddeeeb270 con 0x557ddf042890 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d2c00fab0 con 0x557ddf042890 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d56447b80 10 client.28507 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d56447b80 10 client.28507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.369+0000 7f9d56447b80 20 client.28507 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf05bed0 0x557ddf05e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d56447b80 4 client.28507 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ddf011810 con 0x557ddf05bed0 2024-05-08T23:14:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf05bed0 0x557ddf05e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d3b400700 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf05bed0 0x557ddf05e2f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3add16ae2684999f server_cookie=2dc611b79652b7cb in_seq=0 out_seq=0 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.370+0000 7f9d25600700 10 client.28507 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d25600700 1 -- 172.21.15.46:0/2759972688 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ddf011810 con 0x557ddf05bed0 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d25600700 10 client.28507 handle_command_reply: tid=0 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 client.28507 shutdown 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 msgr2=0x557ddf04bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005320 tx=0x7f9d2c00ba00 comp rx=0 tx=0).stop 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 shutdown_connections 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf04c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf04bbf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf05bed0 0x557ddf05e2f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.371+0000 7f9d56447b80 1 --2- 172.21.15.46:0/2759972688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf04c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.372+0000 7f9d25600700 0 client.28507 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.372+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 >> 172.21.15.46:0/2759972688 conn(0x557ddefa0210 msgr2=0x557ddf03c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.372+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 shutdown_connections 2024-05-08T23:14:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.372+0000 7f9d56447b80 1 -- 172.21.15.46:0/2759972688 wait complete. 2024-05-08T23:14:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.372+0000 7f9d56447b80 20 client.28507 trim_cache size 0 max 16384 2024-05-08T23:14:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.389+0000 7f9d56447b80 1 Processor -- start 2024-05-08T23:14:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 -- start start 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf042890 0x557ddf047570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 0x557ddf04abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf063500 con 0x557ddf048720 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf049bc0 con 0x557ddf042890 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d56447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf045870 con 0x557ddf047e00 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50976/0 (socket says 172.21.15.46:50976) 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3b400700 1 -- 172.21.15.46:0/3456758679 learned_addr learned my addr 172.21.15.46:0/3456758679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3be00700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf042890 0x557ddf047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d40a00700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 0x557ddf04abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3b400700 1 -- 172.21.15.46:0/3456758679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf042890 msgr2=0x557ddf047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.390+0000 7f9d3b400700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf042890 0x557ddf047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d3b400700 1 -- 172.21.15.46:0/3456758679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 msgr2=0x557ddf04abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d3b400700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 0x557ddf04abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d3b400700 1 -- 172.21.15.46:0/3456758679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ddf042f40 con 0x557ddf047e00 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d40a00700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 0x557ddf04abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d3b400700 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d28016000 tx=0x7f9d28019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951c0cfd74c3de45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d31c00700 1 -- 172.21.15.46:0/3456758679 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2802b070 con 0x557ddf047e00 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d31c00700 1 -- 172.21.15.46:0/3456758679 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d2801ce00 con 0x557ddf047e00 2024-05-08T23:14:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.391+0000 7f9d31c00700 1 -- 172.21.15.46:0/3456758679 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28022040 con 0x557ddf047e00 2024-05-08T23:14:13.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- 172.21.15.46:0/3456758679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 msgr2=0x557ddf03cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d28016000 tx=0x7f9d28019000 comp rx=0 tx=0).stop 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- 172.21.15.46:0/3456758679 shutdown_connections 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf048720 0x557ddf04abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf042890 0x557ddf047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 --2- 172.21.15.46:0/3456758679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf047e00 0x557ddf03cd20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- 172.21.15.46:0/3456758679 >> 172.21.15.46:0/3456758679 conn(0x557ddefa0210 msgr2=0x557ddf04a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- 172.21.15.46:0/3456758679 shutdown_connections 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- 172.21.15.46:0/3456758679 wait complete. 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.392+0000 7f9d56447b80 1 -- start start 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 Processor -- start 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf0518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf0b1910 con 0x557ddf047e00 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf0b1a50 con 0x557ddf048720 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ddf0b1b90 con 0x557ddf042890 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d56447b80 10 client.0 _ll_register_callbacks cb 0x557ddef37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:50980/0 (socket says 172.21.15.46:50980) 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.393+0000 7f9d3be00700 1 -- 172.21.15.46:0/1852506389 learned_addr learned my addr 172.21.15.46:0/1852506389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3b400700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf0518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d40a00700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 -- 172.21.15.46:0/1852506389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 msgr2=0x557ddf0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 -- 172.21.15.46:0/1852506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 msgr2=0x557ddf0518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf0518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 -- 172.21.15.46:0/1852506389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d28017050 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3b400700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf0518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d3be00700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00fe40 tx=0x7f9d2c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b714feb7aa6c922d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d25600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01d070 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d2c01e040 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f9d2c02c550 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d2c005bc0 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d56447b80 10 client.28513 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.394+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ddedbfb60 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d2c01e040 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d2c00f460 con 0x557ddf042890 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 10 client.28513 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 10 client.28513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 20 client.28513 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf0b2eb0 0x557ddf0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 4 client.28513 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ddef8aec0 con 0x557ddf0b2eb0 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.395+0000 7f9d3b400700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf0b2eb0 0x557ddf0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.396+0000 7f9d3b400700 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf0b2eb0 0x557ddf0b52d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecea50ebe327ed5f server_cookie=5e46b7a9706e3acc in_seq=0 out_seq=0 2024-05-08T23:14:13.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.396+0000 7f9d25600700 10 client.28513 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:13.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.397+0000 7f9d25600700 1 -- 172.21.15.46:0/1852506389 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ddef8aec0 con 0x557ddf0b2eb0 2024-05-08T23:14:13.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.397+0000 7f9d25600700 10 client.28513 handle_command_reply: tid=0 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.397+0000 7f9d56447b80 1 client.28513 shutdown 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.397+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 msgr2=0x557ddf051390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.397+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00fe40 tx=0x7f9d2c00be60 comp rx=0 tx=0).stop 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 shutdown_connections 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ddf048720 0x557ddf0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ddf042890 0x557ddf051390 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557ddf0b2eb0 0x557ddf0b52d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 --2- 172.21.15.46:0/1852506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ddf047e00 0x557ddf0518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d25600700 0 client.28513 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 >> 172.21.15.46:0/1852506389 conn(0x557ddefa0210 msgr2=0x557ddf0473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 shutdown_connections 2024-05-08T23:14:13.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 1 -- 172.21.15.46:0/1852506389 wait complete. 2024-05-08T23:14:13.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d56447b80 20 client.28513 trim_cache size 0 max 16384 2024-05-08T23:14:13.402 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:13.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 msgr2=0x7f9d200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.398+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 0x7f9d200a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c19ebd0 tx=0x7f9d28009260 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 msgr2=0x7f9d3c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400ef90 tx=0x7f9d3400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 shutdown_connections 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c111e20 0x7f9d3c19d7b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9d200a1290 0x7f9d200a3750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3c073340 0x7f9d3c1a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 --2- 172.21.15.46:0/2512463973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d3c073c60 0x7f9d3c1a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 >> 172.21.15.46:0/2512463973 conn(0x7f9d3c0fc8e0 msgr2=0x7f9d3c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 shutdown_connections 2024-05-08T23:14:13.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.399+0000 7f9d42a00700 1 -- 172.21.15.46:0/2512463973 wait complete. 2024-05-08T23:14:13.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:13 smithi046 ceph-mon[36898]: pgmap v322: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-05-08T23:14:13.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2082541936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:13.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 -- 172.21.15.46:0/3164735222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 msgr2=0x7f633810d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 --2- 172.21.15.46:0/3164735222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f633810d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6324009990 tx=0x7f6324009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 -- 172.21.15.46:0/3164735222 shutdown_connections 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 --2- 172.21.15.46:0/3164735222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633810d6d0 0x7f633810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 --2- 172.21.15.46:0/3164735222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6338073370 0x7f6338073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 --2- 172.21.15.46:0/3164735222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f633810d190 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 -- 172.21.15.46:0/3164735222 >> 172.21.15.46:0/3164735222 conn(0x7f63380fc960 msgr2=0x7f63380fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 -- 172.21.15.46:0/3164735222 shutdown_connections 2024-05-08T23:14:13.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.649+0000 7f633ee00700 1 -- 172.21.15.46:0/3164735222 wait complete. 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.650+0000 7f633ee00700 1 Processor -- start 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.650+0000 7f633ee00700 1 -- start start 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6338073370 0x7f633810cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f633810d6d0 0x7f633810a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633810abd0 con 0x7f633810d6d0 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633810ad40 con 0x7f6338073370 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381a9ef0 con 0x7f6338073c90 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51002/0 (socket says 172.21.15.46:51002) 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 -- 172.21.15.46:0/1648465050 learned_addr learned my addr 172.21.15.46:0/1648465050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f633ce00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f633810d6d0 0x7f633810a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337e00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6338073370 0x7f633810cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 -- 172.21.15.46:0/1648465050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6338073370 msgr2=0x7f633810cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6338073370 0x7f633810cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 -- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f633810d6d0 msgr2=0x7f633810a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f633810d6d0 0x7f633810a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.651+0000 7f6337400700 1 -- 172.21.15.46:0/1648465050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240095f0 con 0x7f6338073c90 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f6337400700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6324005910 tx=0x7f6324004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f6334c00700 1 -- 172.21.15.46:0/1648465050 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632401d070 con 0x7f6338073c90 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f6334c00700 1 -- 172.21.15.46:0/1648465050 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6324022470 con 0x7f6338073c90 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f6334c00700 1 -- 172.21.15.46:0/1648465050 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324017650 con 0x7f6338073c90 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63381aa090 con 0x7f6338073c90 2024-05-08T23:14:13.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.652+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63381aa5a0 con 0x7f6338073c90 2024-05-08T23:14:13.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.656+0000 7f6334c00700 1 -- 172.21.15.46:0/1648465050 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f63240225e0 con 0x7f6338073c90 2024-05-08T23:14:13.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 Processor -- start 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 -- start start 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6334c00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 0x7f631c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6334c00700 1 -- 172.21.15.46:0/1648465050 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f63240ce350 con 0x7f6338073c90 2024-05-08T23:14:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533cad150 con 0x55c533d5be00 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.657+0000 7f6352937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533caab10 con 0x55c533d56890 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6352937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533ca6400 con 0x55c533d5c720 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f633ce00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 0x7f631c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58444/0 (socket says 172.21.15.46:58444) 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 -- 172.21.15.46:0/1676461399 learned_addr learned my addr 172.21.15.46:0/1676461399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f633ce00700 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337400700 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 -- 172.21.15.46:0/1676461399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 msgr2=0x55c533d56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 -- 172.21.15.46:0/1676461399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 msgr2=0x55c533d56cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d56cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.658+0000 7f6337e00700 1 -- 172.21.15.46:0/1676461399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c533b273b0 con 0x55c533d5be00 2024-05-08T23:14:13.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.659+0000 7f633ce00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 0x7f631c0a37d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6330009ec0 tx=0x7f63300134a0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.659+0000 7f6337e00700 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6328009d80 tx=0x7f6328011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add23b405319e1a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.659+0000 7f632d600700 1 -- 172.21.15.46:0/1676461399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328004230 con 0x55c533d5be00 2024-05-08T23:14:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.659+0000 7f632d600700 1 -- 172.21.15.46:0/1676461399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328004390 con 0x55c533d5be00 2024-05-08T23:14:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.659+0000 7f632d600700 1 -- 172.21.15.46:0/1676461399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280197a0 con 0x55c533d5be00 2024-05-08T23:14:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 -- 172.21.15.46:0/1676461399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 msgr2=0x55c533d5c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6328009d80 tx=0x7f6328011720 comp rx=0 tx=0).stop 2024-05-08T23:14:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 -- 172.21.15.46:0/1676461399 shutdown_connections 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5c1e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 --2- 172.21.15.46:0/1676461399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 -- 172.21.15.46:0/1676461399 >> 172.21.15.46:0/1676461399 conn(0x55c533cb4210 msgr2=0x55c533d50df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 -- 172.21.15.46:0/1676461399 shutdown_connections 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.660+0000 7f6352937b80 1 -- 172.21.15.46:0/1676461399 wait complete. 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 -- start start 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 Processor -- start 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533c42560 con 0x55c533d5be00 2024-05-08T23:14:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533c42200 con 0x55c533d56890 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533b296a0 con 0x55c533d5c720 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.661+0000 7f6352937b80 10 client.0 _ll_register_callbacks cb 0x55c533c4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55058/0 (socket says 172.21.15.46:55058) 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 -- 172.21.15.46:0/4210048600 learned_addr learned my addr 172.21.15.46:0/4210048600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f633ce00700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 -- 172.21.15.46:0/4210048600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 msgr2=0x55c533d604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 -- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 msgr2=0x55c533d5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 -- 172.21.15.46:0/4210048600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280099e0 con 0x55c533d56890 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6337e00700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f632801b040 tx=0x7f63280038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f2fdf6c4342f36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6322000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.662+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328003c30 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f6328005040 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c533bff270 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6328005240 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6352937b80 10 client.28433 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c533ad3b60 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c533bff270 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.663+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6328028020 con 0x55c533d56890 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 10 client.28433 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 10 client.28433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 20 client.28433 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533d70160 0x55c533d72580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 4 client.28433 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c533d25810 con 0x55c533d70160 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.664+0000 7f6337400700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533d70160 0x55c533d72580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6337400700 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533d70160 0x55c533d72580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9a3be5bb3c18a54 server_cookie=6b4221c358668347 in_seq=0 out_seq=0 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6322000700 10 client.28433 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6322000700 1 -- 172.21.15.46:0/4210048600 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c533d25810 con 0x55c533d70160 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6322000700 10 client.28433 handle_command_reply: tid=0 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6352937b80 1 client.28433 shutdown 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 msgr2=0x55c533d5fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.665+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f632801b040 tx=0x7f63280038b0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 shutdown_connections 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d56890 0x55c533d5fa70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533d70160 0x55c533d72580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 --2- 172.21.15.46:0/4210048600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d5be00 0x55c533d5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 >> 172.21.15.46:0/4210048600 conn(0x55c533cb4210 msgr2=0x55c533d505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6322000700 0 client.28433 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 shutdown_connections 2024-05-08T23:14:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 1 -- 172.21.15.46:0/4210048600 wait complete. 2024-05-08T23:14:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.666+0000 7f6352937b80 20 client.28433 trim_cache size 0 max 16384 2024-05-08T23:14:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 Processor -- start 2024-05-08T23:14:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 -- start start 2024-05-08T23:14:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533d5e0a0 con 0x55c533d56890 2024-05-08T23:14:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533d6cc20 con 0x55c533d5be00 2024-05-08T23:14:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6352937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533d5c1e0 con 0x55c533d5c720 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6337400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f633ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6337400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55066/0 (socket says 172.21.15.46:55066) 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58464/0 (socket says 172.21.15.46:58464) 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f6337e00700 1 -- 172.21.15.46:0/1653698594 learned_addr learned my addr 172.21.15.46:0/1653698594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.682+0000 7f633ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51038/0 (socket says 172.21.15.46:51038) 2024-05-08T23:14:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 -- 172.21.15.46:0/1653698594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 msgr2=0x55c533d50d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 -- 172.21.15.46:0/1653698594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 msgr2=0x55c533d56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 -- 172.21.15.46:0/1653698594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c533c6c560 con 0x55c533d5c720 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f6337400700 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f6337e00700 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f633ce00700 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6330006130 tx=0x7f633001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ccc9fdae8f49bae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.683+0000 7f632e000700 1 -- 172.21.15.46:0/1653698594 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000e040 con 0x55c533d5c720 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f632e000700 1 -- 172.21.15.46:0/1653698594 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f633000f040 con 0x55c533d5c720 2024-05-08T23:14:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f632e000700 1 -- 172.21.15.46:0/1653698594 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330008d80 con 0x55c533d5c720 2024-05-08T23:14:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 -- 172.21.15.46:0/1653698594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 msgr2=0x55c533d5ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6330006130 tx=0x7f633001d000 comp rx=0 tx=0).stop 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 -- 172.21.15.46:0/1653698594 shutdown_connections 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533d5ea70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 --2- 172.21.15.46:0/1653698594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.684+0000 7f6352937b80 1 -- 172.21.15.46:0/1653698594 >> 172.21.15.46:0/1653698594 conn(0x55c533cb4210 msgr2=0x55c533d5e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.685+0000 7f6352937b80 1 -- 172.21.15.46:0/1653698594 shutdown_connections 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.685+0000 7f6352937b80 1 -- 172.21.15.46:0/1653698594 wait complete. 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.685+0000 7f6352937b80 1 -- start start 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.685+0000 7f6352937b80 1 Processor -- start 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d63a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533dbf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533dbfbf0 con 0x55c533d56890 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533dbfd60 con 0x55c533d5be00 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c533dbff20 con 0x55c533d5c720 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6352937b80 10 client.0 _ll_register_callbacks cb 0x55c533c4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58478/0 (socket says 172.21.15.46:58478) 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6337e00700 1 -- 172.21.15.46:0/2908485134 learned_addr learned my addr 172.21.15.46:0/2908485134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.686+0000 7f6337400700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d63a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 -- 172.21.15.46:0/2908485134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 msgr2=0x55c533dbf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533dbf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 -- 172.21.15.46:0/2908485134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 msgr2=0x55c533d63a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d63a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 -- 172.21.15.46:0/2908485134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c533ad3b60 con 0x55c533d56890 2024-05-08T23:14:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337400700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d63a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6337e00700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f632800e5d0 tx=0x7f6328004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9652899130439e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.687+0000 7f6322000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:14:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800e3b0 con 0x55c533d56890 2024-05-08T23:14:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f63280224b0 con 0x55c533d56890 2024-05-08T23:14:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6328004390 con 0x55c533d56890 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6328004710 con 0x55c533d56890 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6352937b80 10 client.18432 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f632409d040 con 0x55c533d56890 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.688+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328004390 con 0x55c533d56890 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6328011790 con 0x55c533d56890 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 10 client.18432 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 10 client.18432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 20 client.18432 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533dc7320 0x55c533dc9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 4 client.18432 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c533c9eec0 con 0x55c533dc7320 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.689+0000 7f6337400700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533dc7320 0x55c533dc9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6337400700 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533dc7320 0x55c533dc9710 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5105c93461a36062 server_cookie=8f552b8983e850d0 in_seq=0 out_seq=0 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6322000700 10 client.18432 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6322000700 1 -- 172.21.15.46:0/2908485134 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c533c9eec0 con 0x55c533dc7320 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6322000700 10 client.18432 handle_command_reply: tid=0 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 client.18432 shutdown 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 msgr2=0x55c533d63550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f632800e5d0 tx=0x7f6328004280 comp rx=0 tx=0).stop 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 shutdown_connections 2024-05-08T23:14:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c533d5c720 0x55c533dbf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c533dc7320 0x55c533dc9710 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c533d56890 0x55c533d63550 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 --2- 172.21.15.46:0/2908485134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c533d5be00 0x55c533d63a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 >> 172.21.15.46:0/2908485134 conn(0x55c533cb4210 msgr2=0x55c533d5d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.690+0000 7f6322000700 0 client.18432 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.691+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 shutdown_connections 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.691+0000 7f6352937b80 1 -- 172.21.15.46:0/2908485134 wait complete. 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.691+0000 7f6352937b80 20 client.18432 trim_cache size 0 max 16384 2024-05-08T23:14:13.694 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 msgr2=0x7f631c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 0x7f631c0a37d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6330009ec0 tx=0x7f63300134a0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 msgr2=0x7f6338109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6324005910 tx=0x7f6324004730 comp rx=0 tx=0).stop 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 shutdown_connections 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f631c0a1310 0x7f631c0a37d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f633810d6d0 0x7f633810a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6338073370 0x7f633810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 --2- 172.21.15.46:0/1648465050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338073c90 0x7f6338109fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 >> 172.21.15.46:0/1648465050 conn(0x7f63380fc960 msgr2=0x7f63380fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 shutdown_connections 2024-05-08T23:14:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.692+0000 7f633ee00700 1 -- 172.21.15.46:0/1648465050 wait complete. 2024-05-08T23:14:13.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:14:13.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:13 smithi086 ceph-mon[78610]: pgmap v322: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-05-08T23:14:13.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2082541936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:13.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:13 smithi053 ceph-mon[77587]: pgmap v322: 129 pgs: 129 active+clean; 537 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-05-08T23:14:13.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2082541936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:13.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 903241 9.49 MB/sec execute 395 sec latency 17.947 ms 2024-05-08T23:14:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 -- 172.21.15.46:0/1368067384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514101a30 msgr2=0x7f8514105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 --2- 172.21.15.46:0/1368067384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514101a30 0x7f8514105a70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8500009990 tx=0x7f8500009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 -- 172.21.15.46:0/1368067384 shutdown_connections 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 --2- 172.21.15.46:0/1368067384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f851410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 --2- 172.21.15.46:0/1368067384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 0x7f8514101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 --2- 172.21.15.46:0/1368067384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514101a30 0x7f8514105a70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 -- 172.21.15.46:0/1368067384 >> 172.21.15.46:0/1368067384 conn(0x7f85140fc8e0 msgr2=0x7f85140fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 -- 172.21.15.46:0/1368067384 shutdown_connections 2024-05-08T23:14:13.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.937+0000 7f8518c00700 1 -- 172.21.15.46:0/1368067384 wait complete. 2024-05-08T23:14:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.938+0000 7f8518c00700 1 Processor -- start 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.938+0000 7f8518c00700 1 -- start start 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 0x7f85140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f85140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141a9fb0 con 0x7f85141061a0 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141aa120 con 0x7f8514101080 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8518c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141aa2e0 con 0x7f8514074f60 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51056/0 (socket says 172.21.15.46:51056) 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 -- 172.21.15.46:0/2443313293 learned_addr learned my addr 172.21.15.46:0/2443313293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8511400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f85140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8511e00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 0x7f85140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 -- 172.21.15.46:0/2443313293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 msgr2=0x7f85140732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 0x7f85140732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 -- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 msgr2=0x7f85140737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f85140737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8512800700 1 -- 172.21.15.46:0/2443313293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009c40 con 0x7f8514074f60 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.939+0000 7f8511400700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f85140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f8512800700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c370 tx=0x7f850c00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f850aa00700 1 -- 172.21.15.46:0/2443313293 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c017070 con 0x7f8514074f60 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85000095f0 con 0x7f8514074f60 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f850aa00700 1 -- 172.21.15.46:0/2443313293 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c00f040 con 0x7f8514074f60 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f850aa00700 1 -- 172.21.15.46:0/2443313293 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00e050 con 0x7f8514074f60 2024-05-08T23:14:13.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.940+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141aa900 con 0x7f8514074f60 2024-05-08T23:14:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.944+0000 7f850aa00700 1 -- 172.21.15.46:0/2443313293 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f850c004520 con 0x7f8514074f60 2024-05-08T23:14:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.945+0000 7f852c6edb80 1 Processor -- start 2024-05-08T23:14:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 -- start start 2024-05-08T23:14:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f850aa00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 0x7f84f80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 0x558a461a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f850aa00700 1 -- 172.21.15.46:0/2443313293 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f850c0cca10 con 0x7f8514074f60 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 0x558a461a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a460f7150 con 0x558a461a5e00 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a460f4b10 con 0x558a461a0890 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f852c6edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a460f0400 con 0x558a461a6720 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f8511400700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 0x7f84f80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 0x558a461a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.946+0000 7f8511e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 0x558a461a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51072/0 (socket says 172.21.15.46:51072) 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 0x558a461a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8511400700 1 -- 172.21.15.46:0/2364181788 learned_addr learned my addr 172.21.15.46:0/2364181788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 -- 172.21.15.46:0/2364181788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 msgr2=0x558a461a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 0x558a461a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 -- 172.21.15.46:0/2364181788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 msgr2=0x558a461a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 -- 172.21.15.46:0/2364181788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a45f713b0 con 0x558a461a0890 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8512800700 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 0x558a461a0cd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f850c0c9ca0 tx=0x7f850c099430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a252a3c335c9e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f8511400700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 0x7f84f80a37d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8500005cd0 tx=0x7f8500005c60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.947+0000 7f84ff000700 1 -- 172.21.15.46:0/2364181788 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c099d60 con 0x558a461a0890 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f84ff000700 1 -- 172.21.15.46:0/2364181788 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c099660 con 0x558a461a0890 2024-05-08T23:14:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f84ff000700 1 -- 172.21.15.46:0/2364181788 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0047d0 con 0x558a461a0890 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 -- 172.21.15.46:0/2364181788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 msgr2=0x558a461a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 0x558a461a0cd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f850c0c9ca0 tx=0x7f850c099430 comp rx=0 tx=0).stop 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 -- 172.21.15.46:0/2364181788 shutdown_connections 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a0890 0x558a461a0cd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2364181788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a6720 0x558a461a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 -- 172.21.15.46:0/2364181788 >> 172.21.15.46:0/2364181788 conn(0x558a460fe210 msgr2=0x558a4619adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 -- 172.21.15.46:0/2364181788 shutdown_connections 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.948+0000 7f852c6edb80 1 -- 172.21.15.46:0/2364181788 wait complete. 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.949+0000 7f852c6edb80 1 -- start start 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.949+0000 7f852c6edb80 1 Processor -- start 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4608c560 con 0x558a461a0890 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4608c200 con 0x558a461a6720 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a45f736a0 con 0x558a461a5e00 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f852c6edb80 10 client.0 _ll_register_callbacks cb 0x558a46095b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51086/0 (socket says 172.21.15.46:51086) 2024-05-08T23:14:13.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 -- 172.21.15.46:0/2872712924 learned_addr learned my addr 172.21.15.46:0/2872712924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8512800700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511e00700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 -- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 msgr2=0x558a461aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 -- 172.21.15.46:0/2872712924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 msgr2=0x558a461a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511400700 1 -- 172.21.15.46:0/2872712924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141aa900 con 0x558a461a5e00 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.950+0000 7f8511e00700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f8511400700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8500006010 tx=0x7f8500009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d276d27924102a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f84f3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d070 con 0x558a461a5e00 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a46049270 con 0x558a461a5e00 2024-05-08T23:14:13.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f8500004e00 con 0x558a461a5e00 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8500029470 con 0x558a461a5e00 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f852c6edb80 10 client.28540 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558a45f1db60 con 0x558a461a5e00 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.951+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a46049270 con 0x558a461a5e00 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8500031070 con 0x558a461a5e00 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 10 client.28540 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 10 client.28540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 20 client.28540 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a461ba050 0x558a461bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 4 client.28540 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a4616f810 con 0x558a461ba050 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.952+0000 7f8511e00700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a461ba050 0x558a461bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.953+0000 7f8511e00700 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a461ba050 0x558a461bc470 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9730dc09b4bb1f85 server_cookie=bd041ffa3f913cfc in_seq=0 out_seq=0 2024-05-08T23:14:13.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.953+0000 7f84f3400700 10 client.28540 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.953+0000 7f84f3400700 1 -- 172.21.15.46:0/2872712924 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a4616f810 con 0x558a461ba050 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.953+0000 7f84f3400700 10 client.28540 handle_command_reply: tid=0 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.953+0000 7f852c6edb80 1 client.28540 shutdown 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 msgr2=0x558a461a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8500006010 tx=0x7f8500009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 shutdown_connections 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a461ba050 0x558a461bc470 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2872712924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461a9ff0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 >> 172.21.15.46:0/2872712924 conn(0x558a460fe210 msgr2=0x558a4619a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f84f3400700 0 client.28540 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:13.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 shutdown_connections 2024-05-08T23:14:13.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 1 -- 172.21.15.46:0/2872712924 wait complete. 2024-05-08T23:14:13.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.954+0000 7f852c6edb80 20 client.28540 trim_cache size 0 max 16384 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 Processor -- start 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 -- start start 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a4619ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a461ab120 con 0x558a461a5e00 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a46197dc0 con 0x558a461a6720 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.969+0000 7f852c6edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4619c800 con 0x558a461a0890 2024-05-08T23:14:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8512800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a4619ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51102/0 (socket says 172.21.15.46:51102) 2024-05-08T23:14:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 -- 172.21.15.46:0/2627701247 learned_addr learned my addr 172.21.15.46:0/2627701247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 -- 172.21.15.46:0/2627701247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 msgr2=0x558a4619ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a4619ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 -- 172.21.15.46:0/2627701247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 msgr2=0x558a461a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 -- 172.21.15.46:0/2627701247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a460b6560 con 0x558a461a0890 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f8511e00700 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8504009c90 tx=0x7f8504009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41edabf3adb2991c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.970+0000 7f84ffa00700 1 -- 172.21.15.46:0/2627701247 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850401f270 con 0x558a461a0890 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f84ffa00700 1 -- 172.21.15.46:0/2627701247 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8504018040 con 0x558a461a0890 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f84ffa00700 1 -- 172.21.15.46:0/2627701247 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504011eb0 con 0x558a461a0890 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 -- 172.21.15.46:0/2627701247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 msgr2=0x558a461a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8504009c90 tx=0x7f8504009260 comp rx=0 tx=0).stop 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 -- 172.21.15.46:0/2627701247 shutdown_connections 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a4619ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a0890 0x558a461a0350 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 --2- 172.21.15.46:0/2627701247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a5e00 0x558a461a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 -- 172.21.15.46:0/2627701247 >> 172.21.15.46:0/2627701247 conn(0x558a460fe210 msgr2=0x558a461a3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.971+0000 7f852c6edb80 1 -- 172.21.15.46:0/2627701247 shutdown_connections 2024-05-08T23:14:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.972+0000 7f852c6edb80 1 -- 172.21.15.46:0/2627701247 wait complete. 2024-05-08T23:14:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.972+0000 7f852c6edb80 1 -- start start 2024-05-08T23:14:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.972+0000 7f852c6edb80 1 Processor -- start 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a461afcb0 con 0x558a461a0890 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a461afe20 con 0x558a461a6720 2024-05-08T23:14:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4620f510 con 0x558a461a5e00 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f852c6edb80 10 client.0 _ll_register_callbacks cb 0x558a46095b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51108/0 (socket says 172.21.15.46:51108) 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 -- 172.21.15.46:0/3995425421 learned_addr learned my addr 172.21.15.46:0/3995425421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8512800700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 -- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 msgr2=0x558a461af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 -- 172.21.15.46:0/3995425421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 msgr2=0x558a461b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.973+0000 7f8511400700 1 -- 172.21.15.46:0/3995425421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8504009a70 con 0x558a461a5e00 2024-05-08T23:14:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f8511400700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f850000f690 tx=0x7f850000fa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49126eb58dfc3284 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f84f3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d070 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850c099660 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f850000fdd0 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850000f460 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f852c6edb80 10 client.28546 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.974+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558a45f75cd0 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c099660 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8500004500 con 0x558a461a5e00 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 10 client.28546 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 10 client.28546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 20 client.28546 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a462107b0 0x558a46212bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 4 client.28546 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a460e8ec0 con 0x558a462107b0 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.975+0000 7f8511e00700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a462107b0 0x558a46212bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f8511e00700 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a462107b0 0x558a46212bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964dec38d269d932 server_cookie=9bbdefd936a60cab in_seq=0 out_seq=0 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f84f3400700 10 client.28546 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f84f3400700 1 -- 172.21.15.46:0/3995425421 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a460e8ec0 con 0x558a462107b0 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f84f3400700 10 client.28546 handle_command_reply: tid=0 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 client.28546 shutdown 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 msgr2=0x558a461af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f850000f690 tx=0x7f850000fa60 comp rx=0 tx=0).stop 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 shutdown_connections 2024-05-08T23:14:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a461a6720 0x558a461af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558a462107b0 0x558a46212bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a461a0890 0x558a461b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 --2- 172.21.15.46:0/3995425421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a461a5e00 0x558a461af090 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f84f3400700 0 client.28546 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 >> 172.21.15.46:0/3995425421 conn(0x558a460fe210 msgr2=0x558a461a7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 shutdown_connections 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 1 -- 172.21.15.46:0/3995425421 wait complete. 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.976+0000 7f852c6edb80 20 client.28546 trim_cache size 0 max 16384 2024-05-08T23:14:13.980 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 msgr2=0x7f84f80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 0x7f84f80a37d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8500005cd0 tx=0x7f8500005c60 comp rx=0 tx=0).stop 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 msgr2=0x7f8514073d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c370 tx=0x7f850c00c680 comp rx=0 tx=0).stop 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 shutdown_connections 2024-05-08T23:14:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8514074f60 0x7f8514073d60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8514101080 0x7f85140732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f80a1310 0x7f84f80a37d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 --2- 172.21.15.46:0/2443313293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85141061a0 0x7f85140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 >> 172.21.15.46:0/2443313293 conn(0x7f85140fc8e0 msgr2=0x7f85140fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 shutdown_connections 2024-05-08T23:14:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:13.977+0000 7f8518c00700 1 -- 172.21.15.46:0/2443313293 wait complete. 2024-05-08T23:14:14.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 905300 9.50 MB/sec execute 396 sec latency 17.258 ms 2024-05-08T23:14:14.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:14:14.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:14:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 -- 172.21.15.46:0/116241856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c010d6d0 msgr2=0x7f70c010fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 --2- 172.21.15.46:0/116241856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c010d6d0 0x7f70c010fb00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f70b800b0a0 tx=0x7f70b800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 -- 172.21.15.46:0/116241856 shutdown_connections 2024-05-08T23:14:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 --2- 172.21.15.46:0/116241856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c010d6d0 0x7f70c010fb00 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 --2- 172.21.15.46:0/116241856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 0x7f70c0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 --2- 172.21.15.46:0/116241856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c0073c90 0x7f70c010d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 -- 172.21.15.46:0/116241856 >> 172.21.15.46:0/116241856 conn(0x7f70c00fc960 msgr2=0x7f70c00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 -- 172.21.15.46:0/116241856 shutdown_connections 2024-05-08T23:14:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.221+0000 7f70c6e00700 1 -- 172.21.15.46:0/116241856 wait complete. 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.222+0000 7f70c6e00700 1 Processor -- start 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 -- start start 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 0x7f70c0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 0x7f70c019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019e5b0 con 0x7f70c0073370 2024-05-08T23:14:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019e720 con 0x7f70c010d6d0 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019e8e0 con 0x7f70c0073c90 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 0x7f70c0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51128/0 (socket says 172.21.15.46:51128) 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70bf400700 1 -- 172.21.15.46:0/2220588007 learned_addr learned my addr 172.21.15.46:0/2220588007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.223+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 0x7f70c019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 -- 172.21.15.46:0/2220588007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 msgr2=0x7f70c019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 0x7f70c019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 -- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 msgr2=0x7f70c0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 0x7f70c0199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 -- 172.21.15.46:0/2220588007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b8009d00 con 0x7f70c0073c90 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 0x7f70c019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bf400700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9c0 tx=0x7f70ac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bcc00700 1 -- 172.21.15.46:0/2220588007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0097b0 con 0x7f70c0073c90 2024-05-08T23:14:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c019ebc0 con 0x7f70c0073c90 2024-05-08T23:14:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bcc00700 1 -- 172.21.15.46:0/2220588007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac00ee60 con 0x7f70c0073c90 2024-05-08T23:14:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70bcc00700 1 -- 172.21.15.46:0/2220588007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0054f0 con 0x7f70c0073c90 2024-05-08T23:14:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.224+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c01a6400 con 0x7f70c0073c90 2024-05-08T23:14:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.228+0000 7f70bcc00700 1 -- 172.21.15.46:0/2220588007 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f70ac004750 con 0x7f70c0073c90 2024-05-08T23:14:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.229+0000 7f70da979b80 1 Processor -- start 2024-05-08T23:14:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70bcc00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 0x7f70a40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 -- start start 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70bcc00700 1 -- 172.21.15.46:0/2220588007 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f70ac014070 con 0x7f70c0073c90 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 0x7f70a40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702fe80 0x55b8d7030260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6f811d0 con 0x55b8d702fe80 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6f7eb90 con 0x55b8d702a910 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70da979b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6f7a480 con 0x55b8d70307a0 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.230+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702fe80 0x55b8d7030260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51134/0 (socket says 172.21.15.46:51134) 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 -- 172.21.15.46:0/3474955762 learned_addr learned my addr 172.21.15.46:0/3474955762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 -- 172.21.15.46:0/3474955762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 msgr2=0x55b8d702ad50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702ad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 -- 172.21.15.46:0/3474955762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702fe80 msgr2=0x55b8d7030260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702fe80 0x55b8d7030260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 -- 172.21.15.46:0/3474955762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8d6dfb430 con 0x55b8d70307a0 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 0x7f70a40a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b800b780 tx=0x7f70b8006230 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70bf400700 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0a1ec0 tx=0x7f70ac005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ec3546bf8e8af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70b5600700 1 -- 172.21.15.46:0/3474955762 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac099f70 con 0x55b8d70307a0 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70b5600700 1 -- 172.21.15.46:0/3474955762 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac09dda0 con 0x55b8d70307a0 2024-05-08T23:14:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.231+0000 7f70b5600700 1 -- 172.21.15.46:0/3474955762 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac09a1d0 con 0x55b8d70307a0 2024-05-08T23:14:15.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 -- 172.21.15.46:0/3474955762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 msgr2=0x55b8d702a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0a1ec0 tx=0x7f70ac005730 comp rx=0 tx=0).stop 2024-05-08T23:14:15.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 -- 172.21.15.46:0/3474955762 shutdown_connections 2024-05-08T23:14:15.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702fe80 0x55b8d7030260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 --2- 172.21.15.46:0/3474955762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d70307a0 0x55b8d702a3d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 -- 172.21.15.46:0/3474955762 >> 172.21.15.46:0/3474955762 conn(0x55b8d6f88290 msgr2=0x55b8d7024e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.232+0000 7f70da979b80 1 -- 172.21.15.46:0/3474955762 shutdown_connections 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.233+0000 7f70da979b80 1 -- 172.21.15.46:0/3474955762 wait complete. 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.233+0000 7f70da979b80 1 -- start start 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.233+0000 7f70da979b80 1 Processor -- start 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 0x55b8d7033bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d70307a0 0x55b8d7034640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6f165e0 con 0x55b8d702a910 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6f16280 con 0x55b8d70307a0 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d6dfd720 con 0x55b8d702fe80 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70da979b80 10 client.0 _ll_register_callbacks cb 0x55b8d6f1fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51148/0 (socket says 172.21.15.46:51148) 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70bf400700 1 -- 172.21.15.46:0/2129225911 learned_addr learned my addr 172.21.15.46:0/2129225911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d70307a0 0x55b8d7034640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.234+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 0x55b8d7033bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 -- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d70307a0 msgr2=0x55b8d7034640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d70307a0 0x55b8d7034640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 -- 172.21.15.46:0/2129225911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 msgr2=0x55b8d7033bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 0x55b8d7033bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 -- 172.21.15.46:0/2129225911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c01a6400 con 0x55b8d702fe80 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 0x55b8d7033bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70bf400700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c5b0 tx=0x7f70ac0a2460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c8ad8af4d0af7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70aa000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0b8200 con 0x55b8d702fe80 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8d6ed32f0 con 0x55b8d702fe80 2024-05-08T23:14:15.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f70ac0ade40 con 0x55b8d702fe80 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70ac0b8360 con 0x55b8d702fe80 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70da979b80 10 client.28558 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.235+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b8d6da7be0 con 0x55b8d702fe80 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8d6ed32f0 con 0x55b8d702fe80 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70ac09a1d0 con 0x55b8d702fe80 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 10 client.28558 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 10 client.28558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 20 client.28558 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d7043d40 0x55b8d7046160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 4 client.28558 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8d6ff9890 con 0x55b8d7043d40 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.236+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d7043d40 0x55b8d7046160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.237+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d7043d40 0x55b8d7046160 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f82a09793f0a653 server_cookie=6dd1be1492e72851 in_seq=0 out_seq=0 2024-05-08T23:14:15.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.237+0000 7f70aa000700 10 client.28558 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70aa000700 1 -- 172.21.15.46:0/2129225911 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8d6ff9890 con 0x55b8d7043d40 2024-05-08T23:14:15.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70aa000700 10 client.28558 handle_command_reply: tid=0 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 client.28558 shutdown 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 msgr2=0x55b8d7034100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c5b0 tx=0x7f70ac0a2460 comp rx=0 tx=0).stop 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 shutdown_connections 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d70307a0 0x55b8d7034640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d7043d40 0x55b8d7046160 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d702a910 0x55b8d7033bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 --2- 172.21.15.46:0/2129225911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7034100 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 >> 172.21.15.46:0/2129225911 conn(0x55b8d6f88290 msgr2=0x55b8d7024620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70aa000700 0 client.28558 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 shutdown_connections 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.238+0000 7f70da979b80 1 -- 172.21.15.46:0/2129225911 wait complete. 2024-05-08T23:14:15.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.239+0000 7f70da979b80 20 client.28558 trim_cache size 0 max 16384 2024-05-08T23:14:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.260+0000 7f70da979b80 1 Processor -- start 2024-05-08T23:14:15.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.260+0000 7f70da979b80 1 -- start start 2024-05-08T23:14:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d703b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d701f8d0 con 0x55b8d70307a0 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d70366b0 con 0x55b8d702a910 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70da979b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d7035120 con 0x55b8d702fe80 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51162/0 (socket says 172.21.15.46:51162) 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 -- 172.21.15.46:0/2138802448 learned_addr learned my addr 172.21.15.46:0/2138802448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d703b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 -- 172.21.15.46:0/2138802448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 msgr2=0x55b8d702acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 -- 172.21.15.46:0/2138802448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 msgr2=0x55b8d703b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d703b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.261+0000 7f70bf400700 1 -- 172.21.15.46:0/2138802448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8d7021560 con 0x55b8d702fe80 2024-05-08T23:14:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.262+0000 7f70bf400700 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00f010 tx=0x7f70ac0963e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1843c88fb365de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.262+0000 7f70b6000700 1 -- 172.21.15.46:0/2138802448 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac07c080 con 0x55b8d702fe80 2024-05-08T23:14:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.262+0000 7f70b6000700 1 -- 172.21.15.46:0/2138802448 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac005730 con 0x55b8d702fe80 2024-05-08T23:14:15.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.262+0000 7f70b6000700 1 -- 172.21.15.46:0/2138802448 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac098430 con 0x55b8d702fe80 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- 172.21.15.46:0/2138802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 msgr2=0x55b8d7024da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00f010 tx=0x7f70ac0963e0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- 172.21.15.46:0/2138802448 shutdown_connections 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d703b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d702acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 --2- 172.21.15.46:0/2138802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7024da0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- 172.21.15.46:0/2138802448 >> 172.21.15.46:0/2138802448 conn(0x55b8d6f88290 msgr2=0x55b8d702f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- 172.21.15.46:0/2138802448 shutdown_connections 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- 172.21.15.46:0/2138802448 wait complete. 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.263+0000 7f70da979b80 1 -- start start 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 Processor -- start 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7037820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d7037d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d70382e0 con 0x55b8d70307a0 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d7038450 con 0x55b8d702a910 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d7038610 con 0x55b8d702fe80 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.264+0000 7f70da979b80 10 client.0 _ll_register_callbacks cb 0x55b8d6f1fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7037820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55178/0 (socket says 172.21.15.46:55178) 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 -- 172.21.15.46:0/2137714973 learned_addr learned my addr 172.21.15.46:0/2137714973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70c4e00700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d7037d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 -- 172.21.15.46:0/2137714973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 msgr2=0x55b8d7037820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7037820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 -- 172.21.15.46:0/2137714973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 msgr2=0x55b8d7037d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d7037d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 -- 172.21.15.46:0/2137714973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8d6da7be0 con 0x55b8d702a910 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bf400700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7037820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70bfe00700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55b8d70469d0 tx=0x7f70b001e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14bd4b1cdc27e44f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.265+0000 7f70aa000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004a30 con 0x55b8d702a910 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70ac09dda0 con 0x55b8d702a910 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f70b0004b90 con 0x55b8d702a910 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b0004d50 con 0x55b8d702a910 2024-05-08T23:14:15.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70da979b80 10 client.28472 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b8d6f405e0 con 0x55b8d702a910 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.266+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac09dda0 con 0x55b8d702a910 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70b00039b0 con 0x55b8d702a910 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 10 client.28472 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 10 client.28472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 20 client.28472 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d709b800 0x55b8d709dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 4 client.28472 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b8d6f72f40 con 0x55b8d709b800 2024-05-08T23:14:15.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70bf400700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d709b800 0x55b8d709dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70bf400700 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d709b800 0x55b8d709dc20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b2f7b2c87d3a0f server_cookie=57ed7a37f0315cfa in_seq=0 out_seq=0 2024-05-08T23:14:15.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.267+0000 7f70aa000700 10 client.28472 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70aa000700 1 -- 172.21.15.46:0/2137714973 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b8d6f72f40 con 0x55b8d709b800 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70aa000700 10 client.28472 handle_command_reply: tid=0 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 client.28472 shutdown 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 msgr2=0x55b8d703b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55b8d70469d0 tx=0x7f70b001e7c0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 shutdown_connections 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b8d709b800 0x55b8d709dc20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8d70307a0 0x55b8d7037d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8d702a910 0x55b8d703b400 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 --2- 172.21.15.46:0/2137714973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8d702fe80 0x55b8d7037820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 >> 172.21.15.46:0/2137714973 conn(0x55b8d6f88290 msgr2=0x55b8d7035350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70aa000700 0 client.28472 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 shutdown_connections 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 1 -- 172.21.15.46:0/2137714973 wait complete. 2024-05-08T23:14:15.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.269+0000 7f70da979b80 20 client.28472 trim_cache size 0 max 16384 2024-05-08T23:14:15.272 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:14:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "de3556bf-032f-4cdf-b1ae-41575c1b6eae", 2024-05-08T23:14:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:14:15.272 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 msgr2=0x7f70a40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 0x7f70a40a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b800b780 tx=0x7f70b8006230 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 msgr2=0x7f70c0199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9c0 tx=0x7f70ac00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 shutdown_connections 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c010d6d0 0x7f70c019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f70a40a1290 0x7f70a40a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c0073370 0x7f70c0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 --2- 172.21.15.46:0/2220588007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c0073c90 0x7f70c0199bd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 >> 172.21.15.46:0/2220588007 conn(0x7f70c00fc960 msgr2=0x7f70c00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 shutdown_connections 2024-05-08T23:14:15.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.270+0000 7f70c6e00700 1 -- 172.21.15.46:0/2220588007 wait complete. 2024-05-08T23:14:15.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:14:15.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:15 smithi046 ceph-mon[36898]: pgmap v323: 129 pgs: 129 active+clean; 533 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 -- 172.21.15.46:0/1617500014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 msgr2=0x7f15a8103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 --2- 172.21.15.46:0/1617500014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a8103690 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15980098f0 tx=0x7f1598009c00 comp rx=0 tx=0).stop 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 -- 172.21.15.46:0/1617500014 shutdown_connections 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 --2- 172.21.15.46:0/1617500014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 --2- 172.21.15.46:0/1617500014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a8103690 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 --2- 172.21.15.46:0/1617500014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a8103c60 0x7f15a8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 -- 172.21.15.46:0/1617500014 >> 172.21.15.46:0/1617500014 conn(0x7f15a80feb10 msgr2=0x7f15a8100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 -- 172.21.15.46:0/1617500014 shutdown_connections 2024-05-08T23:14:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.525+0000 7f15ae800700 1 -- 172.21.15.46:0/1617500014 wait complete. 2024-05-08T23:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 Processor -- start 2024-05-08T23:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 -- start start 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a819de50 0x7f15a819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819e8d0 con 0x7f15a81032b0 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819ea40 con 0x7f15a81083d0 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.527+0000 7f15ae800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819ec00 con 0x7f15a819de50 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58558/0 (socket says 172.21.15.46:58558) 2024-05-08T23:14:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 -- 172.21.15.46:0/4117332211 learned_addr learned my addr 172.21.15.46:0/4117332211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7e00700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a819de50 0x7f15a819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 -- 172.21.15.46:0/4117332211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a819de50 msgr2=0x7f15a819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a819de50 0x7f15a819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 -- 172.21.15.46:0/4117332211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 msgr2=0x7f15a8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a7400700 1 -- 172.21.15.46:0/4117332211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940095f0 con 0x7f15a81032b0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.528+0000 7f15a6a00700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f15a7400700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1598005950 tx=0x7f159800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f159fe00700 1 -- 172.21.15.46:0/4117332211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159801c070 con 0x7f15a81032b0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f159fe00700 1 -- 172.21.15.46:0/4117332211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159800b9c0 con 0x7f15a81032b0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1598009520 con 0x7f15a81032b0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f159fe00700 1 -- 172.21.15.46:0/4117332211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15980038e0 con 0x7f15a81032b0 2024-05-08T23:14:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.529+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a81a63f0 con 0x7f15a81032b0 2024-05-08T23:14:15.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.534+0000 7f159fe00700 1 -- 172.21.15.46:0/4117332211 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1598020030 con 0x7f15a81032b0 2024-05-08T23:14:15.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.534+0000 7f15c2407b80 1 Processor -- start 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 -- start start 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c496f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c4969cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f159fe00700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 0x7f158c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c48c0150 con 0x55f1c496ee00 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c48bdb10 con 0x55f1c4969890 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15c2407b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c48b9400 con 0x55f1c496f720 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f159fe00700 1 -- 172.21.15.46:0/4117332211 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f15980cde60 con 0x7f15a81032b0 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51198/0 (socket says 172.21.15.46:51198) 2024-05-08T23:14:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15a7400700 1 -- 172.21.15.46:0/640750264 learned_addr learned my addr 172.21.15.46:0/640750264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15a7e00700 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c496f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.535+0000 7f15a6a00700 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c4969cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7e00700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 0x7f158c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 -- 172.21.15.46:0/640750264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 msgr2=0x55f1c4969cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c4969cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 -- 172.21.15.46:0/640750264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 msgr2=0x55f1c496f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c496f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 -- 172.21.15.46:0/640750264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1c473a3b0 con 0x55f1c496f720 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f15a7400700 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1598009810 tx=0x7f1598009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95edb93bfc83e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f159cc00700 1 -- 172.21.15.46:0/640750264 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598021380 con 0x55f1c496f720 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f159cc00700 1 -- 172.21.15.46:0/640750264 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159809b240 con 0x55f1c496f720 2024-05-08T23:14:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.536+0000 7f159cc00700 1 -- 172.21.15.46:0/640750264 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598005140 con 0x55f1c496f720 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 -- 172.21.15.46:0/640750264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 msgr2=0x55f1c4969350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1598009810 tx=0x7f1598009260 comp rx=0 tx=0).stop 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15a7e00700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 0x7f158c0a37d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009ec0 tx=0x7f15a000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 -- 172.21.15.46:0/640750264 shutdown_connections 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c4969cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c496f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 --2- 172.21.15.46:0/640750264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4969350 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 -- 172.21.15.46:0/640750264 >> 172.21.15.46:0/640750264 conn(0x55f1c48c7210 msgr2=0x55f1c4963df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 -- 172.21.15.46:0/640750264 shutdown_connections 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.537+0000 7f15c2407b80 1 -- 172.21.15.46:0/640750264 wait complete. 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 -- start start 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 Processor -- start 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c4972a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496f720 0x55f1c49730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c4855560 con 0x55f1c496ee00 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c4855200 con 0x55f1c496f720 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.538+0000 7f15c2407b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c473c6a0 con 0x55f1c4969890 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15c2407b80 10 client.0 _ll_register_callbacks cb 0x55f1c485eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51206/0 (socket says 172.21.15.46:51206) 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 -- 172.21.15.46:0/2882337021 learned_addr learned my addr 172.21.15.46:0/2882337021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7e00700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496f720 0x55f1c49730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a6a00700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c4972a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 -- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496f720 msgr2=0x55f1c49730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496f720 0x55f1c49730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 -- 172.21.15.46:0/2882337021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 msgr2=0x55f1c4972a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c4972a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 -- 172.21.15.46:0/2882337021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a81a63f0 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a6a00700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c4972a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f15a7400700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1598097040 tx=0x7f1598064870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f09cd4174de11e3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.539+0000 7f1591600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598005350 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f159809e970 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1c4812270 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f159800bb30 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 10 client.28576 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f1c46e6b60 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1c4812270 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15980a2aa0 con 0x55f1c4969890 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 10 client.28576 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 10 client.28576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 20 client.28576 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c4982930 0x55f1c4984d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 4 client.28576 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.540+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1c4938810 con 0x55f1c4982930 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.541+0000 7f15a6a00700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c4982930 0x55f1c4984d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.541+0000 7f15a6a00700 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c4982930 0x55f1c4984d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4c3ca0a510fece server_cookie=f851db0f2ac611d5 in_seq=0 out_seq=0 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.541+0000 7f1591600700 10 client.28576 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f1591600700 1 -- 172.21.15.46:0/2882337021 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1c4938810 con 0x55f1c4982930 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f1591600700 10 client.28576 handle_command_reply: tid=0 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 client.28576 shutdown 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 msgr2=0x55f1c49756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1598097040 tx=0x7f1598064870 comp rx=0 tx=0).stop 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 shutdown_connections 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496f720 0x55f1c49730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c4969890 0x55f1c49756a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c4982930 0x55f1c4984d20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2882337021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496ee00 0x55f1c4972a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 >> 172.21.15.46:0/2882337021 conn(0x55f1c48c7210 msgr2=0x55f1c49635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.542+0000 7f1591600700 0 client.28576 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.543+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 shutdown_connections 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.543+0000 7f15c2407b80 1 -- 172.21.15.46:0/2882337021 wait complete. 2024-05-08T23:14:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.543+0000 7f15c2407b80 20 client.28576 trim_cache size 0 max 16384 2024-05-08T23:14:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 Processor -- start 2024-05-08T23:14:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 -- start start 2024-05-08T23:14:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c4969890 0x55f1c4969350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 0x55f1c4963d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c4960e00 con 0x55f1c4969890 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c495e800 con 0x55f1c496ee00 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.562+0000 7f15c2407b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c4989f50 con 0x55f1c496f720 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 0x55f1c4963d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51214/0 (socket says 172.21.15.46:51214) 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c4969890 0x55f1c4969350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 -- 172.21.15.46:0/2492286386 learned_addr learned my addr 172.21.15.46:0/2492286386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 -- 172.21.15.46:0/2492286386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 msgr2=0x55f1c4963d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 0x55f1c4963d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 -- 172.21.15.46:0/2492286386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c4969890 msgr2=0x55f1c4969350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c4969890 0x55f1c4969350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 -- 172.21.15.46:0/2492286386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1c4969f40 con 0x55f1c496f720 2024-05-08T23:14:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a6a00700 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 0x55f1c4963d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.563+0000 7f15a7e00700 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f15a00186e0 tx=0x7f15a0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8d0e234f049c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f159d600700 1 -- 172.21.15.46:0/2492286386 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0028070 con 0x55f1c496f720 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f159d600700 1 -- 172.21.15.46:0/2492286386 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a000f040 con 0x55f1c496f720 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f159d600700 1 -- 172.21.15.46:0/2492286386 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000e070 con 0x55f1c496f720 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 -- 172.21.15.46:0/2492286386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 msgr2=0x55f1c4979e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f15a00186e0 tx=0x7f15a0009920 comp rx=0 tx=0).stop 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 -- 172.21.15.46:0/2492286386 shutdown_connections 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496f720 0x55f1c4979e70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c4969890 0x55f1c4969350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 --2- 172.21.15.46:0/2492286386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c496ee00 0x55f1c4963d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.564+0000 7f15c2407b80 1 -- 172.21.15.46:0/2492286386 >> 172.21.15.46:0/2492286386 conn(0x55f1c48c7210 msgr2=0x55f1c496cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.565+0000 7f15c2407b80 1 -- 172.21.15.46:0/2492286386 shutdown_connections 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.565+0000 7f15c2407b80 1 -- 172.21.15.46:0/2492286386 wait complete. 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.565+0000 7f15c2407b80 1 -- start start 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.565+0000 7f15c2407b80 1 Processor -- start 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c497a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 0x55f1c496b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c496b960 con 0x55f1c496f720 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c496bad0 con 0x55f1c4969890 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1c496bc90 con 0x55f1c496ee00 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15c2407b80 10 client.0 _ll_register_callbacks cb 0x55f1c485eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51220/0 (socket says 172.21.15.46:51220) 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 -- 172.21.15.46:0/396456209 learned_addr learned my addr 172.21.15.46:0/396456209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a7400700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c497a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a7e00700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 0x55f1c496b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 -- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 msgr2=0x55f1c497a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c497a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 -- 172.21.15.46:0/396456209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 msgr2=0x55f1c496b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 0x55f1c496b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a6a00700 1 -- 172.21.15.46:0/396456209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1594009e80 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.566+0000 7f15a7e00700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 0x55f1c496b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f15a6a00700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f15a819aac0 tx=0x7f159401f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7852e62733f3d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f1591600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594004a30 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f1594004b90 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f159809b240 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1594004d50 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f15c2407b80 10 client.28582 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.567+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f15a0020040 con 0x55f1c496ee00 2024-05-08T23:14:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159809b240 con 0x55f1c496ee00 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15940277b0 con 0x55f1c496ee00 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 10 client.28582 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 10 client.28582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 20 client.28582 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c49da3c0 0x55f1c49dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 4 client.28582 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f1c48b1ec0 con 0x55f1c49da3c0 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.568+0000 7f15a7400700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c49da3c0 0x55f1c49dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15a7400700 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c49da3c0 0x55f1c49dc7e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf657a82e135f3a server_cookie=b439897309fa5a4c in_seq=0 out_seq=0 2024-05-08T23:14:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f1591600700 10 client.28582 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f1591600700 1 -- 172.21.15.46:0/396456209 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f1c48b1ec0 con 0x55f1c49da3c0 2024-05-08T23:14:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f1591600700 10 client.28582 handle_command_reply: tid=0 2024-05-08T23:14:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 client.28582 shutdown 2024-05-08T23:14:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 msgr2=0x55f1c496aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f15a819aac0 tx=0x7f159401f8f0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 shutdown_connections 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f1c49da3c0 0x55f1c49dc7e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1c496f720 0x55f1c496b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f1c4969890 0x55f1c497a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f1591600700 0 client.28582 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 --2- 172.21.15.46:0/396456209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1c496ee00 0x55f1c496aee0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 >> 172.21.15.46:0/396456209 conn(0x55f1c48c7210 msgr2=0x55f1c497e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 shutdown_connections 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 1 -- 172.21.15.46:0/396456209 wait complete. 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.569+0000 7f15c2407b80 20 client.28582 trim_cache size 0 max 16384 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:14:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "de3556bf-032f-4cdf-b1ae-41575c1b6eae": { 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "de3556bf-032f-4cdf-b1ae-41575c1b6eae", 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:14:15.574 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 msgr2=0x7f158c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 0x7f158c0a37d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009ec0 tx=0x7f15a000b040 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 msgr2=0x7f15a81995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1598005950 tx=0x7f159800fa00 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 shutdown_connections 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a819de50 0x7f15a819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f158c0a1310 0x7f158c0a37d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15a81032b0 0x7f15a81995c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 --2- 172.21.15.46:0/4117332211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a81083d0 0x7f15a8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 >> 172.21.15.46:0/4117332211 conn(0x7f15a80feb10 msgr2=0x7f15a80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 shutdown_connections 2024-05-08T23:14:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:15.570+0000 7f15ae800700 1 -- 172.21.15.46:0/4117332211 wait complete. 2024-05-08T23:14:15.589 INFO:tasks.cephfs.filesystem:scrub status for tag:de3556bf-032f-4cdf-b1ae-41575c1b6eae - {'path': '/', 'tag': 'de3556bf-032f-4cdf-b1ae-41575c1b6eae', 'options': 'recursive,force'} 2024-05-08T23:14:15.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:15 smithi086 ceph-mon[78610]: pgmap v323: 129 pgs: 129 active+clean; 533 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-05-08T23:14:15.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:15 smithi053 ceph-mon[77587]: pgmap v323: 129 pgs: 129 active+clean; 533 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-05-08T23:14:15.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 906204 9.48 MB/sec execute 397 sec latency 23.101 ms 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:16.907 INFO:tasks.workunit.client.0.smithi046.stdout: 1 908288 9.48 MB/sec execute 398 sec latency 25.339 ms 2024-05-08T23:14:17.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:17 smithi086 ceph-mon[78610]: pgmap v324: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:17 smithi053 ceph-mon[77587]: pgmap v324: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:17 smithi046 ceph-mon[36898]: pgmap v324: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:17.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 909396 9.48 MB/sec execute 399 sec latency 29.592 ms 2024-05-08T23:14:18.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 911268 9.47 MB/sec execute 400 sec latency 22.707 ms 2024-05-08T23:14:19.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:19 smithi086 ceph-mon[78610]: pgmap v325: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:19 smithi053 ceph-mon[77587]: pgmap v325: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:19 smithi046 ceph-mon[36898]: pgmap v325: 129 pgs: 129 active+clean; 540 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-08T23:14:19.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 912603 9.48 MB/sec execute 401 sec latency 18.518 ms 2024-05-08T23:14:20.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 913874 9.47 MB/sec execute 402 sec latency 19.295 ms 2024-05-08T23:14:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:20 smithi086 ceph-mon[78610]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:14:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:20 smithi053 ceph-mon[77587]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:14:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:20 smithi046 ceph-mon[36898]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:14:21.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 915751 9.47 MB/sec execute 403 sec latency 49.930 ms 2024-05-08T23:14:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:21 smithi086 ceph-mon[78610]: pgmap v327: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:21 smithi053 ceph-mon[77587]: pgmap v327: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:21 smithi046 ceph-mon[36898]: pgmap v327: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:22.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 916385 9.45 MB/sec execute 404 sec latency 217.808 ms 2024-05-08T23:14:23.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 916977 9.46 MB/sec execute 405 sec latency 169.963 ms 2024-05-08T23:14:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:23 smithi086 ceph-mon[78610]: pgmap v328: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:23 smithi053 ceph-mon[77587]: pgmap v328: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:23 smithi046 ceph-mon[36898]: pgmap v328: 129 pgs: 129 active+clean; 547 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-08T23:14:24.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 917961 9.46 MB/sec execute 406 sec latency 60.339 ms 2024-05-08T23:14:25.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 920221 9.48 MB/sec execute 407 sec latency 15.947 ms 2024-05-08T23:14:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:25 smithi086 ceph-mon[78610]: pgmap v329: 129 pgs: 129 active+clean; 554 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-05-08T23:14:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:25 smithi053 ceph-mon[77587]: pgmap v329: 129 pgs: 129 active+clean; 554 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-05-08T23:14:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:25 smithi046 ceph-mon[36898]: pgmap v329: 129 pgs: 129 active+clean; 554 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-05-08T23:14:26.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 921657 9.47 MB/sec execute 408 sec latency 15.655 ms 2024-05-08T23:14:27.908 INFO:tasks.workunit.client.0.smithi046.stdout: 1 923433 9.47 MB/sec execute 409 sec latency 90.254 ms 2024-05-08T23:14:28.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:27 smithi086 ceph-mon[78610]: pgmap v330: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:28.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:27 smithi053 ceph-mon[77587]: pgmap v330: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:27 smithi046 ceph-mon[36898]: pgmap v330: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:28.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 924473 9.46 MB/sec execute 410 sec latency 84.125 ms 2024-05-08T23:14:29.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 926689 9.47 MB/sec execute 411 sec latency 17.059 ms 2024-05-08T23:14:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:29 smithi086 ceph-mon[78610]: pgmap v331: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:29 smithi053 ceph-mon[77587]: pgmap v331: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:29 smithi046 ceph-mon[36898]: pgmap v331: 129 pgs: 129 active+clean; 562 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 212 op/s 2024-05-08T23:14:30.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 928117 9.46 MB/sec execute 412 sec latency 16.666 ms 2024-05-08T23:14:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:14:31.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:14:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:14:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:14:31.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:14:31.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:14:31.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 930313 9.47 MB/sec execute 413 sec latency 17.274 ms 2024-05-08T23:14:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:31 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:14:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:31 smithi086 ceph-mon[78610]: pgmap v332: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 262 op/s 2024-05-08T23:14:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:31 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:14:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:31 smithi053 ceph-mon[77587]: pgmap v332: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 262 op/s 2024-05-08T23:14:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:31 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:14:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:31 smithi046 ceph-mon[36898]: pgmap v332: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 262 op/s 2024-05-08T23:14:32.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 931710 9.46 MB/sec execute 414 sec latency 17.170 ms 2024-05-08T23:14:33.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 933797 9.47 MB/sec execute 415 sec latency 16.302 ms 2024-05-08T23:14:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:33 smithi086 ceph-mon[78610]: pgmap v333: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 229 op/s 2024-05-08T23:14:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:33 smithi053 ceph-mon[77587]: pgmap v333: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 229 op/s 2024-05-08T23:14:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:33 smithi046 ceph-mon[36898]: pgmap v333: 129 pgs: 129 active+clean; 548 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 229 op/s 2024-05-08T23:14:34.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 935239 9.46 MB/sec execute 416 sec latency 16.720 ms 2024-05-08T23:14:35.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 937480 9.47 MB/sec execute 417 sec latency 16.164 ms 2024-05-08T23:14:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:35 smithi086 ceph-mon[78610]: pgmap v334: 129 pgs: 129 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 284 op/s 2024-05-08T23:14:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:35 smithi053 ceph-mon[77587]: pgmap v334: 129 pgs: 129 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 284 op/s 2024-05-08T23:14:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:35 smithi046 ceph-mon[36898]: pgmap v334: 129 pgs: 129 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 284 op/s 2024-05-08T23:14:36.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 938981 9.46 MB/sec execute 418 sec latency 14.373 ms 2024-05-08T23:14:37.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 941007 9.47 MB/sec execute 419 sec latency 26.023 ms 2024-05-08T23:14:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:37 smithi086 ceph-mon[78610]: pgmap v335: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-05-08T23:14:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:37 smithi053 ceph-mon[77587]: pgmap v335: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-05-08T23:14:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:37 smithi046 ceph-mon[36898]: pgmap v335: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-05-08T23:14:38.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 942286 9.46 MB/sec execute 420 sec latency 29.617 ms 2024-05-08T23:14:39.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 944435 9.47 MB/sec execute 421 sec latency 29.135 ms 2024-05-08T23:14:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:39 smithi086 ceph-mon[78610]: pgmap v336: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-05-08T23:14:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:39 smithi053 ceph-mon[77587]: pgmap v336: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-05-08T23:14:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:39 smithi046 ceph-mon[36898]: pgmap v336: 129 pgs: 129 active+clean; 555 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-05-08T23:14:40.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 945698 9.46 MB/sec execute 422 sec latency 15.414 ms 2024-05-08T23:14:41.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 947823 9.47 MB/sec execute 423 sec latency 15.985 ms 2024-05-08T23:14:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:41 smithi086 ceph-mon[78610]: pgmap v337: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-05-08T23:14:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:41 smithi053 ceph-mon[77587]: pgmap v337: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-05-08T23:14:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:41 smithi046 ceph-mon[36898]: pgmap v337: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-05-08T23:14:42.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 949319 9.46 MB/sec execute 424 sec latency 15.595 ms 2024-05-08T23:14:43.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 951426 9.47 MB/sec execute 425 sec latency 17.142 ms 2024-05-08T23:14:44.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:43 smithi086 ceph-mon[78610]: pgmap v338: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:43 smithi053 ceph-mon[77587]: pgmap v338: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:43 smithi046 ceph-mon[36898]: pgmap v338: 129 pgs: 129 active+clean; 557 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:44.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 952789 9.46 MB/sec execute 426 sec latency 23.950 ms 2024-05-08T23:14:45.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:14:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 -- 172.21.15.46:0/3424103296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101a10 msgr2=0x7f2f8c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 --2- 172.21.15.46:0/3424103296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101a10 0x7f2f8c105a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009990 tx=0x7f2f78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 -- 172.21.15.46:0/3424103296 shutdown_connections 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 --2- 172.21.15.46:0/3424103296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 0x7f2f8c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 --2- 172.21.15.46:0/3424103296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c101060 0x7f2f8c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 --2- 172.21.15.46:0/3424103296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101a10 0x7f2f8c105a50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 -- 172.21.15.46:0/3424103296 >> 172.21.15.46:0/3424103296 conn(0x7f2f8c0fc900 msgr2=0x7f2f8c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 -- 172.21.15.46:0/3424103296 shutdown_connections 2024-05-08T23:14:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.843+0000 7f2f94200700 1 -- 172.21.15.46:0/3424103296 wait complete. 2024-05-08T23:14:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.844+0000 7f2f94200700 1 Processor -- start 2024-05-08T23:14:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 -- start start 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 0x7f2f8c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 0x7f2f8c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 0x7f2f8c19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c19e840 con 0x7f2f8c106180 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c19e9b0 con 0x7f2f8c101060 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f94200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c19eb70 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 0x7f2f8c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f92200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 0x7f2f8c19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 0x7f2f8c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47196/0 (socket says 172.21.15.46:47196) 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.845+0000 7f2f91800700 1 -- 172.21.15.46:0/2487457485 learned_addr learned my addr 172.21.15.46:0/2487457485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 0x7f2f8c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 -- 172.21.15.46:0/2487457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 msgr2=0x7f2f8c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 0x7f2f8c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 -- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 msgr2=0x7f2f8c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 0x7f2f8c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 -- 172.21.15.46:0/2487457485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80009520 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f92200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 0x7f2f8c19e240 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400efb0 tx=0x7f2f8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f8a000700 1 -- 172.21.15.46:0/2487457485 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400ca50 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f8a000700 1 -- 172.21.15.46:0/2487457485 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f8400f040 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84009c40 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f8a000700 1 -- 172.21.15.46:0/2487457485 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84004660 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.846+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780095f0 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.850+0000 7f2f8a000700 1 -- 172.21.15.46:0/2487457485 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2f840047c0 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.852+0000 7f2fa7cb0b80 1 Processor -- start 2024-05-08T23:14:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.852+0000 7f2f8a000700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 0x7f2f740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.852+0000 7f2fa7cb0b80 1 -- start start 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f8a000700 1 -- 172.21.15.46:0/2487457485 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2f840ccd60 con 0x7f2f8c19ddc0 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d12cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2c69150 con 0x5582c2d18720 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2c66b10 con 0x5582c2d12890 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2c62400 con 0x5582c2d17e00 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 0x7f2f740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f91800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f91800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58572/0 (socket says 172.21.15.46:58572) 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f91800700 1 -- 172.21.15.46:0/3406103108 learned_addr learned my addr 172.21.15.46:0/3406103108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f92200700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d12cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.853+0000 7f2f90e00700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 -- 172.21.15.46:0/3406103108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 msgr2=0x5582c2d181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 -- 172.21.15.46:0/3406103108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 msgr2=0x5582c2d12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 -- 172.21.15.46:0/3406103108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c2ae33b0 con 0x5582c2d12890 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f91800700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f90e00700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f92200700 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d12cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f840ca080 tx=0x7f2f84099970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602984168a995ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.854+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 0x7f2f740a37d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b2e0 tx=0x7f2f78005ee0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.855+0000 7f2f7ea00700 1 -- 172.21.15.46:0/3406103108 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8409a1d0 con 0x5582c2d12890 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.855+0000 7f2f7ea00700 1 -- 172.21.15.46:0/3406103108 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f84099ad0 con 0x5582c2d12890 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.855+0000 7f2f7ea00700 1 -- 172.21.15.46:0/3406103108 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840961e0 con 0x5582c2d12890 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3406103108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 msgr2=0x5582c2d12cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d12cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f840ca080 tx=0x7f2f84099970 comp rx=0 tx=0).stop 2024-05-08T23:14:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3406103108 shutdown_connections 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d12cd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3406103108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3406103108 >> 172.21.15.46:0/3406103108 conn(0x5582c2c70210 msgr2=0x5582c2d0cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3406103108 shutdown_connections 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3406103108 wait complete. 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.856+0000 7f2fa7cb0b80 1 -- start start 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 Processor -- start 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2bfe560 con 0x5582c2d18720 2024-05-08T23:14:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2bfe200 con 0x5582c2d12890 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2ae56a0 con 0x5582c2d17e00 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.857+0000 7f2fa7cb0b80 10 client.0 _ll_register_callbacks cb 0x5582c2c07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47206/0 (socket says 172.21.15.46:47206) 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f91800700 1 -- 172.21.15.46:0/2181337067 learned_addr learned my addr 172.21.15.46:0/2181337067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f92200700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 -- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 msgr2=0x5582c2d1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 -- 172.21.15.46:0/2181337067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 msgr2=0x5582c2d1c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d1c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 -- 172.21.15.46:0/2181337067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f80009c90 con 0x5582c2d17e00 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f91800700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f90e00700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d1c040 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2f780098e0 tx=0x7f2f78009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191da1756c3027e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.858+0000 7f2f73400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78004b60 con 0x5582c2d17e00 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2f78004cc0 con 0x5582c2d17e00 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582c2bbb270 con 0x5582c2d17e00 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f78004e80 con 0x5582c2d17e00 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 10 client.28594 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2f780095f0 con 0x5582c2d17e00 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582c2bbb270 con 0x5582c2d17e00 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f78033440 con 0x5582c2d17e00 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 10 client.28594 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 10 client.28594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 20 client.28594 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 4 client.28594 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.859+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582c2ce1810 con 0x5582c2d2c260 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.860+0000 7f2f91800700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.860+0000 7f2f91800700 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d2e680 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4801df95c5f5b3e server_cookie=99603e9615db423a in_seq=0 out_seq=0 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.860+0000 7f2f73400700 10 client.28594 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.861+0000 7f2f73400700 1 -- 172.21.15.46:0/2181337067 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582c2ce1810 con 0x5582c2d2c260 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.861+0000 7f2f73400700 10 client.28594 handle_command_reply: tid=0 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.861+0000 7f2fa7cb0b80 1 client.28594 shutdown 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 msgr2=0x5582c2d1c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d1c040 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2f780098e0 tx=0x7f2f78009310 comp rx=0 tx=0).stop 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 shutdown_connections 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d2e680 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d18720 0x5582c2d1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2f73400700 0 client.28594 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/2181337067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d1c040 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 >> 172.21.15.46:0/2181337067 conn(0x5582c2c70210 msgr2=0x5582c2d0c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 shutdown_connections 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/2181337067 wait complete. 2024-05-08T23:14:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.862+0000 7f2fa7cb0b80 20 client.28594 trim_cache size 0 max 16384 2024-05-08T23:14:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 Processor -- start 2024-05-08T23:14:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 -- start start 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d12890 0x5582c2d12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 0x5582c2d0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2bbb270 con 0x5582c2d12890 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2d19b10 con 0x5582c2d18720 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.883+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2d1d170 con 0x5582c2d17e00 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f92200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 0x5582c2d0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58590/0 (socket says 172.21.15.46:58590) 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f92200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 0x5582c2d0cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47218/0 (socket says 172.21.15.46:47218) 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 -- 172.21.15.46:0/4014033731 learned_addr learned my addr 172.21.15.46:0/4014033731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f91800700 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d12890 0x5582c2d12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 -- 172.21.15.46:0/4014033731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 msgr2=0x5582c2d0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 0x5582c2d0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 -- 172.21.15.46:0/4014033731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d12890 msgr2=0x5582c2d12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d12890 0x5582c2d12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 -- 172.21.15.46:0/4014033731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c2a8fb60 con 0x5582c2d17e00 2024-05-08T23:14:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f90e00700 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2f78005f50 tx=0x7f2f7800bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ae328590364dad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.884+0000 7f2f7f400700 1 -- 172.21.15.46:0/4014033731 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78033a60 con 0x5582c2d17e00 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2f7f400700 1 -- 172.21.15.46:0/4014033731 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f78004ca0 con 0x5582c2d17e00 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2f7f400700 1 -- 172.21.15.46:0/4014033731 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800f7e0 con 0x5582c2d17e00 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/4014033731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 msgr2=0x5582c2d282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2f78005f50 tx=0x7f2f7800bf70 comp rx=0 tx=0).stop 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/4014033731 shutdown_connections 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d18720 0x5582c2d0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d12890 0x5582c2d12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/4014033731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d17e00 0x5582c2d282b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/4014033731 >> 172.21.15.46:0/4014033731 conn(0x5582c2c70210 msgr2=0x5582c2d1a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.885+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/4014033731 shutdown_connections 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.886+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/4014033731 wait complete. 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.886+0000 7f2fa7cb0b80 1 -- start start 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.886+0000 7f2fa7cb0b80 1 Processor -- start 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d17e00 0x5582c2d210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 0x5582c2d21830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2d21f00 con 0x5582c2d17e00 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2d81760 con 0x5582c2d12890 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c2d818a0 con 0x5582c2d18720 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2fa7cb0b80 10 client.0 _ll_register_callbacks cb 0x5582c2c07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2f91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47224/0 (socket says 172.21.15.46:47224) 2024-05-08T23:14:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2f91800700 1 -- 172.21.15.46:0/3369710569 learned_addr learned my addr 172.21.15.46:0/3369710569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2f92200700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 0x5582c2d21830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.887+0000 7f2f90e00700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d17e00 0x5582c2d210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 -- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 msgr2=0x5582c2d23630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 -- 172.21.15.46:0/3369710569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d17e00 msgr2=0x5582c2d210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d17e00 0x5582c2d210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 -- 172.21.15.46:0/3369710569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780095f0 con 0x5582c2d18720 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f91800700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f92200700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 0x5582c2d21830 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2f840a2080 tx=0x7f2f8409a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd9e912594549e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f73400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840b2210 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2f840b2370 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f84099ad0 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.888+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f84099d30 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 10 client.28600 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5582c2c28560 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f84099ad0 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f84004bd0 con 0x5582c2d18720 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 10 client.28600 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 10 client.28600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 20 client.28600 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 4 client.28600 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:14:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582c2c5aec0 con 0x5582c2d2c260 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.889+0000 7f2f91800700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.890+0000 7f2f91800700 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d84ae0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57e16305bd0b769 server_cookie=f042f7c13827295f in_seq=0 out_seq=0 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.890+0000 7f2f73400700 10 client.28600 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.890+0000 7f2f73400700 1 -- 172.21.15.46:0/3369710569 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5582c2c5aec0 con 0x5582c2d2c260 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.890+0000 7f2f73400700 10 client.28600 handle_command_reply: tid=0 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.890+0000 7f2fa7cb0b80 1 client.28600 shutdown 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 msgr2=0x5582c2d21830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 0x5582c2d21830 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2f840a2080 tx=0x7f2f8409a280 comp rx=0 tx=0).stop 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 shutdown_connections 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c2d18720 0x5582c2d21830 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582c2d12890 0x5582c2d23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582c2d2c260 0x5582c2d84ae0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 --2- 172.21.15.46:0/3369710569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582c2d17e00 0x5582c2d210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 >> 172.21.15.46:0/3369710569 conn(0x5582c2c70210 msgr2=0x5582c2d1a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2f73400700 0 client.28600 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 shutdown_connections 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 1 -- 172.21.15.46:0/3369710569 wait complete. 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.891+0000 7f2fa7cb0b80 20 client.28600 trim_cache size 0 max 16384 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:14:45.895 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 msgr2=0x7f2f740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 0x7f2f740a37d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b2e0 tx=0x7f2f78005ee0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 msgr2=0x7f2f8c19e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 0x7f2f8c19e240 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400efb0 tx=0x7f2f8400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 shutdown_connections 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f8c19ddc0 0x7f2f8c19e240 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f8c101060 0x7f2f8c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2f740a1310 0x7f2f740a37d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 --2- 172.21.15.46:0/2487457485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c106180 0x7f2f8c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 >> 172.21.15.46:0/2487457485 conn(0x7f2f8c0fc900 msgr2=0x7f2f8c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 shutdown_connections 2024-05-08T23:14:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:45.892+0000 7f2f94200700 1 -- 172.21.15.46:0/2487457485 wait complete. 2024-05-08T23:14:45.909 INFO:tasks.workunit.client.0.smithi046.stdout: 1 954637 9.46 MB/sec execute 427 sec latency 16.101 ms 2024-05-08T23:14:45.911 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:14:45.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:14:45.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:45 smithi046 ceph-mon[36898]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 -- 172.21.15.46:0/1544858903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 msgr2=0x7f29901023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 --2- 172.21.15.46:0/1544858903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901023a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2980009990 tx=0x7f2980009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 -- 172.21.15.46:0/1544858903 shutdown_connections 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 --2- 172.21.15.46:0/1544858903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f299010b3e0 0x7f299010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 --2- 172.21.15.46:0/1544858903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901023a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 --2- 172.21.15.46:0/1544858903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29901028e0 0x7f299010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 -- 172.21.15.46:0/1544858903 >> 172.21.15.46:0/1544858903 conn(0x7f29900fb820 msgr2=0x7f29900fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 -- 172.21.15.46:0/1544858903 shutdown_connections 2024-05-08T23:14:46.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.141+0000 7f2996e00700 1 -- 172.21.15.46:0/1544858903 wait complete. 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.142+0000 7f2996e00700 1 Processor -- start 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.142+0000 7f2996e00700 1 -- start start 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29901028e0 0x7f2990199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299010b3e0 0x7f299019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019e410 con 0x7f29901028e0 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019e580 con 0x7f299010b3e0 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2996e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019e740 con 0x7f2990101fc0 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58608/0 (socket says 172.21.15.46:58608) 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298fe00700 1 -- 172.21.15.46:0/3989902929 learned_addr learned my addr 172.21.15.46:0/3989902929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f2994e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299010b3e0 0x7f299019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298f400700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29901028e0 0x7f2990199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298fe00700 1 -- 172.21.15.46:0/3989902929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299010b3e0 msgr2=0x7f299019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.143+0000 7f298fe00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299010b3e0 0x7f299019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298fe00700 1 -- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29901028e0 msgr2=0x7f2990199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298fe00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29901028e0 0x7f2990199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298fe00700 1 -- 172.21.15.46:0/3989902929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800095f0 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298fe00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2980005740 tx=0x7f2980005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298001d070 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2980022470 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299019e9c0 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980017650 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.144+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29901a6260 con 0x7f2990101fc0 2024-05-08T23:14:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.146+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29901088e0 con 0x7f2990101fc0 2024-05-08T23:14:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.147+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f29800225e0 con 0x7f2990101fc0 2024-05-08T23:14:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.149+0000 7f298cc00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 0x7f29740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.149+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f29800ce3e0 con 0x7f2990101fc0 2024-05-08T23:14:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.149+0000 7f298f400700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 0x7f29740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.150+0000 7f298f400700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 0x7f29740a37d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f299019aa40 tx=0x7f297c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.153+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f298005cd60 con 0x7f2990101fc0 2024-05-08T23:14:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:45 smithi086 ceph-mon[78610]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-08T23:14:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:45 smithi053 ceph-mon[77587]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-05-08T23:14:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.468+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2990068a40 con 0x7f2990101fc0 2024-05-08T23:14:46.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.469+0000 7f298cc00700 1 -- 172.21.15.46:0/3989902929 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f298009fe60 con 0x7f2990101fc0 2024-05-08T23:14:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:14:46.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:14:46.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.471+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 msgr2=0x7f29740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.471+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 0x7f29740a37d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f299019aa40 tx=0x7f297c009260 comp rx=0 tx=0).stop 2024-05-08T23:14:46.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.471+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 msgr2=0x7f29901994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.471+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2980005740 tx=0x7f2980005770 comp rx=0 tx=0).stop 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 shutdown_connections 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299010b3e0 0x7f299019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2990101fc0 0x7f29901994f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f29740a1310 0x7f29740a37d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 --2- 172.21.15.46:0/3989902929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29901028e0 0x7f2990199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 >> 172.21.15.46:0/3989902929 conn(0x7f29900fb820 msgr2=0x7f29900fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 shutdown_connections 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.472+0000 7f2996e00700 1 -- 172.21.15.46:0/3989902929 wait complete. 2024-05-08T23:14:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:14:46.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 -- 172.21.15.46:0/127418723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 msgr2=0x7ffbac10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 --2- 172.21.15.46:0/127418723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac10dd20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b0a0 tx=0x7ffba400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 -- 172.21.15.46:0/127418723 shutdown_connections 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 --2- 172.21.15.46:0/127418723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac10dd20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 --2- 172.21.15.46:0/127418723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac101080 0x7ffbac101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 --2- 172.21.15.46:0/127418723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101a30 0x7ffbac105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 -- 172.21.15.46:0/127418723 >> 172.21.15.46:0/127418723 conn(0x7ffbac0fc8e0 msgr2=0x7ffbac0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 -- 172.21.15.46:0/127418723 shutdown_connections 2024-05-08T23:14:46.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.702+0000 7ffbb3200700 1 -- 172.21.15.46:0/127418723 wait complete. 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 Processor -- start 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 -- start start 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 0x7ffbac0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 0x7ffbac1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1a5c60 con 0x7ffbac101080 2024-05-08T23:14:46.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1a5dd0 con 0x7ffbac073d20 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.704+0000 7ffbb3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1a5f90 con 0x7ffbac1061a0 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 0x7ffbac0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58634/0 (socket says 172.21.15.46:58634) 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 -- 172.21.15.46:0/908869859 learned_addr learned my addr 172.21.15.46:0/908869859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbb1200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 0x7ffbac1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 -- 172.21.15.46:0/908869859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 msgr2=0x7ffbac1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 0x7ffbac1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 -- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 msgr2=0x7ffbac0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 0x7ffbac0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 -- 172.21.15.46:0/908869859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009d00 con 0x7ffbac1061a0 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbb1200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 0x7ffbac1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbabe00700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 0x7ffbac0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.705+0000 7ffbab400700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e970 tx=0x7ffb9800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.706+0000 7ffba8c00700 1 -- 172.21.15.46:0/908869859 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98009790 con 0x7ffbac1061a0 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.706+0000 7ffba8c00700 1 -- 172.21.15.46:0/908869859 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9800eec0 con 0x7ffbac1061a0 2024-05-08T23:14:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.706+0000 7ffba8c00700 1 -- 172.21.15.46:0/908869859 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98005590 con 0x7ffbac1061a0 2024-05-08T23:14:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.706+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac1a6270 con 0x7ffbac1061a0 2024-05-08T23:14:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.706+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1a6730 con 0x7ffbac1061a0 2024-05-08T23:14:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.709+0000 7ffba8c00700 1 -- 172.21.15.46:0/908869859 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffb9800cbd0 con 0x7ffbac1061a0 2024-05-08T23:14:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.710+0000 7ffbc6cc7b80 1 Processor -- start 2024-05-08T23:14:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.710+0000 7ffba8c00700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.710+0000 7ffbc6cc7b80 1 -- start start 2024-05-08T23:14:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.710+0000 7ffba8c00700 1 -- 172.21.15.46:0/908869859 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ffb98014070 con 0x7ffbac1061a0 2024-05-08T23:14:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.710+0000 7ffbb1200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cfe7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cfe7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cf3e150 con 0x55678cfed720 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cf3bb10 con 0x55678cfe7890 2024-05-08T23:14:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cf37400 con 0x55678cfece00 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58638/0 (socket says 172.21.15.46:58638) 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 -- 172.21.15.46:0/3497020303 learned_addr learned my addr 172.21.15.46:0/3497020303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbab400700 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cfe7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbb1200700 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cfe7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 -- 172.21.15.46:0/3497020303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 msgr2=0x55678cfe7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cfe7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 -- 172.21.15.46:0/3497020303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 msgr2=0x55678cfe7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cfe7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbabe00700 1 -- 172.21.15.46:0/3497020303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678cdb83b0 con 0x55678cfece00 2024-05-08T23:14:46.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.711+0000 7ffbb1200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 0x7ffb900a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffba40062a0 tx=0x7ffba40095c0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.712+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0115c0 tx=0x7ffb9c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f437d204a89ecf71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.712+0000 7ffba1600700 1 -- 172.21.15.46:0/3497020303 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c019630 con 0x55678cfece00 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.712+0000 7ffba1600700 1 -- 172.21.15.46:0/3497020303 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9c019c50 con 0x55678cfece00 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.712+0000 7ffba1600700 1 -- 172.21.15.46:0/3497020303 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ece0 con 0x55678cfece00 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3497020303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 msgr2=0x55678cfed1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0115c0 tx=0x7ffb9c0118d0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3497020303 shutdown_connections 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cfe7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfece00 0x55678cfed1e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3497020303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cfe7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3497020303 >> 172.21.15.46:0/3497020303 conn(0x55678cf45210 msgr2=0x55678cfe1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3497020303 shutdown_connections 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3497020303 wait complete. 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.713+0000 7ffbc6cc7b80 1 -- start start 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 Processor -- start 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfece00 0x55678cff10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cff1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678ced3560 con 0x55678cfed720 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678ced3200 con 0x55678cfece00 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cdba6a0 con 0x55678cfe7890 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbc6cc7b80 10 client.0 _ll_register_callbacks cb 0x55678cedcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.714+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58640/0 (socket says 172.21.15.46:58640) 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 -- 172.21.15.46:0/3754803086 learned_addr learned my addr 172.21.15.46:0/3754803086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbab400700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfece00 0x55678cff10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbb1200700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cff1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 -- 172.21.15.46:0/3754803086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfece00 msgr2=0x55678cff10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfece00 0x55678cff10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 -- 172.21.15.46:0/3754803086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 msgr2=0x55678cff1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cff1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 -- 172.21.15.46:0/3754803086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c009dd0 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbb1200700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cff1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffbabe00700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009da0 tx=0x7ffb9c005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2de5cc95dc62cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffb96000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.715+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c019630 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55678ce90270 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ffb9c019790 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9c01ae50 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 10 client.28618 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55678cd64b60 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55678ce90270 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb9c0048d0 con 0x55678cfe7890 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 10 client.28618 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 10 client.28618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 20 client.28618 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d0010e0 0x55678d003500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 4 client.28618 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.716+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55678cfb6810 con 0x55678d0010e0 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.717+0000 7ffbab400700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d0010e0 0x55678d003500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.717+0000 7ffbab400700 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d0010e0 0x55678d003500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c9ffbd7a420c0a4 server_cookie=5c5dbb05a9cd8714 in_seq=0 out_seq=0 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.717+0000 7ffb96000700 10 client.28618 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffb96000700 1 -- 172.21.15.46:0/3754803086 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55678cfb6810 con 0x55678d0010e0 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffb96000700 10 client.28618 handle_command_reply: tid=0 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 client.28618 shutdown 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 msgr2=0x55678cff0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009da0 tx=0x7ffb9c005ce0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 shutdown_connections 2024-05-08T23:14:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d0010e0 0x55678d003500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfed720 0x55678cff1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffb96000700 0 client.28618 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cff0b80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/3754803086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfece00 0x55678cff10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 >> 172.21.15.46:0/3754803086 conn(0x55678cf45210 msgr2=0x55678cfe15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 shutdown_connections 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.718+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/3754803086 wait complete. 2024-05-08T23:14:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.719+0000 7ffbc6cc7b80 20 client.28618 trim_cache size 0 max 16384 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 Processor -- start 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 -- start start 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cffcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfed720 0x55678cfe1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cfdedc0 con 0x55678cfece00 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678d008710 con 0x55678cfed720 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cfeeb10 con 0x55678cfe7890 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58656/0 (socket says 172.21.15.46:58656) 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbabe00700 1 -- 172.21.15.46:0/2666863797 learned_addr learned my addr 172.21.15.46:0/2666863797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbb1200700 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfed720 0x55678cfe1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.733+0000 7ffbab400700 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cffcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 -- 172.21.15.46:0/2666863797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfed720 msgr2=0x55678cfe1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfed720 0x55678cfe1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 -- 172.21.15.46:0/2666863797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 msgr2=0x55678cffcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cffcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 -- 172.21.15.46:0/2666863797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678cefd560 con 0x55678cfe7890 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffbabe00700 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c011510 tx=0x7ffb9c004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafa298faf97d2ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffba2000700 1 -- 172.21.15.46:0/2666863797 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c005d50 con 0x55678cfe7890 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffba2000700 1 -- 172.21.15.46:0/2666863797 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9c0198f0 con 0x55678cfe7890 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.734+0000 7ffba2000700 1 -- 172.21.15.46:0/2666863797 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c003680 con 0x55678cfe7890 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/2666863797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 msgr2=0x55678cfe3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c011510 tx=0x7ffb9c004db0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/2666863797 shutdown_connections 2024-05-08T23:14:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfed720 0x55678cfe1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfe7890 0x55678cfe3a30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/2666863797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cffcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/2666863797 >> 172.21.15.46:0/2666863797 conn(0x55678cf45210 msgr2=0x55678cfde210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/2666863797 shutdown_connections 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/2666863797 wait complete. 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 -- start start 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.735+0000 7ffbc6cc7b80 1 Processor -- start 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cff6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cff6bd0 con 0x55678cfece00 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cff6d40 con 0x55678cfe7890 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cff6f00 con 0x55678cfed720 2024-05-08T23:14:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbc6cc7b80 10 client.0 _ll_register_callbacks cb 0x55678cedcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbab400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cff6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbb1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbab400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cff6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52838/0 (socket says 172.21.15.46:52838) 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbb1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58664/0 (socket says 172.21.15.46:58664) 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbab400700 1 -- 172.21.15.46:0/533380938 learned_addr learned my addr 172.21.15.46:0/533380938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.736+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47294/0 (socket says 172.21.15.46:47294) 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 -- 172.21.15.46:0/533380938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 msgr2=0x55678cff8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 -- 172.21.15.46:0/533380938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 msgr2=0x55678cff6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cff6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 -- 172.21.15.46:0/533380938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55678cd64b60 con 0x55678cfed720 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbabe00700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbb1200700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009fd0 tx=0x7ffba400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aedad932889742 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffb96000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba401d650 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ffba401d7b0 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.737+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb9c019c50 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba401d970 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 10 client.28624 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55678cdbccd0 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9c019c50 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffba4031060 con 0x55678cfed720 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 10 client.28624 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 10 client.28624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 20 client.28624 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d057ba0 0x55678d059fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 4 client.28624 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55678cf2fec0 con 0x55678d057ba0 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.738+0000 7ffbabe00700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d057ba0 0x55678d059fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbabe00700 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d057ba0 0x55678d059fc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e4d412a2aafb1a6 server_cookie=7d9fc0c68fd557a1 in_seq=0 out_seq=0 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffb96000700 10 client.28624 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffb96000700 1 -- 172.21.15.46:0/533380938 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55678cf2fec0 con 0x55678d057ba0 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffb96000700 10 client.28624 handle_command_reply: tid=0 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 client.28624 shutdown 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 msgr2=0x55678cff6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009fd0 tx=0x7ffba400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 shutdown_connections 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55678d057ba0 0x55678d059fc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55678cfed720 0x55678cff6690 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55678cfe7890 0x55678cff8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffb96000700 0 client.28624 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 --2- 172.21.15.46:0/533380938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55678cfece00 0x55678cff6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.739+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 >> 172.21.15.46:0/533380938 conn(0x55678cf45210 msgr2=0x55678cfef130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 shutdown_connections 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbc6cc7b80 1 -- 172.21.15.46:0/533380938 wait complete. 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbc6cc7b80 20 client.28624 trim_cache size 0 max 16384 2024-05-08T23:14:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 msgr2=0x7ffb900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 0x7ffb900a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffba40062a0 tx=0x7ffba40095c0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 msgr2=0x7ffbac0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e970 tx=0x7ffb9800ec80 comp rx=0 tx=0).stop 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 shutdown_connections 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbac073d20 0x7ffbac1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbac101080 0x7ffbac0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 --2- 172.21.15.46:0/908869859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbac1061a0 0x7ffbac0737e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.740+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 >> 172.21.15.46:0/908869859 conn(0x7ffbac0fc8e0 msgr2=0x7ffbac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.741+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 shutdown_connections 2024-05-08T23:14:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.741+0000 7ffbb3200700 1 -- 172.21.15.46:0/908869859 wait complete. 2024-05-08T23:14:46.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:14:46.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 956141 9.46 MB/sec execute 428 sec latency 16.646 ms 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 -- 172.21.15.46:0/938848762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0101080 msgr2=0x7ff7c0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/938848762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0101080 0x7ff7c0101460 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009990 tx=0x7ff7b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 -- 172.21.15.46:0/938848762 shutdown_connections 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/938848762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 0x7ff7c010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/938848762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0101080 0x7ff7c0101460 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/938848762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101a30 0x7ff7c0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.966+0000 7ff7c6a00700 1 -- 172.21.15.46:0/938848762 >> 172.21.15.46:0/938848762 conn(0x7ff7c00fc8e0 msgr2=0x7ff7c00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.967+0000 7ff7c6a00700 1 -- 172.21.15.46:0/938848762 shutdown_connections 2024-05-08T23:14:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.967+0000 7ff7c6a00700 1 -- 172.21.15.46:0/938848762 wait complete. 2024-05-08T23:14:46.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 Processor -- start 2024-05-08T23:14:46.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 -- start start 2024-05-08T23:14:46.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 0x7ff7c00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a5b10 con 0x7ff7c0101080 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a5c50 con 0x7ff7c01061a0 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c6a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a5e10 con 0x7ff7c0073d20 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 0x7ff7c00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58688/0 (socket says 172.21.15.46:58688) 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7c4a00700 1 -- 172.21.15.46:0/3323902575 learned_addr learned my addr 172.21.15.46:0/3323902575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.968+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52862/0 (socket says 172.21.15.46:52862) 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 -- 172.21.15.46:0/3323902575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 msgr2=0x7ff7c00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 0x7ff7c00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 -- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 msgr2=0x7ff7c00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 -- 172.21.15.46:0/3323902575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b00095f0 con 0x7ff7c0073d20 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800ef30 tx=0x7ff7b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7bcc00700 1 -- 172.21.15.46:0/3323902575 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8018070 con 0x7ff7c0073d20 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7bcc00700 1 -- 172.21.15.46:0/3323902575 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b800f040 con 0x7ff7c0073d20 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c01a6090 con 0x7ff7c0073d20 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.969+0000 7ff7bcc00700 1 -- 172.21.15.46:0/3323902575 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b800cd00 con 0x7ff7c0073d20 2024-05-08T23:14:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.970+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01a65f0 con 0x7ff7c0073d20 2024-05-08T23:14:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.972+0000 7ff7bcc00700 1 -- 172.21.15.46:0/3323902575 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff7b8003a00 con 0x7ff7c0073d20 2024-05-08T23:14:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7bcc00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 0x7ff7a40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 Processor -- start 2024-05-08T23:14:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7bcc00700 1 -- 172.21.15.46:0/3323902575 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7b80ccb90 con 0x7ff7c0073d20 2024-05-08T23:14:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 -- start start 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 0x7ff7a40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fcdb150 con 0x55745fd8a720 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fcd8b10 con 0x55745fd84890 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.974+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fcd4400 con 0x55745fd89e00 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52870/0 (socket says 172.21.15.46:52870) 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 -- 172.21.15.46:0/1233475570 learned_addr learned my addr 172.21.15.46:0/1233475570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 0x7ff7a40a37d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006010 tx=0x7ff7b000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 -- 172.21.15.46:0/1233475570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 msgr2=0x55745fd8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 -- 172.21.15.46:0/1233475570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 msgr2=0x55745fd84cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd84cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7bf400700 1 -- 172.21.15.46:0/1233475570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745fb553b0 con 0x55745fd8a720 2024-05-08T23:14:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.975+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd84cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.976+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009cf0 tx=0x7ff7ac011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df38d2f11a6203c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.976+0000 7ff7b5200700 1 -- 172.21.15.46:0/1233475570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac011d80 con 0x55745fd8a720 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.976+0000 7ff7b5200700 1 -- 172.21.15.46:0/1233475570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ac004510 con 0x55745fd8a720 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.976+0000 7ff7b5200700 1 -- 172.21.15.46:0/1233475570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac005280 con 0x55745fd8a720 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1233475570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 msgr2=0x55745fd84350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009cf0 tx=0x7ff7ac011440 comp rx=0 tx=0).stop 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1233475570 shutdown_connections 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd84cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1233475570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd84350 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1233475570 >> 172.21.15.46:0/1233475570 conn(0x55745fce2210 msgr2=0x55745fd7edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.977+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1233475570 shutdown_connections 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.978+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1233475570 wait complete. 2024-05-08T23:14:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.978+0000 7ff7da5fcb80 1 -- start start 2024-05-08T23:14:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 Processor -- start 2024-05-08T23:14:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd84890 0x55745fd8dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fc70560 con 0x55745fd8a720 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fc70200 con 0x55745fd89e00 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fb576a0 con 0x55745fd84890 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.979+0000 7ff7da5fcb80 10 client.0 _ll_register_callbacks cb 0x55745fc79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47320/0 (socket says 172.21.15.46:47320) 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 -- 172.21.15.46:0/1479840353 learned_addr learned my addr 172.21.15.46:0/1479840353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd84890 0x55745fd8dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 -- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd84890 msgr2=0x55745fd8dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd84890 0x55745fd8dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 -- 172.21.15.46:0/1479840353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 msgr2=0x55745fd8e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.980+0000 7ff7bf400700 1 -- 172.21.15.46:0/1479840353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01a65f0 con 0x55745fd89e00 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.981+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.981+0000 7ff7bf400700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0046e0 tx=0x7ff7ac005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b21bec97c6bcb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.981+0000 7ff7a9600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.981+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac026070 con 0x55745fd89e00 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.982+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55745fc2d270 con 0x55745fd89e00 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.982+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7ac004260 con 0x55745fd89e00 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.982+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ac003680 con 0x55745fd89e00 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.982+0000 7ff7da5fcb80 10 client.28544 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.982+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff7ac009920 con 0x55745fd89e00 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55745fc2d270 con 0x55745fd89e00 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7ac029020 con 0x55745fd89e00 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 10 client.28544 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 10 client.28544 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 20 client.28544 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:46.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fd9e080 0x55745fda04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 4 client.28544 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.983+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55745fd53810 con 0x55745fd9e080 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.984+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fd9e080 0x55745fda04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.984+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fd9e080 0x55745fda04a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a49978619cf966b6 server_cookie=6d1eceeccecccfaf in_seq=0 out_seq=0 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.984+0000 7ff7a9600700 10 client.28544 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7a9600700 1 -- 172.21.15.46:0/1479840353 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55745fd53810 con 0x55745fd9e080 2024-05-08T23:14:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7a9600700 10 client.28544 handle_command_reply: tid=0 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 client.28544 shutdown 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 msgr2=0x55745fd8dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0046e0 tx=0x7ff7ac005d70 comp rx=0 tx=0).stop 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 shutdown_connections 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fd9e080 0x55745fda04a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd84890 0x55745fd8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7a9600700 0 client.28544 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/1479840353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd89e00 0x55745fd8dff0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 >> 172.21.15.46:0/1479840353 conn(0x55745fce2210 msgr2=0x55745fd7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 shutdown_connections 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.985+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/1479840353 wait complete. 2024-05-08T23:14:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:46.986+0000 7ff7da5fcb80 20 client.28544 trim_cache size 0 max 16384 2024-05-08T23:14:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 Processor -- start 2024-05-08T23:14:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 -- start start 2024-05-08T23:14:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fd77040 con 0x55745fd8a720 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fd7bdc0 con 0x55745fd84890 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.000+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fda56b0 con 0x55745fd89e00 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47330/0 (socket says 172.21.15.46:47330) 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7c4a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bfe00700 1 -- 172.21.15.46:0/412627771 learned_addr learned my addr 172.21.15.46:0/412627771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 -- 172.21.15.46:0/412627771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 msgr2=0x55745fd99c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 -- 172.21.15.46:0/412627771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 msgr2=0x55745fd8e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 -- 172.21.15.46:0/412627771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745fb01b60 con 0x55745fd89e00 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.001+0000 7ff7bf400700 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd7ed20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009c10 tx=0x7ff7ac011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfa7c6dd014608c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.002+0000 7ff7b5c00700 1 -- 172.21.15.46:0/412627771 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac026040 con 0x55745fd89e00 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.002+0000 7ff7b5c00700 1 -- 172.21.15.46:0/412627771 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ac0050f0 con 0x55745fd89e00 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.002+0000 7ff7b5c00700 1 -- 172.21.15.46:0/412627771 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003af0 con 0x55745fd89e00 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/412627771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 msgr2=0x55745fd7ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd7ed20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009c10 tx=0x7ff7ac011440 comp rx=0 tx=0).stop 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/412627771 shutdown_connections 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd8a720 0x55745fd8e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/412627771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd89e00 0x55745fd7ed20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/412627771 >> 172.21.15.46:0/412627771 conn(0x55745fce2210 msgr2=0x55745fd7b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/412627771 shutdown_connections 2024-05-08T23:14:47.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/412627771 wait complete. 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 -- start start 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.003+0000 7ff7da5fcb80 1 Processor -- start 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd89e00 0x55745fd93150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 0x55745fd936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fd93e30 con 0x55745fd89e00 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fd93fa0 con 0x55745fd84890 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745fdf3580 con 0x55745fd8a720 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7da5fcb80 10 client.0 _ll_register_callbacks cb 0x55745fc79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7bf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd89e00 0x55745fd93150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47340/0 (socket says 172.21.15.46:47340) 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7bfe00700 1 -- 172.21.15.46:0/2737180248 learned_addr learned my addr 172.21.15.46:0/2737180248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.004+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 0x55745fd936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 -- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 msgr2=0x55745fd936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 0x55745fd936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 -- 172.21.15.46:0/2737180248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd89e00 msgr2=0x55745fd93150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd89e00 0x55745fd93150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 -- 172.21.15.46:0/2737180248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c010e590 con 0x55745fd84890 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7c4a00700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 0x55745fd936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7bfe00700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00057b0 tx=0x7ff7b001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26d7a0fd91ee0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7a9600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:47.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0004500 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7ac004510 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.005+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7b0004660 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b0004820 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 10 client.28550 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff7c01a6090 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ac004510 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7b000f8d0 con 0x55745fd84890 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 10 client.28550 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 10 client.28550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 20 client.28550 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fdf4c90 0x55745fdf70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 4 client.28550 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.006+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55745fcccec0 con 0x55745fdf4c90 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7bf400700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fdf4c90 0x55745fdf70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7bf400700 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fdf4c90 0x55745fdf70b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6cb482334acf737d server_cookie=a38c8a45b1643d8a in_seq=0 out_seq=0 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7a9600700 10 client.28550 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7a9600700 1 -- 172.21.15.46:0/2737180248 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55745fcccec0 con 0x55745fdf4c90 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7a9600700 10 client.28550 handle_command_reply: tid=0 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7da5fcb80 1 client.28550 shutdown 2024-05-08T23:14:47.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 msgr2=0x55745fd955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00057b0 tx=0x7ff7b001a050 comp rx=0 tx=0).stop 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 shutdown_connections 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.007+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55745fdf4c90 0x55745fdf70b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745fd8a720 0x55745fd936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55745fd84890 0x55745fd955a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7a9600700 0 client.28550 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 --2- 172.21.15.46:0/2737180248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55745fd89e00 0x55745fd93150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 >> 172.21.15.46:0/2737180248 conn(0x55745fce2210 msgr2=0x55745fd8e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 shutdown_connections 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 1 -- 172.21.15.46:0/2737180248 wait complete. 2024-05-08T23:14:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7da5fcb80 20 client.28550 trim_cache size 0 max 16384 2024-05-08T23:14:47.016 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 msgr2=0x7ff7a40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 0x7ff7a40a37d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006010 tx=0x7ff7b000b260 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 msgr2=0x7ff7c01a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.008+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800ef30 tx=0x7ff7b800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 shutdown_connections 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c0073d20 0x7ff7c01a5540 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff7a40a1310 0x7ff7a40a37d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7c0101080 0x7ff7c00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 --2- 172.21.15.46:0/3323902575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7c01061a0 0x7ff7c00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 >> 172.21.15.46:0/3323902575 conn(0x7ff7c00fc8e0 msgr2=0x7ff7c00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 shutdown_connections 2024-05-08T23:14:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.009+0000 7ff7c6a00700 1 -- 172.21.15.46:0/3323902575 wait complete. 2024-05-08T23:14:47.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:46 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3989902929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:47.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:14:47.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:46 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3989902929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 -- 172.21.15.46:0/3313423997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 msgr2=0x7f5468103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 --2- 172.21.15.46:0/3313423997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468103650 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5458009990 tx=0x7f5458009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 -- 172.21.15.46:0/3313423997 shutdown_connections 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 --2- 172.21.15.46:0/3313423997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 0x7f546810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 --2- 172.21.15.46:0/3313423997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468103650 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 --2- 172.21.15.46:0/3313423997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468103c20 0x7f5468107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 -- 172.21.15.46:0/3313423997 >> 172.21.15.46:0/3313423997 conn(0x7f54680feab0 msgr2=0x7f5468100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 -- 172.21.15.46:0/3313423997 shutdown_connections 2024-05-08T23:14:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.245+0000 7f546de00700 1 -- 172.21.15.46:0/3313423997 wait complete. 2024-05-08T23:14:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.246+0000 7f546de00700 1 Processor -- start 2024-05-08T23:14:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.246+0000 7f546de00700 1 -- start start 2024-05-08T23:14:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 0x7f5468199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546819de00 0x7f546819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819e880 con 0x7f546819de00 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819e9f0 con 0x7f5468108390 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f546de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819ebb0 con 0x7f5468103270 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58766/0 (socket says 172.21.15.46:58766) 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 -- 172.21.15.46:0/3864436610 learned_addr learned my addr 172.21.15.46:0/3864436610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466000700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 0x7f5468199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5467400700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546819de00 0x7f546819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 -- 172.21.15.46:0/3864436610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 msgr2=0x7f5468199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 0x7f5468199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 -- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546819de00 msgr2=0x7f546819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546819de00 0x7f546819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.247+0000 7f5466a00700 1 -- 172.21.15.46:0/3864436610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454009520 con 0x7f5468103270 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f5466a00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5458005910 tx=0x7f5458004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f546ca00700 1 -- 172.21.15.46:0/3864436610 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545801d070 con 0x7f5468103270 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f546ca00700 1 -- 172.21.15.46:0/3864436610 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5458022470 con 0x7f5468103270 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54580095f0 con 0x7f5468103270 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f546ca00700 1 -- 172.21.15.46:0/3864436610 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458017650 con 0x7f5468103270 2024-05-08T23:14:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.248+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681a6340 con 0x7f5468103270 2024-05-08T23:14:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.251+0000 7f546ca00700 1 -- 172.21.15.46:0/3864436610 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f54580225e0 con 0x7f5468103270 2024-05-08T23:14:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.252+0000 7f548194fb80 1 Processor -- start 2024-05-08T23:14:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.252+0000 7f546ca00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 0x7f544c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.252+0000 7f546ca00700 1 -- 172.21.15.46:0/3864436610 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f54580ce370 con 0x7f5468103270 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.252+0000 7f548194fb80 1 -- start start 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5467400700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 0x7f544c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 0x55bfee1e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee131150 con 0x55bfee1e8f80 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee12eb10 con 0x55bfee1e30f0 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f548194fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee12a400 con 0x55bfee1e8660 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5466a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5466000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 0x55bfee1e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5466a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58774/0 (socket says 172.21.15.46:58774) 2024-05-08T23:14:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5466a00700 1 -- 172.21.15.46:0/2935508441 learned_addr learned my addr 172.21.15.46:0/2935508441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5467400700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5467400700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 0x7f544c0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f546000f590 tx=0x7f546000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.253+0000 7f5466a00700 1 -- 172.21.15.46:0/2935508441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 msgr2=0x55bfee1e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466a00700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466a00700 1 -- 172.21.15.46:0/2935508441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 msgr2=0x55bfee1e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466a00700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 0x55bfee1e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466a00700 1 -- 172.21.15.46:0/2935508441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfedfab3b0 con 0x55bfee1e8660 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466000700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 0x55bfee1e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5467400700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5466a00700 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f54580a2f60 tx=0x7f5458022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770e2dd62b8fc5df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5453e00700 1 -- 172.21.15.46:0/2935508441 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545809b7a0 con 0x55bfee1e8660 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5453e00700 1 -- 172.21.15.46:0/2935508441 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f545809b0a0 con 0x55bfee1e8660 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.254+0000 7f5453e00700 1 -- 172.21.15.46:0/2935508441 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580229b0 con 0x55bfee1e8660 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- 172.21.15.46:0/2935508441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 msgr2=0x55bfee1e8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f54580a2f60 tx=0x7f5458022c90 comp rx=0 tx=0).stop 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- 172.21.15.46:0/2935508441 shutdown_connections 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8660 0x55bfee1e8a40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 --2- 172.21.15.46:0/2935508441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8f80 0x55bfee1e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- 172.21.15.46:0/2935508441 >> 172.21.15.46:0/2935508441 conn(0x55bfee138210 msgr2=0x55bfee1d4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- 172.21.15.46:0/2935508441 shutdown_connections 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- 172.21.15.46:0/2935508441 wait complete. 2024-05-08T23:14:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.255+0000 7f548194fb80 1 -- start start 2024-05-08T23:14:47.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 Processor -- start 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee0c6560 con 0x55bfee1e30f0 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee0c6200 con 0x55bfee1e8660 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfedfad6a0 con 0x55bfee1e8f80 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f548194fb80 10 client.0 _ll_register_callbacks cb 0x55bfee0cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5466a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5466000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5466a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52936/0 (socket says 172.21.15.46:52936) 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58790/0 (socket says 172.21.15.46:58790) 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.256+0000 7f5466a00700 1 -- 172.21.15.46:0/2793405550 learned_addr learned my addr 172.21.15.46:0/2793405550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 -- 172.21.15.46:0/2793405550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 msgr2=0x55bfee1ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 -- 172.21.15.46:0/2793405550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 msgr2=0x55bfee1eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 -- 172.21.15.46:0/2793405550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5454009a70 con 0x55bfee1e8f80 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5466a00700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5467400700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f546000f400 tx=0x7f5460004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ddf313cd8d4508 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5450c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000e040 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.257+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfee083270 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5460009020 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f546000fe50 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f548194fb80 10 client.28654 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f54681a6340 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfee083270 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.258+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5460008a00 con 0x55bfee1e8f80 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 10 client.28654 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 10 client.28654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 20 client.28654 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee1fc640 0x55bfee1fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 4 client.28654 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfee1a9810 con 0x55bfee1fc640 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f5466a00700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee1fc640 0x55bfee1fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f5466a00700 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee1fc640 0x55bfee1fea60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f66f28a01df7e39 server_cookie=502dbc8794b32e82 in_seq=0 out_seq=0 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.259+0000 7f5450c00700 10 client.28654 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f5450c00700 1 -- 172.21.15.46:0/2793405550 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfee1a9810 con 0x55bfee1fc640 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f5450c00700 10 client.28654 handle_command_reply: tid=0 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 client.28654 shutdown 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 msgr2=0x55bfee1ec8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f546000f400 tx=0x7f5460004ad0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 shutdown_connections 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1ec8d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee1fc640 0x55bfee1fea60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 --2- 172.21.15.46:0/2793405550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 >> 172.21.15.46:0/2793405550 conn(0x55bfee138210 msgr2=0x55bfee1d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f5450c00700 0 client.28654 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 shutdown_connections 2024-05-08T23:14:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.260+0000 7f548194fb80 1 -- 172.21.15.46:0/2793405550 wait complete. 2024-05-08T23:14:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.261+0000 7f548194fb80 20 client.28654 trim_cache size 0 max 16384 2024-05-08T23:14:47.266 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:46 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3989902929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:14:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.275+0000 7f548194fb80 1 Processor -- start 2024-05-08T23:14:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.275+0000 7f548194fb80 1 -- start start 2024-05-08T23:14:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8660 0x55bfee1d4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee1d6800 con 0x55bfee1e8660 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee203c70 con 0x55bfee1e30f0 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f548194fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee1cd040 con 0x55bfee1e8f80 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5466000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8660 0x55bfee1d4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5466a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58798/0 (socket says 172.21.15.46:58798) 2024-05-08T23:14:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 -- 172.21.15.46:0/2933636153 learned_addr learned my addr 172.21.15.46:0/2933636153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 -- 172.21.15.46:0/2933636153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 msgr2=0x55bfee1ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 -- 172.21.15.46:0/2933636153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8660 msgr2=0x55bfee1d4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.276+0000 7f5467400700 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8660 0x55bfee1d4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f5467400700 1 -- 172.21.15.46:0/2933636153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfedf57b60 con 0x55bfee1e8f80 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f5466a00700 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f5467400700 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5460009c10 tx=0x7f5460021500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799d2671921af64c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f545d200700 1 -- 172.21.15.46:0/2933636153 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000e070 con 0x55bfee1e8f80 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f545d200700 1 -- 172.21.15.46:0/2933636153 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546000f040 con 0x55bfee1e8f80 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.277+0000 7f545d200700 1 -- 172.21.15.46:0/2933636153 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600088d0 con 0x55bfee1e8f80 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- 172.21.15.46:0/2933636153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 msgr2=0x55bfee1eea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5460009c10 tx=0x7f5460021500 comp rx=0 tx=0).stop 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- 172.21.15.46:0/2933636153 shutdown_connections 2024-05-08T23:14:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1eea90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e30f0 0x55bfee1ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 --2- 172.21.15.46:0/2933636153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e8660 0x55bfee1d4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- 172.21.15.46:0/2933636153 >> 172.21.15.46:0/2933636153 conn(0x55bfee138210 msgr2=0x55bfee1f8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- 172.21.15.46:0/2933636153 shutdown_connections 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- 172.21.15.46:0/2933636153 wait complete. 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 -- start start 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.278+0000 7f548194fb80 1 Processor -- start 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee1e5d10 con 0x55bfee1e30f0 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee1e4060 con 0x55bfee1e8660 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfee1e4220 con 0x55bfee1e8f80 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f548194fb80 10 client.0 _ll_register_callbacks cb 0x55bfee0cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f5466a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f5467400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58810/0 (socket says 172.21.15.46:58810) 2024-05-08T23:14:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f5467400700 1 -- 172.21.15.46:0/2794575322 learned_addr learned my addr 172.21.15.46:0/2794575322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.279+0000 7f5466000700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 -- 172.21.15.46:0/2794575322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 msgr2=0x55bfee1e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 -- 172.21.15.46:0/2794575322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 msgr2=0x55bfee1f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 -- 172.21.15.46:0/2794575322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfedfab3b0 con 0x55bfee1e8f80 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5466a00700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5467400700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f546000f400 tx=0x7f5460009160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddb644705c05ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5450c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000e040 con 0x55bfee1e8f80 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f545809b0a0 con 0x55bfee1e8f80 2024-05-08T23:14:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.280+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5460004ca0 con 0x55bfee1e8f80 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5460004e60 con 0x55bfee1e8f80 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 10 client.28660 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f54681a6340 con 0x55bfee1e8f80 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f545809b0a0 con 0x55bfee1e8f80 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f546003c030 con 0x55bfee1e8f80 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 10 client.28660 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 10 client.28660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 20 client.28660 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee252f90 0x55bfee2553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 4 client.28660 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfee122ec0 con 0x55bfee252f90 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.281+0000 7f5466a00700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee252f90 0x55bfee2553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.282+0000 7f5466a00700 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee252f90 0x55bfee2553b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d06524a1fefee314 server_cookie=44510d9bb965ce1f in_seq=0 out_seq=0 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.282+0000 7f5450c00700 10 client.28660 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f5450c00700 1 -- 172.21.15.46:0/2794575322 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfee122ec0 con 0x55bfee252f90 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f5450c00700 10 client.28660 handle_command_reply: tid=0 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 client.28660 shutdown 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 msgr2=0x55bfee1e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f546000f400 tx=0x7f5460009160 comp rx=0 tx=0).stop 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 shutdown_connections 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfee1e8f80 0x55bfee1e5620 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55bfee252f90 0x55bfee2553b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bfee1e30f0 0x55bfee1f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 --2- 172.21.15.46:0/2794575322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfee1e8660 0x55bfee1e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f5450c00700 0 client.28660 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 >> 172.21.15.46:0/2794575322 conn(0x55bfee138210 msgr2=0x55bfee1e7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 shutdown_connections 2024-05-08T23:14:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 1 -- 172.21.15.46:0/2794575322 wait complete. 2024-05-08T23:14:47.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.284+0000 7f548194fb80 20 client.28660 trim_cache size 0 max 16384 2024-05-08T23:14:47.288 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:47.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 msgr2=0x7f544c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 0x7f544c0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f546000f590 tx=0x7f546000b040 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 msgr2=0x7f5468199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5458005910 tx=0x7f5458004730 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 shutdown_connections 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f544c0a1290 0x7f544c0a3750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546819de00 0x7f546819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5468103270 0x7f5468199520 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 --2- 172.21.15.46:0/3864436610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5468108390 0x7f5468199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 >> 172.21.15.46:0/3864436610 conn(0x7f54680feab0 msgr2=0x7f54680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 shutdown_connections 2024-05-08T23:14:47.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.285+0000 7f546de00700 1 -- 172.21.15.46:0/3864436610 wait complete. 2024-05-08T23:14:47.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:14:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 -- 172.21.15.46:0/753172187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff928103c50 msgr2=0x7ff928107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 --2- 172.21.15.46:0/753172187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff928103c50 0x7ff928107c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff914009990 tx=0x7ff914009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 -- 172.21.15.46:0/753172187 shutdown_connections 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 --2- 172.21.15.46:0/753172187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281083c0 0x7ff92810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 --2- 172.21.15.46:0/753172187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281032a0 0x7ff928103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 --2- 172.21.15.46:0/753172187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff928103c50 0x7ff928107c90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.521+0000 7ff930600700 1 -- 172.21.15.46:0/753172187 >> 172.21.15.46:0/753172187 conn(0x7ff9280feac0 msgr2=0x7ff928100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.522+0000 7ff930600700 1 -- 172.21.15.46:0/753172187 shutdown_connections 2024-05-08T23:14:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.522+0000 7ff930600700 1 -- 172.21.15.46:0/753172187 wait complete. 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 Processor -- start 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 -- start start 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281032a0 0x7ff92819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281996d0 0x7ff928199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819a150 con 0x7ff9281996d0 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819a290 con 0x7ff9281032a0 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff930600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819a450 con 0x7ff9281083c0 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff92d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff92d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58834/0 (socket says 172.21.15.46:58834) 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff92d200700 1 -- 172.21.15.46:0/1362151143 learned_addr learned my addr 172.21.15.46:0/1362151143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281032a0 0x7ff92819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.523+0000 7ff92e600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281996d0 0x7ff928199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 -- 172.21.15.46:0/1362151143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281032a0 msgr2=0x7ff92819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281032a0 0x7ff92819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 -- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281996d0 msgr2=0x7ff928199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281996d0 0x7ff928199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 -- 172.21.15.46:0/1362151143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140095f0 con 0x7ff9281083c0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff92d200700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff914000c00 tx=0x7ff914004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff926000700 1 -- 172.21.15.46:0/1362151143 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91401d070 con 0x7ff9281083c0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff926000700 1 -- 172.21.15.46:0/1362151143 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff914021410 con 0x7ff9281083c0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9281a63b0 con 0x7ff9281083c0 2024-05-08T23:14:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.524+0000 7ff926000700 1 -- 172.21.15.46:0/1362151143 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914017650 con 0x7ff9281083c0 2024-05-08T23:14:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.525+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9281a6880 con 0x7ff9281083c0 2024-05-08T23:14:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.528+0000 7ff926000700 1 -- 172.21.15.46:0/1362151143 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff914021580 con 0x7ff9281083c0 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.529+0000 7ff926000700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 0x7ff9100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.529+0000 7ff9441abb80 1 Processor -- start 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.529+0000 7ff926000700 1 -- 172.21.15.46:0/1362151143 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff9140ce2b0 con 0x7ff9281083c0 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.529+0000 7ff9441abb80 1 -- start start 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.529+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 0x7ff9100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f47e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f478350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f478890 0x55702f478cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f3cf150 con 0x55702f478890 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f3ccb10 con 0x55702f47de00 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff9441abb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f3c8400 con 0x55702f47e720 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f47e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f478350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f47e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47438/0 (socket says 172.21.15.46:47438) 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92dc00700 1 -- 172.21.15.46:0/3473753035 learned_addr learned my addr 172.21.15.46:0/3473753035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92e600700 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f478890 0x55702f478cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 -- 172.21.15.46:0/3473753035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 msgr2=0x55702f47e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f47e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 -- 172.21.15.46:0/3473753035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f478890 msgr2=0x55702f478cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f478890 0x55702f478cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92d200700 1 -- 172.21.15.46:0/3473753035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702f2493b0 con 0x55702f47e720 2024-05-08T23:14:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.530+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 0x7ff9100a37d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff91c011630 tx=0x7ff91c0119f0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff92d200700 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f478350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff9140cb590 tx=0x7ff914097320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5b7f7738af7d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff91aa00700 1 -- 172.21.15.46:0/3473753035 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91409ab20 con 0x55702f47e720 2024-05-08T23:14:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff91aa00700 1 -- 172.21.15.46:0/3473753035 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91409a940 con 0x55702f47e720 2024-05-08T23:14:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff91aa00700 1 -- 172.21.15.46:0/3473753035 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9140a33d0 con 0x55702f47e720 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff9441abb80 1 -- 172.21.15.46:0/3473753035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 msgr2=0x55702f478350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.531+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f478350 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff9140cb590 tx=0x7ff914097320 comp rx=0 tx=0).stop 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 -- 172.21.15.46:0/3473753035 shutdown_connections 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f478890 0x55702f478cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f47e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3473753035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f478350 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 -- 172.21.15.46:0/3473753035 >> 172.21.15.46:0/3473753035 conn(0x55702f3d6210 msgr2=0x55702f472df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 -- 172.21.15.46:0/3473753035 shutdown_connections 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 -- 172.21.15.46:0/3473753035 wait complete. 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 -- start start 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.532+0000 7ff9441abb80 1 Processor -- start 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f4846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47de00 0x55702f481a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f364560 con 0x55702f47de00 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f364200 con 0x55702f478890 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f24b6a0 con 0x55702f47e720 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff9441abb80 10 client.0 _ll_register_callbacks cb 0x55702f36db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff92e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff92e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58852/0 (socket says 172.21.15.46:58852) 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff92e600700 1 -- 172.21.15.46:0/2533424510 learned_addr learned my addr 172.21.15.46:0/2533424510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff92d200700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47de00 0x55702f481a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.533+0000 7ff92dc00700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f4846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 -- 172.21.15.46:0/2533424510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 msgr2=0x55702f4846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f4846c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 -- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47de00 msgr2=0x55702f481a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47de00 0x55702f481a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 -- 172.21.15.46:0/2533424510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9281a6880 con 0x55702f47e720 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff92e600700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff920009fd0 tx=0x7ff920007b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68228ce148158f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff90f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000e040 con 0x55702f47e720 2024-05-08T23:14:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55702f321270 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.534+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff9200138c0 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff920013a80 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 10 client.28672 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55702f1f5b60 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55702f321270 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff92001e3b0 con 0x55702f47e720 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 10 client.28672 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 10 client.28672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 20 client.28672 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f491de0 0x55702f494200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 4 client.28672 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55702f447810 con 0x55702f491de0 2024-05-08T23:14:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.535+0000 7ff92dc00700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f491de0 0x55702f494200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.536+0000 7ff92dc00700 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f491de0 0x55702f494200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8e528e5362e0db0 server_cookie=268d347fe6af57d8 in_seq=0 out_seq=0 2024-05-08T23:14:47.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.536+0000 7ff90f400700 10 client.28672 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff90f400700 1 -- 172.21.15.46:0/2533424510 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55702f447810 con 0x55702f491de0 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff90f400700 10 client.28672 handle_command_reply: tid=0 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 client.28672 shutdown 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 msgr2=0x55702f4820f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff920009fd0 tx=0x7ff920007b10 comp rx=0 tx=0).stop 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 shutdown_connections 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47e720 0x55702f4820f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f4846c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f491de0 0x55702f494200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 --2- 172.21.15.46:0/2533424510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47de00 0x55702f481a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 >> 172.21.15.46:0/2533424510 conn(0x55702f3d6210 msgr2=0x55702f4725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff90f400700 0 client.28672 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 shutdown_connections 2024-05-08T23:14:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 1 -- 172.21.15.46:0/2533424510 wait complete. 2024-05-08T23:14:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.537+0000 7ff9441abb80 20 client.28672 trim_cache size 0 max 16384 2024-05-08T23:14:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 Processor -- start 2024-05-08T23:14:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 -- start start 2024-05-08T23:14:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f481fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f472d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f473e80 con 0x55702f47e720 2024-05-08T23:14:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f474070 con 0x55702f47de00 2024-05-08T23:14:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff9441abb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f46fdc0 con 0x55702f478890 2024-05-08T23:14:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.553+0000 7ff92dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f481fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58866/0 (socket says 172.21.15.46:58866) 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 -- 172.21.15.46:0/3916335088 learned_addr learned my addr 172.21.15.46:0/3916335088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92e600700 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f472d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 -- 172.21.15.46:0/3916335088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 msgr2=0x55702f481fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f481fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 -- 172.21.15.46:0/3916335088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 msgr2=0x55702f472d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f472d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 -- 172.21.15.46:0/3916335088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702f46f4e0 con 0x55702f478890 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff92dc00700 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff91c013010 tx=0x7ff91c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd3d51cd94d49ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff91b400700 1 -- 172.21.15.46:0/3916335088 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c02f070 con 0x55702f478890 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff91b400700 1 -- 172.21.15.46:0/3916335088 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91c009dc0 con 0x55702f478890 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.554+0000 7ff91b400700 1 -- 172.21.15.46:0/3916335088 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c025040 con 0x55702f478890 2024-05-08T23:14:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 -- 172.21.15.46:0/3916335088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 msgr2=0x55702f478350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff91c013010 tx=0x7ff91c01f000 comp rx=0 tx=0).stop 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 -- 172.21.15.46:0/3916335088 shutdown_connections 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f472d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f478890 0x55702f478350 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 --2- 172.21.15.46:0/3916335088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f47de00 0x55702f481fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 -- 172.21.15.46:0/3916335088 >> 172.21.15.46:0/3916335088 conn(0x55702f3d6210 msgr2=0x55702f47bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 -- 172.21.15.46:0/3916335088 shutdown_connections 2024-05-08T23:14:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.555+0000 7ff9441abb80 1 -- 172.21.15.46:0/3916335088 wait complete. 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 -- start start 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 Processor -- start 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f488e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f479930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f479e70 con 0x55702f47e720 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f479fe0 con 0x55702f478890 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.556+0000 7ff9441abb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f47a1a0 con 0x55702f47de00 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff9441abb80 10 client.0 _ll_register_callbacks cb 0x55702f36db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92e600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f479930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58868/0 (socket says 172.21.15.46:58868) 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 -- 172.21.15.46:0/1326608790 learned_addr learned my addr 172.21.15.46:0/1326608790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f488e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 -- 172.21.15.46:0/1326608790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 msgr2=0x55702f488e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f488e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 -- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 msgr2=0x55702f479930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f479930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 -- 172.21.15.46:0/1326608790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c004150 con 0x55702f47de00 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92e600700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f479930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff92d200700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff914099330 tx=0x7ff91400b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ddd4eeb415d6dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.557+0000 7ff90f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91409ab20 con 0x55702f47de00 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff91409b6e0 con 0x55702f47de00 2024-05-08T23:14:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff91409a940 con 0x55702f47de00 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9140b92f0 con 0x55702f47de00 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff9441abb80 10 client.28678 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55702f38e560 con 0x55702f47de00 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff914080080 con 0x55702f47de00 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9140a2f40 con 0x55702f47de00 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff9441abb80 10 client.28678 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.558+0000 7ff9441abb80 10 client.28678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff9441abb80 20 client.28678 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f4e8cc0 0x55702f4eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff9441abb80 4 client.28678 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55702f3c0ec0 con 0x55702f4e8cc0 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f4e8cc0 0x55702f4eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff92dc00700 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f4e8cc0 0x55702f4eb0b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dd0edc2c8114cab server_cookie=125066a96fc6bdef in_seq=0 out_seq=0 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.559+0000 7ff90f400700 10 client.28678 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff90f400700 1 -- 172.21.15.46:0/1326608790 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55702f3c0ec0 con 0x55702f4e8cc0 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff90f400700 10 client.28678 handle_command_reply: tid=0 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 client.28678 shutdown 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 msgr2=0x55702f4893c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff914099330 tx=0x7ff91400b640 comp rx=0 tx=0).stop 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 shutdown_connections 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55702f4e8cc0 0x55702f4eb0b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702f47e720 0x55702f479930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff90f400700 0 client.28678 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702f478890 0x55702f488e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 --2- 172.21.15.46:0/1326608790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702f47de00 0x55702f4893c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 >> 172.21.15.46:0/1326608790 conn(0x55702f3d6210 msgr2=0x55702f4801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 shutdown_connections 2024-05-08T23:14:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 1 -- 172.21.15.46:0/1326608790 wait complete. 2024-05-08T23:14:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.560+0000 7ff9441abb80 20 client.28678 trim_cache size 0 max 16384 2024-05-08T23:14:47.565 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 msgr2=0x7ff9100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 0x7ff9100a37d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff91c011630 tx=0x7ff91c0119f0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 msgr2=0x7ff92819fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff914000c00 tx=0x7ff914004730 comp rx=0 tx=0).stop 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 shutdown_connections 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff9100a1310 0x7ff9100a37d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9281996d0 0x7ff928199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9281032a0 0x7ff92819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 --2- 172.21.15.46:0/1362151143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9281083c0 0x7ff92819fae0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 >> 172.21.15.46:0/1362151143 conn(0x7ff9280feac0 msgr2=0x7ff9280ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 shutdown_connections 2024-05-08T23:14:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.561+0000 7ff930600700 1 -- 172.21.15.46:0/1362151143 wait complete. 2024-05-08T23:14:47.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 -- 172.21.15.46:0/2740025707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 msgr2=0x7f96f010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 --2- 172.21.15.46:0/2740025707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f010d880 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b0a0 tx=0x7f96e800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 -- 172.21.15.46:0/2740025707 shutdown_connections 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 --2- 172.21.15.46:0/2740025707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f010d880 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 --2- 172.21.15.46:0/2740025707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f0101fa0 0x7f96f0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 --2- 172.21.15.46:0/2740025707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f01028c0 0x7f96f010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 -- 172.21.15.46:0/2740025707 >> 172.21.15.46:0/2740025707 conn(0x7f96f00fb760 msgr2=0x7f96f00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.803+0000 7f96f7200700 1 -- 172.21.15.46:0/2740025707 shutdown_connections 2024-05-08T23:14:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.804+0000 7f96f7200700 1 -- 172.21.15.46:0/2740025707 wait complete. 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 Processor -- start 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 -- start start 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f01028c0 0x7f96f0100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f0100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01a5870 con 0x7f96f01028c0 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01a59e0 con 0x7f96f0101fa0 2024-05-08T23:14:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.805+0000 7f96f7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01a5ba0 con 0x7f96f010b3c0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47508/0 (socket says 172.21.15.46:47508) 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 -- 172.21.15.46:0/203104538 learned_addr learned my addr 172.21.15.46:0/203104538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96ef400700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f01028c0 0x7f96f0100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96f5200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f0100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 -- 172.21.15.46:0/203104538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 msgr2=0x7f96f0100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f0100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 -- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f01028c0 msgr2=0x7f96f0100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f01028c0 0x7f96f0100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 -- 172.21.15.46:0/203104538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0009520 con 0x7f96f0101fa0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96efe00700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f96e000eb30 tx=0x7f96e000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.806+0000 7f96ecc00700 1 -- 172.21.15.46:0/203104538 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0009880 con 0x7f96f0101fa0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.807+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e8009d00 con 0x7f96f0101fa0 2024-05-08T23:14:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.807+0000 7f96ecc00700 1 -- 172.21.15.46:0/203104538 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e00044e0 con 0x7f96f0101fa0 2024-05-08T23:14:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.807+0000 7f96ecc00700 1 -- 172.21.15.46:0/203104538 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0003d80 con 0x7f96f0101fa0 2024-05-08T23:14:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.807+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01a61f0 con 0x7f96f0101fa0 2024-05-08T23:14:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.810+0000 7f96ecc00700 1 -- 172.21.15.46:0/203104538 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f96e0003f20 con 0x7f96f0101fa0 2024-05-08T23:14:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.811+0000 7f970acd3b80 1 Processor -- start 2024-05-08T23:14:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f96ecc00700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 0x7f96d40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 -- start start 2024-05-08T23:14:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f96ecc00700 1 -- 172.21.15.46:0/203104538 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f96e0014070 con 0x7f96f0101fa0 2024-05-08T23:14:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f96f5200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 0x7f96d40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 0x55b66d3c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c2890 0x55b66d3c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d319150 con 0x55b66d3c7e00 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d316b10 con 0x55b66d3c8720 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f970acd3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d312400 con 0x55b66d3c2890 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f96ef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.812+0000 7f96ef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47524/0 (socket says 172.21.15.46:47524) 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96ef400700 1 -- 172.21.15.46:0/1023501168 learned_addr learned my addr 172.21.15.46:0/1023501168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 0x55b66d3c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96f5200700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c2890 0x55b66d3c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 -- 172.21.15.46:0/1023501168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c2890 msgr2=0x55b66d3c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c2890 0x55b66d3c2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96f5200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 0x7f96d40a37d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b780 tx=0x7f96e8006230 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 -- 172.21.15.46:0/1023501168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 msgr2=0x55b66d3c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 -- 172.21.15.46:0/1023501168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66d1933b0 con 0x55b66d3c7e00 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96ef400700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96efe00700 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 0x55b66d3c81e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ca8f0 tx=0x7f96e0099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de28b29b0b328dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.813+0000 7f96e5600700 1 -- 172.21.15.46:0/1023501168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e009a2b0 con 0x55b66d3c7e00 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f96e5600700 1 -- 172.21.15.46:0/1023501168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e009e0e0 con 0x55b66d3c7e00 2024-05-08T23:14:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f96e5600700 1 -- 172.21.15.46:0/1023501168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e009a510 con 0x55b66d3c7e00 2024-05-08T23:14:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 -- 172.21.15.46:0/1023501168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 msgr2=0x55b66d3c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 0x55b66d3c81e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f96e00ca8f0 tx=0x7f96e0099ec0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 -- 172.21.15.46:0/1023501168 shutdown_connections 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c2890 0x55b66d3c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c7e00 0x55b66d3c81e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 --2- 172.21.15.46:0/1023501168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c8720 0x55b66d3c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 -- 172.21.15.46:0/1023501168 >> 172.21.15.46:0/1023501168 conn(0x55b66d320210 msgr2=0x55b66d3bcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.814+0000 7f970acd3b80 1 -- 172.21.15.46:0/1023501168 shutdown_connections 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.815+0000 7f970acd3b80 1 -- 172.21.15.46:0/1023501168 wait complete. 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.815+0000 7f970acd3b80 1 -- start start 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.815+0000 7f970acd3b80 1 Processor -- start 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c2890 0x55b66d3cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 0x55b66d3cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d2ae560 con 0x55b66d3c8720 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d2ae200 con 0x55b66d3c2890 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d1956a0 con 0x55b66d3c7e00 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f970acd3b80 10 client.0 _ll_register_callbacks cb 0x55b66d2b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58908/0 (socket says 172.21.15.46:58908) 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 -- 172.21.15.46:0/601297365 learned_addr learned my addr 172.21.15.46:0/601297365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96f5200700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 0x55b66d3cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96efe00700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c2890 0x55b66d3cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 -- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c2890 msgr2=0x55b66d3cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c2890 0x55b66d3cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 -- 172.21.15.46:0/601297365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 msgr2=0x55b66d3cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 0x55b66d3cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96ef400700 1 -- 172.21.15.46:0/601297365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0098b0 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.816+0000 7f96f5200700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 0x55b66d3cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f96ef400700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f96dc011850 tx=0x7f96dc011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2ca39efca51b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f96da000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc016070 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f96dc004320 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b66d26b270 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96dc0044e0 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f970acd3b80 10 client.28690 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.817+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f96f01a61f0 con 0x55b66d3c7e00 2024-05-08T23:14:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b66d26b270 con 0x55b66d3c7e00 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96dc00f9a0 con 0x55b66d3c7e00 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 10 client.28690 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 10 client.28690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 20 client.28690 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d3dc380 0x55b66d3de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 4 client.28690 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b66d391810 con 0x55b66d3dc380 2024-05-08T23:14:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.818+0000 7f96efe00700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d3dc380 0x55b66d3de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.819+0000 7f96efe00700 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d3dc380 0x55b66d3de7a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3689edafd880801e server_cookie=2bade3471b91b7a0 in_seq=0 out_seq=0 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.819+0000 7f96da000700 10 client.28690 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f96da000700 1 -- 172.21.15.46:0/601297365 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b66d391810 con 0x55b66d3dc380 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f96da000700 10 client.28690 handle_command_reply: tid=0 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 client.28690 shutdown 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 msgr2=0x55b66d3cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f96dc011850 tx=0x7f96dc011b60 comp rx=0 tx=0).stop 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 shutdown_connections 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d3dc380 0x55b66d3de7a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c8720 0x55b66d3cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c2890 0x55b66d3cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f96da000700 0 client.28690 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 --2- 172.21.15.46:0/601297365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c7e00 0x55b66d3cc100 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 >> 172.21.15.46:0/601297365 conn(0x55b66d320210 msgr2=0x55b66d3bc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 shutdown_connections 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 1 -- 172.21.15.46:0/601297365 wait complete. 2024-05-08T23:14:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.820+0000 7f970acd3b80 20 client.28690 trim_cache size 0 max 16384 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 Processor -- start 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 -- start start 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3d9a00 con 0x55b66d3c2890 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3be7e0 con 0x55b66d3c7e00 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f970acd3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3cd170 con 0x55b66d3c8720 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f96efe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.836+0000 7f96f5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53044/0 (socket says 172.21.15.46:53044) 2024-05-08T23:14:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96ef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 -- 172.21.15.46:0/3516236015 learned_addr learned my addr 172.21.15.46:0/3516236015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 -- 172.21.15.46:0/3516236015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 msgr2=0x55b66d3bcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3bcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 -- 172.21.15.46:0/3516236015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 msgr2=0x55b66d3d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 -- 172.21.15.46:0/3516236015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66d13fb60 con 0x55b66d3c2890 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96ef400700 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96f5200700 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3bcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96efe00700 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96e00044b0 tx=0x7f96e0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4183ef7d7711af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.837+0000 7f96e6000700 1 -- 172.21.15.46:0/3516236015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e009a2b0 con 0x55b66d3c2890 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f96e6000700 1 -- 172.21.15.46:0/3516236015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e0098650 con 0x55b66d3c2890 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f96e6000700 1 -- 172.21.15.46:0/3516236015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e009a510 con 0x55b66d3c2890 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 -- 172.21.15.46:0/3516236015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 msgr2=0x55b66d3de770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96e00044b0 tx=0x7f96e0009f60 comp rx=0 tx=0).stop 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 -- 172.21.15.46:0/3516236015 shutdown_connections 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3bcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3de770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 --2- 172.21.15.46:0/3516236015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 -- 172.21.15.46:0/3516236015 >> 172.21.15.46:0/3516236015 conn(0x55b66d320210 msgr2=0x55b66d3c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 -- 172.21.15.46:0/3516236015 shutdown_connections 2024-05-08T23:14:47.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.838+0000 7f970acd3b80 1 -- 172.21.15.46:0/3516236015 wait complete. 2024-05-08T23:14:47.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.839+0000 7f970acd3b80 1 -- start start 2024-05-08T23:14:47.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.839+0000 7f970acd3b80 1 Processor -- start 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.839+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.839+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.839+0000 7f970acd3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f970acd3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3d1870 con 0x55b66d3c2890 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f970acd3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3d19e0 con 0x55b66d3c7e00 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f970acd3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66d3d1ba0 con 0x55b66d3c8720 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f970acd3b80 10 client.0 _ll_register_callbacks cb 0x55b66d2b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96efe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96ef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58926/0 (socket says 172.21.15.46:58926) 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 -- 172.21.15.46:0/2019247100 learned_addr learned my addr 172.21.15.46:0/2019247100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 -- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 msgr2=0x55b66d3d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 -- 172.21.15.46:0/2019247100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 msgr2=0x55b66d3d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 -- 172.21.15.46:0/2019247100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01a61f0 con 0x55b66d3c8720 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96efe00700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96ef400700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96f5200700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96e8015000 tx=0x7f96e80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14faac10d682c716 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.840+0000 7f96da000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e800e040 con 0x55b66d3c8720 2024-05-08T23:14:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e009e0e0 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f96e801c850 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e8007e70 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f970acd3b80 10 client.28696 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f96f010e540 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e009e0e0 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96e80056d0 con 0x55b66d3c8720 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.841+0000 7f970acd3b80 10 client.28696 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f970acd3b80 10 client.28696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f970acd3b80 20 client.28696 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d432a50 0x55b66d434e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f970acd3b80 4 client.28696 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b66d30aec0 con 0x55b66d432a50 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f96efe00700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d432a50 0x55b66d434e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f96efe00700 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d432a50 0x55b66d434e70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9175a0ef40c8f1 server_cookie=349320137bf1f95d in_seq=0 out_seq=0 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.842+0000 7f96da000700 10 client.28696 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f96da000700 1 -- 172.21.15.46:0/2019247100 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b66d30aec0 con 0x55b66d432a50 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f96da000700 10 client.28696 handle_command_reply: tid=0 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 client.28696 shutdown 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 msgr2=0x55b66d3d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96e8015000 tx=0x7f96e80099e0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 shutdown_connections 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66d3c8720 0x55b66d3d11a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b66d432a50 0x55b66d434e70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b66d3c2890 0x55b66d3d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f96da000700 0 client.28696 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 --2- 172.21.15.46:0/2019247100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b66d3c7e00 0x55b66d3d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 >> 172.21.15.46:0/2019247100 conn(0x55b66d320210 msgr2=0x55b66d3ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 shutdown_connections 2024-05-08T23:14:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 1 -- 172.21.15.46:0/2019247100 wait complete. 2024-05-08T23:14:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.843+0000 7f970acd3b80 20 client.28696 trim_cache size 0 max 16384 2024-05-08T23:14:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:14:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 msgr2=0x7f96d40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 0x7f96d40a37d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b780 tx=0x7f96e8006230 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 msgr2=0x7f96f00ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f96e000eb30 tx=0x7f96e000ee40 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 shutdown_connections 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f010b3c0 0x7f96f0100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96f0101fa0 0x7f96f00ffbf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f96d40a1310 0x7f96d40a37d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 --2- 172.21.15.46:0/203104538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f01028c0 0x7f96f0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 >> 172.21.15.46:0/203104538 conn(0x7f96f00fb760 msgr2=0x7f96f00fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 shutdown_connections 2024-05-08T23:14:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:47.844+0000 7f96f7200700 1 -- 172.21.15.46:0/203104538 wait complete. 2024-05-08T23:14:47.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 957724 9.46 MB/sec execute 429 sec latency 28.065 ms 2024-05-08T23:14:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:47 smithi086 ceph-mon[78610]: pgmap v340: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-05-08T23:14:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:47 smithi053 ceph-mon[77587]: pgmap v340: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-05-08T23:14:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:47 smithi046 ceph-mon[36898]: pgmap v340: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-05-08T23:14:48.865 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:14:48.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:14:48.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 959583 9.46 MB/sec execute 430 sec latency 16.788 ms 2024-05-08T23:14:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 -- 172.21.15.46:0/938898224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1994105fb0 msgr2=0x7f199410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 --2- 172.21.15.46:0/938898224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1994105fb0 0x7f199410db30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f198c00b0a0 tx=0x7f198c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 -- 172.21.15.46:0/938898224 shutdown_connections 2024-05-08T23:14:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 --2- 172.21.15.46:0/938898224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1994105fb0 0x7f199410db30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 --2- 172.21.15.46:0/938898224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f1994101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 --2- 172.21.15.46:0/938898224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994101840 0x7f1994105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 -- 172.21.15.46:0/938898224 >> 172.21.15.46:0/938898224 conn(0x7f19940fc750 msgr2=0x7f19940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.084+0000 7f1993e00700 1 -- 172.21.15.46:0/938898224 shutdown_connections 2024-05-08T23:14:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.085+0000 7f1993e00700 1 -- 172.21.15.46:0/938898224 wait complete. 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 Processor -- start 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 -- start start 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994105fb0 0x7f19941976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 0x7f199419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419c510 con 0x7f1994105fb0 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419c680 con 0x7f1994100e90 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419c840 con 0x7f199419ba90 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1992a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1992a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47568/0 (socket says 172.21.15.46:47568) 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1992a00700 1 -- 172.21.15.46:0/3693209410 learned_addr learned my addr 172.21.15.46:0/3693209410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1993400700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 0x7f199419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.086+0000 7f1992000700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994105fb0 0x7f19941976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 -- 172.21.15.46:0/3693209410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 msgr2=0x7f199419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 0x7f199419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 -- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994105fb0 msgr2=0x7f19941976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994105fb0 0x7f19941976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 -- 172.21.15.46:0/3693209410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009520 con 0x7f1994100e90 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1993400700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 0x7f199419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f1992a00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f198400e9c0 tx=0x7f198400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.087+0000 7f198b400700 1 -- 172.21.15.46:0/3693209410 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840097b0 con 0x7f1994100e90 2024-05-08T23:14:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.088+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c009d00 con 0x7f1994100e90 2024-05-08T23:14:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.088+0000 7f198b400700 1 -- 172.21.15.46:0/3693209410 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f198400ee60 con 0x7f1994100e90 2024-05-08T23:14:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.088+0000 7f198b400700 1 -- 172.21.15.46:0/3693209410 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840054f0 con 0x7f1994100e90 2024-05-08T23:14:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.088+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a40a0 con 0x7f1994100e90 2024-05-08T23:14:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.091+0000 7f198b400700 1 -- 172.21.15.46:0/3693209410 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1984004750 con 0x7f1994100e90 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.092+0000 7f19ad7eeb80 1 Processor -- start 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.092+0000 7f19ad7eeb80 1 -- start start 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.092+0000 7f198b400700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 0x7f19780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f198b400700 1 -- 172.21.15.46:0/3693209410 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1984014070 con 0x7f1994100e90 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 0x5607c4e557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e56160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4da81d0 con 0x5607c4e55d20 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4da5b90 con 0x5607c4e5b450 2024-05-08T23:14:49.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4da1480 con 0x5607c4e5bd70 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1993400700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 0x7f19780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1992a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1992000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 0x5607c4e557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1992a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47574/0 (socket says 172.21.15.46:47574) 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1992a00700 1 -- 172.21.15.46:0/2582910146 learned_addr learned my addr 172.21.15.46:0/2582910146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.093+0000 7f1993400700 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e56160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 -- 172.21.15.46:0/2582910146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 msgr2=0x5607c4e5b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 -- 172.21.15.46:0/2582910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 msgr2=0x5607c4e56160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e56160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 -- 172.21.15.46:0/2582910146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c4c22430 con 0x5607c4e5bd70 2024-05-08T23:14:49.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992a00700 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1993400700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 0x7f19780a37f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f198c00b780 tx=0x7f198c007bf0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f1992000700 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 0x5607c4e557e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f199406a380 tx=0x7f1980011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31347d481573564 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f197fe00700 1 -- 172.21.15.46:0/2582910146 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980019630 con 0x5607c4e5bd70 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f197fe00700 1 -- 172.21.15.46:0/2582910146 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1980019c50 con 0x5607c4e5bd70 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.094+0000 7f197fe00700 1 -- 172.21.15.46:0/2582910146 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000eb90 con 0x5607c4e5bd70 2024-05-08T23:14:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/2582910146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 msgr2=0x5607c4e557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 0x5607c4e557e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f199406a380 tx=0x7f1980011720 comp rx=0 tx=0).stop 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/2582910146 shutdown_connections 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e56160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/2582910146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5bd70 0x5607c4e557e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/2582910146 >> 172.21.15.46:0/2582910146 conn(0x5607c4daf290 msgr2=0x5607c4e4c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/2582910146 shutdown_connections 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.095+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/2582910146 wait complete. 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.096+0000 7f19ad7eeb80 1 -- start start 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.096+0000 7f19ad7eeb80 1 Processor -- start 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 0x5607c4e5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4d3d5e0 con 0x5607c4e5bd70 2024-05-08T23:14:49.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4d3d280 con 0x5607c4e5b450 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4c24720 con 0x5607c4e55d20 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f19ad7eeb80 10 client.0 _ll_register_callbacks cb 0x5607c4d46bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58942/0 (socket says 172.21.15.46:58942) 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 -- 172.21.15.46:0/3774184850 learned_addr learned my addr 172.21.15.46:0/3774184850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1993400700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 0x5607c4e5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 -- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 msgr2=0x5607c4e5f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 -- 172.21.15.46:0/3774184850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 msgr2=0x5607c4e5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 0x5607c4e5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 -- 172.21.15.46:0/3774184850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a40a0 con 0x5607c4e55d20 2024-05-08T23:14:49.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1993400700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 0x5607c4e5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f1992a00700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1984096490 tx=0x7f198409a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84843fc77adfd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.097+0000 7f197cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840a2460 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1984005690 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607c4cfa2f0 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1984098380 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f19ad7eeb80 10 client.28708 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1980009a70 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f198407b080 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f198409a240 con 0x5607c4e55d20 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f19ad7eeb80 10 client.28708 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.098+0000 7f19ad7eeb80 10 client.28708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.099+0000 7f19ad7eeb80 20 client.28708 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:49.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.099+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4e6f720 0x5607c4e71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.099+0000 7f19ad7eeb80 4 client.28708 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:49.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.099+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607c4e20890 con 0x5607c4e6f720 2024-05-08T23:14:49.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.099+0000 7f1992000700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4e6f720 0x5607c4e71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.100+0000 7f1992000700 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4e6f720 0x5607c4e71b10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3067177cbf09da3 server_cookie=d094069f8a30ef0b in_seq=0 out_seq=0 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.100+0000 7f197cc00700 10 client.28708 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.100+0000 7f197cc00700 1 -- 172.21.15.46:0/3774184850 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607c4e20890 con 0x5607c4e6f720 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.100+0000 7f197cc00700 10 client.28708 handle_command_reply: tid=0 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.100+0000 7f19ad7eeb80 1 client.28708 shutdown 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 msgr2=0x5607c4e5f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1984096490 tx=0x7f198409a030 comp rx=0 tx=0).stop 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 shutdown_connections 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4e6f720 0x5607c4e71b10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e5bd70 0x5607c4e5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e55d20 0x5607c4e5f100 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/3774184850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5b450 0x5607c4e5f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 >> 172.21.15.46:0/3774184850 conn(0x5607c4daf290 msgr2=0x5607c4e4b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f197cc00700 0 client.28708 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 shutdown_connections 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/3774184850 wait complete. 2024-05-08T23:14:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.101+0000 7f19ad7eeb80 20 client.28708 trim_cache size 0 max 16384 2024-05-08T23:14:49.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.119+0000 7f19ad7eeb80 1 Processor -- start 2024-05-08T23:14:49.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.119+0000 7f19ad7eeb80 1 -- start start 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e76470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4e607a0 con 0x5607c4e55d20 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4e5b0d0 con 0x5607c4e5bd70 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4e468b0 con 0x5607c4e5b450 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f1992000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e76470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f1993400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f1992000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e76470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58948/0 (socket says 172.21.15.46:58948) 2024-05-08T23:14:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f1992000700 1 -- 172.21.15.46:0/1112778783 learned_addr learned my addr 172.21.15.46:0/1112778783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.120+0000 7f1992a00700 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 -- 172.21.15.46:0/1112778783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 msgr2=0x5607c4e76470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e76470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 -- 172.21.15.46:0/1112778783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 msgr2=0x5607c4e5f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e5f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 -- 172.21.15.46:0/1112778783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c4bcebe0 con 0x5607c4e5bd70 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1993400700 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e59210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f198c0062a0 tx=0x7f198c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac60b0a54e85d4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1988c00700 1 -- 172.21.15.46:0/1112778783 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00e040 con 0x5607c4e5bd70 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1988c00700 1 -- 172.21.15.46:0/1112778783 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f198c02a920 con 0x5607c4e5bd70 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.121+0000 7f1988c00700 1 -- 172.21.15.46:0/1112778783 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c01f8e0 con 0x5607c4e5bd70 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/1112778783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 msgr2=0x5607c4e59210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e59210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f198c0062a0 tx=0x7f198c023000 comp rx=0 tx=0).stop 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/1112778783 shutdown_connections 2024-05-08T23:14:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e59210 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/1112778783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e76470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/1112778783 >> 172.21.15.46:0/1112778783 conn(0x5607c4daf290 msgr2=0x5607c4e56100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/1112778783 shutdown_connections 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.122+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/1112778783 wait complete. 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 -- start start 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 Processor -- start 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e63360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e63de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4ec5bf0 con 0x5607c4e55d20 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4ec5d30 con 0x5607c4e5bd70 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c4ec5e70 con 0x5607c4e5b450 2024-05-08T23:14:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.123+0000 7f19ad7eeb80 10 client.0 _ll_register_callbacks cb 0x5607c4d46bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e63360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58950/0 (socket says 172.21.15.46:58950) 2024-05-08T23:14:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 -- 172.21.15.46:0/653820175 learned_addr learned my addr 172.21.15.46:0/653820175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1993400700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e63de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 -- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 msgr2=0x5607c4e63de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e63de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 -- 172.21.15.46:0/653820175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 msgr2=0x5607c4e63360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e63360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 -- 172.21.15.46:0/653820175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c026050 con 0x5607c4e5b450 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f1992000700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f199406a380 tx=0x7f19800053e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f085e7170bf95a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.124+0000 7f197cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980019630 con 0x5607c4e5b450 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1980019790 con 0x5607c4e5b450 2024-05-08T23:14:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1980019c50 con 0x5607c4e5b450 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f198000e3b0 con 0x5607c4e5b450 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 10 client.28714 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5607c4d675e0 con 0x5607c4e5b450 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1980019c50 con 0x5607c4e5b450 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f198000e650 con 0x5607c4e5b450 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 10 client.28714 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 10 client.28714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 20 client.28714 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4ec7690 0x5607c4ec9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 4 client.28714 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:14:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.125+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5607c4d99f40 con 0x5607c4ec7690 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.126+0000 7f1992a00700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4ec7690 0x5607c4ec9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.126+0000 7f1992a00700 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4ec7690 0x5607c4ec9ab0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe026831a8078b24 server_cookie=4f21f9b4c29ae371 in_seq=0 out_seq=0 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.126+0000 7f197cc00700 10 client.28714 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.127+0000 7f197cc00700 1 -- 172.21.15.46:0/653820175 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5607c4d99f40 con 0x5607c4ec7690 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.127+0000 7f197cc00700 10 client.28714 handle_command_reply: tid=0 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.127+0000 7f19ad7eeb80 1 client.28714 shutdown 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 msgr2=0x5607c4e638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f199406a380 tx=0x7f19800053e0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 shutdown_connections 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607c4e5bd70 0x5607c4e63de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5607c4ec7690 0x5607c4ec9ab0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607c4e55d20 0x5607c4e63360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 --2- 172.21.15.46:0/653820175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607c4e5b450 0x5607c4e638a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f197cc00700 0 client.28714 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 >> 172.21.15.46:0/653820175 conn(0x5607c4daf290 msgr2=0x5607c4e76470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 shutdown_connections 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 1 -- 172.21.15.46:0/653820175 wait complete. 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.128+0000 7f19ad7eeb80 20 client.28714 trim_cache size 0 max 16384 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4711546a-5d5d-460d-aa5b-e3eb5ced928a", 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:14:49.132 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 msgr2=0x7f19780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 0x7f19780a37f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f198c00b780 tx=0x7f198c007bf0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 msgr2=0x7f19941971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f198400e9c0 tx=0x7f198400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 shutdown_connections 2024-05-08T23:14:49.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f199419ba90 0x7f199419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994100e90 0x7f19941971b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f19780a1330 0x7f19780a37f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 --2- 172.21.15.46:0/3693209410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994105fb0 0x7f19941976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 >> 172.21.15.46:0/3693209410 conn(0x7f19940fc750 msgr2=0x7f19940fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 shutdown_connections 2024-05-08T23:14:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.129+0000 7f1993e00700 1 -- 172.21.15.46:0/3693209410 wait complete. 2024-05-08T23:14:49.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 -- 172.21.15.46:0/178611148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24101a30 msgr2=0x7faf24105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 --2- 172.21.15.46:0/178611148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24101a30 0x7faf24105a70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf10009990 tx=0x7faf10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 -- 172.21.15.46:0/178611148 shutdown_connections 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 --2- 172.21.15.46:0/178611148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf2410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 --2- 172.21.15.46:0/178611148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 0x7faf24101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 --2- 172.21.15.46:0/178611148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24101a30 0x7faf24105a70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 -- 172.21.15.46:0/178611148 >> 172.21.15.46:0/178611148 conn(0x7faf240fc8e0 msgr2=0x7faf240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 -- 172.21.15.46:0/178611148 shutdown_connections 2024-05-08T23:14:49.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.416+0000 7faf2c200700 1 -- 172.21.15.46:0/178611148 wait complete. 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.417+0000 7faf2c200700 1 Processor -- start 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 -- start start 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 0x7faf240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 0x7faf241a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf241a5b00 con 0x7faf24101080 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf241a5c70 con 0x7faf241061a0 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf241a5e30 con 0x7faf24073d20 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf28e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf29800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 0x7faf240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf2a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 0x7faf241a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf28e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47626/0 (socket says 172.21.15.46:47626) 2024-05-08T23:14:49.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.418+0000 7faf28e00700 1 -- 172.21.15.46:0/2741390529 learned_addr learned my addr 172.21.15.46:0/2741390529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 -- 172.21.15.46:0/2741390529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 msgr2=0x7faf241a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 0x7faf241a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 -- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 msgr2=0x7faf240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 0x7faf240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 -- 172.21.15.46:0/2741390529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100095f0 con 0x7faf241061a0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf2a200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 0x7faf241a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf28e00700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faf10005950 tx=0x7faf10005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.419+0000 7faf22000700 1 -- 172.21.15.46:0/2741390529 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1001d070 con 0x7faf241061a0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.420+0000 7faf22000700 1 -- 172.21.15.46:0/2741390529 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf10005da0 con 0x7faf241061a0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.420+0000 7faf22000700 1 -- 172.21.15.46:0/2741390529 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10021e70 con 0x7faf241061a0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.420+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf241a60b0 con 0x7faf241061a0 2024-05-08T23:14:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.420+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf241a6480 con 0x7faf241061a0 2024-05-08T23:14:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.424+0000 7faf22000700 1 -- 172.21.15.46:0/2741390529 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf100215a0 con 0x7faf241061a0 2024-05-08T23:14:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.425+0000 7faf3fcb3b80 1 Processor -- start 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.425+0000 7faf3fcb3b80 1 -- start start 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c7f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691bd6150 con 0x562691c85720 2024-05-08T23:14:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691bd3b10 con 0x562691c7f890 2024-05-08T23:14:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691bcf400 con 0x562691c84e00 2024-05-08T23:14:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf2a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf28e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf2a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58988/0 (socket says 172.21.15.46:58988) 2024-05-08T23:14:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf2a200700 1 -- 172.21.15.46:0/3165120848 learned_addr learned my addr 172.21.15.46:0/3165120848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.426+0000 7faf22000700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 0x7faf0c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf29800700 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c7f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf2a200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 0x7faf0c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf22000700 1 -- 172.21.15.46:0/2741390529 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf100ce120 con 0x7faf241061a0 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 -- 172.21.15.46:0/3165120848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 msgr2=0x562691c851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 -- 172.21.15.46:0/3165120848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 msgr2=0x562691c7f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c7f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 -- 172.21.15.46:0/3165120848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562691a503b0 con 0x562691c7f890 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf29800700 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c7f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf28e00700 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c7fcd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf100cb400 tx=0x7faf10009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf42e299034cde40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf2a200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 0x7faf0c0a37d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf1c004280 tx=0x7faf1c004b40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.427+0000 7faf16a00700 1 -- 172.21.15.46:0/3165120848 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf100a10b0 con 0x562691c7f890 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf16a00700 1 -- 172.21.15.46:0/3165120848 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf1009aab0 con 0x562691c7f890 2024-05-08T23:14:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf16a00700 1 -- 172.21.15.46:0/3165120848 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1006e720 con 0x562691c7f890 2024-05-08T23:14:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3165120848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 msgr2=0x562691c7fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c7fcd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf100cb400 tx=0x7faf10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3165120848 shutdown_connections 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c7fcd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3165120848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c7f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3165120848 >> 172.21.15.46:0/3165120848 conn(0x562691bdd210 msgr2=0x562691c79df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3165120848 shutdown_connections 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.428+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3165120848 wait complete. 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.429+0000 7faf3fcb3b80 1 -- start start 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.429+0000 7faf3fcb3b80 1 Processor -- start 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c88990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691b6b560 con 0x562691c84e00 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691b6b200 con 0x562691c7f890 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691a526a0 con 0x562691c85720 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf28e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c88990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47648/0 (socket says 172.21.15.46:47648) 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf29800700 1 -- 172.21.15.46:0/3448053129 learned_addr learned my addr 172.21.15.46:0/3448053129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.430+0000 7faf2a200700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf3fcb3b80 10 client.0 _ll_register_callbacks cb 0x562691b74b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 -- 172.21.15.46:0/3448053129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 msgr2=0x562691c89080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c89080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 -- 172.21.15.46:0/3448053129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 msgr2=0x562691c88990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c88990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 -- 172.21.15.46:0/3448053129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf241a6480 con 0x562691c7f890 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562691b28270 con 0x562691c7f890 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf29800700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faf18007ee0 tx=0x7faf18012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358c65935290915d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.431+0000 7faf0b400700 10 client.28634 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf0b400700 1 -- 172.21.15.46:0/3448053129 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18010930 con 0x562691c7f890 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf0b400700 1 -- 172.21.15.46:0/3448053129 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf18010a90 con 0x562691c7f890 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf0b400700 1 -- 172.21.15.46:0/3448053129 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf18010c50 con 0x562691c7f890 2024-05-08T23:14:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 10 client.28634 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x5626919fcb60 con 0x562691c7f890 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf0b400700 1 -- 172.21.15.46:0/3448053129 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf18025470 con 0x562691c7f890 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 10 client.28634 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 10 client.28634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 20 client.28634 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691c98c50 0x562691c9b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 4 client.28634 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.432+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562691c4e810 con 0x562691c98c50 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.433+0000 7faf28e00700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691c98c50 0x562691c9b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.433+0000 7faf28e00700 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691c98c50 0x562691c9b080 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cd09a96f36de7c server_cookie=94d3d265e9e767b0 in_seq=0 out_seq=0 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.433+0000 7faf0b400700 10 client.28634 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf0b400700 1 -- 172.21.15.46:0/3448053129 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562691c4e810 con 0x562691c98c50 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf0b400700 10 client.28634 handle_command_reply: tid=0 2024-05-08T23:14:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 client.28634 shutdown 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 msgr2=0x562691c8b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faf18007ee0 tx=0x7faf18012730 comp rx=0 tx=0).stop 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 shutdown_connections 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c89080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8b710 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691c98c50 0x562691c9b080 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/3448053129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c88990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf0b400700 0 client.28634 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 >> 172.21.15.46:0/3448053129 conn(0x562691bdd210 msgr2=0x562691c795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.434+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 shutdown_connections 2024-05-08T23:14:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.435+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/3448053129 wait complete. 2024-05-08T23:14:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.435+0000 7faf3fcb3b80 20 client.28634 trim_cache size 0 max 16384 2024-05-08T23:14:49.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 Processor -- start 2024-05-08T23:14:49.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 -- start start 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c79d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c8b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691c76dc0 con 0x562691c85720 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691ca02d0 con 0x562691c7f890 2024-05-08T23:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.455+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691c7af80 con 0x562691c84e00 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47664/0 (socket says 172.21.15.46:47664) 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 -- 172.21.15.46:0/2564954937 learned_addr learned my addr 172.21.15.46:0/2564954937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf2a200700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c8b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf28e00700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c79d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 -- 172.21.15.46:0/2564954937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 msgr2=0x562691c79d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c79d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 -- 172.21.15.46:0/2564954937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 msgr2=0x562691c8b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c8b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf29800700 1 -- 172.21.15.46:0/2564954937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562691c7ff40 con 0x562691c7f890 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.456+0000 7faf28e00700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.457+0000 7faf29800700 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf18012770 tx=0x7faf18015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f64655552e16340 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.457+0000 7faf17400700 1 -- 172.21.15.46:0/2564954937 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18004030 con 0x562691c7f890 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.457+0000 7faf17400700 1 -- 172.21.15.46:0/2564954937 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf1802b070 con 0x562691c7f890 2024-05-08T23:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.457+0000 7faf17400700 1 -- 172.21.15.46:0/2564954937 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18004e90 con 0x562691c7f890 2024-05-08T23:14:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/2564954937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 msgr2=0x562691c88930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf18012770 tx=0x7faf18015000 comp rx=0 tx=0).stop 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/2564954937 shutdown_connections 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c85720 0x562691c8b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c88930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/2564954937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c84e00 0x562691c79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/2564954937 >> 172.21.15.46:0/2564954937 conn(0x562691bdd210 msgr2=0x562691c88220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/2564954937 shutdown_connections 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/2564954937 wait complete. 2024-05-08T23:14:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.458+0000 7faf3fcb3b80 1 -- start start 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 Processor -- start 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c90360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691c80f20 con 0x562691c84e00 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691c81090 con 0x562691c7f890 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562691c81250 con 0x562691c85720 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.459+0000 7faf3fcb3b80 10 client.0 _ll_register_callbacks cb 0x562691b74b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c90360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf29800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47680/0 (socket says 172.21.15.46:47680) 2024-05-08T23:14:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf29800700 1 -- 172.21.15.46:0/19653004 learned_addr learned my addr 172.21.15.46:0/19653004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf2a200700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 -- 172.21.15.46:0/19653004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 msgr2=0x562691c80990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c80990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 -- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 msgr2=0x562691c8fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf28e00700 1 -- 172.21.15.46:0/19653004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562691a503b0 con 0x562691c84e00 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.460+0000 7faf29800700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf28e00700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c90360 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf1009ac20 tx=0x7faf10096cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecef04e34c9b996f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf0b400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf100a3360 con 0x562691c84e00 2024-05-08T23:14:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf1009b670 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf1009aab0 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf100a10b0 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf3fcb3b80 10 client.18633 fetch_fsmap learned FSMap version 30 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626919fcb60 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.461+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf1009aab0 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf1006c310 con 0x562691c84e00 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 10 client.18633 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 10 client.18633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 20 client.18633 populate_metadata read hostname 'smithi046' 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691cefd80 0x562691cf2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 4 client.18633 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:14:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562691bc7ec0 con 0x562691cefd80 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.462+0000 7faf29800700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691cefd80 0x562691cf2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf29800700 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691cefd80 0x562691cf2170 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158aead5505379f7 server_cookie=e2cb2b0605c23c81 in_seq=0 out_seq=0 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf0b400700 10 client.18633 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf0b400700 1 -- 172.21.15.46:0/19653004 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562691bc7ec0 con 0x562691cefd80 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf0b400700 10 client.18633 handle_command_reply: tid=0 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 client.18633 shutdown 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 msgr2=0x562691c90360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c90360 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf1009ac20 tx=0x7faf10096cb0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 shutdown_connections 2024-05-08T23:14:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562691c85720 0x562691c80990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562691c7f890 0x562691c8fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562691cefd80 0x562691cf2170 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 --2- 172.21.15.46:0/19653004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562691c84e00 0x562691c90360 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf0b400700 0 client.18633 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:14:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.463+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 >> 172.21.15.46:0/19653004 conn(0x562691bdd210 msgr2=0x562691c83a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 shutdown_connections 2024-05-08T23:14:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf3fcb3b80 1 -- 172.21.15.46:0/19653004 wait complete. 2024-05-08T23:14:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf3fcb3b80 20 client.18633 trim_cache size 0 max 16384 2024-05-08T23:14:49.473 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:14:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:14:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:14:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "4711546a-5d5d-460d-aa5b-e3eb5ced928a": { 2024-05-08T23:14:49.473 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:14:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4711546a-5d5d-460d-aa5b-e3eb5ced928a", 2024-05-08T23:14:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:14:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:14:49.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:14:49.474 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:14:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 msgr2=0x7faf0c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 0x7faf0c0a37d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf1c004280 tx=0x7faf1c004b40 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 msgr2=0x7faf240737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faf10005950 tx=0x7faf10005740 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 shutdown_connections 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf24073d20 0x7faf241a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf0c0a1310 0x7faf0c0a37d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf24101080 0x7faf240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 --2- 172.21.15.46:0/2741390529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf241061a0 0x7faf240737e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.464+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 >> 172.21.15.46:0/2741390529 conn(0x7faf240fc8e0 msgr2=0x7faf240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.465+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 shutdown_connections 2024-05-08T23:14:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:14:49.465+0000 7faf2c200700 1 -- 172.21.15.46:0/2741390529 wait complete. 2024-05-08T23:14:49.483 INFO:tasks.cephfs.filesystem:scrub status for tag:4711546a-5d5d-460d-aa5b-e3eb5ced928a - {'path': '/', 'tag': '4711546a-5d5d-460d-aa5b-e3eb5ced928a', 'options': 'recursive,force'} 2024-05-08T23:14:49.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 961175 9.46 MB/sec execute 431 sec latency 22.978 ms 2024-05-08T23:14:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:49 smithi086 ceph-mon[78610]: pgmap v341: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:49 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:49 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:49 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:49 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:49 smithi053 ceph-mon[77587]: pgmap v341: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:49 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:49 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:49 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:49 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:49 smithi046 ceph-mon[36898]: pgmap v341: 129 pgs: 129 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:14:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:49 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:49 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:49 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:49 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:50.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 963060 9.46 MB/sec execute 432 sec latency 17.378 ms 2024-05-08T23:14:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:50 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:14:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:50 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:14:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:50 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:14:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:50 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:14:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:50 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:14:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:50 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:14:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:14:51.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 964484 9.46 MB/sec execute 433 sec latency 52.367 ms 2024-05-08T23:14:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:51 smithi086 ceph-mon[78610]: pgmap v342: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 185 op/s 2024-05-08T23:14:52.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:51 smithi053 ceph-mon[77587]: pgmap v342: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 185 op/s 2024-05-08T23:14:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:51 smithi046 ceph-mon[36898]: pgmap v342: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 185 op/s 2024-05-08T23:14:52.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 966516 9.46 MB/sec execute 434 sec latency 18.226 ms 2024-05-08T23:14:53.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 967599 9.45 MB/sec execute 435 sec latency 68.650 ms 2024-05-08T23:14:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:53 smithi086 ceph-mon[78610]: pgmap v343: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-05-08T23:14:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:53 smithi053 ceph-mon[77587]: pgmap v343: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-05-08T23:14:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:53 smithi046 ceph-mon[36898]: pgmap v343: 129 pgs: 129 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-05-08T23:14:54.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 969364 9.45 MB/sec execute 436 sec latency 30.974 ms 2024-05-08T23:14:55.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 970463 9.44 MB/sec execute 437 sec latency 21.545 ms 2024-05-08T23:14:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:55 smithi086 ceph-mon[78610]: pgmap v344: 129 pgs: 129 active+clean; 590 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-05-08T23:14:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:55 smithi053 ceph-mon[77587]: pgmap v344: 129 pgs: 129 active+clean; 590 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-05-08T23:14:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:55 smithi046 ceph-mon[36898]: pgmap v344: 129 pgs: 129 active+clean; 590 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-05-08T23:14:56.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 972314 9.44 MB/sec execute 438 sec latency 27.808 ms 2024-05-08T23:14:57.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 973629 9.44 MB/sec execute 439 sec latency 15.973 ms 2024-05-08T23:14:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:57 smithi086 ceph-mon[78610]: pgmap v345: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 156 op/s 2024-05-08T23:14:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:57 smithi053 ceph-mon[77587]: pgmap v345: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 156 op/s 2024-05-08T23:14:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:57 smithi046 ceph-mon[36898]: pgmap v345: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 156 op/s 2024-05-08T23:14:58.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 974288 9.42 MB/sec execute 440 sec latency 267.302 ms 2024-05-08T23:14:59.910 INFO:tasks.workunit.client.0.smithi046.stdout: 1 975609 9.42 MB/sec execute 441 sec latency 179.291 ms 2024-05-08T23:15:00.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:14:59 smithi086 ceph-mon[78610]: pgmap v346: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 141 op/s 2024-05-08T23:15:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:14:59 smithi053 ceph-mon[77587]: pgmap v346: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 141 op/s 2024-05-08T23:15:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:14:59 smithi046 ceph-mon[36898]: pgmap v346: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 141 op/s 2024-05-08T23:15:00.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 977119 9.42 MB/sec execute 442 sec latency 34.021 ms 2024-05-08T23:15:01.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:01.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 977315 9.40 MB/sec execute 443 sec latency 59.657 ms 2024-05-08T23:15:02.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:01 smithi086 ceph-mon[78610]: pgmap v347: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 253 op/s 2024-05-08T23:15:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:01 smithi053 ceph-mon[77587]: pgmap v347: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 253 op/s 2024-05-08T23:15:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:01 smithi046 ceph-mon[36898]: pgmap v347: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 253 op/s 2024-05-08T23:15:02.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 978723 9.39 MB/sec execute 444 sec latency 23.085 ms 2024-05-08T23:15:03.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 980655 9.40 MB/sec execute 445 sec latency 15.913 ms 2024-05-08T23:15:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:03 smithi086 ceph-mon[78610]: pgmap v348: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 194 op/s 2024-05-08T23:15:04.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:03 smithi053 ceph-mon[77587]: pgmap v348: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 194 op/s 2024-05-08T23:15:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:03 smithi046 ceph-mon[36898]: pgmap v348: 129 pgs: 129 active+clean; 622 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 194 op/s 2024-05-08T23:15:04.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 981942 9.39 MB/sec execute 446 sec latency 25.017 ms 2024-05-08T23:15:05.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 983946 9.39 MB/sec execute 447 sec latency 16.009 ms 2024-05-08T23:15:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:05 smithi086 ceph-mon[78610]: pgmap v349: 129 pgs: 129 active+clean; 597 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 302 op/s 2024-05-08T23:15:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:05 smithi053 ceph-mon[77587]: pgmap v349: 129 pgs: 129 active+clean; 597 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 302 op/s 2024-05-08T23:15:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:05 smithi046 ceph-mon[36898]: pgmap v349: 129 pgs: 129 active+clean; 597 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 302 op/s 2024-05-08T23:15:06.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 984850 9.38 MB/sec execute 448 sec latency 24.750 ms 2024-05-08T23:15:07.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 986875 9.39 MB/sec execute 449 sec latency 16.899 ms 2024-05-08T23:15:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:07 smithi086 ceph-mon[78610]: pgmap v350: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-05-08T23:15:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:07 smithi053 ceph-mon[77587]: pgmap v350: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-05-08T23:15:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:07 smithi046 ceph-mon[36898]: pgmap v350: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-05-08T23:15:08.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 988025 9.38 MB/sec execute 450 sec latency 16.419 ms 2024-05-08T23:15:09.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:08 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:09.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:08 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:09.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:08 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:09.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 989478 9.38 MB/sec execute 451 sec latency 51.844 ms 2024-05-08T23:15:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:09 smithi086 ceph-mon[78610]: pgmap v351: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 240 op/s 2024-05-08T23:15:10.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:09 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:10.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:09 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:10.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:09 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:15:10.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:09 smithi053 ceph-mon[77587]: pgmap v351: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 240 op/s 2024-05-08T23:15:10.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:09 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:10.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:09 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:10.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:09 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:15:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:09 smithi046 ceph-mon[36898]: pgmap v351: 129 pgs: 129 active+clean; 590 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 240 op/s 2024-05-08T23:15:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:09 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:09 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:09 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:15:10.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 991349 9.38 MB/sec execute 452 sec latency 20.487 ms 2024-05-08T23:15:11.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 992641 9.38 MB/sec execute 453 sec latency 45.623 ms 2024-05-08T23:15:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:11 smithi086 ceph-mon[78610]: pgmap v352: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.5 MiB/s wr, 299 op/s 2024-05-08T23:15:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:11 smithi086 ceph-mon[78610]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:15:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:11 smithi053 ceph-mon[77587]: pgmap v352: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.5 MiB/s wr, 299 op/s 2024-05-08T23:15:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:11 smithi053 ceph-mon[77587]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:15:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:11 smithi046 ceph-mon[36898]: pgmap v352: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.5 MiB/s wr, 299 op/s 2024-05-08T23:15:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:11 smithi046 ceph-mon[36898]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:15:12.911 INFO:tasks.workunit.client.0.smithi046.stdout: 1 994530 9.38 MB/sec execute 454 sec latency 27.497 ms 2024-05-08T23:15:13.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 995451 9.37 MB/sec execute 455 sec latency 36.415 ms 2024-05-08T23:15:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:13 smithi086 ceph-mon[78610]: pgmap v354: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-05-08T23:15:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:13 smithi053 ceph-mon[77587]: pgmap v354: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-05-08T23:15:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:13 smithi046 ceph-mon[36898]: pgmap v354: 129 pgs: 129 active+clean; 593 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-05-08T23:15:14.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 997492 9.37 MB/sec execute 456 sec latency 16.049 ms 2024-05-08T23:15:15.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 998664 9.37 MB/sec execute 457 sec latency 14.415 ms 2024-05-08T23:15:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:15 smithi086 ceph-mon[78610]: pgmap v355: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-08T23:15:16.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:15 smithi053 ceph-mon[77587]: pgmap v355: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-08T23:15:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:15 smithi046 ceph-mon[36898]: pgmap v355: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-05-08T23:15:16.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1000184 9.36 MB/sec execute 458 sec latency 32.828 ms 2024-05-08T23:15:17.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1002005 9.36 MB/sec execute 459 sec latency 18.557 ms 2024-05-08T23:15:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:17 smithi086 ceph-mon[78610]: pgmap v356: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:18.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:17 smithi053 ceph-mon[77587]: pgmap v356: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:17 smithi046 ceph-mon[36898]: pgmap v356: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:18.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1003383 9.36 MB/sec execute 460 sec latency 25.061 ms 2024-05-08T23:15:19.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:15:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.721+0000 7f4d6d000700 1 -- 172.21.15.46:0/568166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68073340 msgr2=0x7f4d68073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.721+0000 7f4d6d000700 1 --2- 172.21.15.46:0/568166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68073340 0x7f4d68073720 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 -- 172.21.15.46:0/568166715 shutdown_connections 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 --2- 172.21.15.46:0/568166715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68111df0 0x7f4d681141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 --2- 172.21.15.46:0/568166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68073340 0x7f4d68073720 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 --2- 172.21.15.46:0/568166715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073c60 0x7f4d681118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 -- 172.21.15.46:0/568166715 >> 172.21.15.46:0/568166715 conn(0x7f4d680fc8f0 msgr2=0x7f4d680fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 -- 172.21.15.46:0/568166715 shutdown_connections 2024-05-08T23:15:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.722+0000 7f4d6d000700 1 -- 172.21.15.46:0/568166715 wait complete. 2024-05-08T23:15:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 Processor -- start 2024-05-08T23:15:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 -- start start 2024-05-08T23:15:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073340 0x7f4d681a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68073c60 0x7f4d681a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6819de40 con 0x7f4d68073c60 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6819dfb0 con 0x7f4d68073340 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.723+0000 7f4d6d000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6819e170 con 0x7f4d68111df0 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53680/0 (socket says 172.21.15.46:53680) 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 -- 172.21.15.46:0/4134334308 learned_addr learned my addr 172.21.15.46:0/4134334308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073340 0x7f4d681a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d65400700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68073c60 0x7f4d681a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 -- 172.21.15.46:0/4134334308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073340 msgr2=0x7f4d681a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073340 0x7f4d681a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 -- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68073c60 msgr2=0x7f4d681a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68073c60 0x7f4d681a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.724+0000 7f4d66800700 1 -- 172.21.15.46:0/4134334308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580095f0 con 0x7f4d68111df0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d66800700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000ef90 tx=0x7f4d6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d5ea00700 1 -- 172.21.15.46:0/4134334308 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60018070 con 0x7f4d68111df0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d5ea00700 1 -- 172.21.15.46:0/4134334308 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d6000f040 con 0x7f4d68111df0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6819e3f0 con 0x7f4d68111df0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d5ea00700 1 -- 172.21.15.46:0/4134334308 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003ea0 con 0x7f4d68111df0 2024-05-08T23:15:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.725+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681aa690 con 0x7f4d68111df0 2024-05-08T23:15:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.729+0000 7f4d5ea00700 1 -- 172.21.15.46:0/4134334308 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4d6000ca50 con 0x7f4d68111df0 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.730+0000 7f4d5ea00700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 0x7f4d4c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.730+0000 7f4d80b3cb80 1 Processor -- start 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.730+0000 7f4d5ea00700 1 -- 172.21.15.46:0/4134334308 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f4d600cccc0 con 0x7f4d68111df0 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.730+0000 7f4d80b3cb80 1 -- start start 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.730+0000 7f4d65e00700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 0x7f4d4c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e518781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e51872350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e517c9150 con 0x563e51878720 2024-05-08T23:15:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e517c6b10 con 0x563e51877e00 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e517c2400 con 0x563e51872890 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53696/0 (socket says 172.21.15.46:53696) 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 -- 172.21.15.46:0/3180636161 learned_addr learned my addr 172.21.15.46:0/3180636161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d65e00700 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e518781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d65400700 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e51872350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d65e00700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 0x7f4d4c0a37d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800b2e0 tx=0x7f4d58005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 -- 172.21.15.46:0/3180636161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 msgr2=0x563e518781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e518781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 -- 172.21.15.46:0/3180636161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 msgr2=0x563e51872350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e51872350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.731+0000 7f4d66800700 1 -- 172.21.15.46:0/3180636161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e516433b0 con 0x563e51872890 2024-05-08T23:15:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.732+0000 7f4d66800700 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a1860 tx=0x7f4d600956e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be116591d825bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.732+0000 7f4d53400700 1 -- 172.21.15.46:0/3180636161 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004ad0 con 0x563e51872890 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.732+0000 7f4d53400700 1 -- 172.21.15.46:0/3180636161 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d6009a0a0 con 0x563e51872890 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.732+0000 7f4d53400700 1 -- 172.21.15.46:0/3180636161 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6005ec60 con 0x563e51872890 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/3180636161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 msgr2=0x563e51872cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a1860 tx=0x7f4d600956e0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/3180636161 shutdown_connections 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872cd0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e518781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/3180636161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e51872350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/3180636161 >> 172.21.15.46:0/3180636161 conn(0x563e517d0210 msgr2=0x563e5186cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/3180636161 shutdown_connections 2024-05-08T23:15:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.733+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/3180636161 wait complete. 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 -- start start 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 Processor -- start 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e5187bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e5187c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5175e560 con 0x563e51878720 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5175e200 con 0x563e51872890 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.734+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e516456a0 con 0x563e51877e00 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d80b3cb80 10 client.0 _ll_register_callbacks cb 0x563e51767b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33934/0 (socket says 172.21.15.46:33934) 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 -- 172.21.15.46:0/2470282041 learned_addr learned my addr 172.21.15.46:0/2470282041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d66800700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e5187c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65400700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e5187bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 -- 172.21.15.46:0/2470282041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 msgr2=0x563e5187bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e5187bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 -- 172.21.15.46:0/2470282041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 msgr2=0x563e5187c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e5187c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d65e00700 1 -- 172.21.15.46:0/2470282041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681aa690 con 0x563e51872890 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.735+0000 7f4d66800700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e5187c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e5f1b51927a5384 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d47e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58004bc0 con 0x563e51872890 2024-05-08T23:15:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e5171b270 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f4d58004d20 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d5800f460 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d80b3cb80 10 client.28652 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.736+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563e515efb60 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e5171b270 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4d580305e0 con 0x563e51872890 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 10 client.28652 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 10 client.28652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 20 client.28652 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e5188c1b0 0x563e5188e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 4 client.28652 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e51841810 con 0x563e5188c1b0 2024-05-08T23:15:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.737+0000 7f4d65400700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e5188c1b0 0x563e5188e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.738+0000 7f4d65400700 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e5188c1b0 0x563e5188e5d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3499df2ff1eccee server_cookie=bfb987bcdc19c57c in_seq=0 out_seq=0 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.738+0000 7f4d47e00700 10 client.28652 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d47e00700 1 -- 172.21.15.46:0/2470282041 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e51841810 con 0x563e5188c1b0 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d47e00700 10 client.28652 handle_command_reply: tid=0 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 client.28652 shutdown 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 msgr2=0x563e5187e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 shutdown_connections 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e5188c1b0 0x563e5188e5d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e5187c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d47e00700 0 client.28652 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e5187e7c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2470282041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e5187bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 >> 172.21.15.46:0/2470282041 conn(0x563e517d0210 msgr2=0x563e5186c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 shutdown_connections 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2470282041 wait complete. 2024-05-08T23:15:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.739+0000 7f4d80b3cb80 20 client.28652 trim_cache size 0 max 16384 2024-05-08T23:15:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.764+0000 7f4d80b3cb80 1 Processor -- start 2024-05-08T23:15:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.764+0000 7f4d80b3cb80 1 -- start start 2024-05-08T23:15:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e5186cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e51878590 con 0x563e51878720 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5186dac0 con 0x563e51877e00 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e51865040 con 0x563e51872890 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e5186cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53716/0 (socket says 172.21.15.46:53716) 2024-05-08T23:15:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 -- 172.21.15.46:0/2557117745 learned_addr learned my addr 172.21.15.46:0/2557117745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d66800700 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 -- 172.21.15.46:0/2557117745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 msgr2=0x563e5186cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e5186cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 -- 172.21.15.46:0/2557117745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 msgr2=0x563e518831c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518831c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65e00700 1 -- 172.21.15.46:0/2557117745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e51788560 con 0x563e51872890 2024-05-08T23:15:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.765+0000 7f4d65400700 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e5186cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.766+0000 7f4d65e00700 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c000 tx=0x7f4d58005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b598b33fa36b43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.766+0000 7f4d53e00700 1 -- 172.21.15.46:0/2557117745 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58030ea0 con 0x563e51872890 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.766+0000 7f4d53e00700 1 -- 172.21.15.46:0/2557117745 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5801e040 con 0x563e51872890 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.766+0000 7f4d53e00700 1 -- 172.21.15.46:0/2557117745 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800f750 con 0x563e51872890 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2557117745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 msgr2=0x563e51872350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c000 tx=0x7f4d58005480 comp rx=0 tx=0).stop 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2557117745 shutdown_connections 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51872890 0x563e51872350 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/2557117745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51877e00 0x563e5186cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2557117745 >> 172.21.15.46:0/2557117745 conn(0x563e517d0210 msgr2=0x563e51887c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2557117745 shutdown_connections 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.767+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/2557117745 wait complete. 2024-05-08T23:15:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.768+0000 7f4d80b3cb80 1 -- start start 2024-05-08T23:15:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.768+0000 7f4d80b3cb80 1 Processor -- start 2024-05-08T23:15:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e51874a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e518739a0 con 0x563e51878720 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e51873b10 con 0x563e51872890 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e51873cd0 con 0x563e51877e00 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d80b3cb80 10 client.0 _ll_register_callbacks cb 0x563e51767b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33950/0 (socket says 172.21.15.46:33950) 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e51874a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 -- 172.21.15.46:0/622133044 learned_addr learned my addr 172.21.15.46:0/622133044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d66800700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 -- 172.21.15.46:0/622133044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 msgr2=0x563e51874a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e51874a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 -- 172.21.15.46:0/622133044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 msgr2=0x563e518751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65e00700 1 -- 172.21.15.46:0/622133044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d54009df0 con 0x563e51872890 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.769+0000 7f4d65400700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e51874a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d65e00700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005f50 tx=0x7f4d5800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dffed21825406ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d47e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58030ea0 con 0x563e51872890 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d6009a0a0 con 0x563e51872890 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f4d58030620 con 0x563e51872890 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d5800f460 con 0x563e51872890 2024-05-08T23:15:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d80b3cb80 10 client.28658 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.770+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563e515efb60 con 0x563e51872890 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d6009a0a0 con 0x563e51872890 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4d58038070 con 0x563e51872890 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 10 client.28658 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 10 client.28658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 20 client.28658 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e518e2bb0 0x563e518e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 4 client.28658 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e517baec0 con 0x563e518e2bb0 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.771+0000 7f4d65400700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e518e2bb0 0x563e518e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d47e00700 10 client.28658 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d65400700 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e518e2bb0 0x563e518e4fa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36a253dcdc09536 server_cookie=e74f028a4b27194a in_seq=0 out_seq=0 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d47e00700 1 -- 172.21.15.46:0/622133044 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563e517baec0 con 0x563e518e2bb0 2024-05-08T23:15:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d47e00700 10 client.28658 handle_command_reply: tid=0 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 client.28658 shutdown 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 msgr2=0x563e51883720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005f50 tx=0x7f4d5800b410 comp rx=0 tx=0).stop 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 shutdown_connections 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563e518e2bb0 0x563e518e4fa0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e51878720 0x563e518751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e51872890 0x563e51883720 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 --2- 172.21.15.46:0/622133044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e51877e00 0x563e51874a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d47e00700 0 client.28658 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.772+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 >> 172.21.15.46:0/622133044 conn(0x563e517d0210 msgr2=0x563e5187a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 shutdown_connections 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d80b3cb80 1 -- 172.21.15.46:0/622133044 wait complete. 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d80b3cb80 20 client.28658 trim_cache size 0 max 16384 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:15:19.778 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 msgr2=0x7f4d4c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 0x7f4d4c0a37d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800b2e0 tx=0x7f4d58005fd0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 msgr2=0x7f4d6819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000ef90 tx=0x7f4d6000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 shutdown_connections 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d68111df0 0x7f4d6819d7d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d68073340 0x7f4d681a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4d4c0a1310 0x7f4d4c0a37d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 --2- 172.21.15.46:0/4134334308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d68073c60 0x7f4d681a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.773+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 >> 172.21.15.46:0/4134334308 conn(0x7f4d680fc8f0 msgr2=0x7f4d680fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.774+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 shutdown_connections 2024-05-08T23:15:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:19.774+0000 7f4d6d000700 1 -- 172.21.15.46:0/4134334308 wait complete. 2024-05-08T23:15:19.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:15:19.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:15:19.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1005403 9.36 MB/sec execute 461 sec latency 16.343 ms 2024-05-08T23:15:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 -- 172.21.15.46:0/3371894274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3e0 msgr2=0x7f830810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 --2- 172.21.15.46:0/3371894274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3e0 0x7f830810d8a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f830000b0a0 tx=0x7f830000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 -- 172.21.15.46:0/3371894274 shutdown_connections 2024-05-08T23:15:20.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 --2- 172.21.15.46:0/3371894274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3e0 0x7f830810d8a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 --2- 172.21.15.46:0/3371894274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fc0 0x7f83081023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 --2- 172.21.15.46:0/3371894274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f830810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 -- 172.21.15.46:0/3371894274 >> 172.21.15.46:0/3371894274 conn(0x7f83080fb820 msgr2=0x7f83080fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.019+0000 7f830fa00700 1 -- 172.21.15.46:0/3371894274 shutdown_connections 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.020+0000 7f830fa00700 1 -- 172.21.15.46:0/3371894274 wait complete. 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.021+0000 7f830fa00700 1 Processor -- start 2024-05-08T23:15:20.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.021+0000 7f830fa00700 1 -- start start 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101fc0 0x7f83080ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830810b3e0 0x7f8308100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a57f0 con 0x7f830810b3e0 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a5960 con 0x7f83081028e0 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a5b20 con 0x7f8308101fc0 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33968/0 (socket says 172.21.15.46:33968) 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 -- 172.21.15.46:0/1743080532 learned_addr learned my addr 172.21.15.46:0/1743080532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f830d000700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101fc0 0x7f83080ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 -- 172.21.15.46:0/1743080532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101fc0 msgr2=0x7f83080ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101fc0 0x7f83080ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 -- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830810b3e0 msgr2=0x7f8308100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.022+0000 7f8307e00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830810b3e0 0x7f8308100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f8307e00700 1 -- 172.21.15.46:0/1743080532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009d00 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f8307e00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e9f0 tx=0x7f82f400edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400cbd0 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081a5e00 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f400cd30 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4005650 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.023+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a6230 con 0x7f83081028e0 2024-05-08T23:15:20.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.024+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83081064e0 con 0x7f83081028e0 2024-05-08T23:15:20.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.027+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82f4004750 con 0x7f83081028e0 2024-05-08T23:15:20.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.028+0000 7f8305600700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.029+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f82f4014070 con 0x7f83081028e0 2024-05-08T23:15:20.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.029+0000 7f830d000700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.029+0000 7f830d000700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 0x7f82f00a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005fd0 tx=0x7f82fc005f60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.030+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82f4096910 con 0x7f83081028e0 2024-05-08T23:15:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:19 smithi086 ceph-mon[78610]: pgmap v357: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:20.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:19 smithi086 ceph-mon[78610]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:15:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:19 smithi053 ceph-mon[77587]: pgmap v357: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:20.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:19 smithi053 ceph-mon[77587]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:15:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:19 smithi046 ceph-mon[36898]: pgmap v357: 129 pgs: 129 active+clean; 576 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-05-08T23:15:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:19 smithi046 ceph-mon[36898]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:15:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.325+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8308068a40 con 0x7f83081028e0 2024-05-08T23:15:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.326+0000 7f8305600700 1 -- 172.21.15.46:0/1743080532 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f82f4099f30 con 0x7f83081028e0 2024-05-08T23:15:20.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:15:20.327 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:15:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 msgr2=0x7f82f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 0x7f82f00a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005fd0 tx=0x7f82fc005f60 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 msgr2=0x7f8308100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e9f0 tx=0x7f82f400edb0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 shutdown_connections 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830810b3e0 0x7f8308100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8308101fc0 0x7f83080ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 --2- 172.21.15.46:0/1743080532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028e0 0x7f8308100150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 >> 172.21.15.46:0/1743080532 conn(0x7f83080fb820 msgr2=0x7f83080fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 shutdown_connections 2024-05-08T23:15:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.329+0000 7f830fa00700 1 -- 172.21.15.46:0/1743080532 wait complete. 2024-05-08T23:15:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:15:20.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:15:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 -- 172.21.15.46:0/3572542708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101a30 msgr2=0x7f2384105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 --2- 172.21.15.46:0/3572542708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101a30 0x7f2384105a70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2370009990 tx=0x7f2370009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 -- 172.21.15.46:0/3572542708 shutdown_connections 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 --2- 172.21.15.46:0/3572542708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23841061a0 0x7f238410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 --2- 172.21.15.46:0/3572542708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2384101080 0x7f2384101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 --2- 172.21.15.46:0/3572542708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101a30 0x7f2384105a70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 -- 172.21.15.46:0/3572542708 >> 172.21.15.46:0/3572542708 conn(0x7f23840fc8e0 msgr2=0x7f23840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 -- 172.21.15.46:0/3572542708 shutdown_connections 2024-05-08T23:15:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.572+0000 7f2389e00700 1 -- 172.21.15.46:0/3572542708 wait complete. 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.573+0000 7f2389e00700 1 Processor -- start 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.573+0000 7f2389e00700 1 -- start start 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23841061a0 0x7f2384199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 0x7f238419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238419e890 con 0x7f238419de10 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238419ea00 con 0x7f23841061a0 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2389e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238419ebc0 con 0x7f2384101080 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53756/0 (socket says 172.21.15.46:53756) 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 -- 172.21.15.46:0/1384309313 learned_addr learned my addr 172.21.15.46:0/1384309313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2383400700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 0x7f238419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382000700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23841061a0 0x7f2384199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 -- 172.21.15.46:0/1384309313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23841061a0 msgr2=0x7f2384199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23841061a0 0x7f2384199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 -- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 msgr2=0x7f238419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 0x7f238419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2382a00700 1 -- 172.21.15.46:0/1384309313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700095f0 con 0x7f2384101080 2024-05-08T23:15:20.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.574+0000 7f2383400700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 0x7f238419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2382a00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f237400e9a0 tx=0x7f237400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2388a00700 1 -- 172.21.15.46:0/1384309313 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374009790 con 0x7f2384101080 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23841a6120 con 0x7f2384101080 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2388a00700 1 -- 172.21.15.46:0/1384309313 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2374010430 con 0x7f2384101080 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2388a00700 1 -- 172.21.15.46:0/1384309313 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374018680 con 0x7f2384101080 2024-05-08T23:15:20.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.575+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841a6600 con 0x7f2384101080 2024-05-08T23:15:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.578+0000 7f2388a00700 1 -- 172.21.15.46:0/1384309313 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f23740105a0 con 0x7f2384101080 2024-05-08T23:15:20.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f2388a00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 0x7f23680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 Processor -- start 2024-05-08T23:15:20.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f2388a00700 1 -- 172.21.15.46:0/1384309313 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2374014070 con 0x7f2384101080 2024-05-08T23:15:20.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 -- start start 2024-05-08T23:15:20.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f2382000700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 0x7f23680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a802150 con 0x56314a8ab890 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a7ffb10 con 0x56314a8b0e00 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.581+0000 7f239da05b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a7fb400 con 0x56314a8b1720 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34000/0 (socket says 172.21.15.46:34000) 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382000700 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 -- 172.21.15.46:0/4280107666 learned_addr learned my addr 172.21.15.46:0/4280107666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2383400700 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382000700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 0x7f23680a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f238419aa80 tx=0x7f237000b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 -- 172.21.15.46:0/4280107666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 msgr2=0x56314a8ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 -- 172.21.15.46:0/4280107666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 msgr2=0x56314a8abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 -- 172.21.15.46:0/4280107666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314a67c3b0 con 0x56314a8b0e00 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f2382a00700 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2374096a40 tx=0x7f2374099c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a50131d9525e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.582+0000 7f236fe00700 1 -- 172.21.15.46:0/4280107666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740adea0 con 0x56314a8b0e00 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f236fe00700 1 -- 172.21.15.46:0/4280107666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237409de00 con 0x56314a8b0e00 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f236fe00700 1 -- 172.21.15.46:0/4280107666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237409a230 con 0x56314a8b0e00 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 -- 172.21.15.46:0/4280107666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 msgr2=0x56314a8b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2374096a40 tx=0x7f2374099c40 comp rx=0 tx=0).stop 2024-05-08T23:15:20.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 -- 172.21.15.46:0/4280107666 shutdown_connections 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b11e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 --2- 172.21.15.46:0/4280107666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.583+0000 7f239da05b80 1 -- 172.21.15.46:0/4280107666 >> 172.21.15.46:0/4280107666 conn(0x56314a809210 msgr2=0x56314a8a5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.584+0000 7f239da05b80 1 -- 172.21.15.46:0/4280107666 shutdown_connections 2024-05-08T23:15:20.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.584+0000 7f239da05b80 1 -- 172.21.15.46:0/4280107666 wait complete. 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.584+0000 7f239da05b80 1 -- start start 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 Processor -- start 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a797560 con 0x56314a8ab890 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a797200 con 0x56314a8b0e00 2024-05-08T23:15:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a67e6a0 con 0x56314a8b1720 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.585+0000 7f239da05b80 10 client.0 _ll_register_callbacks cb 0x56314a7a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53784/0 (socket says 172.21.15.46:53784) 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 -- 172.21.15.46:0/1949220467 learned_addr learned my addr 172.21.15.46:0/1949220467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2382000700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2382a00700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 -- 172.21.15.46:0/1949220467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 msgr2=0x56314a8b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 -- 172.21.15.46:0/1949220467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 msgr2=0x56314a8b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 -- 172.21.15.46:0/1949220467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841a6600 con 0x56314a8b1720 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f2383400700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f237c0046e0 tx=0x7f237c0049f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1ea458eef6c3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f236cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:20.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.586+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c00eb00 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56314a754270 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f237c00ec60 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237c00ee20 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 10 client.28768 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56314a628b60 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56314a754270 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f237c01c020 con 0x56314a8b1720 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 10 client.28768 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 10 client.28768 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 20 client.28768 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a8c5160 0x56314a8c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 4 client.28768 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.587+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56314a87a810 con 0x56314a8c5160 2024-05-08T23:15:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.588+0000 7f2382a00700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a8c5160 0x56314a8c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.588+0000 7f2382a00700 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a8c5160 0x56314a8c7580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c740c545bdc95d4 server_cookie=10e8616dcd07c6cb in_seq=0 out_seq=0 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.588+0000 7f236cc00700 10 client.28768 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f236cc00700 1 -- 172.21.15.46:0/1949220467 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56314a87a810 con 0x56314a8c5160 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f236cc00700 10 client.28768 handle_command_reply: tid=0 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 client.28768 shutdown 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 msgr2=0x56314a8b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f237c0046e0 tx=0x7f237c0049f0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 shutdown_connections 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a8c5160 0x56314a8c7580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b1720 0x56314a8b55e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8ab890 0x56314a8b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f236cc00700 0 client.28768 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:20.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 --2- 172.21.15.46:0/1949220467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 >> 172.21.15.46:0/1949220467 conn(0x56314a809210 msgr2=0x56314a8a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 shutdown_connections 2024-05-08T23:15:20.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.589+0000 7f239da05b80 1 -- 172.21.15.46:0/1949220467 wait complete. 2024-05-08T23:15:20.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.590+0000 7f239da05b80 20 client.28768 trim_cache size 0 max 16384 2024-05-08T23:15:20.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.603+0000 7f239da05b80 1 Processor -- start 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.603+0000 7f239da05b80 1 -- start start 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a8a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a8a0800 con 0x56314a8b1720 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a8b2b10 con 0x56314a8b0e00 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f239da05b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a8b08d0 con 0x56314a8ab890 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f2382a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f2382a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53788/0 (socket says 172.21.15.46:53788) 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f2382a00700 1 -- 172.21.15.46:0/3093739926 learned_addr learned my addr 172.21.15.46:0/3093739926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f2382000700 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.604+0000 7f2383400700 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a8a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 -- 172.21.15.46:0/3093739926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 msgr2=0x56314a8b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 -- 172.21.15.46:0/3093739926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 msgr2=0x56314a8a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a8a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 -- 172.21.15.46:0/3093739926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314a8abf40 con 0x56314a8ab890 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2382a00700 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2374009230 tx=0x7f2374010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b675c6ce82c9eb8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2379200700 1 -- 172.21.15.46:0/3093739926 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740adea0 con 0x56314a8ab890 2024-05-08T23:15:20.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2379200700 1 -- 172.21.15.46:0/3093739926 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23740a2450 con 0x56314a8ab890 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.605+0000 7f2379200700 1 -- 172.21.15.46:0/3093739926 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237409a490 con 0x56314a8ab890 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 -- 172.21.15.46:0/3093739926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 msgr2=0x56314a8ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2374009230 tx=0x7f2374010c80 comp rx=0 tx=0).stop 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 -- 172.21.15.46:0/3093739926 shutdown_connections 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a8a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8ab890 0x56314a8ab350 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 --2- 172.21.15.46:0/3093739926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8b0e00 0x56314a8b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 -- 172.21.15.46:0/3093739926 >> 172.21.15.46:0/3093739926 conn(0x56314a809210 msgr2=0x56314a8afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 -- 172.21.15.46:0/3093739926 shutdown_connections 2024-05-08T23:15:20.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.606+0000 7f239da05b80 1 -- 172.21.15.46:0/3093739926 wait complete. 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.607+0000 7f239da05b80 1 -- start start 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.607+0000 7f239da05b80 1 Processor -- start 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b0e00 0x56314a8ba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a91a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a91ab80 con 0x56314a8b1720 2024-05-08T23:15:20.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a91acc0 con 0x56314a8ab890 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314a91ae30 con 0x56314a8b0e00 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f239da05b80 10 client.0 _ll_register_callbacks cb 0x56314a7a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b0e00 0x56314a8ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34026/0 (socket says 172.21.15.46:34026) 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 -- 172.21.15.46:0/512794242 learned_addr learned my addr 172.21.15.46:0/512794242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2383400700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a91a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 -- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b0e00 msgr2=0x56314a8ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b0e00 0x56314a8ba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 -- 172.21.15.46:0/512794242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 msgr2=0x56314a91a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.608+0000 7f2382a00700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a91a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f2382a00700 1 -- 172.21.15.46:0/512794242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56314a67c3b0 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f2383400700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a91a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f2382a00700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23740a1f20 tx=0x7f237409a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee351e4e09e07306 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f236cc00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374094be0 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2374094d40 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f237409de00 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237409a230 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f239da05b80 10 client.28682 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56314a7c1560 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.609+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237409de00 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23740a1fc0 con 0x56314a8ab890 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 10 client.28682 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 10 client.28682 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 20 client.28682 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a91c150 0x56314a91e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 4 client.28682 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56314a7f3ec0 con 0x56314a91c150 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.610+0000 7f2382000700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a91c150 0x56314a91e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f236cc00700 10 client.28682 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f2382000700 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a91c150 0x56314a91e540 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73da5fe13c31dac4 server_cookie=d934e0c6d8523dd0 in_seq=0 out_seq=0 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f236cc00700 1 -- 172.21.15.46:0/512794242 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56314a7f3ec0 con 0x56314a91c150 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f236cc00700 10 client.28682 handle_command_reply: tid=0 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 client.28682 shutdown 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 msgr2=0x56314a8ba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23740a1f20 tx=0x7f237409a750 comp rx=0 tx=0).stop 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 shutdown_connections 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x56314a91c150 0x56314a91e540 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56314a8b1720 0x56314a91a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f236cc00700 0 client.28682 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56314a8ab890 0x56314a8ba240 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 --2- 172.21.15.46:0/512794242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56314a8b0e00 0x56314a8ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 >> 172.21.15.46:0/512794242 conn(0x56314a809210 msgr2=0x56314a8b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 shutdown_connections 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 1 -- 172.21.15.46:0/512794242 wait complete. 2024-05-08T23:15:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.611+0000 7f239da05b80 20 client.28682 trim_cache size 0 max 16384 2024-05-08T23:15:20.618 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:20.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 msgr2=0x7f23680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 0x7f23680a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f238419aa80 tx=0x7f237000b260 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 msgr2=0x7f2384199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f237400e9a0 tx=0x7f237400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 shutdown_connections 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f23680a1290 0x7f23680a3750 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238419de10 0x7f238419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2384101080 0x7f2384199580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 --2- 172.21.15.46:0/1384309313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23841061a0 0x7f2384199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 >> 172.21.15.46:0/1384309313 conn(0x7f23840fc8e0 msgr2=0x7f23840fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 shutdown_connections 2024-05-08T23:15:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.612+0000 7f2389e00700 1 -- 172.21.15.46:0/1384309313 wait complete. 2024-05-08T23:15:20.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:15:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.871+0000 7fa1bc000700 1 -- 172.21.15.46:0/4205169772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b4101fa0 msgr2=0x7fa1b4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.871+0000 7fa1bc000700 1 --2- 172.21.15.46:0/4205169772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b4101fa0 0x7fa1b4102380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009990 tx=0x7fa1a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 -- 172.21.15.46:0/4205169772 shutdown_connections 2024-05-08T23:15:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 --2- 172.21.15.46:0/4205169772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b410b3c0 0x7fa1b410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 --2- 172.21.15.46:0/4205169772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b4101fa0 0x7fa1b4102380 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 --2- 172.21.15.46:0/4205169772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 0x7fa1b410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 -- 172.21.15.46:0/4205169772 >> 172.21.15.46:0/4205169772 conn(0x7fa1b40fb760 msgr2=0x7fa1b40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 -- 172.21.15.46:0/4205169772 shutdown_connections 2024-05-08T23:15:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.872+0000 7fa1bc000700 1 -- 172.21.15.46:0/4205169772 wait complete. 2024-05-08T23:15:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.873+0000 7fa1bc000700 1 Processor -- start 2024-05-08T23:15:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.873+0000 7fa1bc000700 1 -- start start 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 0x7fa1b4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 0x7fa1b419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e3e0 con 0x7fa1b41028c0 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e550 con 0x7fa1b4101fa0 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1bc000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b419e710 con 0x7fa1b410b3c0 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34044/0 (socket says 172.21.15.46:34044) 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1b9600700 1 -- 172.21.15.46:0/2555631065 learned_addr learned my addr 172.21.15.46:0/2555631065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 0x7fa1b4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1ba000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 0x7fa1b419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.874+0000 7fa1b9600700 1 -- 172.21.15.46:0/2555631065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 msgr2=0x7fa1b419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b9600700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 0x7fa1b419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b9600700 1 -- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 msgr2=0x7fa1b4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b9600700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 0x7fa1b4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b9600700 1 -- 172.21.15.46:0/2555631065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80095f0 con 0x7fa1b4101fa0 2024-05-08T23:15:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1ba000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 0x7fa1b419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b9600700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005770 tx=0x7fa1a80057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b2000700 1 -- 172.21.15.46:0/2555631065 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801d070 con 0x7fa1b4101fa0 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b419e990 con 0x7fa1b4101fa0 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b2000700 1 -- 172.21.15.46:0/2555631065 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a8021410 con 0x7fa1b4101fa0 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1b2000700 1 -- 172.21.15.46:0/2555631065 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8017650 con 0x7fa1b4101fa0 2024-05-08T23:15:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.875+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b41a6230 con 0x7fa1b4101fa0 2024-05-08T23:15:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1b2000700 1 -- 172.21.15.46:0/2555631065 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa1a800fb40 con 0x7fa1b4101fa0 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 Processor -- start 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 -- start start 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 0x55a4279541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a42794ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4278a5150 con 0x55a42794e890 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4278a2b10 con 0x55a427953e00 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42789e400 con 0x55a427954720 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.880+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a42794ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1ba000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 0x55a4279541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53822/0 (socket says 172.21.15.46:53822) 2024-05-08T23:15:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 -- 172.21.15.46:0/1184799714 learned_addr learned my addr 172.21.15.46:0/1184799714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 -- 172.21.15.46:0/1184799714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 msgr2=0x55a4279541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 0x55a4279541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 -- 172.21.15.46:0/1184799714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 msgr2=0x55a42794ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a42794ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 -- 172.21.15.46:0/1184799714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42771f3b0 con 0x55a427954720 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1ba000700 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 0x55a4279541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1b9600700 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cb700 tx=0x7fa1a80d4730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d38675c2f93b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1b2000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 0x7fa19c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.881+0000 7fa1a6600700 1 -- 172.21.15.46:0/1184799714 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80ceca0 con 0x55a427954720 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1a6600700 1 -- 172.21.15.46:0/1184799714 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a8021e90 con 0x55a427954720 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1a6600700 1 -- 172.21.15.46:0/1184799714 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8097920 con 0x55a427954720 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1b2000700 1 -- 172.21.15.46:0/2555631065 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa1a80ce4e0 con 0x7fa1b4101fa0 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1ba000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 0x7fa19c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1184799714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 msgr2=0x55a42794e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.882+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cb700 tx=0x7fa1a80d4730 comp rx=0 tx=0).stop 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1ba000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 0x7fa19c0a3750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009550 tx=0x7fa1ac009100 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1184799714 shutdown_connections 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a42794ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427953e00 0x55a4279541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1184799714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a42794e350 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1184799714 >> 172.21.15.46:0/1184799714 conn(0x55a4278ac210 msgr2=0x55a427948df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1184799714 shutdown_connections 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1184799714 wait complete. 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 -- start start 2024-05-08T23:15:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.883+0000 7fa1cfa1db80 1 Processor -- start 2024-05-08T23:15:20.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a4279580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42783a560 con 0x55a427953e00 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42783a200 con 0x55a427954720 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4277216a0 con 0x55a42794e890 2024-05-08T23:15:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1cfa1db80 10 client.0 _ll_register_callbacks cb 0x55a427843b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53828/0 (socket says 172.21.15.46:53828) 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1b9600700 1 -- 172.21.15.46:0/262728694 learned_addr learned my addr 172.21.15.46:0/262728694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1ba000700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a4279580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.884+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 -- 172.21.15.46:0/262728694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 msgr2=0x55a4279580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a4279580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 -- 172.21.15.46:0/262728694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 msgr2=0x55a4279579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 -- 172.21.15.46:0/262728694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac009d10 con 0x55a42794e890 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1b9600700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098e0 tx=0x7fa1a8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb63beac35a04ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa19aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:20.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8097550 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa1a809f040 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4277f7270 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a809afe0 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1cfa1db80 10 client.28786 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa1b41a6230 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.885+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4277f7270 con 0x55a42794e890 2024-05-08T23:15:20.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1a80ce160 con 0x55a42794e890 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 10 client.28786 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 10 client.28786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 20 client.28786 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279678e0 0x55a427969cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 4 client.28786 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42791d810 con 0x55a4279678e0 2024-05-08T23:15:20.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279678e0 0x55a427969cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279678e0 0x55a427969cd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4753163a229fe26 server_cookie=752931c9f26482d0 in_seq=0 out_seq=0 2024-05-08T23:15:20.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.886+0000 7fa19aa00700 10 client.28786 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:20.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.887+0000 7fa19aa00700 1 -- 172.21.15.46:0/262728694 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a42791d810 con 0x55a4279678e0 2024-05-08T23:15:20.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.887+0000 7fa19aa00700 10 client.28786 handle_command_reply: tid=0 2024-05-08T23:15:20.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.887+0000 7fa1cfa1db80 1 client.28786 shutdown 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.887+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 msgr2=0x55a42795a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.887+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098e0 tx=0x7fa1a8005ef0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 shutdown_connections 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279678e0 0x55a427969cd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a4279580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a42794e890 0x55a42795a730 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa19aa00700 0 client.28786 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:20.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/262728694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 >> 172.21.15.46:0/262728694 conn(0x55a4278ac210 msgr2=0x55a4279485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 shutdown_connections 2024-05-08T23:15:20.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/262728694 wait complete. 2024-05-08T23:15:20.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.888+0000 7fa1cfa1db80 20 client.28786 trim_cache size 0 max 16384 2024-05-08T23:15:20.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.905+0000 7fa1cfa1db80 1 Processor -- start 2024-05-08T23:15:20.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.905+0000 7fa1cfa1db80 1 -- start start 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a427948d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a42795a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42794a020 con 0x55a42794e890 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a427943b40 con 0x55a427954720 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42796a330 con 0x55a427953e00 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a427948d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53838/0 (socket says 172.21.15.46:53838) 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 -- 172.21.15.46:0/1935475007 learned_addr learned my addr 172.21.15.46:0/1935475007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1ba000700 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a42795a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 -- 172.21.15.46:0/1935475007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 msgr2=0x55a42795a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a42795a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 -- 172.21.15.46:0/1935475007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 msgr2=0x55a427948d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a427948d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.906+0000 7fa1b8c00700 1 -- 172.21.15.46:0/1935475007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42794ef40 con 0x55a427953e00 2024-05-08T23:15:20.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.907+0000 7fa1b8c00700 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x55a427958ff0 tx=0x7fa1a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219ebe2a73331bec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.907+0000 7fa1a7000700 1 -- 172.21.15.46:0/1935475007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0010df0 con 0x55a427953e00 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.907+0000 7fa1a7000700 1 -- 172.21.15.46:0/1935475007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a0023070 con 0x55a427953e00 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.907+0000 7fa1a7000700 1 -- 172.21.15.46:0/1935475007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005280 con 0x55a427953e00 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1935475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 msgr2=0x55a4279531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x55a427958ff0 tx=0x7fa1a0009260 comp rx=0 tx=0).stop 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1935475007 shutdown_connections 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a427954720 0x55a42795a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a42794e890 0x55a427948d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/1935475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427953e00 0x55a4279531b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1935475007 >> 172.21.15.46:0/1935475007 conn(0x55a4278ac210 msgr2=0x55a427952aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1935475007 shutdown_connections 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.908+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/1935475007 wait complete. 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 -- start start 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 Processor -- start 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 0x55a42795f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a427950ce0 con 0x55a427953e00 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a427950e50 con 0x55a42794e890 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4279c6580 con 0x55a427954720 2024-05-08T23:15:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.909+0000 7fa1cfa1db80 10 client.0 _ll_register_callbacks cb 0x55a427843b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 0x55a42795f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1b8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53852/0 (socket says 172.21.15.46:53852) 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 -- 172.21.15.46:0/4212046892 learned_addr learned my addr 172.21.15.46:0/4212046892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1b9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 0x55a42795f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34090/0 (socket says 172.21.15.46:34090) 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 -- 172.21.15.46:0/4212046892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 msgr2=0x55a42795f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 0x55a42795f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 -- 172.21.15.46:0/4212046892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 msgr2=0x55a4279500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 -- 172.21.15.46:0/4212046892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a0009ab0 con 0x55a427954720 2024-05-08T23:15:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa1ba000700 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac004a00 tx=0x7fa1ac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38a4827a8e8dddd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.910+0000 7fa19aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac016da0 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1a8021e90 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa1ac01d070 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1ac026470 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 10 client.28792 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a4276cbb60 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a8021e90 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1ac025670 con 0x55a427954720 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 10 client.28792 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 10 client.28792 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:15:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 20 client.28792 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:20.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.911+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279c7cf0 0x55a4279ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:20.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa1cfa1db80 4 client.28792 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:20.917 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1006326 9.35 MB/sec execute 462 sec latency 20.615 ms 2024-05-08T23:15:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a427896ec0 con 0x55a4279c7cf0 2024-05-08T23:15:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa1b9600700 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279c7cf0 0x55a4279ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa1b9600700 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279c7cf0 0x55a4279ca110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2aca2ab3d2830aa server_cookie=5f1720cad278c3d1 in_seq=0 out_seq=0 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa19aa00700 10 client.28792 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa19aa00700 1 -- 172.21.15.46:0/4212046892 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a427896ec0 con 0x55a4279c7cf0 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa19aa00700 10 client.28792 handle_command_reply: tid=0 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.912+0000 7fa1cfa1db80 1 client.28792 shutdown 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 msgr2=0x55a427950600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac004a00 tx=0x7fa1ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 shutdown_connections 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55a4279c7cf0 0x55a4279ca110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a427954720 0x55a427950600 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42794e890 0x55a42795f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 --2- 172.21.15.46:0/4212046892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a427953e00 0x55a4279500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa19aa00700 0 client.28792 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 >> 172.21.15.46:0/4212046892 conn(0x55a4278ac210 msgr2=0x55a427952aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 shutdown_connections 2024-05-08T23:15:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 1 -- 172.21.15.46:0/4212046892 wait complete. 2024-05-08T23:15:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.913+0000 7fa1cfa1db80 20 client.28792 trim_cache size 0 max 16384 2024-05-08T23:15:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 msgr2=0x7fa19c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 0x7fa19c0a3750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009550 tx=0x7fa1ac009100 comp rx=0 tx=0).stop 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 msgr2=0x7fa1b41994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005770 tx=0x7fa1a80057a0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 shutdown_connections 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b410b3c0 0x7fa1b419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b4101fa0 0x7fa1b41994c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa19c0a1290 0x7fa19c0a3750 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 --2- 172.21.15.46:0/2555631065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b41028c0 0x7fa1b4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 >> 172.21.15.46:0/2555631065 conn(0x7fa1b40fb760 msgr2=0x7fa1b410c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 shutdown_connections 2024-05-08T23:15:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:20.914+0000 7fa1bc000700 1 -- 172.21.15.46:0/2555631065 wait complete. 2024-05-08T23:15:20.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 -- 172.21.15.46:0/2094557951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 msgr2=0x7f4424073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 --2- 172.21.15.46:0/2094557951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424073750 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4418009990 tx=0x7f4418009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 -- 172.21.15.46:0/2094557951 shutdown_connections 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 --2- 172.21.15.46:0/2094557951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442410d6d0 0x7f442410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 --2- 172.21.15.46:0/2094557951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424073750 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 --2- 172.21.15.46:0/2094557951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4424073c90 0x7f442410d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 -- 172.21.15.46:0/2094557951 >> 172.21.15.46:0/2094557951 conn(0x7f44240fc960 msgr2=0x7f44240fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 -- 172.21.15.46:0/2094557951 shutdown_connections 2024-05-08T23:15:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.181+0000 7f442c000700 1 -- 172.21.15.46:0/2094557951 wait complete. 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.182+0000 7f442c000700 1 Processor -- start 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 -- start start 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4424073c90 0x7f4424199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442410d6d0 0x7f442419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e420 con 0x7f442410d6d0 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e590 con 0x7f4424073c90 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e750 con 0x7f4424073370 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f4429600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f442a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442410d6d0 0x7f442419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f4429600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53868/0 (socket says 172.21.15.46:53868) 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f4429600700 1 -- 172.21.15.46:0/348869771 learned_addr learned my addr 172.21.15.46:0/348869771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.183+0000 7f4428c00700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4424073c90 0x7f4424199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 -- 172.21.15.46:0/348869771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4424073c90 msgr2=0x7f4424199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4424073c90 0x7f4424199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 -- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442410d6d0 msgr2=0x7f442419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442410d6d0 0x7f442419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 -- 172.21.15.46:0/348869771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180095f0 con 0x7f4424073370 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4429600700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4418005910 tx=0x7f4418004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4422000700 1 -- 172.21.15.46:0/348869771 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441801d070 con 0x7f4424073370 2024-05-08T23:15:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4422000700 1 -- 172.21.15.46:0/348869771 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4418022470 con 0x7f4424073370 2024-05-08T23:15:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442419e9d0 con 0x7f4424073370 2024-05-08T23:15:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f4422000700 1 -- 172.21.15.46:0/348869771 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418017650 con 0x7f4424073370 2024-05-08T23:15:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.184+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a6270 con 0x7f4424073370 2024-05-08T23:15:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.188+0000 7f4422000700 1 -- 172.21.15.46:0/348869771 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f44180225e0 con 0x7f4424073370 2024-05-08T23:15:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.189+0000 7f443fb3cb80 1 Processor -- start 2024-05-08T23:15:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f4422000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 0x7f440c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 -- start start 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f4422000700 1 -- 172.21.15.46:0/348869771 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f44180ce300 con 0x7f4424073370 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x563861992350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f442a000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 0x7f440c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x563861992cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638618e9150 con 0x563861998720 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638618e6b10 con 0x563861997e00 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638618e2400 con 0x563861992890 2024-05-08T23:15:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f442a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x563861992cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34114/0 (socket says 172.21.15.46:34114) 2024-05-08T23:15:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.190+0000 7f4429600700 1 -- 172.21.15.46:0/615991239 learned_addr learned my addr 172.21.15.46:0/615991239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4428c00700 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x563861992350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 -- 172.21.15.46:0/615991239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 msgr2=0x563861992cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x563861992cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 -- 172.21.15.46:0/615991239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 msgr2=0x563861992350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x563861992350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 -- 172.21.15.46:0/615991239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638617633b0 con 0x563861997e00 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4429600700 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f44180a3010 tx=0x7f4418009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2e1d85c872e9fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f442a000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 0x7f440c0a37d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f441c009ec0 tx=0x7f441c013480 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4416600700 1 -- 172.21.15.46:0/615991239 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441809fea0 con 0x563861997e00 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4416600700 1 -- 172.21.15.46:0/615991239 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441809b850 con 0x563861997e00 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.191+0000 7f4416600700 1 -- 172.21.15.46:0/615991239 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418099580 con 0x563861997e00 2024-05-08T23:15:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.192+0000 7f443fb3cb80 1 -- 172.21.15.46:0/615991239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 msgr2=0x5638619981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.192+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f44180a3010 tx=0x7f4418009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 -- 172.21.15.46:0/615991239 shutdown_connections 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x563861992cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x5638619981e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/615991239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x563861992350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 -- 172.21.15.46:0/615991239 >> 172.21.15.46:0/615991239 conn(0x5638618f0210 msgr2=0x56386198cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 -- 172.21.15.46:0/615991239 shutdown_connections 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 -- 172.21.15.46:0/615991239 wait complete. 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.193+0000 7f443fb3cb80 1 -- start start 2024-05-08T23:15:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 Processor -- start 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x56386199e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x56386199c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56386187e560 con 0x563861998720 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56386187e200 con 0x563861997e00 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638617656a0 con 0x563861992890 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f443fb3cb80 10 client.0 _ll_register_callbacks cb 0x563861887b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f4428c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f4429600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x56386199e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f4428c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34126/0 (socket says 172.21.15.46:34126) 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.194+0000 7f4428c00700 1 -- 172.21.15.46:0/2223586631 learned_addr learned my addr 172.21.15.46:0/2223586631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f442a000700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x56386199c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 -- 172.21.15.46:0/2223586631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 msgr2=0x56386199b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 -- 172.21.15.46:0/2223586631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 msgr2=0x56386199c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x56386199c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 -- 172.21.15.46:0/2223586631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a6270 con 0x563861992890 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4428c00700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f4429600700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x56386199e700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f4418009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1baaa80a53525091 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f440aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441809fea0 con 0x563861992890 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f441809ef80 con 0x563861992890 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.195+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56386183b270 con 0x563861992890 2024-05-08T23:15:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f441809b3b0 con 0x563861992890 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 10 client.28804 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56386170fb60 con 0x563861992890 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56386183b270 con 0x563861992890 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f44180a30b0 con 0x563861992890 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 10 client.28804 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 10 client.28804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 20 client.28804 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638619b41a0 0x5638619b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 4 client.28804 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.196+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563861961810 con 0x5638619b41a0 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.197+0000 7f4428c00700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638619b41a0 0x5638619b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.197+0000 7f4428c00700 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638619b41a0 0x5638619b6590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adc2128209949f00 server_cookie=dd0faed36d0236e1 in_seq=0 out_seq=0 2024-05-08T23:15:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.197+0000 7f440aa00700 10 client.28804 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.198+0000 7f440aa00700 1 -- 172.21.15.46:0/2223586631 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563861961810 con 0x5638619b41a0 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.198+0000 7f440aa00700 10 client.28804 handle_command_reply: tid=0 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.198+0000 7f443fb3cb80 1 client.28804 shutdown 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 msgr2=0x56386199e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x56386199e700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4418000c00 tx=0x7f4418009310 comp rx=0 tx=0).stop 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 shutdown_connections 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638619b41a0 0x5638619b6590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x56386199c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861992890 0x56386199e700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/2223586631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861997e00 0x56386199b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f440aa00700 0 client.28804 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 >> 172.21.15.46:0/2223586631 conn(0x5638618f0210 msgr2=0x56386198c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.199+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 shutdown_connections 2024-05-08T23:15:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.200+0000 7f443fb3cb80 1 -- 172.21.15.46:0/2223586631 wait complete. 2024-05-08T23:15:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.200+0000 7f443fb3cb80 20 client.28804 trim_cache size 0 max 16384 2024-05-08T23:15:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:20 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1743080532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 Processor -- start 2024-05-08T23:15:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 -- start start 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 0x56386198cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x5638619b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56386198e640 con 0x563861998720 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563861987800 con 0x563861992890 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.226+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638619bb750 con 0x563861997e00 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4428c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 0x56386198cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34134/0 (socket says 172.21.15.46:34134) 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4428c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 0x56386198cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53904/0 (socket says 172.21.15.46:53904) 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 -- 172.21.15.46:0/1187545485 learned_addr learned my addr 172.21.15.46:0/1187545485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f442a000700 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x5638619b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 -- 172.21.15.46:0/1187545485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 msgr2=0x56386198cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 0x56386198cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 -- 172.21.15.46:0/1187545485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 msgr2=0x5638619b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x5638619b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 -- 172.21.15.46:0/1187545485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563861992f40 con 0x563861992890 2024-05-08T23:15:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.227+0000 7f4429600700 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f441800f740 tx=0x7f4418022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dabd769d2e7a52e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.228+0000 7f4417000700 1 -- 172.21.15.46:0/1187545485 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441809fea0 con 0x563861992890 2024-05-08T23:15:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.228+0000 7f4417000700 1 -- 172.21.15.46:0/1187545485 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441809af20 con 0x563861992890 2024-05-08T23:15:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.228+0000 7f4417000700 1 -- 172.21.15.46:0/1187545485 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800fc10 con 0x563861992890 2024-05-08T23:15:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1187545485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 msgr2=0x563861992350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f441800f740 tx=0x7f4418022f90 comp rx=0 tx=0).stop 2024-05-08T23:15:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1187545485 shutdown_connections 2024-05-08T23:15:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861998720 0x5638619b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x563861992350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1187545485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861997e00 0x56386198cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1187545485 >> 172.21.15.46:0/1187545485 conn(0x5638618f0210 msgr2=0x563861995b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1187545485 shutdown_connections 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1187545485 wait complete. 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.229+0000 7f443fb3cb80 1 -- start start 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 Processor -- start 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861997e00 0x5638619939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861998720 0x563861993ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638619945c0 con 0x563861997e00 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563861994730 con 0x563861992890 2024-05-08T23:15:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.230+0000 7f443fb3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638619948f0 con 0x563861998720 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f443fb3cb80 10 client.0 _ll_register_callbacks cb 0x7f441c013970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34144/0 (socket says 172.21.15.46:34144) 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 -- 172.21.15.46:0/1080234656 learned_addr learned my addr 172.21.15.46:0/1080234656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f442a000700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861998720 0x563861993ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4428c00700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861997e00 0x5638619939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 -- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861998720 msgr2=0x563861993ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861998720 0x563861993ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 -- 172.21.15.46:0/1080234656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861997e00 msgr2=0x5638619939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861997e00 0x5638619939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.231+0000 7f4429600700 1 -- 172.21.15.46:0/1080234656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56386170fb60 con 0x563861992890 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f4429600700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44180a3700 tx=0x7f44180a3470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78bf38a6414557bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f440aa00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441809fea0 con 0x563861992890 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f44180a3540 con 0x563861992890 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f441809b850 con 0x563861992890 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f441809ef80 con 0x563861992890 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f443fb3cb80 10 client.28718 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.232+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5638618a8560 con 0x563861992890 2024-05-08T23:15:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.233+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441809b850 con 0x563861992890 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f441800fb50 con 0x563861992890 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 10 client.28718 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 10 client.28718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 20 client.28718 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563861a0a900 0x563861a0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 4 client.28718 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638618daec0 con 0x563861a0a900 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.234+0000 7f4428c00700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563861a0a900 0x563861a0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.235+0000 7f440aa00700 10 client.28718 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.235+0000 7f4428c00700 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563861a0a900 0x563861a0ccf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=508662c40b65388 server_cookie=672a3f3f709578ec in_seq=0 out_seq=0 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.236+0000 7f440aa00700 1 -- 172.21.15.46:0/1080234656 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638618daec0 con 0x563861a0a900 2024-05-08T23:15:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.236+0000 7f440aa00700 10 client.28718 handle_command_reply: tid=0 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.236+0000 7f443fb3cb80 1 client.28718 shutdown 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.236+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 msgr2=0x5638619ab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.236+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44180a3700 tx=0x7f44180a3470 comp rx=0 tx=0).stop 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 shutdown_connections 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563861998720 0x563861993ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563861992890 0x5638619ab9e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x563861a0a900 0x563861a0ccf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 --2- 172.21.15.46:0/1080234656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563861997e00 0x5638619939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f440aa00700 0 client.28718 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 >> 172.21.15.46:0/1080234656 conn(0x5638618f0210 msgr2=0x5638619b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 shutdown_connections 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 1 -- 172.21.15.46:0/1080234656 wait complete. 2024-05-08T23:15:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.237+0000 7f443fb3cb80 20 client.28718 trim_cache size 0 max 16384 2024-05-08T23:15:21.240 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 msgr2=0x7f440c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 0x7f440c0a37d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f441c009ec0 tx=0x7f441c013480 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 msgr2=0x7f4424199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4418005910 tx=0x7f4418004730 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 shutdown_connections 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f440c0a1310 0x7f440c0a37d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f442410d6d0 0x7f442419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4424073370 0x7f4424199500 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 --2- 172.21.15.46:0/348869771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4424073c90 0x7f4424199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 >> 172.21.15.46:0/348869771 conn(0x7f44240fc960 msgr2=0x7f44240fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 shutdown_connections 2024-05-08T23:15:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.238+0000 7f442c000700 1 -- 172.21.15.46:0/348869771 wait complete. 2024-05-08T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:20 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1743080532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:21.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:15:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:20 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1743080532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 -- 172.21.15.46:0/1719190611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c102980 msgr2=0x7f5b0c10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 --2- 172.21.15.46:0/1719190611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c102980 0x7f5b0c10ae60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009990 tx=0x7f5af8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 -- 172.21.15.46:0/1719190611 shutdown_connections 2024-05-08T23:15:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 --2- 172.21.15.46:0/1719190611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c10b430 0x7f5b0c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 --2- 172.21.15.46:0/1719190611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102060 0x7f5b0c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 --2- 172.21.15.46:0/1719190611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c102980 0x7f5b0c10ae60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 -- 172.21.15.46:0/1719190611 >> 172.21.15.46:0/1719190611 conn(0x7f5b0c0fb760 msgr2=0x7f5b0c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 -- 172.21.15.46:0/1719190611 shutdown_connections 2024-05-08T23:15:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.485+0000 7f5b14600700 1 -- 172.21.15.46:0/1719190611 wait complete. 2024-05-08T23:15:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.486+0000 7f5b14600700 1 Processor -- start 2024-05-08T23:15:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 -- start start 2024-05-08T23:15:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102980 0x7f5b0c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c10b430 0x7f5b0c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19e440 con 0x7f5b0c102980 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19e5b0 con 0x7f5b0c10b430 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b14600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19e770 con 0x7f5b0c102060 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53926/0 (socket says 172.21.15.46:53926) 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b11c00700 1 -- 172.21.15.46:0/4259056686 learned_addr learned my addr 172.21.15.46:0/4259056686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b12600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c10b430 0x7f5b0c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.487+0000 7f5b11200700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102980 0x7f5b0c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 -- 172.21.15.46:0/4259056686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c10b430 msgr2=0x7f5b0c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c10b430 0x7f5b0c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 -- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102980 msgr2=0x7f5b0c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102980 0x7f5b0c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 -- 172.21.15.46:0/4259056686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80095f0 con 0x7f5b0c102060 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b11c00700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000e9a0 tx=0x7f5b0000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b0a000700 1 -- 172.21.15.46:0/4259056686 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00009790 con 0x7f5b0c102060 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c19ea50 con 0x7f5b0c102060 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b0a000700 1 -- 172.21.15.46:0/4259056686 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b00010430 con 0x7f5b0c102060 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b0a000700 1 -- 172.21.15.46:0/4259056686 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00018680 con 0x7f5b0c102060 2024-05-08T23:15:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.488+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1a6290 con 0x7f5b0c102060 2024-05-08T23:15:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.492+0000 7f5b0a000700 1 -- 172.21.15.46:0/4259056686 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5b000105a0 con 0x7f5b0c102060 2024-05-08T23:15:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.493+0000 7f5b2818ab80 1 Processor -- start 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.493+0000 7f5b2818ab80 1 -- start start 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.493+0000 7f5b0a000700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 0x7f5af40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd0688d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b0a000700 1 -- 172.21.15.46:0/4259056686 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5b00014070 con 0x7f5b0c102060 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd06887890 0x55dd06887cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd067de150 con 0x55dd0688ce00 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd067dbb10 con 0x55dd0688d720 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd067d7400 con 0x55dd06887890 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b12600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 0x7f5af40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd0688d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34178/0 (socket says 172.21.15.46:34178) 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 -- 172.21.15.46:0/1509969441 learned_addr learned my addr 172.21.15.46:0/1509969441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b12600700 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd06887890 0x55dd06887cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 -- 172.21.15.46:0/1509969441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd06887890 msgr2=0x55dd06887cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd06887890 0x55dd06887cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 -- 172.21.15.46:0/1509969441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 msgr2=0x55dd0688d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd0688d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.494+0000 7f5b11200700 1 -- 172.21.15.46:0/1509969441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd066583b0 con 0x55dd0688d720 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.495+0000 7f5b12600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 0x7f5af40a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400f870 tx=0x7f5b0400fc30 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.495+0000 7f5b11200700 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5af8015000 tx=0x7f5af8005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46d9ad15deb0d63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.495+0000 7f5afea00700 1 -- 172.21.15.46:0/1509969441 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801d070 con 0x55dd0688d720 2024-05-08T23:15:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.495+0000 7f5afea00700 1 -- 172.21.15.46:0/1509969441 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af801e040 con 0x55dd0688d720 2024-05-08T23:15:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.495+0000 7f5afea00700 1 -- 172.21.15.46:0/1509969441 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8004880 con 0x55dd0688d720 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 -- 172.21.15.46:0/1509969441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 msgr2=0x55dd06887350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5af8015000 tx=0x7f5af8005ec0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 -- 172.21.15.46:0/1509969441 shutdown_connections 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd06887890 0x55dd06887cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd0688d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/1509969441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06887350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 -- 172.21.15.46:0/1509969441 >> 172.21.15.46:0/1509969441 conn(0x55dd067e5210 msgr2=0x55dd06881df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 -- 172.21.15.46:0/1509969441 shutdown_connections 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.496+0000 7f5b2818ab80 1 -- 172.21.15.46:0/1509969441 wait complete. 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 -- start start 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 Processor -- start 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd068937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd068911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06773560 con 0x55dd06887890 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06773200 con 0x55dd0688d720 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0665a6a0 con 0x55dd0688ce00 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.497+0000 7f5b2818ab80 10 client.0 _ll_register_callbacks cb 0x55dd0677cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd068937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53944/0 (socket says 172.21.15.46:53944) 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 -- 172.21.15.46:0/988376286 learned_addr learned my addr 172.21.15.46:0/988376286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b12600700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd068911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 -- 172.21.15.46:0/988376286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 msgr2=0x55dd068911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd068911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 -- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 msgr2=0x55dd068937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd068937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 -- 172.21.15.46:0/988376286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c10e590 con 0x55dd0688ce00 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b12600700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd068911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11c00700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd068937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5b11200700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009350 tx=0x7f5af800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f845c42d7a3231 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.498+0000 7f5af3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801d030 con 0x55dd0688ce00 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5af8004700 con 0x55dd0688ce00 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd06730270 con 0x55dd0688ce00 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af8005bc0 con 0x55dd0688ce00 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5b2818ab80 10 client.28822 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd06604b60 con 0x55dd0688ce00 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.499+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd06730270 con 0x55dd0688ce00 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5af800f950 con 0x55dd0688ce00 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 10 client.28822 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 10 client.28822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 20 client.28822 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068a0dc0 0x55dd068a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 4 client.28822 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd06856810 con 0x55dd068a0dc0 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.500+0000 7f5b11c00700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068a0dc0 0x55dd068a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.501+0000 7f5b11c00700 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068a0dc0 0x55dd068a31e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2823651f36fac46e server_cookie=fc7dd2bfdf3b00a1 in_seq=0 out_seq=0 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.501+0000 7f5af3400700 10 client.28822 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.501+0000 7f5af3400700 1 -- 172.21.15.46:0/988376286 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd06856810 con 0x55dd068a0dc0 2024-05-08T23:15:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.501+0000 7f5af3400700 10 client.28822 handle_command_reply: tid=0 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.501+0000 7f5b2818ab80 1 client.28822 shutdown 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 msgr2=0x55dd06890ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009350 tx=0x7f5af800b9e0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 shutdown_connections 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd068911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068a0dc0 0x55dd068a31e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd068937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/988376286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd06890ab0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 >> 172.21.15.46:0/988376286 conn(0x55dd067e5210 msgr2=0x55dd068815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5af3400700 0 client.28822 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 shutdown_connections 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 1 -- 172.21.15.46:0/988376286 wait complete. 2024-05-08T23:15:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.502+0000 7f5b2818ab80 20 client.28822 trim_cache size 0 max 16384 2024-05-08T23:15:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.526+0000 7f5b2818ab80 1 Processor -- start 2024-05-08T23:15:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.526+0000 7f5b2818ab80 1 -- start start 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd06881d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 0x55dd068a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06891a60 con 0x55dd0688ce00 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0687edc0 con 0x55dd06887890 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06882e80 con 0x55dd0688d720 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b12600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 0x55dd068a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:34190/0 (socket says 172.21.15.46:34190) 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b11c00700 1 -- 172.21.15.46:0/3744616753 learned_addr learned my addr 172.21.15.46:0/3744616753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.527+0000 7f5b11200700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd06881d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 -- 172.21.15.46:0/3744616753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 msgr2=0x55dd068a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 0x55dd068a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 -- 172.21.15.46:0/3744616753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 msgr2=0x55dd068911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 -- 172.21.15.46:0/3744616753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd0679d560 con 0x55dd0688ce00 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b12600700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 0x55dd068a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11c00700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5b11200700 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd06881d20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009990 tx=0x7f5af800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4a3da6f504b428 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.528+0000 7f5aff400700 1 -- 172.21.15.46:0/3744616753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801d040 con 0x55dd0688ce00 2024-05-08T23:15:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.529+0000 7f5aff400700 1 -- 172.21.15.46:0/3744616753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af800bb00 con 0x55dd0688ce00 2024-05-08T23:15:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.529+0000 7f5aff400700 1 -- 172.21.15.46:0/3744616753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af802c990 con 0x55dd0688ce00 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.529+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3744616753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 msgr2=0x55dd06881d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.529+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd06881d20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009990 tx=0x7f5af800b260 comp rx=0 tx=0).stop 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3744616753 shutdown_connections 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688d720 0x55dd068a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd06887890 0x55dd068911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3744616753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd0688ce00 0x55dd06881d20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3744616753 >> 172.21.15.46:0/3744616753 conn(0x55dd067e5210 msgr2=0x55dd0687e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3744616753 shutdown_connections 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3744616753 wait complete. 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.530+0000 7f5b2818ab80 1 -- start start 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 Processor -- start 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd06898180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06894140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06894820 con 0x55dd06887890 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06894990 con 0x55dd0688d720 2024-05-08T23:15:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd06894b50 con 0x55dd0688ce00 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.531+0000 7f5b2818ab80 10 client.0 _ll_register_callbacks cb 0x55dd0677cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd06898180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53970/0 (socket says 172.21.15.46:53970) 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 -- 172.21.15.46:0/3942405496 learned_addr learned my addr 172.21.15.46:0/3942405496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 -- 172.21.15.46:0/3942405496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 msgr2=0x55dd06894140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b12600700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06894140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06894140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 -- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 msgr2=0x55dd06898180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd06898180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 -- 172.21.15.46:0/3942405496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0005fe70 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5b11200700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5af8005320 tx=0x7f5af800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f307ffeb5ab454 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.532+0000 7f5af3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af80058e0 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5af8000e40 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd067cfec0 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af8004500 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5b2818ab80 10 client.28828 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5b0400fdb0 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af8004be0 con 0x55dd0688ce00 2024-05-08T23:15:21.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.533+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5af802c7a0 con 0x55dd0688ce00 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 10 client.28828 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 10 client.28828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 20 client.28828 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068f6ee0 0x55dd068f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 4 client.28828 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd067cfec0 con 0x55dd068f6ee0 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.534+0000 7f5b11c00700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068f6ee0 0x55dd068f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b11c00700 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068f6ee0 0x55dd068f92d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99786150f69afd61 server_cookie=8a3b8a52ed00ce54 in_seq=0 out_seq=0 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5af3400700 10 client.28828 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5af3400700 1 -- 172.21.15.46:0/3942405496 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd067cfec0 con 0x55dd068f6ee0 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5af3400700 10 client.28828 handle_command_reply: tid=0 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 client.28828 shutdown 2024-05-08T23:15:21.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 msgr2=0x55dd068986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5af8005320 tx=0x7f5af800b9e0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 shutdown_connections 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd0688d720 0x55dd06894140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd068f6ee0 0x55dd068f92d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd06887890 0x55dd06898180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 --2- 172.21.15.46:0/3942405496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd0688ce00 0x55dd068986c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5af3400700 0 client.28828 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 >> 172.21.15.46:0/3942405496 conn(0x55dd067e5210 msgr2=0x55dd068a7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.535+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 shutdown_connections 2024-05-08T23:15:21.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b2818ab80 1 -- 172.21.15.46:0/3942405496 wait complete. 2024-05-08T23:15:21.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b2818ab80 20 client.28828 trim_cache size 0 max 16384 2024-05-08T23:15:21.542 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 msgr2=0x7f5af40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 0x7f5af40a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400f870 tx=0x7f5b0400fc30 comp rx=0 tx=0).stop 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 msgr2=0x7f5b0c199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000e9a0 tx=0x7f5b0000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 shutdown_connections 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c10b430 0x7f5b0c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c102060 0x7f5b0c199520 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5af40a1290 0x7f5af40a3750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 --2- 172.21.15.46:0/4259056686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b0c102980 0x7f5b0c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.536+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 >> 172.21.15.46:0/4259056686 conn(0x7f5b0c0fb760 msgr2=0x7f5b0c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.537+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 shutdown_connections 2024-05-08T23:15:21.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.537+0000 7f5b14600700 1 -- 172.21.15.46:0/4259056686 wait complete. 2024-05-08T23:15:21.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 -- 172.21.15.46:0/3462680699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 msgr2=0x7f2dd8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/3462680699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd8101460 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009990 tx=0x7f2dcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 -- 172.21.15.46:0/3462680699 shutdown_connections 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/3462680699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/3462680699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd8101460 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/3462680699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd8101a30 0x7f2dd8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 -- 172.21.15.46:0/3462680699 >> 172.21.15.46:0/3462680699 conn(0x7f2dd80fc8e0 msgr2=0x7f2dd80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 -- 172.21.15.46:0/3462680699 shutdown_connections 2024-05-08T23:15:21.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.770+0000 7f2ddfa00700 1 -- 172.21.15.46:0/3462680699 wait complete. 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.771+0000 7f2ddfa00700 1 Processor -- start 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 -- start start 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd81061a0 0x7f2dd81978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 0x7f2dd819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819c6f0 con 0x7f2dd819bc70 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819c860 con 0x7f2dd81061a0 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddfa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819ca20 con 0x7f2dd8101080 2024-05-08T23:15:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53990/0 (socket says 172.21.15.46:53990) 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.772+0000 7f2ddd000700 1 -- 172.21.15.46:0/1487746471 learned_addr learned my addr 172.21.15.46:0/1487746471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd81061a0 0x7f2dd81978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 0x7f2dd819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 -- 172.21.15.46:0/1487746471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd81061a0 msgr2=0x7f2dd81978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd81061a0 0x7f2dd81978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 -- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 msgr2=0x7f2dd819c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 0x7f2dd819c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 -- 172.21.15.46:0/1487746471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0095f0 con 0x7f2dd8101080 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 0x7f2dd819c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2dd5600700 1 -- 172.21.15.46:0/1487746471 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc01d070 con 0x7f2dd8101080 2024-05-08T23:15:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd81a3f20 con 0x7f2dd8101080 2024-05-08T23:15:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2dd5600700 1 -- 172.21.15.46:0/1487746471 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dcc022470 con 0x7f2dd8101080 2024-05-08T23:15:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.773+0000 7f2dd5600700 1 -- 172.21.15.46:0/1487746471 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc017650 con 0x7f2dd8101080 2024-05-08T23:15:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.774+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd81a4290 con 0x7f2dd8101080 2024-05-08T23:15:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.778+0000 7f2dd5600700 1 -- 172.21.15.46:0/1487746471 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2dcc022850 con 0x7f2dd8101080 2024-05-08T23:15:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.779+0000 7f2df3493b80 1 Processor -- start 2024-05-08T23:15:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.779+0000 7f2df3493b80 1 -- start start 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d06641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d065e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2dd5600700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 0x7f2dc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d05b5150 con 0x5583d0664720 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d05b2b10 con 0x5583d0663e00 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2df3493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d05ae400 con 0x5583d065e890 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2dd5600700 1 -- 172.21.15.46:0/1487746471 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dcc0ce520 con 0x7f2dd8101080 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2ddda00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d06641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2ddda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 0x7f2dc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2ddda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53994/0 (socket says 172.21.15.46:53994) 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2ddda00700 1 -- 172.21.15.46:0/1375580340 learned_addr learned my addr 172.21.15.46:0/1375580340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.780+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d065e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 -- 172.21.15.46:0/1375580340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 msgr2=0x5583d06641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d06641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 -- 172.21.15.46:0/1375580340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 msgr2=0x5583d065e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d065e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 -- 172.21.15.46:0/1375580340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d042f3b0 con 0x5583d065e890 2024-05-08T23:15:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0012ea0 tx=0x7f2dd0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a0044452703a8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 0x7f2dc00a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2dd81988e0 tx=0x7f2dc4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2dca000700 1 -- 172.21.15.46:0/1375580340 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000e040 con 0x5583d065e890 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2dca000700 1 -- 172.21.15.46:0/1375580340 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd0009310 con 0x5583d065e890 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.781+0000 7f2dca000700 1 -- 172.21.15.46:0/1375580340 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0004260 con 0x5583d065e890 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 -- 172.21.15.46:0/1375580340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 msgr2=0x5583d065ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0012ea0 tx=0x7f2dd0009f90 comp rx=0 tx=0).stop 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 -- 172.21.15.46:0/1375580340 shutdown_connections 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d065e890 0x5583d065ecd0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d06641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1375580340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d065e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 -- 172.21.15.46:0/1375580340 >> 172.21.15.46:0/1375580340 conn(0x5583d05bc210 msgr2=0x5583d0658df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 -- 172.21.15.46:0/1375580340 shutdown_connections 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.782+0000 7f2df3493b80 1 -- 172.21.15.46:0/1375580340 wait complete. 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 -- start start 2024-05-08T23:15:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 Processor -- start 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d065e890 0x5583d0667b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d06685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d054a560 con 0x5583d0664720 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d054a200 con 0x5583d065e890 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.783+0000 7f2df3493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d04316a0 con 0x5583d0663e00 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2df3493b80 10 client.0 _ll_register_callbacks cb 0x5583d0553b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2ddda00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d06685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:53996/0 (socket says 172.21.15.46:53996) 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 -- 172.21.15.46:0/3600843722 learned_addr learned my addr 172.21.15.46:0/3600843722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2ddda00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d06685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50854/0 (socket says 172.21.15.46:50854) 2024-05-08T23:15:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2ddd000700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d065e890 0x5583d0667b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 -- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d065e890 msgr2=0x5583d0667b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d065e890 0x5583d0667b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 -- 172.21.15.46:0/3600843722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 msgr2=0x5583d06685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d06685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 -- 172.21.15.46:0/3600843722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd0012b50 con 0x5583d0663e00 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40126d0 tx=0x7f2dc40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f224a220bdc494 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.784+0000 7f2dbea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc401e610 con 0x5583d0663e00 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583d0507270 con 0x5583d0663e00 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dc4018070 con 0x5583d0663e00 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc401e770 con 0x5583d0663e00 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2df3493b80 10 client.28840 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5583d03dbb60 con 0x5583d0663e00 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583d0507270 con 0x5583d0663e00 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.785+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dc4024020 con 0x5583d0663e00 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 10 client.28840 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 10 client.28840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 20 client.28840 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d067c750 0x5583d067eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 4 client.28840 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583d062d810 con 0x5583d067c750 2024-05-08T23:15:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.786+0000 7f2ddd000700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d067c750 0x5583d067eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.787+0000 7f2ddd000700 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d067c750 0x5583d067eb70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80b48fc399ef7b server_cookie=bb0d42dadb0e315d in_seq=0 out_seq=0 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.787+0000 7f2dbea00700 10 client.28840 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.787+0000 7f2dbea00700 1 -- 172.21.15.46:0/3600843722 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583d062d810 con 0x5583d067c750 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.787+0000 7f2dbea00700 10 client.28840 handle_command_reply: tid=0 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.787+0000 7f2df3493b80 1 client.28840 shutdown 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 msgr2=0x5583d06680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40126d0 tx=0x7f2dc40047c0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 shutdown_connections 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d067c750 0x5583d067eb70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d0664720 0x5583d06685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d065e890 0x5583d0667b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2dbea00700 0 client.28840 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 --2- 172.21.15.46:0/3600843722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d06680a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 >> 172.21.15.46:0/3600843722 conn(0x5583d05bc210 msgr2=0x5583d06585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 shutdown_connections 2024-05-08T23:15:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 1 -- 172.21.15.46:0/3600843722 wait complete. 2024-05-08T23:15:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.788+0000 7f2df3493b80 20 client.28840 trim_cache size 0 max 16384 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 Processor -- start 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 -- start start 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d0667460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0664720 0x5583d0663340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d06691a0 con 0x5583d065e890 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d0659910 con 0x5583d0663e00 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2df3493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d065ae60 con 0x5583d0664720 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.802+0000 7f2ddd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50866/0 (socket says 172.21.15.46:50866) 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 -- 172.21.15.46:0/1483761054 learned_addr learned my addr 172.21.15.46:0/1483761054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d0667460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddda00700 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0664720 0x5583d0663340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 -- 172.21.15.46:0/1483761054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0664720 msgr2=0x5583d0663340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0664720 0x5583d0663340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 -- 172.21.15.46:0/1483761054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 msgr2=0x5583d0667460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d0667460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 -- 172.21.15.46:0/1483761054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d0574560 con 0x5583d065e890 2024-05-08T23:15:21.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2ddd000700 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc09af50 tx=0x7f2dcc022c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a491154dd36bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2dcaa00700 1 -- 172.21.15.46:0/1483761054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0b1070 con 0x5583d065e890 2024-05-08T23:15:21.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2dcaa00700 1 -- 172.21.15.46:0/1483761054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dcc0b3030 con 0x5583d065e890 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.803+0000 7f2dcaa00700 1 -- 172.21.15.46:0/1483761054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc09b2e0 con 0x5583d065e890 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 -- 172.21.15.46:0/1483761054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 msgr2=0x5583d065e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc09af50 tx=0x7f2dcc022c50 comp rx=0 tx=0).stop 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 -- 172.21.15.46:0/1483761054 shutdown_connections 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0664720 0x5583d0663340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d065e350 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 --2- 172.21.15.46:0/1483761054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0663e00 0x5583d0667460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 -- 172.21.15.46:0/1483761054 >> 172.21.15.46:0/1483761054 conn(0x5583d05bc210 msgr2=0x5583d0662c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 -- 172.21.15.46:0/1483761054 shutdown_connections 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.804+0000 7f2df3493b80 1 -- 172.21.15.46:0/1483761054 wait complete. 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 -- start start 2024-05-08T23:15:21.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 Processor -- start 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d06715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 0x5583d0672060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d06d1d20 con 0x5583d065e890 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d06d1e90 con 0x5583d0664720 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.805+0000 7f2df3493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d06d2050 con 0x5583d0663e00 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2df3493b80 10 client.0 _ll_register_callbacks cb 0x5583d0553b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2dd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d06715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2dd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:54014/0 (socket says 172.21.15.46:54014) 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2dd7e00700 1 -- 172.21.15.46:0/2988601587 learned_addr learned my addr 172.21.15.46:0/2988601587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddda00700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 0x5583d0672060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 -- 172.21.15.46:0/2988601587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 msgr2=0x5583d0671b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 -- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 msgr2=0x5583d0672060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 0x5583d0672060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 -- 172.21.15.46:0/2988601587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd81a4290 con 0x5583d065e890 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddda00700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 0x5583d0672060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2ddd000700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d06715e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098e0 tx=0x7f2dcc0cb770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d12bf659f5dba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.806+0000 7f2dbea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0b1040 con 0x5583d065e890 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dcc09f080 con 0x5583d065e890 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dd0009310 con 0x5583d065e890 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dcc09abb0 con 0x5583d065e890 2024-05-08T23:15:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 10 client.18741 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5583d0433cd0 con 0x5583d065e890 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd0009310 con 0x5583d065e890 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dcc0ce1b0 con 0x5583d065e890 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 10 client.18741 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 20 client.18741 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d06d3310 0x5583d06d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 4 client.18741 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.807+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583d05a6ec0 con 0x5583d06d3310 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.808+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d06d3310 0x5583d06d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.808+0000 7f2dd7e00700 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d06d3310 0x5583d06d5700 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5044fb34807dd39c server_cookie=31eadeba9dde87e9 in_seq=0 out_seq=0 2024-05-08T23:15:21.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.808+0000 7f2dbea00700 10 client.18741 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:21.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.808+0000 7f2dbea00700 1 -- 172.21.15.46:0/2988601587 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583d05a6ec0 con 0x5583d06d3310 2024-05-08T23:15:21.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2dbea00700 10 client.18741 handle_command_reply: tid=0 2024-05-08T23:15:21.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 client.18741 shutdown 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 msgr2=0x5583d06715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d06715e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098e0 tx=0x7f2dcc0cb770 comp rx=0 tx=0).stop 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 shutdown_connections 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583d0664720 0x5583d0672060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5583d06d3310 0x5583d06d5700 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583d065e890 0x5583d06715e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 --2- 172.21.15.46:0/2988601587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583d0663e00 0x5583d0671b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2dbea00700 0 client.18741 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 >> 172.21.15.46:0/2988601587 conn(0x5583d05bc210 msgr2=0x5583d0667b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 shutdown_connections 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 1 -- 172.21.15.46:0/2988601587 wait complete. 2024-05-08T23:15:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.809+0000 7f2df3493b80 20 client.18741 trim_cache size 0 max 16384 2024-05-08T23:15:21.815 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 msgr2=0x7f2dc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 0x7f2dc00a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2dd81988e0 tx=0x7f2dc4009260 comp rx=0 tx=0).stop 2024-05-08T23:15:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 msgr2=0x7f2dd8197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc0057f0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 shutdown_connections 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dc00a1290 0x7f2dc00a3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd819bc70 0x7f2dd819c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8197390 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 --2- 172.21.15.46:0/1487746471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd81061a0 0x7f2dd81978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 >> 172.21.15.46:0/1487746471 conn(0x7f2dd80fc8e0 msgr2=0x7f2dd80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 shutdown_connections 2024-05-08T23:15:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:21.810+0000 7f2ddfa00700 1 -- 172.21.15.46:0/1487746471 wait complete. 2024-05-08T23:15:21.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1008328 9.35 MB/sec execute 463 sec latency 19.386 ms 2024-05-08T23:15:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:21 smithi086 ceph-mon[78610]: pgmap v359: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 162 op/s 2024-05-08T23:15:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:21 smithi053 ceph-mon[77587]: pgmap v359: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 162 op/s 2024-05-08T23:15:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:21 smithi046 ceph-mon[36898]: pgmap v359: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 162 op/s 2024-05-08T23:15:22.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:15:22.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:15:22.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1009427 9.35 MB/sec execute 464 sec latency 14.793 ms 2024-05-08T23:15:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 -- 172.21.15.46:0/2084946927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f80733d0 msgr2=0x7f00f80737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 --2- 172.21.15.46:0/2084946927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f80733d0 0x7f00f80737b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009990 tx=0x7f00e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 -- 172.21.15.46:0/2084946927 shutdown_connections 2024-05-08T23:15:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 --2- 172.21.15.46:0/2084946927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f810d630 0x7f00f810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 --2- 172.21.15.46:0/2084946927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f80733d0 0x7f00f80737b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 --2- 172.21.15.46:0/2084946927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f8073cf0 0x7f00f810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 -- 172.21.15.46:0/2084946927 >> 172.21.15.46:0/2084946927 conn(0x7f00f80fc900 msgr2=0x7f00f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 -- 172.21.15.46:0/2084946927 shutdown_connections 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.044+0000 7f00ff000700 1 -- 172.21.15.46:0/2084946927 wait complete. 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.046+0000 7f00ff000700 1 Processor -- start 2024-05-08T23:15:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.046+0000 7f00ff000700 1 -- start start 2024-05-08T23:15:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f80733d0 0x7f00f8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f810d630 0x7f00f819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f819e420 con 0x7f00f810d630 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f819e590 con 0x7f00f80733d0 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00ff000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f819e750 con 0x7f00f8073cf0 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39898/0 (socket says 172.21.15.46:39898) 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 -- 172.21.15.46:0/2689032271 learned_addr learned my addr 172.21.15.46:0/2689032271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f80733d0 0x7f00f8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00fd000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f810d630 0x7f00f819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 -- 172.21.15.46:0/2689032271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f80733d0 msgr2=0x7f00f8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f80733d0 0x7f00f8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 -- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f810d630 msgr2=0x7f00f819dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f810d630 0x7f00f819dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.047+0000 7f00f7400700 1 -- 172.21.15.46:0/2689032271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e80095f0 con 0x7f00f8073cf0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00f7400700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f00e400e9a0 tx=0x7f00e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00f4c00700 1 -- 172.21.15.46:0/2689032271 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4009790 con 0x7f00f8073cf0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00f4c00700 1 -- 172.21.15.46:0/2689032271 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e4010430 con 0x7f00f8073cf0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f819ea30 con 0x7f00f8073cf0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00f4c00700 1 -- 172.21.15.46:0/2689032271 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4018680 con 0x7f00f8073cf0 2024-05-08T23:15:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.048+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f81a6270 con 0x7f00f8073cf0 2024-05-08T23:15:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.053+0000 7f00f4c00700 1 -- 172.21.15.46:0/2689032271 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f00e40105a0 con 0x7f00f8073cf0 2024-05-08T23:15:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 Processor -- start 2024-05-08T23:15:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 -- start start 2024-05-08T23:15:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c1b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f00f4c00700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 0x7f00dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c1bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40b721d0 con 0x555b40c1b910 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40b6fb90 con 0x555b40c217a0 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.054+0000 7f0112becb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40b6b480 con 0x555b40c20e80 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00f4c00700 1 -- 172.21.15.46:0/2689032271 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f00e4014070 con 0x7f00f8073cf0 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00f7400700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 0x7f00dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c1b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39910/0 (socket says 172.21.15.46:39910) 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 -- 172.21.15.46:0/2536837345 learned_addr learned my addr 172.21.15.46:0/2536837345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c1bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 -- 172.21.15.46:0/2536837345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 msgr2=0x555b40c1b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c1b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 -- 172.21.15.46:0/2536837345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 msgr2=0x555b40c1bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c1bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00fd000700 1 -- 172.21.15.46:0/2536837345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b409ec430 con 0x555b40c20e80 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.055+0000 7f00f7e00700 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c1b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.056+0000 7f00f7400700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 0x7f00dc0a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00e40964c0 tx=0x7f00e400c480 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.056+0000 7f00fd000700 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f00f0008cd0 tx=0x7f00f0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50517800e7f68c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.056+0000 7f00ed600700 1 -- 172.21.15.46:0/2536837345 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000e050 con 0x555b40c20e80 2024-05-08T23:15:23.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.056+0000 7f00ed600700 1 -- 172.21.15.46:0/2536837345 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f000f040 con 0x555b40c20e80 2024-05-08T23:15:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.056+0000 7f00ed600700 1 -- 172.21.15.46:0/2536837345 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004280 con 0x555b40c20e80 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- 172.21.15.46:0/2536837345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 msgr2=0x555b40c21260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f00f0008cd0 tx=0x7f00f0009920 comp rx=0 tx=0).stop 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- 172.21.15.46:0/2536837345 shutdown_connections 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c1bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c20e80 0x555b40c21260 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 --2- 172.21.15.46:0/2536837345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c1b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- 172.21.15.46:0/2536837345 >> 172.21.15.46:0/2536837345 conn(0x555b40b79290 msgr2=0x555b40c15e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- 172.21.15.46:0/2536837345 shutdown_connections 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- 172.21.15.46:0/2536837345 wait complete. 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.057+0000 7f0112becb80 1 -- start start 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 Processor -- start 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40b075e0 con 0x555b40c20e80 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40b07280 con 0x555b40c1b910 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b409ee720 con 0x555b40c217a0 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f0112becb80 10 client.0 _ll_register_callbacks cb 0x555b40b10bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f00fd000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f00fd000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50670/0 (socket says 172.21.15.46:50670) 2024-05-08T23:15:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.058+0000 7f00fd000700 1 -- 172.21.15.46:0/291800965 learned_addr learned my addr 172.21.15.46:0/291800965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 -- 172.21.15.46:0/291800965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 msgr2=0x555b40c25630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c25630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 -- 172.21.15.46:0/291800965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 msgr2=0x555b40c250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 -- 172.21.15.46:0/291800965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f8110760 con 0x555b40c1b910 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00f7e00700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00fd000700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009ec0 tx=0x7f00f0018c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b0677961f1696b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00e2000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.059+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002c070 con 0x555b40c1b910 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b40ac42f0 con 0x555b40c1b910 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f00f0003a00 con 0x555b40c1b910 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f000e030 con 0x555b40c1b910 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 10 client.28763 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f00f00089b0 con 0x555b40c1b910 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b40ac42f0 con 0x555b40c1b910 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00f002f020 con 0x555b40c1b910 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 10 client.28763 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 10 client.28763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 20 client.28763 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c35130 0x555b40c37520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 4 client.28763 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.060+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b40bea890 con 0x555b40c35130 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.061+0000 7f00f7e00700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c35130 0x555b40c37520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.061+0000 7f00f7e00700 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c35130 0x555b40c37520 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97aab1cb3054ee71 server_cookie=8b7d0f2ad0eb7c68 in_seq=0 out_seq=0 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.061+0000 7f00e2000700 10 client.28763 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f00e2000700 1 -- 172.21.15.46:0/291800965 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b40bea890 con 0x555b40c35130 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f00e2000700 10 client.28763 handle_command_reply: tid=0 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 client.28763 shutdown 2024-05-08T23:15:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 msgr2=0x555b40c24bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009ec0 tx=0x7f00f0018c50 comp rx=0 tx=0).stop 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 shutdown_connections 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c1b910 0x555b40c24bb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c35130 0x555b40c37520 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 --2- 172.21.15.46:0/291800965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f00e2000700 0 client.28763 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 >> 172.21.15.46:0/291800965 conn(0x555b40b79290 msgr2=0x555b40c15620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.062+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 shutdown_connections 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.063+0000 7f0112becb80 1 -- 172.21.15.46:0/291800965 wait complete. 2024-05-08T23:15:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.063+0000 7f0112becb80 20 client.28763 trim_cache size 0 max 16384 2024-05-08T23:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.089+0000 7f0112becb80 1 Processor -- start 2024-05-08T23:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.089+0000 7f0112becb80 1 -- start start 2024-05-08T23:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c1b910 0x555b40c30ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c17b10 con 0x555b40c20e80 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c1e8f0 con 0x555b40c217a0 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f0112becb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c27720 con 0x555b40c1b910 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60376/0 (socket says 172.21.15.46:60376) 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 -- 172.21.15.46:0/2953216761 learned_addr learned my addr 172.21.15.46:0/2953216761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00fd000700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c1b910 0x555b40c30ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00fd000700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 -- 172.21.15.46:0/2953216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c1b910 msgr2=0x555b40c30ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c1b910 0x555b40c30ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 -- 172.21.15.46:0/2953216761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 msgr2=0x555b40c2c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c2c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00f7e00700 1 -- 172.21.15.46:0/2953216761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b40c1bfc0 con 0x555b40c20e80 2024-05-08T23:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.090+0000 7f00fd000700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c2c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.091+0000 7f00f7e00700 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f00e80098e0 tx=0x7f00e8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d4a5d4de564678 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.091+0000 7f00ee000700 1 -- 172.21.15.46:0/2953216761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e801d070 con 0x555b40c20e80 2024-05-08T23:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.091+0000 7f00ee000700 1 -- 172.21.15.46:0/2953216761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e801e040 con 0x555b40c20e80 2024-05-08T23:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.091+0000 7f00ee000700 1 -- 172.21.15.46:0/2953216761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800f460 con 0x555b40c20e80 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- 172.21.15.46:0/2953216761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 msgr2=0x555b40c15da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f00e80098e0 tx=0x7f00e8009310 comp rx=0 tx=0).stop 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- 172.21.15.46:0/2953216761 shutdown_connections 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c217a0 0x555b40c2c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c1b910 0x555b40c30ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 --2- 172.21.15.46:0/2953216761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c20e80 0x555b40c15da0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- 172.21.15.46:0/2953216761 >> 172.21.15.46:0/2953216761 conn(0x555b40b79290 msgr2=0x555b40c25490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- 172.21.15.46:0/2953216761 shutdown_connections 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- 172.21.15.46:0/2953216761 wait complete. 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 -- start start 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.092+0000 7f0112becb80 1 Processor -- start 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c20e80 0x555b40c84f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c7f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c7fc60 con 0x555b40c1b910 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c7fdd0 con 0x555b40c20e80 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b40c7ff90 con 0x555b40c217a0 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f0112becb80 10 client.0 _ll_register_callbacks cb 0x555b40b10bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60390/0 (socket says 172.21.15.46:60390) 2024-05-08T23:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 -- 172.21.15.46:0/1923112189 learned_addr learned my addr 172.21.15.46:0/1923112189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00f7e00700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c20e80 0x555b40c84f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c7f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 -- 172.21.15.46:0/1923112189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 msgr2=0x555b40c7f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c7f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 -- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c20e80 msgr2=0x555b40c84f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c20e80 0x555b40c84f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.093+0000 7f00fd000700 1 -- 172.21.15.46:0/1923112189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8005530 con 0x555b40c1b910 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f00fd000700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00f001a440 tx=0x7f00f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fc9bde34c8f588 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f00e2000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002c040 con 0x555b40c1b910 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f00f0018a90 con 0x555b40c1b910 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f000f040 con 0x555b40c1b910 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f000e040 con 0x555b40c1b910 2024-05-08T23:15:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f0112becb80 10 client.18759 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.094+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555b40998be0 con 0x555b40c1b910 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f000f040 con 0x555b40c1b910 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00f002f070 con 0x555b40c1b910 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 10 client.18759 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 20 client.18759 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c8c550 0x555b40c8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 4 client.18759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555b40b63f40 con 0x555b40c8c550 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.095+0000 7f00f7e00700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c8c550 0x555b40c8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.096+0000 7f00f7e00700 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c8c550 0x555b40c8e970 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca44b575a0dfa03 server_cookie=67cba8933824ee8 in_seq=0 out_seq=0 2024-05-08T23:15:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.096+0000 7f00e2000700 10 client.18759 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f00e2000700 1 -- 172.21.15.46:0/1923112189 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555b40b63f40 con 0x555b40c8c550 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f00e2000700 10 client.18759 handle_command_reply: tid=0 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 client.18759 shutdown 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 msgr2=0x555b40c84a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00f001a440 tx=0x7f00f000b040 comp rx=0 tx=0).stop 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 shutdown_connections 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b40c217a0 0x555b40c7f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b40c8c550 0x555b40c8e970 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b40c1b910 0x555b40c84a40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 --2- 172.21.15.46:0/1923112189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b40c20e80 0x555b40c84f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 >> 172.21.15.46:0/1923112189 conn(0x555b40b79290 msgr2=0x555b40c1fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f00e2000700 0 client.18759 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 shutdown_connections 2024-05-08T23:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.097+0000 7f0112becb80 1 -- 172.21.15.46:0/1923112189 wait complete. 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f0112becb80 20 client.18759 trim_cache size 0 max 16384 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "c5b6660f-2451-4395-808e-b37b155fbe6a", 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:15:23.102 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 msgr2=0x7f00dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 0x7f00dc0a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00e40964c0 tx=0x7f00e400c480 comp rx=0 tx=0).stop 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 msgr2=0x7f00f8199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f00e400e9a0 tx=0x7f00e400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 shutdown_connections 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00dc0a1290 0x7f00dc0a3750 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00f810d630 0x7f00f819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00f80733d0 0x7f00f8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 --2- 172.21.15.46:0/2689032271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00f8073cf0 0x7f00f8199a90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.098+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 >> 172.21.15.46:0/2689032271 conn(0x7f00f80fc900 msgr2=0x7f00f80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.099+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 shutdown_connections 2024-05-08T23:15:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.099+0000 7f00ff000700 1 -- 172.21.15.46:0/2689032271 wait complete. 2024-05-08T23:15:23.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:15:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 -- 172.21.15.46:0/2674197758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca8105190 msgr2=0x7f0ca8105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 --2- 172.21.15.46:0/2674197758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca8105190 0x7f0ca8105640 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b0a0 tx=0x7f0ca000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 -- 172.21.15.46:0/2674197758 shutdown_connections 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 --2- 172.21.15.46:0/2674197758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca8105190 0x7f0ca8105640 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 --2- 172.21.15.46:0/2674197758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca80ffd90 0x7f0ca8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 --2- 172.21.15.46:0/2674197758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 0x7f0ca8104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 -- 172.21.15.46:0/2674197758 >> 172.21.15.46:0/2674197758 conn(0x7f0ca80fb760 msgr2=0x7f0ca80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 -- 172.21.15.46:0/2674197758 shutdown_connections 2024-05-08T23:15:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.380+0000 7f0cb0200700 1 -- 172.21.15.46:0/2674197758 wait complete. 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 Processor -- start 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 -- start start 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 0x7f0ca81a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca8105190 0x7f0ca819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.381+0000 7f0cb0200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca819dec0 con 0x7f0ca81006b0 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cb0200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca819e030 con 0x7f0ca8105190 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cb0200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca819e1f0 con 0x7f0ca80ffd90 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cae200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca8105190 0x7f0ca819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39962/0 (socket says 172.21.15.46:39962) 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 -- 172.21.15.46:0/981701413 learned_addr learned my addr 172.21.15.46:0/981701413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cace00700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 0x7f0ca81a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 -- 172.21.15.46:0/981701413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca8105190 msgr2=0x7f0ca819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca8105190 0x7f0ca819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 -- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 msgr2=0x7f0ca81a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 0x7f0ca81a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 -- 172.21.15.46:0/981701413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca0009d00 con 0x7f0ca80ffd90 2024-05-08T23:15:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.382+0000 7f0cad800700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00e9c0 tx=0x7f0c9c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.383+0000 7f0ca6000700 1 -- 172.21.15.46:0/981701413 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0097b0 con 0x7f0ca80ffd90 2024-05-08T23:15:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.383+0000 7f0ca6000700 1 -- 172.21.15.46:0/981701413 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9c00ee60 con 0x7f0ca80ffd90 2024-05-08T23:15:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.383+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca819e4d0 con 0x7f0ca80ffd90 2024-05-08T23:15:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.383+0000 7f0ca6000700 1 -- 172.21.15.46:0/981701413 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0054f0 con 0x7f0ca80ffd90 2024-05-08T23:15:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.383+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81aa710 con 0x7f0ca80ffd90 2024-05-08T23:15:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.386+0000 7f0ca6000700 1 -- 172.21.15.46:0/981701413 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0c9c004750 con 0x7f0ca80ffd90 2024-05-08T23:15:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.387+0000 7f0cc3c53b80 1 Processor -- start 2024-05-08T23:15:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.387+0000 7f0cc3c53b80 1 -- start start 2024-05-08T23:15:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0ca6000700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 0x7f0c900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b382e150 con 0x5615b38dce00 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b382bb10 con 0x5615b38d7890 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b3827400 con 0x5615b38dd720 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0ca6000700 1 -- 172.21.15.46:0/981701413 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0c9c014070 con 0x7f0ca80ffd90 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cae200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39964/0 (socket says 172.21.15.46:39964) 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cae200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60416/0 (socket says 172.21.15.46:60416) 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cad800700 1 -- 172.21.15.46:0/352136273 learned_addr learned my addr 172.21.15.46:0/352136273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cace00700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 0x7f0c900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cae200700 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cad800700 1 -- 172.21.15.46:0/352136273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 msgr2=0x5615b38d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.388+0000 7f0cad800700 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0cad800700 1 -- 172.21.15.46:0/352136273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 msgr2=0x5615b38dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0cad800700 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0cad800700 1 -- 172.21.15.46:0/352136273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b36a83b0 con 0x5615b38dd720 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0cad800700 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0a1f30 tx=0x7f0c9c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25968c8ae7702963 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0cace00700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 0x7f0c900a37d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ca819ec70 tx=0x7f0c94009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0c9aa00700 1 -- 172.21.15.46:0/352136273 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0cd160 con 0x5615b38dd720 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0c9aa00700 1 -- 172.21.15.46:0/352136273 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9c099940 con 0x5615b38dd720 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.389+0000 7f0c9aa00700 1 -- 172.21.15.46:0/352136273 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c098460 con 0x5615b38dd720 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/352136273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 msgr2=0x5615b38d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0a1f30 tx=0x7f0c9c01d000 comp rx=0 tx=0).stop 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/352136273 shutdown_connections 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/352136273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b38d7350 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/352136273 >> 172.21.15.46:0/352136273 conn(0x5615b3835210 msgr2=0x5615b38d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/352136273 shutdown_connections 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/352136273 wait complete. 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.390+0000 7f0cc3c53b80 1 -- start start 2024-05-08T23:15:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.391+0000 7f0cc3c53b80 1 Processor -- start 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b37c3560 con 0x5615b38dce00 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b37c3200 con 0x5615b38dd720 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b36aa6a0 con 0x5615b38d7890 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cc3c53b80 10 client.0 _ll_register_callbacks cb 0x5615b37ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cace00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39968/0 (socket says 172.21.15.46:39968) 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 -- 172.21.15.46:0/3184914203 learned_addr learned my addr 172.21.15.46:0/3184914203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cae200700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 -- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 msgr2=0x5615b38e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 -- 172.21.15.46:0/3184914203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 msgr2=0x5615b38e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.392+0000 7f0cad800700 1 -- 172.21.15.46:0/3184914203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81aa710 con 0x5615b38d7890 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cad800700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c099ab0 tx=0x7f0c9c0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7a94c35f90c38e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0caaa0 con 0x5615b38d7890 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0c9c099fe0 con 0x5615b38d7890 2024-05-08T23:15:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615b3780270 con 0x5615b38d7890 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9c09a6e0 con 0x5615b38d7890 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 10 client.28876 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5615b3654b60 con 0x5615b38d7890 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615b3780270 con 0x5615b38d7890 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c9c09a240 con 0x5615b38d7890 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 10 client.28876 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 10 client.28876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 20 client.28876 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.393+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b38f0e60 0x5615b38f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.394+0000 7f0cc3c53b80 4 client.28876 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.394+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615b38a6810 con 0x5615b38f0e60 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.394+0000 7f0cace00700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b38f0e60 0x5615b38f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.394+0000 7f0cace00700 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b38f0e60 0x5615b38f3280 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe067062b968e24 server_cookie=ba3fdb8483cd0e09 in_seq=0 out_seq=0 2024-05-08T23:15:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.394+0000 7f0c8f400700 10 client.28876 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0c8f400700 1 -- 172.21.15.46:0/3184914203 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615b38a6810 con 0x5615b38f0e60 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0c8f400700 10 client.28876 handle_command_reply: tid=0 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 client.28876 shutdown 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 msgr2=0x5615b38e0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c099ab0 tx=0x7f0c9c0056d0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 shutdown_connections 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38d7890 0x5615b38e0ab0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b38f0e60 0x5615b38f3280 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3184914203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0c8f400700 0 client.28876 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 >> 172.21.15.46:0/3184914203 conn(0x5615b3835210 msgr2=0x5615b38d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.395+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 shutdown_connections 2024-05-08T23:15:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.396+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3184914203 wait complete. 2024-05-08T23:15:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.396+0000 7f0cc3c53b80 20 client.28876 trim_cache size 0 max 16384 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 Processor -- start 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 -- start start 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 0x5615b38d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dce00 0x5615b38dc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b38cedc0 con 0x5615b38d7890 2024-05-08T23:15:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b38cca90 con 0x5615b38dd720 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b38f8490 con 0x5615b38dce00 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cad800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 0x5615b38d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cae200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dce00 0x5615b38dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cae200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50730/0 (socket says 172.21.15.46:50730) 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cae200700 1 -- 172.21.15.46:0/3218930183 learned_addr learned my addr 172.21.15.46:0/3218930183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.414+0000 7f0cad800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 0x5615b38d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60436/0 (socket says 172.21.15.46:60436) 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 -- 172.21.15.46:0/3218930183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dce00 msgr2=0x5615b38dc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dce00 0x5615b38dc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 -- 172.21.15.46:0/3218930183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 msgr2=0x5615b38d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 0x5615b38d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 -- 172.21.15.46:0/3218930183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b37ed560 con 0x5615b38dd720 2024-05-08T23:15:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0cae200700 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b0a0 tx=0x7f0ca00094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120488b40743f42a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0c9b400700 1 -- 172.21.15.46:0/3218930183 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000e040 con 0x5615b38dd720 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0c9b400700 1 -- 172.21.15.46:0/3218930183 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ca000bcb0 con 0x5615b38dd720 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.415+0000 7f0c9b400700 1 -- 172.21.15.46:0/3218930183 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00206e0 con 0x5615b38dd720 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3218930183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 msgr2=0x5615b38d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b0a0 tx=0x7f0ca00094c0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3218930183 shutdown_connections 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38dd720 0x5615b38d1d20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38d7890 0x5615b38d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3218930183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dce00 0x5615b38dc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3218930183 >> 172.21.15.46:0/3218930183 conn(0x5615b3835210 msgr2=0x5615b38dbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3218930183 shutdown_connections 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3218930183 wait complete. 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.416+0000 7f0cc3c53b80 1 -- start start 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 Processor -- start 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b39408d0 con 0x5615b38dce00 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b3940a40 con 0x5615b38d7890 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b3940c00 con 0x5615b38dd720 2024-05-08T23:15:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cc3c53b80 10 client.0 _ll_register_callbacks cb 0x5615b37ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cae200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cae200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39986/0 (socket says 172.21.15.46:39986) 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38e4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50738/0 (socket says 172.21.15.46:50738) 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.417+0000 7f0cae200700 1 -- 172.21.15.46:0/3302446798 learned_addr learned my addr 172.21.15.46:0/3302446798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cace00700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 -- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 msgr2=0x5615b38e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 -- 172.21.15.46:0/3302446798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 msgr2=0x5615b38e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 -- 172.21.15.46:0/3302446798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca8114b40 con 0x5615b38dd720 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cae200700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009610 tx=0x7f0ca0007550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=262176eca5881046 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0c8f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000e040 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0ca0003e10 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.418+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c9c099940 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ca0020480 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 10 client.28882 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5615b36accd0 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9c099940 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ca002e030 con 0x5615b38dd720 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 10 client.28882 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 10 client.28882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 20 client.28882 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b39481c0 0x5615b394a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 4 client.28882 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615b381fec0 con 0x5615b39481c0 2024-05-08T23:15:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.419+0000 7f0cad800700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b39481c0 0x5615b394a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cad800700 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b39481c0 0x5615b394a5e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb5a4e6f3e59994 server_cookie=f2ef7421c1a8c328 in_seq=0 out_seq=0 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0c8f400700 10 client.28882 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0c8f400700 1 -- 172.21.15.46:0/3302446798 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5615b381fec0 con 0x5615b39481c0 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0c8f400700 10 client.28882 handle_command_reply: tid=0 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 client.28882 shutdown 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 msgr2=0x5615b3940390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009610 tx=0x7f0ca0007550 comp rx=0 tx=0).stop 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 shutdown_connections 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615b38dd720 0x5615b3940390 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615b38d7890 0x5615b38e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5615b39481c0 0x5615b394a5e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 --2- 172.21.15.46:0/3302446798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615b38dce00 0x5615b38e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0c8f400700 0 client.28882 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 >> 172.21.15.46:0/3302446798 conn(0x5615b3835210 msgr2=0x5615b38dbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.420+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 shutdown_connections 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cc3c53b80 1 -- 172.21.15.46:0/3302446798 wait complete. 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cc3c53b80 20 client.28882 trim_cache size 0 max 16384 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "c5b6660f-2451-4395-808e-b37b155fbe6a": { 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "c5b6660f-2451-4395-808e-b37b155fbe6a", 2024-05-08T23:15:23.426 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:15:23.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:15:23.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:15:23.427 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 msgr2=0x7f0c900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 0x7f0c900a37d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ca819ec70 tx=0x7f0c94009260 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 msgr2=0x7f0ca81a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00e9c0 tx=0x7f0c9c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 shutdown_connections 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca8105190 0x7f0ca819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca80ffd90 0x7f0ca81a3720 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c900a1310 0x7f0c900a37d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 --2- 172.21.15.46:0/981701413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ca81006b0 0x7f0ca81a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 >> 172.21.15.46:0/981701413 conn(0x7f0ca80fb760 msgr2=0x7f0ca8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.421+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 shutdown_connections 2024-05-08T23:15:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:23.422+0000 7f0cb0200700 1 -- 172.21.15.46:0/981701413 wait complete. 2024-05-08T23:15:23.441 INFO:tasks.cephfs.filesystem:scrub status for tag:c5b6660f-2451-4395-808e-b37b155fbe6a - {'path': '/', 'tag': 'c5b6660f-2451-4395-808e-b37b155fbe6a', 'options': 'recursive,force'} 2024-05-08T23:15:23.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1011207 9.34 MB/sec execute 465 sec latency 25.355 ms 2024-05-08T23:15:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: pgmap v360: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: pgmap v360: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: pgmap v360: 129 pgs: 129 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:24.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1012662 9.35 MB/sec execute 466 sec latency 16.760 ms 2024-05-08T23:15:25.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1014063 9.34 MB/sec execute 467 sec latency 24.748 ms 2024-05-08T23:15:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:25 smithi086 ceph-mon[78610]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-08T23:15:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:25 smithi053 ceph-mon[77587]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-08T23:15:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:25 smithi046 ceph-mon[36898]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-08T23:15:26.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1016057 9.35 MB/sec execute 468 sec latency 15.807 ms 2024-05-08T23:15:27.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1016916 9.33 MB/sec execute 469 sec latency 27.840 ms 2024-05-08T23:15:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:27 smithi086 ceph-mon[78610]: pgmap v362: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:27 smithi053 ceph-mon[77587]: pgmap v362: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:27 smithi046 ceph-mon[36898]: pgmap v362: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:28.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1018947 9.34 MB/sec execute 470 sec latency 23.946 ms 2024-05-08T23:15:29.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1019603 9.33 MB/sec execute 471 sec latency 399.277 ms 2024-05-08T23:15:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:29 smithi086 ceph-mon[78610]: pgmap v363: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:29 smithi053 ceph-mon[77587]: pgmap v363: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:29 smithi046 ceph-mon[36898]: pgmap v363: 129 pgs: 129 active+clean; 586 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-05-08T23:15:30.912 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1020043 9.31 MB/sec execute 472 sec latency 66.261 ms 2024-05-08T23:15:31.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1021934 9.31 MB/sec execute 473 sec latency 24.530 ms 2024-05-08T23:15:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:31 smithi086 ceph-mon[78610]: pgmap v364: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-08T23:15:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:31 smithi053 ceph-mon[77587]: pgmap v364: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-08T23:15:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:31 smithi046 ceph-mon[36898]: pgmap v364: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-08T23:15:32.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1023291 9.31 MB/sec execute 474 sec latency 19.100 ms 2024-05-08T23:15:33.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1024668 9.30 MB/sec execute 475 sec latency 29.819 ms 2024-05-08T23:15:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:33 smithi086 ceph-mon[78610]: pgmap v365: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-05-08T23:15:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:33 smithi053 ceph-mon[77587]: pgmap v365: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-05-08T23:15:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:33 smithi046 ceph-mon[36898]: pgmap v365: 129 pgs: 129 active+clean; 597 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-05-08T23:15:34.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1026693 9.31 MB/sec execute 476 sec latency 30.233 ms 2024-05-08T23:15:35.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1027560 9.30 MB/sec execute 477 sec latency 30.922 ms 2024-05-08T23:15:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:35 smithi086 ceph-mon[78610]: pgmap v366: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-05-08T23:15:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:15:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:35 smithi053 ceph-mon[77587]: pgmap v366: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-05-08T23:15:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:15:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:35 smithi046 ceph-mon[36898]: pgmap v366: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 255 op/s 2024-05-08T23:15:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:15:36.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1029606 9.30 MB/sec execute 478 sec latency 30.206 ms 2024-05-08T23:15:37.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1030655 9.30 MB/sec execute 479 sec latency 15.104 ms 2024-05-08T23:15:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:37 smithi086 ceph-mon[78610]: pgmap v367: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 242 op/s 2024-05-08T23:15:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:37 smithi053 ceph-mon[77587]: pgmap v367: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 242 op/s 2024-05-08T23:15:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:37 smithi046 ceph-mon[36898]: pgmap v367: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 242 op/s 2024-05-08T23:15:38.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1031926 9.29 MB/sec execute 480 sec latency 71.380 ms 2024-05-08T23:15:39.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:38 smithi086 ceph-mon[78610]: pgmap v368: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 223 op/s 2024-05-08T23:15:39.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:38 smithi053 ceph-mon[77587]: pgmap v368: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 223 op/s 2024-05-08T23:15:39.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:38 smithi046 ceph-mon[36898]: pgmap v368: 129 pgs: 129 active+clean; 585 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 223 op/s 2024-05-08T23:15:39.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1033849 9.29 MB/sec execute 481 sec latency 15.914 ms 2024-05-08T23:15:40.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1034965 9.28 MB/sec execute 482 sec latency 25.525 ms 2024-05-08T23:15:41.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:41 smithi086 ceph-mon[78610]: pgmap v369: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.3 MiB/s wr, 289 op/s 2024-05-08T23:15:41.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:41 smithi053 ceph-mon[77587]: pgmap v369: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.3 MiB/s wr, 289 op/s 2024-05-08T23:15:41.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:41 smithi046 ceph-mon[36898]: pgmap v369: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.3 MiB/s wr, 289 op/s 2024-05-08T23:15:41.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1036974 9.29 MB/sec execute 483 sec latency 25.647 ms 2024-05-08T23:15:42.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1037868 9.28 MB/sec execute 484 sec latency 52.418 ms 2024-05-08T23:15:43.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:43 smithi086 ceph-mon[78610]: pgmap v370: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 217 op/s 2024-05-08T23:15:43.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:43 smithi053 ceph-mon[77587]: pgmap v370: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 217 op/s 2024-05-08T23:15:43.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:43 smithi046 ceph-mon[36898]: pgmap v370: 129 pgs: 129 active+clean; 590 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 217 op/s 2024-05-08T23:15:43.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1039761 9.27 MB/sec execute 485 sec latency 17.646 ms 2024-05-08T23:15:44.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1041145 9.28 MB/sec execute 486 sec latency 18.716 ms 2024-05-08T23:15:45.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:45 smithi086 ceph-mon[78610]: pgmap v371: 129 pgs: 129 active+clean; 600 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-05-08T23:15:45.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:45 smithi053 ceph-mon[77587]: pgmap v371: 129 pgs: 129 active+clean; 600 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-05-08T23:15:45.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:45 smithi046 ceph-mon[36898]: pgmap v371: 129 pgs: 129 active+clean; 600 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-05-08T23:15:45.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1042700 9.27 MB/sec execute 487 sec latency 24.310 ms 2024-05-08T23:15:46.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1044536 9.28 MB/sec execute 488 sec latency 15.168 ms 2024-05-08T23:15:47.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:47 smithi086 ceph-mon[78610]: pgmap v372: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 155 op/s 2024-05-08T23:15:47.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:47 smithi053 ceph-mon[77587]: pgmap v372: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 155 op/s 2024-05-08T23:15:47.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:47 smithi046 ceph-mon[36898]: pgmap v372: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 155 op/s 2024-05-08T23:15:47.913 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1045863 9.27 MB/sec execute 489 sec latency 14.797 ms 2024-05-08T23:15:48.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1047926 9.28 MB/sec execute 490 sec latency 35.946 ms 2024-05-08T23:15:49.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:49 smithi086 ceph-mon[78610]: pgmap v373: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-05-08T23:15:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:49 smithi053 ceph-mon[77587]: pgmap v373: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-05-08T23:15:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:49 smithi046 ceph-mon[36898]: pgmap v373: 129 pgs: 129 active+clean; 592 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-05-08T23:15:49.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1048791 9.26 MB/sec execute 491 sec latency 16.147 ms 2024-05-08T23:15:50.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1050931 9.27 MB/sec execute 492 sec latency 31.518 ms 2024-05-08T23:15:51.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:51 smithi086 ceph-mon[78610]: pgmap v374: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:15:51.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:51 smithi053 ceph-mon[77587]: pgmap v374: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:15:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:51 smithi046 ceph-mon[36898]: pgmap v374: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-05-08T23:15:51.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1052041 9.26 MB/sec execute 493 sec latency 24.162 ms 2024-05-08T23:15:52.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1053892 9.26 MB/sec execute 494 sec latency 26.549 ms 2024-05-08T23:15:53.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 -- 172.21.15.46:0/4110234312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101a00 msgr2=0x7f432c105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 --2- 172.21.15.46:0/4110234312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101a00 0x7f432c105a40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4318009990 tx=0x7f4318009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 -- 172.21.15.46:0/4110234312 shutdown_connections 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 --2- 172.21.15.46:0/4110234312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 0x7f432c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 --2- 172.21.15.46:0/4110234312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c101050 0x7f432c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 --2- 172.21.15.46:0/4110234312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101a00 0x7f432c105a40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 -- 172.21.15.46:0/4110234312 >> 172.21.15.46:0/4110234312 conn(0x7f432c0fc8b0 msgr2=0x7f432c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 -- 172.21.15.46:0/4110234312 shutdown_connections 2024-05-08T23:15:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.656+0000 7f4334000700 1 -- 172.21.15.46:0/4110234312 wait complete. 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.657+0000 7f4334000700 1 Processor -- start 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.657+0000 7f4334000700 1 -- start start 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 0x7f432c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 0x7f432c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c19e8c0 con 0x7f432c106170 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c19ea30 con 0x7f432c19de40 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4334000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c19ebf0 con 0x7f432c101050 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48768/0 (socket says 172.21.15.46:48768) 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4331600700 1 -- 172.21.15.46:0/2140337299 learned_addr learned my addr 172.21.15.46:0/2140337299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4330c00700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 0x7f432c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.658+0000 7f4332000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 0x7f432c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 -- 172.21.15.46:0/2140337299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 msgr2=0x7f432c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 0x7f432c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 -- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 msgr2=0x7f432c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 0x7f432c199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 -- 172.21.15.46:0/2140337299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180095f0 con 0x7f432c101050 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4332000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 0x7f432c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4331600700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f432000e9c0 tx=0x7f432000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f432a000700 1 -- 172.21.15.46:0/2140337299 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43200097b0 con 0x7f432c101050 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f432a000700 1 -- 172.21.15.46:0/2140337299 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4320004cf0 con 0x7f432c101050 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f432a000700 1 -- 172.21.15.46:0/2140337299 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320005600 con 0x7f432c101050 2024-05-08T23:15:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c1a6150 con 0x7f432c101050 2024-05-08T23:15:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.659+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c1a65e0 con 0x7f432c101050 2024-05-08T23:15:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.663+0000 7f432a000700 1 -- 172.21.15.46:0/2140337299 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f432000cc60 con 0x7f432c101050 2024-05-08T23:15:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.664+0000 7f4347a63b80 1 Processor -- start 2024-05-08T23:15:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.664+0000 7f4347a63b80 1 -- start start 2024-05-08T23:15:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f432a000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 0x7f43140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a693720 0x555a0a68d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a68dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a5e4150 con 0x555a0a68d890 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a5e1b10 con 0x555a0a693720 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4347a63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a5dd400 con 0x555a0a692e00 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f432a000700 1 -- 172.21.15.46:0/2140337299 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4320014070 con 0x7f432c101050 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48772/0 (socket says 172.21.15.46:48772) 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 -- 172.21.15.46:0/1196247188 learned_addr learned my addr 172.21.15.46:0/1196247188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4332000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 0x7f43140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4330c00700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a693720 0x555a0a68d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 -- 172.21.15.46:0/1196247188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a693720 msgr2=0x555a0a68d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a693720 0x555a0a68d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 -- 172.21.15.46:0/1196247188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 msgr2=0x555a0a68dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4332000700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a68dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a68dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4331600700 1 -- 172.21.15.46:0/1196247188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0a45e3b0 con 0x555a0a692e00 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.665+0000 7f4332000700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a68dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.666+0000 7f4331600700 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f43200a2000 tx=0x7f4320099d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59611636046f6fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.666+0000 7f4332000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 0x7f43140a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4324012740 tx=0x7f4324009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.666+0000 7f431e600700 1 -- 172.21.15.46:0/1196247188 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320093360 con 0x555a0a692e00 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.666+0000 7f431e600700 1 -- 172.21.15.46:0/1196247188 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f432009a0b0 con 0x555a0a692e00 2024-05-08T23:15:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.666+0000 7f431e600700 1 -- 172.21.15.46:0/1196247188 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43200860b0 con 0x555a0a692e00 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- 172.21.15.46:0/1196247188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 msgr2=0x555a0a6931e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f43200a2000 tx=0x7f4320099d20 comp rx=0 tx=0).stop 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- 172.21.15.46:0/1196247188 shutdown_connections 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a68dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a6931e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1196247188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a693720 0x555a0a68d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- 172.21.15.46:0/1196247188 >> 172.21.15.46:0/1196247188 conn(0x555a0a5eb210 msgr2=0x555a0a687df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- 172.21.15.46:0/1196247188 shutdown_connections 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- 172.21.15.46:0/1196247188 wait complete. 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.667+0000 7f4347a63b80 1 -- start start 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 Processor -- start 2024-05-08T23:15:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a696ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a697550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a579560 con 0x555a0a68d890 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a579200 con 0x555a0a692e00 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a4606a0 con 0x555a0a693720 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4347a63b80 10 client.0 _ll_register_callbacks cb 0x555a0a582b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4330c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4331600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a696ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4332000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a697550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4330c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37186/0 (socket says 172.21.15.46:37186) 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.668+0000 7f4330c00700 1 -- 172.21.15.46:0/719767755 learned_addr learned my addr 172.21.15.46:0/719767755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 -- 172.21.15.46:0/719767755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 msgr2=0x555a0a697550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a697550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 -- 172.21.15.46:0/719767755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 msgr2=0x555a0a696ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a696ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 -- 172.21.15.46:0/719767755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c1a65e0 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4332000700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a697550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4330c00700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4318009990 tx=0x7f4318005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f76a06f4c69335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4312a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.669+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431801d070 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a0a536270 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4318004500 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f431800bee0 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 10 client.28799 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555a0a40ab60 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a0a536270 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43180245a0 con 0x555a0a692e00 2024-05-08T23:15:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 10 client.28799 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 10 client.28799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.670+0000 7f4347a63b80 20 client.28799 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6a71d0 0x555a0a6a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4347a63b80 4 client.28799 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a0a65c810 con 0x555a0a6a71d0 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4331600700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6a71d0 0x555a0a6a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4331600700 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6a71d0 0x555a0a6a95f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a711e8a87833fa server_cookie=9ed0b0fb32e649a6 in_seq=0 out_seq=0 2024-05-08T23:15:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.671+0000 7f4312a00700 10 client.28799 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4312a00700 1 -- 172.21.15.46:0/719767755 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a0a65c810 con 0x555a0a6a71d0 2024-05-08T23:15:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4312a00700 10 client.28799 handle_command_reply: tid=0 2024-05-08T23:15:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 client.28799 shutdown 2024-05-08T23:15:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 msgr2=0x555a0a697010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4318009990 tx=0x7f4318005750 comp rx=0 tx=0).stop 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 shutdown_connections 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a697550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6a71d0 0x555a0a6a95f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a68d890 0x555a0a696ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 --2- 172.21.15.46:0/719767755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a692e00 0x555a0a697010 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 >> 172.21.15.46:0/719767755 conn(0x555a0a5eb210 msgr2=0x555a0a6875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4312a00700 0 client.28799 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 shutdown_connections 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.672+0000 7f4347a63b80 1 -- 172.21.15.46:0/719767755 wait complete. 2024-05-08T23:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.673+0000 7f4347a63b80 20 client.28799 trim_cache size 0 max 16384 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 Processor -- start 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 -- start start 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a68d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a692e00 0x555a0a687d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a689020 con 0x555a0a692e00 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a695040 con 0x555a0a68d890 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4347a63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a682a90 con 0x555a0a693720 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4332000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4332000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48794/0 (socket says 172.21.15.46:48794) 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4331600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a68d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4332000700 1 -- 172.21.15.46:0/4007886361 learned_addr learned my addr 172.21.15.46:0/4007886361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.692+0000 7f4330c00700 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a692e00 0x555a0a687d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 -- 172.21.15.46:0/4007886361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 msgr2=0x555a0a68d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a68d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 -- 172.21.15.46:0/4007886361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a692e00 msgr2=0x555a0a687d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a692e00 0x555a0a687d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 -- 172.21.15.46:0/4007886361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0a5a3560 con 0x555a0a693720 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4331600700 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a68d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f4332000700 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f43240165f0 tx=0x7f4324016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f585f51485262368 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f431f000700 1 -- 172.21.15.46:0/4007886361 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432400e040 con 0x555a0a693720 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f431f000700 1 -- 172.21.15.46:0/4007886361 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4324022ae0 con 0x555a0a693720 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.693+0000 7f431f000700 1 -- 172.21.15.46:0/4007886361 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324007b60 con 0x555a0a693720 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 -- 172.21.15.46:0/4007886361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 msgr2=0x555a0a6922b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f43240165f0 tx=0x7f4324016000 comp rx=0 tx=0).stop 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 -- 172.21.15.46:0/4007886361 shutdown_connections 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a693720 0x555a0a6922b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a68d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 --2- 172.21.15.46:0/4007886361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a692e00 0x555a0a687d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 -- 172.21.15.46:0/4007886361 >> 172.21.15.46:0/4007886361 conn(0x555a0a5eb210 msgr2=0x555a0a691ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 -- 172.21.15.46:0/4007886361 shutdown_connections 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.694+0000 7f4347a63b80 1 -- 172.21.15.46:0/4007886361 wait complete. 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 -- start start 2024-05-08T23:15:53.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 Processor -- start 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a69e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 0x555a0a68edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a68f490 con 0x555a0a693720 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a68f600 con 0x555a0a68d890 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0a68f7c0 con 0x555a0a692e00 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.695+0000 7f4347a63b80 10 client.0 _ll_register_callbacks cb 0x555a0a582b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4330c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a69e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37194/0 (socket says 172.21.15.46:37194) 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 -- 172.21.15.46:0/1697380581 learned_addr learned my addr 172.21.15.46:0/1697380581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4332000700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 0x555a0a68edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 -- 172.21.15.46:0/1697380581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 msgr2=0x555a0a69e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a69e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 -- 172.21.15.46:0/1697380581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 msgr2=0x555a0a68edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 0x555a0a68edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 -- 172.21.15.46:0/1697380581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0a45e3b0 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4332000700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 0x555a0a68edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4331600700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f43200965b0 tx=0x7f432000efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c68179e7a2d4e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4312a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.696+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43200cd230 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f43200cae40 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f432009a0b0 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43200ca970 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 10 client.28805 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f4324012df0 con 0x555a0a68d890 2024-05-08T23:15:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f432009a0b0 con 0x555a0a68d890 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43200b4d90 con 0x555a0a68d890 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 10 client.28805 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 10 client.28805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 20 client.28805 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6fe520 0x555a0a700910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 4 client.28805 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.697+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a0a5d5ec0 con 0x555a0a6fe520 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4330c00700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6fe520 0x555a0a700910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4330c00700 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6fe520 0x555a0a700910 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e542977c58c7fb24 server_cookie=341ac5b842eb4994 in_seq=0 out_seq=0 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4312a00700 10 client.28805 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4312a00700 1 -- 172.21.15.46:0/1697380581 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555a0a5d5ec0 con 0x555a0a6fe520 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4312a00700 10 client.28805 handle_command_reply: tid=0 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.698+0000 7f4347a63b80 1 client.28805 shutdown 2024-05-08T23:15:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 msgr2=0x555a0a69e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f43200965b0 tx=0x7f432000efa0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 shutdown_connections 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555a0a6fe520 0x555a0a700910 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a0a693720 0x555a0a68edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a0a68d890 0x555a0a69e170 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4312a00700 0 client.28805 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 --2- 172.21.15.46:0/1697380581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a0a692e00 0x555a0a69e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 >> 172.21.15.46:0/1697380581 conn(0x555a0a5eb210 msgr2=0x555a0a695c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 shutdown_connections 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 1 -- 172.21.15.46:0/1697380581 wait complete. 2024-05-08T23:15:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.699+0000 7f4347a63b80 20 client.28805 trim_cache size 0 max 16384 2024-05-08T23:15:53.703 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:15:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:15:53.703 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 msgr2=0x7f43140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 0x7f43140a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4324012740 tx=0x7f4324009920 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 msgr2=0x7f432c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f432000e9c0 tx=0x7f432000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 shutdown_connections 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f432c19de40 0x7f432c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c101050 0x7f432c199560 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43140a1290 0x7f43140a3750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 --2- 172.21.15.46:0/2140337299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c106170 0x7f432c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 >> 172.21.15.46:0/2140337299 conn(0x7f432c0fc8b0 msgr2=0x7f432c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 shutdown_connections 2024-05-08T23:15:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.700+0000 7f4334000700 1 -- 172.21.15.46:0/2140337299 wait complete. 2024-05-08T23:15:53.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:53 smithi086 ceph-mon[78610]: pgmap v375: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-05-08T23:15:53.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:53 smithi046 ceph-mon[36898]: pgmap v375: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-05-08T23:15:53.718 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:15:53.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:15:53.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:53 smithi053 ceph-mon[77587]: pgmap v375: 129 pgs: 129 active+clean; 594 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-05-08T23:15:53.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1055309 9.26 MB/sec execute 495 sec latency 14.592 ms 2024-05-08T23:15:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.940+0000 7fafaa200700 1 -- 172.21.15.46:0/891206030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 msgr2=0x7fafa410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.940+0000 7fafaa200700 1 --2- 172.21.15.46:0/891206030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa410d850 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b0a0 tx=0x7faf9c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 -- 172.21.15.46:0/891206030 shutdown_connections 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 --2- 172.21.15.46:0/891206030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa410d850 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 --2- 172.21.15.46:0/891206030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4101f70 0x7fafa4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 --2- 172.21.15.46:0/891206030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4102890 0x7fafa410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 -- 172.21.15.46:0/891206030 >> 172.21.15.46:0/891206030 conn(0x7fafa40fb770 msgr2=0x7fafa40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 -- 172.21.15.46:0/891206030 shutdown_connections 2024-05-08T23:15:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.941+0000 7fafaa200700 1 -- 172.21.15.46:0/891206030 wait complete. 2024-05-08T23:15:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.942+0000 7fafaa200700 1 Processor -- start 2024-05-08T23:15:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.942+0000 7fafaa200700 1 -- start start 2024-05-08T23:15:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4101f70 0x7fafa40ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4102890 0x7fafa4100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41a5770 con 0x7fafa4102890 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41a58e0 con 0x7fafa4101f70 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafaa200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41a5aa0 con 0x7fafa410b390 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4102890 0x7fafa4100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48830/0 (socket says 172.21.15.46:48830) 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 -- 172.21.15.46:0/1850770007 learned_addr learned my addr 172.21.15.46:0/1850770007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3400700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4101f70 0x7fafa40ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 -- 172.21.15.46:0/1850770007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4101f70 msgr2=0x7fafa40ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4101f70 0x7fafa40ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 -- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4102890 msgr2=0x7fafa4100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4102890 0x7fafa4100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.943+0000 7fafa3e00700 1 -- 172.21.15.46:0/1850770007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c009d00 con 0x7fafa410b390 2024-05-08T23:15:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7fafa3e00700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b070 tx=0x7faf9c007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00e040 con 0x7fafa410b390 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf9c01d470 con 0x7fafa410b390 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa41a5d20 con 0x7fafa410b390 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0052a0 con 0x7fafa410b390 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.944+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa41a61e0 con 0x7fafa410b390 2024-05-08T23:15:53.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.945+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa41064c0 con 0x7fafa410b390 2024-05-08T23:15:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.947+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf9c01d850 con 0x7fafa410b390 2024-05-08T23:15:53.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.948+0000 7faf9be00700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 0x7faf880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.948+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf9c0ce4f0 con 0x7fafa410b390 2024-05-08T23:15:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.948+0000 7fafa3400700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 0x7faf880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:53.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.949+0000 7fafa3400700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 0x7faf880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf94009c60 tx=0x7faf94009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:53.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:53.952+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf9c097560 con 0x7fafa410b390 2024-05-08T23:15:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.284+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafa4101140 con 0x7fafa410b390 2024-05-08T23:15:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.285+0000 7faf9be00700 1 -- 172.21.15.46:0/1850770007 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7faf9c01d5e0 con 0x7fafa410b390 2024-05-08T23:15:54.285 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:15:54.285 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:15:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 msgr2=0x7faf880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 0x7faf880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf94009c60 tx=0x7faf94009260 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 msgr2=0x7fafa4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b070 tx=0x7faf9c007930 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 shutdown_connections 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa410b390 0x7fafa4100640 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa4101f70 0x7fafa40ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf880a1290 0x7faf880a3750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 --2- 172.21.15.46:0/1850770007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafa4102890 0x7fafa4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 >> 172.21.15.46:0/1850770007 conn(0x7fafa40fb770 msgr2=0x7fafa40fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 shutdown_connections 2024-05-08T23:15:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.288+0000 7fafaa200700 1 -- 172.21.15.46:0/1850770007 wait complete. 2024-05-08T23:15:54.289 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:15:54.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:15:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.549+0000 7fa59e800700 1 -- 172.21.15.46:0/1586827246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5981028c0 msgr2=0x7fa59810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.549+0000 7fa59e800700 1 --2- 172.21.15.46:0/1586827246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5981028c0 0x7fa59810ada0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098f0 tx=0x7fa584009c00 comp rx=0 tx=0).stop 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 -- 172.21.15.46:0/1586827246 shutdown_connections 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 --2- 172.21.15.46:0/1586827246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa59810b3c0 0x7fa59810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 --2- 172.21.15.46:0/1586827246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598101fa0 0x7fa598102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 --2- 172.21.15.46:0/1586827246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5981028c0 0x7fa59810ada0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 -- 172.21.15.46:0/1586827246 >> 172.21.15.46:0/1586827246 conn(0x7fa5980fb760 msgr2=0x7fa5980fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 -- 172.21.15.46:0/1586827246 shutdown_connections 2024-05-08T23:15:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.550+0000 7fa59e800700 1 -- 172.21.15.46:0/1586827246 wait complete. 2024-05-08T23:15:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 Processor -- start 2024-05-08T23:15:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 -- start start 2024-05-08T23:15:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 0x7fa5980ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5981028c0 0x7fa598100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5981a5720 con 0x7fa5981028c0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5981a5890 con 0x7fa59810b3c0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.551+0000 7fa59e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5981a5a50 con 0x7fa598101fa0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa596a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5981028c0 0x7fa598100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 0x7fa5980ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37240/0 (socket says 172.21.15.46:37240) 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 -- 172.21.15.46:0/4231566301 learned_addr learned my addr 172.21.15.46:0/4231566301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 -- 172.21.15.46:0/4231566301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 msgr2=0x7fa5980ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 0x7fa5980ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 -- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5981028c0 msgr2=0x7fa598100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5981028c0 0x7fa598100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597e00700 1 -- 172.21.15.46:0/4231566301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880095f0 con 0x7fa59810b3c0 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.552+0000 7fa597400700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 0x7fa5980ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa597e00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c370 tx=0x7fa59000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa58fe00700 1 -- 172.21.15.46:0/4231566301 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590017070 con 0x7fa59810b3c0 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa590009c40 con 0x7fa59810b3c0 2024-05-08T23:15:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa58fe00700 1 -- 172.21.15.46:0/4231566301 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59000f040 con 0x7fa59810b3c0 2024-05-08T23:15:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa58fe00700 1 -- 172.21.15.46:0/4231566301 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000e050 con 0x7fa59810b3c0 2024-05-08T23:15:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.553+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584009520 con 0x7fa59810b3c0 2024-05-08T23:15:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.557+0000 7fa58fe00700 1 -- 172.21.15.46:0/4231566301 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa59000cb60 con 0x7fa59810b3c0 2024-05-08T23:15:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa5b23f5b80 1 Processor -- start 2024-05-08T23:15:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa5b23f5b80 1 -- start start 2024-05-08T23:15:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa58fe00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 0x7fa57c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa58fe00700 1 -- 172.21.15.46:0/4231566301 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa5900cc990 con 0x7fa59810b3c0 2024-05-08T23:15:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f6e00 0x556b383f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f7720 0x556b383f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.558+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b38348150 con 0x556b383f7720 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b38345b10 con 0x556b383f6e00 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa596a00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 0x7fa57c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b38341400 con 0x556b383f1890 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa597e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa596a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f7720 0x556b383f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa597e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48858/0 (socket says 172.21.15.46:48858) 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa597e00700 1 -- 172.21.15.46:0/2219847746 learned_addr learned my addr 172.21.15.46:0/2219847746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.559+0000 7fa597400700 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f6e00 0x556b383f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 -- 172.21.15.46:0/2219847746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f6e00 msgr2=0x556b383f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f6e00 0x556b383f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 -- 172.21.15.46:0/2219847746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f7720 msgr2=0x556b383f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f7720 0x556b383f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 -- 172.21.15.46:0/2219847746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b381c23b0 con 0x556b383f1890 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa596a00700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 0x7fa57c0a37d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098c0 tx=0x7fa584010f00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa597e00700 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a1580 tx=0x7fa5900994f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7655a4e208571fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa58cc00700 1 -- 172.21.15.46:0/2219847746 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b90d0 con 0x556b383f1890 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa58cc00700 1 -- 172.21.15.46:0/2219847746 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5900092e0 con 0x556b383f1890 2024-05-08T23:15:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.560+0000 7fa58cc00700 1 -- 172.21.15.46:0/2219847746 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590003a00 con 0x556b383f1890 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2219847746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 msgr2=0x556b383f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a1580 tx=0x7fa5900994f0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2219847746 shutdown_connections 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383f1cd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f6e00 0x556b383f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2219847746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f7720 0x556b383f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2219847746 >> 172.21.15.46:0/2219847746 conn(0x556b3834f210 msgr2=0x556b383ebdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.561+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2219847746 shutdown_connections 2024-05-08T23:15:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.562+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2219847746 wait complete. 2024-05-08T23:15:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.562+0000 7fa5b23f5b80 1 -- start start 2024-05-08T23:15:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.563+0000 7fa5b23f5b80 1 Processor -- start 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.563+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.563+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b383fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.563+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b382dd560 con 0x556b383f6e00 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b382dd200 con 0x556b383f7720 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b381c46a0 con 0x556b383f1890 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa5b23f5b80 10 client.0 _ll_register_callbacks cb 0x556b382e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37264/0 (socket says 172.21.15.46:37264) 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597e00700 1 -- 172.21.15.46:0/2160730293 learned_addr learned my addr 172.21.15.46:0/2160730293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 -- 172.21.15.46:0/2160730293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 msgr2=0x556b383fb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 -- 172.21.15.46:0/2160730293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 msgr2=0x556b383fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b383fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597400700 1 -- 172.21.15.46:0/2160730293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584009520 con 0x556b383f1890 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.564+0000 7fa597e00700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa597400700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383fab30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa588012cf0 tx=0x7fa58800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1444372643f0fb12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa581600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588017070 con 0x556b383f1890 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa58800fa10 con 0x556b383f1890 2024-05-08T23:15:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b3829a270 con 0x556b383f1890 2024-05-08T23:15:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58800fbd0 con 0x556b383f1890 2024-05-08T23:15:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa5b23f5b80 10 client.28918 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.565+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556b3816eb60 con 0x556b383f1890 2024-05-08T23:15:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b3829a270 con 0x556b383f1890 2024-05-08T23:15:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa588003aa0 con 0x556b383f1890 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 10 client.28918 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 10 client.28918 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 20 client.28918 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b38413af0 0x556b38415f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 4 client.28918 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b383c0810 con 0x556b38413af0 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.566+0000 7fa596a00700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b38413af0 0x556b38415f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.567+0000 7fa596a00700 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b38413af0 0x556b38415f10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f46e6de2aa2a77d5 server_cookie=c45070a368e751f9 in_seq=0 out_seq=0 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.567+0000 7fa581600700 10 client.28918 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa581600700 1 -- 172.21.15.46:0/2160730293 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b383c0810 con 0x556b38413af0 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa581600700 10 client.28918 handle_command_reply: tid=0 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 client.28918 shutdown 2024-05-08T23:15:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 msgr2=0x556b383fab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383fab30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa588012cf0 tx=0x7fa58800f3c0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 shutdown_connections 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b38413af0 0x556b38415f10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b383fab30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa581600700 0 client.28918 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2160730293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b383fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 >> 172.21.15.46:0/2160730293 conn(0x556b3834f210 msgr2=0x556b383eb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 shutdown_connections 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.568+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2160730293 wait complete. 2024-05-08T23:15:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.569+0000 7fa5b23f5b80 20 client.28918 trim_cache size 0 max 16384 2024-05-08T23:15:54.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.584+0000 7fa5b23f5b80 1 Processor -- start 2024-05-08T23:15:54.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.584+0000 7fa5b23f5b80 1 -- start start 2024-05-08T23:15:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f1890 0x556b383f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b383f8b10 con 0x556b383f1890 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b3841b120 con 0x556b383f7720 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b383ed7e0 con 0x556b383f6e00 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa597400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f1890 0x556b383f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48874/0 (socket says 172.21.15.46:48874) 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 -- 172.21.15.46:0/2998091722 learned_addr learned my addr 172.21.15.46:0/2998091722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa597e00700 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 -- 172.21.15.46:0/2998091722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 msgr2=0x556b383ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 -- 172.21.15.46:0/2998091722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f1890 msgr2=0x556b383f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f1890 0x556b383f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.585+0000 7fa596a00700 1 -- 172.21.15.46:0/2998091722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b38307560 con 0x556b383f6e00 2024-05-08T23:15:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.586+0000 7fa596a00700 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa584009810 tx=0x7fa58401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de5c6aa1916c83c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.586+0000 7fa58d600700 1 -- 172.21.15.46:0/2998091722 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584030460 con 0x556b383f6e00 2024-05-08T23:15:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.586+0000 7fa58d600700 1 -- 172.21.15.46:0/2998091722 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58402a070 con 0x556b383f6e00 2024-05-08T23:15:54.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.586+0000 7fa58d600700 1 -- 172.21.15.46:0/2998091722 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400b7c0 con 0x556b383f6e00 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2998091722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 msgr2=0x556b383eda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa584009810 tx=0x7fa58401c000 comp rx=0 tx=0).stop 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2998091722 shutdown_connections 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b383ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f1890 0x556b383f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/2998091722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f6e00 0x556b383eda00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2998091722 >> 172.21.15.46:0/2998091722 conn(0x556b3834f210 msgr2=0x556b383f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2998091722 shutdown_connections 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.587+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/2998091722 wait complete. 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 -- start start 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 Processor -- start 2024-05-08T23:15:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b384089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b38408f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b38409460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.588+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b38469080 con 0x556b383f6e00 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b384691f0 con 0x556b383f7720 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa5b23f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b384693b0 con 0x556b383f1890 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa5b23f5b80 10 client.0 _ll_register_callbacks cb 0x556b382e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa596a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b38408f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b38409460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b384089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b38409460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37284/0 (socket says 172.21.15.46:37284) 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597e00700 1 -- 172.21.15.46:0/1180365902 learned_addr learned my addr 172.21.15.46:0/1180365902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 -- 172.21.15.46:0/1180365902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 msgr2=0x556b38409460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b38409460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 -- 172.21.15.46:0/1180365902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 msgr2=0x556b38408f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b38408f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa597400700 1 -- 172.21.15.46:0/1180365902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58401f040 con 0x556b383f1890 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.589+0000 7fa596a00700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b38408f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa597400700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b384089e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa588012420 tx=0x7fa58800fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55f9820f87e377c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588017070 con 0x556b383f1890 2024-05-08T23:15:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5900092e0 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa588000e40 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa588010500 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa5b23f5b80 10 client.28924 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556b381c6cd0 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5900092e0 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.590+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa588010740 con 0x556b383f1890 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 10 client.28924 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 10 client.28924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 20 client.28924 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b3846ab50 0x556b3846cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 4 client.28924 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556b38339ec0 con 0x556b3846ab50 2024-05-08T23:15:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa596a00700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b3846ab50 0x556b3846cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa596a00700 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b3846ab50 0x556b3846cf70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c07f0345236696e3 server_cookie=504e66fe555f5a0a in_seq=0 out_seq=0 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.591+0000 7fa581600700 10 client.28924 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa581600700 1 -- 172.21.15.46:0/1180365902 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556b38339ec0 con 0x556b3846ab50 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa581600700 10 client.28924 handle_command_reply: tid=0 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 client.28924 shutdown 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 msgr2=0x556b384089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b384089e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa588012420 tx=0x7fa58800fc40 comp rx=0 tx=0).stop 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 shutdown_connections 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x556b3846ab50 0x556b3846cf70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556b383f7720 0x556b38409460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b383f1890 0x556b384089e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa581600700 0 client.28924 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:15:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 --2- 172.21.15.46:0/1180365902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b383f6e00 0x556b38408f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 >> 172.21.15.46:0/1180365902 conn(0x556b3834f210 msgr2=0x556b383f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 shutdown_connections 2024-05-08T23:15:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 1 -- 172.21.15.46:0/1180365902 wait complete. 2024-05-08T23:15:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.592+0000 7fa5b23f5b80 20 client.28924 trim_cache size 0 max 16384 2024-05-08T23:15:54.596 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 msgr2=0x7fa57c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 0x7fa57c0a37d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098c0 tx=0x7fa584010f00 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 msgr2=0x7fa598100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c370 tx=0x7fa59000c680 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 shutdown_connections 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59810b3c0 0x7fa598100670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa598101fa0 0x7fa5980ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa57c0a1310 0x7fa57c0a37d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 --2- 172.21.15.46:0/4231566301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5981028c0 0x7fa598100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 >> 172.21.15.46:0/4231566301 conn(0x7fa5980fb760 msgr2=0x7fa5980fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 shutdown_connections 2024-05-08T23:15:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.593+0000 7fa59e800700 1 -- 172.21.15.46:0/4231566301 wait complete. 2024-05-08T23:15:54.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:54 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1850770007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:15:54.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:54 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1850770007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:54 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1850770007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.833+0000 7f676fe00700 1 -- 172.21.15.46:0/1664621255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 msgr2=0x7f67701012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.833+0000 7f676fe00700 1 --2- 172.21.15.46:0/1664621255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f67701012a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6760009990 tx=0x7f6760009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 -- 172.21.15.46:0/1664621255 shutdown_connections 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 --2- 172.21.15.46:0/1664621255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6770105fe0 0x7f677010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 --2- 172.21.15.46:0/1664621255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f67701012a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 --2- 172.21.15.46:0/1664621255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770101870 0x7f67701058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 -- 172.21.15.46:0/1664621255 >> 172.21.15.46:0/1664621255 conn(0x7f67700fc740 msgr2=0x7f67700feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 -- 172.21.15.46:0/1664621255 shutdown_connections 2024-05-08T23:15:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.834+0000 7f676fe00700 1 -- 172.21.15.46:0/1664621255 wait complete. 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.835+0000 7f676fe00700 1 Processor -- start 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.835+0000 7f676fe00700 1 -- start start 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770105fe0 0x7f6770199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 0x7f677019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677019e940 con 0x7f677019dec0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677019eab0 con 0x7f6770105fe0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677019ec70 con 0x7f6770100ec0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48906/0 (socket says 172.21.15.46:48906) 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770105fe0 0x7f6770199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676ea00700 1 -- 172.21.15.46:0/519987131 learned_addr learned my addr 172.21.15.46:0/519987131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.836+0000 7f676f400700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 0x7f677019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 -- 172.21.15.46:0/519987131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770105fe0 msgr2=0x7f6770199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770105fe0 0x7f6770199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 -- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 msgr2=0x7f677019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 0x7f677019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 -- 172.21.15.46:0/519987131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600095f0 con 0x7f6770100ec0 2024-05-08T23:15:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676f400700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 0x7f677019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676ea00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6760000c00 tx=0x7f6760005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f6767400700 1 -- 172.21.15.46:0/519987131 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676001d070 con 0x7f6770100ec0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f6767400700 1 -- 172.21.15.46:0/519987131 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6760022470 con 0x7f6770100ec0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f6767400700 1 -- 172.21.15.46:0/519987131 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760017650 con 0x7f6770100ec0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.837+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67701a6170 con 0x7f6770100ec0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.838+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701a6540 con 0x7f6770100ec0 2024-05-08T23:15:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.840+0000 7f6767400700 1 -- 172.21.15.46:0/519987131 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f67600225e0 con 0x7f6770100ec0 2024-05-08T23:15:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 Processor -- start 2024-05-08T23:15:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 -- start start 2024-05-08T23:15:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f6767400700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 0x7f67540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da315150 con 0x55b8da3c7520 2024-05-08T23:15:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f6767400700 1 -- 172.21.15.46:0/519987131 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67600ce440 con 0x7f6770100ec0 2024-05-08T23:15:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da312b10 con 0x55b8da3c6b60 2024-05-08T23:15:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.843+0000 7f67894a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da30e400 con 0x55b8da3cc870 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676f400700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 0x7f67540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37306/0 (socket says 172.21.15.46:37306) 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 -- 172.21.15.46:0/2615283464 learned_addr learned my addr 172.21.15.46:0/2615283464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 -- 172.21.15.46:0/2615283464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 msgr2=0x55b8da3ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676f400700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 -- 172.21.15.46:0/2615283464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 msgr2=0x55b8da3d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676e000700 1 -- 172.21.15.46:0/2615283464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8da18f3b0 con 0x55b8da3c6b60 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676ea00700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.844+0000 7f676f400700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.845+0000 7f676e000700 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f675c0046c0 tx=0x7f675c00b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74754a7faf08c8b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.845+0000 7f676f400700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 0x7f67540a37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6768009e90 tx=0x7f6768009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.845+0000 7f675be00700 1 -- 172.21.15.46:0/2615283464 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c004a30 con 0x55b8da3c6b60 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.845+0000 7f675be00700 1 -- 172.21.15.46:0/2615283464 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675c0054e0 con 0x55b8da3c6b60 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.845+0000 7f675be00700 1 -- 172.21.15.46:0/2615283464 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c003cb0 con 0x55b8da3c6b60 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 -- 172.21.15.46:0/2615283464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 msgr2=0x55b8da3c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f675c0046c0 tx=0x7f675c00b9e0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 -- 172.21.15.46:0/2615283464 shutdown_connections 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 --2- 172.21.15.46:0/2615283464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3c6fe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 -- 172.21.15.46:0/2615283464 >> 172.21.15.46:0/2615283464 conn(0x55b8da31c210 msgr2=0x55b8da3b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 -- 172.21.15.46:0/2615283464 shutdown_connections 2024-05-08T23:15:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.846+0000 7f67894a3b80 1 -- 172.21.15.46:0/2615283464 wait complete. 2024-05-08T23:15:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 -- start start 2024-05-08T23:15:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 Processor -- start 2024-05-08T23:15:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da2aa560 con 0x55b8da3c7520 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da2aa200 con 0x55b8da3c6b60 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da1916a0 con 0x55b8da3cc870 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.847+0000 7f67894a3b80 10 client.0 _ll_register_callbacks cb 0x55b8da2b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48918/0 (socket says 172.21.15.46:48918) 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 -- 172.21.15.46:0/654056311 learned_addr learned my addr 172.21.15.46:0/654056311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676ea00700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 -- 172.21.15.46:0/654056311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 msgr2=0x55b8da3d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3d35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 -- 172.21.15.46:0/654056311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 msgr2=0x55b8da3d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 -- 172.21.15.46:0/654056311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c0042f0 con 0x55b8da3cc870 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f676f400700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f676800eef0 tx=0x7f6768004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9eb0a42043388f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.848+0000 7f6758c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676801ade0 con 0x55b8da3cc870 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67680048c0 con 0x55b8da3cc870 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8da267270 con 0x55b8da3cc870 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6768018440 con 0x55b8da3cc870 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f67894a3b80 10 client.28933 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:54.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b8da13bb60 con 0x55b8da3cc870 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8da267270 con 0x55b8da3cc870 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.849+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f676802b060 con 0x55b8da3cc870 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 10 client.28933 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 10 client.28933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 20 client.28933 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da3e0b60 0x55b8da3e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 4 client.28933 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8da38d810 con 0x55b8da3e0b60 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f676ea00700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da3e0b60 0x55b8da3e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f676ea00700 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da3e0b60 0x55b8da3e2f80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14a3c5e045996b38 server_cookie=3f0955d6ffa0991b in_seq=0 out_seq=0 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.850+0000 7f6758c00700 10 client.28933 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f6758c00700 1 -- 172.21.15.46:0/654056311 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f6760066a90 con 0x55b8da3e0b60 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f6758c00700 10 client.28933 handle_command_reply: tid=0 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 client.28933 shutdown 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 msgr2=0x55b8da3d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f676800eef0 tx=0x7f6768004ad0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 shutdown_connections 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da3e0b60 0x55b8da3e2f80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0fd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f6758c00700 0 client.28933 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 --2- 172.21.15.46:0/654056311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.851+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 >> 172.21.15.46:0/654056311 conn(0x55b8da31c210 msgr2=0x55b8da3b9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.852+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 shutdown_connections 2024-05-08T23:15:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.852+0000 7f67894a3b80 1 -- 172.21.15.46:0/654056311 wait complete. 2024-05-08T23:15:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.852+0000 7f67894a3b80 20 client.28933 trim_cache size 0 max 16384 2024-05-08T23:15:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.867+0000 7f67894a3b80 1 Processor -- start 2024-05-08T23:15:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.867+0000 7f67894a3b80 1 -- start start 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.867+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f67894a3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3bac20 con 0x55b8da3c7520 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f67894a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3e8190 con 0x55b8da3c6b60 2024-05-08T23:15:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f67894a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3b3800 con 0x55b8da3cc870 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 -- 172.21.15.46:0/839016170 learned_addr learned my addr 172.21.15.46:0/839016170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676e000700 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 -- 172.21.15.46:0/839016170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 msgr2=0x55b8da3d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 -- 172.21.15.46:0/839016170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 msgr2=0x55b8da3dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3dcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676ea00700 1 -- 172.21.15.46:0/839016170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8da3cc4f0 con 0x55b8da3c6b60 2024-05-08T23:15:54.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.868+0000 7f676f400700 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.869+0000 7f676ea00700 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6760005cb0 tx=0x7f67600cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2bb35c8d6a5d991 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.869+0000 7f6764c00700 1 -- 172.21.15.46:0/839016170 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600b7070 con 0x55b8da3c6b60 2024-05-08T23:15:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.869+0000 7f6764c00700 1 -- 172.21.15.46:0/839016170 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8da38d810 con 0x55b8da3c6b60 2024-05-08T23:15:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.869+0000 7f6764c00700 1 -- 172.21.15.46:0/839016170 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600a3560 con 0x55b8da3c6b60 2024-05-08T23:15:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- 172.21.15.46:0/839016170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 msgr2=0x55b8da3ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6760005cb0 tx=0x7f67600cd040 comp rx=0 tx=0).stop 2024-05-08T23:15:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- 172.21.15.46:0/839016170 shutdown_connections 2024-05-08T23:15:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3cc870 0x55b8da3d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3c6b60 0x55b8da3ce4f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 --2- 172.21.15.46:0/839016170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- 172.21.15.46:0/839016170 >> 172.21.15.46:0/839016170 conn(0x55b8da31c210 msgr2=0x55b8da3b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- 172.21.15.46:0/839016170 shutdown_connections 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- 172.21.15.46:0/839016170 wait complete. 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 -- start start 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.870+0000 7f67894a3b80 1 Processor -- start 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 0x55b8da3d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3d4220 con 0x55b8da3c7520 2024-05-08T23:15:54.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3d4390 con 0x55b8da3cc870 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8da3d4550 con 0x55b8da3c6b60 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f67894a3b80 10 client.0 _ll_register_callbacks cb 0x55b8da2b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f676ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f676f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 0x55b8da3d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f676e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f676ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48940/0 (socket says 172.21.15.46:48940) 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.871+0000 7f676ea00700 1 -- 172.21.15.46:0/1512016285 learned_addr learned my addr 172.21.15.46:0/1512016285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 -- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 msgr2=0x55b8da3d6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 -- 172.21.15.46:0/1512016285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 msgr2=0x55b8da3d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 -- 172.21.15.46:0/1512016285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8da13bb60 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676ea00700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f676f400700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 0x55b8da3d3c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f676802d7b0 tx=0x7f676802d9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d223ae0e806523bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f6758c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67680243f0 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675c0054e0 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.872+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6768004b40 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6768004d00 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 10 client.28847 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f67600cbe70 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675c0054e0 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67680248e0 con 0x55b8da3cc870 2024-05-08T23:15:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 10 client.28847 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 10 client.28847 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 20 client.28847 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da437640 0x55b8da439a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 4 client.28847 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b8da306ec0 con 0x55b8da437640 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.873+0000 7f676ea00700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da437640 0x55b8da439a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f676ea00700 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da437640 0x55b8da439a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d27ed76dc0eaf834 server_cookie=1a058f205ddcf46 in_seq=0 out_seq=0 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f6758c00700 10 client.28847 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f6758c00700 1 -- 172.21.15.46:0/1512016285 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b8da306ec0 con 0x55b8da437640 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f6758c00700 10 client.28847 handle_command_reply: tid=0 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 client.28847 shutdown 2024-05-08T23:15:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 msgr2=0x55b8da3d3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 0x55b8da3d3c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f676802d7b0 tx=0x7f676802d9d0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 shutdown_connections 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b8da437640 0x55b8da439a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b8da3cc870 0x55b8da3d3c50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8da3c6b60 0x55b8da3d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 --2- 172.21.15.46:0/1512016285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b8da3c7520 0x55b8da3d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f6758c00700 0 client.28847 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.874+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 >> 172.21.15.46:0/1512016285 conn(0x55b8da31c210 msgr2=0x55b8da3ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 shutdown_connections 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f67894a3b80 1 -- 172.21.15.46:0/1512016285 wait complete. 2024-05-08T23:15:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f67894a3b80 20 client.28847 trim_cache size 0 max 16384 2024-05-08T23:15:54.880 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 msgr2=0x7f67540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 0x7f67540a37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6768009e90 tx=0x7f6768009920 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 msgr2=0x7f6770199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6760000c00 tx=0x7f6760005740 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 shutdown_connections 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67540a1330 0x7f67540a37f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677019dec0 0x7f677019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6770100ec0 0x7f6770199630 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 --2- 172.21.15.46:0/519987131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6770105fe0 0x7f6770199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.875+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 >> 172.21.15.46:0/519987131 conn(0x7f67700fc740 msgr2=0x7f67700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.876+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 shutdown_connections 2024-05-08T23:15:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:54.876+0000 7f676fe00700 1 -- 172.21.15.46:0/519987131 wait complete. 2024-05-08T23:15:54.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:15:54.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1056704 9.26 MB/sec execute 496 sec latency 23.515 ms 2024-05-08T23:15:55.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 -- 172.21.15.46:0/2291099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 msgr2=0x7eff7c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 --2- 172.21.15.46:0/2291099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c10dd20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b0a0 tx=0x7eff7400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 -- 172.21.15.46:0/2291099725 shutdown_connections 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 --2- 172.21.15.46:0/2291099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c10dd20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 --2- 172.21.15.46:0/2291099725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 --2- 172.21.15.46:0/2291099725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c101a30 0x7eff7c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 -- 172.21.15.46:0/2291099725 >> 172.21.15.46:0/2291099725 conn(0x7eff7c0fc8e0 msgr2=0x7eff7c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 -- 172.21.15.46:0/2291099725 shutdown_connections 2024-05-08T23:15:55.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.126+0000 7eff83400700 1 -- 172.21.15.46:0/2291099725 wait complete. 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.127+0000 7eff83400700 1 Processor -- start 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.127+0000 7eff83400700 1 -- start start 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 0x7eff7c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19e960 con 0x7eff7c101080 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19ead0 con 0x7eff7c19dee0 2024-05-08T23:15:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff83400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19ec90 con 0x7eff7c1061a0 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39630/0 (socket says 172.21.15.46:39630) 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 -- 172.21.15.46:0/2960019265 learned_addr learned my addr 172.21.15.46:0/2960019265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff81400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 0x7eff7c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff7be00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 -- 172.21.15.46:0/2960019265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 msgr2=0x7eff7c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 -- 172.21.15.46:0/2960019265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 msgr2=0x7eff7c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.128+0000 7eff80a00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 0x7eff7c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff80a00700 1 -- 172.21.15.46:0/2960019265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74009d00 con 0x7eff7c101080 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff81400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 0x7eff7c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff80a00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e9a0 tx=0x7eff7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff79600700 1 -- 172.21.15.46:0/2960019265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70009790 con 0x7eff7c101080 2024-05-08T23:15:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c1a61f0 con 0x7eff7c101080 2024-05-08T23:15:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c1a6720 con 0x7eff7c101080 2024-05-08T23:15:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff79600700 1 -- 172.21.15.46:0/2960019265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff70010430 con 0x7eff7c101080 2024-05-08T23:15:55.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.129+0000 7eff79600700 1 -- 172.21.15.46:0/2960019265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70018680 con 0x7eff7c101080 2024-05-08T23:15:55.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.133+0000 7eff79600700 1 -- 172.21.15.46:0/2960019265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff700187e0 con 0x7eff7c101080 2024-05-08T23:15:55.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.134+0000 7eff79600700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 0x7eff640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.134+0000 7eff96e35b80 1 Processor -- start 2024-05-08T23:15:55.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.134+0000 7eff79600700 1 -- 172.21.15.46:0/2960019265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7eff70014070 con 0x7eff7c101080 2024-05-08T23:15:55.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 -- start start 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff7be00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 0x7eff640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deb7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3deb7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3de0e150 con 0x55ee3debd720 2024-05-08T23:15:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3de0bb10 con 0x55ee3deb7890 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff96e35b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3de07400 con 0x55ee3debce00 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff80a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff80a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48978/0 (socket says 172.21.15.46:48978) 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.135+0000 7eff80a00700 1 -- 172.21.15.46:0/1266414613 learned_addr learned my addr 172.21.15.46:0/1266414613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff81400700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3deb7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff7be00700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deb7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 -- 172.21.15.46:0/1266414613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 msgr2=0x55ee3deb7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff7be00700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 0x7eff640a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff7c19ab50 tx=0x7eff68009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3deb7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 -- 172.21.15.46:0/1266414613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 msgr2=0x55ee3deb7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deb7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 -- 172.21.15.46:0/1266414613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3dc883b0 con 0x55ee3debce00 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff81400700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3deb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff80a00700 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7eff700a1e30 tx=0x7eff7009a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eacf98bc1b2e1d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff6dc00700 1 -- 172.21.15.46:0/1266414613 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700a2360 con 0x55ee3debce00 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff6dc00700 1 -- 172.21.15.46:0/1266414613 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff7009a140 con 0x55ee3debce00 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.136+0000 7eff6dc00700 1 -- 172.21.15.46:0/1266414613 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700a2130 con 0x55ee3debce00 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 -- 172.21.15.46:0/1266414613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 msgr2=0x55ee3debd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7eff700a1e30 tx=0x7eff7009a6b0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 -- 172.21.15.46:0/1266414613 shutdown_connections 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3deb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3debd1e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1266414613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deb7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 -- 172.21.15.46:0/1266414613 >> 172.21.15.46:0/1266414613 conn(0x55ee3de15210 msgr2=0x55ee3deb1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 -- 172.21.15.46:0/1266414613 shutdown_connections 2024-05-08T23:15:55.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.137+0000 7eff96e35b80 1 -- 172.21.15.46:0/1266414613 wait complete. 2024-05-08T23:15:55.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.138+0000 7eff96e35b80 1 -- start start 2024-05-08T23:15:55.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.138+0000 7eff96e35b80 1 Processor -- start 2024-05-08T23:15:55.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3dec1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3dda3560 con 0x55ee3debd720 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3dda3200 con 0x55ee3deb7890 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3dc8a6a0 con 0x55ee3debce00 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff96e35b80 10 client.0 _ll_register_callbacks cb 0x55ee3ddacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:48994/0 (socket says 172.21.15.46:48994) 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 -- 172.21.15.46:0/1548071434 learned_addr learned my addr 172.21.15.46:0/1548071434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff81400700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3dec1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 -- 172.21.15.46:0/1548071434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 msgr2=0x55ee3dec0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 -- 172.21.15.46:0/1548071434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 msgr2=0x55ee3dec1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3dec1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.139+0000 7eff7be00700 1 -- 172.21.15.46:0/1548071434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c10e590 con 0x55ee3debce00 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff7be00700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7eff68004370 tx=0x7eff680047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26ec90f64ec2343 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff62000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68027440 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee3dd60270 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7eff680275a0 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff68027760 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff96e35b80 10 client.28951 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.140+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7eff7c1a6720 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee3dd60270 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff6801c4a0 con 0x55ee3debce00 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 10 client.28951 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 10 client.28951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 20 client.28951 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3ded1290 0x55ee3ded36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 4 client.28951 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee3de86810 con 0x55ee3ded1290 2024-05-08T23:15:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff80a00700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3ded1290 0x55ee3ded36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.141+0000 7eff80a00700 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3ded1290 0x55ee3ded36b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bd781d9b3bf29c3 server_cookie=3144d13174f182a5 in_seq=0 out_seq=0 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.142+0000 7eff62000700 10 client.28951 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.142+0000 7eff62000700 1 -- 172.21.15.46:0/1548071434 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee3de86810 con 0x55ee3ded1290 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.142+0000 7eff62000700 10 client.28951 handle_command_reply: tid=0 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.142+0000 7eff96e35b80 1 client.28951 shutdown 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 msgr2=0x55ee3dec0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7eff68004370 tx=0x7eff680047c0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 shutdown_connections 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3ded1290 0x55ee3ded36b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3dec1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 --2- 172.21.15.46:0/1548071434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3dec0ff0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff62000700 0 client.28951 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 >> 172.21.15.46:0/1548071434 conn(0x55ee3de15210 msgr2=0x55ee3deb15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 shutdown_connections 2024-05-08T23:15:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 1 -- 172.21.15.46:0/1548071434 wait complete. 2024-05-08T23:15:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.143+0000 7eff96e35b80 20 client.28951 trim_cache size 0 max 16384 2024-05-08T23:15:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.164+0000 7eff96e35b80 1 Processor -- start 2024-05-08T23:15:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.164+0000 7eff96e35b80 1 -- start start 2024-05-08T23:15:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3ded3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debce00 0x55ee3deb1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 0x55ee3dec8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3dec3660 con 0x55ee3debce00 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3deaedf0 con 0x55ee3deb7890 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff96e35b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3ded8940 con 0x55ee3debd720 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3ded3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3ded3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37386/0 (socket says 172.21.15.46:37386) 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 0x55ee3dec8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff80a00700 1 -- 172.21.15.46:0/466791424 learned_addr learned my addr 172.21.15.46:0/466791424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.165+0000 7eff7be00700 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debce00 0x55ee3deb1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 -- 172.21.15.46:0/466791424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 msgr2=0x55ee3ded3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3ded3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 -- 172.21.15.46:0/466791424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debce00 msgr2=0x55ee3deb1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debce00 0x55ee3deb1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 -- 172.21.15.46:0/466791424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3dc34b60 con 0x55ee3debd720 2024-05-08T23:15:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.166+0000 7eff81400700 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 0x55ee3dec8160 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b780 tx=0x7eff74009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a6566e8a52e3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.167+0000 7eff6e600700 1 -- 172.21.15.46:0/466791424 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400e040 con 0x55ee3debd720 2024-05-08T23:15:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.167+0000 7eff6e600700 1 -- 172.21.15.46:0/466791424 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff74007d40 con 0x55ee3debd720 2024-05-08T23:15:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.167+0000 7eff6e600700 1 -- 172.21.15.46:0/466791424 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740276e0 con 0x55ee3debd720 2024-05-08T23:15:55.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.169+0000 7eff96e35b80 1 -- 172.21.15.46:0/466791424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 msgr2=0x55ee3dec8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.169+0000 7eff96e35b80 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 0x55ee3dec8160 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b780 tx=0x7eff74009520 comp rx=0 tx=0).stop 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 -- 172.21.15.46:0/466791424 shutdown_connections 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debd720 0x55ee3dec8160 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3ded3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 --2- 172.21.15.46:0/466791424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debce00 0x55ee3deb1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 -- 172.21.15.46:0/466791424 >> 172.21.15.46:0/466791424 conn(0x55ee3de15210 msgr2=0x55ee3deb7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 -- 172.21.15.46:0/466791424 shutdown_connections 2024-05-08T23:15:55.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.170+0000 7eff96e35b80 1 -- 172.21.15.46:0/466791424 wait complete. 2024-05-08T23:15:55.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.171+0000 7eff96e35b80 1 -- start start 2024-05-08T23:15:55.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.171+0000 7eff96e35b80 1 Processor -- start 2024-05-08T23:15:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3deba790 con 0x55ee3debd720 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3deb8930 con 0x55ee3deb7890 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3deb8af0 con 0x55ee3debce00 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff96e35b80 10 client.0 _ll_register_callbacks cb 0x55ee3ddacb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49014/0 (socket says 172.21.15.46:49014) 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff7be00700 1 -- 172.21.15.46:0/3331545489 learned_addr learned my addr 172.21.15.46:0/3331545489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37394/0 (socket says 172.21.15.46:37394) 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.172+0000 7eff81400700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff7be00700 1 -- 172.21.15.46:0/3331545489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 msgr2=0x55ee3dec8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff7be00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff7be00700 1 -- 172.21.15.46:0/3331545489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 msgr2=0x55ee3deba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff7be00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff7be00700 1 -- 172.21.15.46:0/3331545489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff740092e0 con 0x55ee3debce00 2024-05-08T23:15:55.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.173+0000 7eff80a00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.174+0000 7eff7be00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff6801c8b0 tx=0x7eff680119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed2c51e23cdb2b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.174+0000 7eff62000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.174+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68003960 con 0x55ee3debce00 2024-05-08T23:15:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.174+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7eff68025340 con 0x55ee3debce00 2024-05-08T23:15:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.174+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff7009a140 con 0x55ee3debce00 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.175+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff6801c9d0 con 0x55ee3debce00 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.175+0000 7eff96e35b80 10 client.28957 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.175+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ee3ddcd560 con 0x55ee3debce00 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.175+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff7009a140 con 0x55ee3debce00 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.175+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff68027550 con 0x55ee3debce00 2024-05-08T23:15:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 10 client.28957 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 10 client.28957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 20 client.28957 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3df308b0 0x55ee3df32cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 4 client.28957 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee3ddffec0 con 0x55ee3df308b0 2024-05-08T23:15:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.176+0000 7eff80a00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3df308b0 0x55ee3df32cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.177+0000 7eff80a00700 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3df308b0 0x55ee3df32cd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de1dab4ce33f5f95 server_cookie=2037119efdefd7a8 in_seq=0 out_seq=0 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.177+0000 7eff62000700 10 client.28957 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.177+0000 7eff62000700 1 -- 172.21.15.46:0/3331545489 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee3ddffec0 con 0x55ee3df308b0 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.177+0000 7eff62000700 10 client.28957 handle_command_reply: tid=0 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.177+0000 7eff96e35b80 1 client.28957 shutdown 2024-05-08T23:15:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 msgr2=0x55ee3deb9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff6801c8b0 tx=0x7eff680119c0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 shutdown_connections 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55ee3df308b0 0x55ee3df32cd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3debd720 0x55ee3deba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3deb7890 0x55ee3dec8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 --2- 172.21.15.46:0/3331545489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3debce00 0x55ee3deb9a20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 >> 172.21.15.46:0/3331545489 conn(0x55ee3de15210 msgr2=0x55ee3debf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff62000700 0 client.28957 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:15:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 shutdown_connections 2024-05-08T23:15:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.178+0000 7eff96e35b80 1 -- 172.21.15.46:0/3331545489 wait complete. 2024-05-08T23:15:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.179+0000 7eff96e35b80 20 client.28957 trim_cache size 0 max 16384 2024-05-08T23:15:55.179 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.179+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 msgr2=0x7eff640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.179+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 0x7eff640a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff7c19ab50 tx=0x7eff68009260 comp rx=0 tx=0).stop 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.179+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 msgr2=0x7eff7c199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.179+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e9a0 tx=0x7eff7000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 shutdown_connections 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff7c19dee0 0x7eff7c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff640a1290 0x7eff640a3750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff7c101080 0x7eff7c199600 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 --2- 172.21.15.46:0/2960019265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff7c1061a0 0x7eff7c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 >> 172.21.15.46:0/2960019265 conn(0x7eff7c0fc8e0 msgr2=0x7eff7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.180+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 shutdown_connections 2024-05-08T23:15:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.181+0000 7eff83400700 1 -- 172.21.15.46:0/2960019265 wait complete. 2024-05-08T23:15:55.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:15:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 -- 172.21.15.46:0/3970033246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 msgr2=0x7fd3ac10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3970033246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac10f910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b0a0 tx=0x7fd3a400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 -- 172.21.15.46:0/3970033246 shutdown_connections 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3970033246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac10f910 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3970033246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac0737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3970033246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 -- 172.21.15.46:0/3970033246 >> 172.21.15.46:0/3970033246 conn(0x7fd3ac0fc770 msgr2=0x7fd3ac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 -- 172.21.15.46:0/3970033246 shutdown_connections 2024-05-08T23:15:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.484+0000 7fd3b2600700 1 -- 172.21.15.46:0/3970033246 wait complete. 2024-05-08T23:15:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.485+0000 7fd3b2600700 1 Processor -- start 2024-05-08T23:15:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.485+0000 7fd3b2600700 1 -- start start 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac19e2d0 con 0x7fd3ac10d4a0 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac19e440 con 0x7fd3ac0733d0 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b2600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac19e600 con 0x7fd3ac073cf0 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49028/0 (socket says 172.21.15.46:49028) 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3abe00700 1 -- 172.21.15.46:0/3606884234 learned_addr learned my addr 172.21.15.46:0/3606884234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b1200700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.486+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 -- 172.21.15.46:0/3606884234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 msgr2=0x7fd3ac1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 -- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 msgr2=0x7fd3ac19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 -- 172.21.15.46:0/3606884234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0009520 con 0x7fd3ac073cf0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3b1200700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e9c0 tx=0x7fd39800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3a9600700 1 -- 172.21.15.46:0/3606884234 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980097b0 con 0x7fd3ac073cf0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3a9600700 1 -- 172.21.15.46:0/3606884234 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd39800ee60 con 0x7fd3ac073cf0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a4009d00 con 0x7fd3ac073cf0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3a9600700 1 -- 172.21.15.46:0/3606884234 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980054f0 con 0x7fd3ac073cf0 2024-05-08T23:15:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.487+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac1a5f40 con 0x7fd3ac073cf0 2024-05-08T23:15:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.492+0000 7fd3a9600700 1 -- 172.21.15.46:0/3606884234 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd398004750 con 0x7fd3ac073cf0 2024-05-08T23:15:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.492+0000 7fd3c7665b80 1 Processor -- start 2024-05-08T23:15:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.492+0000 7fd3c7665b80 1 -- start start 2024-05-08T23:15:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55c0970 0x55c5a55ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55ba890 0x55c5a55bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a5511150 con 0x55c5a55c0050 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a550eb10 con 0x55c5a55ba890 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a550a400 con 0x55c5a55c0970 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39680/0 (socket says 172.21.15.46:39680) 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1550751196 learned_addr learned my addr 172.21.15.46:0/1550751196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3abe00700 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55ba890 0x55c5a55bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.493+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55c0970 0x55c5a55ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1550751196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55c0970 msgr2=0x55c5a55ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55c0970 0x55c5a55ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3a9600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 0x7fd3940a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1550751196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55ba890 msgr2=0x55c5a55bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55ba890 0x55c5a55bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1550751196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a538b3b0 con 0x55c5a55c0050 2024-05-08T23:15:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3a9600700 1 -- 172.21.15.46:0/3606884234 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd398014070 con 0x7fd3ac073cf0 2024-05-08T23:15:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 0x7fd3940a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4015000 tx=0x7fd3a4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c1749c3ea514f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.494+0000 7fd39e000700 1 -- 172.21.15.46:0/1550751196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400e040 con 0x55c5a55c0050 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.495+0000 7fd39e000700 1 -- 172.21.15.46:0/1550751196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a4007d70 con 0x55c5a55c0050 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.495+0000 7fd39e000700 1 -- 172.21.15.46:0/1550751196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a401c840 con 0x55c5a55c0050 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.495+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1550751196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 msgr2=0x55c5a55c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.495+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4015000 tx=0x7fd3a4009c40 comp rx=0 tx=0).stop 2024-05-08T23:15:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.495+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 0x7fd3940a3870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4024000 tx=0x7fd3a40077b0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1550751196 shutdown_connections 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55ba890 0x55c5a55bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c0430 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1550751196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55c0970 0x55c5a55ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1550751196 >> 172.21.15.46:0/1550751196 conn(0x55c5a5518210 msgr2=0x55c5a55b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1550751196 shutdown_connections 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1550751196 wait complete. 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.496+0000 7fd3c7665b80 1 -- start start 2024-05-08T23:15:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 Processor -- start 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a54a6560 con 0x55c5a55c0970 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a54a6200 con 0x55c5a55c0050 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a538d6a0 con 0x55c5a55ba890 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.497+0000 7fd3c7665b80 10 client.0 _ll_register_callbacks cb 0x55c5a54afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39692/0 (socket says 172.21.15.46:39692) 2024-05-08T23:15:55.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1163635276 learned_addr learned my addr 172.21.15.46:0/1163635276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3abe00700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1163635276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 msgr2=0x55c5a55c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1163635276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 msgr2=0x55c5a55c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.498+0000 7fd3b1c00700 1 -- 172.21.15.46:0/1163635276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac1105d0 con 0x55c5a55c0970 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007550 tx=0x7fd3a401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573ea66b47467daa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd392a00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400e030 con 0x55c5a55c0970 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5a5463270 con 0x55c5a55c0970 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd3a40075b0 con 0x55c5a55c0970 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a401c840 con 0x55c5a55c0970 2024-05-08T23:15:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd3c7665b80 10 client.18855 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.499+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5a5337b60 con 0x55c5a55c0970 2024-05-08T23:15:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5a5463270 con 0x55c5a55c0970 2024-05-08T23:15:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd3a40051e0 con 0x55c5a55c0970 2024-05-08T23:15:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 10 client.18855 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 20 client.18855 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a55d3ea0 0x55c5a55d6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 4 client.18855 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5a5589810 con 0x55c5a55d3ea0 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.500+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a55d3ea0 0x55c5a55d6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.501+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a55d3ea0 0x55c5a55d6290 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9e7677446d7a31f server_cookie=8053e7df87eab448 in_seq=0 out_seq=0 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.501+0000 7fd392a00700 10 client.18855 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd392a00700 1 -- 172.21.15.46:0/1163635276 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5a5589810 con 0x55c5a55d3ea0 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd392a00700 10 client.18855 handle_command_reply: tid=0 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 client.18855 shutdown 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 msgr2=0x55c5a55c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007550 tx=0x7fd3a401e000 comp rx=0 tx=0).stop 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 shutdown_connections 2024-05-08T23:15:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a55d3ea0 0x55c5a55d6290 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55c4100 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1163635276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 >> 172.21.15.46:0/1163635276 conn(0x55c5a5518210 msgr2=0x55c5a55b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.502+0000 7fd392a00700 0 client.18855 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.503+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 shutdown_connections 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.503+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1163635276 wait complete. 2024-05-08T23:15:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.503+0000 7fd3c7665b80 20 client.18855 trim_cache size 0 max 16384 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.520+0000 7fd3c7665b80 1 Processor -- start 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 -- start start 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a55bb2f0 con 0x55c5a55c0970 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a55b6e20 con 0x55c5a55c0050 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a55b57c0 con 0x55c5a55ba890 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3b1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3b1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49042/0 (socket says 172.21.15.46:49042) 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3b1200700 1 -- 172.21.15.46:0/1898120200 learned_addr learned my addr 172.21.15.46:0/1898120200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3abe00700 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.521+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 -- 172.21.15.46:0/1898120200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 msgr2=0x55c5a55cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 -- 172.21.15.46:0/1898120200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 msgr2=0x55c5a55cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 -- 172.21.15.46:0/1898120200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a54d0560 con 0x55c5a55ba890 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd3b1200700 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005950 tx=0x7fd3a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff14f1a373cd62c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd39ea00700 1 -- 172.21.15.46:0/1898120200 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0017040 con 0x55c5a55ba890 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd39ea00700 1 -- 172.21.15.46:0/1898120200 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a0018040 con 0x55c5a55ba890 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.522+0000 7fd39ea00700 1 -- 172.21.15.46:0/1898120200 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00210f0 con 0x55c5a55ba890 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1898120200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 msgr2=0x55c5a55c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005950 tx=0x7fd3a0009260 comp rx=0 tx=0).stop 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1898120200 shutdown_connections 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0970 0x55c5a55cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c63e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/1898120200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0050 0x55c5a55cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1898120200 >> 172.21.15.46:0/1898120200 conn(0x55c5a5518210 msgr2=0x55c5a55dabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.523+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1898120200 shutdown_connections 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.524+0000 7fd3c7665b80 1 -- 172.21.15.46:0/1898120200 wait complete. 2024-05-08T23:15:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.524+0000 7fd3c7665b80 1 -- start start 2024-05-08T23:15:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.524+0000 7fd3c7665b80 1 Processor -- start 2024-05-08T23:15:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 0x55c5a55c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a5631c80 con 0x55c5a55c0050 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a5631df0 con 0x55c5a55c0970 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a5631fb0 con 0x55c5a55ba890 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3c7665b80 10 client.0 _ll_register_callbacks cb 0x55c5a54afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3b1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3b1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49052/0 (socket says 172.21.15.46:49052) 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3b1200700 1 -- 172.21.15.46:0/3223079747 learned_addr learned my addr 172.21.15.46:0/3223079747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.525+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 0x55c5a55c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 -- 172.21.15.46:0/3223079747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 msgr2=0x55c5a55c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 0x55c5a55c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 -- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 msgr2=0x55c5a55c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 -- 172.21.15.46:0/3223079747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a0004040 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1c00700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 0x55c5a55c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3b1200700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0000c00 tx=0x7fd3a00121a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4cd23147c7cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd392a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0017040 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd3a0003880 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3a4007d70 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a0003a40 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3c7665b80 10 client.28975 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.526+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5a538fcd0 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a4007d70 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd3a0010840 con 0x55c5a55ba890 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 10 client.28975 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 10 client.28975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:15:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 20 client.28975 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a56336d0 0x55c5a5635af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 4 client.28975 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5a5502ec0 con 0x55c5a56336d0 2024-05-08T23:15:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.527+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a56336d0 0x55c5a5635af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:55 smithi046 ceph-mon[36898]: pgmap v376: 129 pgs: 129 active+clean; 598 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-05-08T23:15:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd392a00700 10 client.28975 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd3abe00700 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a56336d0 0x55c5a5635af0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7a130ca90928395 server_cookie=eb28268e8559d40a in_seq=0 out_seq=0 2024-05-08T23:15:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd392a00700 1 -- 172.21.15.46:0/3223079747 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5a5502ec0 con 0x55c5a56336d0 2024-05-08T23:15:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd392a00700 10 client.28975 handle_command_reply: tid=0 2024-05-08T23:15:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd3c7665b80 1 client.28975 shutdown 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 msgr2=0x55c5a55c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.528+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0000c00 tx=0x7fd3a00121a0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 shutdown_connections 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5a55c0970 0x55c5a55c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5a55ba890 0x55c5a55c73b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c5a56336d0 0x55c5a5635af0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 --2- 172.21.15.46:0/3223079747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5a55c0050 0x55c5a55c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 >> 172.21.15.46:0/3223079747 conn(0x55c5a5518210 msgr2=0x55c5a55bdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd392a00700 0 client.28975 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 shutdown_connections 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 1 -- 172.21.15.46:0/3223079747 wait complete. 2024-05-08T23:15:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.529+0000 7fd3c7665b80 20 client.28975 trim_cache size 0 max 16384 2024-05-08T23:15:55.533 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 msgr2=0x7fd3940a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 0x7fd3940a3870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4024000 tx=0x7fd3a40077b0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 msgr2=0x7fd3ac1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e9c0 tx=0x7fd39800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 shutdown_connections 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd3940a13b0 0x7fd3940a3870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac10d4a0 0x7fd3ac19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3ac0733d0 0x7fd3ac1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 --2- 172.21.15.46:0/3606884234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3ac073cf0 0x7fd3ac1998f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 >> 172.21.15.46:0/3606884234 conn(0x7fd3ac0fc770 msgr2=0x7fd3ac0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 shutdown_connections 2024-05-08T23:15:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.530+0000 7fd3b2600700 1 -- 172.21.15.46:0/3606884234 wait complete. 2024-05-08T23:15:55.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:15:55.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:55 smithi086 ceph-mon[78610]: pgmap v376: 129 pgs: 129 active+clean; 598 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-05-08T23:15:55.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:55 smithi053 ceph-mon[77587]: pgmap v376: 129 pgs: 129 active+clean; 598 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 -- 172.21.15.46:0/937527687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073c70 msgr2=0x7ff46010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 --2- 172.21.15.46:0/937527687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073c70 0x7ff46010d160 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009990 tx=0x7ff44c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 -- 172.21.15.46:0/937527687 shutdown_connections 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 --2- 172.21.15.46:0/937527687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46010d6a0 0x7ff46010fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 --2- 172.21.15.46:0/937527687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff460073350 0x7ff460073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 --2- 172.21.15.46:0/937527687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073c70 0x7ff46010d160 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 -- 172.21.15.46:0/937527687 >> 172.21.15.46:0/937527687 conn(0x7ff4600fc8b0 msgr2=0x7ff4600fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 -- 172.21.15.46:0/937527687 shutdown_connections 2024-05-08T23:15:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.783+0000 7ff466600700 1 -- 172.21.15.46:0/937527687 wait complete. 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.784+0000 7ff466600700 1 Processor -- start 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.784+0000 7ff466600700 1 -- start start 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff460073c70 0x7ff460199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff46010d6a0 0x7ff46019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46019e480 con 0x7ff460073c70 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46019e5f0 con 0x7ff46010d6a0 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff466600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46019e7b0 con 0x7ff460073350 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49068/0 (socket says 172.21.15.46:49068) 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff45f400700 1 -- 172.21.15.46:0/2581170907 learned_addr learned my addr 172.21.15.46:0/2581170907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff45ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff460073c70 0x7ff460199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.785+0000 7ff45fe00700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff46010d6a0 0x7ff46019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 -- 172.21.15.46:0/2581170907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff46010d6a0 msgr2=0x7ff46019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff46010d6a0 0x7ff46019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 -- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff460073c70 msgr2=0x7ff460199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff460073c70 0x7ff460199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 -- 172.21.15.46:0/2581170907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0095f0 con 0x7ff460073350 2024-05-08T23:15:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff45f400700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff45000e9c0 tx=0x7ff45000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff457e00700 1 -- 172.21.15.46:0/2581170907 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500097b0 con 0x7ff460073350 2024-05-08T23:15:55.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff457e00700 1 -- 172.21.15.46:0/2581170907 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff45000ee60 con 0x7ff460073350 2024-05-08T23:15:55.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46019ea90 con 0x7ff460073350 2024-05-08T23:15:55.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff457e00700 1 -- 172.21.15.46:0/2581170907 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500054f0 con 0x7ff460073350 2024-05-08T23:15:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.786+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601a62d0 con 0x7ff460073350 2024-05-08T23:15:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.789+0000 7ff457e00700 1 -- 172.21.15.46:0/2581170907 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff450004750 con 0x7ff460073350 2024-05-08T23:15:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.791+0000 7ff457e00700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 0x7ff4440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.791+0000 7ff457e00700 1 -- 172.21.15.46:0/2581170907 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff450014070 con 0x7ff460073350 2024-05-08T23:15:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.791+0000 7ff47a01fb80 1 Processor -- start 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.791+0000 7ff45ea00700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 0x7ff4440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.791+0000 7ff47a01fb80 1 -- start start 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210ffe150 con 0x5582110a7890 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210ffbb10 con 0x5582110ad720 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210ff7400 con 0x5582110ace00 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49070/0 (socket says 172.21.15.46:49070) 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 -- 172.21.15.46:0/490754799 learned_addr learned my addr 172.21.15.46:0/490754799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45fe00700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45ea00700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 0x7ff4440a37d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff46019aab0 tx=0x7ff44c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45ea00700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 -- 172.21.15.46:0/490754799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 msgr2=0x5582110a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 -- 172.21.15.46:0/490754799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 msgr2=0x5582110a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 -- 172.21.15.46:0/490754799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558210e783b0 con 0x5582110ace00 2024-05-08T23:15:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45fe00700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45ea00700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.792+0000 7ff45f400700 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff4500a1f00 tx=0x7ff45009a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6168662a94a4996 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.793+0000 7ff454c00700 1 -- 172.21.15.46:0/490754799 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500a2430 con 0x5582110ace00 2024-05-08T23:15:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.793+0000 7ff454c00700 1 -- 172.21.15.46:0/490754799 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4500980f0 con 0x5582110ace00 2024-05-08T23:15:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.793+0000 7ff454c00700 1 -- 172.21.15.46:0/490754799 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500994a0 con 0x5582110ace00 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.793+0000 7ff47a01fb80 1 -- 172.21.15.46:0/490754799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 msgr2=0x5582110ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.793+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff4500a1f00 tx=0x7ff45009a730 comp rx=0 tx=0).stop 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 -- 172.21.15.46:0/490754799 shutdown_connections 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110ad1e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/490754799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 -- 172.21.15.46:0/490754799 >> 172.21.15.46:0/490754799 conn(0x558211005210 msgr2=0x5582110a1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 -- 172.21.15.46:0/490754799 shutdown_connections 2024-05-08T23:15:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 -- 172.21.15.46:0/490754799 wait complete. 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.794+0000 7ff47a01fb80 1 -- start start 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 Processor -- start 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210f93560 con 0x5582110a7890 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210f93200 con 0x5582110ad720 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210e7a6a0 con 0x5582110ace00 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.795+0000 7ff47a01fb80 10 client.0 _ll_register_callbacks cb 0x558210f9cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49084/0 (socket says 172.21.15.46:49084) 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 -- 172.21.15.46:0/3137689548 learned_addr learned my addr 172.21.15.46:0/3137689548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45fe00700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45f400700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 -- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 msgr2=0x5582110b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 -- 172.21.15.46:0/3137689548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 msgr2=0x5582110b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45ea00700 1 -- 172.21.15.46:0/3137689548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601a62d0 con 0x5582110ace00 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.796+0000 7ff45f400700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff45ea00700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff44c000c00 tx=0x7ff44c01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e599ed7a232e069 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff449600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c004d70 con 0x5582110ace00 2024-05-08T23:15:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558210f50270 con 0x5582110ace00 2024-05-08T23:15:55.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff44c030470 con 0x5582110ace00 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff44c004ed0 con 0x5582110ace00 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff47a01fb80 10 client.28987 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.797+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558210e24b60 con 0x5582110ace00 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558210f50270 con 0x5582110ace00 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff44c00f760 con 0x5582110ace00 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 10 client.28987 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 10 client.28987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 20 client.28987 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582110c10a0 0x5582110c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 4 client.28987 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558211076810 con 0x5582110c10a0 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.798+0000 7ff45f400700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582110c10a0 0x5582110c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.799+0000 7ff45f400700 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582110c10a0 0x5582110c34c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89c502cda3a0528 server_cookie=3683952a975e9900 in_seq=0 out_seq=0 2024-05-08T23:15:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.799+0000 7ff449600700 10 client.28987 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.799+0000 7ff449600700 1 -- 172.21.15.46:0/3137689548 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558211076810 con 0x5582110c10a0 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.799+0000 7ff449600700 10 client.28987 handle_command_reply: tid=0 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.799+0000 7ff47a01fb80 1 client.28987 shutdown 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 msgr2=0x5582110b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff44c000c00 tx=0x7ff44c01a050 comp rx=0 tx=0).stop 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 shutdown_connections 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5582110c10a0 0x5582110c34c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110a7890 0x5582110b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/3137689548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110ace00 0x5582110b11c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 >> 172.21.15.46:0/3137689548 conn(0x558211005210 msgr2=0x5582110a15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff449600700 0 client.28987 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:55.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 shutdown_connections 2024-05-08T23:15:55.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 1 -- 172.21.15.46:0/3137689548 wait complete. 2024-05-08T23:15:55.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.800+0000 7ff47a01fb80 20 client.28987 trim_cache size 0 max 16384 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 Processor -- start 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 -- start start 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ace00 0x5582110b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 0x5582110abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821109edc0 con 0x5582110ad720 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582110bde70 con 0x5582110ace00 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.816+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582110c86d0 con 0x5582110a7890 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49100/0 (socket says 172.21.15.46:49100) 2024-05-08T23:15:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 -- 172.21.15.46:0/614295579 learned_addr learned my addr 172.21.15.46:0/614295579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45ea00700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ace00 0x5582110b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 -- 172.21.15.46:0/614295579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ace00 msgr2=0x5582110b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ace00 0x5582110b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 -- 172.21.15.46:0/614295579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 msgr2=0x5582110abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45fe00700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 0x5582110abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 0x5582110abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 -- 172.21.15.46:0/614295579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558210fbd560 con 0x5582110a7890 2024-05-08T23:15:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45fe00700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 0x5582110abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.817+0000 7ff45f400700 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff4500cd280 tx=0x7ff45009a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34082a4037fbf0d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.818+0000 7ff455600700 1 -- 172.21.15.46:0/614295579 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450098350 con 0x5582110a7890 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.818+0000 7ff455600700 1 -- 172.21.15.46:0/614295579 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4500966c0 con 0x5582110a7890 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.818+0000 7ff455600700 1 -- 172.21.15.46:0/614295579 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45009a210 con 0x5582110a7890 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.818+0000 7ff47a01fb80 1 -- 172.21.15.46:0/614295579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 msgr2=0x5582110a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.818+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff4500cd280 tx=0x7ff45009a130 comp rx=0 tx=0).stop 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 -- 172.21.15.46:0/614295579 shutdown_connections 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ad720 0x5582110abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110a1d20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/614295579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ace00 0x5582110b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 -- 172.21.15.46:0/614295579 >> 172.21.15.46:0/614295579 conn(0x558211005210 msgr2=0x5582110af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 -- 172.21.15.46:0/614295579 shutdown_connections 2024-05-08T23:15:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 -- 172.21.15.46:0/614295579 wait complete. 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 -- start start 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.819+0000 7ff47a01fb80 1 Processor -- start 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ace00 0x5582110b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211116720 con 0x5582110ace00 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211116890 con 0x5582110ad720 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211116a50 con 0x5582110a7890 2024-05-08T23:15:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff47a01fb80 10 client.0 _ll_register_callbacks cb 0x558210f9cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49112/0 (socket says 172.21.15.46:49112) 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45f400700 1 -- 172.21.15.46:0/2823522200 learned_addr learned my addr 172.21.15.46:0/2823522200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45ea00700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ace00 0x5582110b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45fe00700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45f400700 1 -- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 msgr2=0x5582110b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.820+0000 7ff45f400700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff45f400700 1 -- 172.21.15.46:0/2823522200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ace00 msgr2=0x5582110b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff45f400700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ace00 0x5582110b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff45f400700 1 -- 172.21.15.46:0/2823522200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558210e783b0 con 0x5582110a7890 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff45f400700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff450096460 tx=0x7ff450098260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d4a5cb6e4f2ce5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff449600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450096830 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff45009a210 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4500980f0 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff45009a3d0 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff47a01fb80 10 client.28993 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.821+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558210e7ccd0 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4500980f0 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff450094bc0 con 0x5582110a7890 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 10 client.28993 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 10 client.28993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 20 client.28993 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558211117c90 0x55821111a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 4 client.28993 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558210fefec0 con 0x558211117c90 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.822+0000 7ff45ea00700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558211117c90 0x55821111a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff45ea00700 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558211117c90 0x55821111a080 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29fe94b569e8b46 server_cookie=790c15792c7d2608 in_seq=0 out_seq=0 2024-05-08T23:15:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff449600700 10 client.28993 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff449600700 1 -- 172.21.15.46:0/2823522200 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558210fefec0 con 0x558211117c90 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff449600700 10 client.28993 handle_command_reply: tid=0 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 client.28993 shutdown 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 msgr2=0x5582110b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff450096460 tx=0x7ff450098260 comp rx=0 tx=0).stop 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 shutdown_connections 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582110ad720 0x5582110b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582110a7890 0x5582110b61a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558211117c90 0x55821111a080 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 --2- 172.21.15.46:0/2823522200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5582110ace00 0x5582110b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff449600700 0 client.28993 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 >> 172.21.15.46:0/2823522200 conn(0x558211005210 msgr2=0x5582110abd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 shutdown_connections 2024-05-08T23:15:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.823+0000 7ff47a01fb80 1 -- 172.21.15.46:0/2823522200 wait complete. 2024-05-08T23:15:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff47a01fb80 20 client.28993 trim_cache size 0 max 16384 2024-05-08T23:15:55.827 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:15:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 msgr2=0x7ff4440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 0x7ff4440a37d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff46019aab0 tx=0x7ff44c00b260 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 msgr2=0x7ff460199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff45000e9c0 tx=0x7ff45000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 shutdown_connections 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff46010d6a0 0x7ff46019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff460073350 0x7ff460199560 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff4440a1310 0x7ff4440a37d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 --2- 172.21.15.46:0/2581170907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff460073c70 0x7ff460199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 >> 172.21.15.46:0/2581170907 conn(0x7ff4600fc8b0 msgr2=0x7ff4600fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 shutdown_connections 2024-05-08T23:15:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:55.824+0000 7ff466600700 1 -- 172.21.15.46:0/2581170907 wait complete. 2024-05-08T23:15:55.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1058612 9.26 MB/sec execute 497 sec latency 18.148 ms 2024-05-08T23:15:56.844 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:15:56.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:15:56.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1059523 9.25 MB/sec execute 498 sec latency 23.949 ms 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 -- 172.21.15.46:0/601598478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a581061a0 msgr2=0x7f9a5810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 --2- 172.21.15.46:0/601598478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a581061a0 0x7f9a5810dd20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400b0a0 tx=0x7f9a5400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 -- 172.21.15.46:0/601598478 shutdown_connections 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 --2- 172.21.15.46:0/601598478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a581061a0 0x7f9a5810dd20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 --2- 172.21.15.46:0/601598478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a58101080 0x7f9a58101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 --2- 172.21.15.46:0/601598478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58101a30 0x7f9a58105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 -- 172.21.15.46:0/601598478 >> 172.21.15.46:0/601598478 conn(0x7f9a580fc8e0 msgr2=0x7f9a580fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 -- 172.21.15.46:0/601598478 shutdown_connections 2024-05-08T23:15:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.060+0000 7f9a60800700 1 -- 172.21.15.46:0/601598478 wait complete. 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.061+0000 7f9a60800700 1 Processor -- start 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.061+0000 7f9a60800700 1 -- start start 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 0x7f9a580737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58073d20 0x7f9a581a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581a5c60 con 0x7f9a58073d20 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581a5dd0 con 0x7f9a581061a0 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a60800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581a5f90 con 0x7f9a58101080 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 0x7f9a580737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49124/0 (socket says 172.21.15.46:49124) 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 0x7f9a580737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37488/0 (socket says 172.21.15.46:37488) 2024-05-08T23:15:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 -- 172.21.15.46:0/574665929 learned_addr learned my addr 172.21.15.46:0/574665929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5e800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58073d20 0x7f9a581a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 -- 172.21.15.46:0/574665929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 msgr2=0x7f9a580737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 0x7f9a580737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 -- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58073d20 msgr2=0x7f9a581a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58073d20 0x7f9a581a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.062+0000 7f9a5de00700 1 -- 172.21.15.46:0/574665929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54009d00 con 0x7f9a58101080 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a5de00700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9c0 tx=0x7f9a4c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a52a00700 1 -- 172.21.15.46:0/574665929 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0097b0 con 0x7f9a58101080 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a581a6270 con 0x7f9a58101080 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a52a00700 1 -- 172.21.15.46:0/574665929 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c00ee60 con 0x7f9a58101080 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a52a00700 1 -- 172.21.15.46:0/574665929 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0054f0 con 0x7f9a58101080 2024-05-08T23:15:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.063+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a581a6730 con 0x7f9a58101080 2024-05-08T23:15:57.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.067+0000 7f9a52a00700 1 -- 172.21.15.46:0/574665929 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9a4c004750 con 0x7f9a58101080 2024-05-08T23:15:57.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 Processor -- start 2024-05-08T23:15:57.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a52a00700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 0x7f9a400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 -- start start 2024-05-08T23:15:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a52a00700 1 -- 172.21.15.46:0/574665929 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9a4c014070 con 0x7f9a58101080 2024-05-08T23:15:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a5e800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 0x7f9a400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bd7a0 0x55b89a9b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a90e1d0 con 0x55b89a9bd7a0 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a90bb90 con 0x55b89a9b7910 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a907480 con 0x55b89a9bce80 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49126/0 (socket says 172.21.15.46:49126) 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.069+0000 7f9a5de00700 1 -- 172.21.15.46:0/1351690229 learned_addr learned my addr 172.21.15.46:0/1351690229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5e800700 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bd7a0 0x55b89a9b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 -- 172.21.15.46:0/1351690229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 msgr2=0x55b89a9b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 -- 172.21.15.46:0/1351690229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bd7a0 msgr2=0x55b89a9b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bd7a0 0x55b89a9b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 -- 172.21.15.46:0/1351690229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89a788430 con 0x55b89a9bce80 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c096ad0 tx=0x7f9a4c09a830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda9b36ea9e27ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a5e800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 0x7f9a400a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400b780 tx=0x7f9a54007d70 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a47000700 1 -- 172.21.15.46:0/1351690229 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00cbd0 con 0x55b89a9bce80 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a47000700 1 -- 172.21.15.46:0/1351690229 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c0056b0 con 0x55b89a9bce80 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.070+0000 7f9a47000700 1 -- 172.21.15.46:0/1351690229 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c016330 con 0x55b89a9bce80 2024-05-08T23:15:57.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1351690229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 msgr2=0x55b89a9bd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c096ad0 tx=0x7f9a4c09a830 comp rx=0 tx=0).stop 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1351690229 shutdown_connections 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bce80 0x55b89a9bd260 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1351690229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bd7a0 0x55b89a9b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1351690229 >> 172.21.15.46:0/1351690229 conn(0x55b89a915290 msgr2=0x55b89a9b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1351690229 shutdown_connections 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.071+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1351690229 wait complete. 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.072+0000 7f9a743f2b80 1 -- start start 2024-05-08T23:15:57.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.072+0000 7f9a743f2b80 1 Processor -- start 2024-05-08T23:15:57.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.072+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 0x55b89a9c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.072+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 0x55b89a9c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a8a35e0 con 0x55b89a9b7910 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a8a3280 con 0x55b89a9bce80 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a78a720 con 0x55b89a9bd7a0 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a743f2b80 10 client.0 _ll_register_callbacks cb 0x55b89a8acbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 0x55b89a9c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49130/0 (socket says 172.21.15.46:49130) 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 -- 172.21.15.46:0/3082433667 learned_addr learned my addr 172.21.15.46:0/3082433667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5d400700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 0x55b89a9c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 -- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 msgr2=0x55b89a9c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 0x55b89a9c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 -- 172.21.15.46:0/3082433667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 msgr2=0x55b89a9c0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 0x55b89a9c0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5e800700 1 -- 172.21.15.46:0/3082433667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a581a6730 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5de00700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 0x55b89a9c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.073+0000 7f9a5d400700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 0x55b89a9c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a5e800700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9a54015000 tx=0x7f9a5400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7400b76d6613cd63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400e040 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b89a8602f0 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9a5401f650 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a54003e90 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a743f2b80 10 client.29005 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b89a734be0 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b89a8602f0 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.074+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a5402a970 con 0x55b89a9bd7a0 2024-05-08T23:15:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 10 client.29005 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 10 client.29005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 20 client.29005 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89a9d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 4 client.29005 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89a986890 con 0x55b89a9d12e0 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a5de00700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89a9d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a5de00700 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89a9d3700 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19d673443532a97 server_cookie=e8b7fc856b6b0ff0 in_seq=0 out_seq=0 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.075+0000 7f9a3b400700 10 client.29005 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.076+0000 7f9a3b400700 1 -- 172.21.15.46:0/3082433667 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b89a986890 con 0x55b89a9d12e0 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.076+0000 7f9a3b400700 10 client.29005 handle_command_reply: tid=0 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.076+0000 7f9a743f2b80 1 client.29005 shutdown 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 msgr2=0x55b89a9c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9a54015000 tx=0x7f9a5400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 shutdown_connections 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c15a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89a9d3700 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9b7910 0x55b89a9c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a3b400700 0 client.29005 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/3082433667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bce80 0x55b89a9c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 >> 172.21.15.46:0/3082433667 conn(0x55b89a915290 msgr2=0x55b89a9b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 shutdown_connections 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 1 -- 172.21.15.46:0/3082433667 wait complete. 2024-05-08T23:15:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.077+0000 7f9a743f2b80 20 client.29005 trim_cache size 0 max 16384 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 Processor -- start 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 -- start start 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bd7a0 0x55b89a9bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9c3690 con 0x55b89a9bce80 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9aee40 con 0x55b89a9bd7a0 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.099+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9bd5e0 con 0x55b89a9b7910 2024-05-08T23:15:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49140/0 (socket says 172.21.15.46:49140) 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 -- 172.21.15.46:0/1116430044 learned_addr learned my addr 172.21.15.46:0/1116430044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5e800700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bd7a0 0x55b89a9bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 -- 172.21.15.46:0/1116430044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bd7a0 msgr2=0x55b89a9bc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bd7a0 0x55b89a9bc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 -- 172.21.15.46:0/1116430044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 msgr2=0x55b89a9b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 -- 172.21.15.46:0/1116430044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89a9ae560 con 0x55b89a9b7910 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.100+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c096a80 tx=0x7f9a4c099ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739f1f7ed91ed68e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a47a00700 1 -- 172.21.15.46:0/1116430044 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00cbd0 con 0x55b89a9b7910 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a47a00700 1 -- 172.21.15.46:0/1116430044 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c098400 con 0x55b89a9b7910 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a47a00700 1 -- 172.21.15.46:0/1116430044 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c09a2c0 con 0x55b89a9b7910 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1116430044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 msgr2=0x55b89a9b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c096a80 tx=0x7f9a4c099ba0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1116430044 shutdown_connections 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9bd7a0 0x55b89a9bc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9b7910 0x55b89a9b73d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1116430044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.101+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1116430044 >> 172.21.15.46:0/1116430044 conn(0x55b89a915290 msgr2=0x55b89a9bbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.102+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1116430044 shutdown_connections 2024-05-08T23:15:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.102+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1116430044 wait complete. 2024-05-08T23:15:57.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.102+0000 7f9a743f2b80 1 -- start start 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.102+0000 7f9a743f2b80 1 Processor -- start 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9c7060 con 0x55b89a9bce80 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9c71d0 con 0x55b89a9b7910 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a9c7390 con 0x55b89a9bd7a0 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a743f2b80 10 client.0 _ll_register_callbacks cb 0x55b89a8acbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37532/0 (socket says 172.21.15.46:37532) 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 -- 172.21.15.46:0/1226649521 learned_addr learned my addr 172.21.15.46:0/1226649521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5e800700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 -- 172.21.15.46:0/1226649521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 msgr2=0x55b89a9c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 -- 172.21.15.46:0/1226649521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 msgr2=0x55b89a9c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.103+0000 7f9a5de00700 1 -- 172.21.15.46:0/1226649521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89a788430 con 0x55b89a9b7910 2024-05-08T23:15:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a5de00700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c5b0 tx=0x7f9a4c09e010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957ee421256d396d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a3b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c016530 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9a4c0b81c0 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a4c0056b0 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a4c0b8380 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a743f2b80 10 client.28916 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b89a9b7fc0 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.104+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c0056b0 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a4c0a2050 con 0x55b89a9b7910 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 10 client.28916 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 10 client.28916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 20 client.28916 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89aa2a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 4 client.28916 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b89a8fff40 con 0x55b89a9d12e0 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.105+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89aa2a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.106+0000 7f9a5d400700 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89aa2a630 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a268546bd55e1d5a server_cookie=f8d76481a6b564f7 in_seq=0 out_seq=0 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.106+0000 7f9a3b400700 10 client.28916 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a3b400700 1 -- 172.21.15.46:0/1226649521 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b89a8fff40 con 0x55b89a9d12e0 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a3b400700 10 client.28916 handle_command_reply: tid=0 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 client.28916 shutdown 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 msgr2=0x55b89a9c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c5b0 tx=0x7f9a4c09e010 comp rx=0 tx=0).stop 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 shutdown_connections 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89a9bd7a0 0x55b89a9c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b89a9b7910 0x55b89a9c8250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b89a9d12e0 0x55b89aa2a630 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 --2- 172.21.15.46:0/1226649521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b89a9bce80 0x55b89a9c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 >> 172.21.15.46:0/1226649521 conn(0x55b89a915290 msgr2=0x55b89a9c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a3b400700 0 client.28916 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 shutdown_connections 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.107+0000 7f9a743f2b80 1 -- 172.21.15.46:0/1226649521 wait complete. 2024-05-08T23:15:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a743f2b80 20 client.28916 trim_cache size 0 max 16384 2024-05-08T23:15:57.113 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:15:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ca8f0452-659a-436f-86c8-d3f60ff0cc17", 2024-05-08T23:15:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:15:57.113 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 msgr2=0x7f9a400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 0x7f9a400a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400b780 tx=0x7f9a54007d70 comp rx=0 tx=0).stop 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 msgr2=0x7f9a580732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9c0 tx=0x7f9a4c00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 shutdown_connections 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9a400a1290 0x7f9a400a3750 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a58073d20 0x7f9a581a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a58101080 0x7f9a580732a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 --2- 172.21.15.46:0/574665929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a581061a0 0x7f9a580737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.108+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 >> 172.21.15.46:0/574665929 conn(0x7f9a580fc8e0 msgr2=0x7f9a580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.109+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 shutdown_connections 2024-05-08T23:15:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.109+0000 7f9a60800700 1 -- 172.21.15.46:0/574665929 wait complete. 2024-05-08T23:15:57.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:15:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 -- 172.21.15.46:0/1576543998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101a00 msgr2=0x7f0660105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 --2- 172.21.15.46:0/1576543998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101a00 0x7f0660105a40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f064c009990 tx=0x7f064c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 -- 172.21.15.46:0/1576543998 shutdown_connections 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 --2- 172.21.15.46:0/1576543998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f066010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 --2- 172.21.15.46:0/1576543998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0660101050 0x7f0660101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 --2- 172.21.15.46:0/1576543998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101a00 0x7f0660105a40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 -- 172.21.15.46:0/1576543998 >> 172.21.15.46:0/1576543998 conn(0x7f06600fc8b0 msgr2=0x7f06600fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 -- 172.21.15.46:0/1576543998 shutdown_connections 2024-05-08T23:15:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.374+0000 7f0665800700 1 -- 172.21.15.46:0/1576543998 wait complete. 2024-05-08T23:15:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.375+0000 7f0665800700 1 Processor -- start 2024-05-08T23:15:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 -- start start 2024-05-08T23:15:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f06601978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066019bc40 0x7f066019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066019c6c0 con 0x7f0660106170 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066019c830 con 0x7f066019bc40 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.376+0000 7f0665800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066019c9f0 con 0x7f0660101050 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f06601978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f06601978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39838/0 (socket says 172.21.15.46:39838) 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065e000700 1 -- 172.21.15.46:0/636986285 learned_addr learned my addr 172.21.15.46:0/636986285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49164/0 (socket says 172.21.15.46:49164) 2024-05-08T23:15:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065f400700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066019bc40 0x7f066019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 -- 172.21.15.46:0/636986285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066019bc40 msgr2=0x7f066019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066019bc40 0x7f066019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 -- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 msgr2=0x7f06601978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f06601978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.377+0000 7f065ea00700 1 -- 172.21.15.46:0/636986285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0095f0 con 0x7f0660101050 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f065ea00700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0650009be0 tx=0x7f0650009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f0657400700 1 -- 172.21.15.46:0/636986285 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650015070 con 0x7f0660101050 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f0657400700 1 -- 172.21.15.46:0/636986285 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0650004830 con 0x7f0660101050 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f0657400700 1 -- 172.21.15.46:0/636986285 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500054c0 con 0x7f0660101050 2024-05-08T23:15:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0660073230 con 0x7f0660101050 2024-05-08T23:15:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.378+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06600736f0 con 0x7f0660101050 2024-05-08T23:15:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.381+0000 7f0657400700 1 -- 172.21.15.46:0/636986285 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0650005620 con 0x7f0660101050 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0657400700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 0x7f06440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 Processor -- start 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 -- start start 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0657400700 1 -- 172.21.15.46:0/636986285 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06500d08f0 con 0x7f0660101050 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f065e000700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 0x7f06440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683d77150 con 0x561683e25e00 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683d74b10 con 0x561683e26720 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f0679262b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683d70400 con 0x561683e20890 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f065ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f065f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f065f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49172/0 (socket says 172.21.15.46:49172) 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.383+0000 7f065f400700 1 -- 172.21.15.46:0/709867091 learned_addr learned my addr 172.21.15.46:0/709867091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065e000700 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 -- 172.21.15.46:0/709867091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 msgr2=0x561683e20cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 -- 172.21.15.46:0/709867091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 msgr2=0x561683e20350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e20350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 -- 172.21.15.46:0/709867091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561683bf13b0 con 0x561683e25e00 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065f400700 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065e000700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 0x7f06440a37d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06601988b0 tx=0x7f064c005110 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f065ea00700 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e261e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f06500cdbd0 tx=0x7f065009d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1849683295c54bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f064be00700 1 -- 172.21.15.46:0/709867091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065009dc90 con 0x561683e25e00 2024-05-08T23:15:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.384+0000 7f064be00700 1 -- 172.21.15.46:0/709867091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06500b7070 con 0x561683e25e00 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f064be00700 1 -- 172.21.15.46:0/709867091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500a1200 con 0x561683e25e00 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 -- 172.21.15.46:0/709867091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 msgr2=0x561683e261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e261e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f06500cdbd0 tx=0x7f065009d1c0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 -- 172.21.15.46:0/709867091 shutdown_connections 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e20cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e261e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 --2- 172.21.15.46:0/709867091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e20350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 -- 172.21.15.46:0/709867091 >> 172.21.15.46:0/709867091 conn(0x561683d7e210 msgr2=0x561683e1adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 -- 172.21.15.46:0/709867091 shutdown_connections 2024-05-08T23:15:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.385+0000 7f0679262b80 1 -- 172.21.15.46:0/709867091 wait complete. 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.386+0000 7f0679262b80 1 -- start start 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.386+0000 7f0679262b80 1 Processor -- start 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e29fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e2a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683d0c560 con 0x561683e25e00 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683d0c200 con 0x561683e26720 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683bf36a0 con 0x561683e20890 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f0679262b80 10 client.0 _ll_register_callbacks cb 0x561683d15b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:57.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f065ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f065ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49180/0 (socket says 172.21.15.46:49180) 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f065ea00700 1 -- 172.21.15.46:0/3876808230 learned_addr learned my addr 172.21.15.46:0/3876808230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f065e000700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e29fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.387+0000 7f065f400700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e2a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 -- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 msgr2=0x561683e2a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e2a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 -- 172.21.15.46:0/3876808230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 msgr2=0x561683e29fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e29fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 -- 172.21.15.46:0/3876808230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0658011330 con 0x561683e20890 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f065ea00700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bb0 tx=0x7f0650099ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e7e72298502df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0648c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:15:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500a13c0 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06500a5a10 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561683cc9270 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06500a1170 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0679262b80 10 client.29023 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.388+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f06600736f0 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561683cc9270 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0650083020 con 0x561683e20890 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 10 client.29023 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 10 client.29023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 20 client.29023 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e39e30 0x561683e3c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 4 client.29023 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:15:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561683def810 con 0x561683e39e30 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.389+0000 7f065e000700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e39e30 0x561683e3c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.390+0000 7f065e000700 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e39e30 0x561683e3c220 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d17f541ab7017c3 server_cookie=cf0260350dee5326 in_seq=0 out_seq=0 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.390+0000 7f0648c00700 10 client.29023 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.390+0000 7f0648c00700 1 -- 172.21.15.46:0/3876808230 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561683def810 con 0x561683e39e30 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.390+0000 7f0648c00700 10 client.29023 handle_command_reply: tid=0 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 client.29023 shutdown 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 msgr2=0x561683e29a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bb0 tx=0x7f0650099ed0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 shutdown_connections 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e2a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e20890 0x561683e29a70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e39e30 0x561683e3c220 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 --2- 172.21.15.46:0/3876808230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e25e00 0x561683e29fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 >> 172.21.15.46:0/3876808230 conn(0x561683d7e210 msgr2=0x561683e1a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0648c00700 0 client.29023 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 shutdown_connections 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 1 -- 172.21.15.46:0/3876808230 wait complete. 2024-05-08T23:15:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.391+0000 7f0679262b80 20 client.29023 trim_cache size 0 max 16384 2024-05-08T23:15:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.416+0000 7f0679262b80 1 Processor -- start 2024-05-08T23:15:57.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.416+0000 7f0679262b80 1 -- start start 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e1ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e26720 0x561683e2a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e13040 con 0x561683e26720 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e1b260 con 0x561683e20890 2024-05-08T23:15:57.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f0679262b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e36980 con 0x561683e25e00 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e1ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37572/0 (socket says 172.21.15.46:37572) 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 -- 172.21.15.46:0/1444534736 learned_addr learned my addr 172.21.15.46:0/1444534736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065f400700 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e26720 0x561683e2a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 -- 172.21.15.46:0/1444534736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 msgr2=0x561683e1ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e1ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 -- 172.21.15.46:0/1444534736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e26720 msgr2=0x561683e2a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e26720 0x561683e2a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065ea00700 1 -- 172.21.15.46:0/1444534736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561683b9db60 con 0x561683e20890 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.417+0000 7f065e000700 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.418+0000 7f065ea00700 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f065009a000 tx=0x7f065009d1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cda30ca3e0b8259 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.418+0000 7f0654c00700 1 -- 172.21.15.46:0/1444534736 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065009dc90 con 0x561683e20890 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.418+0000 7f0654c00700 1 -- 172.21.15.46:0/1444534736 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065009cd10 con 0x561683e20890 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.418+0000 7f0654c00700 1 -- 172.21.15.46:0/1444534736 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065001f2c0 con 0x561683e20890 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- 172.21.15.46:0/1444534736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 msgr2=0x561683e20350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f065009a000 tx=0x7f065009d1c0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- 172.21.15.46:0/1444534736 shutdown_connections 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e26720 0x561683e2a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e20890 0x561683e20350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 --2- 172.21.15.46:0/1444534736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- 172.21.15.46:0/1444534736 >> 172.21.15.46:0/1444534736 conn(0x561683d7e210 msgr2=0x561683e23bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- 172.21.15.46:0/1444534736 shutdown_connections 2024-05-08T23:15:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- 172.21.15.46:0/1444534736 wait complete. 2024-05-08T23:15:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 -- start start 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.419+0000 7f0679262b80 1 Processor -- start 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e23570 con 0x561683e20890 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e236e0 con 0x561683e26720 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683e21e70 con 0x561683e25e00 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f0679262b80 10 client.0 _ll_register_callbacks cb 0x561683d15b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f065ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f065e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f065ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39872/0 (socket says 172.21.15.46:39872) 2024-05-08T23:15:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f065ea00700 1 -- 172.21.15.46:0/3949588908 learned_addr learned my addr 172.21.15.46:0/3949588908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.420+0000 7f065f400700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 -- 172.21.15.46:0/3949588908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 msgr2=0x561683e22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 -- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 msgr2=0x561683e21930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e21930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 -- 172.21.15.46:0/3949588908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066010e560 con 0x561683e20890 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f065ea00700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f065009d520 tx=0x7f0650099eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4291c6fbd72a06e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0648c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065009dc90 con 0x561683e20890 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06500a13c0 con 0x561683e20890 2024-05-08T23:15:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06500b7070 con 0x561683e20890 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06500a1170 con 0x561683e20890 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0679262b80 10 client.18909 fetch_fsmap learned FSMap version 30 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.421+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f06600736f0 con 0x561683e20890 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06500b7070 con 0x561683e20890 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0650083070 con 0x561683e20890 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 10 client.18909 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 20 client.18909 populate_metadata read hostname 'smithi046' 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e90a30 0x561683e92e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 4 client.18909 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561683d68ec0 con 0x561683e90a30 2024-05-08T23:15:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.422+0000 7f065e000700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e90a30 0x561683e92e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f065e000700 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e90a30 0x561683e92e20 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97fc01f3e3dad3f9 server_cookie=778b5e42df874b3c in_seq=0 out_seq=0 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0648c00700 10 client.18909 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0648c00700 1 -- 172.21.15.46:0/3949588908 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561683d68ec0 con 0x561683e90a30 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0648c00700 10 client.18909 handle_command_reply: tid=0 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 client.18909 shutdown 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 msgr2=0x561683e31690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f065009d520 tx=0x7f0650099eb0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 shutdown_connections 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561683e26720 0x561683e21930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561683e90a30 0x561683e92e20 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561683e20890 0x561683e31690 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 --2- 172.21.15.46:0/3949588908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561683e25e00 0x561683e22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0648c00700 0 client.18909 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 >> 172.21.15.46:0/3949588908 conn(0x561683d7e210 msgr2=0x561683e28320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.423+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 shutdown_connections 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0679262b80 1 -- 172.21.15.46:0/3949588908 wait complete. 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0679262b80 20 client.18909 trim_cache size 0 max 16384 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:15:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "ca8f0452-659a-436f-86c8-d3f60ff0cc17": { 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ca8f0452-659a-436f-86c8-d3f60ff0cc17", 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:15:57.430 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 msgr2=0x7f06440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 0x7f06440a37d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06601988b0 tx=0x7f064c005110 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 msgr2=0x7f06601973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0650009be0 tx=0x7f0650009ef0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 shutdown_connections 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066019bc40 0x7f066019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660101050 0x7f06601973b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06440a1310 0x7f06440a37d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 --2- 172.21.15.46:0/636986285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0660106170 0x7f06601978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 >> 172.21.15.46:0/636986285 conn(0x7f06600fc8b0 msgr2=0x7f06600fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 shutdown_connections 2024-05-08T23:15:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:15:57.424+0000 7f0665800700 1 -- 172.21.15.46:0/636986285 wait complete. 2024-05-08T23:15:57.443 INFO:tasks.cephfs.filesystem:scrub status for tag:ca8f0452-659a-436f-86c8-d3f60ff0cc17 - {'path': '/', 'tag': 'ca8f0452-659a-436f-86c8-d3f60ff0cc17', 'options': 'recursive,force'} 2024-05-08T23:15:57.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:57 smithi046 ceph-mon[36898]: pgmap v377: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:15:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:57 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:15:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:57 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:57 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:57 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:57 smithi086 ceph-mon[78610]: pgmap v377: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:15:57.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:57 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:15:57.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:57 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:57 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:57.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:57 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:57 smithi053 ceph-mon[77587]: pgmap v377: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 126 op/s 2024-05-08T23:15:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:57 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:15:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:57 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:57 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:57 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:57.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1060945 9.24 MB/sec execute 499 sec latency 34.614 ms 2024-05-08T23:15:58.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:58 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:58.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:58 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:15:58.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:58 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:58 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:58 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:58 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:15:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:58 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:58 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:58 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:58 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:15:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:58 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:58 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:15:58.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1062301 9.25 MB/sec execute 500 sec latency 55.719 ms 2024-05-08T23:15:59.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:15:59 smithi086 ceph-mon[78610]: pgmap v378: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-08T23:15:59.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:15:59 smithi053 ceph-mon[77587]: pgmap v378: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-08T23:15:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:15:59 smithi046 ceph-mon[36898]: pgmap v378: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-05-08T23:15:59.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1063785 9.24 MB/sec execute 501 sec latency 32.901 ms 2024-05-08T23:16:00.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1064438 9.23 MB/sec execute 502 sec latency 164.721 ms 2024-05-08T23:16:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:01 smithi086 ceph-mon[78610]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-05-08T23:16:01.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:01 smithi053 ceph-mon[77587]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-05-08T23:16:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:01 smithi046 ceph-mon[36898]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-05-08T23:16:01.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1065809 9.23 MB/sec execute 503 sec latency 48.755 ms 2024-05-08T23:16:02.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1067073 9.22 MB/sec execute 504 sec latency 27.885 ms 2024-05-08T23:16:03.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:03 smithi086 ceph-mon[78610]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-05-08T23:16:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:03 smithi053 ceph-mon[77587]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-05-08T23:16:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:03 smithi046 ceph-mon[36898]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-05-08T23:16:03.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1069203 9.23 MB/sec execute 505 sec latency 14.738 ms 2024-05-08T23:16:04.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1070028 9.22 MB/sec execute 506 sec latency 19.512 ms 2024-05-08T23:16:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:05 smithi086 ceph-mon[78610]: pgmap v381: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-05-08T23:16:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:05 smithi053 ceph-mon[77587]: pgmap v381: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-05-08T23:16:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:05 smithi046 ceph-mon[36898]: pgmap v381: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-05-08T23:16:05.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1070623 9.21 MB/sec execute 507 sec latency 367.073 ms 2024-05-08T23:16:06.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1071955 9.20 MB/sec execute 508 sec latency 47.426 ms 2024-05-08T23:16:07.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:07 smithi086 ceph-mon[78610]: pgmap v382: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:16:07.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:07 smithi053 ceph-mon[77587]: pgmap v382: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:16:07.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:07 smithi046 ceph-mon[36898]: pgmap v382: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-08T23:16:07.914 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1073014 9.20 MB/sec execute 509 sec latency 25.680 ms 2024-05-08T23:16:08.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1074421 9.19 MB/sec execute 510 sec latency 16.681 ms 2024-05-08T23:16:09.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:09 smithi086 ceph-mon[78610]: pgmap v383: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:16:09.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:09 smithi053 ceph-mon[77587]: pgmap v383: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:16:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:09 smithi046 ceph-mon[36898]: pgmap v383: 129 pgs: 129 active+clean; 598 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-05-08T23:16:09.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1076397 9.20 MB/sec execute 511 sec latency 15.006 ms 2024-05-08T23:16:10.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:10 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:10.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:10 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:10.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:10 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:10.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:10 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:10.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:10 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:10.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:10 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:10.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1077240 9.18 MB/sec execute 512 sec latency 37.264 ms 2024-05-08T23:16:11.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:11.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:11.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: pgmap v384: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-05-08T23:16:11.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:16:11.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:11.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:11.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: pgmap v384: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: pgmap v384: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:16:11.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1079261 9.19 MB/sec execute 513 sec latency 24.429 ms 2024-05-08T23:16:12.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1080330 9.18 MB/sec execute 514 sec latency 17.314 ms 2024-05-08T23:16:13.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:13 smithi086 ceph-mon[78610]: pgmap v386: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 303 op/s 2024-05-08T23:16:13.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:13 smithi053 ceph-mon[77587]: pgmap v386: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 303 op/s 2024-05-08T23:16:13.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:13 smithi046 ceph-mon[36898]: pgmap v386: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 303 op/s 2024-05-08T23:16:13.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1082071 9.18 MB/sec execute 515 sec latency 19.362 ms 2024-05-08T23:16:14.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1083575 9.18 MB/sec execute 516 sec latency 49.342 ms 2024-05-08T23:16:15.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:15 smithi086 ceph-mon[78610]: pgmap v387: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:16:15.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:15 smithi053 ceph-mon[77587]: pgmap v387: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:16:15.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:15 smithi046 ceph-mon[36898]: pgmap v387: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 284 op/s 2024-05-08T23:16:15.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1085029 9.18 MB/sec execute 517 sec latency 25.557 ms 2024-05-08T23:16:16.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1087036 9.18 MB/sec execute 518 sec latency 15.067 ms 2024-05-08T23:16:17.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:17 smithi086 ceph-mon[78610]: pgmap v388: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:17.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:17 smithi053 ceph-mon[77587]: pgmap v388: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:17 smithi046 ceph-mon[36898]: pgmap v388: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:17.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1088115 9.17 MB/sec execute 519 sec latency 24.995 ms 2024-05-08T23:16:18.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1090172 9.18 MB/sec execute 520 sec latency 33.109 ms 2024-05-08T23:16:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:19 smithi086 ceph-mon[78610]: pgmap v389: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:19 smithi053 ceph-mon[77587]: pgmap v389: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:19 smithi046 ceph-mon[36898]: pgmap v389: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 272 op/s 2024-05-08T23:16:19.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1091104 9.17 MB/sec execute 521 sec latency 30.049 ms 2024-05-08T23:16:20.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1092989 9.17 MB/sec execute 522 sec latency 17.265 ms 2024-05-08T23:16:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:20 smithi086 ceph-mon[78610]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:16:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:20 smithi053 ceph-mon[77587]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:16:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:20 smithi046 ceph-mon[36898]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:16:21.915 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1094392 9.17 MB/sec execute 523 sec latency 16.080 ms 2024-05-08T23:16:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:21 smithi086 ceph-mon[78610]: pgmap v391: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-08T23:16:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:21 smithi053 ceph-mon[77587]: pgmap v391: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-08T23:16:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:21 smithi046 ceph-mon[36898]: pgmap v391: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-05-08T23:16:22.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1095915 9.16 MB/sec execute 524 sec latency 16.074 ms 2024-05-08T23:16:23.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1097720 9.17 MB/sec execute 525 sec latency 15.768 ms 2024-05-08T23:16:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:23 smithi086 ceph-mon[78610]: pgmap v392: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-05-08T23:16:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:23 smithi053 ceph-mon[77587]: pgmap v392: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-05-08T23:16:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:23 smithi046 ceph-mon[36898]: pgmap v392: 129 pgs: 129 active+clean; 596 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-05-08T23:16:24.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1099010 9.16 MB/sec execute 526 sec latency 75.909 ms 2024-05-08T23:16:25.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1101109 9.17 MB/sec execute 527 sec latency 17.953 ms 2024-05-08T23:16:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:25 smithi086 ceph-mon[78610]: pgmap v393: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-05-08T23:16:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:25 smithi053 ceph-mon[77587]: pgmap v393: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-05-08T23:16:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:25 smithi046 ceph-mon[36898]: pgmap v393: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-05-08T23:16:26.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1101944 9.16 MB/sec execute 528 sec latency 22.726 ms 2024-05-08T23:16:27.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:16:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 -- 172.21.15.46:0/2000335099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf641061a0 msgr2=0x7fbf6410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 --2- 172.21.15.46:0/2000335099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf641061a0 0x7fbf6410dd20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 -- 172.21.15.46:0/2000335099 shutdown_connections 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 --2- 172.21.15.46:0/2000335099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf641061a0 0x7fbf6410dd20 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 --2- 172.21.15.46:0/2000335099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf64101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 --2- 172.21.15.46:0/2000335099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf64101a30 0x7fbf64105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 -- 172.21.15.46:0/2000335099 >> 172.21.15.46:0/2000335099 conn(0x7fbf640fc8e0 msgr2=0x7fbf640fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.668+0000 7fbf6a400700 1 -- 172.21.15.46:0/2000335099 shutdown_connections 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.669+0000 7fbf6a400700 1 -- 172.21.15.46:0/2000335099 wait complete. 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.670+0000 7fbf6a400700 1 Processor -- start 2024-05-08T23:16:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.670+0000 7fbf6a400700 1 -- start start 2024-05-08T23:16:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf641061a0 0x7fbf641979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 0x7fbf6419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419c7b0 con 0x7fbf64101080 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419c920 con 0x7fbf641061a0 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf6a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419cae0 con 0x7fbf6419bd30 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57708/0 (socket says 172.21.15.46:57708) 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 -- 172.21.15.46:0/3720275789 learned_addr learned my addr 172.21.15.46:0/3720275789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63e00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 0x7fbf6419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf62a00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf641061a0 0x7fbf641979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 -- 172.21.15.46:0/3720275789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 msgr2=0x7fbf6419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 0x7fbf6419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 -- 172.21.15.46:0/3720275789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf641061a0 msgr2=0x7fbf641979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf641061a0 0x7fbf641979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63400700 1 -- 172.21.15.46:0/3720275789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009d00 con 0x7fbf64101080 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.671+0000 7fbf63e00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 0x7fbf6419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf63400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400d980 tx=0x7fbf5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf5be00700 1 -- 172.21.15.46:0/3720275789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54004220 con 0x7fbf64101080 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf5be00700 1 -- 172.21.15.46:0/3720275789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf54007640 con 0x7fbf64101080 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf641a3fe0 con 0x7fbf64101080 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf5be00700 1 -- 172.21.15.46:0/3720275789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54005690 con 0x7fbf64101080 2024-05-08T23:16:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.672+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf641a44c0 con 0x7fbf64101080 2024-05-08T23:16:27.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.675+0000 7fbf5be00700 1 -- 172.21.15.46:0/3720275789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbf54004750 con 0x7fbf64101080 2024-05-08T23:16:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.676+0000 7fbf7dee2b80 1 Processor -- start 2024-05-08T23:16:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 -- start start 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf5be00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 0x7fbf480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e27f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 0x56281e279cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e1d0150 con 0x56281e27ee00 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e1cdb10 con 0x56281e27f720 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e1c9400 con 0x56281e279890 2024-05-08T23:16:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.677+0000 7fbf5be00700 1 -- 172.21.15.46:0/3720275789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbf540ccf20 con 0x7fbf64101080 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e27f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 0x56281e279cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 0x56281e279cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37946/0 (socket says 172.21.15.46:37946) 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51522/0 (socket says 172.21.15.46:51522) 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63e00700 1 -- 172.21.15.46:0/1440979143 learned_addr learned my addr 172.21.15.46:0/1440979143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63e00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 0x7fbf480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 -- 172.21.15.46:0/1440979143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 msgr2=0x56281e279cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 0x56281e279cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 -- 172.21.15.46:0/1440979143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 msgr2=0x56281e27f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e27f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 -- 172.21.15.46:0/1440979143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281e04a3b0 con 0x56281e27f720 2024-05-08T23:16:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf6406a4a0 tx=0x7fbf50009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b1f8846da0fe0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.678+0000 7fbf63e00700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 0x7fbf480a3750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c007a60 tx=0x7fbf5c006210 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf58c00700 1 -- 172.21.15.46:0/1440979143 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50011cb0 con 0x56281e27f720 2024-05-08T23:16:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf58c00700 1 -- 172.21.15.46:0/1440979143 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf50011e10 con 0x56281e27f720 2024-05-08T23:16:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf58c00700 1 -- 172.21.15.46:0/1440979143 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50005280 con 0x56281e27f720 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1440979143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 msgr2=0x56281e279350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf6406a4a0 tx=0x7fbf50009f40 comp rx=0 tx=0).stop 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1440979143 shutdown_connections 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e279890 0x56281e279cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e27f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.679+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1440979143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e279350 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.680+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1440979143 >> 172.21.15.46:0/1440979143 conn(0x56281e1d7210 msgr2=0x56281e273df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.680+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1440979143 shutdown_connections 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.680+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1440979143 wait complete. 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.680+0000 7fbf7dee2b80 1 -- start start 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.680+0000 7fbf7dee2b80 1 Processor -- start 2024-05-08T23:16:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e2829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e165560 con 0x56281e27ee00 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e165200 con 0x56281e279890 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e04c6a0 con 0x56281e27f720 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf7dee2b80 10 client.0 _ll_register_callbacks cb 0x56281e16eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf62a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e2829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf63e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37958/0 (socket says 172.21.15.46:37958) 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51532/0 (socket says 172.21.15.46:51532) 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.681+0000 7fbf63e00700 1 -- 172.21.15.46:0/620103145 learned_addr learned my addr 172.21.15.46:0/620103145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 -- 172.21.15.46:0/620103145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 msgr2=0x56281e2830e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 -- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 msgr2=0x56281e285700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 -- 172.21.15.46:0/620103145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf500098c0 con 0x56281e27ee00 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf63e00700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf63400700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf62a00700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e2829f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009230 tx=0x7fbf50005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f9b9501205d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf4d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf500048d0 con 0x56281e27ee00 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbf50004260 con 0x56281e27ee00 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.682+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56281e122270 con 0x56281e27ee00 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf50004440 con 0x56281e27ee00 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 10 client.18921 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56281dff6b60 con 0x56281e27ee00 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56281e122270 con 0x56281e27ee00 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf50005050 con 0x56281e27ee00 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 10 client.18921 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 20 client.18921 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e293030 0x56281e295450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 4 client.18921 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.683+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56281e248810 con 0x56281e293030 2024-05-08T23:16:27.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.684+0000 7fbf63400700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e293030 0x56281e295450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.684+0000 7fbf63400700 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e293030 0x56281e295450 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce2d6e86d8c7e95 server_cookie=467c2c4a6a903bef in_seq=0 out_seq=0 2024-05-08T23:16:27.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.684+0000 7fbf4d600700 10 client.18921 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf4d600700 1 -- 172.21.15.46:0/620103145 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56281e248810 con 0x56281e293030 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf4d600700 10 client.18921 handle_command_reply: tid=0 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 client.18921 shutdown 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 msgr2=0x56281e2829f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e2829f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009230 tx=0x7fbf50005df0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 shutdown_connections 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e285700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e293030 0x56281e295450 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/620103145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e2829f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 >> 172.21.15.46:0/620103145 conn(0x56281e1d7210 msgr2=0x56281e2735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.685+0000 7fbf4d600700 0 client.18921 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.686+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 shutdown_connections 2024-05-08T23:16:27.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.686+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/620103145 wait complete. 2024-05-08T23:16:27.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.686+0000 7fbf7dee2b80 20 client.18921 trim_cache size 0 max 16384 2024-05-08T23:16:27.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 Processor -- start 2024-05-08T23:16:27.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 -- start start 2024-05-08T23:16:27.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e279350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e273d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e28fd10 con 0x56281e27ee00 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e26c070 con 0x56281e279890 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.705+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e2748e0 con 0x56281e27f720 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e279350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e279350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51544/0 (socket says 172.21.15.46:51544) 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63400700 1 -- 172.21.15.46:0/821715631 learned_addr learned my addr 172.21.15.46:0/821715631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf62a00700 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e273d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 -- 172.21.15.46:0/821715631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 msgr2=0x56281e279350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e279350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 -- 172.21.15.46:0/821715631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 msgr2=0x56281e273d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e273d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.706+0000 7fbf63e00700 1 -- 172.21.15.46:0/821715631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281e279f40 con 0x56281e27f720 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.707+0000 7fbf63e00700 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2812c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c664965ed20b7db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.707+0000 7fbf59600700 1 -- 172.21.15.46:0/821715631 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00e040 con 0x56281e27f720 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.707+0000 7fbf59600700 1 -- 172.21.15.46:0/821715631 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5c01fb00 con 0x56281e27f720 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.707+0000 7fbf59600700 1 -- 172.21.15.46:0/821715631 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c005540 con 0x56281e27f720 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/821715631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 msgr2=0x56281e2812c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2812c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c006000 comp rx=0 tx=0).stop 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/821715631 shutdown_connections 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27f720 0x56281e2812c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e279890 0x56281e279350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/821715631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e27ee00 0x56281e273d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/821715631 >> 172.21.15.46:0/821715631 conn(0x56281e1d7210 msgr2=0x56281e28ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/821715631 shutdown_connections 2024-05-08T23:16:27.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.708+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/821715631 wait complete. 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.709+0000 7fbf7dee2b80 1 -- start start 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.709+0000 7fbf7dee2b80 1 Processor -- start 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e279890 0x56281e28a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e27a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e27c570 con 0x56281e279890 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e27c6e0 con 0x56281e27f720 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281e27ada0 con 0x56281e27ee00 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf7dee2b80 10 client.0 _ll_register_callbacks cb 0x7fbf54095f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:37980/0 (socket says 172.21.15.46:37980) 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e279890 0x56281e28a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf62a00700 1 -- 172.21.15.46:0/1150998680 learned_addr learned my addr 172.21.15.46:0/1150998680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.710+0000 7fbf63e00700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e27a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 -- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 msgr2=0x56281e27a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e27a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 -- 172.21.15.46:0/1150998680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e279890 msgr2=0x56281e28a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e279890 0x56281e28a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 -- 172.21.15.46:0/1150998680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6410e590 con 0x56281e27ee00 2024-05-08T23:16:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf62a00700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005380 tx=0x7fbf500118c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91b4e3ddaf1d785 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf4d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50005180 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf50011e10 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.711+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbf50004260 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf50005c80 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 10 client.29047 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56281e2704e0 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf50011e10 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf50003cd0 con 0x56281e27ee00 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 10 client.29047 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 10 client.29047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 20 client.29047 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e2e9800 0x56281e2ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 4 client.29047 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.712+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56281e1c1ec0 con 0x56281e2e9800 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.713+0000 7fbf63400700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e2e9800 0x56281e2ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.713+0000 7fbf4d600700 10 client.29047 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.713+0000 7fbf63400700 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e2e9800 0x56281e2ebc20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb83ae2d9635b76d server_cookie=7cf6b8c45fdfbeba in_seq=0 out_seq=0 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.713+0000 7fbf4d600700 1 -- 172.21.15.46:0/1150998680 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56281e1c1ec0 con 0x56281e2e9800 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.713+0000 7fbf4d600700 10 client.29047 handle_command_reply: tid=0 2024-05-08T23:16:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 client.29047 shutdown 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 msgr2=0x56281e27bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005380 tx=0x7fbf500118c0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 shutdown_connections 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281e27f720 0x56281e27a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56281e2e9800 0x56281e2ebc20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56281e279890 0x56281e28a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 --2- 172.21.15.46:0/1150998680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281e27ee00 0x56281e27bc60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf4d600700 0 client.29047 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 >> 172.21.15.46:0/1150998680 conn(0x56281e1d7210 msgr2=0x56281e28ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 shutdown_connections 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 1 -- 172.21.15.46:0/1150998680 wait complete. 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.714+0000 7fbf7dee2b80 20 client.29047 trim_cache size 0 max 16384 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:16:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 msgr2=0x7fbf480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 0x7fbf480a3750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c007a60 tx=0x7fbf5c006210 comp rx=0 tx=0).stop 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 msgr2=0x7fbf641974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400d980 tx=0x7fbf5400dc90 comp rx=0 tx=0).stop 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 shutdown_connections 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf6419bd30 0x7fbf6419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbf480a1290 0x7fbf480a3750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf64101080 0x7fbf641974a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 --2- 172.21.15.46:0/3720275789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf641061a0 0x7fbf641979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 >> 172.21.15.46:0/3720275789 conn(0x7fbf640fc8e0 msgr2=0x7fbf640fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 shutdown_connections 2024-05-08T23:16:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.715+0000 7fbf6a400700 1 -- 172.21.15.46:0/3720275789 wait complete. 2024-05-08T23:16:27.736 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:16:27.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:16:27.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1104017 9.16 MB/sec execute 529 sec latency 30.021 ms 2024-05-08T23:16:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 -- 172.21.15.46:0/3033866175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00100e90 msgr2=0x7f0e00101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 --2- 172.21.15.46:0/3033866175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00100e90 0x7f0e00101270 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009990 tx=0x7f0df0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 -- 172.21.15.46:0/3033866175 shutdown_connections 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 --2- 172.21.15.46:0/3033866175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00105fb0 0x7f0e0010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 --2- 172.21.15.46:0/3033866175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00100e90 0x7f0e00101270 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 --2- 172.21.15.46:0/3033866175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00101840 0x7f0e00105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.973+0000 7f0e04c00700 1 -- 172.21.15.46:0/3033866175 >> 172.21.15.46:0/3033866175 conn(0x7f0e000fc730 msgr2=0x7f0e000feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.974+0000 7f0e04c00700 1 -- 172.21.15.46:0/3033866175 shutdown_connections 2024-05-08T23:16:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.974+0000 7f0e04c00700 1 -- 172.21.15.46:0/3033866175 wait complete. 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 Processor -- start 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 -- start start 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 0x7f0e000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00073d20 0x7f0e001a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001a59c0 con 0x7f0e00100e90 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001a5b30 con 0x7f0e00105fb0 2024-05-08T23:16:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.975+0000 7f0e04c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001a5cf0 con 0x7f0e00073d20 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dff400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 0x7f0e000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51582/0 (socket says 172.21.15.46:51582) 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 -- 172.21.15.46:0/2155223330 learned_addr learned my addr 172.21.15.46:0/2155223330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dffe00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00073d20 0x7f0e001a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dff400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 0x7f0e000732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57770/0 (socket says 172.21.15.46:57770) 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 -- 172.21.15.46:0/2155223330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00073d20 msgr2=0x7f0e001a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00073d20 0x7f0e001a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 -- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 msgr2=0x7f0e000732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 0x7f0e000732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.976+0000 7f0dfea00700 1 -- 172.21.15.46:0/2155223330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00095f0 con 0x7f0e00105fb0 2024-05-08T23:16:27.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0dfea00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e970 tx=0x7f0dec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec009790 con 0x7f0e00105fb0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e00196f60 con 0x7f0e00105fb0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dec005020 con 0x7f0e00105fb0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00cb80 con 0x7f0e00105fb0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.977+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001974c0 con 0x7f0e00105fb0 2024-05-08T23:16:27.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.979+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e001091d0 con 0x7f0e00105fb0 2024-05-08T23:16:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.981+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0dec005490 con 0x7f0e00105fb0 2024-05-08T23:16:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.982+0000 7f0df7e00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 0x7f0de40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.983+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0dec014070 con 0x7f0e00105fb0 2024-05-08T23:16:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.983+0000 7f0dff400700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 0x7f0de40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.983+0000 7f0dff400700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 0x7f0de40a37d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b2e0 tx=0x7f0df0005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:27.987+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0dec096220 con 0x7f0e00105fb0 2024-05-08T23:16:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:27 smithi086 ceph-mon[78610]: pgmap v394: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:27 smithi053 ceph-mon[77587]: pgmap v394: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:27 smithi046 ceph-mon[36898]: pgmap v394: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.298+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e00074780 con 0x7f0e00105fb0 2024-05-08T23:16:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.299+0000 7f0df7e00700 1 -- 172.21.15.46:0/2155223330 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f0dec099840 con 0x7f0e00105fb0 2024-05-08T23:16:28.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:16:28.300 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:16:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 msgr2=0x7f0de40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 0x7f0de40a37d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b2e0 tx=0x7f0df0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 msgr2=0x7f0e000737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e970 tx=0x7f0dec00ec80 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 shutdown_connections 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e00073d20 0x7f0e001a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0de40a1310 0x7f0de40a37d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e00100e90 0x7f0e000732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 --2- 172.21.15.46:0/2155223330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e00105fb0 0x7f0e000737e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 >> 172.21.15.46:0/2155223330 conn(0x7f0e000fc730 msgr2=0x7f0e000fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 shutdown_connections 2024-05-08T23:16:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.302+0000 7f0e04c00700 1 -- 172.21.15.46:0/2155223330 wait complete. 2024-05-08T23:16:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:16:28.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 -- 172.21.15.46:0/1215669211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1032b0 msgr2=0x7fa41c103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 --2- 172.21.15.46:0/1215669211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1032b0 0x7fa41c103690 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa410009990 tx=0x7fa410009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 -- 172.21.15.46:0/1215669211 shutdown_connections 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 --2- 172.21.15.46:0/1215669211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1083d0 0x7fa41c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 --2- 172.21.15.46:0/1215669211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1032b0 0x7fa41c103690 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 --2- 172.21.15.46:0/1215669211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c103c60 0x7fa41c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 -- 172.21.15.46:0/1215669211 >> 172.21.15.46:0/1215669211 conn(0x7fa41c0feb10 msgr2=0x7fa41c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 -- 172.21.15.46:0/1215669211 shutdown_connections 2024-05-08T23:16:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.547+0000 7fa423800700 1 -- 172.21.15.46:0/1215669211 wait complete. 2024-05-08T23:16:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.548+0000 7fa423800700 1 Processor -- start 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 -- start start 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1032b0 0x7fa41c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c075f30 0x7fa41c1a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0796b0 con 0x7fa41c1032b0 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1aa080 con 0x7fa41c075f30 2024-05-08T23:16:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.549+0000 7fa423800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1aa240 con 0x7fa41c1083d0 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa420e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1032b0 0x7fa41c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa421800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c075f30 0x7fa41c1a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38022/0 (socket says 172.21.15.46:38022) 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 -- 172.21.15.46:0/2324028762 learned_addr learned my addr 172.21.15.46:0/2324028762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 -- 172.21.15.46:0/2324028762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c075f30 msgr2=0x7fa41c1a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c075f30 0x7fa41c1a9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 -- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1032b0 msgr2=0x7fa41c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1032b0 0x7fa41c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 -- 172.21.15.46:0/2324028762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100095f0 con 0x7fa41c1083d0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa41be00700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e970 tx=0x7fa40800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.550+0000 7fa419600700 1 -- 172.21.15.46:0/2324028762 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408009790 con 0x7fa41c1083d0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.551+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c1aa520 con 0x7fa41c1083d0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.551+0000 7fa419600700 1 -- 172.21.15.46:0/2324028762 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40800c7e0 con 0x7fa41c1083d0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.551+0000 7fa419600700 1 -- 172.21.15.46:0/2324028762 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080054e0 con 0x7fa41c1083d0 2024-05-08T23:16:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.551+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c1aaa30 con 0x7fa41c1083d0 2024-05-08T23:16:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.554+0000 7fa419600700 1 -- 172.21.15.46:0/2324028762 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa40800c950 con 0x7fa41c1083d0 2024-05-08T23:16:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.555+0000 7fa4372ecb80 1 Processor -- start 2024-05-08T23:16:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.555+0000 7fa4372ecb80 1 -- start start 2024-05-08T23:16:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79a9e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 0x55aa79a98cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa799ef150 con 0x55aa79a9de00 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa799ecb10 con 0x55aa79a98890 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa419600700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 0x7fa4040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa799e8400 con 0x55aa79a9e720 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38034/0 (socket says 172.21.15.46:38034) 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa420e00700 1 -- 172.21.15.46:0/3623369007 learned_addr learned my addr 172.21.15.46:0/3623369007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa419600700 1 -- 172.21.15.46:0/2324028762 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa408014070 con 0x7fa41c1083d0 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa421800700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79a9e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa41be00700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 0x55aa79a98cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa421800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 0x7fa4040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa420e00700 1 -- 172.21.15.46:0/3623369007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 msgr2=0x55aa79a98cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.556+0000 7fa420e00700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 0x55aa79a98cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa420e00700 1 -- 172.21.15.46:0/3623369007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 msgr2=0x55aa79a9e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa420e00700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79a9e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa420e00700 1 -- 172.21.15.46:0/3623369007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa798693b0 con 0x55aa79a9e720 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa41be00700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 0x55aa79a98cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa420e00700 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa410005ec0 tx=0x7fa410005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca23f446d491592 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa40e000700 1 -- 172.21.15.46:0/3623369007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001d070 con 0x55aa79a9e720 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa40e000700 1 -- 172.21.15.46:0/3623369007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41001e040 con 0x55aa79a9e720 2024-05-08T23:16:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.557+0000 7fa40e000700 1 -- 172.21.15.46:0/3623369007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410004880 con 0x55aa79a9e720 2024-05-08T23:16:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 -- 172.21.15.46:0/3623369007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 msgr2=0x55aa79a98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa410005ec0 tx=0x7fa410005ef0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa421800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 0x7fa4040a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa414009ec0 tx=0x7fa41400b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 -- 172.21.15.46:0/3623369007 shutdown_connections 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a98890 0x55aa79a98cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79a9e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/3623369007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a98350 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 -- 172.21.15.46:0/3623369007 >> 172.21.15.46:0/3623369007 conn(0x55aa799f6210 msgr2=0x55aa79a92df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 -- 172.21.15.46:0/3623369007 shutdown_connections 2024-05-08T23:16:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.558+0000 7fa4372ecb80 1 -- 172.21.15.46:0/3623369007 wait complete. 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.559+0000 7fa4372ecb80 1 -- start start 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.559+0000 7fa4372ecb80 1 Processor -- start 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79aa1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9e720 0x55aa79aa24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79984560 con 0x55aa79a9de00 2024-05-08T23:16:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79984200 con 0x55aa79a9e720 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa7986b6a0 con 0x55aa79a98890 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa4372ecb80 10 client.0 _ll_register_callbacks cb 0x55aa7998db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38046/0 (socket says 172.21.15.46:38046) 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa420e00700 1 -- 172.21.15.46:0/547381089 learned_addr learned my addr 172.21.15.46:0/547381089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa41be00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79aa1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.560+0000 7fa421800700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9e720 0x55aa79aa24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 -- 172.21.15.46:0/547381089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9e720 msgr2=0x55aa79aa24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9e720 0x55aa79aa24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 -- 172.21.15.46:0/547381089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 msgr2=0x55aa79aa1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79aa1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 -- 172.21.15.46:0/547381089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c1107c0 con 0x55aa79a98890 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa420e00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa410000c00 tx=0x7fa410005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9824ea245e7ec3a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa402a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001d030 con 0x55aa79a98890 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa410004500 con 0x55aa79a98890 2024-05-08T23:16:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.561+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa79941270 con 0x55aa79a98890 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4100046c0 con 0x55aa79a98890 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 10 client.29065 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aa79815b60 con 0x55aa79a98890 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa79941270 con 0x55aa79a98890 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa41000f800 con 0x55aa79a98890 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 10 client.29065 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 10 client.29065 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:16:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 20 client.29065 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79ab1c60 0x55aa79ab4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 4 client.29065 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.562+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa79a67810 con 0x55aa79ab1c60 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.563+0000 7fa41be00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79ab1c60 0x55aa79ab4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.563+0000 7fa41be00700 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79ab1c60 0x55aa79ab4050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d231fe8457d4d60 server_cookie=db32f35a6f455ab in_seq=0 out_seq=0 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.563+0000 7fa402a00700 10 client.29065 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa402a00700 1 -- 172.21.15.46:0/547381089 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa79a67810 con 0x55aa79ab1c60 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa402a00700 10 client.29065 handle_command_reply: tid=0 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 client.29065 shutdown 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 msgr2=0x55aa79aa1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa410000c00 tx=0x7fa410005af0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 shutdown_connections 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79ab1c60 0x55aa79ab4050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9e720 0x55aa79aa24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79aa1a30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/547381089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9de00 0x55aa79aa1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 >> 172.21.15.46:0/547381089 conn(0x55aa799f6210 msgr2=0x55aa79a925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa402a00700 0 client.29065 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 shutdown_connections 2024-05-08T23:16:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 1 -- 172.21.15.46:0/547381089 wait complete. 2024-05-08T23:16:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.564+0000 7fa4372ecb80 20 client.29065 trim_cache size 0 max 16384 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 Processor -- start 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 -- start start 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a98c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9e720 0x55aa79a94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a93a20 con 0x55aa79a9e720 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a8b040 con 0x55aa79a9de00 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.580+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a9c410 con 0x55aa79a98890 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38056/0 (socket says 172.21.15.46:38056) 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 -- 172.21.15.46:0/632313619 learned_addr learned my addr 172.21.15.46:0/632313619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa41be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a98c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa421800700 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9e720 0x55aa79a94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 -- 172.21.15.46:0/632313619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 msgr2=0x55aa79a98c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a98c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 -- 172.21.15.46:0/632313619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9e720 msgr2=0x55aa79a94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9e720 0x55aa79a94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 -- 172.21.15.46:0/632313619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa799ae560 con 0x55aa79a98890 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa420e00700 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa41000fb60 tx=0x7fa410009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7d383a1bfb4ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa40ea00700 1 -- 172.21.15.46:0/632313619 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001d040 con 0x55aa79a98890 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa40ea00700 1 -- 172.21.15.46:0/632313619 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41000f920 con 0x55aa79a98890 2024-05-08T23:16:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.581+0000 7fa40ea00700 1 -- 172.21.15.46:0/632313619 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41002b9a0 con 0x55aa79a98890 2024-05-08T23:16:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 -- 172.21.15.46:0/632313619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 msgr2=0x55aa79a98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa41000fb60 tx=0x7fa410009290 comp rx=0 tx=0).stop 2024-05-08T23:16:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 -- 172.21.15.46:0/632313619 shutdown_connections 2024-05-08T23:16:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a9e720 0x55aa79a94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a98890 0x55aa79a98350 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/632313619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a98c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 -- 172.21.15.46:0/632313619 >> 172.21.15.46:0/632313619 conn(0x55aa799f6210 msgr2=0x55aa79aa0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 -- 172.21.15.46:0/632313619 shutdown_connections 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.582+0000 7fa4372ecb80 1 -- 172.21.15.46:0/632313619 wait complete. 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 -- start start 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 Processor -- start 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a98890 0x55aa79aa9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a99930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a9a3b0 con 0x55aa79a98890 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a9a4f0 con 0x55aa79a9de00 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa79a9a6b0 con 0x55aa79a9e720 2024-05-08T23:16:28.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.583+0000 7fa4372ecb80 10 client.0 _ll_register_callbacks cb 0x55aa7998db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa420e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a98890 0x55aa79aa9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa41be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a99930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38068/0 (socket says 172.21.15.46:38068) 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 -- 172.21.15.46:0/381856480 learned_addr learned my addr 172.21.15.46:0/381856480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 -- 172.21.15.46:0/381856480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 msgr2=0x55aa79a99930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a99930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 -- 172.21.15.46:0/381856480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a98890 msgr2=0x55aa79aa9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a98890 0x55aa79aa9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 -- 172.21.15.46:0/381856480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa79815b60 con 0x55aa79a9e720 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa421800700 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa414018bf0 tx=0x7fa414004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f772e39d51cd5c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.584+0000 7fa402a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41402a070 con 0x55aa79a9e720 2024-05-08T23:16:28.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa41001e040 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa414004bb0 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa41400e070 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 10 client.29071 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aa7986dcd0 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41001e040 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa41402d020 con 0x55aa79a9e720 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 10 client.29071 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 10 client.29071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 20 client.29071 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79b08790 0x55aa79b0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 4 client.29071 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.585+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa799e0ec0 con 0x55aa79b08790 2024-05-08T23:16:28.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa420e00700 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79b08790 0x55aa79b0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa420e00700 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79b08790 0x55aa79b0abb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3589d37d0110d359 server_cookie=4a8f58d967e84e73 in_seq=0 out_seq=0 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa402a00700 10 client.29071 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa402a00700 1 -- 172.21.15.46:0/381856480 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa799e0ec0 con 0x55aa79b08790 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa402a00700 10 client.29071 handle_command_reply: tid=0 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 client.29071 shutdown 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 msgr2=0x55aa79a99e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa414018bf0 tx=0x7fa414004ad0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 shutdown_connections 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55aa79b08790 0x55aa79b0abb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa79a9e720 0x55aa79a99e70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa79a98890 0x55aa79aa9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 --2- 172.21.15.46:0/381856480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa79a9de00 0x55aa79a99930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa402a00700 0 client.29071 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:16:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.586+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 >> 172.21.15.46:0/381856480 conn(0x55aa799f6210 msgr2=0x55aa79a9eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 shutdown_connections 2024-05-08T23:16:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa4372ecb80 1 -- 172.21.15.46:0/381856480 wait complete. 2024-05-08T23:16:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa4372ecb80 20 client.29071 trim_cache size 0 max 16384 2024-05-08T23:16:28.592 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 msgr2=0x7fa4040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 0x7fa4040a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa414009ec0 tx=0x7fa41400b040 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 msgr2=0x7fa41c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e970 tx=0x7fa40800ec80 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 shutdown_connections 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41c075f30 0x7fa41c1a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa4040a1290 0x7fa4040a3750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa41c1032b0 0x7fa41c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 --2- 172.21.15.46:0/2324028762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c1083d0 0x7fa41c0759f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.587+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 >> 172.21.15.46:0/2324028762 conn(0x7fa41c0feb10 msgr2=0x7fa41c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.588+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 shutdown_connections 2024-05-08T23:16:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.588+0000 7fa423800700 1 -- 172.21.15.46:0/2324028762 wait complete. 2024-05-08T23:16:28.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:16:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 -- 172.21.15.46:0/1058929171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101a30 msgr2=0x7fc08c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 --2- 172.21.15.46:0/1058929171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101a30 0x7fc08c105a70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098f0 tx=0x7fc07c009c00 comp rx=0 tx=0).stop 2024-05-08T23:16:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 -- 172.21.15.46:0/1058929171 shutdown_connections 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 --2- 172.21.15.46:0/1058929171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 --2- 172.21.15.46:0/1058929171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c101080 0x7fc08c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 --2- 172.21.15.46:0/1058929171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101a30 0x7fc08c105a70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.824+0000 7fc093c00700 1 -- 172.21.15.46:0/1058929171 >> 172.21.15.46:0/1058929171 conn(0x7fc08c0fc8e0 msgr2=0x7fc08c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.825+0000 7fc093c00700 1 -- 172.21.15.46:0/1058929171 shutdown_connections 2024-05-08T23:16:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.825+0000 7fc093c00700 1 -- 172.21.15.46:0/1058929171 wait complete. 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 Processor -- start 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 -- start start 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c073d20 0x7fc08c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1a5b50 con 0x7fc08c073d20 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1a5cc0 con 0x7fc08c101080 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc093c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1a5e80 con 0x7fc08c1061a0 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc091200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.826+0000 7fc08be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc091200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51676/0 (socket says 172.21.15.46:51676) 2024-05-08T23:16:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc091200700 1 -- 172.21.15.46:0/217099020 learned_addr learned my addr 172.21.15.46:0/217099020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38078/0 (socket says 172.21.15.46:38078) 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc091c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c073d20 0x7fc08c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 -- 172.21.15.46:0/217099020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 msgr2=0x7fc08c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 -- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c073d20 msgr2=0x7fc08c1a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c073d20 0x7fc08c1a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 -- 172.21.15.46:0/217099020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009520 con 0x7fc08c1061a0 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc091200700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.827+0000 7fc08be00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000c00 tx=0x7fc07c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.829+0000 7fc089600700 1 -- 172.21.15.46:0/217099020 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c01c070 con 0x7fc08c1061a0 2024-05-08T23:16:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.829+0000 7fc089600700 1 -- 172.21.15.46:0/217099020 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c00bb20 con 0x7fc08c1061a0 2024-05-08T23:16:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.829+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c197090 con 0x7fc08c1061a0 2024-05-08T23:16:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.829+0000 7fc089600700 1 -- 172.21.15.46:0/217099020 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c005720 con 0x7fc08c1061a0 2024-05-08T23:16:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.829+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1975c0 con 0x7fc08c1061a0 2024-05-08T23:16:28.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.833+0000 7fc089600700 1 -- 172.21.15.46:0/217099020 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc07c020030 con 0x7fc08c1061a0 2024-05-08T23:16:28.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.835+0000 7fc089600700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 0x7fc0700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.835+0000 7fc089600700 1 -- 172.21.15.46:0/217099020 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc07c0cdd80 con 0x7fc08c1061a0 2024-05-08T23:16:28.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.835+0000 7fc0a76ffb80 1 Processor -- start 2024-05-08T23:16:28.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.835+0000 7fc091200700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 0x7fc0700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.835+0000 7fc0a76ffb80 1 -- start start 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1dafbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1da52150 con 0x562c1db01720 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1da4fb10 con 0x562c1dafb890 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1da4b400 con 0x562c1db00e00 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 0x7fc0700a37d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc084009c10 tx=0x7fc08400b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38092/0 (socket says 172.21.15.46:38092) 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 -- 172.21.15.46:0/256539245 learned_addr learned my addr 172.21.15.46:0/256539245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091c00700 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1dafbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc08be00700 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 -- 172.21.15.46:0/256539245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 msgr2=0x562c1dafbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1dafbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 -- 172.21.15.46:0/256539245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 msgr2=0x562c1dafb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.836+0000 7fc091200700 1 -- 172.21.15.46:0/256539245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1d8cc3b0 con 0x562c1db00e00 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.837+0000 7fc091200700 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e810 tx=0x7fc084014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355501e1ea89eb55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.837+0000 7fc082000700 1 -- 172.21.15.46:0/256539245 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084015860 con 0x562c1db00e00 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.837+0000 7fc082000700 1 -- 172.21.15.46:0/256539245 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08401b070 con 0x562c1db00e00 2024-05-08T23:16:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.837+0000 7fc082000700 1 -- 172.21.15.46:0/256539245 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084022d40 con 0x562c1db00e00 2024-05-08T23:16:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/256539245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 msgr2=0x562c1db011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e810 tx=0x7fc084014000 comp rx=0 tx=0).stop 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/256539245 shutdown_connections 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1dafbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db011e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/256539245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/256539245 >> 172.21.15.46:0/256539245 conn(0x562c1da59210 msgr2=0x562c1daf5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/256539245 shutdown_connections 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.838+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/256539245 wait complete. 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 -- start start 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 Processor -- start 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1db04a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1db054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1d9e7560 con 0x562c1db01720 2024-05-08T23:16:28.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1d9e7200 con 0x562c1dafb890 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.839+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1d8ce6a0 con 0x562c1db00e00 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc0a76ffb80 10 client.0 _ll_register_callbacks cb 0x562c1d9f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38104/0 (socket says 172.21.15.46:38104) 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 -- 172.21.15.46:0/634411896 learned_addr learned my addr 172.21.15.46:0/634411896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc091200700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1db04a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc091c00700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1db054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 -- 172.21.15.46:0/634411896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 msgr2=0x562c1db04a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1db04a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 -- 172.21.15.46:0/634411896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 msgr2=0x562c1db054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1db054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 -- 172.21.15.46:0/634411896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08400e4f0 con 0x562c1db00e00 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc091c00700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1db054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:28.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc08be00700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009810 tx=0x7fc07c09a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723276b5d5572ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc07aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.840+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c09ab10 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc07c09e940 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c1d9a4270 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c0a2f00 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 10 client.29083 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c1d878b60 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c1d9a4270 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc07c09ae90 con 0x562c1db00e00 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 10 client.29083 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 10 client.29083 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 20 client.29083 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:28.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db14e10 0x562c1db17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 4 client.29083 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.841+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c1daca810 con 0x562c1db14e10 2024-05-08T23:16:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.842+0000 7fc091200700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db14e10 0x562c1db17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.842+0000 7fc07aa00700 10 client.29083 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:16:28.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.842+0000 7fc091200700 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db14e10 0x562c1db17230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9043ae0255cb15e server_cookie=a3426bb0ae51272c in_seq=0 out_seq=0 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc07aa00700 1 -- 172.21.15.46:0/634411896 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c1daca810 con 0x562c1db14e10 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc07aa00700 10 client.29083 handle_command_reply: tid=0 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 client.29083 shutdown 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 msgr2=0x562c1db04f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009810 tx=0x7fc07c09a470 comp rx=0 tx=0).stop 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 shutdown_connections 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db14e10 0x562c1db17230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1db054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1dafb890 0x562c1db04a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/634411896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db00e00 0x562c1db04f90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc07aa00700 0 client.29083 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:16:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 >> 172.21.15.46:0/634411896 conn(0x562c1da59210 msgr2=0x562c1daf55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 shutdown_connections 2024-05-08T23:16:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/634411896 wait complete. 2024-05-08T23:16:28.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.843+0000 7fc0a76ffb80 20 client.29083 trim_cache size 0 max 16384 2024-05-08T23:16:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 Processor -- start 2024-05-08T23:16:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 -- start start 2024-05-08T23:16:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db17200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1daf7640 con 0x562c1db01720 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1db1c440 con 0x562c1db00e00 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.858+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1daf0800 con 0x562c1dafb890 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38112/0 (socket says 172.21.15.46:38112) 2024-05-08T23:16:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 -- 172.21.15.46:0/898574320 learned_addr learned my addr 172.21.15.46:0/898574320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc08be00700 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db17200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091c00700 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 -- 172.21.15.46:0/898574320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 msgr2=0x562c1db17200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db17200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 -- 172.21.15.46:0/898574320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 msgr2=0x562c1dafbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 -- 172.21.15.46:0/898574320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1da11560 con 0x562c1dafb890 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc091200700 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc08400efd0 tx=0x7fc084009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96024b1a891aaa93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.859+0000 7fc082a00700 1 -- 172.21.15.46:0/898574320 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084034d70 con 0x562c1dafb890 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc082a00700 1 -- 172.21.15.46:0/898574320 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08400f040 con 0x562c1dafb890 2024-05-08T23:16:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc082a00700 1 -- 172.21.15.46:0/898574320 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084022e50 con 0x562c1dafb890 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/898574320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 msgr2=0x562c1dafb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc08400efd0 tx=0x7fc084009330 comp rx=0 tx=0).stop 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/898574320 shutdown_connections 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1db01720 0x562c1dafbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1dafb890 0x562c1dafb350 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/898574320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db17200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/898574320 >> 172.21.15.46:0/898574320 conn(0x562c1da59210 msgr2=0x562c1daf2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.860+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/898574320 shutdown_connections 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.861+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/898574320 wait complete. 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.861+0000 7fc0a76ffb80 1 -- start start 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.861+0000 7fc0a76ffb80 1 Processor -- start 2024-05-08T23:16:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.861+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db0a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db01720 0x562c1db0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1db0ad80 con 0x562c1dafb890 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1db0aef0 con 0x562c1db00e00 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1db6a310 con 0x562c1db01720 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc0a76ffb80 10 client.0 _ll_register_callbacks cb 0x562c1d9f0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57888/0 (socket says 172.21.15.46:57888) 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 -- 172.21.15.46:0/2246592074 learned_addr learned my addr 172.21.15.46:0/2246592074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc08be00700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db0a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091c00700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db01720 0x562c1db0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 -- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db01720 msgr2=0x562c1db0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db01720 0x562c1db0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 -- 172.21.15.46:0/2246592074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 msgr2=0x562c1db0a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db0a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.862+0000 7fc091200700 1 -- 172.21.15.46:0/2246592074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c1d8cc3b0 con 0x562c1dafb890 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc091200700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc08400b010 tx=0x7fc0840226a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16968c0c912f39ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc07aa00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084004a10 con 0x562c1dafb890 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc084004b70 con 0x562c1dafb890 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc08401b070 con 0x562c1dafb890 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc084004d30 con 0x562c1dafb890 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc0a76ffb80 10 client.18969 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c1d8d0cd0 con 0x562c1dafb890 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.863+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08401b070 con 0x562c1dafb890 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc084005160 con 0x562c1dafb890 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 10 client.18969 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 10 client.18969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 20 client.18969 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db6b670 0x562c1db6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 4 client.18969 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c1da43ec0 con 0x562c1db6b670 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc08be00700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db6b670 0x562c1db6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc08be00700 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db6b670 0x562c1db6da60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60b54f82bc1a6eaa server_cookie=ad0e282fc4aaaaee in_seq=0 out_seq=0 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.864+0000 7fc07aa00700 10 client.18969 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc07aa00700 1 -- 172.21.15.46:0/2246592074 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c1da43ec0 con 0x562c1db6b670 2024-05-08T23:16:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc07aa00700 10 client.18969 handle_command_reply: tid=0 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 client.18969 shutdown 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 msgr2=0x562c1db0c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc08400b010 tx=0x7fc0840226a0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 shutdown_connections 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562c1db6b670 0x562c1db6da60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c1db01720 0x562c1db0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c1dafb890 0x562c1db0c4c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 --2- 172.21.15.46:0/2246592074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c1db00e00 0x562c1db0a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 >> 172.21.15.46:0/2246592074 conn(0x562c1da59210 msgr2=0x562c1dafbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc07aa00700 0 client.18969 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 shutdown_connections 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 1 -- 172.21.15.46:0/2246592074 wait complete. 2024-05-08T23:16:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.865+0000 7fc0a76ffb80 20 client.18969 trim_cache size 0 max 16384 2024-05-08T23:16:28.872 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:16:28.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 msgr2=0x7fc0700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 0x7fc0700a37d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc084009c10 tx=0x7fc08400b040 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 msgr2=0x7fc08c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000c00 tx=0x7fc07c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 shutdown_connections 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc0700a1310 0x7fc0700a37d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc08c073d20 0x7fc08c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc08c101080 0x7fc08c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 --2- 172.21.15.46:0/217099020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c1061a0 0x7fc08c0737e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 >> 172.21.15.46:0/217099020 conn(0x7fc08c0fc8e0 msgr2=0x7fc08c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 shutdown_connections 2024-05-08T23:16:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:28.866+0000 7fc093c00700 1 -- 172.21.15.46:0/217099020 wait complete. 2024-05-08T23:16:28.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:28 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2155223330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:16:28.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:16:28.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1105068 9.16 MB/sec execute 530 sec latency 18.117 ms 2024-05-08T23:16:29.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.101+0000 7f328dc00700 1 -- 172.21.15.46:0/1505938644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288106d30 msgr2=0x7f3288107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.101+0000 7f328dc00700 1 --2- 172.21.15.46:0/1505938644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288106d30 0x7f3288107110 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3278009990 tx=0x7f3278009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.101+0000 7f328dc00700 1 -- 172.21.15.46:0/1505938644 shutdown_connections 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 --2- 172.21.15.46:0/1505938644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288068b20 0x7f328810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 --2- 172.21.15.46:0/1505938644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288106d30 0x7f3288107110 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 --2- 172.21.15.46:0/1505938644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068160 0x7f32880685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 -- 172.21.15.46:0/1505938644 >> 172.21.15.46:0/1505938644 conn(0x7f3288075710 msgr2=0x7f3288075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 -- 172.21.15.46:0/1505938644 shutdown_connections 2024-05-08T23:16:29.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.102+0000 7f328dc00700 1 -- 172.21.15.46:0/1505938644 wait complete. 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 Processor -- start 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 -- start start 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068b20 0x7f32881003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288106d30 0x7f32881008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a5330 con 0x7f3288068b20 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a54a0 con 0x7f3288106d30 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f328dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881a5660 con 0x7f3288068160 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.103+0000 7f3287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38132/0 (socket says 172.21.15.46:38132) 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 -- 172.21.15.46:0/3853360484 learned_addr learned my addr 172.21.15.46:0/3853360484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287400700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068b20 0x7f32881003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f328d200700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288106d30 0x7f32881008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 -- 172.21.15.46:0/3853360484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288106d30 msgr2=0x7f32881008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288106d30 0x7f32881008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 -- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068b20 msgr2=0x7f32881003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068b20 0x7f32881003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 -- 172.21.15.46:0/3853360484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780095f0 con 0x7f3288068160 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.104+0000 7f3287e00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3278000c00 tx=0x7f3278004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.105+0000 7f3284c00700 1 -- 172.21.15.46:0/3853360484 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327801d070 con 0x7f3288068160 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.105+0000 7f3284c00700 1 -- 172.21.15.46:0/3853360484 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3278022470 con 0x7f3288068160 2024-05-08T23:16:29.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.105+0000 7f3284c00700 1 -- 172.21.15.46:0/3853360484 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278017650 con 0x7f3288068160 2024-05-08T23:16:29.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.105+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32881a58e0 con 0x7f3288068160 2024-05-08T23:16:29.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.105+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32881a5d00 con 0x7f3288068160 2024-05-08T23:16:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.109+0000 7f3284c00700 1 -- 172.21.15.46:0/3853360484 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f32780225e0 con 0x7f3288068160 2024-05-08T23:16:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.110+0000 7f32a2d17b80 1 Processor -- start 2024-05-08T23:16:29.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.110+0000 7f32a2d17b80 1 -- start start 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c6b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515bbc150 con 0x55f515c6b970 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515bb9b10 con 0x55f515c6b050 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515bb5400 con 0x55f515c65890 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f328d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c6b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f3284c00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 0x7f326c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f3287e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f328d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c6b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51736/0 (socket says 172.21.15.46:51736) 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f328d200700 1 -- 172.21.15.46:0/4165948617 learned_addr learned my addr 172.21.15.46:0/4165948617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f3287400700 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.111+0000 7f3284c00700 1 -- 172.21.15.46:0/3853360484 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f32780ce3e0 con 0x7f3288068160 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 -- 172.21.15.46:0/4165948617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 msgr2=0x55f515c6b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c6b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 -- 172.21.15.46:0/4165948617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 msgr2=0x55f515c65350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f328d200700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 0x7f326c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c65350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 -- 172.21.15.46:0/4165948617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f515a363b0 con 0x55f515c65890 2024-05-08T23:16:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f3287400700 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3274009c80 tx=0x7f3274009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc93147bc63eb38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f327d600700 1 -- 172.21.15.46:0/4165948617 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274011bf0 con 0x55f515c65890 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f327d600700 1 -- 172.21.15.46:0/4165948617 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32740044f0 con 0x55f515c65890 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.112+0000 7f327d600700 1 -- 172.21.15.46:0/4165948617 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274005280 con 0x55f515c65890 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 -- 172.21.15.46:0/4165948617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 msgr2=0x55f515c65cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3274009c80 tx=0x7f3274009f90 comp rx=0 tx=0).stop 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f328d200700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 0x7f326c0a3870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3280009510 tx=0x7f3280004b40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 -- 172.21.15.46:0/4165948617 shutdown_connections 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65cd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/4165948617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 -- 172.21.15.46:0/4165948617 >> 172.21.15.46:0/4165948617 conn(0x55f515bc3210 msgr2=0x55f515c602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 -- 172.21.15.46:0/4165948617 shutdown_connections 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.113+0000 7f32a2d17b80 1 -- 172.21.15.46:0/4165948617 wait complete. 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.114+0000 7f32a2d17b80 1 -- start start 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.114+0000 7f32a2d17b80 1 Processor -- start 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c65890 0x55f515c71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c6f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515b51560 con 0x55f515c6b970 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515b51200 con 0x55f515c65890 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515a386a0 con 0x55f515c6b050 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f32a2d17b80 10 client.0 _ll_register_callbacks cb 0x55f515b5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f3287400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f328d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c6f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f3287400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38158/0 (socket says 172.21.15.46:38158) 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f3287400700 1 -- 172.21.15.46:0/3713905712 learned_addr learned my addr 172.21.15.46:0/3713905712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.115+0000 7f3287e00700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c65890 0x55f515c71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 -- 172.21.15.46:0/3713905712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c65890 msgr2=0x55f515c71880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c65890 0x55f515c71880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 -- 172.21.15.46:0/3713905712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 msgr2=0x55f515c6f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c6f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 -- 172.21.15.46:0/3713905712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32740098e0 con 0x55f515c6b050 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f328d200700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3287400700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3274004a60 tx=0x7f3274011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29bdb1ae22eeb34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3272000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274005110 con 0x55f515c6b050 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f515b0e270 con 0x55f515c6b050 2024-05-08T23:16:29.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3274005270 con 0x55f515c6b050 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3274005430 con 0x55f515c6b050 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f32a2d17b80 10 client.29101 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.116+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5159e2b60 con 0x55f515c6b050 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f515b0e270 con 0x55f515c6b050 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32740193f0 con 0x55f515c6b050 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 10 client.29101 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 10 client.29101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 20 client.29101 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515c7f200 0x55f515c81620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 4 client.29101 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f515c34810 con 0x55f515c7f200 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.117+0000 7f3287e00700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515c7f200 0x55f515c81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.118+0000 7f3287e00700 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515c7f200 0x55f515c81620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24bdb0cfb04ff377 server_cookie=fe121a64b4c3f92 in_seq=0 out_seq=0 2024-05-08T23:16:29.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.118+0000 7f3272000700 10 client.29101 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f3272000700 1 -- 172.21.15.46:0/3713905712 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f515c34810 con 0x55f515c7f200 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f3272000700 10 client.29101 handle_command_reply: tid=0 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 client.29101 shutdown 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 msgr2=0x55f515c6ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3274004a60 tx=0x7f3274011c90 comp rx=0 tx=0).stop 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 shutdown_connections 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515c7f200 0x55f515c81620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b970 0x55f515c6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c65890 0x55f515c71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/3713905712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b050 0x55f515c6ebc0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f3272000700 0 client.29101 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:16:29.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 >> 172.21.15.46:0/3713905712 conn(0x55f515bc3210 msgr2=0x55f515c600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 shutdown_connections 2024-05-08T23:16:29.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.119+0000 7f32a2d17b80 1 -- 172.21.15.46:0/3713905712 wait complete. 2024-05-08T23:16:29.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.120+0000 7f32a2d17b80 20 client.29101 trim_cache size 0 max 16384 2024-05-08T23:16:29.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 Processor -- start 2024-05-08T23:16:29.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 -- start start 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b050 0x55f515c6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b970 0x55f515c65c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c5cdf0 con 0x55f515c6b050 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c867f0 con 0x55f515c6b970 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.136+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c5a800 con 0x55f515c65890 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38172/0 (socket says 172.21.15.46:38172) 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 -- 172.21.15.46:0/1731034477 learned_addr learned my addr 172.21.15.46:0/1731034477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287400700 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b050 0x55f515c6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f328d200700 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b970 0x55f515c65c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 -- 172.21.15.46:0/1731034477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b970 msgr2=0x55f515c65c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b970 0x55f515c65c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 -- 172.21.15.46:0/1731034477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b050 msgr2=0x55f515c6f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b050 0x55f515c6f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 -- 172.21.15.46:0/1731034477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f515b7b560 con 0x55f515c65890 2024-05-08T23:16:29.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.137+0000 7f3287e00700 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f327809b0a0 tx=0x7f32780a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77b2263ec3130e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f327e000700 1 -- 172.21.15.46:0/1731034477 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327809ef40 con 0x55f515c65890 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f327e000700 1 -- 172.21.15.46:0/1731034477 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32780a3500 con 0x55f515c65890 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f327e000700 1 -- 172.21.15.46:0/1731034477 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780228e0 con 0x55f515c65890 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 -- 172.21.15.46:0/1731034477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 msgr2=0x55f515c65350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f327809b0a0 tx=0x7f32780a1040 comp rx=0 tx=0).stop 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 -- 172.21.15.46:0/1731034477 shutdown_connections 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b970 0x55f515c65c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c65890 0x55f515c65350 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/1731034477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c6b050 0x55f515c6f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.138+0000 7f32a2d17b80 1 -- 172.21.15.46:0/1731034477 >> 172.21.15.46:0/1731034477 conn(0x55f515bc3210 msgr2=0x55f515c5ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.139+0000 7f32a2d17b80 1 -- 172.21.15.46:0/1731034477 shutdown_connections 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.139+0000 7f32a2d17b80 1 -- 172.21.15.46:0/1731034477 wait complete. 2024-05-08T23:16:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.139+0000 7f32a2d17b80 1 -- start start 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.139+0000 7f32a2d17b80 1 Processor -- start 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c65890 0x55f515c76330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b970 0x55f515c66a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c67050 con 0x55f515c65890 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c671c0 con 0x55f515c6b050 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f515c67380 con 0x55f515c6b970 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f32a2d17b80 10 client.0 _ll_register_callbacks cb 0x55f515b5ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f3287400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f3287400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51756/0 (socket says 172.21.15.46:51756) 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f3287400700 1 -- 172.21.15.46:0/397285813 learned_addr learned my addr 172.21.15.46:0/397285813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f328d200700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b970 0x55f515c66a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.140+0000 7f3287e00700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c65890 0x55f515c76330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 -- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b970 msgr2=0x55f515c66a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b970 0x55f515c66a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 -- 172.21.15.46:0/397285813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c65890 msgr2=0x55f515c76330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c65890 0x55f515c76330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 -- 172.21.15.46:0/397285813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32881a5d00 con 0x55f515c6b050 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3287400700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3274004a90 tx=0x7f3274019620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc61727b91df2d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3272000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274005320 con 0x55f515c6b050 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3274004b30 con 0x55f515c6b050 2024-05-08T23:16:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.141+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32740044f0 con 0x55f515c6b050 2024-05-08T23:16:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3274019a70 con 0x55f515c6b050 2024-05-08T23:16:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 10 client.29012 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f515a3acd0 con 0x55f515c6b050 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32740044f0 con 0x55f515c6b050 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3274011440 con 0x55f515c6b050 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 10 client.29012 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 10 client.29012 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 20 client.29012 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515cd56c0 0x55f515cd7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 4 client.29012 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.142+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f515badec0 con 0x55f515cd56c0 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3287e00700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515cd56c0 0x55f515cd7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3287e00700 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515cd56c0 0x55f515cd7ab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1800b75b4e32350 server_cookie=32d25d1a56743097 in_seq=0 out_seq=0 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3272000700 10 client.29012 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3272000700 1 -- 172.21.15.46:0/397285813 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f515badec0 con 0x55f515cd56c0 2024-05-08T23:16:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3272000700 10 client.29012 handle_command_reply: tid=0 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 client.29012 shutdown 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 msgr2=0x55f515c76870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3274004a90 tx=0x7f3274019620 comp rx=0 tx=0).stop 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 shutdown_connections 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f515c6b970 0x55f515c66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55f515cd56c0 0x55f515cd7ab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f515c65890 0x55f515c76330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f32a2d17b80 1 --2- 172.21.15.46:0/397285813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f515c6b050 0x55f515c76870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.143+0000 7f3272000700 0 client.29012 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.144+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 >> 172.21.15.46:0/397285813 conn(0x55f515bc3210 msgr2=0x55f515c6d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.144+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 shutdown_connections 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.144+0000 7f32a2d17b80 1 -- 172.21.15.46:0/397285813 wait complete. 2024-05-08T23:16:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.144+0000 7f32a2d17b80 20 client.29012 trim_cache size 0 max 16384 2024-05-08T23:16:29.149 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:16:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 msgr2=0x7f326c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 0x7f326c0a3870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3280009510 tx=0x7f3280004b40 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 msgr2=0x7f32880ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3278000c00 tx=0x7f3278004730 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 shutdown_connections 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3288106d30 0x7f32881008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3288068160 0x7f32880ffe60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f326c0a13b0 0x7f326c0a3870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 --2- 172.21.15.46:0/3853360484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3288068b20 0x7f32881003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 >> 172.21.15.46:0/3853360484 conn(0x7f3288075710 msgr2=0x7f32880fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 shutdown_connections 2024-05-08T23:16:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.145+0000 7f328dc00700 1 -- 172.21.15.46:0/3853360484 wait complete. 2024-05-08T23:16:29.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:16:29.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:28 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2155223330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:16:29.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:28 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2155223330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:16:29.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 -- 172.21.15.46:0/879754496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 msgr2=0x7ff538103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 --2- 172.21.15.46:0/879754496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538103690 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff528009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 -- 172.21.15.46:0/879754496 shutdown_connections 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 --2- 172.21.15.46:0/879754496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5381083d0 0x7ff53810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 --2- 172.21.15.46:0/879754496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538103690 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 --2- 172.21.15.46:0/879754496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff538103c60 0x7ff538107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 -- 172.21.15.46:0/879754496 >> 172.21.15.46:0/879754496 conn(0x7ff5380feb10 msgr2=0x7ff538100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 -- 172.21.15.46:0/879754496 shutdown_connections 2024-05-08T23:16:29.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.457+0000 7ff53e200700 1 -- 172.21.15.46:0/879754496 wait complete. 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 Processor -- start 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 -- start start 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5381083d0 0x7ff538199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff53819de40 0x7ff53819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819e8c0 con 0x7ff5381083d0 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819ea30 con 0x7ff53819de40 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.458+0000 7ff53e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819ebf0 con 0x7ff5381032b0 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38198/0 (socket says 172.21.15.46:38198) 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff536a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5381083d0 0x7ff538199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 -- 172.21.15.46:0/2659125618 learned_addr learned my addr 172.21.15.46:0/2659125618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537e00700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff53819de40 0x7ff53819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 -- 172.21.15.46:0/2659125618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff53819de40 msgr2=0x7ff53819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff53819de40 0x7ff53819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 -- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5381083d0 msgr2=0x7ff538199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5381083d0 0x7ff538199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 -- 172.21.15.46:0/2659125618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280095f0 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff537400700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff528005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.459+0000 7ff52fe00700 1 -- 172.21.15.46:0/2659125618 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52801d070 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.460+0000 7ff52fe00700 1 -- 172.21.15.46:0/2659125618 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff528022470 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.460+0000 7ff52fe00700 1 -- 172.21.15.46:0/2659125618 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528017650 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.460+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5381a60f0 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.460+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381a64c0 con 0x7ff5381032b0 2024-05-08T23:16:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.463+0000 7ff52fe00700 1 -- 172.21.15.46:0/2659125618 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff5280225e0 con 0x7ff5381032b0 2024-05-08T23:16:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.464+0000 7ff52fe00700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 0x7ff51c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.464+0000 7ff551d80b80 1 Processor -- start 2024-05-08T23:16:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.464+0000 7ff52fe00700 1 -- 172.21.15.46:0/2659125618 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff5280ce270 con 0x7ff5381032b0 2024-05-08T23:16:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.464+0000 7ff536a00700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 0x7ff51c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 -- start start 2024-05-08T23:16:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c88cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64bdf150 con 0x55df64c8e720 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64bdcb10 con 0x55df64c8de00 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff551d80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64bd8400 con 0x55df64c88890 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff536a00700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 0x7ff51c0a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff53819aab0 tx=0x7ff524009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.465+0000 7ff537400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51782/0 (socket says 172.21.15.46:51782) 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff536a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 -- 172.21.15.46:0/4217293218 learned_addr learned my addr 172.21.15.46:0/4217293218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537e00700 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c88cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 -- 172.21.15.46:0/4217293218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 msgr2=0x55df64c88cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c88cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 -- 172.21.15.46:0/4217293218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 msgr2=0x55df64c88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 -- 172.21.15.46:0/4217293218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df64a593b0 con 0x55df64c8de00 2024-05-08T23:16:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537e00700 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c88cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff537400700 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff5280a2e60 tx=0x7ff528009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22236215f993c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.466+0000 7ff523e00700 1 -- 172.21.15.46:0/4217293218 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280a10b0 con 0x55df64c8de00 2024-05-08T23:16:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff523e00700 1 -- 172.21.15.46:0/4217293218 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52809b6a0 con 0x55df64c8de00 2024-05-08T23:16:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff523e00700 1 -- 172.21.15.46:0/4217293218 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52809b220 con 0x55df64c8de00 2024-05-08T23:16:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 -- 172.21.15.46:0/4217293218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 msgr2=0x55df64c8e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff5280a2e60 tx=0x7ff528009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 -- 172.21.15.46:0/4217293218 shutdown_connections 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c88cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c8e1e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4217293218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.467+0000 7ff551d80b80 1 -- 172.21.15.46:0/4217293218 >> 172.21.15.46:0/4217293218 conn(0x55df64be6210 msgr2=0x55df64c82df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.468+0000 7ff551d80b80 1 -- 172.21.15.46:0/4217293218 shutdown_connections 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.468+0000 7ff551d80b80 1 -- 172.21.15.46:0/4217293218 wait complete. 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.468+0000 7ff551d80b80 1 -- start start 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 Processor -- start 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8e720 0x55df64c92210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64b74560 con 0x55df64c88890 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64b74200 con 0x55df64c8e720 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64a5b6a0 con 0x55df64c8de00 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.469+0000 7ff551d80b80 10 client.0 _ll_register_callbacks cb 0x55df64b7db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff536a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff536a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38222/0 (socket says 172.21.15.46:38222) 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff536a00700 1 -- 172.21.15.46:0/1541314033 learned_addr learned my addr 172.21.15.46:0/1541314033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537e00700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8e720 0x55df64c92210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 -- 172.21.15.46:0/1541314033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 msgr2=0x55df64c91ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 -- 172.21.15.46:0/1541314033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8e720 msgr2=0x55df64c92210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8e720 0x55df64c92210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 -- 172.21.15.46:0/1541314033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff530009050 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff536a00700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff537400700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c947e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff52809aca0 tx=0x7ff52809aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4133f97871afce2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.470+0000 7ff520c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280a3390 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff52809afa0 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df64b31270 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5280a10b0 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 10 client.18999 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff5381a64c0 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df64b31270 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff52809b200 con 0x55df64c88890 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 10 client.18999 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 10 client.18999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 20 client.18999 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64ca19a0 0x55df64ca3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 4 client.18999 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.471+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df64c57810 con 0x55df64ca19a0 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.472+0000 7ff536a00700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64ca19a0 0x55df64ca3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.472+0000 7ff536a00700 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64ca19a0 0x55df64ca3d90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44e7ac3ff81dd7c7 server_cookie=546fb1559009606c in_seq=0 out_seq=0 2024-05-08T23:16:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.472+0000 7ff520c00700 10 client.18999 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:16:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff520c00700 1 -- 172.21.15.46:0/1541314033 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df64c57810 con 0x55df64ca19a0 2024-05-08T23:16:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff520c00700 10 client.18999 handle_command_reply: tid=0 2024-05-08T23:16:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 client.18999 shutdown 2024-05-08T23:16:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 msgr2=0x55df64c947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c947e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff52809aca0 tx=0x7ff52809aa30 comp rx=0 tx=0).stop 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 shutdown_connections 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8e720 0x55df64c92210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64ca19a0 0x55df64ca3d90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c947e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 --2- 172.21.15.46:0/1541314033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8de00 0x55df64c91ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 >> 172.21.15.46:0/1541314033 conn(0x55df64be6210 msgr2=0x55df64c825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.473+0000 7ff520c00700 0 client.18999 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.474+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 shutdown_connections 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.474+0000 7ff551d80b80 1 -- 172.21.15.46:0/1541314033 wait complete. 2024-05-08T23:16:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.474+0000 7ff551d80b80 20 client.18999 trim_cache size 0 max 16384 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.489+0000 7ff551d80b80 1 Processor -- start 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.489+0000 7ff551d80b80 1 -- start start 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c82d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64c92a50 con 0x55df64c88890 2024-05-08T23:16:29.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64c838e0 con 0x55df64c8de00 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff551d80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64c91e30 con 0x55df64c8e720 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38230/0 (socket says 172.21.15.46:38230) 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 -- 172.21.15.46:0/519422844 learned_addr learned my addr 172.21.15.46:0/519422844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff536a00700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537400700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c82d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 -- 172.21.15.46:0/519422844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 msgr2=0x55df64c913b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c913b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 -- 172.21.15.46:0/519422844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 msgr2=0x55df64c82d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.490+0000 7ff537e00700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c82d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff537e00700 1 -- 172.21.15.46:0/519422844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df64c88f40 con 0x55df64c8e720 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff536a00700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:29.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff537e00700 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff5300077d0 tx=0x7ff530009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ac212556b9374e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff52d200700 1 -- 172.21.15.46:0/519422844 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000e040 con 0x55df64c8e720 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff52d200700 1 -- 172.21.15.46:0/519422844 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53000f040 con 0x55df64c8e720 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.491+0000 7ff52d200700 1 -- 172.21.15.46:0/519422844 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53001fc60 con 0x55df64c8e720 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- 172.21.15.46:0/519422844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 msgr2=0x55df64c918f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff5300077d0 tx=0x7ff530009920 comp rx=0 tx=0).stop 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- 172.21.15.46:0/519422844 shutdown_connections 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c8e720 0x55df64c918f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c88890 0x55df64c82d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 --2- 172.21.15.46:0/519422844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- 172.21.15.46:0/519422844 >> 172.21.15.46:0/519422844 conn(0x55df64be6210 msgr2=0x55df64ca39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- 172.21.15.46:0/519422844 shutdown_connections 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- 172.21.15.46:0/519422844 wait complete. 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.492+0000 7ff551d80b80 1 -- start start 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 Processor -- start 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c97300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c97840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64cf6d90 con 0x55df64c8e720 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64cf6ed0 con 0x55df64c8de00 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df64cf7040 con 0x55df64c88890 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff551d80b80 10 client.0 _ll_register_callbacks cb 0x55df64b7db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff537400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38240/0 (socket says 172.21.15.46:38240) 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff537400700 1 -- 172.21.15.46:0/4047727167 learned_addr learned my addr 172.21.15.46:0/4047727167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff536a00700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c97300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.493+0000 7ff537e00700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c97840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 -- 172.21.15.46:0/4047727167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 msgr2=0x55df64c97300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c97300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 -- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 msgr2=0x55df64c97840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c97840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 -- 172.21.15.46:0/4047727167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff524017070 con 0x55df64c88890 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537e00700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff537400700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff528022890 tx=0x7ff528009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52807d489941abae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff520c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280a3390 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff52809afa0 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52809b6a0 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5280a10b0 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff551d80b80 10 client.29125 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff530007500 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.494+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52809b6a0 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5280a2f50 con 0x55df64c88890 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 10 client.29125 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 10 client.29125 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 20 client.29125 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64cf83b0 0x55df64cfa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 4 client.29125 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:16:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df64bd0ec0 con 0x55df64cf83b0 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff536a00700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64cf83b0 0x55df64cfa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff536a00700 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64cf83b0 0x55df64cfa7a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81ee51138262baf5 server_cookie=732d8bdf19bf49d9 in_seq=0 out_seq=0 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.495+0000 7ff520c00700 10 client.29125 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff520c00700 1 -- 172.21.15.46:0/4047727167 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df64bd0ec0 con 0x55df64cf83b0 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff520c00700 10 client.29125 handle_command_reply: tid=0 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 client.29125 shutdown 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 msgr2=0x55df64c96dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff528022890 tx=0x7ff528009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 shutdown_connections 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55df64cf83b0 0x55df64cfa7a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df64c8e720 0x55df64c97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df64c88890 0x55df64c96dc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 --2- 172.21.15.46:0/4047727167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55df64c8de00 0x55df64c97300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff520c00700 0 client.29125 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 >> 172.21.15.46:0/4047727167 conn(0x55df64be6210 msgr2=0x55df64c8cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 shutdown_connections 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 1 -- 172.21.15.46:0/4047727167 wait complete. 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.496+0000 7ff551d80b80 20 client.29125 trim_cache size 0 max 16384 2024-05-08T23:16:29.502 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 msgr2=0x7ff51c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 0x7ff51c0a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff53819aab0 tx=0x7ff524009260 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 msgr2=0x7ff538199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff528000c00 tx=0x7ff528005740 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 shutdown_connections 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff53819de40 0x7ff53819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5381032b0 0x7ff538199560 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff51c0a1310 0x7ff51c0a37d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 --2- 172.21.15.46:0/2659125618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5381083d0 0x7ff538199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 >> 172.21.15.46:0/2659125618 conn(0x7ff5380feb10 msgr2=0x7ff5380fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 shutdown_connections 2024-05-08T23:16:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.497+0000 7ff53e200700 1 -- 172.21.15.46:0/2659125618 wait complete. 2024-05-08T23:16:29.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:16:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 -- 172.21.15.46:0/531884611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98100ec0 msgr2=0x7efe981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 --2- 172.21.15.46:0/531884611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98100ec0 0x7efe981012a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009990 tx=0x7efe8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 -- 172.21.15.46:0/531884611 shutdown_connections 2024-05-08T23:16:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 --2- 172.21.15.46:0/531884611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 0x7efe9810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 --2- 172.21.15.46:0/531884611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98100ec0 0x7efe981012a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 --2- 172.21.15.46:0/531884611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98101870 0x7efe981058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 -- 172.21.15.46:0/531884611 >> 172.21.15.46:0/531884611 conn(0x7efe980fc740 msgr2=0x7efe980feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 -- 172.21.15.46:0/531884611 shutdown_connections 2024-05-08T23:16:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.744+0000 7efe9ea00700 1 -- 172.21.15.46:0/531884611 wait complete. 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.745+0000 7efe9ea00700 1 Processor -- start 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.745+0000 7efe9ea00700 1 -- start start 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 0x7efe98197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe9819bca0 0x7efe9819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9819c720 con 0x7efe98100ec0 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9819c890 con 0x7efe98105fe0 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9819ca50 con 0x7efe9819bca0 2024-05-08T23:16:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe9819bca0 0x7efe9819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58032/0 (socket says 172.21.15.46:58032) 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9d600700 1 -- 172.21.15.46:0/3288130514 learned_addr learned my addr 172.21.15.46:0/3288130514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.746+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 0x7efe98197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 -- 172.21.15.46:0/3288130514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe9819bca0 msgr2=0x7efe9819c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe9819bca0 0x7efe9819c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 -- 172.21.15.46:0/3288130514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 msgr2=0x7efe98197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 0x7efe98197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 -- 172.21.15.46:0/3288130514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009520 con 0x7efe98100ec0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9d600700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe96000700 1 -- 172.21.15.46:0/3288130514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c01d070 con 0x7efe98100ec0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe96000700 1 -- 172.21.15.46:0/3288130514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe8c022470 con 0x7efe98100ec0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c0095f0 con 0x7efe98100ec0 2024-05-08T23:16:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe96000700 1 -- 172.21.15.46:0/3288130514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c017650 con 0x7efe98100ec0 2024-05-08T23:16:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.747+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe981a41e0 con 0x7efe98100ec0 2024-05-08T23:16:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.752+0000 7efe96000700 1 -- 172.21.15.46:0/3288130514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efe8c0225e0 con 0x7efe98100ec0 2024-05-08T23:16:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.753+0000 7efeb3b58b80 1 Processor -- start 2024-05-08T23:16:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.753+0000 7efeb3b58b80 1 -- start start 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe96000700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 0x7efe800a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4251430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc424bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc41a2150 con 0x564dc424b890 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc419fb10 con 0x564dc4251050 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc419b400 con 0x564dc4251970 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe96000700 1 -- 172.21.15.46:0/3288130514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe8c0ce3e0 con 0x7efe98100ec0 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38264/0 (socket says 172.21.15.46:38264) 2024-05-08T23:16:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9cc00700 1 -- 172.21.15.46:0/1748577107 learned_addr learned my addr 172.21.15.46:0/1748577107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9e000700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 0x7efe800a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9d600700 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4251430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.754+0000 7efe9e000700 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc424bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 -- 172.21.15.46:0/1748577107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 msgr2=0x564dc4251430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4251430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 -- 172.21.15.46:0/1748577107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 msgr2=0x564dc424bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc424bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 -- 172.21.15.46:0/1748577107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc401c3b0 con 0x564dc4251970 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e80 tx=0x7efe84011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f854def136be0c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe9e000700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 0x7efe800a3870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe90009550 tx=0x7efe9000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe8a600700 1 -- 172.21.15.46:0/1748577107 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84004250 con 0x564dc4251970 2024-05-08T23:16:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe8a600700 1 -- 172.21.15.46:0/1748577107 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe840043b0 con 0x564dc4251970 2024-05-08T23:16:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.755+0000 7efe8a600700 1 -- 172.21.15.46:0/1748577107 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84019920 con 0x564dc4251970 2024-05-08T23:16:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1748577107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 msgr2=0x564dc424b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e80 tx=0x7efe84011720 comp rx=0 tx=0).stop 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1748577107 shutdown_connections 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc424bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4251430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1748577107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc424b350 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1748577107 >> 172.21.15.46:0/1748577107 conn(0x564dc41a9210 msgr2=0x564dc42462d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1748577107 shutdown_connections 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.756+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1748577107 wait complete. 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 -- start start 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 Processor -- start 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc4254ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4255220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4137560 con 0x564dc424b890 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4137200 con 0x564dc4251050 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc401e6a0 con 0x564dc4251970 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.757+0000 7efeb3b58b80 10 client.0 _ll_register_callbacks cb 0x564dc4140b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4255220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc4254ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38266/0 (socket says 172.21.15.46:38266) 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4255220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51848/0 (socket says 172.21.15.46:51848) 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 -- 172.21.15.46:0/2112080798 learned_addr learned my addr 172.21.15.46:0/2112080798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 -- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 msgr2=0x564dc4255220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4255220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 -- 172.21.15.46:0/2112080798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 msgr2=0x564dc4254ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc4254ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 -- 172.21.15.46:0/2112080798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe84009ab0 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe9e000700 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efe9001ab70 tx=0x7efe90004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1e14ce915d3d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.758+0000 7efe7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9001f070 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe90015780 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dc40f4270 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9001ae80 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efeb3b58b80 10 client.29137 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564dc3fc8b60 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564dc40f4270 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe90025a20 con 0x564dc4251970 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efeb3b58b80 10 client.29137 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.759+0000 7efeb3b58b80 10 client.29137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:29.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efeb3b58b80 20 client.29137 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42697d0 0x564dc426bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efeb3b58b80 4 client.29137 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dc421a810 con 0x564dc42697d0 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efe9d600700 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42697d0 0x564dc426bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efe9d600700 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42697d0 0x564dc426bbf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c757ad217fea5b server_cookie=5f5b0bff2779f44 in_seq=0 out_seq=0 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.760+0000 7efe7ea00700 10 client.29137 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efe7ea00700 1 -- 172.21.15.46:0/2112080798 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564dc421a810 con 0x564dc42697d0 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efe7ea00700 10 client.29137 handle_command_reply: tid=0 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 client.29137 shutdown 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 msgr2=0x564dc4255760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efe9001ab70 tx=0x7efe90004b40 comp rx=0 tx=0).stop 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 shutdown_connections 2024-05-08T23:16:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc4255760 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42697d0 0x564dc426bbf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc424b890 0x564dc4254ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2112080798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251050 0x564dc4255220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.761+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 >> 172.21.15.46:0/2112080798 conn(0x564dc41a9210 msgr2=0x564dc42460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.762+0000 7efe7ea00700 0 client.29137 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.762+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 shutdown_connections 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.762+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2112080798 wait complete. 2024-05-08T23:16:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.762+0000 7efeb3b58b80 20 client.29137 trim_cache size 0 max 16384 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.777+0000 7efeb3b58b80 1 Processor -- start 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 -- start start 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc424b890 0x564dc424b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc4255580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc42562e0 con 0x564dc4251050 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4246df0 con 0x564dc424b890 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc4242e10 con 0x564dc4251970 2024-05-08T23:16:29.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efe9e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efe9cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc4255580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efe9e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38274/0 (socket says 172.21.15.46:38274) 2024-05-08T23:16:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efe9e000700 1 -- 172.21.15.46:0/2783693683 learned_addr learned my addr 172.21.15.46:0/2783693683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.778+0000 7efe9d600700 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc424b890 0x564dc424b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 -- 172.21.15.46:0/2783693683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc424b890 msgr2=0x564dc424b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc424b890 0x564dc424b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 -- 172.21.15.46:0/2783693683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 msgr2=0x564dc4255580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc4255580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 -- 172.21.15.46:0/2783693683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc42424e0 con 0x564dc4251970 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.779+0000 7efe9e000700 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe90007fd0 tx=0x7efe900075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac78fe92c5414bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efe8b000700 1 -- 172.21.15.46:0/2783693683 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9001f050 con 0x564dc4251970 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efe8b000700 1 -- 172.21.15.46:0/2783693683 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe900258e0 con 0x564dc4251970 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efe8b000700 1 -- 172.21.15.46:0/2783693683 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90018e00 con 0x564dc4251970 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2783693683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 msgr2=0x564dc426b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe90007fd0 tx=0x7efe900075c0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2783693683 shutdown_connections 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc4251970 0x564dc426b7e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc424b890 0x564dc424b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/2783693683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc4255580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.780+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2783693683 >> 172.21.15.46:0/2783693683 conn(0x564dc41a9210 msgr2=0x564dc424bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.781+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2783693683 shutdown_connections 2024-05-08T23:16:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.781+0000 7efeb3b58b80 1 -- 172.21.15.46:0/2783693683 wait complete. 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.781+0000 7efeb3b58b80 1 -- start start 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.781+0000 7efeb3b58b80 1 Processor -- start 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc425ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251970 0x564dc425d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc42bed60 con 0x564dc4251050 2024-05-08T23:16:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc42beed0 con 0x564dc4251970 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc42bf090 con 0x564dc424b890 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efeb3b58b80 10 client.0 _ll_register_callbacks cb 0x564dc4140b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efe9d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efe9d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38276/0 (socket says 172.21.15.46:38276) 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efe9d600700 1 -- 172.21.15.46:0/1983920880 learned_addr learned my addr 172.21.15.46:0/1983920880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc425ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.782+0000 7efe9e000700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251970 0x564dc425d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 -- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251970 msgr2=0x564dc425d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251970 0x564dc425d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 -- 172.21.15.46:0/1983920880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 msgr2=0x564dc425ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc425ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 -- 172.21.15.46:0/1983920880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc3fc8b60 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc425ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe9d600700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe8c097b80 tx=0x7efe8c0a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd992125ca3d6a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c09ef40 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7efe8c09e040 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe840043b0 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8c0a3500 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.783+0000 7efeb3b58b80 10 client.29143 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564dc4161560 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe840043b0 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe8c0b92a0 con 0x564dc424b890 2024-05-08T23:16:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 10 client.29143 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 10 client.29143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 20 client.29143 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42c0540 0x564dc42c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 4 client.29143 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564dc4193ec0 con 0x564dc42c0540 2024-05-08T23:16:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.784+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42c0540 0x564dc42c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.785+0000 7efe7ea00700 10 client.29143 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.785+0000 7efe9cc00700 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42c0540 0x564dc42c2930 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7888aad946f07341 server_cookie=6e207b08745aa32a in_seq=0 out_seq=0 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.785+0000 7efe7ea00700 1 -- 172.21.15.46:0/1983920880 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564dc4193ec0 con 0x564dc42c0540 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.785+0000 7efe7ea00700 10 client.29143 handle_command_reply: tid=0 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.785+0000 7efeb3b58b80 1 client.29143 shutdown 2024-05-08T23:16:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 msgr2=0x564dc425c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe8c097b80 tx=0x7efe8c0a1040 comp rx=0 tx=0).stop 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 shutdown_connections 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc4251970 0x564dc425d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dc424b890 0x564dc425c790 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564dc42c0540 0x564dc42c2930 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 --2- 172.21.15.46:0/1983920880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564dc4251050 0x564dc425ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efe7ea00700 0 client.29143 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 >> 172.21.15.46:0/1983920880 conn(0x564dc41a9210 msgr2=0x564dc4246060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 shutdown_connections 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 1 -- 172.21.15.46:0/1983920880 wait complete. 2024-05-08T23:16:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.786+0000 7efeb3b58b80 20 client.29143 trim_cache size 0 max 16384 2024-05-08T23:16:29.791 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:16:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 msgr2=0x7efe800a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 0x7efe800a3870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe90009550 tx=0x7efe9000b040 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 msgr2=0x7efe981973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c005740 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 shutdown_connections 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe9819bca0 0x7efe9819c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efe800a13b0 0x7efe800a3870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe98100ec0 0x7efe981973c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 --2- 172.21.15.46:0/3288130514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe98105fe0 0x7efe98197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 >> 172.21.15.46:0/3288130514 conn(0x7efe980fc740 msgr2=0x7efe980feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 shutdown_connections 2024-05-08T23:16:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:29.787+0000 7efe9ea00700 1 -- 172.21.15.46:0/3288130514 wait complete. 2024-05-08T23:16:29.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1106470 9.15 MB/sec execute 531 sec latency 25.634 ms 2024-05-08T23:16:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:29 smithi086 ceph-mon[78610]: pgmap v395: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:29 smithi053 ceph-mon[77587]: pgmap v395: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:29 smithi046 ceph-mon[36898]: pgmap v395: 129 pgs: 129 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-05-08T23:16:30.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:16:30.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:16:30.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1108333 9.16 MB/sec execute 532 sec latency 15.572 ms 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 -- 172.21.15.46:0/2691441324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c106180 msgr2=0x7f850c10dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 --2- 172.21.15.46:0/2691441324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c106180 0x7f850c10dd00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8504009f90 tx=0x7f850400b330 comp rx=0 tx=0).stop 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 -- 172.21.15.46:0/2691441324 shutdown_connections 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 --2- 172.21.15.46:0/2691441324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c106180 0x7f850c10dd00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 --2- 172.21.15.46:0/2691441324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c101060 0x7f850c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 --2- 172.21.15.46:0/2691441324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c101a10 0x7f850c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 -- 172.21.15.46:0/2691441324 >> 172.21.15.46:0/2691441324 conn(0x7f850c0fc900 msgr2=0x7f850c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 -- 172.21.15.46:0/2691441324 shutdown_connections 2024-05-08T23:16:31.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.034+0000 7f8512600700 1 -- 172.21.15.46:0/2691441324 wait complete. 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.035+0000 7f8512600700 1 Processor -- start 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.035+0000 7f8512600700 1 -- start start 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c106180 0x7f850c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 0x7f850c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c19e9a0 con 0x7f850c106180 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c19eb10 con 0x7f850c19df20 2024-05-08T23:16:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f8512600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c19ecd0 con 0x7f850c101060 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f850b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f850b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38294/0 (socket says 172.21.15.46:38294) 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f850b400700 1 -- 172.21.15.46:0/4013906161 learned_addr learned my addr 172.21.15.46:0/4013906161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f850aa00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c106180 0x7f850c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.036+0000 7f850be00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 0x7f850c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 -- 172.21.15.46:0/4013906161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 msgr2=0x7f850c19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 0x7f850c19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 -- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c106180 msgr2=0x7f850c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c106180 0x7f850c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 -- 172.21.15.46:0/4013906161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009c40 con 0x7f850c101060 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850be00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 0x7f850c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f850b400700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00e9a0 tx=0x7f84fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f8503e00700 1 -- 172.21.15.46:0/4013906161 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc009790 con 0x7f850c101060 2024-05-08T23:16:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f8503e00700 1 -- 172.21.15.46:0/4013906161 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc00ce80 con 0x7f850c101060 2024-05-08T23:16:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c1a6230 con 0x7f850c101060 2024-05-08T23:16:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f8503e00700 1 -- 172.21.15.46:0/4013906161 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc018680 con 0x7f850c101060 2024-05-08T23:16:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.037+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1a6760 con 0x7f850c101060 2024-05-08T23:16:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.042+0000 7f8503e00700 1 -- 172.21.15.46:0/4013906161 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f84fc010430 con 0x7f850c101060 2024-05-08T23:16:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.043+0000 7f8526185b80 1 Processor -- start 2024-05-08T23:16:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.043+0000 7f8526185b80 1 -- start start 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.043+0000 7f8503e00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 0x7f84f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8503e00700 1 -- 172.21.15.46:0/4013906161 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f84fc014070 con 0x7f850c101060 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850be00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 0x7f84f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258ddc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258ddcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258d331d0 con 0x556258de27a0 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258d30b90 con 0x556258de1e80 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f8526185b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258d2c480 con 0x556258ddc910 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258ddcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258ddcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38296/0 (socket says 172.21.15.46:38296) 2024-05-08T23:16:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850be00700 1 -- 172.21.15.46:0/3412190903 learned_addr learned my addr 172.21.15.46:0/3412190903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.044+0000 7f850aa00700 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258ddc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 -- 172.21.15.46:0/3412190903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 msgr2=0x556258ddcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258ddcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 -- 172.21.15.46:0/3412190903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 msgr2=0x556258ddc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258ddc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 -- 172.21.15.46:0/3412190903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556258bad430 con 0x556258de1e80 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850aa00700 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258ddc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850be00700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 0x7f84f00a37d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8504003c30 tx=0x7f8504003c60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f850b400700 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de2260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca560 tx=0x7f84fc09a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd302168f1227ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f8500c00700 1 -- 172.21.15.46:0/3412190903 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc099fa0 con 0x556258de1e80 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f8500c00700 1 -- 172.21.15.46:0/3412190903 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc09dd50 con 0x556258de1e80 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.045+0000 7f8500c00700 1 -- 172.21.15.46:0/3412190903 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc099670 con 0x556258de1e80 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 -- 172.21.15.46:0/3412190903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 msgr2=0x556258de2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de2260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ca560 tx=0x7f84fc09a620 comp rx=0 tx=0).stop 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 -- 172.21.15.46:0/3412190903 shutdown_connections 2024-05-08T23:16:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258ddcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de2260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 --2- 172.21.15.46:0/3412190903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258ddc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 -- 172.21.15.46:0/3412190903 >> 172.21.15.46:0/3412190903 conn(0x556258d3a290 msgr2=0x556258dd6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 -- 172.21.15.46:0/3412190903 shutdown_connections 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.046+0000 7f8526185b80 1 -- 172.21.15.46:0/3412190903 wait complete. 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.047+0000 7f8526185b80 1 -- start start 2024-05-08T23:16:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.047+0000 7f8526185b80 1 Processor -- start 2024-05-08T23:16:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258de66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258cc85e0 con 0x556258de27a0 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258cc8280 con 0x556258de1e80 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258baf720 con 0x556258ddc910 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f8526185b80 10 client.0 _ll_register_callbacks cb 0x556258cd1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38302/0 (socket says 172.21.15.46:38302) 2024-05-08T23:16:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 -- 172.21.15.46:0/206637646 learned_addr learned my addr 172.21.15.46:0/206637646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850be00700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258de66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 -- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 msgr2=0x556258de61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 -- 172.21.15.46:0/206637646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 msgr2=0x556258de66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258de66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.048+0000 7f850b400700 1 -- 172.21.15.46:0/206637646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1a6760 con 0x556258ddc910 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f850b400700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84fc099c20 tx=0x7f84fc010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb9b766d48e0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0a23a0 con 0x556258ddc910 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f84fc09a390 con 0x556258ddc910 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556258c852f0 con 0x556258ddc910 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc0b26c0 con 0x556258ddc910 2024-05-08T23:16:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f8526185b80 10 client.29155 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556258b59be0 con 0x556258ddc910 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc0b6080 con 0x556258ddc910 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.049+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84fc0b7f80 con 0x556258ddc910 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 10 client.29155 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 10 client.29155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 20 client.29155 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258df5f80 0x556258df83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 4 client.29155 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556258dab890 con 0x556258df5f80 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.050+0000 7f850aa00700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258df5f80 0x556258df83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.051+0000 7f850aa00700 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258df5f80 0x556258df83a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a835922609486ab7 server_cookie=944713bd3ba810eb in_seq=0 out_seq=0 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.051+0000 7f84f5600700 10 client.29155 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.051+0000 7f84f5600700 1 -- 172.21.15.46:0/206637646 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556258dab890 con 0x556258df5f80 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.051+0000 7f84f5600700 10 client.29155 handle_command_reply: tid=0 2024-05-08T23:16:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.051+0000 7f8526185b80 1 client.29155 shutdown 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 msgr2=0x556258de5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84fc099c20 tx=0x7f84fc010940 comp rx=0 tx=0).stop 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 shutdown_connections 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258df5f80 0x556258df83a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258de66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258ddc910 0x556258de5c70 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f84f5600700 0 client.29155 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 --2- 172.21.15.46:0/206637646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258de61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 >> 172.21.15.46:0/206637646 conn(0x556258d3a290 msgr2=0x556258dd6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 shutdown_connections 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 1 -- 172.21.15.46:0/206637646 wait complete. 2024-05-08T23:16:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.052+0000 7f8526185b80 20 client.29155 trim_cache size 0 max 16384 2024-05-08T23:16:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 Processor -- start 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 -- start start 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 0x556258de1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de27a0 0x556258de6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258de8ce0 con 0x556258ddc910 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258df2e20 con 0x556258de1e80 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.073+0000 7f8526185b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258de71d0 con 0x556258de27a0 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 0x556258de1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de27a0 0x556258de6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51920/0 (socket says 172.21.15.46:51920) 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 -- 172.21.15.46:0/1132582882 learned_addr learned my addr 172.21.15.46:0/1132582882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 -- 172.21.15.46:0/1132582882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de27a0 msgr2=0x556258de6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de27a0 0x556258de6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 -- 172.21.15.46:0/1132582882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 msgr2=0x556258de1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 0x556258de1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 -- 172.21.15.46:0/1132582882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556258dd3560 con 0x556258de1e80 2024-05-08T23:16:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850b400700 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 0x556258de1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.074+0000 7f850aa00700 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f84f80095c0 tx=0x7f84f8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f222aec6bd7a2508 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8501600700 1 -- 172.21.15.46:0/1132582882 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8017040 con 0x556258de1e80 2024-05-08T23:16:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8501600700 1 -- 172.21.15.46:0/1132582882 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f8018040 con 0x556258de1e80 2024-05-08T23:16:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8501600700 1 -- 172.21.15.46:0/1132582882 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003b40 con 0x556258de1e80 2024-05-08T23:16:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 -- 172.21.15.46:0/1132582882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 msgr2=0x556258dd6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f84f80095c0 tx=0x7f84f8009310 comp rx=0 tx=0).stop 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 -- 172.21.15.46:0/1132582882 shutdown_connections 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de27a0 0x556258de6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258ddc910 0x556258de1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 --2- 172.21.15.46:0/1132582882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258de1e80 0x556258dd6da0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 -- 172.21.15.46:0/1132582882 >> 172.21.15.46:0/1132582882 conn(0x556258d3a290 msgr2=0x556258de0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 -- 172.21.15.46:0/1132582882 shutdown_connections 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.075+0000 7f8526185b80 1 -- 172.21.15.46:0/1132582882 wait complete. 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.076+0000 7f8526185b80 1 -- start start 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.076+0000 7f8526185b80 1 Processor -- start 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.076+0000 7f8526185b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258ddc910 0x556258dec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258e4c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258decee0 con 0x556258de27a0 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258e4c990 con 0x556258ddc910 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556258e4cad0 con 0x556258de1e80 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f8526185b80 10 client.0 _ll_register_callbacks cb 0x556258cd1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258e4c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38312/0 (socket says 172.21.15.46:38312) 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 -- 172.21.15.46:0/3529003077 learned_addr learned my addr 172.21.15.46:0/3529003077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258ddc910 0x556258dec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 -- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258ddc910 msgr2=0x556258dec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258ddc910 0x556258dec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 -- 172.21.15.46:0/3529003077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 msgr2=0x556258e4c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258e4c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.077+0000 7f850aa00700 1 -- 172.21.15.46:0/3529003077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556258b59be0 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f850aa00700 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84f8022040 tx=0x7f84f8012dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc2427d962e14a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f84f5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8004260 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84fc09dd50 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f84f80043c0 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f8004580 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f8526185b80 10 client.29161 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.078+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f84f8009d80 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc09dd50 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84f8004750 con 0x556258de1e80 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 10 client.29161 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 10 client.29161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 20 client.29161 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258e4e1e0 0x556258e505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 4 client.29161 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:16:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556258d24f40 con 0x556258e4e1e0 2024-05-08T23:16:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.079+0000 7f850b400700 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258e4e1e0 0x556258e505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.080+0000 7f850b400700 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258e4e1e0 0x556258e505d0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe2bc3355db8f4d server_cookie=60867f40ba007566 in_seq=0 out_seq=0 2024-05-08T23:16:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.080+0000 7f84f5600700 10 client.29161 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f84f5600700 1 -- 172.21.15.46:0/3529003077 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556258d24f40 con 0x556258e4e1e0 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f84f5600700 10 client.29161 handle_command_reply: tid=0 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 client.29161 shutdown 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 msgr2=0x556258dec7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84f8022040 tx=0x7f84f8012dd0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 shutdown_connections 2024-05-08T23:16:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556258e4e1e0 0x556258e505d0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556258de27a0 0x556258e4c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556258ddc910 0x556258dec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 --2- 172.21.15.46:0/3529003077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556258de1e80 0x556258dec7f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f84f5600700 0 client.29161 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.081+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 >> 172.21.15.46:0/3529003077 conn(0x556258d3a290 msgr2=0x556258de2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 shutdown_connections 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8526185b80 1 -- 172.21.15.46:0/3529003077 wait complete. 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8526185b80 20 client.29161 trim_cache size 0 max 16384 2024-05-08T23:16:31.086 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:16:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:16:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9099e7ae-45ba-438e-9856-6cea590c3c76", 2024-05-08T23:16:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:16:31.087 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:16:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 msgr2=0x7f84f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 0x7f84f00a37d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8504003c30 tx=0x7f8504003c60 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 msgr2=0x7f850c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.082+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00e9a0 tx=0x7f84fc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 shutdown_connections 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c19df20 0x7f850c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f850c101060 0x7f850c199640 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f84f00a1310 0x7f84f00a37d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 --2- 172.21.15.46:0/4013906161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850c106180 0x7f850c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 >> 172.21.15.46:0/4013906161 conn(0x7f850c0fc900 msgr2=0x7f850c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 shutdown_connections 2024-05-08T23:16:31.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.083+0000 7f8512600700 1 -- 172.21.15.46:0/4013906161 wait complete. 2024-05-08T23:16:31.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:16:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 -- 172.21.15.46:0/2197381630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc101080 msgr2=0x7f07fc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 --2- 172.21.15.46:0/2197381630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc101080 0x7f07fc101460 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009990 tx=0x7f07ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 -- 172.21.15.46:0/2197381630 shutdown_connections 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 --2- 172.21.15.46:0/2197381630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc1061a0 0x7f07fc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 --2- 172.21.15.46:0/2197381630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc101080 0x7f07fc101460 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 --2- 172.21.15.46:0/2197381630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc101a30 0x7f07fc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 -- 172.21.15.46:0/2197381630 >> 172.21.15.46:0/2197381630 conn(0x7f07fc0fc8e0 msgr2=0x7f07fc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.330+0000 7f0801a00700 1 -- 172.21.15.46:0/2197381630 shutdown_connections 2024-05-08T23:16:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.331+0000 7f0801a00700 1 -- 172.21.15.46:0/2197381630 wait complete. 2024-05-08T23:16:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 Processor -- start 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 -- start start 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 0x7f07fc197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc19bbd0 0x7f07fc19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19c650 con 0x7f07fc1061a0 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19c7c0 con 0x7f07fc101080 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.332+0000 7f0801a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19c980 con 0x7f07fc19bbd0 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07fa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 0x7f07fc197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51950/0 (socket says 172.21.15.46:51950) 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 -- 172.21.15.46:0/4220973757 learned_addr learned my addr 172.21.15.46:0/4220973757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07fb400700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc19bbd0 0x7f07fc19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 -- 172.21.15.46:0/4220973757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc19bbd0 msgr2=0x7f07fc19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc19bbd0 0x7f07fc19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 -- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 msgr2=0x7f07fc197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 0x7f07fc197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07faa00700 1 -- 172.21.15.46:0/4220973757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0095f0 con 0x7f07fc101080 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.333+0000 7f07fa000700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 0x7f07fc197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f07faa00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005770 tx=0x7f07ec0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f07f3400700 1 -- 172.21.15.46:0/4220973757 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec01d070 con 0x7f07fc101080 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc0731d0 con 0x7f07fc101080 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f07f3400700 1 -- 172.21.15.46:0/4220973757 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec022470 con 0x7f07fc101080 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f07f3400700 1 -- 172.21.15.46:0/4220973757 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec017650 con 0x7f07fc101080 2024-05-08T23:16:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.334+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc0736d0 con 0x7f07fc101080 2024-05-08T23:16:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.337+0000 7f07f3400700 1 -- 172.21.15.46:0/4220973757 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f07ec00fb00 con 0x7f07fc101080 2024-05-08T23:16:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.338+0000 7f0815505b80 1 Processor -- start 2024-05-08T23:16:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.338+0000 7f0815505b80 1 -- start start 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe993b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9892150 con 0x55dbe993b890 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe988fb10 con 0x55dbe9940e00 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f0815505b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe988b400 con 0x55dbe9941720 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07f3400700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 0x7f07e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07fa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07fa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58174/0 (socket says 172.21.15.46:58174) 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07faa00700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe993b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07fa000700 1 -- 172.21.15.46:0/2683870342 learned_addr learned my addr 172.21.15.46:0/2683870342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07fb400700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07f3400700 1 -- 172.21.15.46:0/4220973757 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f07ec0ce490 con 0x7f07fc101080 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.339+0000 7f07fb400700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 0x7f07e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 -- 172.21.15.46:0/2683870342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 msgr2=0x55dbe99411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 -- 172.21.15.46:0/2683870342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 msgr2=0x55dbe993bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 -- 172.21.15.46:0/2683870342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe970c3b0 con 0x55dbe9941720 2024-05-08T23:16:31.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07fa000700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07faa00700 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe993b350 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0cb770 tx=0x7f07ec090a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83963a2f6935311a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07fb400700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 0x7f07e00a3750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007b90 tx=0x7f07f4009100 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07e7e00700 1 -- 172.21.15.46:0/2683870342 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec09b0b0 con 0x55dbe9941720 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07e7e00700 1 -- 172.21.15.46:0/2683870342 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec09ae20 con 0x55dbe9941720 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.340+0000 7f07e7e00700 1 -- 172.21.15.46:0/2683870342 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec09b370 con 0x55dbe9941720 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 -- 172.21.15.46:0/2683870342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 msgr2=0x55dbe993b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe993b350 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0cb770 tx=0x7f07ec090a30 comp rx=0 tx=0).stop 2024-05-08T23:16:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 -- 172.21.15.46:0/2683870342 shutdown_connections 2024-05-08T23:16:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 --2- 172.21.15.46:0/2683870342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe993b350 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 -- 172.21.15.46:0/2683870342 >> 172.21.15.46:0/2683870342 conn(0x55dbe9899210 msgr2=0x55dbe9935df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 -- 172.21.15.46:0/2683870342 shutdown_connections 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.341+0000 7f0815505b80 1 -- 172.21.15.46:0/2683870342 wait complete. 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 -- start start 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 Processor -- start 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe9947800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9941720 0x55dbe9945230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9827560 con 0x55dbe993b890 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9827200 con 0x55dbe9941720 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe970e6a0 con 0x55dbe9940e00 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.342+0000 7f0815505b80 10 client.0 _ll_register_callbacks cb 0x55dbe9830b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07faa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe9947800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38364/0 (socket says 172.21.15.46:38364) 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 -- 172.21.15.46:0/493466084 learned_addr learned my addr 172.21.15.46:0/493466084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fb400700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9941720 0x55dbe9945230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 -- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9941720 msgr2=0x55dbe9945230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9941720 0x55dbe9945230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 -- 172.21.15.46:0/493466084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 msgr2=0x55dbe9947800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe9947800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 -- 172.21.15.46:0/493466084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e8010e30 con 0x55dbe9940e00 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07faa00700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe9947800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07fa000700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009f10 tx=0x7f07e800b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a375ebd917c8ec96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.343+0000 7f07e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015040 con 0x55dbe9940e00 2024-05-08T23:16:31.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f07e8004830 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbe97e4270 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e80049f0 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 10 client.29173 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f07fc0736d0 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbe97e4270 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07e8025400 con 0x55dbe9940e00 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 10 client.29173 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 10 client.29173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 20 client.29173 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe9954ea0 0x55dbe99572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 4 client.29173 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.344+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbe990a810 con 0x55dbe9954ea0 2024-05-08T23:16:31.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.345+0000 7f07faa00700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe9954ea0 0x55dbe99572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.345+0000 7f07faa00700 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe9954ea0 0x55dbe99572c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aeb6121fabf3990 server_cookie=5eec09753913446e in_seq=0 out_seq=0 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.345+0000 7f07e4c00700 10 client.29173 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f07e4c00700 1 -- 172.21.15.46:0/493466084 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbe990a810 con 0x55dbe9954ea0 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f07e4c00700 10 client.29173 handle_command_reply: tid=0 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 client.29173 shutdown 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 msgr2=0x55dbe9944af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009f10 tx=0x7f07e800b5a0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 shutdown_connections 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9941720 0x55dbe9945230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe9954ea0 0x55dbe99572c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe9947800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 --2- 172.21.15.46:0/493466084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe9944af0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 >> 172.21.15.46:0/493466084 conn(0x55dbe9899210 msgr2=0x55dbe99355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f07e4c00700 0 client.29173 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 shutdown_connections 2024-05-08T23:16:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.346+0000 7f0815505b80 1 -- 172.21.15.46:0/493466084 wait complete. 2024-05-08T23:16:31.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.347+0000 7f0815505b80 20 client.29173 trim_cache size 0 max 16384 2024-05-08T23:16:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 Processor -- start 2024-05-08T23:16:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 -- start start 2024-05-08T23:16:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe9935d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe99525a0 con 0x55dbe993b890 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.365+0000 7f0815505b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe99460f0 con 0x55dbe9940e00 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f0815505b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9932dc0 con 0x55dbe9941720 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07faa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51988/0 (socket says 172.21.15.46:51988) 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 -- 172.21.15.46:0/1369098159 learned_addr learned my addr 172.21.15.46:0/1369098159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe9935d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 -- 172.21.15.46:0/1369098159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 msgr2=0x55dbe9935d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe9935d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 -- 172.21.15.46:0/1369098159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 msgr2=0x55dbe993b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 -- 172.21.15.46:0/1369098159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe96b8b60 con 0x55dbe9940e00 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.366+0000 7f07fa000700 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f07e8025860 tx=0x7f07e8009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf3827484282335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.367+0000 7f07f0c00700 1 -- 172.21.15.46:0/1369098159 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015070 con 0x55dbe9940e00 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.367+0000 7f07f0c00700 1 -- 172.21.15.46:0/1369098159 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e8016040 con 0x55dbe9940e00 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.367+0000 7f07f0c00700 1 -- 172.21.15.46:0/1369098159 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8003cb0 con 0x55dbe9940e00 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.367+0000 7f0815505b80 1 -- 172.21.15.46:0/1369098159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 msgr2=0x55dbe99402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.367+0000 7f0815505b80 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f07e8025860 tx=0x7f07e8009480 comp rx=0 tx=0).stop 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 -- 172.21.15.46:0/1369098159 shutdown_connections 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9941720 0x55dbe9935d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe993b890 0x55dbe993b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 --2- 172.21.15.46:0/1369098159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe9940e00 0x55dbe99402c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 -- 172.21.15.46:0/1369098159 >> 172.21.15.46:0/1369098159 conn(0x55dbe9899210 msgr2=0x55dbe993fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 -- 172.21.15.46:0/1369098159 shutdown_connections 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 -- 172.21.15.46:0/1369098159 wait complete. 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 -- start start 2024-05-08T23:16:31.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.368+0000 7f0815505b80 1 Processor -- start 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 0x55dbe994bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe9941720 0x55dbe9948200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9948870 con 0x55dbe9941720 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe99489e0 con 0x55dbe993b890 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe9948ba0 con 0x55dbe9940e00 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f0815505b80 10 client.0 _ll_register_callbacks cb 0x55dbe9830b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07faa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 0x55dbe994bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07faa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 0x55dbe994bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51996/0 (socket says 172.21.15.46:51996) 2024-05-08T23:16:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38386/0 (socket says 172.21.15.46:38386) 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07faa00700 1 -- 172.21.15.46:0/2177956222 learned_addr learned my addr 172.21.15.46:0/2177956222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.369+0000 7f07fb400700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe9941720 0x55dbe9948200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 -- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 msgr2=0x55dbe994bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 0x55dbe994bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 -- 172.21.15.46:0/2177956222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe9941720 msgr2=0x55dbe9948200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe9941720 0x55dbe9948200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 -- 172.21.15.46:0/2177956222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc0736d0 con 0x55dbe9940e00 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07fa000700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x55dbe9945aa0 tx=0x7f07e8025bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c2fbbcd9323d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015040 con 0x55dbe9940e00 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07ec09ae20 con 0x55dbe9940e00 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f07e8003680 con 0x55dbe9940e00 2024-05-08T23:16:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.370+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e8003840 con 0x55dbe9940e00 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 10 client.29179 fetch_fsmap learned FSMap version 30 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dbe9851560 con 0x55dbe9940e00 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec09ae20 con 0x55dbe9940e00 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07e800b2c0 con 0x55dbe9940e00 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 10 client.29179 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 10 client.29179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 20 client.29179 populate_metadata read hostname 'smithi046' 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe99ac1c0 0x55dbe99ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 4 client.29179 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dbe9883ec0 con 0x55dbe99ac1c0 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.371+0000 7f07faa00700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe99ac1c0 0x55dbe99ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f07faa00700 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe99ac1c0 0x55dbe99ae5b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a008f6a4d8d02a65 server_cookie=28960d08040b893e in_seq=0 out_seq=0 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f07e4c00700 10 client.29179 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f07e4c00700 1 -- 172.21.15.46:0/2177956222 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dbe9883ec0 con 0x55dbe99ac1c0 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f07e4c00700 10 client.29179 handle_command_reply: tid=0 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 client.29179 shutdown 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 msgr2=0x55dbe994c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x55dbe9945aa0 tx=0x7f07e8025bb0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 shutdown_connections 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55dbe99ac1c0 0x55dbe99ae5b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbe9941720 0x55dbe9948200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe993b890 0x55dbe994bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 --2- 172.21.15.46:0/2177956222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbe9940e00 0x55dbe994c050 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f07e4c00700 0 client.29179 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.372+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 >> 172.21.15.46:0/2177956222 conn(0x55dbe9899210 msgr2=0x55dbe9941bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 shutdown_connections 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0815505b80 1 -- 172.21.15.46:0/2177956222 wait complete. 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0815505b80 20 client.29179 trim_cache size 0 max 16384 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "9099e7ae-45ba-438e-9856-6cea590c3c76": { 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9099e7ae-45ba-438e-9856-6cea590c3c76", 2024-05-08T23:16:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:16:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:16:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:16:31.379 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:16:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 msgr2=0x7f07e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 0x7f07e00a3750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007b90 tx=0x7f07f4009100 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 msgr2=0x7f07fc197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005770 tx=0x7f07ec0057a0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 shutdown_connections 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc19bbd0 0x7f07fc19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc101080 0x7f07fc197340 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f07e00a1290 0x7f07e00a3750 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 --2- 172.21.15.46:0/4220973757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07fc1061a0 0x7f07fc197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.373+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 >> 172.21.15.46:0/4220973757 conn(0x7f07fc0fc8e0 msgr2=0x7f07fc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.374+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 shutdown_connections 2024-05-08T23:16:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:16:31.374+0000 7f0801a00700 1 -- 172.21.15.46:0/4220973757 wait complete. 2024-05-08T23:16:31.392 INFO:tasks.cephfs.filesystem:scrub status for tag:9099e7ae-45ba-438e-9856-6cea590c3c76 - {'path': '/', 'tag': '9099e7ae-45ba-438e-9856-6cea590c3c76', 'options': 'recursive,force'} 2024-05-08T23:16:31.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1109466 9.15 MB/sec execute 533 sec latency 29.544 ms 2024-05-08T23:16:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: pgmap v396: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:31 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: pgmap v396: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:31 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: pgmap v396: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:31 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:32.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1111726 9.16 MB/sec execute 534 sec latency 18.575 ms 2024-05-08T23:16:33.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:32 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:16:33.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:32 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:32 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:16:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:32 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:32 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:16:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:32 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:16:33.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1112592 9.15 MB/sec execute 535 sec latency 18.348 ms 2024-05-08T23:16:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:33 smithi086 ceph-mon[78610]: pgmap v397: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-05-08T23:16:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:33 smithi053 ceph-mon[77587]: pgmap v397: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-05-08T23:16:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:33 smithi046 ceph-mon[36898]: pgmap v397: 129 pgs: 129 active+clean; 602 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-05-08T23:16:34.916 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1114542 9.15 MB/sec execute 536 sec latency 89.238 ms 2024-05-08T23:16:35.918 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1115459 9.14 MB/sec execute 537 sec latency 194.273 ms 2024-05-08T23:16:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:35 smithi086 ceph-mon[78610]: pgmap v398: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-05-08T23:16:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:35 smithi053 ceph-mon[77587]: pgmap v398: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-05-08T23:16:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:35 smithi046 ceph-mon[36898]: pgmap v398: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-05-08T23:16:36.918 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1115890 9.13 MB/sec execute 538 sec latency 310.901 ms 2024-05-08T23:16:37.918 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1117469 9.12 MB/sec execute 539 sec latency 23.259 ms 2024-05-08T23:16:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:37 smithi086 ceph-mon[78610]: pgmap v399: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-05-08T23:16:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:37 smithi053 ceph-mon[77587]: pgmap v399: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-05-08T23:16:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:37 smithi046 ceph-mon[36898]: pgmap v399: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-05-08T23:16:38.918 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1119004 9.13 MB/sec execute 540 sec latency 15.963 ms 2024-05-08T23:16:39.918 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1120393 9.12 MB/sec execute 541 sec latency 16.505 ms 2024-05-08T23:16:40.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:39 smithi086 ceph-mon[78610]: pgmap v400: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-05-08T23:16:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:39 smithi053 ceph-mon[77587]: pgmap v400: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-05-08T23:16:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:39 smithi046 ceph-mon[36898]: pgmap v400: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-05-08T23:16:40.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1122426 9.13 MB/sec execute 542 sec latency 56.100 ms 2024-05-08T23:16:41.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1123169 9.12 MB/sec execute 543 sec latency 136.045 ms 2024-05-08T23:16:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:41 smithi086 ceph-mon[78610]: pgmap v401: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2024-05-08T23:16:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:41 smithi053 ceph-mon[77587]: pgmap v401: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2024-05-08T23:16:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:41 smithi046 ceph-mon[36898]: pgmap v401: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2024-05-08T23:16:42.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1125012 9.11 MB/sec execute 544 sec latency 19.317 ms 2024-05-08T23:16:43.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1126273 9.12 MB/sec execute 545 sec latency 19.930 ms 2024-05-08T23:16:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:43 smithi086 ceph-mon[78610]: pgmap v402: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-05-08T23:16:44.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:43 smithi053 ceph-mon[77587]: pgmap v402: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-05-08T23:16:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:43 smithi046 ceph-mon[36898]: pgmap v402: 129 pgs: 129 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-05-08T23:16:44.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1127634 9.11 MB/sec execute 546 sec latency 35.376 ms 2024-05-08T23:16:45.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1129571 9.12 MB/sec execute 547 sec latency 16.628 ms 2024-05-08T23:16:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:45 smithi086 ceph-mon[78610]: pgmap v403: 129 pgs: 129 active+clean; 605 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 271 op/s 2024-05-08T23:16:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:16:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:45 smithi053 ceph-mon[77587]: pgmap v403: 129 pgs: 129 active+clean; 605 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 271 op/s 2024-05-08T23:16:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:16:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:45 smithi046 ceph-mon[36898]: pgmap v403: 129 pgs: 129 active+clean; 605 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 271 op/s 2024-05-08T23:16:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:16:46.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1130475 9.10 MB/sec execute 548 sec latency 28.767 ms 2024-05-08T23:16:47.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1132585 9.11 MB/sec execute 549 sec latency 16.286 ms 2024-05-08T23:16:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:47 smithi086 ceph-mon[78610]: pgmap v404: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 222 op/s 2024-05-08T23:16:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:47 smithi053 ceph-mon[77587]: pgmap v404: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 222 op/s 2024-05-08T23:16:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:47 smithi046 ceph-mon[36898]: pgmap v404: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 222 op/s 2024-05-08T23:16:48.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1133668 9.10 MB/sec execute 550 sec latency 29.626 ms 2024-05-08T23:16:49.919 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1135559 9.10 MB/sec execute 551 sec latency 27.838 ms 2024-05-08T23:16:50.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:49 smithi086 ceph-mon[78610]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-05-08T23:16:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:49 smithi053 ceph-mon[77587]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-05-08T23:16:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:49 smithi046 ceph-mon[36898]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-05-08T23:16:50.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1136976 9.10 MB/sec execute 552 sec latency 18.934 ms 2024-05-08T23:16:51.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1138348 9.10 MB/sec execute 553 sec latency 25.442 ms 2024-05-08T23:16:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:51 smithi086 ceph-mon[78610]: pgmap v406: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 279 op/s 2024-05-08T23:16:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:51 smithi053 ceph-mon[77587]: pgmap v406: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 279 op/s 2024-05-08T23:16:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:51 smithi046 ceph-mon[36898]: pgmap v406: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 279 op/s 2024-05-08T23:16:52.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1140288 9.10 MB/sec execute 554 sec latency 14.657 ms 2024-05-08T23:16:53.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1141385 9.09 MB/sec execute 555 sec latency 20.869 ms 2024-05-08T23:16:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:53 smithi086 ceph-mon[78610]: pgmap v407: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 144 op/s 2024-05-08T23:16:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:53 smithi053 ceph-mon[77587]: pgmap v407: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 144 op/s 2024-05-08T23:16:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:53 smithi046 ceph-mon[36898]: pgmap v407: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 144 op/s 2024-05-08T23:16:54.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1143557 9.10 MB/sec execute 556 sec latency 24.832 ms 2024-05-08T23:16:55.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1144454 9.09 MB/sec execute 557 sec latency 42.822 ms 2024-05-08T23:16:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:55 smithi086 ceph-mon[78610]: pgmap v408: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-05-08T23:16:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:55 smithi053 ceph-mon[77587]: pgmap v408: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-05-08T23:16:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:55 smithi046 ceph-mon[36898]: pgmap v408: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 199 op/s 2024-05-08T23:16:56.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1146510 9.10 MB/sec execute 558 sec latency 28.662 ms 2024-05-08T23:16:57.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1147671 9.09 MB/sec execute 559 sec latency 29.071 ms 2024-05-08T23:16:58.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:57 smithi086 ceph-mon[78610]: pgmap v409: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-05-08T23:16:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:57 smithi053 ceph-mon[77587]: pgmap v409: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-05-08T23:16:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:57 smithi046 ceph-mon[36898]: pgmap v409: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-05-08T23:16:58.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1149138 9.09 MB/sec execute 560 sec latency 24.493 ms 2024-05-08T23:16:59.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1150922 9.09 MB/sec execute 561 sec latency 20.375 ms 2024-05-08T23:17:00.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:16:59 smithi086 ceph-mon[78610]: pgmap v410: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-05-08T23:17:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:16:59 smithi053 ceph-mon[77587]: pgmap v410: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-05-08T23:17:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:16:59 smithi046 ceph-mon[36898]: pgmap v410: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-05-08T23:17:00.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1151071 9.08 MB/sec execute 562 sec latency 62.733 ms 2024-05-08T23:17:01.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:17:01.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.626+0000 7f72b0800700 1 -- 172.21.15.46:0/959004829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102000 msgr2=0x7f72a81023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.626+0000 7f72b0800700 1 --2- 172.21.15.46:0/959004829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102000 0x7f72a81023e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f729c009990 tx=0x7f729c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 -- 172.21.15.46:0/959004829 shutdown_connections 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 --2- 172.21.15.46:0/959004829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a810b3d0 0x7f72a810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 --2- 172.21.15.46:0/959004829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102000 0x7f72a81023e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 --2- 172.21.15.46:0/959004829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a8102920 0x7f72a810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 -- 172.21.15.46:0/959004829 >> 172.21.15.46:0/959004829 conn(0x7f72a80fb760 msgr2=0x7f72a80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 -- 172.21.15.46:0/959004829 shutdown_connections 2024-05-08T23:17:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.627+0000 7f72b0800700 1 -- 172.21.15.46:0/959004829 wait complete. 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 Processor -- start 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 -- start start 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8102000 0x7f72a81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 0x7f72a819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a819e410 con 0x7f72a8102000 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a819e580 con 0x7f72a810b3d0 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72b0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a819e740 con 0x7f72a8102920 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72ad400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72ad400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41844/0 (socket says 172.21.15.46:41844) 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72ad400700 1 -- 172.21.15.46:0/2889923028 learned_addr learned my addr 172.21.15.46:0/2889923028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.628+0000 7f72ade00700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8102000 0x7f72a81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ae800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 0x7f72a819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 -- 172.21.15.46:0/2889923028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 msgr2=0x7f72a819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 0x7f72a819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 -- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8102000 msgr2=0x7f72a81994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8102000 0x7f72a81994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 -- 172.21.15.46:0/2889923028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0095f0 con 0x7f72a8102920 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ae800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 0x7f72a819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72ad400700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f729800d950 tx=0x7f729800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72a2a00700 1 -- 172.21.15.46:0/2889923028 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009940 con 0x7f72a8102920 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72a2a00700 1 -- 172.21.15.46:0/2889923028 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7298010460 con 0x7f72a8102920 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a819ea20 con 0x7f72a8102920 2024-05-08T23:17:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.629+0000 7f72a2a00700 1 -- 172.21.15.46:0/2889923028 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800f620 con 0x7f72a8102920 2024-05-08T23:17:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.630+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a81a6260 con 0x7f72a8102920 2024-05-08T23:17:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.632+0000 7f72a2a00700 1 -- 172.21.15.46:0/2889923028 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f72980108d0 con 0x7f72a8102920 2024-05-08T23:17:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.634+0000 7f72a2a00700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.634+0000 7f72a2a00700 1 -- 172.21.15.46:0/2889923028 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f72980ccf60 con 0x7f72a8102920 2024-05-08T23:17:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.634+0000 7f72c4287b80 1 Processor -- start 2024-05-08T23:17:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.634+0000 7f72ade00700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.634+0000 7f72c4287b80 1 -- start start 2024-05-08T23:17:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c0b0150 con 0x55d03c15ee00 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c0adb10 con 0x55d03c15f720 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72c4287b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c0a9400 con 0x55d03c159890 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ade00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ade00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41204/0 (socket says 172.21.15.46:41204) 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ade00700 1 -- 172.21.15.46:0/2322604638 learned_addr learned my addr 172.21.15.46:0/2322604638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ade00700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 0x7f72900a37d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f729c006010 tx=0x7f729c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 -- 172.21.15.46:0/2322604638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 msgr2=0x55d03c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 -- 172.21.15.46:0/2322604638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 msgr2=0x55d03c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ad400700 1 -- 172.21.15.46:0/2322604638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03bf2a3b0 con 0x55d03c15f720 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.635+0000 7f72ade00700 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.636+0000 7f72ad400700 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c159350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72980ca240 tx=0x7f729809a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b50a72ceadd1a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.636+0000 7f7297000700 1 -- 172.21.15.46:0/2322604638 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980a2080 con 0x55d03c15f720 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.636+0000 7f7297000700 1 -- 172.21.15.46:0/2322604638 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7298096310 con 0x55d03c15f720 2024-05-08T23:17:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.636+0000 7f7297000700 1 -- 172.21.15.46:0/2322604638 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980a1bf0 con 0x55d03c15f720 2024-05-08T23:17:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- 172.21.15.46:0/2322604638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 msgr2=0x55d03c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c159350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72980ca240 tx=0x7f729809a300 comp rx=0 tx=0).stop 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- 172.21.15.46:0/2322604638 shutdown_connections 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15ee00 0x55d03c15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2322604638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c159350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- 172.21.15.46:0/2322604638 >> 172.21.15.46:0/2322604638 conn(0x55d03c0b7210 msgr2=0x55d03c153df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- 172.21.15.46:0/2322604638 shutdown_connections 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- 172.21.15.46:0/2322604638 wait complete. 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.637+0000 7f72c4287b80 1 -- start start 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 Processor -- start 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c162ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15f720 0x55d03c163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c045560 con 0x55d03c159890 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c045200 con 0x55d03c15ee00 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03bf2c6a0 con 0x55d03c15f720 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72c4287b80 10 client.0 _ll_register_callbacks cb 0x55d03c04eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72ad400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72ade00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c162ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72ad400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42366/0 (socket says 172.21.15.46:42366) 2024-05-08T23:17:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.638+0000 7f72ad400700 1 -- 172.21.15.46:0/1204421684 learned_addr learned my addr 172.21.15.46:0/1204421684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ae800700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15f720 0x55d03c163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 -- 172.21.15.46:0/1204421684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15f720 msgr2=0x55d03c163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15f720 0x55d03c163550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 -- 172.21.15.46:0/1204421684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 msgr2=0x55d03c162ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c162ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 -- 172.21.15.46:0/1204421684 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a810e530 con 0x55d03c15ee00 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72ad400700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7298010fd0 tx=0x7f729809a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857099a7b12f7556 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f728b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980a2080 con 0x55d03c15ee00 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d03c002270 con 0x55d03c15ee00 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.639+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7298099300 con 0x55d03c15ee00 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72980a1bf0 con 0x55d03c15ee00 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 10 client.29096 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f72a81a6260 con 0x55d03c15ee00 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d03c002270 con 0x55d03c15ee00 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7298098b60 con 0x55d03c15ee00 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 10 client.29096 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 10 client.29096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 20 client.29096 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c172d90 0x55d03c175180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 4 client.29096 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.640+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d03c128810 con 0x55d03c172d90 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.641+0000 7f72ade00700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c172d90 0x55d03c175180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.641+0000 7f72ade00700 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c172d90 0x55d03c175180 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0d5fdbe7eca041 server_cookie=c80cdfb66a366bf2 in_seq=0 out_seq=0 2024-05-08T23:17:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.641+0000 7f728b400700 10 client.29096 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f728b400700 1 -- 172.21.15.46:0/1204421684 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d03c128810 con 0x55d03c172d90 2024-05-08T23:17:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f728b400700 10 client.29096 handle_command_reply: tid=0 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 client.29096 shutdown 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 msgr2=0x55d03c163010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7298010fd0 tx=0x7f729809a090 comp rx=0 tx=0).stop 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 shutdown_connections 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15f720 0x55d03c163550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c172d90 0x55d03c175180 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c162ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 --2- 172.21.15.46:0/1204421684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c163010 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f728b400700 0 client.29096 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.642+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 >> 172.21.15.46:0/1204421684 conn(0x55d03c0b7210 msgr2=0x55d03c1535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.643+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 shutdown_connections 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.643+0000 7f72c4287b80 1 -- 172.21.15.46:0/1204421684 wait complete. 2024-05-08T23:17:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.643+0000 7f72c4287b80 20 client.29096 trim_cache size 0 max 16384 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.663+0000 7f72c4287b80 1 Processor -- start 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.663+0000 7f72c4287b80 1 -- start start 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15f720 0x55d03c16a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c165640 con 0x55d03c15f720 2024-05-08T23:17:01.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c150dc0 con 0x55d03c15ee00 2024-05-08T23:17:01.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72c4287b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c17a320 con 0x55d03c159890 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41854/0 (socket says 172.21.15.46:41854) 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72ade00700 1 -- 172.21.15.46:0/2236499587 learned_addr learned my addr 172.21.15.46:0/2236499587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72ae800700 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15f720 0x55d03c16a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.664+0000 7f72ad400700 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 -- 172.21.15.46:0/2236499587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 msgr2=0x55d03c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 -- 172.21.15.46:0/2236499587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15f720 msgr2=0x55d03c16a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15f720 0x55d03c16a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 -- 172.21.15.46:0/2236499587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03bed6b60 con 0x55d03c159890 2024-05-08T23:17:01.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f72ade00700 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b6f0 tx=0x7f729c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6610c6a616be525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f7297a00700 1 -- 172.21.15.46:0/2236499587 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00bd90 con 0x55d03c159890 2024-05-08T23:17:01.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f7297a00700 1 -- 172.21.15.46:0/2236499587 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729c01e040 con 0x55d03c159890 2024-05-08T23:17:01.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.665+0000 7f7297a00700 1 -- 172.21.15.46:0/2236499587 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c035720 con 0x55d03c159890 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 -- 172.21.15.46:0/2236499587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 msgr2=0x55d03c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b6f0 tx=0x7f729c020000 comp rx=0 tx=0).stop 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 -- 172.21.15.46:0/2236499587 shutdown_connections 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c15f720 0x55d03c16a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c159890 0x55d03c159350 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 --2- 172.21.15.46:0/2236499587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15ee00 0x55d03c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 -- 172.21.15.46:0/2236499587 >> 172.21.15.46:0/2236499587 conn(0x55d03c0b7210 msgr2=0x55d03c1557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 -- 172.21.15.46:0/2236499587 shutdown_connections 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.666+0000 7f72c4287b80 1 -- 172.21.15.46:0/2236499587 wait complete. 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 -- start start 2024-05-08T23:17:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 Processor -- start 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15ee00 0x55d03c16a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c15b0a0 con 0x55d03c159890 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c15b210 con 0x55d03c15f720 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03c15b3d0 con 0x55d03c15ee00 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.667+0000 7f72c4287b80 10 client.0 _ll_register_callbacks cb 0x55d03c04eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41230/0 (socket says 172.21.15.46:41230) 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 -- 172.21.15.46:0/3725790707 learned_addr learned my addr 172.21.15.46:0/3725790707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ae800700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ad400700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15ee00 0x55d03c16a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 -- 172.21.15.46:0/3725790707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15ee00 msgr2=0x55d03c16a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15ee00 0x55d03c16a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 -- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 msgr2=0x55d03c15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.668+0000 7f72ade00700 1 -- 172.21.15.46:0/3725790707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c02c040 con 0x55d03c159890 2024-05-08T23:17:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f72ade00700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f729c005800 tx=0x7f729c0350f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c58543006d86d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f728b400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:17:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c043070 con 0x55d03c159890 2024-05-08T23:17:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f729c0353a0 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7298096310 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f729c00be90 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f72c4287b80 10 client.19077 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.669+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d03c06f560 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7298096310 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f729c046030 con 0x55d03c159890 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 10 client.19077 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 20 client.19077 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c1c9e60 0x55d03c1cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 4 client.19077 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d03c0a1ec0 con 0x55d03c1c9e60 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.670+0000 7f72ad400700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c1c9e60 0x55d03c1cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f72ad400700 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c1c9e60 0x55d03c1cc250 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0d728c71835f5e server_cookie=4f6cd84728243197 in_seq=0 out_seq=0 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f728b400700 10 client.19077 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f728b400700 1 -- 172.21.15.46:0/3725790707 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d03c0a1ec0 con 0x55d03c1c9e60 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f728b400700 10 client.19077 handle_command_reply: tid=0 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f72c4287b80 1 client.19077 shutdown 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 msgr2=0x55d03c16a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.671+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f729c005800 tx=0x7f729c0350f0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 shutdown_connections 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d03c15f720 0x55d03c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d03c1c9e60 0x55d03c1cc250 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d03c159890 0x55d03c16a1a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 --2- 172.21.15.46:0/3725790707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d03c15ee00 0x55d03c16a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 >> 172.21.15.46:0/3725790707 conn(0x55d03c0b7210 msgr2=0x55d03c161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f728b400700 0 client.19077 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 shutdown_connections 2024-05-08T23:17:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 1 -- 172.21.15.46:0/3725790707 wait complete. 2024-05-08T23:17:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.672+0000 7f72c4287b80 20 client.19077 trim_cache size 0 max 16384 2024-05-08T23:17:01.676 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:01.676 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:17:01.676 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:17:01.676 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 msgr2=0x7f72900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 0x7f72900a37d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f729c006010 tx=0x7f729c00b260 comp rx=0 tx=0).stop 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 msgr2=0x7f72a8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f729800d950 tx=0x7f729800dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 shutdown_connections 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a810b3d0 0x7f72a819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a8102000 0x7f72a81994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 --2- 172.21.15.46:0/2889923028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8102920 0x7f72a8199a30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 >> 172.21.15.46:0/2889923028 conn(0x7f72a80fb760 msgr2=0x7f72a80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 shutdown_connections 2024-05-08T23:17:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.673+0000 7f72b0800700 1 -- 172.21.15.46:0/2889923028 wait complete. 2024-05-08T23:17:01.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:17:01.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:01.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1152245 9.07 MB/sec execute 563 sec latency 62.940 ms 2024-05-08T23:17:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.920+0000 7ff2b6200700 1 -- 172.21.15.46:0/1714745563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 msgr2=0x7ff2b010d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.920+0000 7ff2b6200700 1 --2- 172.21.15.46:0/1714745563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b010d210 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098f0 tx=0x7ff29c009c00 comp rx=0 tx=0).stop 2024-05-08T23:17:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 -- 172.21.15.46:0/1714745563 shutdown_connections 2024-05-08T23:17:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 --2- 172.21.15.46:0/1714745563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 0x7ff2b010fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 --2- 172.21.15.46:0/1714745563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 0x7ff2b00fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 --2- 172.21.15.46:0/1714745563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b010d210 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 -- 172.21.15.46:0/1714745563 >> 172.21.15.46:0/1714745563 conn(0x7ff2b00fb820 msgr2=0x7ff2b00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 -- 172.21.15.46:0/1714745563 shutdown_connections 2024-05-08T23:17:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.921+0000 7ff2b6200700 1 -- 172.21.15.46:0/1714745563 wait complete. 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 Processor -- start 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 -- start start 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 0x7ff2b0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 0x7ff2b019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b019e460 con 0x7ff2b00ffba0 2024-05-08T23:17:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b019e5d0 con 0x7ff2b010d750 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.922+0000 7ff2b6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b019e790 con 0x7ff2b0100550 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2af400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 0x7ff2b0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2afe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 0x7ff2b019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40694/0 (socket says 172.21.15.46:40694) 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 -- 172.21.15.46:0/2221881369 learned_addr learned my addr 172.21.15.46:0/2221881369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 -- 172.21.15.46:0/2221881369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 msgr2=0x7ff2b019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 0x7ff2b019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 -- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 msgr2=0x7ff2b0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 0x7ff2b0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 -- 172.21.15.46:0/2221881369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00095f0 con 0x7ff2b0100550 2024-05-08T23:17:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.923+0000 7ff2aea00700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff29c005950 tx=0x7ff29c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.924+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01c070 con 0x7ff2b0100550 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.924+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29c00baa0 con 0x7ff2b0100550 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.924+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0038e0 con 0x7ff2b0100550 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.924+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c009520 con 0x7ff2b0100550 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.924+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b019ec50 con 0x7ff2b0100550 2024-05-08T23:17:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.925+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0108970 con 0x7ff2b0100550 2024-05-08T23:17:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.928+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff29c020030 con 0x7ff2b0100550 2024-05-08T23:17:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.929+0000 7ff2a7e00700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 0x7ff2940a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.929+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff29c0cde60 con 0x7ff2b0100550 2024-05-08T23:17:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.930+0000 7ff2af400700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 0x7ff2940a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:01.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.931+0000 7ff2af400700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 0x7ff2940a37d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005fd0 tx=0x7ff2a0009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:01.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:01.934+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff29c096ed0 con 0x7ff2b0100550 2024-05-08T23:17:02.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:01 smithi046 ceph-mon[36898]: pgmap v411: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-05-08T23:17:02.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:01 smithi086 ceph-mon[78610]: pgmap v411: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-05-08T23:17:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.245+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b00689e0 con 0x7ff2b0100550 2024-05-08T23:17:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.247+0000 7ff2a7e00700 1 -- 172.21.15.46:0/2221881369 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7ff29c09a4f0 con 0x7ff2b0100550 2024-05-08T23:17:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:17:02.248 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 msgr2=0x7ff2940a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 0x7ff2940a37d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005fd0 tx=0x7ff2a0009310 comp rx=0 tx=0).stop 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 msgr2=0x7ff2b0199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff29c005950 tx=0x7ff29c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 shutdown_connections 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b010d750 0x7ff2b019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2940a1310 0x7ff2940a37d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b00ffba0 0x7ff2b0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 --2- 172.21.15.46:0/2221881369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2b0100550 0x7ff2b0199a80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 >> 172.21.15.46:0/2221881369 conn(0x7ff2b00fb820 msgr2=0x7ff2b00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 shutdown_connections 2024-05-08T23:17:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.251+0000 7ff2b6200700 1 -- 172.21.15.46:0/2221881369 wait complete. 2024-05-08T23:17:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:17:02.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:01 smithi053 ceph-mon[77587]: pgmap v411: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-05-08T23:17:02.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:17:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 -- 172.21.15.46:0/1872057872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 msgr2=0x7f6fe410d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 --2- 172.21.15.46:0/1872057872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe410d140 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009990 tx=0x7f6fd0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 -- 172.21.15.46:0/1872057872 shutdown_connections 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 --2- 172.21.15.46:0/1872057872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe410d680 0x7f6fe410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 --2- 172.21.15.46:0/1872057872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe4073370 0x7f6fe4073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 --2- 172.21.15.46:0/1872057872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe410d140 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 -- 172.21.15.46:0/1872057872 >> 172.21.15.46:0/1872057872 conn(0x7f6fe40fc8f0 msgr2=0x7f6fe40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 -- 172.21.15.46:0/1872057872 shutdown_connections 2024-05-08T23:17:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.492+0000 7f6feb600700 1 -- 172.21.15.46:0/1872057872 wait complete. 2024-05-08T23:17:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 Processor -- start 2024-05-08T23:17:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 -- start start 2024-05-08T23:17:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 0x7f6fe4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe410d680 0x7f6fe419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419e460 con 0x7f6fe410d680 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419e5d0 con 0x7f6fe4073370 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.493+0000 7f6feb600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419e790 con 0x7f6fe4073c90 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 0x7f6fe4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe9600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe410d680 0x7f6fe419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 0x7f6fe4199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48230/0 (socket says 172.21.15.46:48230) 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40714/0 (socket says 172.21.15.46:40714) 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe8c00700 1 -- 172.21.15.46:0/14337334 learned_addr learned my addr 172.21.15.46:0/14337334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 -- 172.21.15.46:0/14337334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 msgr2=0x7f6fe4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 0x7f6fe4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 -- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe410d680 msgr2=0x7f6fe419de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe410d680 0x7f6fe419de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 -- 172.21.15.46:0/14337334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00095f0 con 0x7f6fe4073c90 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.494+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00098e0 tx=0x7f6fd0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.495+0000 7f6fe1600700 1 -- 172.21.15.46:0/14337334 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd001d070 con 0x7f6fe4073c90 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.495+0000 7f6fe1600700 1 -- 172.21.15.46:0/14337334 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fd0022470 con 0x7f6fe4073c90 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.495+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe419ea10 con 0x7f6fe4073c90 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.495+0000 7f6fe1600700 1 -- 172.21.15.46:0/14337334 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0017650 con 0x7f6fe4073c90 2024-05-08T23:17:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.495+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41a62b0 con 0x7f6fe4073c90 2024-05-08T23:17:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.498+0000 7f6fe1600700 1 -- 172.21.15.46:0/14337334 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6fd00225e0 con 0x7f6fe4073c90 2024-05-08T23:17:02.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.500+0000 7f6fe1600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 0x7f6fcc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.500+0000 7f6fff102b80 1 Processor -- start 2024-05-08T23:17:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.500+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 0x7f6fcc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.500+0000 7f6fe1600700 1 -- 172.21.15.46:0/14337334 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6fd00ce480 con 0x7f6fe4073c90 2024-05-08T23:17:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.500+0000 7f6fff102b80 1 -- start start 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293539350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293490150 con 0x56429353ee00 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429348db10 con 0x56429353f720 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fff102b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293489400 con 0x564293539890 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 0x7f6fcc0a37d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d20 tx=0x7f6fd8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45374/0 (socket says 172.21.15.46:45374) 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe8c00700 1 -- 172.21.15.46:0/2393309781 learned_addr learned my addr 172.21.15.46:0/2393309781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40716/0 (socket says 172.21.15.46:40716) 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293539350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 -- 172.21.15.46:0/2393309781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 msgr2=0x564293539350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293539350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 -- 172.21.15.46:0/2393309781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 msgr2=0x56429353f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.501+0000 7f6fe9600700 1 -- 172.21.15.46:0/2393309781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429330a3b0 con 0x564293539890 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.502+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.502+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc011550 tx=0x7f6fdc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384dbbaf0d838aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.502+0000 7f6fd5c00700 1 -- 172.21.15.46:0/2393309781 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00e040 con 0x564293539890 2024-05-08T23:17:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.502+0000 7f6fd5c00700 1 -- 172.21.15.46:0/2393309781 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fdc00f040 con 0x564293539890 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.502+0000 7f6fd5c00700 1 -- 172.21.15.46:0/2393309781 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004380 con 0x564293539890 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- 172.21.15.46:0/2393309781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 msgr2=0x564293539cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc011550 tx=0x7f6fdc009920 comp rx=0 tx=0).stop 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- 172.21.15.46:0/2393309781 shutdown_connections 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564293539890 0x564293539cd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353ee00 0x56429353f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2393309781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293539350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- 172.21.15.46:0/2393309781 >> 172.21.15.46:0/2393309781 conn(0x564293497210 msgr2=0x564293533df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- 172.21.15.46:0/2393309781 shutdown_connections 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- 172.21.15.46:0/2393309781 wait complete. 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.503+0000 7f6fff102b80 1 -- start start 2024-05-08T23:17:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 Processor -- start 2024-05-08T23:17:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293542aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293543520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293425560 con 0x564293539890 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293425200 con 0x56429353f720 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429330c6a0 con 0x56429353ee00 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fff102b80 10 client.0 _ll_register_callbacks cb 0x56429342eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40718/0 (socket says 172.21.15.46:40718) 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293542aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.504+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2122544905 learned_addr learned my addr 172.21.15.46:0/2122544905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293543520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2122544905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 msgr2=0x564293543520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293543520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2122544905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 msgr2=0x564293542aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293542aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2122544905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd8017070 con 0x56429353ee00 2024-05-08T23:17:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293543520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd009b170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650ce30d4deaf8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fca000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd009b730 con 0x56429353ee00 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6fd009b2d0 con 0x56429353ee00 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.505+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642933e2270 con 0x56429353ee00 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd009f100 con 0x56429353ee00 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 10 client.29212 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f6fdc011230 con 0x56429353ee00 2024-05-08T23:17:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642933e2270 con 0x56429353ee00 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6fd00a3230 con 0x56429353ee00 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 10 client.29212 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 10 client.29212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 20 client.29212 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564293552de0 0x564293555200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 4 client.29212 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564293508810 con 0x564293552de0 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.506+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564293552de0 0x564293555200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.507+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564293552de0 0x564293555200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=111b0af8486ea888 server_cookie=8d0ca4ec3e0ecc71 in_seq=0 out_seq=0 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.507+0000 7f6fca000700 10 client.29212 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fca000700 1 -- 172.21.15.46:0/2122544905 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564293508810 con 0x564293552de0 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fca000700 10 client.29212 handle_command_reply: tid=0 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 client.29212 shutdown 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 msgr2=0x564293542fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd009b170 comp rx=0 tx=0).stop 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 shutdown_connections 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x564293552de0 0x564293555200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x564293543520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293542aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fca000700 0 client.29212 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2122544905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293542fe0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 >> 172.21.15.46:0/2122544905 conn(0x564293497210 msgr2=0x5642935335a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 shutdown_connections 2024-05-08T23:17:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 1 -- 172.21.15.46:0/2122544905 wait complete. 2024-05-08T23:17:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.508+0000 7f6fff102b80 20 client.29212 trim_cache size 0 max 16384 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.523+0000 7f6fff102b80 1 Processor -- start 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.523+0000 7f6fff102b80 1 -- start start 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564293539890 0x564293539350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 0x56429354ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293545610 con 0x56429353f720 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564293530dc0 con 0x564293539890 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fff102b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429353f560 con 0x56429353ee00 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40730/0 (socket says 172.21.15.46:40730) 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2753759651 learned_addr learned my addr 172.21.15.46:0/2753759651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 0x56429354ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564293539890 0x564293539350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2753759651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564293539890 msgr2=0x564293539350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564293539890 0x564293539350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2753759651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 msgr2=0x56429354ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 0x56429354ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2753759651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642932b6b60 con 0x56429353ee00 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.524+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 0x56429354ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.525+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0097d40 tx=0x7f6fd00094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e9d217dc1a0f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.525+0000 7f6fd6600700 1 -- 172.21.15.46:0/2753759651 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00a1400 con 0x56429353ee00 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.525+0000 7f6fd6600700 1 -- 172.21.15.46:0/2753759651 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fd00b92b0 con 0x56429353ee00 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.525+0000 7f6fd6600700 1 -- 172.21.15.46:0/2753759651 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00b94a0 con 0x56429353ee00 2024-05-08T23:17:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 -- 172.21.15.46:0/2753759651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 msgr2=0x564293555200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0097d40 tx=0x7f6fd00094b0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 -- 172.21.15.46:0/2753759651 shutdown_connections 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56429353f720 0x56429354ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564293539890 0x564293539350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2753759651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293555200 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 -- 172.21.15.46:0/2753759651 >> 172.21.15.46:0/2753759651 conn(0x564293497210 msgr2=0x564293543380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 -- 172.21.15.46:0/2753759651 shutdown_connections 2024-05-08T23:17:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.526+0000 7f6fff102b80 1 -- 172.21.15.46:0/2753759651 wait complete. 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.527+0000 7f6fff102b80 1 -- start start 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.527+0000 7f6fff102b80 1 Processor -- start 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293549860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x5642935a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642935a08f0 con 0x564293539890 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642935a0a60 con 0x56429353f720 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642935a0c20 con 0x56429353ee00 2024-05-08T23:17:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fff102b80 10 client.0 _ll_register_callbacks cb 0x56429342eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293549860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40746/0 (socket says 172.21.15.46:40746) 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293549860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45402/0 (socket says 172.21.15.46:45402) 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2211647689 learned_addr learned my addr 172.21.15.46:0/2211647689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x5642935a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2211647689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 msgr2=0x5642935a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x5642935a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2211647689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 msgr2=0x564293549860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.528+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293549860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fe3e00700 1 -- 172.21.15.46:0/2211647689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc011230 con 0x56429353ee00 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fe9600700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x5642935a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fe3e00700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd00ce3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17dbf56c742d5af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fca000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd009b2d0 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6fd009f100 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fdc00f040 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd00a3230 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fff102b80 10 client.29218 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56429344f560 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.529+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fdc00f040 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6fd00a1090 con 0x56429353ee00 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 10 client.29218 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 10 client.29218 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 20 client.29218 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5642935a9180 0x5642935ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 4 client.29218 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564293481ec0 con 0x5642935a9180 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5642935a9180 0x5642935ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fca000700 10 client.29218 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.530+0000 7f6fe8c00700 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5642935a9180 0x5642935ab570 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=663b575f6aded137 server_cookie=70511b4078659d99 in_seq=0 out_seq=0 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fca000700 1 -- 172.21.15.46:0/2211647689 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564293481ec0 con 0x5642935a9180 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fca000700 10 client.29218 handle_command_reply: tid=0 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 client.29218 shutdown 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 msgr2=0x564293549da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd00ce3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 shutdown_connections 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5642935a9180 0x5642935ab570 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56429353f720 0x5642935a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564293539890 0x564293549860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 --2- 172.21.15.46:0/2211647689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56429353ee00 0x564293549da0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fca000700 0 client.29218 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 >> 172.21.15.46:0/2211647689 conn(0x564293497210 msgr2=0x56429353dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 shutdown_connections 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 1 -- 172.21.15.46:0/2211647689 wait complete. 2024-05-08T23:17:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.531+0000 7f6fff102b80 20 client.29218 trim_cache size 0 max 16384 2024-05-08T23:17:02.539 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 msgr2=0x7f6fcc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 0x7f6fcc0a37d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d20 tx=0x7f6fd8009260 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 msgr2=0x7f6fe4199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00098e0 tx=0x7f6fd0005740 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 shutdown_connections 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6fcc0a1310 0x7f6fcc0a37d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe410d680 0x7f6fe419de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fe4073370 0x7f6fe4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 --2- 172.21.15.46:0/14337334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fe4073c90 0x7f6fe4199a80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 >> 172.21.15.46:0/14337334 conn(0x7f6fe40fc8f0 msgr2=0x7f6fe40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 shutdown_connections 2024-05-08T23:17:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.532+0000 7f6feb600700 1 -- 172.21.15.46:0/14337334 wait complete. 2024-05-08T23:17:02.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 -- 172.21.15.46:0/4092723333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a410b3c0 msgr2=0x7ff1a410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 --2- 172.21.15.46:0/4092723333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a410b3c0 0x7ff1a410d880 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00b0a0 tx=0x7ff19c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 -- 172.21.15.46:0/4092723333 shutdown_connections 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 --2- 172.21.15.46:0/4092723333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a410b3c0 0x7ff1a410d880 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 --2- 172.21.15.46:0/4092723333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 --2- 172.21.15.46:0/4092723333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41028c0 0x7ff1a410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 -- 172.21.15.46:0/4092723333 >> 172.21.15.46:0/4092723333 conn(0x7ff1a40fb760 msgr2=0x7ff1a40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 -- 172.21.15.46:0/4092723333 shutdown_connections 2024-05-08T23:17:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.788+0000 7ff1ac600700 1 -- 172.21.15.46:0/4092723333 wait complete. 2024-05-08T23:17:02.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.789+0000 7ff1ac600700 1 Processor -- start 2024-05-08T23:17:02.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 -- start start 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 0x7ff1a4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 0x7ff1a419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a419e530 con 0x7ff1a41028c0 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a419e6a0 con 0x7ff1a4101fa0 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1ac600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a419e860 con 0x7ff1a410b3c0 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1a9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1a9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48266/0 (socket says 172.21.15.46:48266) 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1a9c00700 1 -- 172.21.15.46:0/2182774963 learned_addr learned my addr 172.21.15.46:0/2182774963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.790+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 0x7ff1a4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 0x7ff1a419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 -- 172.21.15.46:0/2182774963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 msgr2=0x7ff1a419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 0x7ff1a419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 -- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 msgr2=0x7ff1a4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 0x7ff1a4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 -- 172.21.15.46:0/2182774963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff198009520 con 0x7ff1a4101fa0 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 0x7ff1a4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 0x7ff1a419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff19800eb30 tx=0x7ff19800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.791+0000 7ff1a2000700 1 -- 172.21.15.46:0/2182774963 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198009880 con 0x7ff1a4101fa0 2024-05-08T23:17:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.792+0000 7ff1a2000700 1 -- 172.21.15.46:0/2182774963 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1980044e0 con 0x7ff1a4101fa0 2024-05-08T23:17:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.792+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c009d00 con 0x7ff1a4101fa0 2024-05-08T23:17:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.792+0000 7ff1a2000700 1 -- 172.21.15.46:0/2182774963 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198003d80 con 0x7ff1a4101fa0 2024-05-08T23:17:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.792+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41a61a0 con 0x7ff1a4101fa0 2024-05-08T23:17:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.795+0000 7ff1a2000700 1 -- 172.21.15.46:0/2182774963 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff198003f20 con 0x7ff1a4101fa0 2024-05-08T23:17:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.796+0000 7ff1a2000700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 0x7ff18c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 Processor -- start 2024-05-08T23:17:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1a2000700 1 -- 172.21.15.46:0/2182774963 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff198014070 con 0x7ff1a4101fa0 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 0x7ff18c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 -- start start 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e249150 con 0x559c1e2f8720 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e246b10 con 0x559c1e2f7e00 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.797+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e242400 con 0x559c1e2f2890 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40772/0 (socket says 172.21.15.46:40772) 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 -- 172.21.15.46:0/2428125962 learned_addr learned my addr 172.21.15.46:0/2428125962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 0x7ff18c0a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff1a419ab60 tx=0x7ff190009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 -- 172.21.15.46:0/2428125962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 msgr2=0x559c1e2f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 -- 172.21.15.46:0/2428125962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 msgr2=0x559c1e2f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 -- 172.21.15.46:0/2428125962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c1e0c33b0 con 0x559c1e2f2890 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.798+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff19c015000 tx=0x7ff19c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a681c1b68b9631 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.799+0000 7ff196a00700 1 -- 172.21.15.46:0/2428125962 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00e040 con 0x559c1e2f2890 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.799+0000 7ff196a00700 1 -- 172.21.15.46:0/2428125962 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c004a30 con 0x559c1e2f2890 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.799+0000 7ff196a00700 1 -- 172.21.15.46:0/2428125962 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0052a0 con 0x559c1e2f2890 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.799+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2428125962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 msgr2=0x559c1e2f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.799+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff19c015000 tx=0x7ff19c009b80 comp rx=0 tx=0).stop 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2428125962 shutdown_connections 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2cd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2428125962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2428125962 >> 172.21.15.46:0/2428125962 conn(0x559c1e250210 msgr2=0x559c1e2ecdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2428125962 shutdown_connections 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2428125962 wait complete. 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.800+0000 7ff1c0156b80 1 -- start start 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 Processor -- start 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e2fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f8720 0x559c1e2fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e1de560 con 0x559c1e2f2890 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e1de200 con 0x559c1e2f8720 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e0c56a0 con 0x559c1e2f7e00 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1c0156b80 10 client.0 _ll_register_callbacks cb 0x559c1e1e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40782/0 (socket says 172.21.15.46:40782) 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1a9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e2fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.801+0000 7ff1a9200700 1 -- 172.21.15.46:0/2717005054 learned_addr learned my addr 172.21.15.46:0/2717005054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1aa600700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f8720 0x559c1e2fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 -- 172.21.15.46:0/2717005054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f8720 msgr2=0x559c1e2fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f8720 0x559c1e2fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 -- 172.21.15.46:0/2717005054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 msgr2=0x559c1e2fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e2fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 -- 172.21.15.46:0/2717005054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41a61a0 con 0x559c1e2f7e00 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1a9200700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff1900039b0 tx=0x7ff1900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e84ba7ffe03001 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff18b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000ea40 con 0x559c1e2f7e00 2024-05-08T23:17:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c1e19b270 con 0x559c1e2f7e00 2024-05-08T23:17:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.802+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff19000eba0 con 0x559c1e2f7e00 2024-05-08T23:17:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19000ed60 con 0x559c1e2f7e00 2024-05-08T23:17:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 10 client.29230 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff1a410e540 con 0x559c1e2f7e00 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c1e19b270 con 0x559c1e2f7e00 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff19001f400 con 0x559c1e2f7e00 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 10 client.29230 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 10 client.29230 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 20 client.29230 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e30c0f0 0x559c1e30e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 4 client.29230 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.803+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c1e2c1810 con 0x559c1e30c0f0 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.804+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e30c0f0 0x559c1e30e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.804+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e30c0f0 0x559c1e30e510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3984555e57f0c71d server_cookie=3961fbcdba2e751 in_seq=0 out_seq=0 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.804+0000 7ff18b400700 10 client.29230 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff18b400700 1 -- 172.21.15.46:0/2717005054 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c1e2c1810 con 0x559c1e30c0f0 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff18b400700 10 client.29230 handle_command_reply: tid=0 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 client.29230 shutdown 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 msgr2=0x559c1e2fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff1900039b0 tx=0x7ff1900047c0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 shutdown_connections 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e30c0f0 0x559c1e30e510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f8720 0x559c1e2fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e2fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/2717005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f7e00 0x559c1e2fbfb0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff18b400700 0 client.29230 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 >> 172.21.15.46:0/2717005054 conn(0x559c1e250210 msgr2=0x559c1e2ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 shutdown_connections 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 1 -- 172.21.15.46:0/2717005054 wait complete. 2024-05-08T23:17:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.805+0000 7ff1c0156b80 20 client.29230 trim_cache size 0 max 16384 2024-05-08T23:17:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 Processor -- start 2024-05-08T23:17:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 -- start start 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e313720 con 0x559c1e2f8720 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e2ede80 con 0x559c1e2f7e00 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.821+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e309880 con 0x559c1e2f2890 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48290/0 (socket says 172.21.15.46:48290) 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 -- 172.21.15.46:0/62591254 learned_addr learned my addr 172.21.15.46:0/62591254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1aa600700 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 -- 172.21.15.46:0/62591254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 msgr2=0x559c1e2f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 -- 172.21.15.46:0/62591254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 msgr2=0x559c1e2fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9200700 1 -- 172.21.15.46:0/62591254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c1e06fb60 con 0x559c1e2f7e00 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.822+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.823+0000 7ff1a9200700 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff19001f790 tx=0x7ff190005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafe759623bc7bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.823+0000 7ff197400700 1 -- 172.21.15.46:0/62591254 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19001fa80 con 0x559c1e2f7e00 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.823+0000 7ff197400700 1 -- 172.21.15.46:0/62591254 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19001f400 con 0x559c1e2f7e00 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.823+0000 7ff197400700 1 -- 172.21.15.46:0/62591254 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190028830 con 0x559c1e2f7e00 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- 172.21.15.46:0/62591254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 msgr2=0x559c1e2ecd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff19001f790 tx=0x7ff190005a00 comp rx=0 tx=0).stop 2024-05-08T23:17:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- 172.21.15.46:0/62591254 shutdown_connections 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f8720 0x559c1e2fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f2890 0x559c1e2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/62591254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2ecd20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- 172.21.15.46:0/62591254 >> 172.21.15.46:0/62591254 conn(0x559c1e250210 msgr2=0x559c1e2fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- 172.21.15.46:0/62591254 shutdown_connections 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- 172.21.15.46:0/62591254 wait complete. 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 -- start start 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.824+0000 7ff1c0156b80 1 Processor -- start 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e303300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e3039e0 con 0x559c1e2f2890 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e2f49d0 con 0x559c1e2f7e00 2024-05-08T23:17:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1e2f4b40 con 0x559c1e2f8720 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1c0156b80 10 client.0 _ll_register_callbacks cb 0x559c1e1e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1aa600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1a9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e303300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1aa600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40806/0 (socket says 172.21.15.46:40806) 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1aa600700 1 -- 172.21.15.46:0/3961706320 learned_addr learned my addr 172.21.15.46:0/3961706320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.825+0000 7ff1a9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 -- 172.21.15.46:0/3961706320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 msgr2=0x559c1e2f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 -- 172.21.15.46:0/3961706320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 msgr2=0x559c1e303300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e303300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 -- 172.21.15.46:0/3961706320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c009d00 con 0x559c1e2f8720 2024-05-08T23:17:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1aa600700 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff19c015000 tx=0x7ff19c00b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a4abb42378c3ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff18b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00e040 con 0x559c1e2f8720 2024-05-08T23:17:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff19c0074a0 con 0x559c1e2f8720 2024-05-08T23:17:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.826+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff19c004a30 con 0x559c1e2f8720 2024-05-08T23:17:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c007660 con 0x559c1e2f8720 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 10 client.29236 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff19002d040 con 0x559c1e2f8720 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c004a30 con 0x559c1e2f8720 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff19c024030 con 0x559c1e2f8720 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 10 client.29236 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 10 client.29236 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 20 client.29236 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e363680 0x559c1e365aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 4 client.29236 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c1e23aec0 con 0x559c1e363680 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.827+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e363680 0x559c1e365aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1a9c00700 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e363680 0x559c1e365aa0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18277a1b2ee7c62 server_cookie=ac025e6d6f0411c8 in_seq=0 out_seq=0 2024-05-08T23:17:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff18b400700 10 client.29236 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff18b400700 1 -- 172.21.15.46:0/3961706320 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c1e23aec0 con 0x559c1e363680 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff18b400700 10 client.29236 handle_command_reply: tid=0 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 client.29236 shutdown 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 msgr2=0x559c1e2f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff19c015000 tx=0x7ff19c00b700 comp rx=0 tx=0).stop 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 shutdown_connections 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c1e363680 0x559c1e365aa0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c1e2f8720 0x559c1e2f4490 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c1e2f2890 0x559c1e303300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 --2- 172.21.15.46:0/3961706320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c1e2f7e00 0x559c1e2f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff18b400700 0 client.29236 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 >> 172.21.15.46:0/3961706320 conn(0x559c1e250210 msgr2=0x559c1e2f6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.828+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 shutdown_connections 2024-05-08T23:17:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1c0156b80 1 -- 172.21.15.46:0/3961706320 wait complete. 2024-05-08T23:17:02.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1c0156b80 20 client.29236 trim_cache size 0 max 16384 2024-05-08T23:17:02.834 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 msgr2=0x7ff18c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 0x7ff18c0a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff1a419ab60 tx=0x7ff190009260 comp rx=0 tx=0).stop 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 msgr2=0x7ff1a4199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff19800eb30 tx=0x7ff19800ee40 comp rx=0 tx=0).stop 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 shutdown_connections 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a410b3c0 0x7ff1a419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1a4101fa0 0x7ff1a4199610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff18c0a1290 0x7ff18c0a3750 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.829+0000 7ff1ac600700 1 --2- 172.21.15.46:0/2182774963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a41028c0 0x7ff1a4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.830+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 >> 172.21.15.46:0/2182774963 conn(0x7ff1a40fb760 msgr2=0x7ff1a410c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.830+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 shutdown_connections 2024-05-08T23:17:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:02.830+0000 7ff1ac600700 1 -- 172.21.15.46:0/2182774963 wait complete. 2024-05-08T23:17:02.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:17:02.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1154045 9.07 MB/sec execute 564 sec latency 19.984 ms 2024-05-08T23:17:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.079+0000 7fb1a2200700 1 -- 172.21.15.46:0/1616766604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 msgr2=0x7fb19c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.079+0000 7fb1a2200700 1 --2- 172.21.15.46:0/1616766604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c10dd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb194009f90 tx=0x7fb19400b330 comp rx=0 tx=0).stop 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.079+0000 7fb1a2200700 1 -- 172.21.15.46:0/1616766604 shutdown_connections 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 --2- 172.21.15.46:0/1616766604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c10dd20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 --2- 172.21.15.46:0/1616766604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 --2- 172.21.15.46:0/1616766604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c101a30 0x7fb19c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 -- 172.21.15.46:0/1616766604 >> 172.21.15.46:0/1616766604 conn(0x7fb19c0fc8e0 msgr2=0x7fb19c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 -- 172.21.15.46:0/1616766604 shutdown_connections 2024-05-08T23:17:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.080+0000 7fb1a2200700 1 -- 172.21.15.46:0/1616766604 wait complete. 2024-05-08T23:17:03.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.081+0000 7fb1a2200700 1 Processor -- start 2024-05-08T23:17:03.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.081+0000 7fb1a2200700 1 -- start start 2024-05-08T23:17:03.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 0x7fb19c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c19e9c0 con 0x7fb19c101080 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c19eb30 con 0x7fb19c19df40 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb1a2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c19ecf0 con 0x7fb19c1061a0 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45486/0 (socket says 172.21.15.46:45486) 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40826/0 (socket says 172.21.15.46:40826) 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19b400700 1 -- 172.21.15.46:0/2616449104 learned_addr learned my addr 172.21.15.46:0/2616449104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.082+0000 7fb19be00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 0x7fb19c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 -- 172.21.15.46:0/2616449104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 msgr2=0x7fb19c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 -- 172.21.15.46:0/2616449104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 msgr2=0x7fb19c19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 0x7fb19c19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 -- 172.21.15.46:0/2616449104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb194009c40 con 0x7fb19c101080 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19aa00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19be00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 0x7fb19c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb19b400700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9a0 tx=0x7fb18c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb193e00700 1 -- 172.21.15.46:0/2616449104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c009790 con 0x7fb19c101080 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb193e00700 1 -- 172.21.15.46:0/2616449104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c00ce80 con 0x7fb19c101080 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c1a6250 con 0x7fb19c101080 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb193e00700 1 -- 172.21.15.46:0/2616449104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c018680 con 0x7fb19c101080 2024-05-08T23:17:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.083+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1a6730 con 0x7fb19c101080 2024-05-08T23:17:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.088+0000 7fb193e00700 1 -- 172.21.15.46:0/2616449104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb18c010430 con 0x7fb19c101080 2024-05-08T23:17:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.089+0000 7fb1b5e0eb80 1 Processor -- start 2024-05-08T23:17:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.089+0000 7fb1b5e0eb80 1 -- start start 2024-05-08T23:17:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb193e00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 0x7fb1800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1ea4f150 con 0x558d1eaf8890 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb193e00700 1 -- 172.21.15.46:0/2616449104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb18c014070 con 0x7fb19c101080 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1ea4cb10 con 0x558d1eafe720 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1ea48400 con 0x558d1eafde00 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19be00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 0x7fb1800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45488/0 (socket says 172.21.15.46:45488) 2024-05-08T23:17:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19be00700 1 -- 172.21.15.46:0/2180673705 learned_addr learned my addr 172.21.15.46:0/2180673705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.090+0000 7fb19b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40842/0 (socket says 172.21.15.46:40842) 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 -- 172.21.15.46:0/2180673705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 msgr2=0x558d1eaf8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 -- 172.21.15.46:0/2180673705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 msgr2=0x558d1eaf8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 -- 172.21.15.46:0/2180673705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d1e8c93b0 con 0x558d1eafde00 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19b400700 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ca5e0 tx=0x7fb18c099c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18dae45332461d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb19be00700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 0x7fb1800a3750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb1940079a0 tx=0x7fb194009360 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb187e00700 1 -- 172.21.15.46:0/2180673705 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0493c0 con 0x558d1eafde00 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb187e00700 1 -- 172.21.15.46:0/2180673705 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c00cce0 con 0x558d1eafde00 2024-05-08T23:17:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.091+0000 7fb187e00700 1 -- 172.21.15.46:0/2180673705 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c049b00 con 0x558d1eafde00 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2180673705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 msgr2=0x558d1eafe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0ca5e0 tx=0x7fb18c099c10 comp rx=0 tx=0).stop 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2180673705 shutdown_connections 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eafe1e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2180673705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2180673705 >> 172.21.15.46:0/2180673705 conn(0x558d1ea56210 msgr2=0x558d1eaf2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2180673705 shutdown_connections 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.092+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2180673705 wait complete. 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.093+0000 7fb1b5e0eb80 1 -- start start 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.093+0000 7fb1b5e0eb80 1 Processor -- start 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb04790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eb01af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1e9e4560 con 0x558d1eaf8890 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1e9e4200 con 0x558d1eafde00 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1e8cb6a0 con 0x558d1eafe720 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb1b5e0eb80 10 client.0 _ll_register_callbacks cb 0x558d1e9edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb04790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40848/0 (socket says 172.21.15.46:40848) 2024-05-08T23:17:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 -- 172.21.15.46:0/2298452554 learned_addr learned my addr 172.21.15.46:0/2298452554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19aa00700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eb01af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 -- 172.21.15.46:0/2298452554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 msgr2=0x558d1eb01af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eb01af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 -- 172.21.15.46:0/2298452554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 msgr2=0x558d1eb04790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb04790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.094+0000 7fb19be00700 1 -- 172.21.15.46:0/2298452554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1a6730 con 0x558d1eafe720 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb19be00700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb1940093d0 tx=0x7fb19400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4b86d427d78de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb184c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400e040 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d1e9a1270 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb19401c880 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb19401ca40 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb1b5e0eb80 10 client.29248 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.095+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d1e875b60 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d1e9a1270 con 0x558d1eafe720 2024-05-08T23:17:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb19402a070 con 0x558d1eafe720 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 10 client.29248 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 10 client.29248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 20 client.29248 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb12150 0x558d1eb14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 4 client.29248 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d1eac7810 con 0x558d1eb12150 2024-05-08T23:17:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.096+0000 7fb19b400700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb12150 0x558d1eb14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.097+0000 7fb184c00700 10 client.29248 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.097+0000 7fb19b400700 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb12150 0x558d1eb14570 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d22ff7e74dbc8782 server_cookie=6fee9982f38823ed in_seq=0 out_seq=0 2024-05-08T23:17:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.098+0000 7fb184c00700 1 -- 172.21.15.46:0/2298452554 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d1eac7810 con 0x558d1eb12150 2024-05-08T23:17:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.098+0000 7fb184c00700 10 client.29248 handle_command_reply: tid=0 2024-05-08T23:17:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.098+0000 7fb1b5e0eb80 1 client.29248 shutdown 2024-05-08T23:17:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 msgr2=0x558d1eb02230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb1940093d0 tx=0x7fb19400b330 comp rx=0 tx=0).stop 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 shutdown_connections 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eb02230 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb12150 0x558d1eb14570 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb04790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/2298452554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eb01af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 >> 172.21.15.46:0/2298452554 conn(0x558d1ea56210 msgr2=0x558d1eaf25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb184c00700 0 client.29248 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 shutdown_connections 2024-05-08T23:17:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/2298452554 wait complete. 2024-05-08T23:17:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.099+0000 7fb1b5e0eb80 20 client.29248 trim_cache size 0 max 16384 2024-05-08T23:17:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.116+0000 7fb1b5e0eb80 1 Processor -- start 2024-05-08T23:17:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 -- start start 2024-05-08T23:17:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eb14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eaf91b0 con 0x558d1eaf8890 2024-05-08T23:17:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eb0dc20 con 0x558d1eafe720 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eb0dde0 con 0x558d1eafde00 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb19aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eb14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb19b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb19aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eb14540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40854/0 (socket says 172.21.15.46:40854) 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb19aa00700 1 -- 172.21.15.46:0/175095244 learned_addr learned my addr 172.21.15.46:0/175095244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.117+0000 7fb19be00700 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 -- 172.21.15.46:0/175095244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 msgr2=0x558d1eb14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eb14540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 -- 172.21.15.46:0/175095244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 msgr2=0x558d1eaf8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 -- 172.21.15.46:0/175095244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d1ea0e560 con 0x558d1eafe720 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb19be00700 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8c70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb194009040 tx=0x7fb194004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5ca0f139228977 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb191200700 1 -- 172.21.15.46:0/175095244 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400e070 con 0x558d1eafe720 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb191200700 1 -- 172.21.15.46:0/175095244 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb194027040 con 0x558d1eafe720 2024-05-08T23:17:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.118+0000 7fb191200700 1 -- 172.21.15.46:0/175095244 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194020820 con 0x558d1eafe720 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/175095244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 msgr2=0x558d1eaf8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8c70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb194009040 tx=0x7fb194004a90 comp rx=0 tx=0).stop 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/175095244 shutdown_connections 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafe720 0x558d1eaf8c70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eaf8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/175095244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafde00 0x558d1eb14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/175095244 >> 172.21.15.46:0/175095244 conn(0x558d1ea56210 msgr2=0x558d1eaef210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/175095244 shutdown_connections 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/175095244 wait complete. 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.119+0000 7fb1b5e0eb80 1 -- start start 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 Processor -- start 2024-05-08T23:17:03.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eafad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eafb660 con 0x558d1eaf8890 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eafb7d0 con 0x558d1eafde00 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1eaf9ee0 con 0x558d1eafe720 2024-05-08T23:17:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.120+0000 7fb1b5e0eb80 10 client.0 _ll_register_callbacks cb 0x558d1e9edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40862/0 (socket says 172.21.15.46:40862) 2024-05-08T23:17:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 -- 172.21.15.46:0/1554361605 learned_addr learned my addr 172.21.15.46:0/1554361605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19aa00700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eafad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 -- 172.21.15.46:0/1554361605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 msgr2=0x558d1eafad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eafad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 -- 172.21.15.46:0/1554361605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 msgr2=0x558d1eb096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 -- 172.21.15.46:0/1554361605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d1e875b60 con 0x558d1eafe720 2024-05-08T23:17:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19aa00700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eafad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb19be00700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb194009960 tx=0x7fb194007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ce74690142c806 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.121+0000 7fb184c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400e040 con 0x558d1eafe720 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb18c00cce0 con 0x558d1eafe720 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb19400b680 con 0x558d1eafe720 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb194020860 con 0x558d1eafe720 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 10 client.29254 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d1e8cdcd0 con 0x558d1eafe720 2024-05-08T23:17:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c00cce0 con 0x558d1eafe720 2024-05-08T23:17:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb194035030 con 0x558d1eafe720 2024-05-08T23:17:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 10 client.29254 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 10 client.29254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:17:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 20 client.29254 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb68ec0 0x558d1eb6b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 4 client.29254 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.122+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d1ea40ec0 con 0x558d1eb68ec0 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.123+0000 7fb19b400700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb68ec0 0x558d1eb6b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.123+0000 7fb19b400700 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb68ec0 0x558d1eb6b2b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e3d44f457ffcf51 server_cookie=d367e58d46832cbe in_seq=0 out_seq=0 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.123+0000 7fb184c00700 10 client.29254 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb184c00700 1 -- 172.21.15.46:0/1554361605 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d1ea40ec0 con 0x558d1eb68ec0 2024-05-08T23:17:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb184c00700 10 client.29254 handle_command_reply: tid=0 2024-05-08T23:17:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 client.29254 shutdown 2024-05-08T23:17:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 msgr2=0x558d1eaf99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb194009960 tx=0x7fb194007d30 comp rx=0 tx=0).stop 2024-05-08T23:17:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 shutdown_connections 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d1eafe720 0x558d1eaf99a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558d1eb68ec0 0x558d1eb6b2b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d1eaf8890 0x558d1eb096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 --2- 172.21.15.46:0/1554361605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d1eafde00 0x558d1eafad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb184c00700 0 client.29254 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 >> 172.21.15.46:0/1554361605 conn(0x558d1ea56210 msgr2=0x558d1eafbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 shutdown_connections 2024-05-08T23:17:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 1 -- 172.21.15.46:0/1554361605 wait complete. 2024-05-08T23:17:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.124+0000 7fb1b5e0eb80 20 client.29254 trim_cache size 0 max 16384 2024-05-08T23:17:03.141 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:03.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 msgr2=0x7fb1800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 0x7fb1800a3750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb1940079a0 tx=0x7fb194009360 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 msgr2=0x7fb19c199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9a0 tx=0x7fb18c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 shutdown_connections 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c19df40 0x7fb19c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb1800a1290 0x7fb1800a3750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19c101080 0x7fb19c199660 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 --2- 172.21.15.46:0/2616449104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb19c1061a0 0x7fb19c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 >> 172.21.15.46:0/2616449104 conn(0x7fb19c0fc8e0 msgr2=0x7fb19c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 shutdown_connections 2024-05-08T23:17:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.125+0000 7fb1a2200700 1 -- 172.21.15.46:0/2616449104 wait complete. 2024-05-08T23:17:03.144 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:02 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2221881369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:03.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:17:03.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:02 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2221881369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:03.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:02 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2221881369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 -- 172.21.15.46:0/3306999995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c0101060 msgr2=0x7f54c0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 --2- 172.21.15.46:0/3306999995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c0101060 0x7f54c0101440 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f54b40098f0 tx=0x7f54b4009c00 comp rx=0 tx=0).stop 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 -- 172.21.15.46:0/3306999995 shutdown_connections 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 --2- 172.21.15.46:0/3306999995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 --2- 172.21.15.46:0/3306999995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c0101060 0x7f54c0101440 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 --2- 172.21.15.46:0/3306999995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101a10 0x7f54c0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 -- 172.21.15.46:0/3306999995 >> 172.21.15.46:0/3306999995 conn(0x7f54c00fc900 msgr2=0x7f54c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 -- 172.21.15.46:0/3306999995 shutdown_connections 2024-05-08T23:17:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.375+0000 7f54c8400700 1 -- 172.21.15.46:0/3306999995 wait complete. 2024-05-08T23:17:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.376+0000 7f54c8400700 1 Processor -- start 2024-05-08T23:17:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.376+0000 7f54c8400700 1 -- start start 2024-05-08T23:17:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c019e920 con 0x7f54c0101060 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c019ea90 con 0x7f54c0106180 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c019ec50 con 0x7f54c019dea0 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c0199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48364/0 (socket says 172.21.15.46:48364) 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45544/0 (socket says 172.21.15.46:45544) 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5000700 1 -- 172.21.15.46:0/3074576000 learned_addr learned my addr 172.21.15.46:0/3074576000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40890/0 (socket says 172.21.15.46:40890) 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 -- 172.21.15.46:0/3074576000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 msgr2=0x7f54c0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 -- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 msgr2=0x7f54c01995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c6400700 1 -- 172.21.15.46:0/3074576000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4009520 con 0x7f54c019dea0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.377+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54c6400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f54b800efc0 tx=0x7f54b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54be000700 1 -- 172.21.15.46:0/3074576000 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8018070 con 0x7f54c019dea0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54be000700 1 -- 172.21.15.46:0/3074576000 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b800f040 con 0x7f54c019dea0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54be000700 1 -- 172.21.15.46:0/3074576000 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b800ccb0 con 0x7f54c019dea0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c01a6150 con 0x7f54c019dea0 2024-05-08T23:17:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.378+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c01a65e0 con 0x7f54c019dea0 2024-05-08T23:17:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.381+0000 7f54be000700 1 -- 172.21.15.46:0/3074576000 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f54b8003a00 con 0x7f54c019dea0 2024-05-08T23:17:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54be000700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 0x7f54a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54be000700 1 -- 172.21.15.46:0/3074576000 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f54b80ccad0 con 0x7f54c019dea0 2024-05-08T23:17:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54dbfa3b80 1 Processor -- start 2024-05-08T23:17:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 0x7f54a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54dbfa3b80 1 -- start start 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.383+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a75e00 0x55a8e8a761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a70890 0x55a8e8a70cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e89c7150 con 0x55a8e8a70890 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e89c4b10 con 0x55a8e8a76720 2024-05-08T23:17:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e89c0400 con 0x55a8e8a75e00 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 0x7f54a80a37d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fd0 tx=0x7f54b4019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a75e00 0x55a8e8a761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48374/0 (socket says 172.21.15.46:48374) 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 -- 172.21.15.46:0/1995028818 learned_addr learned my addr 172.21.15.46:0/1995028818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c6400700 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a70890 0x55a8e8a70cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 -- 172.21.15.46:0/1995028818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a75e00 msgr2=0x55a8e8a761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a75e00 0x55a8e8a761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 -- 172.21.15.46:0/1995028818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a70890 msgr2=0x55a8e8a70cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a70890 0x55a8e8a70cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.384+0000 7f54c5000700 1 -- 172.21.15.46:0/1995028818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e88413b0 con 0x55a8e8a76720 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.385+0000 7f54c5000700 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009d90 tx=0x7f54ac011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f423567ea79105f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.385+0000 7f54b2a00700 1 -- 172.21.15.46:0/1995028818 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac019630 con 0x55a8e8a76720 2024-05-08T23:17:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.385+0000 7f54b2a00700 1 -- 172.21.15.46:0/1995028818 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac019c50 con 0x55a8e8a76720 2024-05-08T23:17:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.385+0000 7f54b2a00700 1 -- 172.21.15.46:0/1995028818 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00ebe0 con 0x55a8e8a76720 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/1995028818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 msgr2=0x55a8e8a70350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009d90 tx=0x7f54ac011720 comp rx=0 tx=0).stop 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/1995028818 shutdown_connections 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a70890 0x55a8e8a70cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a75e00 0x55a8e8a761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/1995028818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a70350 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/1995028818 >> 172.21.15.46:0/1995028818 conn(0x55a8e89ce210 msgr2=0x55a8e8a6adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/1995028818 shutdown_connections 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/1995028818 wait complete. 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.386+0000 7f54dbfa3b80 1 -- start start 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 Processor -- start 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 0x55a8e8a79b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e895c560 con 0x55a8e8a75e00 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e895c200 con 0x55a8e8a70890 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e88436a0 con 0x55a8e8a76720 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.387+0000 7f54dbfa3b80 10 client.0 _ll_register_callbacks cb 0x55a8e8965b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c5000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40916/0 (socket says 172.21.15.46:40916) 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 -- 172.21.15.46:0/2747993890 learned_addr learned my addr 172.21.15.46:0/2747993890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c5a00700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 0x55a8e8a79b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 -- 172.21.15.46:0/2747993890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 msgr2=0x55a8e8a79b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 0x55a8e8a79b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 -- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 msgr2=0x55a8e8a7a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 -- 172.21.15.46:0/2747993890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac0099f0 con 0x55a8e8a76720 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c5a00700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 0x55a8e8a79b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54c6400700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f54b8099790 tx=0x7f54b8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a824af9660777eab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.388+0000 7f54a7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b809d630 con 0x55a8e8a76720 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f54b80a1bf0 con 0x55a8e8a76720 2024-05-08T23:17:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8e8919270 con 0x55a8e8a76720 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b800ce10 con 0x55a8e8a76720 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 10 client.29266 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a8e87edb60 con 0x55a8e8a76720 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b80a5020 con 0x55a8e8a76720 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54b8098cf0 con 0x55a8e8a76720 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 10 client.29266 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 10 client.29266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 20 client.29266 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8a8a2f0 0x55a8e8a8c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 4 client.29266 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.389+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8e8a3f810 con 0x55a8e8a8a2f0 2024-05-08T23:17:03.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.390+0000 7f54c5a00700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8a8a2f0 0x55a8e8a8c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.390+0000 7f54c5a00700 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8a8a2f0 0x55a8e8a8c6e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3ff4ef7ad3a8ec server_cookie=60d103b2f15114a6 in_seq=0 out_seq=0 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.390+0000 7f54a7400700 10 client.29266 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54a7400700 1 -- 172.21.15.46:0/2747993890 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8e8a3f810 con 0x55a8e8a8a2f0 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54a7400700 10 client.29266 handle_command_reply: tid=0 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 client.29266 shutdown 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 msgr2=0x55a8e8a7a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f54b8099790 tx=0x7f54b8009f90 comp rx=0 tx=0).stop 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 shutdown_connections 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a76720 0x55a8e8a7a5f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a70890 0x55a8e8a79b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8a8a2f0 0x55a8e8a8c6e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/2747993890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 >> 172.21.15.46:0/2747993890 conn(0x55a8e89ce210 msgr2=0x55a8e8a6a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54a7400700 0 client.29266 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 shutdown_connections 2024-05-08T23:17:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.391+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/2747993890 wait complete. 2024-05-08T23:17:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.392+0000 7f54dbfa3b80 20 client.29266 trim_cache size 0 max 16384 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 Processor -- start 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 -- start start 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a85c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8a6c870 con 0x55a8e8a75e00 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8a7b1e0 con 0x55a8e8a76720 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8a71230 con 0x55a8e8a70890 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40930/0 (socket says 172.21.15.46:40930) 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.407+0000 7f54c5a00700 1 -- 172.21.15.46:0/808778284 learned_addr learned my addr 172.21.15.46:0/808778284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c6400700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5000700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a85c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 -- 172.21.15.46:0/808778284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 msgr2=0x55a8e8a6ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a6ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 -- 172.21.15.46:0/808778284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 msgr2=0x55a8e8a85c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a85c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 -- 172.21.15.46:0/808778284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e8986560 con 0x55a8e8a70890 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5000700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a85c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54c5a00700 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fa0 tx=0x7f54b401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a2534c3f8d40dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54b3400700 1 -- 172.21.15.46:0/808778284 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400bba0 con 0x55a8e8a70890 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54b3400700 1 -- 172.21.15.46:0/808778284 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b401d040 con 0x55a8e8a70890 2024-05-08T23:17:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.408+0000 7f54b3400700 1 -- 172.21.15.46:0/808778284 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003e00 con 0x55a8e8a70890 2024-05-08T23:17:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/808778284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 msgr2=0x55a8e8a86310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fa0 tx=0x7f54b401f000 comp rx=0 tx=0).stop 2024-05-08T23:17:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/808778284 shutdown_connections 2024-05-08T23:17:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a6ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a86310 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/808778284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a85c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/808778284 >> 172.21.15.46:0/808778284 conn(0x55a8e89ce210 msgr2=0x55a8e8a67210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/808778284 shutdown_connections 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.409+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/808778284 wait complete. 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.410+0000 7f54dbfa3b80 1 -- start start 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.410+0000 7f54dbfa3b80 1 Processor -- start 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8adf8f0 con 0x55a8e8a75e00 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8adfa60 con 0x55a8e8a76720 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e8adfc20 con 0x55a8e8a70890 2024-05-08T23:17:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54dbfa3b80 10 client.0 _ll_register_callbacks cb 0x55a8e8965b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40942/0 (socket says 172.21.15.46:40942) 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 -- 172.21.15.46:0/3290130182 learned_addr learned my addr 172.21.15.46:0/3290130182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c6400700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5000700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 -- 172.21.15.46:0/3290130182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 msgr2=0x55a8e8a7fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a7fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 -- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 msgr2=0x55a8e8a7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5a00700 1 -- 172.21.15.46:0/3290130182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4028040 con 0x55a8e8a70890 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.411+0000 7f54c5000700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54c5a00700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009c40 tx=0x7f54b4009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3bc793806860b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54a7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400bba0 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54ac019c50 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f54b4003a80 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b4003c40 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54dbfa3b80 10 client.29272 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a8e8845cd0 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.412+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac019c50 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54b4037070 con 0x55a8e8a70890 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 10 client.29272 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 10 client.29272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 20 client.29272 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8ae0fa0 0x55a8e8ae33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 4 client.29272 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a8e89b8ec0 con 0x55a8e8ae0fa0 2024-05-08T23:17:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.413+0000 7f54c5000700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8ae0fa0 0x55a8e8ae33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54c5000700 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8ae0fa0 0x55a8e8ae33c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a8d9469dfdfe400 server_cookie=758087163c4950c7 in_seq=0 out_seq=0 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54a7400700 10 client.29272 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54a7400700 1 -- 172.21.15.46:0/3290130182 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a8e89b8ec0 con 0x55a8e8ae0fa0 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54a7400700 10 client.29272 handle_command_reply: tid=0 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 client.29272 shutdown 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 msgr2=0x55a8e8a7f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009c40 tx=0x7f54b4009870 comp rx=0 tx=0).stop 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 shutdown_connections 2024-05-08T23:17:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8e8a76720 0x55a8e8a7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e8a70890 0x55a8e8a7f150 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55a8e8ae0fa0 0x55a8e8ae33c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 --2- 172.21.15.46:0/3290130182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8e8a75e00 0x55a8e8a7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 >> 172.21.15.46:0/3290130182 conn(0x55a8e89ce210 msgr2=0x55a8e8a782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54a7400700 0 client.29272 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.414+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 shutdown_connections 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54dbfa3b80 1 -- 172.21.15.46:0/3290130182 wait complete. 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54dbfa3b80 20 client.29272 trim_cache size 0 max 16384 2024-05-08T23:17:03.419 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 msgr2=0x7f54a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 0x7f54a80a37d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fd0 tx=0x7f54b4019040 comp rx=0 tx=0).stop 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 msgr2=0x7f54c019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f54b800efc0 tx=0x7f54b800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 shutdown_connections 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54c019dea0 0x7f54c019e320 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f54a80a1310 0x7f54a80a37d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54c0101060 0x7f54c01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 --2- 172.21.15.46:0/3074576000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54c0106180 0x7f54c0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.415+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 >> 172.21.15.46:0/3074576000 conn(0x7f54c00fc900 msgr2=0x7f54c00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.416+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 shutdown_connections 2024-05-08T23:17:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.416+0000 7f54c8400700 1 -- 172.21.15.46:0/3074576000 wait complete. 2024-05-08T23:17:03.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:17:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.665+0000 7f2c19c00700 1 -- 172.21.15.46:0/1332551246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c141028c0 msgr2=0x7f2c1410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.665+0000 7f2c19c00700 1 --2- 172.21.15.46:0/1332551246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c141028c0 0x7f2c1410ada0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009990 tx=0x7f2c00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 -- 172.21.15.46:0/1332551246 shutdown_connections 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 --2- 172.21.15.46:0/1332551246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1410b3c0 0x7f2c1410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 --2- 172.21.15.46:0/1332551246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c14101fa0 0x7f2c14102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 --2- 172.21.15.46:0/1332551246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c141028c0 0x7f2c1410ada0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 -- 172.21.15.46:0/1332551246 >> 172.21.15.46:0/1332551246 conn(0x7f2c140fb760 msgr2=0x7f2c140fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 -- 172.21.15.46:0/1332551246 shutdown_connections 2024-05-08T23:17:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.666+0000 7f2c19c00700 1 -- 172.21.15.46:0/1332551246 wait complete. 2024-05-08T23:17:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 Processor -- start 2024-05-08T23:17:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 -- start start 2024-05-08T23:17:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 0x7f2c1419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14195f50 con 0x7f2c141028c0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141960c0 con 0x7f2c14101fa0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.667+0000 7f2c19c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14196280 con 0x7f2c1410b3c0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48416/0 (socket says 172.21.15.46:48416) 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40966/0 (socket says 172.21.15.46:40966) 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12a00700 1 -- 172.21.15.46:0/3893541287 learned_addr learned my addr 172.21.15.46:0/3893541287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12000700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 0x7f2c1419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 -- 172.21.15.46:0/3893541287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 msgr2=0x7f2c14199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 -- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 msgr2=0x7f2c1419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 0x7f2c1419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 -- 172.21.15.46:0/3893541287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009520 con 0x7f2c1410b3c0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12a00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c12000700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 0x7f2c1419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.668+0000 7f2c13400700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00efc0 tx=0x7f2c0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.669+0000 7f2c0b400700 1 -- 172.21.15.46:0/3893541287 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c018070 con 0x7f2c1410b3c0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.669+0000 7f2c0b400700 1 -- 172.21.15.46:0/3893541287 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c00f040 con 0x7f2c1410b3c0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.669+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c000095f0 con 0x7f2c1410b3c0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.669+0000 7f2c0b400700 1 -- 172.21.15.46:0/3893541287 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00cd40 con 0x7f2c1410b3c0 2024-05-08T23:17:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.669+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141a1e00 con 0x7f2c1410b3c0 2024-05-08T23:17:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.672+0000 7f2c0b400700 1 -- 172.21.15.46:0/3893541287 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2c0c003a00 con 0x7f2c1410b3c0 2024-05-08T23:17:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.673+0000 7f2c2d77fb80 1 Processor -- start 2024-05-08T23:17:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c0b400700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 0x7f2bf80a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 -- start start 2024-05-08T23:17:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c0b400700 1 -- 172.21.15.46:0/3893541287 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2c0c0ccc60 con 0x7f2c1410b3c0 2024-05-08T23:17:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca68e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca688350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c12000700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 0x7f2bf80a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca5df150 con 0x562eca68de00 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca5dcb10 con 0x562eca68e720 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca5d8400 con 0x562eca688890 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca68e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c12000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca688350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca68e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45598/0 (socket says 172.21.15.46:45598) 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c13400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.674+0000 7f2c12a00700 1 -- 172.21.15.46:0/1572004937 learned_addr learned my addr 172.21.15.46:0/1572004937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 -- 172.21.15.46:0/1572004937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 msgr2=0x562eca688350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca688350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 -- 172.21.15.46:0/1572004937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 msgr2=0x562eca68e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca68e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 -- 172.21.15.46:0/1572004937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eca4593b0 con 0x562eca688890 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c13400700 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0c9f40 tx=0x7f2c0c099760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dacdee4ca07bba30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2c12000700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 0x7f2bf80a7be0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005cd0 tx=0x7f2c00005c60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2bffe00700 1 -- 172.21.15.46:0/1572004937 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c004ad0 con 0x562eca688890 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2bffe00700 1 -- 172.21.15.46:0/1572004937 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c01b320 con 0x562eca688890 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.675+0000 7f2bffe00700 1 -- 172.21.15.46:0/1572004937 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c099bf0 con 0x562eca688890 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1572004937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 msgr2=0x562eca688cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0c9f40 tx=0x7f2c0c099760 comp rx=0 tx=0).stop 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1572004937 shutdown_connections 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688cd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca68e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1572004937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca688350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1572004937 >> 172.21.15.46:0/1572004937 conn(0x562eca5e6210 msgr2=0x562eca682df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1572004937 shutdown_connections 2024-05-08T23:17:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.676+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1572004937 wait complete. 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 -- start start 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 Processor -- start 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca6920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca6925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca574560 con 0x562eca68e720 2024-05-08T23:17:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca574200 con 0x562eca68de00 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca45b6a0 con 0x562eca688890 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.677+0000 7f2c2d77fb80 10 client.0 _ll_register_callbacks cb 0x562eca57db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40978/0 (socket says 172.21.15.46:40978) 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 -- 172.21.15.46:0/1389962970 learned_addr learned my addr 172.21.15.46:0/1389962970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12000700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca6920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c13400700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca6925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 -- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 msgr2=0x562eca6920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca6920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 -- 172.21.15.46:0/1389962970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 msgr2=0x562eca6925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca6925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 -- 172.21.15.46:0/1389962970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141a1e00 con 0x562eca688890 2024-05-08T23:17:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c12a00700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2c04012ef0 tx=0x7f2c0400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd594235e8e6270 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2bfcc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04017070 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2c0400fc40 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.678+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eca531270 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c040122e0 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 10 client.29284 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562eca405b60 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eca531270 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c0401c420 con 0x562eca688890 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 10 client.29284 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 10 client.29284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 20 client.29284 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6a2370 0x562eca6a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 4 client.29284 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.679+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eca657810 con 0x562eca6a2370 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.680+0000 7f2c12000700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6a2370 0x562eca6a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.680+0000 7f2c12000700 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6a2370 0x562eca6a4790 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c749cf3c624483 server_cookie=6cc6fcf344f72f10 in_seq=0 out_seq=0 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.680+0000 7f2bfcc00700 10 client.29284 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2bfcc00700 1 -- 172.21.15.46:0/1389962970 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562eca657810 con 0x562eca6a2370 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2bfcc00700 10 client.29284 handle_command_reply: tid=0 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 client.29284 shutdown 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 msgr2=0x562eca691b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2c04012ef0 tx=0x7f2c0400f6a0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 shutdown_connections 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6a2370 0x562eca6a4790 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca6925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca691b60 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/1389962970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca6920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 >> 172.21.15.46:0/1389962970 conn(0x562eca5e6210 msgr2=0x562eca6825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2bfcc00700 0 client.29284 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 shutdown_connections 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/1389962970 wait complete. 2024-05-08T23:17:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.681+0000 7f2c2d77fb80 20 client.29284 trim_cache size 0 max 16384 2024-05-08T23:17:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.696+0000 7f2c2d77fb80 1 Processor -- start 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.696+0000 7f2c2d77fb80 1 -- start start 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca69dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca69e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca67fdc0 con 0x562eca68e720 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca68b870 con 0x562eca68de00 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca6a99a0 con 0x562eca688890 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40988/0 (socket says 172.21.15.46:40988) 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 -- 172.21.15.46:0/541989170 learned_addr learned my addr 172.21.15.46:0/541989170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12000700 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca69dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c13400700 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca69e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 -- 172.21.15.46:0/541989170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 msgr2=0x562eca69dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca69dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 -- 172.21.15.46:0/541989170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 msgr2=0x562eca69e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca69e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.697+0000 7f2c12a00700 1 -- 172.21.15.46:0/541989170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eca59e560 con 0x562eca688890 2024-05-08T23:17:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c12a00700 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2c0401c750 tx=0x7f2c0400f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=137b164c1dfdab9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c08c00700 1 -- 172.21.15.46:0/541989170 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04017030 con 0x562eca688890 2024-05-08T23:17:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c08c00700 1 -- 172.21.15.46:0/541989170 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c04018040 con 0x562eca688890 2024-05-08T23:17:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c08c00700 1 -- 172.21.15.46:0/541989170 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04005330 con 0x562eca688890 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/541989170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 msgr2=0x562eca688350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2c0401c750 tx=0x7f2c0400f9f0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/541989170 shutdown_connections 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68e720 0x562eca69e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca688350 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/541989170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68de00 0x562eca69dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.698+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/541989170 >> 172.21.15.46:0/541989170 conn(0x562eca5e6210 msgr2=0x562eca690430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.699+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/541989170 shutdown_connections 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.699+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/541989170 wait complete. 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.699+0000 7f2c2d77fb80 1 -- start start 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.699+0000 7f2c2d77fb80 1 Processor -- start 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca6988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca698e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca6f7870 con 0x562eca68de00 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca6f79b0 con 0x562eca68e720 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eca6f7b70 con 0x562eca688890 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c2d77fb80 10 client.0 _ll_register_callbacks cb 0x562eca57db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca6988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40994/0 (socket says 172.21.15.46:40994) 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 -- 172.21.15.46:0/3486698993 learned_addr learned my addr 172.21.15.46:0/3486698993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c13400700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca698e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 -- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 msgr2=0x562eca698e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca698e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 -- 172.21.15.46:0/3486698993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 msgr2=0x562eca6988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca6988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12a00700 1 -- 172.21.15.46:0/3486698993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562eca405b60 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.700+0000 7f2c12000700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca6988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2c12a00700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005cb0 tx=0x7f2c0400fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313588b49e6e9773 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04017040 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2c040052b0 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c0c01b320 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0400fce0 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2c2d77fb80 10 client.29290 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562eca45dcd0 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0403b020 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.701+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c04010430 con 0x562eca688890 2024-05-08T23:17:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 10 client.29290 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 10 client.29290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 20 client.29290 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6f8fb0 0x562eca6fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 4 client.29290 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562eca5d0ec0 con 0x562eca6f8fb0 2024-05-08T23:17:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c12000700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6f8fb0 0x562eca6fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2c12000700 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6f8fb0 0x562eca6fb3a0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2216c61601f32aa0 server_cookie=5c513330bcac4475 in_seq=0 out_seq=0 2024-05-08T23:17:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.702+0000 7f2bfcc00700 10 client.29290 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2bfcc00700 1 -- 172.21.15.46:0/3486698993 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562eca5d0ec0 con 0x562eca6f8fb0 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2bfcc00700 10 client.29290 handle_command_reply: tid=0 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 client.29290 shutdown 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 msgr2=0x562eca698390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005cb0 tx=0x7f2c0400fc40 comp rx=0 tx=0).stop 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 shutdown_connections 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562eca68e720 0x562eca698e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562eca688890 0x562eca698390 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562eca6f8fb0 0x562eca6fb3a0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 --2- 172.21.15.46:0/3486698993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eca68de00 0x562eca6988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 >> 172.21.15.46:0/3486698993 conn(0x562eca5e6210 msgr2=0x562eca68cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2bfcc00700 0 client.29290 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 shutdown_connections 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 1 -- 172.21.15.46:0/3486698993 wait complete. 2024-05-08T23:17:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.703+0000 7f2c2d77fb80 20 client.29290 trim_cache size 0 max 16384 2024-05-08T23:17:03.709 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 msgr2=0x7f2bf80a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 0x7f2bf80a7be0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005cd0 tx=0x7f2c00005c60 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 msgr2=0x7f2c141957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00efc0 tx=0x7f2c0c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 shutdown_connections 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1410b3c0 0x7f2c141957b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c14101fa0 0x7f2c14199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2bf80a5720 0x7f2bf80a7be0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 --2- 172.21.15.46:0/3893541287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141028c0 0x7f2c1419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 >> 172.21.15.46:0/3893541287 conn(0x7f2c140fb760 msgr2=0x7f2c1410ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 shutdown_connections 2024-05-08T23:17:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:03.704+0000 7f2c19c00700 1 -- 172.21.15.46:0/3893541287 wait complete. 2024-05-08T23:17:03.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1154978 9.06 MB/sec execute 565 sec latency 23.281 ms 2024-05-08T23:17:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:03 smithi046 ceph-mon[36898]: pgmap v412: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-05-08T23:17:04.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:03 smithi086 ceph-mon[78610]: pgmap v412: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-05-08T23:17:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:03 smithi053 ceph-mon[77587]: pgmap v412: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-05-08T23:17:04.724 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:17:04.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:17:04.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1156942 9.06 MB/sec execute 566 sec latency 17.632 ms 2024-05-08T23:17:04.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.937+0000 7f7343200700 1 -- 172.21.15.46:0/3797973713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c0733d0 msgr2=0x7f733c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.937+0000 7f7343200700 1 --2- 172.21.15.46:0/3797973713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c0733d0 0x7f733c0737b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f732c009990 tx=0x7f732c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 -- 172.21.15.46:0/3797973713 shutdown_connections 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 --2- 172.21.15.46:0/3797973713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c10d650 0x7f733c10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 --2- 172.21.15.46:0/3797973713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c0733d0 0x7f733c0737b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 --2- 172.21.15.46:0/3797973713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 0x7f733c10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 -- 172.21.15.46:0/3797973713 >> 172.21.15.46:0/3797973713 conn(0x7f733c0fc8e0 msgr2=0x7f733c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 -- 172.21.15.46:0/3797973713 shutdown_connections 2024-05-08T23:17:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.938+0000 7f7343200700 1 -- 172.21.15.46:0/3797973713 wait complete. 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 Processor -- start 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 -- start start 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 0x7f733c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c10d650 0x7f733c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c19e440 con 0x7f733c10d650 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c19e5b0 con 0x7f733c073cf0 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f7343200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c19e770 con 0x7f733c0733d0 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.939+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41002/0 (socket says 172.21.15.46:41002) 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 -- 172.21.15.46:0/4249030991 learned_addr learned my addr 172.21.15.46:0/4249030991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733b400700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 0x7f733c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f7341200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c10d650 0x7f733c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 -- 172.21.15.46:0/4249030991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 msgr2=0x7f733c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 0x7f733c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 -- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c10d650 msgr2=0x7f733c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c10d650 0x7f733c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 -- 172.21.15.46:0/4249030991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0095f0 con 0x7f733c0733d0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f733be00700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.940+0000 7f7338c00700 1 -- 172.21.15.46:0/4249030991 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01d070 con 0x7f733c0733d0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.941+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c19e9f0 con 0x7f733c0733d0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.941+0000 7f7338c00700 1 -- 172.21.15.46:0/4249030991 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c022470 con 0x7f733c0733d0 2024-05-08T23:17:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.941+0000 7f7338c00700 1 -- 172.21.15.46:0/4249030991 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c017650 con 0x7f733c0733d0 2024-05-08T23:17:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.941+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c1a6290 con 0x7f733c0733d0 2024-05-08T23:17:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.944+0000 7f7338c00700 1 -- 172.21.15.46:0/4249030991 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f732c0225e0 con 0x7f733c0733d0 2024-05-08T23:17:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.945+0000 7f7356c3eb80 1 Processor -- start 2024-05-08T23:17:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.945+0000 7f7356c3eb80 1 -- start start 2024-05-08T23:17:04.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.945+0000 7f7338c00700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 0x7f73200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7338c00700 1 -- 172.21.15.46:0/4249030991 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f732c0ce550 con 0x7f733c0733d0 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e84260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7341200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 0x7f73200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e7e910 0x55d9d5e7ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5dd51d0 con 0x55d9d5e83e80 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5dd2b90 con 0x55d9d5e7e910 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5dce480 con 0x55d9d5e847a0 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f733b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f733b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41014/0 (socket says 172.21.15.46:41014) 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f733be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e84260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f733b400700 1 -- 172.21.15.46:0/3731181545 learned_addr learned my addr 172.21.15.46:0/3731181545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.946+0000 7f7341200700 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e7e910 0x55d9d5e7ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 -- 172.21.15.46:0/3731181545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e7e910 msgr2=0x55d9d5e7ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e7e910 0x55d9d5e7ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 -- 172.21.15.46:0/3731181545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 msgr2=0x55d9d5e84260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e84260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 -- 172.21.15.46:0/3731181545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9d5c4f430 con 0x55d9d5e847a0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733be00700 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e84260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f7341200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 0x7f73200a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73340076f0 tx=0x7f7334007ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f733b400700 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f7328011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23843f2ef08ff86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f7331600700 1 -- 172.21.15.46:0/3731181545 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328019630 con 0x55d9d5e847a0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f7331600700 1 -- 172.21.15.46:0/3731181545 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7328009e70 con 0x55d9d5e847a0 2024-05-08T23:17:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.947+0000 7f7331600700 1 -- 172.21.15.46:0/3731181545 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800ea40 con 0x55d9d5e847a0 2024-05-08T23:17:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3731181545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 msgr2=0x55d9d5e7e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f7328011720 comp rx=0 tx=0).stop 2024-05-08T23:17:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3731181545 shutdown_connections 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e7e910 0x55d9d5e7ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e84260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3731181545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e847a0 0x55d9d5e7e3d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3731181545 >> 172.21.15.46:0/3731181545 conn(0x55d9d5ddc290 msgr2=0x55d9d5e78e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3731181545 shutdown_connections 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.948+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3731181545 wait complete. 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 -- start start 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 Processor -- start 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 0x55d9d5e87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5d6a5e0 con 0x55d9d5e7e910 2024-05-08T23:17:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5d6a280 con 0x55d9d5e847a0 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.949+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5c51720 con 0x55d9d5e83e80 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f7356c3eb80 10 client.0 _ll_register_callbacks cb 0x55d9d5d73bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41018/0 (socket says 172.21.15.46:41018) 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 -- 172.21.15.46:0/308823266 learned_addr learned my addr 172.21.15.46:0/308823266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733be00700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 0x55d9d5e87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f7341200700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 -- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 msgr2=0x55d9d5e885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 -- 172.21.15.46:0/308823266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 msgr2=0x55d9d5e87b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 0x55d9d5e87b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 -- 172.21.15.46:0/308823266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c1a6290 con 0x55d9d5e83e80 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733be00700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 0x55d9d5e87b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f733b400700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7328009230 tx=0x7f7328005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d200711cee05f148 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.950+0000 7f7326000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328004b60 con 0x55d9d5e83e80 2024-05-08T23:17:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7328004cc0 con 0x55d9d5e83e80 2024-05-08T23:17:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9d5d272f0 con 0x55d9d5e83e80 2024-05-08T23:17:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7328004e80 con 0x55d9d5e83e80 2024-05-08T23:17:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 10 client.29302 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f7328009980 con 0x55d9d5e83e80 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9d5d272f0 con 0x55d9d5e83e80 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f73280225a0 con 0x55d9d5e83e80 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 10 client.29302 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 10 client.29302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 20 client.29302 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5e980b0 0x55d9d5e9a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 4 client.29302 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.951+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9d5e4d890 con 0x55d9d5e980b0 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.952+0000 7f733be00700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5e980b0 0x55d9d5e9a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.952+0000 7f733be00700 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5e980b0 0x55d9d5e9a4d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9d39a0aff3dfc5 server_cookie=4e710960047a2ccf in_seq=0 out_seq=0 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.952+0000 7f7326000700 10 client.29302 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7326000700 1 -- 172.21.15.46:0/308823266 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9d5e4d890 con 0x55d9d5e980b0 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7326000700 10 client.29302 handle_command_reply: tid=0 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 client.29302 shutdown 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 msgr2=0x55d9d5e880b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7328009230 tx=0x7f7328005630 comp rx=0 tx=0).stop 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 shutdown_connections 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5e980b0 0x55d9d5e9a4d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e7e910 0x55d9d5e87b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/308823266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e83e80 0x55d9d5e880b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7326000700 0 client.29302 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 >> 172.21.15.46:0/308823266 conn(0x55d9d5ddc290 msgr2=0x55d9d5e78620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 shutdown_connections 2024-05-08T23:17:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.953+0000 7f7356c3eb80 1 -- 172.21.15.46:0/308823266 wait complete. 2024-05-08T23:17:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.954+0000 7f7356c3eb80 20 client.29302 trim_cache size 0 max 16384 2024-05-08T23:17:04.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.975+0000 7f7356c3eb80 1 Processor -- start 2024-05-08T23:17:04.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.975+0000 7f7356c3eb80 1 -- start start 2024-05-08T23:17:04.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e83e80 0x55d9d5e78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e847a0 0x55d9d5e880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5e79990 con 0x55d9d5e847a0 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5e79b40 con 0x55d9d5e83e80 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5e8a730 con 0x55d9d5e7e910 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41028/0 (socket says 172.21.15.46:41028) 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 -- 172.21.15.46:0/3975420085 learned_addr learned my addr 172.21.15.46:0/3975420085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f7341200700 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e847a0 0x55d9d5e880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733b400700 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e83e80 0x55d9d5e78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 -- 172.21.15.46:0/3975420085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e83e80 msgr2=0x55d9d5e78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e83e80 0x55d9d5e78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 -- 172.21.15.46:0/3975420085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e847a0 msgr2=0x55d9d5e880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e847a0 0x55d9d5e880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.976+0000 7f733be00700 1 -- 172.21.15.46:0/3975420085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9d5e75560 con 0x55d9d5e7e910 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.977+0000 7f733be00700 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f732c097e10 tx=0x7f732c09f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58c66b5ab24bfaf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.977+0000 7f7332000700 1 -- 172.21.15.46:0/3975420085 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c066070 con 0x55d9d5e7e910 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.977+0000 7f7332000700 1 -- 172.21.15.46:0/3975420085 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c09aee0 con 0x55d9d5e7e910 2024-05-08T23:17:04.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.977+0000 7f7332000700 1 -- 172.21.15.46:0/3975420085 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0a3790 con 0x55d9d5e7e910 2024-05-08T23:17:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3975420085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 msgr2=0x55d9d5e83370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f732c097e10 tx=0x7f732c09f040 comp rx=0 tx=0).stop 2024-05-08T23:17:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3975420085 shutdown_connections 2024-05-08T23:17:04.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e847a0 0x55d9d5e880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e83370 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/3975420085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e83e80 0x55d9d5e78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3975420085 >> 172.21.15.46:0/3975420085 conn(0x55d9d5ddc290 msgr2=0x55d9d5e82c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3975420085 shutdown_connections 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- 172.21.15.46:0/3975420085 wait complete. 2024-05-08T23:17:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 -- start start 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.978+0000 7f7356c3eb80 1 Processor -- start 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5eee600 con 0x55d9d5e83e80 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5eee740 con 0x55d9d5e847a0 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9d5eee900 con 0x55d9d5e7e910 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7356c3eb80 10 client.0 _ll_register_callbacks cb 0x55d9d5d73bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f7341200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f733b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f733be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41040/0 (socket says 172.21.15.46:41040) 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.979+0000 7f733be00700 1 -- 172.21.15.46:0/1831699507 learned_addr learned my addr 172.21.15.46:0/1831699507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 -- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 msgr2=0x55d9d5e8e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e8e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 -- 172.21.15.46:0/1831699507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 msgr2=0x55d9d5e8e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e8e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 -- 172.21.15.46:0/1831699507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c19e9f0 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f733be00700 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f732c0978a0 tx=0x7f732c09ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960f3b292bafbbf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7326000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c066050 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f732c09baa0 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7328009e70 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c0a3790 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7356c3eb80 10 client.29308 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.980+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d9d5e7efc0 con 0x55d9d5e7e910 2024-05-08T23:17:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7328009e70 con 0x55d9d5e7e910 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f732c0ce170 con 0x55d9d5e7e910 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 10 client.29308 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 10 client.29308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 20 client.29308 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5eefc10 0x55d9d5ef2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 4 client.29308 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d9d5dc6f40 con 0x55d9d5eefc10 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.981+0000 7f733b400700 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5eefc10 0x55d9d5ef2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.982+0000 7f733b400700 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5eefc10 0x55d9d5ef2030 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d02047ccaf9eec server_cookie=a0a8567614ea1218 in_seq=0 out_seq=0 2024-05-08T23:17:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.982+0000 7f7326000700 10 client.29308 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7326000700 1 -- 172.21.15.46:0/1831699507 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d9d5dc6f40 con 0x55d9d5eefc10 2024-05-08T23:17:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7326000700 10 client.29308 handle_command_reply: tid=0 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 client.29308 shutdown 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 msgr2=0x55d9d5e8ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f732c0978a0 tx=0x7f732c09ad80 comp rx=0 tx=0).stop 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 shutdown_connections 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9d5e847a0 0x55d9d5e8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9d5e7e910 0x55d9d5e8ddb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55d9d5eefc10 0x55d9d5ef2030 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 --2- 172.21.15.46:0/1831699507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d5e83e80 0x55d9d5e8e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 >> 172.21.15.46:0/1831699507 conn(0x55d9d5ddc290 msgr2=0x55d9d5e89410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.983+0000 7f7326000700 0 client.29308 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 shutdown_connections 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7356c3eb80 1 -- 172.21.15.46:0/1831699507 wait complete. 2024-05-08T23:17:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7356c3eb80 20 client.29308 trim_cache size 0 max 16384 2024-05-08T23:17:04.989 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:17:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ed295724-30a3-478b-aa9a-4cf885bb9459", 2024-05-08T23:17:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:17:04.989 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 msgr2=0x7f73200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 0x7f73200a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73340076f0 tx=0x7f7334007ab0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 msgr2=0x7f733c199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.984+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c005740 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 shutdown_connections 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1290 0x7f73200a3750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f733c10d650 0x7f733c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f733c0733d0 0x7f733c199570 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 --2- 172.21.15.46:0/4249030991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f733c073cf0 0x7f733c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 >> 172.21.15.46:0/4249030991 conn(0x7f733c0fc8e0 msgr2=0x7f733c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 shutdown_connections 2024-05-08T23:17:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:04.985+0000 7f7343200700 1 -- 172.21.15.46:0/4249030991 wait complete. 2024-05-08T23:17:05.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:17:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 -- 172.21.15.46:0/1461636286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584068b20 msgr2=0x7f858410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 --2- 172.21.15.46:0/1461636286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584068b20 0x7f858410dad0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b0a0 tx=0x7f857c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 -- 172.21.15.46:0/1461636286 shutdown_connections 2024-05-08T23:17:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 --2- 172.21.15.46:0/1461636286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584068b20 0x7f858410dad0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 --2- 172.21.15.46:0/1461636286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584106d30 0x7f8584107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 --2- 172.21.15.46:0/1461636286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 0x7f85840685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 -- 172.21.15.46:0/1461636286 >> 172.21.15.46:0/1461636286 conn(0x7f8584075710 msgr2=0x7f8584075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 -- 172.21.15.46:0/1461636286 shutdown_connections 2024-05-08T23:17:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.232+0000 7f8589a00700 1 -- 172.21.15.46:0/1461636286 wait complete. 2024-05-08T23:17:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.233+0000 7f8589a00700 1 Processor -- start 2024-05-08T23:17:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.233+0000 7f8589a00700 1 -- start start 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 0x7f85840ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584068b20 0x7f85841003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5520 con 0x7f8584068b20 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5690 con 0x7f8584068160 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5850 con 0x7f8584106d30 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41062/0 (socket says 172.21.15.46:41062) 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 -- 172.21.15.46:0/2703528403 learned_addr learned my addr 172.21.15.46:0/2703528403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8583e00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 0x7f85840ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8583400700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584068b20 0x7f85841003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 -- 172.21.15.46:0/2703528403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 msgr2=0x7f85840ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 0x7f85840ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 -- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584068b20 msgr2=0x7f85841003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584068b20 0x7f85841003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.234+0000 7f8589000700 1 -- 172.21.15.46:0/2703528403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009520 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8589000700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b070 tx=0x7f857c0078c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8580c00700 1 -- 172.21.15.46:0/2703528403 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c00e040 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8580c00700 1 -- 172.21.15.46:0/2703528403 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f857c01d470 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8580c00700 1 -- 172.21.15.46:0/2703528403 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c005280 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c009d00 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.235+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a5d70 con 0x7f8584106d30 2024-05-08T23:17:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.238+0000 7f8580c00700 1 -- 172.21.15.46:0/2703528403 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f857c01d5e0 con 0x7f8584106d30 2024-05-08T23:17:05.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.240+0000 7f859eac0b80 1 Processor -- start 2024-05-08T23:17:05.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.240+0000 7f859eac0b80 1 -- start start 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827970 0x55ba3e821350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e821cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e778150 con 0x55ba3e821890 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e775b10 con 0x55ba3e827970 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f859eac0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e771400 con 0x55ba3e827050 2024-05-08T23:17:05.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8580c00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 0x7f85680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8583400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8589000700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 0x7f85680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8580c00700 1 -- 172.21.15.46:0/2703528403 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f857c0ce3f0 con 0x7f8584106d30 2024-05-08T23:17:05.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8583400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41076/0 (socket says 172.21.15.46:41076) 2024-05-08T23:17:05.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8583400700 1 -- 172.21.15.46:0/1962793146 learned_addr learned my addr 172.21.15.46:0/1962793146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8583e00700 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827970 0x55ba3e821350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.241+0000 7f8583400700 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e821cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 -- 172.21.15.46:0/1962793146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827970 msgr2=0x55ba3e821350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827970 0x55ba3e821350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 -- 172.21.15.46:0/1962793146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 msgr2=0x55ba3e821cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e821cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 -- 172.21.15.46:0/1962793146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba3e5f23b0 con 0x55ba3e827050 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8583400700 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8570011860 tx=0x7f85700058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e1eade867f7ce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8589000700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 0x7f85680a3870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f857c0cb730 tx=0x7f857c01dc60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8579600700 1 -- 172.21.15.46:0/1962793146 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570004030 con 0x55ba3e827050 2024-05-08T23:17:05.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8579600700 1 -- 172.21.15.46:0/1962793146 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8570004830 con 0x55ba3e827050 2024-05-08T23:17:05.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.242+0000 7f8579600700 1 -- 172.21.15.46:0/1962793146 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570005340 con 0x55ba3e827050 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 -- 172.21.15.46:0/1962793146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 msgr2=0x55ba3e827430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8570011860 tx=0x7f85700058e0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 -- 172.21.15.46:0/1962793146 shutdown_connections 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e821cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827050 0x55ba3e827430 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1962793146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827970 0x55ba3e821350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 -- 172.21.15.46:0/1962793146 >> 172.21.15.46:0/1962793146 conn(0x55ba3e77f210 msgr2=0x55ba3e81c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 -- 172.21.15.46:0/1962793146 shutdown_connections 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.243+0000 7f859eac0b80 1 -- 172.21.15.46:0/1962793146 wait complete. 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.244+0000 7f859eac0b80 1 -- start start 2024-05-08T23:17:05.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.244+0000 7f859eac0b80 1 Processor -- start 2024-05-08T23:17:05.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e82ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827970 0x55ba3e82b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e70d560 con 0x55ba3e821890 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e70d200 con 0x55ba3e827050 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e5f46a0 con 0x55ba3e827970 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f859eac0b80 10 client.0 _ll_register_callbacks cb 0x55ba3e716b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e82ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48532/0 (socket says 172.21.15.46:48532) 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583e00700 1 -- 172.21.15.46:0/2520875983 learned_addr learned my addr 172.21.15.46:0/2520875983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583400700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827970 0x55ba3e82b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583e00700 1 -- 172.21.15.46:0/2520875983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827970 msgr2=0x55ba3e82b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.245+0000 7f8583e00700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827970 0x55ba3e82b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f8583e00700 1 -- 172.21.15.46:0/2520875983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 msgr2=0x55ba3e82ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f8583e00700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e82ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f8583e00700 1 -- 172.21.15.46:0/2520875983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8570019040 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f8583e00700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8574011890 tx=0x7f8574011ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b6bf4668b51b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f856e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574016070 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba3e6ca270 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.246+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f857400f850 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f857400fa10 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 10 client.29228 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba3e59eb60 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba3e6ca270 con 0x55ba3e827050 2024-05-08T23:17:05.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8574019390 con 0x55ba3e827050 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 10 client.29228 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 10 client.29228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 20 client.29228 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e83b3a0 0x55ba3e83d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 4 client.29228 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.247+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba3e7f0810 con 0x55ba3e83b3a0 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.248+0000 7f8583400700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e83b3a0 0x55ba3e83d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.248+0000 7f8583400700 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e83b3a0 0x55ba3e83d7c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b89e664e6c8fef server_cookie=93660fdfd5a2997d in_seq=0 out_seq=0 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.248+0000 7f856e000700 10 client.29228 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f856e000700 1 -- 172.21.15.46:0/2520875983 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba3e7f0810 con 0x55ba3e83b3a0 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f856e000700 10 client.29228 handle_command_reply: tid=0 2024-05-08T23:17:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 client.29228 shutdown 2024-05-08T23:17:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 msgr2=0x55ba3e82b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8574011890 tx=0x7f8574011ba0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 shutdown_connections 2024-05-08T23:17:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e827970 0x55ba3e82b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e83b3a0 0x55ba3e83d7c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e82ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 --2- 172.21.15.46:0/2520875983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e82b280 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f856e000700 0 client.29228 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.249+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 >> 172.21.15.46:0/2520875983 conn(0x55ba3e77f210 msgr2=0x55ba3e81c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.250+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 shutdown_connections 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.250+0000 7f859eac0b80 1 -- 172.21.15.46:0/2520875983 wait complete. 2024-05-08T23:17:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.250+0000 7f859eac0b80 20 client.29228 trim_cache size 0 max 16384 2024-05-08T23:17:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 Processor -- start 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 -- start start 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e77f210 0x55ba3e826450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e827050 0x55ba3e829610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e81cdc0 con 0x55ba3e827050 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e826990 con 0x55ba3e77f210 2024-05-08T23:17:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.270+0000 7f859eac0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e81d820 con 0x55ba3e821890 2024-05-08T23:17:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e77f210 0x55ba3e826450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e827050 0x55ba3e829610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41088/0 (socket says 172.21.15.46:41088) 2024-05-08T23:17:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 -- 172.21.15.46:0/1043517776 learned_addr learned my addr 172.21.15.46:0/1043517776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 -- 172.21.15.46:0/1043517776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e77f210 msgr2=0x55ba3e826450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e77f210 0x55ba3e826450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 -- 172.21.15.46:0/1043517776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e827050 msgr2=0x55ba3e829610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e827050 0x55ba3e829610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 -- 172.21.15.46:0/1043517776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba3e737560 con 0x55ba3e821890 2024-05-08T23:17:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.271+0000 7f8583e00700 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8574019770 tx=0x7f8574009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c0562e29b5d0ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.272+0000 7f857a000700 1 -- 172.21.15.46:0/1043517776 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574016070 con 0x55ba3e821890 2024-05-08T23:17:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.272+0000 7f857a000700 1 -- 172.21.15.46:0/1043517776 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8574019390 con 0x55ba3e821890 2024-05-08T23:17:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.272+0000 7f857a000700 1 -- 172.21.15.46:0/1043517776 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574005040 con 0x55ba3e821890 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 -- 172.21.15.46:0/1043517776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 msgr2=0x55ba3e82b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8574019770 tx=0x7f8574009370 comp rx=0 tx=0).stop 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 -- 172.21.15.46:0/1043517776 shutdown_connections 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e827050 0x55ba3e829610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e77f210 0x55ba3e826450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 --2- 172.21.15.46:0/1043517776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e821890 0x55ba3e82b5e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 -- 172.21.15.46:0/1043517776 >> 172.21.15.46:0/1043517776 conn(0x55ba3e821fd0 msgr2=0x55ba3e825d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 -- 172.21.15.46:0/1043517776 shutdown_connections 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.273+0000 7f859eac0b80 1 -- 172.21.15.46:0/1043517776 wait complete. 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 -- start start 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 Processor -- start 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e830870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e830db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e8312f0 con 0x55ba3e821890 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e831460 con 0x55ba3e827050 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3e831620 con 0x55ba3e77f210 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.274+0000 7f859eac0b80 10 client.0 _ll_register_callbacks cb 0x55ba3e716b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e830870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41094/0 (socket says 172.21.15.46:41094) 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583400700 1 -- 172.21.15.46:0/3805877903 learned_addr learned my addr 172.21.15.46:0/3805877903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583400700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e830db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 -- 172.21.15.46:0/3805877903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 msgr2=0x55ba3e832430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 -- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 msgr2=0x55ba3e830db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e830db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583e00700 1 -- 172.21.15.46:0/3805877903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba3e59eb60 con 0x55ba3e821890 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.275+0000 7f8583400700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f8583e00700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e830870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ba3e828e60 tx=0x7f8574019d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b916f931c22b6980 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f856e000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574016040 con 0x55ba3e821890 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f85740054d0 con 0x55ba3e821890 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8570004830 con 0x55ba3e821890 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8574019500 con 0x55ba3e821890 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f859eac0b80 10 client.19209 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.276+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba3e5f6cd0 con 0x55ba3e821890 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8570004830 con 0x55ba3e821890 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8574031070 con 0x55ba3e821890 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 10 client.19209 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 20 client.19209 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e827970 0x55ba3e8949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 4 client.19209 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba3e769ec0 con 0x55ba3e827970 2024-05-08T23:17:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.277+0000 7f8583400700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e827970 0x55ba3e8949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f8583400700 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e827970 0x55ba3e8949f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a9e750a87d6578 server_cookie=1642aadb8cb40e6f in_seq=0 out_seq=0 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f856e000700 10 client.19209 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f856e000700 1 -- 172.21.15.46:0/3805877903 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba3e769ec0 con 0x55ba3e827970 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f856e000700 10 client.19209 handle_command_reply: tid=0 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 client.19209 shutdown 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 msgr2=0x55ba3e830870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e830870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ba3e828e60 tx=0x7f8574019d90 comp rx=0 tx=0).stop 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 shutdown_connections 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba3e827050 0x55ba3e830db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba3e77f210 0x55ba3e832430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ba3e827970 0x55ba3e8949f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 --2- 172.21.15.46:0/3805877903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba3e821890 0x55ba3e830870 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 >> 172.21.15.46:0/3805877903 conn(0x55ba3e821fd0 msgr2=0x55ba3e82b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.278+0000 7f856e000700 0 client.19209 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.279+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 shutdown_connections 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.279+0000 7f859eac0b80 1 -- 172.21.15.46:0/3805877903 wait complete. 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.279+0000 7f859eac0b80 20 client.19209 trim_cache size 0 max 16384 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "ed295724-30a3-478b-aa9a-4cf885bb9459": { 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ed295724-30a3-478b-aa9a-4cf885bb9459", 2024-05-08T23:17:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 msgr2=0x7f85680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 0x7f85680a3870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f857c0cb730 tx=0x7f857c01dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 msgr2=0x7f8584100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b070 tx=0x7f857c0078c0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 shutdown_connections 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8584106d30 0x7f8584100900 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8584068160 0x7f85840ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85680a13b0 0x7f85680a3870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 --2- 172.21.15.46:0/2703528403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8584068b20 0x7f85841003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 >> 172.21.15.46:0/2703528403 conn(0x7f8584075710 msgr2=0x7f85840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 shutdown_connections 2024-05-08T23:17:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:05.280+0000 7f8589a00700 1 -- 172.21.15.46:0/2703528403 wait complete. 2024-05-08T23:17:05.307 INFO:tasks.cephfs.filesystem:scrub status for tag:ed295724-30a3-478b-aa9a-4cf885bb9459 - {'path': '/', 'tag': 'ed295724-30a3-478b-aa9a-4cf885bb9459', 'options': 'recursive,force'} 2024-05-08T23:17:05.920 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1158180 9.07 MB/sec execute 567 sec latency 20.909 ms 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: pgmap v413: 129 pgs: 129 active+clean; 613 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:05 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: pgmap v413: 129 pgs: 129 active+clean; 613 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-05-08T23:17:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:17:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:06.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:06.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:06.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:05 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: pgmap v413: 129 pgs: 129 active+clean; 613 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:05 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:06.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1159420 9.06 MB/sec execute 568 sec latency 25.435 ms 2024-05-08T23:17:07.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:06 smithi086 ceph-mon[78610]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:17:07.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:06 smithi053 ceph-mon[77587]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:17:07.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:06 smithi046 ceph-mon[36898]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:17:07.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1161439 9.07 MB/sec execute 569 sec latency 22.989 ms 2024-05-08T23:17:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:07 smithi086 ceph-mon[78610]: pgmap v415: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:08.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:07 smithi053 ceph-mon[77587]: pgmap v415: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:07 smithi046 ceph-mon[36898]: pgmap v415: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:08.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1162031 9.05 MB/sec execute 570 sec latency 151.989 ms 2024-05-08T23:17:09.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1163864 9.05 MB/sec execute 571 sec latency 27.578 ms 2024-05-08T23:17:10.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:09 smithi086 ceph-mon[78610]: pgmap v416: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:09 smithi053 ceph-mon[77587]: pgmap v416: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:09 smithi046 ceph-mon[36898]: pgmap v416: 129 pgs: 129 active+clean; 615 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-05-08T23:17:10.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1165169 9.05 MB/sec execute 572 sec latency 19.890 ms 2024-05-08T23:17:11.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:10 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:11.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:10 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:11.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:10 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:11.921 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1165776 9.04 MB/sec execute 573 sec latency 113.308 ms 2024-05-08T23:17:12.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: pgmap v417: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:11 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: pgmap v417: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:11 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: pgmap v417: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:11 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:12.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1166547 9.03 MB/sec execute 574 sec latency 171.347 ms 2024-05-08T23:17:13.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:12 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:13.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:12 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:13.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:12 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:12 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:12 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:12 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:12 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:12 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:12 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:17:13.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1168526 9.04 MB/sec execute 575 sec latency 29.172 ms 2024-05-08T23:17:14.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:13 smithi086 ceph-mon[78610]: pgmap v418: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:13 smithi053 ceph-mon[77587]: pgmap v418: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:13 smithi046 ceph-mon[36898]: pgmap v418: 129 pgs: 129 active+clean; 611 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-05-08T23:17:14.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1169348 9.03 MB/sec execute 576 sec latency 30.087 ms 2024-05-08T23:17:15.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1171423 9.03 MB/sec execute 577 sec latency 32.717 ms 2024-05-08T23:17:16.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:15 smithi086 ceph-mon[78610]: pgmap v419: 129 pgs: 129 active+clean; 617 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:17:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:15 smithi086 ceph-mon[78610]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:17:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:15 smithi053 ceph-mon[77587]: pgmap v419: 129 pgs: 129 active+clean; 617 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:17:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:15 smithi053 ceph-mon[77587]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:17:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:15 smithi046 ceph-mon[36898]: pgmap v419: 129 pgs: 129 active+clean; 617 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-05-08T23:17:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:15 smithi046 ceph-mon[36898]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:17:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:16.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1172387 9.03 MB/sec execute 578 sec latency 69.132 ms 2024-05-08T23:17:17.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:17.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:17.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1173534 9.02 MB/sec execute 579 sec latency 54.140 ms 2024-05-08T23:17:18.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:17 smithi086 ceph-mon[78610]: pgmap v421: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:18.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:17 smithi053 ceph-mon[77587]: pgmap v421: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:18.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:17 smithi046 ceph-mon[36898]: pgmap v421: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:18.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1175485 9.02 MB/sec execute 580 sec latency 56.314 ms 2024-05-08T23:17:19.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1176390 9.01 MB/sec execute 581 sec latency 18.655 ms 2024-05-08T23:17:20.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:19 smithi086 ceph-mon[78610]: pgmap v422: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:19 smithi053 ceph-mon[77587]: pgmap v422: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:19 smithi046 ceph-mon[36898]: pgmap v422: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 236 op/s 2024-05-08T23:17:20.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1178551 9.02 MB/sec execute 582 sec latency 15.417 ms 2024-05-08T23:17:21.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1179661 9.01 MB/sec execute 583 sec latency 13.722 ms 2024-05-08T23:17:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:21 smithi086 ceph-mon[78610]: pgmap v423: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:21 smithi053 ceph-mon[77587]: pgmap v423: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:21 smithi046 ceph-mon[36898]: pgmap v423: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:22.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1181401 9.01 MB/sec execute 584 sec latency 43.058 ms 2024-05-08T23:17:23.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1182883 9.01 MB/sec execute 585 sec latency 18.488 ms 2024-05-08T23:17:24.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:23 smithi086 ceph-mon[78610]: pgmap v424: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:23 smithi053 ceph-mon[77587]: pgmap v424: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:23 smithi046 ceph-mon[36898]: pgmap v424: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 266 op/s 2024-05-08T23:17:24.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1184201 9.01 MB/sec execute 586 sec latency 32.775 ms 2024-05-08T23:17:25.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1186236 9.02 MB/sec execute 587 sec latency 16.539 ms 2024-05-08T23:17:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:25 smithi086 ceph-mon[78610]: pgmap v425: 129 pgs: 129 active+clean; 609 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 195 op/s 2024-05-08T23:17:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:25 smithi053 ceph-mon[77587]: pgmap v425: 129 pgs: 129 active+clean; 609 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 195 op/s 2024-05-08T23:17:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:25 smithi046 ceph-mon[36898]: pgmap v425: 129 pgs: 129 active+clean; 609 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 195 op/s 2024-05-08T23:17:26.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1187071 9.00 MB/sec execute 588 sec latency 23.473 ms 2024-05-08T23:17:27.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1189168 9.01 MB/sec execute 589 sec latency 35.842 ms 2024-05-08T23:17:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:27 smithi086 ceph-mon[78610]: pgmap v426: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2024-05-08T23:17:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:27 smithi053 ceph-mon[77587]: pgmap v426: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2024-05-08T23:17:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:27 smithi046 ceph-mon[36898]: pgmap v426: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2024-05-08T23:17:28.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1190284 9.00 MB/sec execute 590 sec latency 15.750 ms 2024-05-08T23:17:29.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1192017 9.00 MB/sec execute 591 sec latency 17.635 ms 2024-05-08T23:17:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:29 smithi086 ceph-mon[78610]: pgmap v427: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-08T23:17:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:29 smithi053 ceph-mon[77587]: pgmap v427: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-08T23:17:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:29 smithi046 ceph-mon[36898]: pgmap v427: 129 pgs: 129 active+clean; 610 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 144 op/s 2024-05-08T23:17:30.922 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1193493 9.00 MB/sec execute 592 sec latency 18.371 ms 2024-05-08T23:17:31.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1194643 9.00 MB/sec execute 593 sec latency 18.973 ms 2024-05-08T23:17:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:31 smithi086 ceph-mon[78610]: pgmap v428: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2024-05-08T23:17:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:31 smithi053 ceph-mon[77587]: pgmap v428: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2024-05-08T23:17:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:31 smithi046 ceph-mon[36898]: pgmap v428: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 205 op/s 2024-05-08T23:17:32.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1196887 9.01 MB/sec execute 594 sec latency 15.643 ms 2024-05-08T23:17:33.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1197711 8.99 MB/sec execute 595 sec latency 17.135 ms 2024-05-08T23:17:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:33 smithi086 ceph-mon[78610]: pgmap v429: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-05-08T23:17:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:33 smithi053 ceph-mon[77587]: pgmap v429: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-05-08T23:17:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:33 smithi046 ceph-mon[36898]: pgmap v429: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-05-08T23:17:34.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1199545 8.99 MB/sec execute 596 sec latency 32.634 ms 2024-05-08T23:17:35.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 -- 172.21.15.46:0/167927147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 msgr2=0x7fb59c103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 --2- 172.21.15.46:0/167927147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c103690 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009990 tx=0x7fb58c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 -- 172.21.15.46:0/167927147 shutdown_connections 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 --2- 172.21.15.46:0/167927147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c1083d0 0x7fb59c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 --2- 172.21.15.46:0/167927147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c103690 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 --2- 172.21.15.46:0/167927147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c103c60 0x7fb59c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 -- 172.21.15.46:0/167927147 >> 172.21.15.46:0/167927147 conn(0x7fb59c0feb10 msgr2=0x7fb59c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 -- 172.21.15.46:0/167927147 shutdown_connections 2024-05-08T23:17:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.558+0000 7fb5a1800700 1 -- 172.21.15.46:0/167927147 wait complete. 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.559+0000 7fb5a1800700 1 Processor -- start 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.559+0000 7fb5a1800700 1 -- start start 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c1083d0 0x7fb59c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 0x7fb59c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19e900 con 0x7fb59c19de80 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19ea70 con 0x7fb59c1083d0 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb5a1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19ec30 con 0x7fb59c1032b0 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33476/0 (socket says 172.21.15.46:33476) 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb59aa00700 1 -- 172.21.15.46:0/1032071576 learned_addr learned my addr 172.21.15.46:0/1032071576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb59a000700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c1083d0 0x7fb59c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.560+0000 7fb59b400700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 0x7fb59c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 -- 172.21.15.46:0/1032071576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c1083d0 msgr2=0x7fb59c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c1083d0 0x7fb59c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 -- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 msgr2=0x7fb59c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 0x7fb59c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 -- 172.21.15.46:0/1032071576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0095f0 con 0x7fb59c1032b0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59b400700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 0x7fb59c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb593400700 1 -- 172.21.15.46:0/1032071576 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c01d070 con 0x7fb59c1032b0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb593400700 1 -- 172.21.15.46:0/1032071576 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb58c022470 con 0x7fb59c1032b0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c1a6130 con 0x7fb59c1032b0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb593400700 1 -- 172.21.15.46:0/1032071576 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c017650 con 0x7fb59c1032b0 2024-05-08T23:17:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.561+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c1a65b0 con 0x7fb59c1032b0 2024-05-08T23:17:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.566+0000 7fb593400700 1 -- 172.21.15.46:0/1032071576 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb58c0225e0 con 0x7fb59c1032b0 2024-05-08T23:17:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 Processor -- start 2024-05-08T23:17:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 -- start start 2024-05-08T23:17:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc5acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.567+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afbb1150 con 0x5641afc5a890 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afbaeb10 con 0x5641afc5fe00 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afbaa400 con 0x5641afc60720 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33488/0 (socket says 172.21.15.46:33488) 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 -- 172.21.15.46:0/4230582345 learned_addr learned my addr 172.21.15.46:0/4230582345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59a000700 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc5acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 -- 172.21.15.46:0/4230582345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 msgr2=0x5641afc601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 -- 172.21.15.46:0/4230582345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 msgr2=0x5641afc5acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc5acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59aa00700 1 -- 172.21.15.46:0/4230582345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641afa2b3b0 con 0x5641afc60720 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb59a000700 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc5acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.568+0000 7fb593400700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 0x7fb5800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb59aa00700 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0cb450 tx=0x7fb58c0d6540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c20c63666e78dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb59b400700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 0x7fb5800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb593400700 1 -- 172.21.15.46:0/1032071576 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb58c0ce230 con 0x7fb59c1032b0 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb587e00700 1 -- 172.21.15.46:0/4230582345 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0972a0 con 0x5641afc60720 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb587e00700 1 -- 172.21.15.46:0/4230582345 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb58c09ad30 con 0x5641afc60720 2024-05-08T23:17:35.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.569+0000 7fb587e00700 1 -- 172.21.15.46:0/4230582345 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c09aaa0 con 0x5641afc60720 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/4230582345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 msgr2=0x5641afc5a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0cb450 tx=0x7fb58c0d6540 comp rx=0 tx=0).stop 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb59b400700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 0x7fb5800a37d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb594009fd0 tx=0x7fb594009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/4230582345 shutdown_connections 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc5acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/4230582345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc60720 0x5641afc5a350 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/4230582345 >> 172.21.15.46:0/4230582345 conn(0x5641afbb8210 msgr2=0x5641afc54df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/4230582345 shutdown_connections 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/4230582345 wait complete. 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.570+0000 7fb5b52d9b80 1 -- start start 2024-05-08T23:17:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 Processor -- start 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afb46560 con 0x5641afc5a890 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afb46200 con 0x5641afc60720 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afa2d6a0 con 0x5641afc5fe00 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb5b52d9b80 10 client.0 _ll_register_callbacks cb 0x5641afb4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb59a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb59a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33498/0 (socket says 172.21.15.46:33498) 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb59a000700 1 -- 172.21.15.46:0/1716011993 learned_addr learned my addr 172.21.15.46:0/1716011993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.571+0000 7fb59b400700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 -- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 msgr2=0x5641afc641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 -- 172.21.15.46:0/1716011993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 msgr2=0x5641afc66730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc66730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 -- 172.21.15.46:0/1716011993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c1a65b0 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb59a000700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb588012700 tx=0x7fb588012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ae9696c7899226 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb584c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588017070 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb588004050 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.572+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641afb03270 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb588004210 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 10 client.29338 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5641af9d7b60 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641afb03270 con 0x5641afc5fe00 2024-05-08T23:17:35.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb588010430 con 0x5641afc5fe00 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 10 client.29338 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 10 client.29338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 20 client.29338 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afc73e40 0x5641afc76260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 4 client.29338 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.573+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641afc29810 con 0x5641afc73e40 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.574+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afc73e40 0x5641afc76260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.574+0000 7fb59aa00700 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afc73e40 0x5641afc76260 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d85dee609d8cfe server_cookie=9d6495fbf7d5bbd5 in_seq=0 out_seq=0 2024-05-08T23:17:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.574+0000 7fb584c00700 10 client.29338 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:35.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb584c00700 1 -- 172.21.15.46:0/1716011993 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641afc29810 con 0x5641afc73e40 2024-05-08T23:17:35.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb584c00700 10 client.29338 handle_command_reply: tid=0 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 client.29338 shutdown 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 msgr2=0x5641afc63a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb588012700 tx=0x7fb588012a10 comp rx=0 tx=0).stop 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 shutdown_connections 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afc73e40 0x5641afc76260 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc66730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/1716011993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc63a90 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb584c00700 0 client.29338 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.575+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 >> 172.21.15.46:0/1716011993 conn(0x5641afbb8210 msgr2=0x5641afc545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.576+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 shutdown_connections 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.576+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/1716011993 wait complete. 2024-05-08T23:17:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.576+0000 7fb5b52d9b80 20 client.29338 trim_cache size 0 max 16384 2024-05-08T23:17:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 Processor -- start 2024-05-08T23:17:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 -- start start 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 0x5641afc6f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 0x5641afc6aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc7b470 con 0x5641afc60720 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc4f800 con 0x5641afc5fe00 2024-05-08T23:17:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.601+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc56a00 con 0x5641afc5a890 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 0x5641afc6aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46588/0 (socket says 172.21.15.46:46588) 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 -- 172.21.15.46:0/2851158583 learned_addr learned my addr 172.21.15.46:0/2851158583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59aa00700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 0x5641afc6f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 -- 172.21.15.46:0/2851158583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 msgr2=0x5641afc6f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 0x5641afc6f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 -- 172.21.15.46:0/2851158583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 msgr2=0x5641afc6aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 0x5641afc6aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59a000700 1 -- 172.21.15.46:0/2851158583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641afb70560 con 0x5641afc5fe00 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59b400700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 0x5641afc6aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.602+0000 7fb59aa00700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 0x5641afc6f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.603+0000 7fb59a000700 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb588000c00 tx=0x7fb588009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9f774fde2a350e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.604+0000 7fb590c00700 1 -- 172.21.15.46:0/2851158583 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588017070 con 0x5641afc5fe00 2024-05-08T23:17:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.605+0000 7fb590c00700 1 -- 172.21.15.46:0/2851158583 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb588010a60 con 0x5641afc5fe00 2024-05-08T23:17:35.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.605+0000 7fb590c00700 1 -- 172.21.15.46:0/2851158583 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588017070 con 0x5641afc5fe00 2024-05-08T23:17:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.606+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/2851158583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 msgr2=0x5641afc54d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.606+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb588000c00 tx=0x7fb588009730 comp rx=0 tx=0).stop 2024-05-08T23:17:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.607+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/2851158583 shutdown_connections 2024-05-08T23:17:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.607+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc60720 0x5641afc6aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.607+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5a890 0x5641afc6f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.607+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/2851158583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc5fe00 0x5641afc54d20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.607+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/2851158583 >> 172.21.15.46:0/2851158583 conn(0x5641afbb8210 msgr2=0x5641afc5dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.608+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/2851158583 shutdown_connections 2024-05-08T23:17:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.608+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/2851158583 wait complete. 2024-05-08T23:17:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.609+0000 7fb5b52d9b80 1 -- start start 2024-05-08T23:17:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.609+0000 7fb5b52d9b80 1 Processor -- start 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc6b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc697b0 con 0x5641afc5a890 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc69920 con 0x5641afc60720 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641afc69ae0 con 0x5641afc5fe00 2024-05-08T23:17:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.610+0000 7fb5b52d9b80 10 client.0 _ll_register_callbacks cb 0x5641afb4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.611+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.611+0000 7fb59aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47944/0 (socket says 172.21.15.46:47944) 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.611+0000 7fb59aa00700 1 -- 172.21.15.46:0/864941031 learned_addr learned my addr 172.21.15.46:0/864941031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.611+0000 7fb59a000700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.611+0000 7fb59b400700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 -- 172.21.15.46:0/864941031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 msgr2=0x5641afc6b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc6b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 -- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 msgr2=0x5641afc690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 -- 172.21.15.46:0/864941031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641afa2b3b0 con 0x5641afc5a890 2024-05-08T23:17:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb59aa00700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb58c006010 tx=0x7fb58c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c21b854033a72c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.612+0000 7fb584c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:17:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.613+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c09a8c0 con 0x5641afc5a890 2024-05-08T23:17:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.613+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb58c09b660 con 0x5641afc5a890 2024-05-08T23:17:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.613+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb58c09ad30 con 0x5641afc5a890 2024-05-08T23:17:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.614+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58c0972a0 con 0x5641afc5a890 2024-05-08T23:17:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.614+0000 7fb5b52d9b80 10 client.19227 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.614+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb594007d20 con 0x5641afc5a890 2024-05-08T23:17:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb58c083080 con 0x5641afc5a890 2024-05-08T23:17:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb58c097670 con 0x5641afc5a890 2024-05-08T23:17:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 10 client.19227 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 20 client.19227 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afce4920 0x5641afce6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 4 client.19227 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5641afba2ec0 con 0x5641afce4920 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.615+0000 7fb59a000700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afce4920 0x5641afce6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.616+0000 7fb584c00700 10 client.19227 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.616+0000 7fb59a000700 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afce4920 0x5641afce6d10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5878c85961a5a9b server_cookie=bf9c9a66864a2883 in_seq=0 out_seq=0 2024-05-08T23:17:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.617+0000 7fb584c00700 1 -- 172.21.15.46:0/864941031 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5641afba2ec0 con 0x5641afce4920 2024-05-08T23:17:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.617+0000 7fb584c00700 10 client.19227 handle_command_reply: tid=0 2024-05-08T23:17:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.617+0000 7fb5b52d9b80 1 client.19227 shutdown 2024-05-08T23:17:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 msgr2=0x5641afc6ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb58c006010 tx=0x7fb58c009310 comp rx=0 tx=0).stop 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 shutdown_connections 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641afc60720 0x5641afc690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5641afce4920 0x5641afce6d10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641afc5a890 0x5641afc6ab40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 --2- 172.21.15.46:0/864941031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641afc5fe00 0x5641afc6b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 >> 172.21.15.46:0/864941031 conn(0x5641afbb8210 msgr2=0x5641afc5eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.618+0000 7fb584c00700 0 client.19227 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.620+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 shutdown_connections 2024-05-08T23:17:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.620+0000 7fb5b52d9b80 1 -- 172.21.15.46:0/864941031 wait complete. 2024-05-08T23:17:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.620+0000 7fb5b52d9b80 20 client.19227 trim_cache size 0 max 16384 2024-05-08T23:17:35.620 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:35.620 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:17:35.620 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:17:35.620 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.621+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 msgr2=0x7fb5800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.621+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 0x7fb5800a37d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb594009fd0 tx=0x7fb594009040 comp rx=0 tx=0).stop 2024-05-08T23:17:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.621+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 msgr2=0x7fb59c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.621+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005740 comp rx=0 tx=0).stop 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 shutdown_connections 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb5800a1310 0x7fb5800a37d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c19de80 0x7fb59c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb59c1032b0 0x7fb59c1995f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 --2- 172.21.15.46:0/1032071576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb59c1083d0 0x7fb59c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 >> 172.21.15.46:0/1032071576 conn(0x7fb59c0feb10 msgr2=0x7fb59c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 shutdown_connections 2024-05-08T23:17:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.622+0000 7fb5a1800700 1 -- 172.21.15.46:0/1032071576 wait complete. 2024-05-08T23:17:35.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:17:35.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 -- 172.21.15.46:0/4165639892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe08101a20 msgr2=0x7fbe08105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 --2- 172.21.15.46:0/4165639892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe08101a20 0x7fbe08105a60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009990 tx=0x7fbdfc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 -- 172.21.15.46:0/4165639892 shutdown_connections 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 --2- 172.21.15.46:0/4165639892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08106190 0x7fbe0810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 --2- 172.21.15.46:0/4165639892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08101070 0x7fbe08101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 --2- 172.21.15.46:0/4165639892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe08101a20 0x7fbe08105a60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 -- 172.21.15.46:0/4165639892 >> 172.21.15.46:0/4165639892 conn(0x7fbe080fc930 msgr2=0x7fbe080fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 -- 172.21.15.46:0/4165639892 shutdown_connections 2024-05-08T23:17:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.873+0000 7fbe10600700 1 -- 172.21.15.46:0/4165639892 wait complete. 2024-05-08T23:17:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 Processor -- start 2024-05-08T23:17:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 -- start start 2024-05-08T23:17:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 0x7fbe08197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe0819bc00 0x7fbe0819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0819c680 con 0x7fbe0819bc00 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0819c7f0 con 0x7fbe08106190 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.874+0000 7fbe10600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0819c9b0 con 0x7fbe08101070 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33526/0 (socket says 172.21.15.46:33526) 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 -- 172.21.15.46:0/3647073288 learned_addr learned my addr 172.21.15.46:0/3647073288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0e600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe0819bc00 0x7fbe0819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0d200700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 0x7fbe08197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 -- 172.21.15.46:0/3647073288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 msgr2=0x7fbe08197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 0x7fbe08197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 -- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe0819bc00 msgr2=0x7fbe0819c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe0819bc00 0x7fbe0819c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 -- 172.21.15.46:0/3647073288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00009c40 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0d200700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 0x7fbe08197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.875+0000 7fbe0dc00700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e9a0 tx=0x7fbdf800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.876+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8009790 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.876+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf800cbd0 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.876+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf800eee0 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.876+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc0095f0 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.876+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe081a41b0 con 0x7fbe08101070 2024-05-08T23:17:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.877+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe08109460 con 0x7fbe08101070 2024-05-08T23:17:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.879+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbdf800cd40 con 0x7fbe08101070 2024-05-08T23:17:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.880+0000 7fbe06000700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 0x7fbdf00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.881+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbdf8014070 con 0x7fbe08101070 2024-05-08T23:17:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.881+0000 7fbe0d200700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 0x7fbdf00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:35.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.881+0000 7fbe0d200700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 0x7fbdf00a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005b40 tx=0x7fbdfc005ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:35.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:35.885+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbdf8052d60 con 0x7fbe08101070 2024-05-08T23:17:35.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1200759 8.99 MB/sec execute 597 sec latency 24.469 ms 2024-05-08T23:17:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:35 smithi046 ceph-mon[36898]: pgmap v430: 129 pgs: 129 active+clean; 618 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 167 op/s 2024-05-08T23:17:36.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.191+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe08067d10 con 0x7fbe08101070 2024-05-08T23:17:36.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.192+0000 7fbe06000700 1 -- 172.21.15.46:0/3647073288 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fbdf8099df0 con 0x7fbe08101070 2024-05-08T23:17:36.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:17:36.191 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:08:15.796608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:17:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.194+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 msgr2=0x7fbdf00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.194+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 0x7fbdf00a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005b40 tx=0x7fbdfc005ab0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.194+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 msgr2=0x7fbe08197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.194+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e9a0 tx=0x7fbdf800ed60 comp rx=0 tx=0).stop 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 shutdown_connections 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbdf00a1290 0x7fbdf00a3750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe0819bc00 0x7fbe0819c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe08101070 0x7fbe08197320 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 --2- 172.21.15.46:0/3647073288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe08106190 0x7fbe08197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 >> 172.21.15.46:0/3647073288 conn(0x7fbe080fc930 msgr2=0x7fbe080fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 shutdown_connections 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.195+0000 7fbe10600700 1 -- 172.21.15.46:0/3647073288 wait complete. 2024-05-08T23:17:36.195 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-05-08T23:17:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:35 smithi086 ceph-mon[78610]: pgmap v430: 129 pgs: 129 active+clean; 618 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 167 op/s 2024-05-08T23:17:36.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:17:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:35 smithi053 ceph-mon[77587]: pgmap v430: 129 pgs: 129 active+clean; 618 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 167 op/s 2024-05-08T23:17:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 -- 172.21.15.46:0/2339236700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628073340 msgr2=0x7f3628073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 --2- 172.21.15.46:0/2339236700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628073340 0x7f3628073720 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f361800b0a0 tx=0x7f361800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 -- 172.21.15.46:0/2339236700 shutdown_connections 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 --2- 172.21.15.46:0/2339236700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628111e00 0x7f36281141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 --2- 172.21.15.46:0/2339236700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628073340 0x7f3628073720 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 --2- 172.21.15.46:0/2339236700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f36281118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.454+0000 7f362da00700 1 -- 172.21.15.46:0/2339236700 >> 172.21.15.46:0/2339236700 conn(0x7f36280fc900 msgr2=0x7f36280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.455+0000 7f362da00700 1 -- 172.21.15.46:0/2339236700 shutdown_connections 2024-05-08T23:17:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.455+0000 7f362da00700 1 -- 172.21.15.46:0/2339236700 wait complete. 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 Processor -- start 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 -- start start 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628073340 0x7f362819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f362819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a2950 con 0x7f3628073c60 2024-05-08T23:17:36.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a2ac0 con 0x7f3628073340 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f362da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a2c80 con 0x7f3628111e00 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f361e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f362819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f3626a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628073340 0x7f362819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33548/0 (socket says 172.21.15.46:33548) 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.456+0000 7f3627400700 1 -- 172.21.15.46:0/2420493937 learned_addr learned my addr 172.21.15.46:0/2420493937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 -- 172.21.15.46:0/2420493937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 msgr2=0x7f36281a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 -- 172.21.15.46:0/2420493937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628073340 msgr2=0x7f362819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628073340 0x7f362819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 -- 172.21.15.46:0/2420493937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009520 con 0x7f3628073c60 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f3627400700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361e000700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f362819dfb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f361000e9f0 tx=0x7f361000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361fe00700 1 -- 172.21.15.46:0/2420493937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000cb70 con 0x7f3628073c60 2024-05-08T23:17:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361fe00700 1 -- 172.21.15.46:0/2420493937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f361000ee90 con 0x7f3628073c60 2024-05-08T23:17:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618009d00 con 0x7f3628073c60 2024-05-08T23:17:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.457+0000 7f361fe00700 1 -- 172.21.15.46:0/2420493937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610003cf0 con 0x7f3628073c60 2024-05-08T23:17:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.458+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281aa630 con 0x7f3628073c60 2024-05-08T23:17:36.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.461+0000 7f361fe00700 1 -- 172.21.15.46:0/2420493937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f361000ccd0 con 0x7f3628073c60 2024-05-08T23:17:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.462+0000 7f36414b3b80 1 Processor -- start 2024-05-08T23:17:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.462+0000 7f36414b3b80 1 -- start start 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f361fe00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 0x7f360c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de202cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de159150 con 0x55c5de207e00 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de156b10 con 0x55c5de202890 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f36414b3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de152400 con 0x55c5de208720 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f361fe00700 1 -- 172.21.15.46:0/2420493937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3610014070 con 0x7f3628073c60 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f361e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f3626a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f361e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33558/0 (socket says 172.21.15.46:33558) 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f361e000700 1 -- 172.21.15.46:0/1430546217 learned_addr learned my addr 172.21.15.46:0/1430546217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f3627400700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 0x7f360c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.463+0000 7f3627400700 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de202cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 -- 172.21.15.46:0/1430546217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 msgr2=0x55c5de202cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de202cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 -- 172.21.15.46:0/1430546217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 msgr2=0x55c5de2081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 -- 172.21.15.46:0/1430546217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ddfd33b0 con 0x55c5de208720 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f3626a00700 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f361e000700 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3610005cb0 tx=0x7f36100999a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6e4da3d39fba06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f3627400700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 0x7f360c0a37d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3620013ab0 tx=0x7f3620009ce0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f3617e00700 1 -- 172.21.15.46:0/1430546217 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361009de70 con 0x55c5de208720 2024-05-08T23:17:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f3617e00700 1 -- 172.21.15.46:0/1430546217 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36100a24c0 con 0x55c5de208720 2024-05-08T23:17:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.464+0000 7f3617e00700 1 -- 172.21.15.46:0/1430546217 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361005d1e0 con 0x55c5de208720 2024-05-08T23:17:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 -- 172.21.15.46:0/1430546217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 msgr2=0x55c5de202350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3610005cb0 tx=0x7f36100999a0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 -- 172.21.15.46:0/1430546217 shutdown_connections 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de202cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1430546217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de202350 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 -- 172.21.15.46:0/1430546217 >> 172.21.15.46:0/1430546217 conn(0x55c5de160210 msgr2=0x55c5de1fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 -- 172.21.15.46:0/1430546217 shutdown_connections 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.465+0000 7f36414b3b80 1 -- 172.21.15.46:0/1430546217 wait complete. 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.466+0000 7f36414b3b80 1 -- start start 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.466+0000 7f36414b3b80 1 Processor -- start 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de20e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de20ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de0ee560 con 0x55c5de202890 2024-05-08T23:17:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de0ee200 con 0x55c5de207e00 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ddfd56a0 con 0x55c5de208720 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f36414b3b80 10 client.0 _ll_register_callbacks cb 0x55c5de0f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33574/0 (socket says 172.21.15.46:33574) 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 -- 172.21.15.46:0/833871631 learned_addr learned my addr 172.21.15.46:0/833871631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f361e000700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de20ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3626a00700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de20e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 -- 172.21.15.46:0/833871631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 msgr2=0x55c5de20ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de20ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 -- 172.21.15.46:0/833871631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 msgr2=0x55c5de20e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de20e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.467+0000 7f3627400700 1 -- 172.21.15.46:0/833871631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281aa630 con 0x55c5de208720 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.468+0000 7f3627400700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3620012040 tx=0x7f3620009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35855285b1c526ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.468+0000 7f3614c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.468+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620020650 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.468+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5de0ab270 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.468+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f36200207b0 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3620020970 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 10 client.29362 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5ddf7fb60 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5de0ab270 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3620010ba0 con 0x55c5de208720 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 10 client.29362 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 10 client.29362 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 20 client.29362 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de21c200 0x55c5de21e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 4 client.29362 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5de1d1810 con 0x55c5de21c200 2024-05-08T23:17:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.469+0000 7f3626a00700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de21c200 0x55c5de21e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.470+0000 7f3626a00700 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de21c200 0x55c5de21e620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb11990d346b458b server_cookie=a4d41fd9319ff5fd in_seq=0 out_seq=0 2024-05-08T23:17:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.470+0000 7f3614c00700 10 client.29362 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.470+0000 7f3614c00700 1 -- 172.21.15.46:0/833871631 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5de1d1810 con 0x55c5de21c200 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.470+0000 7f3614c00700 10 client.29362 handle_command_reply: tid=0 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 client.29362 shutdown 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 msgr2=0x55c5de20c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3620012040 tx=0x7f3620009260 comp rx=0 tx=0).stop 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 shutdown_connections 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de21c200 0x55c5de21e620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20c150 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de20e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f3614c00700 0 client.29362 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 --2- 172.21.15.46:0/833871631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de20ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 >> 172.21.15.46:0/833871631 conn(0x55c5de160210 msgr2=0x55c5de1fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 shutdown_connections 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 1 -- 172.21.15.46:0/833871631 wait complete. 2024-05-08T23:17:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.471+0000 7f36414b3b80 20 client.29362 trim_cache size 0 max 16384 2024-05-08T23:17:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.488+0000 7f36414b3b80 1 Processor -- start 2024-05-08T23:17:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.488+0000 7f36414b3b80 1 -- start start 2024-05-08T23:17:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.488+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de1fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f36414b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de20cda0 con 0x55c5de202890 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f36414b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de20c750 con 0x55c5de207e00 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f36414b3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de209f60 con 0x55c5de208720 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3626a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3626a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47994/0 (socket says 172.21.15.46:47994) 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3626a00700 1 -- 172.21.15.46:0/3935432034 learned_addr learned my addr 172.21.15.46:0/3935432034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f361e000700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de1fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 -- 172.21.15.46:0/3935432034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 msgr2=0x55c5de1fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de1fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 -- 172.21.15.46:0/3935432034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 msgr2=0x55c5de202350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3627400700 1 -- 172.21.15.46:0/3935432034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5de202f40 con 0x55c5de208720 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f3626a00700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.489+0000 7f361e000700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de1fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.490+0000 7f3627400700 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20a2e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3620010f00 tx=0x7f3620005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1751773f6d92506d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.490+0000 7f361cc00700 1 -- 172.21.15.46:0/3935432034 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620039050 con 0x55c5de208720 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.490+0000 7f361cc00700 1 -- 172.21.15.46:0/3935432034 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3620034d30 con 0x55c5de208720 2024-05-08T23:17:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.490+0000 7f361cc00700 1 -- 172.21.15.46:0/3935432034 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620010ad0 con 0x55c5de208720 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- 172.21.15.46:0/3935432034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 msgr2=0x55c5de20a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20a2e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3620010f00 tx=0x7f3620005980 comp rx=0 tx=0).stop 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- 172.21.15.46:0/3935432034 shutdown_connections 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de20a2e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de202890 0x55c5de202350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 --2- 172.21.15.46:0/3935432034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de207e00 0x55c5de1fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- 172.21.15.46:0/3935432034 >> 172.21.15.46:0/3935432034 conn(0x55c5de160210 msgr2=0x55c5de217c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- 172.21.15.46:0/3935432034 shutdown_connections 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- 172.21.15.46:0/3935432034 wait complete. 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.491+0000 7f36414b3b80 1 -- start start 2024-05-08T23:17:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 Processor -- start 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de213170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de203fc0 con 0x55c5de207e00 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de204130 con 0x55c5de202890 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5de2042f0 con 0x55c5de208720 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f36414b3b80 10 client.0 _ll_register_callbacks cb 0x55c5de0f7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f3627400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33598/0 (socket says 172.21.15.46:33598) 2024-05-08T23:17:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f3627400700 1 -- 172.21.15.46:0/1632965884 learned_addr learned my addr 172.21.15.46:0/1632965884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.492+0000 7f3626a00700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de213170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f361e000700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 -- 172.21.15.46:0/1632965884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 msgr2=0x55c5de213170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de213170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 -- 172.21.15.46:0/1632965884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 msgr2=0x55c5de2136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 -- 172.21.15.46:0/1632965884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3628114ba0 con 0x55c5de208720 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3627400700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3620010f80 tx=0x7f3620010530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1a0728b133e195 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3614c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620034ea0 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36100a24c0 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.493+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3620034710 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36200348d0 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 10 client.29368 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c5ddfd33b0 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36100a24c0 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f362002a410 con 0x55c5de208720 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 10 client.29368 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 10 client.29368 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 20 client.29368 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de273280 0x55c5de275670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 4 client.29368 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5de14aec0 con 0x55c5de273280 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.494+0000 7f3626a00700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de273280 0x55c5de275670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f3626a00700 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de273280 0x55c5de275670 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cc22864cacbcbc2 server_cookie=20be1c14bf53404f in_seq=0 out_seq=0 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f3614c00700 10 client.29368 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f3614c00700 1 -- 172.21.15.46:0/1632965884 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5de14aec0 con 0x55c5de273280 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f3614c00700 10 client.29368 handle_command_reply: tid=0 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 client.29368 shutdown 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 msgr2=0x55c5de2039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3620010f80 tx=0x7f3620010530 comp rx=0 tx=0).stop 2024-05-08T23:17:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 shutdown_connections 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c5de273280 0x55c5de275670 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5de208720 0x55c5de2039f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5de202890 0x55c5de213170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f3614c00700 0 client.29368 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 --2- 172.21.15.46:0/1632965884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5de207e00 0x55c5de2136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.495+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 >> 172.21.15.46:0/1632965884 conn(0x55c5de160210 msgr2=0x55c5de217c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 shutdown_connections 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f36414b3b80 1 -- 172.21.15.46:0/1632965884 wait complete. 2024-05-08T23:17:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f36414b3b80 20 client.29368 trim_cache size 0 max 16384 2024-05-08T23:17:36.501 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 msgr2=0x7f360c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 0x7f360c0a37d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3620013ab0 tx=0x7f3620009ce0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 msgr2=0x7f362819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f362819dfb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f361000e9f0 tx=0x7f361000ed00 comp rx=0 tx=0).stop 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 shutdown_connections 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3628111e00 0x7f36281a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.496+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3628073340 0x7f362819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.497+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f360c0a1310 0x7f360c0a37d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.497+0000 7f362da00700 1 --2- 172.21.15.46:0/2420493937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3628073c60 0x7f362819dfb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.497+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 >> 172.21.15.46:0/2420493937 conn(0x7f36280fc900 msgr2=0x7f36280fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.497+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 shutdown_connections 2024-05-08T23:17:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.497+0000 7f362da00700 1 -- 172.21.15.46:0/2420493937 wait complete. 2024-05-08T23:17:36.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 -- 172.21.15.46:0/1600218132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 msgr2=0x7f742010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 --2- 172.21.15.46:0/1600218132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f742010ada0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f740c009990 tx=0x7f740c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 -- 172.21.15.46:0/1600218132 shutdown_connections 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 --2- 172.21.15.46:0/1600218132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742010b3c0 0x7f742010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 --2- 172.21.15.46:0/1600218132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7420101fa0 0x7f7420102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 --2- 172.21.15.46:0/1600218132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f742010ada0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 -- 172.21.15.46:0/1600218132 >> 172.21.15.46:0/1600218132 conn(0x7f74200fb760 msgr2=0x7f74200fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 -- 172.21.15.46:0/1600218132 shutdown_connections 2024-05-08T23:17:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.740+0000 7f7426400700 1 -- 172.21.15.46:0/1600218132 wait complete. 2024-05-08T23:17:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.741+0000 7f7426400700 1 Processor -- start 2024-05-08T23:17:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.741+0000 7f7426400700 1 -- start start 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7420101fa0 0x7f74200ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f742010b3c0 0x7f7420100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74201a5680 con 0x7f742010b3c0 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74201a57f0 con 0x7f7420101fa0 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f7426400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74201a59b0 con 0x7f74201028c0 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33628/0 (socket says 172.21.15.46:33628) 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 -- 172.21.15.46:0/2520571700 learned_addr learned my addr 172.21.15.46:0/2520571700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741fe00700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f742010b3c0 0x7f7420100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741f400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7420101fa0 0x7f74200ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 -- 172.21.15.46:0/2520571700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7420101fa0 msgr2=0x7f74200ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7420101fa0 0x7f74200ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 -- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f742010b3c0 msgr2=0x7f7420100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f742010b3c0 0x7f7420100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.742+0000 7f741ea00700 1 -- 172.21.15.46:0/2520571700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410009520 con 0x7f74201028c0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f741ea00700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f7417e00700 1 -- 172.21.15.46:0/2520571700 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c01d070 con 0x7f74201028c0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f7417e00700 1 -- 172.21.15.46:0/2520571700 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740c022470 con 0x7f74201028c0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c0095f0 con 0x7f74201028c0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f7417e00700 1 -- 172.21.15.46:0/2520571700 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00bee0 con 0x7f74201028c0 2024-05-08T23:17:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.743+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74201a5fa0 con 0x7f74201028c0 2024-05-08T23:17:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.747+0000 7f7417e00700 1 -- 172.21.15.46:0/2520571700 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f740c0225e0 con 0x7f74201028c0 2024-05-08T23:17:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7417e00700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 0x7f74040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 Processor -- start 2024-05-08T23:17:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7417e00700 1 -- 172.21.15.46:0/2520571700 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f740c0ce360 con 0x7f74201028c0 2024-05-08T23:17:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 -- start start 2024-05-08T23:17:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f741f400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 0x7f74040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa9720 0x559c60fa3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa3890 0x559c60fa3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60efa150 con 0x559c60fa9720 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60ef7b10 con 0x559c60fa8e00 2024-05-08T23:17:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.749+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60ef3400 con 0x559c60fa3890 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa9720 0x559c60fa3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46680/0 (socket says 172.21.15.46:46680) 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 -- 172.21.15.46:0/819815352 learned_addr learned my addr 172.21.15.46:0/819815352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741fe00700 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa3890 0x559c60fa3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 0x7f74040a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7410009e40 tx=0x7f7410009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 -- 172.21.15.46:0/819815352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa3890 msgr2=0x559c60fa3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa3890 0x559c60fa3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 -- 172.21.15.46:0/819815352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa9720 msgr2=0x559c60fa3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa9720 0x559c60fa3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 -- 172.21.15.46:0/819815352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c60d743b0 con 0x559c60fa8e00 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.750+0000 7f741f400700 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7410012f40 tx=0x7f7410019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d01afc3af6e28002 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.751+0000 7f7414c00700 1 -- 172.21.15.46:0/819815352 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741001cdb0 con 0x559c60fa8e00 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.751+0000 7f7414c00700 1 -- 172.21.15.46:0/819815352 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74100044e0 con 0x559c60fa8e00 2024-05-08T23:17:36.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.751+0000 7f7414c00700 1 -- 172.21.15.46:0/819815352 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741001b460 con 0x559c60fa8e00 2024-05-08T23:17:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- 172.21.15.46:0/819815352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 msgr2=0x559c60fa91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7410012f40 tx=0x7f7410019000 comp rx=0 tx=0).stop 2024-05-08T23:17:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- 172.21.15.46:0/819815352 shutdown_connections 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa3890 0x559c60fa3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fa91e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/819815352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa9720 0x559c60fa3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- 172.21.15.46:0/819815352 >> 172.21.15.46:0/819815352 conn(0x559c60f01210 msgr2=0x559c60f9ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- 172.21.15.46:0/819815352 shutdown_connections 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- 172.21.15.46:0/819815352 wait complete. 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.752+0000 7f7439fd1b80 1 -- start start 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 Processor -- start 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60facb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c60fad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60e8f560 con 0x559c60fa3890 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60e8f200 con 0x559c60fa8e00 2024-05-08T23:17:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60d766a0 con 0x559c60fa9720 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.753+0000 7f7439fd1b80 10 client.0 _ll_register_callbacks cb 0x559c60e98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60facb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c60fad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46690/0 (socket says 172.21.15.46:46690) 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741ea00700 1 -- 172.21.15.46:0/3553575722 learned_addr learned my addr 172.21.15.46:0/3553575722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 -- 172.21.15.46:0/3553575722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 msgr2=0x559c60fad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 -- 172.21.15.46:0/3553575722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 msgr2=0x559c60facb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60facb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 -- 172.21.15.46:0/3553575722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7410017050 con 0x559c60fa9720 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741f400700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60facb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f741fe00700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c60fad580 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7418011e90 tx=0x7f7418014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665bd95b96c85830 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.754+0000 7f7409600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800e040 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f74180093a0 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c60e4c270 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7418004db0 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7439fd1b80 10 client.29380 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559c60d20b60 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.755+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c60e4c270 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f741801c030 con 0x559c60fa9720 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 10 client.29380 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 10 client.29380 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 20 client.29380 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c60fbd350 0x559c60fbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 4 client.29380 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c60f72810 con 0x559c60fbd350 2024-05-08T23:17:36.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f741f400700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c60fbd350 0x559c60fbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f741f400700 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c60fbd350 0x559c60fbf770 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=202b0981524e6695 server_cookie=7a916ab1d7a3b36c in_seq=0 out_seq=0 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.756+0000 7f7409600700 10 client.29380 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7409600700 1 -- 172.21.15.46:0/3553575722 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c60f72810 con 0x559c60fbd350 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7409600700 10 client.29380 handle_command_reply: tid=0 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 client.29380 shutdown 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 msgr2=0x559c60fad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c60fad580 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7418011e90 tx=0x7f7418014600 comp rx=0 tx=0).stop 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 shutdown_connections 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c60fbd350 0x559c60fbf770 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c60fad580 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60facb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/3553575722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7409600700 0 client.29380 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.757+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 >> 172.21.15.46:0/3553575722 conn(0x559c60f01210 msgr2=0x559c60f9d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.758+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 shutdown_connections 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.758+0000 7f7439fd1b80 1 -- 172.21.15.46:0/3553575722 wait complete. 2024-05-08T23:17:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.758+0000 7f7439fd1b80 20 client.29380 trim_cache size 0 max 16384 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.773+0000 7f7439fd1b80 1 Processor -- start 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 -- start start 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fa3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa9720 0x559c60fad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60f98a90 con 0x559c60fa3890 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60fc4940 con 0x559c60fa9720 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60fba980 con 0x559c60fa8e00 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f741ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f741ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33650/0 (socket says 172.21.15.46:33650) 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f741ea00700 1 -- 172.21.15.46:0/2361033422 learned_addr learned my addr 172.21.15.46:0/2361033422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f741f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fa3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.774+0000 7f741fe00700 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa9720 0x559c60fad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 -- 172.21.15.46:0/2361033422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa9720 msgr2=0x559c60fad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa9720 0x559c60fad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 -- 172.21.15.46:0/2361033422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 msgr2=0x559c60fa3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fa3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 -- 172.21.15.46:0/2361033422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c60fa3f40 con 0x559c60fa8e00 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f741ea00700 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f740c006010 tx=0x7f740c0ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffec4fd2672ba2ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f7415600700 1 -- 172.21.15.46:0/2361033422 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c09ab10 con 0x559c60fa8e00 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f7415600700 1 -- 172.21.15.46:0/2361033422 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740c09b8b0 con 0x559c60fa8e00 2024-05-08T23:17:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.775+0000 7f7415600700 1 -- 172.21.15.46:0/2361033422 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c0a10b0 con 0x559c60fa8e00 2024-05-08T23:17:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 -- 172.21.15.46:0/2361033422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 msgr2=0x559c60f9dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f740c006010 tx=0x7f740c0ce040 comp rx=0 tx=0).stop 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 -- 172.21.15.46:0/2361033422 shutdown_connections 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa9720 0x559c60fad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fa3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/2361033422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa8e00 0x559c60f9dd20 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 -- 172.21.15.46:0/2361033422 >> 172.21.15.46:0/2361033422 conn(0x559c60f01210 msgr2=0x559c60fab3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 -- 172.21.15.46:0/2361033422 shutdown_connections 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.776+0000 7f7439fd1b80 1 -- 172.21.15.46:0/2361033422 wait complete. 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 -- start start 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 Processor -- start 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fb2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c61012810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60fb2f60 con 0x559c60fa3890 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c61012d50 con 0x559c60fa8e00 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c61012e90 con 0x559c60fa9720 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.777+0000 7f7439fd1b80 10 client.0 _ll_register_callbacks cb 0x7f74180117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fb2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48064/0 (socket says 172.21.15.46:48064) 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 -- 172.21.15.46:0/749617714 learned_addr learned my addr 172.21.15.46:0/749617714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741fe00700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c61012810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 -- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 msgr2=0x559c61012810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c61012810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 -- 172.21.15.46:0/749617714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 msgr2=0x559c60fb2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fb2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 -- 172.21.15.46:0/749617714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74201a5fa0 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741fe00700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c61012810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741ea00700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fb2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f741f400700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7410012ee0 tx=0x7f7410009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4262f78fd8b269 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.778+0000 7f7409600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410003b20 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74100044e0 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7410003c80 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7410004000 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7439fd1b80 10 client.19269 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559c60eb9560 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.779+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74100044e0 con 0x559c60fa3890 2024-05-08T23:17:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7410039020 con 0x559c60fa3890 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 10 client.19269 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 10 client.19269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 20 client.19269 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c61014550 0x559c61016940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 4 client.19269 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c60eebec0 con 0x559c61014550 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f741ea00700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c61014550 0x559c61016940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f741ea00700 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c61014550 0x559c61016940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63b5b6b6cf07e2c6 server_cookie=ac992a6d44fec8bd in_seq=0 out_seq=0 2024-05-08T23:17:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.780+0000 7f7409600700 10 client.19269 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7409600700 1 -- 172.21.15.46:0/749617714 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c60eebec0 con 0x559c61014550 2024-05-08T23:17:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7409600700 10 client.19269 handle_command_reply: tid=0 2024-05-08T23:17:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 client.19269 shutdown 2024-05-08T23:17:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 msgr2=0x559c60fb2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7410012ee0 tx=0x7f7410009800 comp rx=0 tx=0).stop 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 shutdown_connections 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x559c61014550 0x559c61016940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c60fa9720 0x559c61012810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c60fa3890 0x559c60fb2340 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 --2- 172.21.15.46:0/749617714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c60fa8e00 0x559c60fb2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7409600700 0 client.19269 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 >> 172.21.15.46:0/749617714 conn(0x559c60f01210 msgr2=0x559c60fa7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 shutdown_connections 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 1 -- 172.21.15.46:0/749617714 wait complete. 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.781+0000 7f7439fd1b80 20 client.19269 trim_cache size 0 max 16384 2024-05-08T23:17:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 msgr2=0x7f74040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 0x7f74040a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7410009e40 tx=0x7f7410009260 comp rx=0 tx=0).stop 2024-05-08T23:17:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 msgr2=0x7f7420100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c005740 comp rx=0 tx=0).stop 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 shutdown_connections 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f74040a1310 0x7f74040a37d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f742010b3c0 0x7f7420100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7420101fa0 0x7f74200ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 --2- 172.21.15.46:0/2520571700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74201028c0 0x7f7420100130 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 >> 172.21.15.46:0/2520571700 conn(0x7f74200fb760 msgr2=0x7f74200fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 shutdown_connections 2024-05-08T23:17:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:36.782+0000 7f7426400700 1 -- 172.21.15.46:0/2520571700 wait complete. 2024-05-08T23:17:36.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:17:36.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1202167 8.99 MB/sec execute 598 sec latency 22.874 ms 2024-05-08T23:17:37.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 -- 172.21.15.46:0/3593372420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c101080 msgr2=0x7f873c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 --2- 172.21.15.46:0/3593372420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c101080 0x7f873c101460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b0a0 tx=0x7f872c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 -- 172.21.15.46:0/3593372420 shutdown_connections 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 --2- 172.21.15.46:0/3593372420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 --2- 172.21.15.46:0/3593372420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c101080 0x7f873c101460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.031+0000 7f8741e00700 1 --2- 172.21.15.46:0/3593372420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101a30 0x7f873c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.032+0000 7f8741e00700 1 -- 172.21.15.46:0/3593372420 >> 172.21.15.46:0/3593372420 conn(0x7f873c0fc8e0 msgr2=0x7f873c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.032+0000 7f8741e00700 1 -- 172.21.15.46:0/3593372420 shutdown_connections 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.032+0000 7f8741e00700 1 -- 172.21.15.46:0/3593372420 wait complete. 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 Processor -- start 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 -- start start 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c19dd90 0x7f873c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19e810 con 0x7f873c101080 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19e980 con 0x7f873c19dd90 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.033+0000 7f8741e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c19eb40 con 0x7f873c1061a0 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f8732000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48080/0 (socket says 172.21.15.46:48080) 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 -- 172.21.15.46:0/1293583495 learned_addr learned my addr 172.21.15.46:0/1293583495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873b400700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c19dd90 0x7f873c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 -- 172.21.15.46:0/1293583495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 msgr2=0x7f873c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 -- 172.21.15.46:0/1293583495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c19dd90 msgr2=0x7f873c19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c19dd90 0x7f873c19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 -- 172.21.15.46:0/1293583495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009520 con 0x7f873c101080 2024-05-08T23:17:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f8732000700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.034+0000 7f873aa00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b070 tx=0x7f872c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.035+0000 7f8740a00700 1 -- 172.21.15.46:0/1293583495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00e040 con 0x7f873c101080 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.035+0000 7f8740a00700 1 -- 172.21.15.46:0/1293583495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c00be20 con 0x7f873c101080 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.035+0000 7f8740a00700 1 -- 172.21.15.46:0/1293583495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c01b460 con 0x7f873c101080 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.035+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c009d00 con 0x7f873c101080 2024-05-08T23:17:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.035+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c1a6280 con 0x7f873c101080 2024-05-08T23:17:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.038+0000 7f8740a00700 1 -- 172.21.15.46:0/1293583495 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f872c01d470 con 0x7f873c101080 2024-05-08T23:17:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.039+0000 7f875599bb80 1 Processor -- start 2024-05-08T23:17:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.039+0000 7f875599bb80 1 -- start start 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f8740a00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 0x7f87200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbcc5150 con 0x55fbdbd73e00 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbcc2b10 con 0x55fbdbd74720 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f875599bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbcbe400 con 0x55fbdbd6e890 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f8740a00700 1 -- 172.21.15.46:0/1293583495 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f872c0ce190 con 0x7f873c101080 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f873b400700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 0x7f87200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f8732000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48092/0 (socket says 172.21.15.46:48092) 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f873b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f873aa00700 1 -- 172.21.15.46:0/3241382038 learned_addr learned my addr 172.21.15.46:0/3241382038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.040+0000 7f8732000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46752/0 (socket says 172.21.15.46:46752) 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 -- 172.21.15.46:0/3241382038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 msgr2=0x55fbdbd6ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 -- 172.21.15.46:0/3241382038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 msgr2=0x55fbdbd6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 -- 172.21.15.46:0/3241382038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbdbb3f3b0 con 0x55fbdbd73e00 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f8732000700 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873aa00700 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b780 tx=0x7f872c0972a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a6f5a0722ab67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f873b400700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 0x7f87200a37d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8734012840 tx=0x7f8734009a50 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f872be00700 1 -- 172.21.15.46:0/3241382038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c09a820 con 0x55fbdbd73e00 2024-05-08T23:17:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f872be00700 1 -- 172.21.15.46:0/3241382038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c087000 con 0x55fbdbd73e00 2024-05-08T23:17:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.041+0000 7f872be00700 1 -- 172.21.15.46:0/3241382038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0a3340 con 0x55fbdbd73e00 2024-05-08T23:17:37.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 -- 172.21.15.46:0/3241382038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 msgr2=0x55fbdbd741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b780 tx=0x7f872c0972a0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 -- 172.21.15.46:0/3241382038 shutdown_connections 2024-05-08T23:17:37.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd741e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 --2- 172.21.15.46:0/3241382038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd74720 0x55fbdbd6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 -- 172.21.15.46:0/3241382038 >> 172.21.15.46:0/3241382038 conn(0x55fbdbccc210 msgr2=0x55fbdbd68df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 -- 172.21.15.46:0/3241382038 shutdown_connections 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.042+0000 7f875599bb80 1 -- 172.21.15.46:0/3241382038 wait complete. 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.043+0000 7f875599bb80 1 -- start start 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.043+0000 7f875599bb80 1 Processor -- start 2024-05-08T23:17:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd77f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 0x55fbdbd784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbc5a560 con 0x55fbdbd74720 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbc5a200 con 0x55fbdbd73e00 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbb416a0 con 0x55fbdbd6e890 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f875599bb80 10 client.0 _ll_register_callbacks cb 0x55fbdbc63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f873aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f873aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33690/0 (socket says 172.21.15.46:33690) 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f873aa00700 1 -- 172.21.15.46:0/242492930 learned_addr learned my addr 172.21.15.46:0/242492930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.044+0000 7f8732000700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd77f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 -- 172.21.15.46:0/242492930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 msgr2=0x55fbdbd77f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd77f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 -- 172.21.15.46:0/242492930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 msgr2=0x55fbdbd784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873b400700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 0x55fbdbd784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 0x55fbdbd784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 -- 172.21.15.46:0/242492930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724009e20 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873b400700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 0x55fbdbd784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f873aa00700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f872c09ae50 tx=0x7f872c01db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30df38e75aa59ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f8728c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0cb900 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f872c085e40 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.045+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbdbc17270 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872c003e10 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 10 client.29398 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f873c1a6280 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbdbc17270 con 0x55fbdbd6e890 2024-05-08T23:17:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f872c01d7b0 con 0x55fbdbd6e890 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 10 client.29398 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 10 client.29398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 20 client.29398 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbd87a80 0x55fbdbd89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 4 client.29398 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbdbd3d810 con 0x55fbdbd87a80 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.046+0000 7f8732000700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbd87a80 0x55fbdbd89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.047+0000 7f8732000700 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbd87a80 0x55fbdbd89e70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41bdab4a0f7bab6c server_cookie=eb8a1cafc29ddcf8 in_seq=0 out_seq=0 2024-05-08T23:17:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.047+0000 7f8728c00700 10 client.29398 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f8728c00700 1 -- 172.21.15.46:0/242492930 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fbdbd3d810 con 0x55fbdbd87a80 2024-05-08T23:17:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f8728c00700 10 client.29398 handle_command_reply: tid=0 2024-05-08T23:17:37.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 client.29398 shutdown 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 msgr2=0x55fbdbd77a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f872c09ae50 tx=0x7f872c01db10 comp rx=0 tx=0).stop 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 shutdown_connections 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbd87a80 0x55fbdbd89e70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd74720 0x55fbdbd784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd6e890 0x55fbdbd77a50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 --2- 172.21.15.46:0/242492930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd77f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 >> 172.21.15.46:0/242492930 conn(0x55fbdbccc210 msgr2=0x55fbdbd685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.048+0000 7f8728c00700 0 client.29398 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.049+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 shutdown_connections 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.049+0000 7f875599bb80 1 -- 172.21.15.46:0/242492930 wait complete. 2024-05-08T23:17:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.049+0000 7f875599bb80 20 client.29398 trim_cache size 0 max 16384 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.064+0000 7f875599bb80 1 Processor -- start 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.064+0000 7f875599bb80 1 -- start start 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbd73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbd6aa30 con 0x55fbdbd6e890 2024-05-08T23:17:37.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbd65dc0 con 0x55fbdbd73e00 2024-05-08T23:17:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f875599bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbd8f000 con 0x55fbdbd74720 2024-05-08T23:17:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48098/0 (socket says 172.21.15.46:48098) 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873aa00700 1 -- 172.21.15.46:0/2129137688 learned_addr learned my addr 172.21.15.46:0/2129137688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbd73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f8732000700 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 -- 172.21.15.46:0/2129137688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 msgr2=0x55fbdbd68d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd68d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 -- 172.21.15.46:0/2129137688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 msgr2=0x55fbdbd6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.065+0000 7f873b400700 1 -- 172.21.15.46:0/2129137688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbdbaebb60 con 0x55fbdbd74720 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.066+0000 7f873b400700 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbd73230 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8734013fd0 tx=0x7f8734020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8e07ce790d2005e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.066+0000 7f8731200700 1 -- 172.21.15.46:0/2129137688 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734012030 con 0x55fbdbd74720 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.066+0000 7f8731200700 1 -- 172.21.15.46:0/2129137688 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8734012190 con 0x55fbdbd74720 2024-05-08T23:17:37.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.066+0000 7f8731200700 1 -- 172.21.15.46:0/2129137688 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734005690 con 0x55fbdbd74720 2024-05-08T23:17:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- 172.21.15.46:0/2129137688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 msgr2=0x55fbdbd73230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbd73230 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8734013fd0 tx=0x7f8734020000 comp rx=0 tx=0).stop 2024-05-08T23:17:37.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- 172.21.15.46:0/2129137688 shutdown_connections 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbd73230 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd6e890 0x55fbdbd6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 --2- 172.21.15.46:0/2129137688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd73e00 0x55fbdbd68d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- 172.21.15.46:0/2129137688 >> 172.21.15.46:0/2129137688 conn(0x55fbdbccc210 msgr2=0x55fbdbd72b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- 172.21.15.46:0/2129137688 shutdown_connections 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- 172.21.15.46:0/2129137688 wait complete. 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.067+0000 7f875599bb80 1 -- start start 2024-05-08T23:17:37.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 Processor -- start 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 0x55fbdbd7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd7b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbddcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbd7bd90 con 0x55fbdbd73e00 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbddd400 con 0x55fbdbd6e890 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.068+0000 7f875599bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbdbddd540 con 0x55fbdbd74720 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f875599bb80 10 client.0 _ll_register_callbacks cb 0x55fbdbc63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 0x55fbdbd7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 0x55fbdbd7af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46772/0 (socket says 172.21.15.46:46772) 2024-05-08T23:17:37.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873aa00700 1 -- 172.21.15.46:0/4100168314 learned_addr learned my addr 172.21.15.46:0/4100168314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbddcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f8732000700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd7b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 -- 172.21.15.46:0/4100168314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 msgr2=0x55fbdbd7af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 0x55fbdbd7af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 -- 172.21.15.46:0/4100168314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 msgr2=0x55fbdbd7b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd7b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.069+0000 7f873b400700 1 -- 172.21.15.46:0/4100168314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873401a040 con 0x55fbdbd74720 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f873b400700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbddcec0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8734009d10 tx=0x7f8734005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cab64e3de3352c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f8728c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734003890 con 0x55fbdbd74720 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f872c087000 con 0x55fbdbd74720 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f87340039f0 con 0x55fbdbd74720 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8734003bb0 con 0x55fbdbd74720 2024-05-08T23:17:37.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f875599bb80 10 client.29404 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.070+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fbdbc84560 con 0x55fbdbd74720 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c087000 con 0x55fbdbd74720 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f873402f020 con 0x55fbdbd74720 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 10 client.29404 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 10 client.29404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 20 client.29404 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbddeb80 0x55fbdbde0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 4 client.29404 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fbdbcb6ec0 con 0x55fbdbddeb80 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.071+0000 7f873aa00700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbddeb80 0x55fbdbde0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f873aa00700 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbddeb80 0x55fbdbde0fa0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6265c846871e78b server_cookie=66c9d2b57d2b7aa5 in_seq=0 out_seq=0 2024-05-08T23:17:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f8728c00700 10 client.29404 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f8728c00700 1 -- 172.21.15.46:0/4100168314 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fbdbcb6ec0 con 0x55fbdbddeb80 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f8728c00700 10 client.29404 handle_command_reply: tid=0 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 client.29404 shutdown 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 msgr2=0x55fbdbddcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbddcec0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8734009d10 tx=0x7f8734005500 comp rx=0 tx=0).stop 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 shutdown_connections 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fbdbd74720 0x55fbdbddcec0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fbdbd6e890 0x55fbdbd7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fbdbddeb80 0x55fbdbde0fa0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 --2- 172.21.15.46:0/4100168314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbdbd73e00 0x55fbdbd7b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f8728c00700 0 client.29404 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.072+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 >> 172.21.15.46:0/4100168314 conn(0x55fbdbccc210 msgr2=0x55fbdbd75440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 shutdown_connections 2024-05-08T23:17:37.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f875599bb80 1 -- 172.21.15.46:0/4100168314 wait complete. 2024-05-08T23:17:37.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f875599bb80 20 client.29404 trim_cache size 0 max 16384 2024-05-08T23:17:37.076 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 msgr2=0x7f87200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 0x7f87200a37d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8734012840 tx=0x7f8734009a50 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 msgr2=0x7f873c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b070 tx=0x7f872c007870 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 shutdown_connections 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f873c19dd90 0x7f873c19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87200a1310 0x7f87200a37d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f873c101080 0x7f873c199500 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.073+0000 7f8741e00700 1 --2- 172.21.15.46:0/1293583495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f873c1061a0 0x7f873c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.074+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 >> 172.21.15.46:0/1293583495 conn(0x7f873c0fc8e0 msgr2=0x7f873c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.074+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 shutdown_connections 2024-05-08T23:17:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.074+0000 7f8741e00700 1 -- 172.21.15.46:0/1293583495 wait complete. 2024-05-08T23:17:37.089 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:36 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3647073288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:37.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:17:37.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:36 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3647073288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:37.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:36 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3647073288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 -- 172.21.15.46:0/88766723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60781061a0 msgr2=0x7f607810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 --2- 172.21.15.46:0/88766723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60781061a0 0x7f607810dd20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f607000b0a0 tx=0x7f607000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 -- 172.21.15.46:0/88766723 shutdown_connections 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 --2- 172.21.15.46:0/88766723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60781061a0 0x7f607810dd20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 --2- 172.21.15.46:0/88766723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6078101080 0x7f6078101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 --2- 172.21.15.46:0/88766723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078101a30 0x7f6078105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 -- 172.21.15.46:0/88766723 >> 172.21.15.46:0/88766723 conn(0x7f60780fc8e0 msgr2=0x7f60780fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 -- 172.21.15.46:0/88766723 shutdown_connections 2024-05-08T23:17:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.319+0000 7f607f600700 1 -- 172.21.15.46:0/88766723 wait complete. 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.320+0000 7f607f600700 1 Processor -- start 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 -- start start 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6078101080 0x7f60780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60781061a0 0x7f60780737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781a5bf0 con 0x7f6078101080 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781a5d60 con 0x7f60781061a0 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781a5f20 con 0x7f6078073d20 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6078101080 0x7f60780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33732/0 (socket says 172.21.15.46:33732) 2024-05-08T23:17:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f607d600700 1 -- 172.21.15.46:0/3338447275 learned_addr learned my addr 172.21.15.46:0/3338447275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.321+0000 7f6077e00700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60781061a0 0x7f60780737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 -- 172.21.15.46:0/3338447275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60781061a0 msgr2=0x7f60780737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60781061a0 0x7f60780737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 -- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6078101080 msgr2=0x7f60780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6078101080 0x7f60780732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 -- 172.21.15.46:0/3338447275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009d00 con 0x7f6078073d20 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607d600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f607000b070 tx=0x7f607000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f6075600700 1 -- 172.21.15.46:0/3338447275 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000e040 con 0x7f6078073d20 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60781a61a0 con 0x7f6078073d20 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f6075600700 1 -- 172.21.15.46:0/3338447275 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607001d480 con 0x7f6078073d20 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.322+0000 7f6075600700 1 -- 172.21.15.46:0/3338447275 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700052a0 con 0x7f6078073d20 2024-05-08T23:17:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.323+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781a6570 con 0x7f6078073d20 2024-05-08T23:17:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.326+0000 7f6075600700 1 -- 172.21.15.46:0/3338447275 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f607001d5f0 con 0x7f6078073d20 2024-05-08T23:17:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.327+0000 7f6093143b80 1 Processor -- start 2024-05-08T23:17:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.327+0000 7f6075600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 0x7f60600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.327+0000 7f6093143b80 1 -- start start 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.327+0000 7f6075600700 1 -- 172.21.15.46:0/3338447275 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f60700ce100 con 0x7f6078073d20 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6077e00700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 0x7f60600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2574cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc24cb150 con 0x563bc257a720 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc24c8b10 con 0x563bc2574890 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6093143b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc24c4400 con 0x563bc2579e00 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33746/0 (socket says 172.21.15.46:33746) 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 -- 172.21.15.46:0/2242387547 learned_addr learned my addr 172.21.15.46:0/2242387547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607d600700 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2574cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f6077e00700 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 -- 172.21.15.46:0/2242387547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 msgr2=0x563bc2574cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2574cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 -- 172.21.15.46:0/2242387547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 msgr2=0x563bc2574350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.328+0000 7f607cc00700 1 -- 172.21.15.46:0/2242387547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc23453b0 con 0x563bc2579e00 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.329+0000 7f6077e00700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 0x7f60600a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f60780749d0 tx=0x7f6064011440 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.329+0000 7f607cc00700 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d80 tx=0x7f606c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2775326a610531 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.329+0000 7f6069c00700 1 -- 172.21.15.46:0/2242387547 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c004230 con 0x563bc2579e00 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.329+0000 7f6069c00700 1 -- 172.21.15.46:0/2242387547 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c004390 con 0x563bc2579e00 2024-05-08T23:17:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.329+0000 7f6069c00700 1 -- 172.21.15.46:0/2242387547 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c019820 con 0x563bc2579e00 2024-05-08T23:17:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 -- 172.21.15.46:0/2242387547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 msgr2=0x563bc257a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d80 tx=0x7f606c011720 comp rx=0 tx=0).stop 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 -- 172.21.15.46:0/2242387547 shutdown_connections 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2574cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2579e00 0x563bc257a1e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 --2- 172.21.15.46:0/2242387547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 -- 172.21.15.46:0/2242387547 >> 172.21.15.46:0/2242387547 conn(0x563bc24d2210 msgr2=0x563bc256edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 -- 172.21.15.46:0/2242387547 shutdown_connections 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.330+0000 7f6093143b80 1 -- 172.21.15.46:0/2242387547 wait complete. 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.331+0000 7f6093143b80 1 -- start start 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.331+0000 7f6093143b80 1 Processor -- start 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc257e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc257a720 0x563bc257e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc2460560 con 0x563bc2579e00 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc2460200 con 0x563bc257a720 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc23476a0 con 0x563bc2574890 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6093143b80 10 client.0 _ll_register_callbacks cb 0x563bc2469b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33760/0 (socket says 172.21.15.46:33760) 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 -- 172.21.15.46:0/1439349301 learned_addr learned my addr 172.21.15.46:0/1439349301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607d600700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc257a720 0x563bc257e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f6077e00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc257e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 -- 172.21.15.46:0/1439349301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc257a720 msgr2=0x563bc257e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc257a720 0x563bc257e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 -- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 msgr2=0x563bc257e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc257e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.332+0000 7f607cc00700 1 -- 172.21.15.46:0/1439349301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c0099e0 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f607cc00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a644385ff894237d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f605e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c005450 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f606c005ce0 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bc241d270 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606c009ed0 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.333+0000 7f6093143b80 10 client.29416 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563bc22f1b60 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bc241d270 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f606c021040 con 0x563bc2574890 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 10 client.29416 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 10 client.29416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 20 client.29416 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc258e160 0x563bc2590580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 4 client.29416 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bc2543810 con 0x563bc258e160 2024-05-08T23:17:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.334+0000 7f6077e00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc258e160 0x563bc2590580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.335+0000 7f6077e00700 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc258e160 0x563bc2590580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c859a7f9caf8e81 server_cookie=5f51f1627c44b2 in_seq=0 out_seq=0 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.335+0000 7f605e000700 10 client.29416 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.335+0000 7f605e000700 1 -- 172.21.15.46:0/1439349301 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563bc2543810 con 0x563bc258e160 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.335+0000 7f605e000700 10 client.29416 handle_command_reply: tid=0 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.335+0000 7f6093143b80 1 client.29416 shutdown 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 msgr2=0x563bc257dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c003960 comp rx=0 tx=0).stop 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 shutdown_connections 2024-05-08T23:17:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc257a720 0x563bc257e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc257dad0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc258e160 0x563bc2590580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 --2- 172.21.15.46:0/1439349301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc257e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 >> 172.21.15.46:0/1439349301 conn(0x563bc24d2210 msgr2=0x563bc256e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f605e000700 0 client.29416 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 shutdown_connections 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 1 -- 172.21.15.46:0/1439349301 wait complete. 2024-05-08T23:17:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.336+0000 7f6093143b80 20 client.29416 trim_cache size 0 max 16384 2024-05-08T23:17:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.352+0000 7f6093143b80 1 Processor -- start 2024-05-08T23:17:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.352+0000 7f6093143b80 1 -- start start 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.352+0000 7f6093143b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.352+0000 7f6093143b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2579e00 0x563bc2570640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.352+0000 7f6093143b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f6093143b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc25957b0 con 0x563bc257a720 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f6093143b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc2569800 con 0x563bc2579e00 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f6093143b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc25784b0 con 0x563bc2574890 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33768/0 (socket says 172.21.15.46:33768) 2024-05-08T23:17:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 -- 172.21.15.46:0/142775238 learned_addr learned my addr 172.21.15.46:0/142775238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607d600700 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f6077e00700 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2579e00 0x563bc2570640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 -- 172.21.15.46:0/142775238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2579e00 msgr2=0x563bc2570640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2579e00 0x563bc2570640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 -- 172.21.15.46:0/142775238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 msgr2=0x563bc2574c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 -- 172.21.15.46:0/142775238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc248a560 con 0x563bc2574890 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.353+0000 7f607cc00700 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f606c0095f0 tx=0x7f606c0194a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fed01ed42a6d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.354+0000 7f606a600700 1 -- 172.21.15.46:0/142775238 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c019860 con 0x563bc2574890 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.354+0000 7f606a600700 1 -- 172.21.15.46:0/142775238 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c003d80 con 0x563bc2574890 2024-05-08T23:17:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.354+0000 7f606a600700 1 -- 172.21.15.46:0/142775238 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0055e0 con 0x563bc2574890 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.354+0000 7f6093143b80 1 -- 172.21.15.46:0/142775238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 msgr2=0x563bc2574350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.354+0000 7f6093143b80 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f606c0095f0 tx=0x7f606c0194a0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 -- 172.21.15.46:0/142775238 shutdown_connections 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc257a720 0x563bc2574c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc2574890 0x563bc2574350 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 --2- 172.21.15.46:0/142775238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2579e00 0x563bc2570640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 -- 172.21.15.46:0/142775238 >> 172.21.15.46:0/142775238 conn(0x563bc24d2210 msgr2=0x563bc257c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 -- 172.21.15.46:0/142775238 shutdown_connections 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 -- 172.21.15.46:0/142775238 wait complete. 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 -- start start 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.355+0000 7f6093143b80 1 Processor -- start 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2585500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc2581110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc2581d10 con 0x563bc2579e00 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc2581e80 con 0x563bc2574890 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc25e3670 con 0x563bc257a720 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6093143b80 10 client.0 _ll_register_callbacks cb 0x563bc2469b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f607d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f607cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2585500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f607d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33772/0 (socket says 172.21.15.46:33772) 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f607d600700 1 -- 172.21.15.46:0/3567794203 learned_addr learned my addr 172.21.15.46:0/3567794203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.356+0000 7f6077e00700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc2581110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 -- 172.21.15.46:0/3567794203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 msgr2=0x563bc2585500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2585500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 -- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 msgr2=0x563bc2581110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc2581110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 -- 172.21.15.46:0/3567794203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bc22f1b60 con 0x563bc257a720 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f6077e00700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc2581110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f607d600700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f607009ae20 tx=0x7f6070009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e60f7364689b040 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f605e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607009bea0 con 0x563bc257a720 2024-05-08T23:17:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f607009b590 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f606c004390 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607001dae0 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f6093143b80 10 client.29422 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.357+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563bc2349cd0 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60700b9030 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f607009b0f0 con 0x563bc257a720 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 10 client.29422 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 10 client.29422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 20 client.29422 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc25e4bf0 0x563bc25e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 4 client.29422 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:37.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563bc24bcec0 con 0x563bc25e4bf0 2024-05-08T23:17:37.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.358+0000 7f607cc00700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc25e4bf0 0x563bc25e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f607cc00700 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc25e4bf0 0x563bc25e6fe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c466dd32e8f0220b server_cookie=3fc53df0ddb6ff60 in_seq=0 out_seq=0 2024-05-08T23:17:37.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f605e000700 10 client.29422 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:37.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f605e000700 1 -- 172.21.15.46:0/3567794203 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563bc24bcec0 con 0x563bc25e4bf0 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f605e000700 10 client.29422 handle_command_reply: tid=0 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 client.29422 shutdown 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 msgr2=0x563bc2581650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f607009ae20 tx=0x7f6070009810 comp rx=0 tx=0).stop 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 shutdown_connections 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563bc257a720 0x563bc2581650 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc2574890 0x563bc2585500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x563bc25e4bf0 0x563bc25e6fe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.359+0000 7f6093143b80 1 --2- 172.21.15.46:0/3567794203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bc2579e00 0x563bc2581110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.360+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 >> 172.21.15.46:0/3567794203 conn(0x563bc24d2210 msgr2=0x563bc2578bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.360+0000 7f605e000700 0 client.29422 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.360+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 shutdown_connections 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.360+0000 7f6093143b80 1 -- 172.21.15.46:0/3567794203 wait complete. 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.360+0000 7f6093143b80 20 client.29422 trim_cache size 0 max 16384 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 msgr2=0x7f60600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 0x7f60600a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f60780749d0 tx=0x7f6064011440 comp rx=0 tx=0).stop 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 msgr2=0x7f60781a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f607000b070 tx=0x7f607000bda0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 shutdown_connections 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6078073d20 0x7f60781a5620 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f60600a1290 0x7f60600a3750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6078101080 0x7f60780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 --2- 172.21.15.46:0/3338447275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60781061a0 0x7f60780737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 >> 172.21.15.46:0/3338447275 conn(0x7f60780fc8e0 msgr2=0x7f60780febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 shutdown_connections 2024-05-08T23:17:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.361+0000 7f607f600700 1 -- 172.21.15.46:0/3338447275 wait complete. 2024-05-08T23:17:37.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:17:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 -- 172.21.15.46:0/3576573840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13681028c0 msgr2=0x7f136810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 --2- 172.21.15.46:0/3576573840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13681028c0 0x7f136810ada0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1354009990 tx=0x7f1354009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 -- 172.21.15.46:0/3576573840 shutdown_connections 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 --2- 172.21.15.46:0/3576573840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136810b3c0 0x7f136810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 --2- 172.21.15.46:0/3576573840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1368101fa0 0x7f1368102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 --2- 172.21.15.46:0/3576573840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13681028c0 0x7f136810ada0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 -- 172.21.15.46:0/3576573840 >> 172.21.15.46:0/3576573840 conn(0x7f13680fb760 msgr2=0x7f13680fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 -- 172.21.15.46:0/3576573840 shutdown_connections 2024-05-08T23:17:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.599+0000 7f136ec00700 1 -- 172.21.15.46:0/3576573840 wait complete. 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.600+0000 7f136ec00700 1 Processor -- start 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.600+0000 7f136ec00700 1 -- start start 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 0x7f13681994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13681028c0 0x7f1368199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 0x7f136819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e3e0 con 0x7f136810b3c0 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e550 con 0x7f13681028c0 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819e710 con 0x7f1368101fa0 2024-05-08T23:17:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 0x7f13681994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 0x7f136819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 0x7f13681994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33786/0 (socket says 172.21.15.46:33786) 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f1367e00700 1 -- 172.21.15.46:0/3613537756 learned_addr learned my addr 172.21.15.46:0/3613537756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f1367400700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13681028c0 0x7f1368199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 -- 172.21.15.46:0/3613537756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 msgr2=0x7f13681994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 0x7f13681994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 -- 172.21.15.46:0/3613537756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13681028c0 msgr2=0x7f1368199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13681028c0 0x7f1368199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.601+0000 7f136cc00700 1 -- 172.21.15.46:0/3613537756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358009520 con 0x7f136810b3c0 2024-05-08T23:17:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f136cc00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 0x7f136819dda0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f136000efc0 tx=0x7f136000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f1364c00700 1 -- 172.21.15.46:0/3613537756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360018070 con 0x7f136810b3c0 2024-05-08T23:17:37.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f1364c00700 1 -- 172.21.15.46:0/3613537756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136000f040 con 0x7f136810b3c0 2024-05-08T23:17:37.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13540095f0 con 0x7f136810b3c0 2024-05-08T23:17:37.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f1364c00700 1 -- 172.21.15.46:0/3613537756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136000cd40 con 0x7f136810b3c0 2024-05-08T23:17:37.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.602+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a6050 con 0x7f136810b3c0 2024-05-08T23:17:37.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.606+0000 7f1364c00700 1 -- 172.21.15.46:0/3613537756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1360003a00 con 0x7f136810b3c0 2024-05-08T23:17:37.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1382771b80 1 Processor -- start 2024-05-08T23:17:37.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1364c00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 0x7f134c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1382771b80 1 -- start start 2024-05-08T23:17:37.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1364c00700 1 -- 172.21.15.46:0/3613537756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f13600ccc40 con 0x7f136810b3c0 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1367400700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 0x7f134c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.608+0000 7f1382771b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1382771b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1382771b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1382771b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda721150 con 0x55fcda7ca890 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1382771b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda71eb10 con 0x55fcda7d0720 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1382771b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda71a400 con 0x55fcda7cfe00 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33788/0 (socket says 172.21.15.46:33788) 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 -- 172.21.15.46:0/1565698876 learned_addr learned my addr 172.21.15.46:0/1565698876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f136cc00700 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367400700 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 -- 172.21.15.46:0/1565698876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 msgr2=0x55fcda7ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 -- 172.21.15.46:0/1565698876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 msgr2=0x55fcda7cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.609+0000 7f1367e00700 1 -- 172.21.15.46:0/1565698876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcda59b3b0 con 0x55fcda7cfe00 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.610+0000 7f1367400700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 0x7f134c0a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f136819aa10 tx=0x7f1354005320 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.610+0000 7f1367e00700 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f13580114c0 tx=0x7f13580117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a7fe6867c27aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.610+0000 7f135d200700 1 -- 172.21.15.46:0/1565698876 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358019630 con 0x55fcda7cfe00 2024-05-08T23:17:37.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.610+0000 7f135d200700 1 -- 172.21.15.46:0/1565698876 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358019c50 con 0x55fcda7cfe00 2024-05-08T23:17:37.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.610+0000 7f135d200700 1 -- 172.21.15.46:0/1565698876 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800fc80 con 0x55fcda7cfe00 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 -- 172.21.15.46:0/1565698876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 msgr2=0x55fcda7d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f13580114c0 tx=0x7f13580117d0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 -- 172.21.15.46:0/1565698876 shutdown_connections 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7cfe00 0x55fcda7d01e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 --2- 172.21.15.46:0/1565698876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 -- 172.21.15.46:0/1565698876 >> 172.21.15.46:0/1565698876 conn(0x55fcda728210 msgr2=0x55fcda7c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 -- 172.21.15.46:0/1565698876 shutdown_connections 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.611+0000 7f1382771b80 1 -- 172.21.15.46:0/1565698876 wait complete. 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 -- start start 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 Processor -- start 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda6b6560 con 0x55fcda7cfe00 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda6b6200 con 0x55fcda7ca890 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda59d6a0 con 0x55fcda7d0720 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.612+0000 7f1382771b80 10 client.0 _ll_register_callbacks cb 0x55fcda6bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f136cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46862/0 (socket says 172.21.15.46:46862) 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 -- 172.21.15.46:0/700727231 learned_addr learned my addr 172.21.15.46:0/700727231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367400700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 -- 172.21.15.46:0/700727231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 msgr2=0x55fcda7d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 -- 172.21.15.46:0/700727231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 msgr2=0x55fcda7d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 -- 172.21.15.46:0/700727231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1358009c90 con 0x55fcda7ca890 2024-05-08T23:17:37.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f136cc00700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1367e00700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c30 tx=0x7f135800f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a9cc09d281e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.613+0000 7f1351600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:17:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358019630 con 0x55fcda7ca890 2024-05-08T23:17:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcda673270 con 0x55fcda7ca890 2024-05-08T23:17:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1358011e40 con 0x55fcda7ca890 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1358019790 con 0x55fcda7ca890 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1382771b80 10 client.29342 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.614+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fcda547b60 con 0x55fcda7ca890 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcda673270 con 0x55fcda7ca890 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1358004850 con 0x55fcda7ca890 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 10 client.29342 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 10 client.29342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 20 client.29342 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda7e4040 0x55fcda7e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 4 client.29342 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcda799810 con 0x55fcda7e4040 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.615+0000 7f1367400700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda7e4040 0x55fcda7e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.616+0000 7f1367400700 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda7e4040 0x55fcda7e6460 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff0766aa81c12e1 server_cookie=2079860612b67458 in_seq=0 out_seq=0 2024-05-08T23:17:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.616+0000 7f1351600700 10 client.29342 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1351600700 1 -- 172.21.15.46:0/700727231 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcda799810 con 0x55fcda7e4040 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1351600700 10 client.29342 handle_command_reply: tid=0 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 client.29342 shutdown 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 msgr2=0x55fcda7d3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c30 tx=0x7f135800f680 comp rx=0 tx=0).stop 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 shutdown_connections 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7ca890 0x55fcda7d3ab0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda7e4040 0x55fcda7e6460 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 --2- 172.21.15.46:0/700727231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 >> 172.21.15.46:0/700727231 conn(0x55fcda728210 msgr2=0x55fcda7c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.617+0000 7f1351600700 0 client.29342 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.618+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 shutdown_connections 2024-05-08T23:17:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.618+0000 7f1382771b80 1 -- 172.21.15.46:0/700727231 wait complete. 2024-05-08T23:17:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.618+0000 7f1382771b80 20 client.29342 trim_cache size 0 max 16384 2024-05-08T23:17:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.633+0000 7f1382771b80 1 Processor -- start 2024-05-08T23:17:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.633+0000 7f1382771b80 1 -- start start 2024-05-08T23:17:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 0x55fcda7c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda7d5120 con 0x55fcda7ca890 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda7c1dc0 con 0x55fcda7cfe00 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1382771b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda7c6e50 con 0x55fcda7d0720 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1367400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 0x55fcda7c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1367400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 0x55fcda7c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46870/0 (socket says 172.21.15.46:46870) 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1367400700 1 -- 172.21.15.46:0/3520537097 learned_addr learned my addr 172.21.15.46:0/3520537097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f1367e00700 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 -- 172.21.15.46:0/3520537097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 msgr2=0x55fcda7c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 0x55fcda7c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 -- 172.21.15.46:0/3520537097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 msgr2=0x55fcda7e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.634+0000 7f136cc00700 1 -- 172.21.15.46:0/3520537097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcda6e0560 con 0x55fcda7d0720 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.635+0000 7f136cc00700 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f13600c9f20 tx=0x7f1360009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd56cd79665c7c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.635+0000 7f135dc00700 1 -- 172.21.15.46:0/3520537097 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600992d0 con 0x55fcda7d0720 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.635+0000 7f135dc00700 1 -- 172.21.15.46:0/3520537097 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1360004ad0 con 0x55fcda7d0720 2024-05-08T23:17:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.635+0000 7f135dc00700 1 -- 172.21.15.46:0/3520537097 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600a18d0 con 0x55fcda7d0720 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- 172.21.15.46:0/3520537097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 msgr2=0x55fcda7d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f13600c9f20 tx=0x7f1360009f90 comp rx=0 tx=0).stop 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- 172.21.15.46:0/3520537097 shutdown_connections 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7d0720 0x55fcda7d4350 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7ca890 0x55fcda7e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 --2- 172.21.15.46:0/3520537097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7cfe00 0x55fcda7c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- 172.21.15.46:0/3520537097 >> 172.21.15.46:0/3520537097 conn(0x55fcda728210 msgr2=0x55fcda7cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- 172.21.15.46:0/3520537097 shutdown_connections 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- 172.21.15.46:0/3520537097 wait complete. 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.636+0000 7f1382771b80 1 -- start start 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 Processor -- start 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7d76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda7d7dd0 con 0x55fcda7cfe00 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda7d7f40 con 0x55fcda7d0720 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcda839550 con 0x55fcda7ca890 2024-05-08T23:17:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1382771b80 10 client.0 _ll_register_callbacks cb 0x55fcda6bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f136cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7d76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33822/0 (socket says 172.21.15.46:33822) 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1367e00700 1 -- 172.21.15.46:0/4079740986 learned_addr learned my addr 172.21.15.46:0/4079740986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.637+0000 7f1367400700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 -- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 msgr2=0x55fcda7d76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7d76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 -- 172.21.15.46:0/4079740986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 msgr2=0x55fcda7d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 -- 172.21.15.46:0/4079740986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a6050 con 0x55fcda7ca890 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1367e00700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1358001eb0 tx=0x7f1358009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf22c3f58a56585c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1351600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358019840 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1358011e40 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.638+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1358019c50 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1358009350 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 10 client.29440 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fcda59fcd0 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358019c50 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1358003710 con 0x55fcda7ca890 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 10 client.29440 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 10 client.29440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 20 client.29440 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda83a9d0 0x55fcda83cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 4 client.29440 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.639+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fcda712ec0 con 0x55fcda83a9d0 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1367400700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda83a9d0 0x55fcda83cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1367400700 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda83a9d0 0x55fcda83cdc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f7d976b9ef3522 server_cookie=749a0e0909b86973 in_seq=0 out_seq=0 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1351600700 10 client.29440 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1351600700 1 -- 172.21.15.46:0/4079740986 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fcda712ec0 con 0x55fcda83a9d0 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1351600700 10 client.29440 handle_command_reply: tid=0 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.640+0000 7f1382771b80 1 client.29440 shutdown 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 msgr2=0x55fcda7db520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1358001eb0 tx=0x7f1358009260 comp rx=0 tx=0).stop 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 shutdown_connections 2024-05-08T23:17:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fcda7d0720 0x55fcda7d76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcda7ca890 0x55fcda7db520 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55fcda83a9d0 0x55fcda83cdc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1351600700 0 client.29440 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 --2- 172.21.15.46:0/4079740986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fcda7cfe00 0x55fcda7d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 >> 172.21.15.46:0/4079740986 conn(0x55fcda728210 msgr2=0x55fcda7ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 shutdown_connections 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 1 -- 172.21.15.46:0/4079740986 wait complete. 2024-05-08T23:17:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.641+0000 7f1382771b80 20 client.29440 trim_cache size 0 max 16384 2024-05-08T23:17:37.646 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 msgr2=0x7f134c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 0x7f134c0a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f136819aa10 tx=0x7f1354005320 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 msgr2=0x7f136819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 0x7f136819dda0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f136000efc0 tx=0x7f136000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 shutdown_connections 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f134c0a1290 0x7f134c0a3750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136810b3c0 0x7f136819dda0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1368101fa0 0x7f13681994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 --2- 172.21.15.46:0/3613537756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13681028c0 0x7f1368199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 >> 172.21.15.46:0/3613537756 conn(0x7f13680fb760 msgr2=0x7f13680fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 shutdown_connections 2024-05-08T23:17:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:37.642+0000 7f136ec00700 1 -- 172.21.15.46:0/3613537756 wait complete. 2024-05-08T23:17:37.923 INFO:tasks.workunit.client.0.smithi046.stdout: 1 1204089 8.99 MB/sec execute 599 sec latency 18.592 ms 2024-05-08T23:17:38.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:37 smithi046 ceph-mon[36898]: pgmap v431: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-05-08T23:17:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:37 smithi086 ceph-mon[78610]: pgmap v431: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-05-08T23:17:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:37 smithi053 ceph-mon[77587]: pgmap v431: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-05-08T23:17:38.663 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:17:38.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 -- 172.21.15.46:0/3912381340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d0101080 msgr2=0x7f32d0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 --2- 172.21.15.46:0/3912381340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d0101080 0x7f32d0101460 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 -- 172.21.15.46:0/3912381340 shutdown_connections 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 --2- 172.21.15.46:0/3912381340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d01061a0 0x7f32d010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 --2- 172.21.15.46:0/3912381340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d0101080 0x7f32d0101460 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 --2- 172.21.15.46:0/3912381340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d0101a30 0x7f32d0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 -- 172.21.15.46:0/3912381340 >> 172.21.15.46:0/3912381340 conn(0x7f32d00fc8e0 msgr2=0x7f32d00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 -- 172.21.15.46:0/3912381340 shutdown_connections 2024-05-08T23:17:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.905+0000 7f32d5000700 1 -- 172.21.15.46:0/3912381340 wait complete. 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.906+0000 7f32d5000700 1 Processor -- start 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 -- start start 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d0101080 0x7f32d01995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 0x7f32d019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019e930 con 0x7f32d019deb0 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019eaa0 con 0x7f32d0101080 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32d5000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d019ec60 con 0x7f32d01061a0 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33848/0 (socket says 172.21.15.46:33848) 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32cd400700 1 -- 172.21.15.46:0/533993356 learned_addr learned my addr 172.21.15.46:0/533993356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32ce800700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 0x7f32d019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.907+0000 7f32cde00700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d0101080 0x7f32d01995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 -- 172.21.15.46:0/533993356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d0101080 msgr2=0x7f32d01995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d0101080 0x7f32d01995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 -- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 msgr2=0x7f32d019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 0x7f32d019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 -- 172.21.15.46:0/533993356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00095f0 con 0x7f32d01061a0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32ce800700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 0x7f32d019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32cd400700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00d980 tx=0x7f32bc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32c6a00700 1 -- 172.21.15.46:0/533993356 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc009930 con 0x7f32d01061a0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32c6a00700 1 -- 172.21.15.46:0/533993356 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32bc010460 con 0x7f32d01061a0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32c6a00700 1 -- 172.21.15.46:0/533993356 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00f620 con 0x7f32d01061a0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d01a61c0 con 0x7f32d01061a0 2024-05-08T23:17:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.908+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01a66a0 con 0x7f32d01061a0 2024-05-08T23:17:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.911+0000 7f32c6a00700 1 -- 172.21.15.46:0/533993356 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f32bc0108d0 con 0x7f32d01061a0 2024-05-08T23:17:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.913+0000 7f32c6a00700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 0x7f32b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.913+0000 7f32c6a00700 1 -- 172.21.15.46:0/533993356 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f32bc0ccb80 con 0x7f32d01061a0 2024-05-08T23:17:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.913+0000 7f32e8bd0b80 1 Processor -- start 2024-05-08T23:17:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.913+0000 7f32cde00700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 0x7f32b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.913+0000 7f32e8bd0b80 1 -- start start 2024-05-08T23:17:38.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561450031e80 0x561450032260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 0x56145002cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ff831d0 con 0x56145002c910 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ff80b90 con 0x561450031e80 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ff7c480 con 0x5614500327a0 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32ce800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 0x56145002cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33854/0 (socket says 172.21.15.46:33854) 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 -- 172.21.15.46:0/1551473709 learned_addr learned my addr 172.21.15.46:0/1551473709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32ce800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 0x56145002cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48256/0 (socket says 172.21.15.46:48256) 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cde00700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 0x7f32b40a37d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006010 tx=0x7f32c000b540 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cde00700 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561450031e80 0x561450032260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 -- 172.21.15.46:0/1551473709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561450031e80 msgr2=0x561450032260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561450031e80 0x561450032260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 -- 172.21.15.46:0/1551473709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 msgr2=0x56145002cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 0x56145002cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.914+0000 7f32cd400700 1 -- 172.21.15.46:0/1551473709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144fdfd430 con 0x5614500327a0 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.915+0000 7f32cd400700 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0a1770 tx=0x7f32bc099fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1835eb979724fac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.915+0000 7f32bb400700 1 -- 172.21.15.46:0/1551473709 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0a1ca0 con 0x5614500327a0 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.915+0000 7f32bb400700 1 -- 172.21.15.46:0/1551473709 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32bc095f30 con 0x5614500327a0 2024-05-08T23:17:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.915+0000 7f32bb400700 1 -- 172.21.15.46:0/1551473709 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc099cc0 con 0x5614500327a0 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/1551473709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 msgr2=0x56145002c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0a1770 tx=0x7f32bc099fa0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/1551473709 shutdown_connections 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56145002c910 0x56145002cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561450031e80 0x561450032260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/1551473709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x56145002c3d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/1551473709 >> 172.21.15.46:0/1551473709 conn(0x56144ff8a290 msgr2=0x561450026e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/1551473709 shutdown_connections 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/1551473709 wait complete. 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.916+0000 7f32e8bd0b80 1 -- start start 2024-05-08T23:17:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 Processor -- start 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450036030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x561450036570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ff185e0 con 0x561450031e80 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ff18280 con 0x56145002c910 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144fdff720 con 0x5614500327a0 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32e8bd0b80 10 client.0 _ll_register_callbacks cb 0x56144ff21bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32cde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32cd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450036030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32cde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46912/0 (socket says 172.21.15.46:46912) 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32cde00700 1 -- 172.21.15.46:0/3171470361 learned_addr learned my addr 172.21.15.46:0/3171470361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.917+0000 7f32ce800700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x561450036570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 -- 172.21.15.46:0/3171470361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 msgr2=0x561450036570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x561450036570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 -- 172.21.15.46:0/3171470361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 msgr2=0x561450036030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450036030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 -- 172.21.15.46:0/3171470361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01a66a0 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cd400700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450036030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32cde00700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b9f0 tx=0x7f32c0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70547c0fb41394d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32afe00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004b10 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56144fed52f0 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.918+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f32c0004c70 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c0004e30 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 10 client.29360 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56144fda9be0 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56144fed52f0 con 0x56145002c910 2024-05-08T23:17:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32c002f440 con 0x56145002c910 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 10 client.29360 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 10 client.29360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 20 client.29360 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561450046170 0x561450048590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 4 client.29360 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.919+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56144fffb890 con 0x561450046170 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.920+0000 7f32cd400700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561450046170 0x561450048590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.920+0000 7f32cd400700 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561450046170 0x561450048590 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fa5755370feeb server_cookie=ca58fdf707b22e72 in_seq=0 out_seq=0 2024-05-08T23:17:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.920+0000 7f32afe00700 10 client.29360 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32afe00700 1 -- 172.21.15.46:0/3171470361 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56144fffb890 con 0x561450046170 2024-05-08T23:17:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32afe00700 10 client.29360 handle_command_reply: tid=0 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 client.29360 shutdown 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 msgr2=0x561450035af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b9f0 tx=0x7f32c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 shutdown_connections 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614500327a0 0x561450036570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x561450035af0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561450046170 0x561450048590 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3171470361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450036030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 >> 172.21.15.46:0/3171470361 conn(0x56144ff8a290 msgr2=0x561450026620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32afe00700 0 client.29360 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 shutdown_connections 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.921+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3171470361 wait complete. 2024-05-08T23:17:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.922+0000 7f32e8bd0b80 20 client.29360 trim_cache size 0 max 16384 2024-05-08T23:17:38.926 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 600 sec 2024-05-08T23:17:38.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.943+0000 7f32e8bd0b80 1 Processor -- start 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 -- start start 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450026da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614500327a0 0x5614500312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561450038dc0 con 0x561450031e80 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561450021880 con 0x5614500327a0 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561450027b40 con 0x56145002c910 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32cde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32cd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450026da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32cde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33870/0 (socket says 172.21.15.46:33870) 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32cde00700 1 -- 172.21.15.46:0/938176872 learned_addr learned my addr 172.21.15.46:0/938176872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.944+0000 7f32ce800700 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614500327a0 0x5614500312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 -- 172.21.15.46:0/938176872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614500327a0 msgr2=0x5614500312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614500327a0 0x5614500312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 -- 172.21.15.46:0/938176872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 msgr2=0x561450026da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450026da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 -- 172.21.15.46:0/938176872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145002cfc0 con 0x56145002c910 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cd400700 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450026da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32cde00700 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c00052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6ded53e438f30b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32bbe00700 1 -- 172.21.15.46:0/938176872 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c002fac0 con 0x56145002c910 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32bbe00700 1 -- 172.21.15.46:0/938176872 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32c001e040 con 0x56145002c910 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.945+0000 7f32bbe00700 1 -- 172.21.15.46:0/938176872 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000f6c0 con 0x56145002c910 2024-05-08T23:17:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/938176872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 msgr2=0x56145002c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c00052a0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/938176872 shutdown_connections 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614500327a0 0x5614500312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56145002c910 0x56145002c3d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/938176872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561450031e80 0x561450026da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/938176872 >> 172.21.15.46:0/938176872 conn(0x56144ff8a290 msgr2=0x561450030be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/938176872 shutdown_connections 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.946+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/938176872 wait complete. 2024-05-08T23:17:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.947+0000 7f32e8bd0b80 1 -- start start 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.947+0000 7f32e8bd0b80 1 Processor -- start 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x56145003d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 0x56145002e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145002eeb0 con 0x5614500327a0 2024-05-08T23:17:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145002f020 con 0x56145002c910 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145009c670 con 0x561450031e80 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32e8bd0b80 10 client.0 _ll_register_callbacks cb 0x56144ff21bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32ce800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 0x56145002e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x56145003d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32ce800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 0x56145002e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48284/0 (socket says 172.21.15.46:48284) 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33886/0 (socket says 172.21.15.46:33886) 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32ce800700 1 -- 172.21.15.46:0/3946006489 learned_addr learned my addr 172.21.15.46:0/3946006489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 -- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 msgr2=0x56145003d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x56145003d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 -- 172.21.15.46:0/3946006489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 msgr2=0x56145002e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 0x56145002e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.948+0000 7f32cd400700 1 -- 172.21.15.46:0/3946006489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56144fdfd430 con 0x561450031e80 2024-05-08T23:17:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32cd400700 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0997b0 tx=0x7f32bc099f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384cbda10b38f096 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0b90c0 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f32bc0b9220 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32bc095f30 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32bc098d10 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 10 client.29458 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56144fda9be0 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32bc095f30 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32bc0a1810 con 0x561450031e80 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 10 client.29458 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 10 client.29458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 20 client.29458 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56145009d9a0 0x56145009fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 4 client.29458 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:17:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.949+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56144ff74f40 con 0x56145009d9a0 2024-05-08T23:17:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.950+0000 7f32cde00700 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56145009d9a0 0x56145009fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.950+0000 7f32cde00700 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56145009d9a0 0x56145009fdc0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551bc4c7cfba97b6 server_cookie=781db5906c8130b9 in_seq=0 out_seq=0 2024-05-08T23:17:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.950+0000 7f32afe00700 10 client.29458 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.951+0000 7f32afe00700 1 -- 172.21.15.46:0/3946006489 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56144ff74f40 con 0x56145009d9a0 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.951+0000 7f32afe00700 10 client.29458 handle_command_reply: tid=0 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.951+0000 7f32e8bd0b80 1 client.29458 shutdown 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 msgr2=0x56145002e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0997b0 tx=0x7f32bc099f20 comp rx=0 tx=0).stop 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 shutdown_connections 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56145009d9a0 0x56145009fdc0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614500327a0 0x56145002e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56145002c910 0x56145003d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32afe00700 0 client.29458 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 --2- 172.21.15.46:0/3946006489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561450031e80 0x56145002e220 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 >> 172.21.15.46:0/3946006489 conn(0x56144ff8a290 msgr2=0x561450033fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 shutdown_connections 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 1 -- 172.21.15.46:0/3946006489 wait complete. 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.952+0000 7f32e8bd0b80 20 client.29458 trim_cache size 0 max 16384 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "95f2ccbb-668e-4a73-8382-14cbc78c1edd", 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:17:38.955 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 msgr2=0x7f32b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 0x7f32b40a37d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006010 tx=0x7f32c000b540 comp rx=0 tx=0).stop 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 msgr2=0x7f32d0199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00d980 tx=0x7f32bc00dc90 comp rx=0 tx=0).stop 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 shutdown_connections 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f32b40a1310 0x7f32b40a37d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32d019deb0 0x7f32d019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d0101080 0x7f32d01995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 --2- 172.21.15.46:0/533993356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32d01061a0 0x7f32d0199b10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 >> 172.21.15.46:0/533993356 conn(0x7f32d00fc8e0 msgr2=0x7f32d00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 shutdown_connections 2024-05-08T23:17:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:38.953+0000 7f32d5000700 1 -- 172.21.15.46:0/533993356 wait complete. 2024-05-08T23:17:38.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: 0 cleanup 600 sec 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: Operation Count AvgLat MaxLat 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: ---------------------------------------- 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: NTCreateX 171696 1.285 400.406 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: Close 126088 0.005 1.279 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: Rename 7272 5.903 163.331 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: Unlink 34702 1.713 339.390 2024-05-08T23:17:39.138 INFO:tasks.workunit.client.0.smithi046.stdout: Deltree 4 94.670 112.173 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Mkdir 2 0.017 0.017 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Qpathinfo 155582 1.135 373.215 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Qfileinfo 27254 0.002 0.145 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Qfsinfo 28557 0.214 5.370 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Sfileinfo 13980 0.046 23.744 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Find 60174 0.263 160.845 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: WriteX 85539 0.126 164.709 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: ReadX 269279 0.030 191.454 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: LockX 560 0.527 1.919 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: UnlockX 560 0.528 3.583 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: Flush 12042 4.461 267.292 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-08T23:17:39.139 INFO:tasks.workunit.client.0.smithi046.stdout:Throughput 8.99469 MB/sec 1 clients 1 procs max_latency=400.413 ms 2024-05-08T23:17:39.140 INFO:tasks.workunit.client.0.smithi046.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-08T23:17:39.140 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-08T23:17:39.140 INFO:tasks.workunit.client.0.smithi046.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-08T23:17:39.142 INFO:tasks.workunit.client.0.smithi046.stdout:9 of 10 processes prepared for launch 0 sec 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 -- 172.21.15.46:0/546396605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c073340 msgr2=0x7faf7c073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 --2- 172.21.15.46:0/546396605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c073340 0x7faf7c073720 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0098f0 tx=0x7faf6c009c00 comp rx=0 tx=0).stop 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 -- 172.21.15.46:0/546396605 shutdown_connections 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 --2- 172.21.15.46:0/546396605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c111e20 0x7faf7c114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 --2- 172.21.15.46:0/546396605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c073340 0x7faf7c073720 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 --2- 172.21.15.46:0/546396605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 0x7faf7c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 -- 172.21.15.46:0/546396605 >> 172.21.15.46:0/546396605 conn(0x7faf7c0fc8e0 msgr2=0x7faf7c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 -- 172.21.15.46:0/546396605 shutdown_connections 2024-05-08T23:17:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.187+0000 7faf81200700 1 -- 172.21.15.46:0/546396605 wait complete. 2024-05-08T23:17:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.188+0000 7faf81200700 1 Processor -- start 2024-05-08T23:17:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.188+0000 7faf81200700 1 -- start start 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 0x7faf7c1a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c111e20 0x7faf7c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c19de50 con 0x7faf7c073340 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c19dfc0 con 0x7faf7c073c60 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf81200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c19e180 con 0x7faf7c111e20 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48302/0 (socket says 172.21.15.46:48302) 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf79e00700 1 -- 172.21.15.46:0/453643215 learned_addr learned my addr 172.21.15.46:0/453643215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf73e00700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 0x7faf7c1a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.189+0000 7faf7a800700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c111e20 0x7faf7c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 -- 172.21.15.46:0/453643215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c111e20 msgr2=0x7faf7c19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c111e20 0x7faf7c19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 -- 172.21.15.46:0/453643215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 msgr2=0x7faf7c1a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 0x7faf7c1a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 -- 172.21.15.46:0/453643215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009520 con 0x7faf7c073340 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf79e00700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005950 tx=0x7faf6c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf72a00700 1 -- 172.21.15.46:0/453643215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01c070 con 0x7faf7c073340 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c19e400 con 0x7faf7c073340 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf72a00700 1 -- 172.21.15.46:0/453643215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6c00baa0 con 0x7faf7c073340 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.190+0000 7faf72a00700 1 -- 172.21.15.46:0/453643215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0038e0 con 0x7faf7c073340 2024-05-08T23:17:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.191+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c1aa6a0 con 0x7faf7c073340 2024-05-08T23:17:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.194+0000 7faf72a00700 1 -- 172.21.15.46:0/453643215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf6c020030 con 0x7faf7c073340 2024-05-08T23:17:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.195+0000 7faf94de0b80 1 Processor -- start 2024-05-08T23:17:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.195+0000 7faf72a00700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.195+0000 7faf94de0b80 1 -- start start 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.195+0000 7faf72a00700 1 -- 172.21.15.46:0/453643215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf6c0cdec0 con 0x7faf7c073340 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf7a800700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626beadb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beadbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626bea32150 con 0x5626beae0e00 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626bea2fb10 con 0x5626beae1720 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf94de0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626bea2b400 con 0x5626beadb890 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48314/0 (socket says 172.21.15.46:48314) 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf79e00700 1 -- 172.21.15.46:0/1255512138 learned_addr learned my addr 172.21.15.46:0/1255512138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf73e00700 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626beadb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.196+0000 7faf7a800700 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beadbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 -- 172.21.15.46:0/1255512138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 msgr2=0x5626beadbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beadbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 -- 172.21.15.46:0/1255512138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 msgr2=0x5626beadb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626beadb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 -- 172.21.15.46:0/1255512138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626be8ac3b0 con 0x5626beae0e00 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf79e00700 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009fd0 tx=0x7faf6c005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c16b1c03bad02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf7a800700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 0x7faf600a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf7c1a43b0 tx=0x7faf7400fc00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf6b400700 1 -- 172.21.15.46:0/1255512138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c09b2f0 con 0x5626beae0e00 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf6b400700 1 -- 172.21.15.46:0/1255512138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6c09abf0 con 0x5626beae0e00 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.197+0000 7faf6b400700 1 -- 172.21.15.46:0/1255512138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0050d0 con 0x5626beae0e00 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 -- 172.21.15.46:0/1255512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 msgr2=0x5626beae11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009fd0 tx=0x7faf6c005470 comp rx=0 tx=0).stop 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 -- 172.21.15.46:0/1255512138 shutdown_connections 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beadbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae11e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1255512138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626beadb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 -- 172.21.15.46:0/1255512138 >> 172.21.15.46:0/1255512138 conn(0x5626bea39210 msgr2=0x5626bead5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 -- 172.21.15.46:0/1255512138 shutdown_connections 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.198+0000 7faf94de0b80 1 -- 172.21.15.46:0/1255512138 wait complete. 2024-05-08T23:17:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.199+0000 7faf94de0b80 1 -- start start 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.199+0000 7faf94de0b80 1 Processor -- start 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.199+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 0x5626beae4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626be9c7560 con 0x5626beae0e00 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626be9c7200 con 0x5626beadb890 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626be8ae6a0 con 0x5626beae1720 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf94de0b80 10 client.0 _ll_register_callbacks cb 0x5626be9d0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf7a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 0x5626beae4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf7a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33928/0 (socket says 172.21.15.46:33928) 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf7a800700 1 -- 172.21.15.46:0/615144667 learned_addr learned my addr 172.21.15.46:0/615144667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf73e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 -- 172.21.15.46:0/615144667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 msgr2=0x5626beae5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 -- 172.21.15.46:0/615144667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 msgr2=0x5626beae4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf79e00700 1 -- 172.21.15.46:0/615144667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c1aa6a0 con 0x5626beadb890 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.200+0000 7faf7a800700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf79e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 0x5626beae4a90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09ab80 tx=0x7faf6c0cb6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e863d12840e5ae12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:17:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0a2fe0 con 0x5626beadb890 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf6c0050d0 con 0x5626beadb890 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626be984270 con 0x5626beadb890 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf6c005290 con 0x5626beadb890 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf94de0b80 10 client.29378 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626be858b60 con 0x5626beadb890 2024-05-08T23:17:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.201+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626be984270 con 0x5626beadb890 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf6c0a2b50 con 0x5626beadb890 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 10 client.29378 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 10 client.29378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 20 client.29378 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb059c0 0x5626beb07db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 4 client.29378 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626beaaa810 con 0x5626beb059c0 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.202+0000 7faf73e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb059c0 0x5626beb07db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.203+0000 7faf73e00700 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb059c0 0x5626beb07db0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3b6232a06ce998 server_cookie=4e055274c6bcc9cd in_seq=0 out_seq=0 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.203+0000 7faf5fe00700 10 client.29378 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.203+0000 7faf5fe00700 1 -- 172.21.15.46:0/615144667 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626beaaa810 con 0x5626beb059c0 2024-05-08T23:17:39.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.203+0000 7faf5fe00700 10 client.29378 handle_command_reply: tid=0 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.203+0000 7faf94de0b80 1 client.29378 shutdown 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 msgr2=0x5626beae4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 0x5626beae4a90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09ab80 tx=0x7faf6c0cb6b0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 shutdown_connections 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae1720 0x5626beae5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beadb890 0x5626beae4a90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb059c0 0x5626beb07db0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 --2- 172.21.15.46:0/615144667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae0e00 0x5626beae4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 >> 172.21.15.46:0/615144667 conn(0x5626bea39210 msgr2=0x5626bead55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf5fe00700 0 client.29378 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 shutdown_connections 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 1 -- 172.21.15.46:0/615144667 wait complete. 2024-05-08T23:17:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.204+0000 7faf94de0b80 20 client.29378 trim_cache size 0 max 16384 2024-05-08T23:17:39.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.222+0000 7faf94de0b80 1 Processor -- start 2024-05-08T23:17:39.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.222+0000 7faf94de0b80 1 -- start start 2024-05-08T23:17:39.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 0x5626beae0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626bead5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626bead2e00 con 0x5626beadb890 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626bead7e50 con 0x5626beae1720 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf94de0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626beb0cfc0 con 0x5626beae0e00 2024-05-08T23:17:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf79e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48332/0 (socket says 172.21.15.46:48332) 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf79e00700 1 -- 172.21.15.46:0/1083713728 learned_addr learned my addr 172.21.15.46:0/1083713728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 0x5626beae0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf7a800700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626bead5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 -- 172.21.15.46:0/1083713728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 msgr2=0x5626bead5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626bead5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 -- 172.21.15.46:0/1083713728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 msgr2=0x5626beadb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf73e00700 1 -- 172.21.15.46:0/1083713728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626beadbf40 con 0x5626beae0e00 2024-05-08T23:17:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.223+0000 7faf79e00700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.224+0000 7faf73e00700 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 0x5626beae0400 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5626beb08690 tx=0x7faf64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f39a46806017d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.224+0000 7faf6be00700 1 -- 172.21.15.46:0/1083713728 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64017070 con 0x5626beae0e00 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.224+0000 7faf6be00700 1 -- 172.21.15.46:0/1083713728 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf64018040 con 0x5626beae0e00 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.224+0000 7faf6be00700 1 -- 172.21.15.46:0/1083713728 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64005020 con 0x5626beae0e00 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- 172.21.15.46:0/1083713728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 msgr2=0x5626beae0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 0x5626beae0400 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5626beb08690 tx=0x7faf64009260 comp rx=0 tx=0).stop 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- 172.21.15.46:0/1083713728 shutdown_connections 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae1720 0x5626bead5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beadb890 0x5626beadb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1083713728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beae0e00 0x5626beae0400 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- 172.21.15.46:0/1083713728 >> 172.21.15.46:0/1083713728 conn(0x5626bea39210 msgr2=0x5626beae3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- 172.21.15.46:0/1083713728 shutdown_connections 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- 172.21.15.46:0/1083713728 wait complete. 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.225+0000 7faf94de0b80 1 -- start start 2024-05-08T23:17:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 Processor -- start 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae0e00 0x5626beafb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae1720 0x5626beafbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626beafc2c0 con 0x5626beae1720 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626beafc430 con 0x5626beae0e00 2024-05-08T23:17:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626beb5be70 con 0x5626beadb890 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf94de0b80 10 client.0 _ll_register_callbacks cb 0x5626be9d0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf7a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae1720 0x5626beafbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf79e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf79e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33932/0 (socket says 172.21.15.46:33932) 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf79e00700 1 -- 172.21.15.46:0/1695978247 learned_addr learned my addr 172.21.15.46:0/1695978247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:17:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.226+0000 7faf73e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae0e00 0x5626beafb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 -- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae0e00 msgr2=0x5626beafb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae0e00 0x5626beafb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 -- 172.21.15.46:0/1695978247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae1720 msgr2=0x5626beafbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae1720 0x5626beafbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 -- 172.21.15.46:0/1695978247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626be858b60 con 0x5626beadb890 2024-05-08T23:17:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf79e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09ab80 tx=0x7faf6c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70eebb4bcedbc40e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c096d80 con 0x5626beadb890 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf6c09abf0 con 0x5626beadb890 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.227+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faf6c0050d0 con 0x5626beadb890 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf6c09a550 con 0x5626beadb890 2024-05-08T23:17:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 10 client.29476 fetch_fsmap learned FSMap version 30 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7faf64009b30 con 0x5626beadb890 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6c09abf0 con 0x5626beadb890 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf6c09ae50 con 0x5626beadb890 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 10 client.29476 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 10 client.29476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:17:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 20 client.29476 populate_metadata read hostname 'smithi046' 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb5d0c0 0x5626beb5f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 4 client.29476 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626bea23ec0 con 0x5626beb5d0c0 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.228+0000 7faf73e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb5d0c0 0x5626beb5f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf73e00700 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb5d0c0 0x5626beb5f4b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665294d6560a390f server_cookie=1393f742604cd675 in_seq=0 out_seq=0 2024-05-08T23:17:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf5fe00700 10 client.29476 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf5fe00700 1 -- 172.21.15.46:0/1695978247 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5626bea23ec0 con 0x5626beb5d0c0 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf5fe00700 10 client.29476 handle_command_reply: tid=0 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 client.29476 shutdown 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 msgr2=0x5626beafd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09ab80 tx=0x7faf6c005040 comp rx=0 tx=0).stop 2024-05-08T23:17:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 shutdown_connections 2024-05-08T23:17:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5626beb5d0c0 0x5626beb5f4b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626beae1720 0x5626beafbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626beadb890 0x5626beafd2a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf5fe00700 0 client.29476 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:17:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 --2- 172.21.15.46:0/1695978247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626beae0e00 0x5626beafb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 >> 172.21.15.46:0/1695978247 conn(0x5626bea39210 msgr2=0x5626beae0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.229+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 shutdown_connections 2024-05-08T23:17:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf94de0b80 1 -- 172.21.15.46:0/1695978247 wait complete. 2024-05-08T23:17:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf94de0b80 20 client.29476 trim_cache size 0 max 16384 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout: "95f2ccbb-668e-4a73-8382-14cbc78c1edd": { 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:17:39.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "95f2ccbb-668e-4a73-8382-14cbc78c1edd", 2024-05-08T23:17:39.249 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:17:39.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:17:39.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:17:39.249 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 msgr2=0x7faf600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 0x7faf600a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf7c1a43b0 tx=0x7faf7400fc00 comp rx=0 tx=0).stop 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 msgr2=0x7faf7c1a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005950 tx=0x7faf6c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 shutdown_connections 2024-05-08T23:17:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c111e20 0x7faf7c19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf7c073340 0x7faf7c1a36b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 --2- 172.21.15.46:0/453643215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf7c073c60 0x7faf7c1a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.230+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 >> 172.21.15.46:0/453643215 conn(0x7faf7c0fc8e0 msgr2=0x7faf7c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.231+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 shutdown_connections 2024-05-08T23:17:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:17:39.231+0000 7faf81200700 1 -- 172.21.15.46:0/453643215 wait complete. 2024-05-08T23:17:39.253 INFO:tasks.workunit.client.0.smithi046.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-08T23:17:39.253 INFO:tasks.workunit.client.0.smithi046.stdout:releasing clients 2024-05-08T23:17:39.254 INFO:tasks.cephfs.filesystem:scrub status for tag:95f2ccbb-668e-4a73-8382-14cbc78c1edd - {'path': '/', 'tag': '95f2ccbb-668e-4a73-8382-14cbc78c1edd', 'options': 'recursive,force'} 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: pgmap v432: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:39 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 363 130.19 MB/sec warmup 1 sec latency 136.007 ms 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: pgmap v432: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:39 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: pgmap v432: 129 pgs: 129 active+clean; 617 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:17:40.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:40.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:39 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:17:41.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 774 101.96 MB/sec warmup 2 sec latency 234.071 ms 2024-05-08T23:17:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:41 smithi086 ceph-mon[78610]: pgmap v433: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2024-05-08T23:17:42.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:41 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:42.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 1286 79.71 MB/sec warmup 3 sec latency 514.841 ms 2024-05-08T23:17:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:41 smithi053 ceph-mon[77587]: pgmap v433: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2024-05-08T23:17:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:41 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:41 smithi046 ceph-mon[36898]: pgmap v433: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 175 op/s 2024-05-08T23:17:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:41 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:43.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 1399 61.35 MB/sec warmup 4 sec latency 1514.905 ms 2024-05-08T23:17:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:43 smithi086 ceph-mon[78610]: pgmap v434: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-05-08T23:17:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:43 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:44.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 2164 53.22 MB/sec warmup 5 sec latency 1598.179 ms 2024-05-08T23:17:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:43 smithi053 ceph-mon[77587]: pgmap v434: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-05-08T23:17:44.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:43 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:43 smithi046 ceph-mon[36898]: pgmap v434: 129 pgs: 129 active+clean; 620 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-05-08T23:17:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:43 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:17:45.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 2748 52.98 MB/sec warmup 6 sec latency 163.529 ms 2024-05-08T23:17:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:45 smithi086 ceph-mon[78610]: pgmap v435: 129 pgs: 129 active+clean; 710 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 301 op/s 2024-05-08T23:17:46.243 INFO:tasks.workunit.client.0.smithi046.stdout: 10 3387 52.37 MB/sec warmup 7 sec latency 181.106 ms 2024-05-08T23:17:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:45 smithi053 ceph-mon[77587]: pgmap v435: 129 pgs: 129 active+clean; 710 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 301 op/s 2024-05-08T23:17:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:45 smithi046 ceph-mon[36898]: pgmap v435: 129 pgs: 129 active+clean; 710 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 301 op/s 2024-05-08T23:17:47.244 INFO:tasks.workunit.client.0.smithi046.stdout: 10 3725 48.27 MB/sec warmup 8 sec latency 159.926 ms 2024-05-08T23:17:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:47 smithi086 ceph-mon[78610]: pgmap v436: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 296 op/s 2024-05-08T23:17:48.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 3948 43.42 MB/sec warmup 9 sec latency 256.903 ms 2024-05-08T23:17:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:47 smithi053 ceph-mon[77587]: pgmap v436: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 296 op/s 2024-05-08T23:17:48.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:47 smithi046 ceph-mon[36898]: pgmap v436: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 296 op/s 2024-05-08T23:17:49.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 4204 40.17 MB/sec warmup 10 sec latency 348.594 ms 2024-05-08T23:17:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:49 smithi086 ceph-mon[78610]: pgmap v437: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 282 op/s 2024-05-08T23:17:50.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 4601 37.91 MB/sec warmup 11 sec latency 247.715 ms 2024-05-08T23:17:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:49 smithi053 ceph-mon[77587]: pgmap v437: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 282 op/s 2024-05-08T23:17:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:49 smithi046 ceph-mon[36898]: pgmap v437: 129 pgs: 129 active+clean; 736 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 282 op/s 2024-05-08T23:17:51.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:50 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:17:51.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 5316 37.53 MB/sec warmup 12 sec latency 174.460 ms 2024-05-08T23:17:51.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:50 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:17:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:50 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:17:52.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:51 smithi086 ceph-mon[78610]: pgmap v438: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 450 op/s 2024-05-08T23:17:52.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 6088 37.32 MB/sec warmup 13 sec latency 162.833 ms 2024-05-08T23:17:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:51 smithi053 ceph-mon[77587]: pgmap v438: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 450 op/s 2024-05-08T23:17:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:51 smithi046 ceph-mon[36898]: pgmap v438: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 450 op/s 2024-05-08T23:17:53.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 6767 38.61 MB/sec warmup 14 sec latency 146.119 ms 2024-05-08T23:17:54.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:53 smithi086 ceph-mon[78610]: pgmap v439: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-05-08T23:17:54.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 7256 38.69 MB/sec warmup 15 sec latency 236.125 ms 2024-05-08T23:17:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:53 smithi053 ceph-mon[77587]: pgmap v439: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-05-08T23:17:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:53 smithi046 ceph-mon[36898]: pgmap v439: 129 pgs: 129 active+clean; 811 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-05-08T23:17:55.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 7533 36.88 MB/sec warmup 16 sec latency 124.344 ms 2024-05-08T23:17:56.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:55 smithi086 ceph-mon[78610]: pgmap v440: 129 pgs: 129 active+clean; 810 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 514 op/s 2024-05-08T23:17:56.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 7767 34.97 MB/sec warmup 17 sec latency 261.822 ms 2024-05-08T23:17:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:55 smithi053 ceph-mon[77587]: pgmap v440: 129 pgs: 129 active+clean; 810 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 514 op/s 2024-05-08T23:17:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:55 smithi046 ceph-mon[36898]: pgmap v440: 129 pgs: 129 active+clean; 810 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 23 MiB/s wr, 514 op/s 2024-05-08T23:17:57.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 8214 33.88 MB/sec warmup 18 sec latency 243.242 ms 2024-05-08T23:17:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:57 smithi086 ceph-mon[78610]: pgmap v441: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 393 op/s 2024-05-08T23:17:58.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 8832 34.00 MB/sec warmup 19 sec latency 185.832 ms 2024-05-08T23:17:58.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:57 smithi053 ceph-mon[77587]: pgmap v441: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 393 op/s 2024-05-08T23:17:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:57 smithi046 ceph-mon[36898]: pgmap v441: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 393 op/s 2024-05-08T23:17:59.246 INFO:tasks.workunit.client.0.smithi046.stdout: 10 9569 33.98 MB/sec warmup 20 sec latency 163.359 ms 2024-05-08T23:18:00.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:17:59 smithi086 ceph-mon[78610]: pgmap v442: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 353 op/s 2024-05-08T23:18:00.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10233 34.95 MB/sec warmup 21 sec latency 134.167 ms 2024-05-08T23:18:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:17:59 smithi053 ceph-mon[77587]: pgmap v442: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 353 op/s 2024-05-08T23:18:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:17:59 smithi046 ceph-mon[36898]: pgmap v442: 129 pgs: 129 active+clean; 814 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 353 op/s 2024-05-08T23:18:01.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10450 33.77 MB/sec warmup 22 sec latency 198.604 ms 2024-05-08T23:18:02.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:01 smithi086 ceph-mon[78610]: pgmap v443: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-05-08T23:18:02.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10503 32.48 MB/sec warmup 23 sec latency 331.804 ms 2024-05-08T23:18:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:01 smithi053 ceph-mon[77587]: pgmap v443: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-05-08T23:18:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:01 smithi046 ceph-mon[36898]: pgmap v443: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-05-08T23:18:03.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10616 31.55 MB/sec warmup 24 sec latency 248.108 ms 2024-05-08T23:18:04.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:03 smithi086 ceph-mon[78610]: pgmap v444: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-05-08T23:18:04.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10871 30.80 MB/sec warmup 25 sec latency 417.894 ms 2024-05-08T23:18:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:03 smithi053 ceph-mon[77587]: pgmap v444: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-05-08T23:18:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:03 smithi046 ceph-mon[36898]: pgmap v444: 129 pgs: 129 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2024-05-08T23:18:05.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 11093 30.06 MB/sec warmup 26 sec latency 433.795 ms 2024-05-08T23:18:06.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:05 smithi086 ceph-mon[78610]: pgmap v445: 129 pgs: 129 active+clean; 817 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-08T23:18:06.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 11406 29.41 MB/sec warmup 27 sec latency 209.018 ms 2024-05-08T23:18:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:05 smithi053 ceph-mon[77587]: pgmap v445: 129 pgs: 129 active+clean; 817 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-08T23:18:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:05 smithi046 ceph-mon[36898]: pgmap v445: 129 pgs: 129 active+clean; 817 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-08T23:18:07.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 11751 28.65 MB/sec warmup 28 sec latency 254.114 ms 2024-05-08T23:18:08.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:07 smithi086 ceph-mon[78610]: pgmap v446: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 504 op/s 2024-05-08T23:18:08.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 12352 28.70 MB/sec warmup 29 sec latency 207.457 ms 2024-05-08T23:18:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:07 smithi053 ceph-mon[77587]: pgmap v446: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 504 op/s 2024-05-08T23:18:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:07 smithi046 ceph-mon[36898]: pgmap v446: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.0 MiB/s wr, 504 op/s 2024-05-08T23:18:09.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 12939 28.48 MB/sec warmup 30 sec latency 291.980 ms 2024-05-08T23:18:09.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:18:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 -- 172.21.15.46:0/2407719659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 msgr2=0x7fbe98113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/2407719659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 0x7fbe98113bb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009990 tx=0x7fbe88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 -- 172.21.15.46:0/2407719659 shutdown_connections 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/2407719659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe981140f0 0x7fbe981164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/2407719659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe98107fa0 0x7fbe98108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/2407719659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 0x7fbe98113bb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 -- 172.21.15.46:0/2407719659 >> 172.21.15.46:0/2407719659 conn(0x7fbe9806bf30 msgr2=0x7fbe9806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.657+0000 7fbe9fa00700 1 -- 172.21.15.46:0/2407719659 shutdown_connections 2024-05-08T23:18:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.658+0000 7fbe9fa00700 1 -- 172.21.15.46:0/2407719659 wait complete. 2024-05-08T23:18:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.661+0000 7fbe9fa00700 1 Processor -- start 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.661+0000 7fbe9fa00700 1 -- start start 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 0x7fbe98109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 0x7fbe98109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 0x7fbe9810a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9810b9e0 con 0x7fbe981140f0 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981a3bb0 con 0x7fbe98107fa0 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981a3d70 con 0x7fbe981088c0 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 0x7fbe9810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 0x7fbe9810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57974/0 (socket says 172.21.15.46:57974) 2024-05-08T23:18:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9da00700 1 -- 172.21.15.46:0/704289827 learned_addr learned my addr 172.21.15.46:0/704289827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.662+0000 7fbe9d000700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 0x7fbe98109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 -- 172.21.15.46:0/704289827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 msgr2=0x7fbe98109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 0x7fbe98109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 -- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 msgr2=0x7fbe9810a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 0x7fbe9810a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 -- 172.21.15.46:0/704289827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880095f0 con 0x7fbe98107fa0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.663+0000 7fbe9d000700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 0x7fbe98109a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbe90007ec0 tx=0x7fbe9000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.664+0000 7fbe95600700 1 -- 172.21.15.46:0/704289827 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000ddd0 con 0x7fbe98107fa0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.664+0000 7fbe95600700 1 -- 172.21.15.46:0/704289827 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9000f040 con 0x7fbe98107fa0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.664+0000 7fbe95600700 1 -- 172.21.15.46:0/704289827 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000ddd0 con 0x7fbe98107fa0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.664+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe981a4050 con 0x7fbe98107fa0 2024-05-08T23:18:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.664+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981a4450 con 0x7fbe98107fa0 2024-05-08T23:18:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.672+0000 7fbe95600700 1 -- 172.21.15.46:0/704289827 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbe900057a0 con 0x7fbe98107fa0 2024-05-08T23:18:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.675+0000 7fbe95600700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 0x7fbe7c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.675+0000 7fbe95600700 1 -- 172.21.15.46:0/704289827 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbe900cc4d0 con 0x7fbe98107fa0 2024-05-08T23:18:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.676+0000 7fbe97e00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 0x7fbe7c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.676+0000 7fbeb3604b80 1 Processor -- start 2024-05-08T23:18:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 -- start start 2024-05-08T23:18:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba017350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba017890 0x55feba017cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9f6e150 con 0x55feba01d720 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9f6bb10 con 0x55feba017890 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.677+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9f67400 con 0x55feba01ce00 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49514/0 (socket says 172.21.15.46:49514) 2024-05-08T23:18:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 -- 172.21.15.46:0/1409195228 learned_addr learned my addr 172.21.15.46:0/1409195228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 -- 172.21.15.46:0/1409195228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba017890 msgr2=0x55feba017cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba017890 0x55feba017cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 -- 172.21.15.46:0/1409195228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 msgr2=0x55feba017350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba017350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 -- 172.21.15.46:0/1409195228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb9de83b0 con 0x55feba01ce00 2024-05-08T23:18:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.678+0000 7fbe9d000700 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a10c0 tx=0x7fbe90098ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15dd429b0b0ae9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.680+0000 7fbe8e000700 1 -- 172.21.15.46:0/1409195228 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9009cfa0 con 0x55feba01ce00 2024-05-08T23:18:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.680+0000 7fbe8e000700 1 -- 172.21.15.46:0/1409195228 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe900a15f0 con 0x55feba01ce00 2024-05-08T23:18:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.680+0000 7fbe8e000700 1 -- 172.21.15.46:0/1409195228 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004cd0 con 0x55feba01ce00 2024-05-08T23:18:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 -- 172.21.15.46:0/1409195228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 msgr2=0x55feba01d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a10c0 tx=0x7fbe90098ad0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 -- 172.21.15.46:0/1409195228 shutdown_connections 2024-05-08T23:18:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba017890 0x55feba017cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba01d1e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/1409195228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba017350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 -- 172.21.15.46:0/1409195228 >> 172.21.15.46:0/1409195228 conn(0x55feb9f75210 msgr2=0x55feba011df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 -- 172.21.15.46:0/1409195228 shutdown_connections 2024-05-08T23:18:09.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.681+0000 7fbeb3604b80 1 -- 172.21.15.46:0/1409195228 wait complete. 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.682+0000 7fbeb3604b80 1 -- start start 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.682+0000 7fbeb3604b80 1 Processor -- start 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.683+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.683+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba021030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.683+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba021570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.683+0000 7fbe9d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbe9d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49526/0 (socket says 172.21.15.46:49526) 2024-05-08T23:18:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbe9d000700 1 -- 172.21.15.46:0/3672559540 learned_addr learned my addr 172.21.15.46:0/3672559540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9f03560 con 0x55feba01d720 2024-05-08T23:18:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9f03200 con 0x55feba01ce00 2024-05-08T23:18:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb9dea6a0 con 0x55feba017890 2024-05-08T23:18:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.684+0000 7fbeb3604b80 10 client.0 _ll_register_callbacks cb 0x55feb9f0cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:09.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.685+0000 7fbe9da00700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba021570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.685+0000 7fbe97e00700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba021030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe97e00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 0x7fbe7c0a38b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55feba01db80 tx=0x7fbe88005fb0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 -- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 msgr2=0x55feba021030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba021030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 -- 172.21.15.46:0/3672559540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 msgr2=0x55feba021570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba021570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981a4450 con 0x55feba017890 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9da00700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba021570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe9d000700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbe90095be0 tx=0x7fbe900a1760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960306c7695a7196 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.686+0000 7fbe82a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900c9c40 con 0x55feba017890 2024-05-08T23:18:09.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbe900cc210 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55feb9ec0270 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe900993d0 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbeb3604b80 10 client.29485 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55feb9d94b60 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.687+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55feb9ec0270 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe900b1d20 con 0x55feba017890 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 10 client.29485 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 10 client.29485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 20 client.29485 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba030e20 0x55feba033240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 4 client.29485 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55feb9fe6810 con 0x55feba030e20 2024-05-08T23:18:09.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.688+0000 7fbe97e00700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba030e20 0x55feba033240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.689+0000 7fbe82a00700 10 client.29485 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:09.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.690+0000 7fbe97e00700 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba030e20 0x55feba033240 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9965249918f9481d server_cookie=723dad3595782f64 in_seq=0 out_seq=0 2024-05-08T23:18:09.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.692+0000 7fbe82a00700 1 -- 172.21.15.46:0/3672559540 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55feb9fe6810 con 0x55feba030e20 2024-05-08T23:18:09.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.692+0000 7fbe82a00700 10 client.29485 handle_command_reply: tid=0 2024-05-08T23:18:09.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 client.29485 shutdown 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 msgr2=0x55feba020af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbe90095be0 tx=0x7fbe900a1760 comp rx=0 tx=0).stop 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 shutdown_connections 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba030e20 0x55feba033240 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba01d720 0x55feba021570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba017890 0x55feba020af0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbe82a00700 0 client.29485 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/3672559540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba021030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.693+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 >> 172.21.15.46:0/3672559540 conn(0x55feb9f75210 msgr2=0x55feba0115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.694+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 shutdown_connections 2024-05-08T23:18:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.695+0000 7fbeb3604b80 1 -- 172.21.15.46:0/3672559540 wait complete. 2024-05-08T23:18:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.695+0000 7fbeb3604b80 20 client.29485 trim_cache size 0 max 16384 2024-05-08T23:18:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.743+0000 7fbeb3604b80 1 Processor -- start 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 -- start start 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba017350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01d720 0x55feba028190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba00c840 con 0x55feba017890 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba01d560 con 0x55feba01d720 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba038410 con 0x55feba01ce00 2024-05-08T23:18:10.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbe97e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbe97e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49532/0 (socket says 172.21.15.46:49532) 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.744+0000 7fbe97e00700 1 -- 172.21.15.46:0/933895870 learned_addr learned my addr 172.21.15.46:0/933895870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe9da00700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01d720 0x55feba028190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe9d000700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba017350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe97e00700 1 -- 172.21.15.46:0/933895870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01d720 msgr2=0x55feba028190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe97e00700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01d720 0x55feba028190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe97e00700 1 -- 172.21.15.46:0/933895870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 msgr2=0x55feba017350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe97e00700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba017350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe97e00700 1 -- 172.21.15.46:0/933895870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb9f2d560 con 0x55feba01ce00 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.745+0000 7fbe9d000700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba017350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:10.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.746+0000 7fbe97e00700 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe98072350 tx=0x7fbe88018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53733631940a5b01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.746+0000 7fbe8ea00700 1 -- 172.21.15.46:0/933895870 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801d070 con 0x55feba01ce00 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.746+0000 7fbe8ea00700 1 -- 172.21.15.46:0/933895870 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8802a040 con 0x55feba01ce00 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.746+0000 7fbe8ea00700 1 -- 172.21.15.46:0/933895870 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe880049a0 con 0x55feba01ce00 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.747+0000 7fbeb3604b80 1 -- 172.21.15.46:0/933895870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 msgr2=0x55feba011d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.747+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe98072350 tx=0x7fbe88018000 comp rx=0 tx=0).stop 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.748+0000 7fbeb3604b80 1 -- 172.21.15.46:0/933895870 shutdown_connections 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.748+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01d720 0x55feba028190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.748+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba017350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.748+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/933895870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01ce00 0x55feba011d20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.748+0000 7fbeb3604b80 1 -- 172.21.15.46:0/933895870 >> 172.21.15.46:0/933895870 conn(0x55feb9f75210 msgr2=0x55feba01f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.750+0000 7fbeb3604b80 1 -- 172.21.15.46:0/933895870 shutdown_connections 2024-05-08T23:18:10.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.750+0000 7fbeb3604b80 1 -- 172.21.15.46:0/933895870 wait complete. 2024-05-08T23:18:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.750+0000 7fbeb3604b80 1 -- start start 2024-05-08T23:18:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.750+0000 7fbeb3604b80 1 Processor -- start 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.751+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba0286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.751+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba019240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.751+0000 7fbeb3604b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.751+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba01a040 con 0x55feba017890 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.751+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba0872e0 con 0x55feba01ce00 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbeb3604b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feba087420 con 0x55feba01d720 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbeb3604b80 10 client.0 _ll_register_callbacks cb 0x55feb9f0cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbe9da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbe9da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49534/0 (socket says 172.21.15.46:49534) 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbe9da00700 1 -- 172.21.15.46:0/4278550960 learned_addr learned my addr 172.21.15.46:0/4278550960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbe97e00700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba019240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.752+0000 7fbe9d000700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba0286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.753+0000 7fbe9da00700 1 -- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 msgr2=0x55feba019240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe9da00700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba019240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe9da00700 1 -- 172.21.15.46:0/4278550960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 msgr2=0x55feba0286f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe9da00700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba0286f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe9da00700 1 -- 172.21.15.46:0/4278550960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe98116de0 con 0x55feba01d720 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe9d000700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba0286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.754+0000 7fbe97e00700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba019240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.755+0000 7fbe9da00700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe84012790 tx=0x7fbe84012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af3680cd89d253e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.755+0000 7fbe82a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.755+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84017070 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.755+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe900a15f0 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.756+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fbe840107c0 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.758+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe84010980 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.758+0000 7fbeb3604b80 10 client.29491 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.758+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55feb9deccd0 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.758+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe900a15f0 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe8401bcf0 con 0x55feba01d720 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 10 client.29491 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 10 client.29491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 20 client.29491 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba088c20 0x55feba08b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 4 client.29491 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.759+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55feb9f5fec0 con 0x55feba088c20 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbe9d000700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba088c20 0x55feba08b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbe9d000700 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba088c20 0x55feba08b040 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d97ae0d2beba599 server_cookie=b003a4cfbe669437 in_seq=0 out_seq=0 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbe82a00700 10 client.29491 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbe82a00700 1 -- 172.21.15.46:0/4278550960 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55feb9f5fec0 con 0x55feba088c20 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbe82a00700 10 client.29491 handle_command_reply: tid=0 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.760+0000 7fbeb3604b80 1 client.29491 shutdown 2024-05-08T23:18:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 msgr2=0x55feba019930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe84012790 tx=0x7fbe84012b50 comp rx=0 tx=0).stop 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 shutdown_connections 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55feba01d720 0x55feba019930 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55feba088c20 0x55feba08b040 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55feba017890 0x55feba0286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 --2- 172.21.15.46:0/4278550960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feba01ce00 0x55feba019240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 >> 172.21.15.46:0/4278550960 conn(0x55feb9f75210 msgr2=0x55feba01c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbe82a00700 0 client.29491 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 shutdown_connections 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.761+0000 7fbeb3604b80 1 -- 172.21.15.46:0/4278550960 wait complete. 2024-05-08T23:18:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.762+0000 7fbeb3604b80 20 client.29491 trim_cache size 0 max 16384 2024-05-08T23:18:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:10.129 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:18:10.129 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:18:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.765+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 msgr2=0x7fbe7c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.765+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 0x7fbe7c0a38b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55feba01db80 tx=0x7fbe88005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.765+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 msgr2=0x7fbe98109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.765+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 0x7fbe98109a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbe90007ec0 tx=0x7fbe9000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 shutdown_connections 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbe7c0a13f0 0x7fbe7c0a38b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe981140f0 0x7fbe9810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe98107fa0 0x7fbe98109a10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 --2- 172.21.15.46:0/704289827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe981088c0 0x7fbe98109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 >> 172.21.15.46:0/704289827 conn(0x7fbe9806bf30 msgr2=0x7fbe9806fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 shutdown_connections 2024-05-08T23:18:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:09.766+0000 7fbe9fa00700 1 -- 172.21.15.46:0/704289827 wait complete. 2024-05-08T23:18:10.132 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:18:10.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:18:10.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:09 smithi046 ceph-mon[36898]: pgmap v447: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-05-08T23:18:10.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 13034 27.79 MB/sec warmup 31 sec latency 702.641 ms 2024-05-08T23:18:10.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:09 smithi053 ceph-mon[77587]: pgmap v447: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-05-08T23:18:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.420+0000 7f981f200700 1 -- 172.21.15.46:0/1736982202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 msgr2=0x7f981810adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.420+0000 7f981f200700 1 --2- 172.21.15.46:0/1736982202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f981810adc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9808009990 tx=0x7f9808009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 -- 172.21.15.46:0/1736982202 shutdown_connections 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 --2- 172.21.15.46:0/1736982202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f981810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 --2- 172.21.15.46:0/1736982202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98181023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 --2- 172.21.15.46:0/1736982202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f981810adc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 -- 172.21.15.46:0/1736982202 >> 172.21.15.46:0/1736982202 conn(0x7f98180fb820 msgr2=0x7f98180fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 -- 172.21.15.46:0/1736982202 shutdown_connections 2024-05-08T23:18:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.421+0000 7f981f200700 1 -- 172.21.15.46:0/1736982202 wait complete. 2024-05-08T23:18:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 Processor -- start 2024-05-08T23:18:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 -- start start 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f9818100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f9818100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98181a56c0 con 0x7f981810b3e0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98181a5830 con 0x7f9818101fc0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.422+0000 7f981f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98181a59f0 con 0x7f98181028e0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43668/0 (socket says 172.21.15.46:43668) 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817e00700 1 -- 172.21.15.46:0/2314106368 learned_addr learned my addr 172.21.15.46:0/2314106368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f981d200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f9818100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f9818100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 -- 172.21.15.46:0/2314106368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 msgr2=0x7f98180ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 -- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 msgr2=0x7f9818100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f9818100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817400700 1 -- 172.21.15.46:0/2314106368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800009520 con 0x7f98181028e0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f9817e00700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.423+0000 7f981d200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f9818100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f9817400700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f9818100150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9808000c00 tx=0x7f9808004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980801d070 con 0x7f98181028e0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9808021410 con 0x7f98181028e0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808017650 con 0x7f98181028e0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98080095f0 con 0x7f98181028e0 2024-05-08T23:18:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.424+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98181a5f90 con 0x7f98181028e0 2024-05-08T23:18:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.426+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98181064b0 con 0x7f98181028e0 2024-05-08T23:18:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.430+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9808021580 con 0x7f98181028e0 2024-05-08T23:18:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.432+0000 7f9814c00700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 0x7f97fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.432+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f98080ce450 con 0x7f98181028e0 2024-05-08T23:18:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.432+0000 7f9817e00700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 0x7f97fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.432+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9808097520 con 0x7f98181028e0 2024-05-08T23:18:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.434+0000 7f9817e00700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 0x7f97fc0a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9810009c10 tx=0x7f981000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:09 smithi086 ceph-mon[78610]: pgmap v447: 129 pgs: 129 active+clean; 819 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 437 op/s 2024-05-08T23:18:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.810+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98181a5bb0 con 0x7f98181028e0 2024-05-08T23:18:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.811+0000 7f9814c00700 1 -- 172.21.15.46:0/2314106368 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10910 (secure 0 0 0) 0x7f980809ab40 con 0x7f98181028e0 2024-05-08T23:18:10.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:18:10.812 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:17:42.880411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 msgr2=0x7f97fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 0x7f97fc0a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9810009c10 tx=0x7f981000b040 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 msgr2=0x7f9818100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f9818100150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9808000c00 tx=0x7f9808004730 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 shutdown_connections 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97fc0a1310 0x7f97fc0a37d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810b3e0 0x7f9818100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9818101fc0 0x7f98180ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 --2- 172.21.15.46:0/2314106368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98181028e0 0x7f9818100150 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 >> 172.21.15.46:0/2314106368 conn(0x7f98180fb820 msgr2=0x7f98180fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 shutdown_connections 2024-05-08T23:18:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:10.814+0000 7f981f200700 1 -- 172.21.15.46:0/2314106368 wait complete. 2024-05-08T23:18:10.816 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-05-08T23:18:10.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:18:11.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.187+0000 7f66ecc00700 1 -- 172.21.15.46:0/4032779002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e80ffd90 msgr2=0x7f66e8100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 --2- 172.21.15.46:0/4032779002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e80ffd90 0x7f66e8100170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 -- 172.21.15.46:0/4032779002 shutdown_connections 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 --2- 172.21.15.46:0/4032779002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e8105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 --2- 172.21.15.46:0/4032779002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e80ffd90 0x7f66e8100170 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 --2- 172.21.15.46:0/4032779002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e81006b0 0x7f66e8104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 -- 172.21.15.46:0/4032779002 >> 172.21.15.46:0/4032779002 conn(0x7f66e80fb760 msgr2=0x7f66e80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 -- 172.21.15.46:0/4032779002 shutdown_connections 2024-05-08T23:18:11.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.188+0000 7f66ecc00700 1 -- 172.21.15.46:0/4032779002 wait complete. 2024-05-08T23:18:11.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.190+0000 7f66ecc00700 1 Processor -- start 2024-05-08T23:18:11.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.190+0000 7f66ecc00700 1 -- start start 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80ffd90 0x7f66e81a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 0x7f66e81a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 0x7f66e81a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 0x7f66e81a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43688/0 (socket says 172.21.15.46:43688) 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819dd70 con 0x7f66e80ffd90 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819dee0 con 0x7f66e81006b0 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66ecc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819e0a0 con 0x7f66e8105190 2024-05-08T23:18:11.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66e5400700 1 -- 172.21.15.46:0/1843265300 learned_addr learned my addr 172.21.15.46:0/1843265300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.191+0000 7f66e6800700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e5e00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80ffd90 0x7f66e81a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 -- 172.21.15.46:0/1843265300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 msgr2=0x7f66e81a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 0x7f66e81a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 -- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80ffd90 msgr2=0x7f66e81a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80ffd90 0x7f66e81a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 -- 172.21.15.46:0/1843265300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80095f0 con 0x7f66e8105190 2024-05-08T23:18:11.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.192+0000 7f66e6800700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e819d710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f66e0007f10 tx=0x7f66e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.193+0000 7f66dea00700 1 -- 172.21.15.46:0/1843265300 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000ddd0 con 0x7f66e8105190 2024-05-08T23:18:11.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.193+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e819e380 con 0x7f66e8105190 2024-05-08T23:18:11.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.193+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e81aa5d0 con 0x7f66e8105190 2024-05-08T23:18:11.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.193+0000 7f66dea00700 1 -- 172.21.15.46:0/1843265300 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e000f040 con 0x7f66e8105190 2024-05-08T23:18:11.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.193+0000 7f66dea00700 1 -- 172.21.15.46:0/1843265300 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0005a90 con 0x7f66e8105190 2024-05-08T23:18:11.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f66dea00700 1 -- 172.21.15.46:0/1843265300 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f66e0004ad0 con 0x7f66e8105190 2024-05-08T23:18:11.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 Processor -- start 2024-05-08T23:18:11.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 -- start start 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8cad150 con 0x5639e8d5c720 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8caab10 con 0x5639e8d5be00 2024-05-08T23:18:11.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.203+0000 7f67007f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8ca6400 con 0x5639e8d56890 2024-05-08T23:18:11.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.204+0000 7f66e5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.204+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.204+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43698/0 (socket says 172.21.15.46:43698) 2024-05-08T23:18:11.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.204+0000 7f66e5400700 1 -- 172.21.15.46:0/2353829512 learned_addr learned my addr 172.21.15.46:0/2353829512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.204+0000 7f66e5400700 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 -- 172.21.15.46:0/2353829512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 msgr2=0x5639e8d56cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d56cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 -- 172.21.15.46:0/2353829512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 msgr2=0x5639e8d56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 -- 172.21.15.46:0/2353829512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e8b273b0 con 0x5639e8d5be00 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5e00700 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66dea00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 0x7f66cc0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.205+0000 7f66e5400700 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f66d4007f80 tx=0x7f66d40078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c42175f3d153a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.206+0000 7f66dea00700 1 -- 172.21.15.46:0/1843265300 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66e00cc890 con 0x7f66e8105190 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.206+0000 7f66d3000700 1 -- 172.21.15.46:0/2353829512 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4004a30 con 0x5639e8d5be00 2024-05-08T23:18:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.206+0000 7f66d3000700 1 -- 172.21.15.46:0/2353829512 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d40054e0 con 0x5639e8d5be00 2024-05-08T23:18:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.206+0000 7f66d3000700 1 -- 172.21.15.46:0/2353829512 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4010730 con 0x5639e8d5be00 2024-05-08T23:18:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.206+0000 7f66e5e00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 0x7f66cc0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 -- 172.21.15.46:0/2353829512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 msgr2=0x5639e8d5c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f66d4007f80 tx=0x7f66d40078b0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f66e5e00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 0x7f66cc0a3850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006010 tx=0x7f66d801a040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 -- 172.21.15.46:0/2353829512 shutdown_connections 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5c1e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2353829512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 -- 172.21.15.46:0/2353829512 >> 172.21.15.46:0/2353829512 conn(0x5639e8cb4210 msgr2=0x5639e8d50df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.207+0000 7f67007f9b80 1 -- 172.21.15.46:0/2353829512 shutdown_connections 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.208+0000 7f67007f9b80 1 -- 172.21.15.46:0/2353829512 wait complete. 2024-05-08T23:18:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.208+0000 7f67007f9b80 1 -- start start 2024-05-08T23:18:11.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.208+0000 7f67007f9b80 1 Processor -- start 2024-05-08T23:18:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.209+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d56890 0x5639e8d5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 0x5639e8d604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f67007f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8c42560 con 0x5639e8d56890 2024-05-08T23:18:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f66e5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43712/0 (socket says 172.21.15.46:43712) 2024-05-08T23:18:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.210+0000 7f66e5400700 1 -- 172.21.15.46:0/3307386531 learned_addr learned my addr 172.21.15.46:0/3307386531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f66e5e00700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d56890 0x5639e8d5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8c42200 con 0x5639e8d5be00 2024-05-08T23:18:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8b296a0 con 0x5639e8d5c720 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f67007f9b80 10 client.0 _ll_register_callbacks cb 0x5639e8c4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f66e6800700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 0x5639e8d604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f66e5400700 1 -- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 msgr2=0x5639e8d604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f66e5400700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 0x5639e8d604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.211+0000 7f66e5400700 1 -- 172.21.15.46:0/3307386531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d56890 msgr2=0x5639e8d5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.212+0000 7f66e5400700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d56890 0x5639e8d5fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.212+0000 7f66e5400700 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e8114b40 con 0x5639e8d5be00 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.212+0000 7f66e6800700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 0x5639e8d604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.212+0000 7f66e5400700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005fa0 tx=0x7f66d40079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35532885f7c702bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.212+0000 7f66c7400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.213+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4023070 con 0x5639e8d5be00 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.213+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639e8bff270 con 0x5639e8d5be00 2024-05-08T23:18:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.213+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66d4004a20 con 0x5639e8d5be00 2024-05-08T23:18:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.214+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66d4005000 con 0x5639e8d5be00 2024-05-08T23:18:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.214+0000 7f67007f9b80 10 client.29417 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.214+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f66d4007df0 con 0x5639e8d5be00 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.214+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639e8bff270 con 0x5639e8d5be00 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.214+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66d4003960 con 0x5639e8d5be00 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 10 client.29417 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 10 client.29417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 20 client.29417 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8d700a0 0x5639e8d724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 4 client.29417 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.215+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639e8d25810 con 0x5639e8d700a0 2024-05-08T23:18:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.218+0000 7f66e5e00700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8d700a0 0x5639e8d724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.219+0000 7f66e5e00700 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8d700a0 0x5639e8d724c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b50c5769a660b9b server_cookie=59e52db0c2a22af9 in_seq=0 out_seq=0 2024-05-08T23:18:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.219+0000 7f66c7400700 10 client.29417 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f66c7400700 1 -- 172.21.15.46:0/3307386531 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639e8d25810 con 0x5639e8d700a0 2024-05-08T23:18:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f66c7400700 10 client.29417 handle_command_reply: tid=0 2024-05-08T23:18:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 client.29417 shutdown 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 msgr2=0x5639e8d5ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005fa0 tx=0x7f66d40079a0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 shutdown_connections 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8d700a0 0x5639e8d724c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d5c720 0x5639e8d604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d56890 0x5639e8d5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f66c7400700 0 client.29417 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 --2- 172.21.15.46:0/3307386531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.222+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 >> 172.21.15.46:0/3307386531 conn(0x5639e8cb4210 msgr2=0x5639e8d505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.223+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 shutdown_connections 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.223+0000 7f67007f9b80 1 -- 172.21.15.46:0/3307386531 wait complete. 2024-05-08T23:18:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.223+0000 7f67007f9b80 20 client.29417 trim_cache size 0 max 16384 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.245+0000 7f67007f9b80 1 Processor -- start 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.245+0000 7f67007f9b80 1 -- start start 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8d52e60 con 0x5639e8d5c720 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8d4ddc0 con 0x5639e8d5be00 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f67007f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8d77690 con 0x5639e8d56890 2024-05-08T23:18:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f66e5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f66e5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49576/0 (socket says 172.21.15.46:49576) 2024-05-08T23:18:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.246+0000 7f66e5e00700 1 -- 172.21.15.46:0/2245598788 learned_addr learned my addr 172.21.15.46:0/2245598788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5400700 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 13529 27.98 MB/sec warmup 32 sec latency 217.024 ms 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e6800700 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5e00700 1 -- 172.21.15.46:0/2245598788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 msgr2=0x5639e8d5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5e00700 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5e00700 1 -- 172.21.15.46:0/2245598788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 msgr2=0x5639e8d604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5e00700 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.247+0000 7f66e5e00700 1 -- 172.21.15.46:0/2245598788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e8d56f40 con 0x5639e8d56890 2024-05-08T23:18:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.248+0000 7f66e5e00700 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d8021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9396d31cdbaae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.248+0000 7f66d3a00700 1 -- 172.21.15.46:0/2245598788 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8036070 con 0x5639e8d56890 2024-05-08T23:18:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.249+0000 7f66d3a00700 1 -- 172.21.15.46:0/2245598788 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d8004c60 con 0x5639e8d56890 2024-05-08T23:18:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.249+0000 7f66d3a00700 1 -- 172.21.15.46:0/2245598788 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d801d040 con 0x5639e8d56890 2024-05-08T23:18:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 -- 172.21.15.46:0/2245598788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 msgr2=0x5639e8d50d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d8021000 comp rx=0 tx=0).stop 2024-05-08T23:18:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 -- 172.21.15.46:0/2245598788 shutdown_connections 2024-05-08T23:18:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5c720 0x5639e8d604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d50d20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 --2- 172.21.15.46:0/2245598788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5be00 0x5639e8d5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.250+0000 7f67007f9b80 1 -- 172.21.15.46:0/2245598788 >> 172.21.15.46:0/2245598788 conn(0x5639e8cb4210 msgr2=0x5639e8d5e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.251+0000 7f67007f9b80 1 -- 172.21.15.46:0/2245598788 shutdown_connections 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.251+0000 7f67007f9b80 1 -- 172.21.15.46:0/2245598788 wait complete. 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.251+0000 7f67007f9b80 1 -- start start 2024-05-08T23:18:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.252+0000 7f67007f9b80 1 Processor -- start 2024-05-08T23:18:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.253+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d65420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.253+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f67007f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5c720 0x5639e8dbf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f67007f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8dbfa70 con 0x5639e8d5be00 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f67007f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8dbfbe0 con 0x5639e8d5c720 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f67007f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e8dbfda0 con 0x5639e8d56890 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f67007f9b80 10 client.0 _ll_register_callbacks cb 0x5639e8c4bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f66e5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f66e5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58056/0 (socket says 172.21.15.46:58056) 2024-05-08T23:18:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f66e5400700 1 -- 172.21.15.46:0/893047346 learned_addr learned my addr 172.21.15.46:0/893047346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f66e6800700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5c720 0x5639e8dbf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.254+0000 7f66e5e00700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d65420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5e00700 1 -- 172.21.15.46:0/893047346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5c720 msgr2=0x5639e8dbf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5e00700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5c720 0x5639e8dbf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5e00700 1 -- 172.21.15.46:0/893047346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 msgr2=0x5639e8d65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5e00700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5e00700 1 -- 172.21.15.46:0/893047346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d802a040 con 0x5639e8d56890 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.255+0000 7f66e5400700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.256+0000 7f66e5e00700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d65420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d800f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e968e91c27b01c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.256+0000 7f66c7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.256+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8036040 con 0x5639e8d56890 2024-05-08T23:18:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.257+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66d8005510 con 0x5639e8d56890 2024-05-08T23:18:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.257+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66d40054e0 con 0x5639e8d56890 2024-05-08T23:18:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.257+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66d801d040 con 0x5639e8d56890 2024-05-08T23:18:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.257+0000 7f67007f9b80 10 client.29515 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.257+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5639e8ad3b60 con 0x5639e8d56890 2024-05-08T23:18:11.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d800f7b0 con 0x5639e8d56890 2024-05-08T23:18:11.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66d8025460 con 0x5639e8d56890 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 10 client.29515 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 10 client.29515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 20 client.29515 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8dc1090 0x5639e8dc1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 4 client.29515 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f66d40054e0 con 0x5639e8dc1090 2024-05-08T23:18:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.259+0000 7f66e5400700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8dc1090 0x5639e8dc1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.260+0000 7f66c7400700 10 client.29515 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.260+0000 7f66e5400700 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8dc1090 0x5639e8dc1470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c51ab5eba4de7a2 server_cookie=3533d4c866ded554 in_seq=0 out_seq=0 2024-05-08T23:18:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f66c7400700 1 -- 172.21.15.46:0/893047346 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f66d40054e0 con 0x5639e8dc1090 2024-05-08T23:18:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f66c7400700 10 client.29515 handle_command_reply: tid=0 2024-05-08T23:18:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 client.29515 shutdown 2024-05-08T23:18:11.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:10 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2314106368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 msgr2=0x5639e8d65420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d65420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d800f460 comp rx=0 tx=0).stop 2024-05-08T23:18:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 shutdown_connections 2024-05-08T23:18:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5639e8dc1090 0x5639e8dc1470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e8d5c720 0x5639e8dbf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639e8d56890 0x5639e8d65420 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 --2- 172.21.15.46:0/893047346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5639e8d5be00 0x5639e8d65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f66c7400700 0 client.29515 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.261+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 >> 172.21.15.46:0/893047346 conn(0x5639e8cb4210 msgr2=0x5639e8d5ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.263+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 shutdown_connections 2024-05-08T23:18:11.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.263+0000 7f67007f9b80 1 -- 172.21.15.46:0/893047346 wait complete. 2024-05-08T23:18:11.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.264+0000 7f67007f9b80 20 client.29515 trim_cache size 0 max 16384 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.264+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 msgr2=0x7f66cc0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.264+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 0x7f66cc0a3850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006010 tx=0x7f66d801a040 comp rx=0 tx=0).stop 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.264+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 msgr2=0x7f66e819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.264+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e819d710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f66e0007f10 tx=0x7f66e000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 shutdown_connections 2024-05-08T23:18:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e8105190 0x7f66e819d710 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66cc0a1390 0x7f66cc0a3850 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80ffd90 0x7f66e81a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 --2- 172.21.15.46:0/1843265300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66e81006b0 0x7f66e81a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 >> 172.21.15.46:0/1843265300 conn(0x7f66e80fb760 msgr2=0x7f66e8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.265+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 shutdown_connections 2024-05-08T23:18:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.266+0000 7f66ecc00700 1 -- 172.21.15.46:0/1843265300 wait complete. 2024-05-08T23:18:11.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:10 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2314106368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:11.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:18:11.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:10 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2314106368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 -- 172.21.15.46:0/1593658983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 msgr2=0x7f478c103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 --2- 172.21.15.46:0/1593658983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c103690 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f477801a9a0 tx=0x7f477801acb0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 -- 172.21.15.46:0/1593658983 shutdown_connections 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 --2- 172.21.15.46:0/1593658983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 0x7f478c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 --2- 172.21.15.46:0/1593658983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c103690 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 --2- 172.21.15.46:0/1593658983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c103c60 0x7f478c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 -- 172.21.15.46:0/1593658983 >> 172.21.15.46:0/1593658983 conn(0x7f478c0feb10 msgr2=0x7f478c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.627+0000 7f4791000700 1 -- 172.21.15.46:0/1593658983 shutdown_connections 2024-05-08T23:18:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.628+0000 7f4791000700 1 -- 172.21.15.46:0/1593658983 wait complete. 2024-05-08T23:18:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.629+0000 7f4791000700 1 Processor -- start 2024-05-08T23:18:11.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.629+0000 7f4791000700 1 -- start start 2024-05-08T23:18:11.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.630+0000 7f4791000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c10f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.630+0000 7f4791000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 0x7f478c10fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4791000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 0x7f478c0754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c10f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c10f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49592/0 (socket says 172.21.15.46:49592) 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4791000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c075ca0 con 0x7f478c1083d0 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4789e00700 1 -- 172.21.15.46:0/3335887635 learned_addr learned my addr 172.21.15.46:0/3335887635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c075e10 con 0x7f478c0793f0 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4789400700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 0x7f478c10fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c075fd0 con 0x7f478c1032b0 2024-05-08T23:18:11.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.631+0000 7f478a800700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 0x7f478c0754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 -- 172.21.15.46:0/3335887635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 msgr2=0x7f478c10f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c10f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 -- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 msgr2=0x7f478c10fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 0x7f478c10fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 -- 172.21.15.46:0/3335887635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784009c40 con 0x7f478c0793f0 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.632+0000 7f478a800700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 0x7f478c0754d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f478400efb0 tx=0x7f478400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.633+0000 7f4782a00700 1 -- 172.21.15.46:0/3335887635 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784003a00 con 0x7f478c0793f0 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.633+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477801a600 con 0x7f478c0793f0 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.633+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c1a2130 con 0x7f478c0793f0 2024-05-08T23:18:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.633+0000 7f4782a00700 1 -- 172.21.15.46:0/3335887635 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478400f040 con 0x7f478c0793f0 2024-05-08T23:18:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.633+0000 7f4782a00700 1 -- 172.21.15.46:0/3335887635 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784004540 con 0x7f478c0793f0 2024-05-08T23:18:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.638+0000 7f4782a00700 1 -- 172.21.15.46:0/3335887635 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4784004780 con 0x7f478c0793f0 2024-05-08T23:18:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.639+0000 7f47a4b8cb80 1 Processor -- start 2024-05-08T23:18:11.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 -- start start 2024-05-08T23:18:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f99720 0x55b458f93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458eea150 con 0x55b458f99720 2024-05-08T23:18:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458ee7b10 con 0x55b458f98e00 2024-05-08T23:18:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.640+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458ee3400 con 0x55b458f93890 2024-05-08T23:18:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43746/0 (socket says 172.21.15.46:43746) 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 -- 172.21.15.46:0/2813089701 learned_addr learned my addr 172.21.15.46:0/2813089701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 -- 172.21.15.46:0/2813089701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 msgr2=0x55b458f991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 -- 172.21.15.46:0/2813089701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f99720 msgr2=0x55b458f93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f99720 0x55b458f93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4789400700 1 -- 172.21.15.46:0/2813089701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b458d643b0 con 0x55b458f93890 2024-05-08T23:18:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.641+0000 7f4782a00700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 0x7f47700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.642+0000 7f4789400700 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f93cd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f477c011820 tx=0x7f477c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5bcd76aab0cf69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.642+0000 7f4777400700 1 -- 172.21.15.46:0/2813089701 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c004830 con 0x55b458f93890 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.642+0000 7f4777400700 1 -- 172.21.15.46:0/2813089701 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c004e50 con 0x55b458f93890 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.642+0000 7f4777400700 1 -- 172.21.15.46:0/2813089701 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c01f6b0 con 0x55b458f93890 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.643+0000 7f4782a00700 1 -- 172.21.15.46:0/3335887635 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f47840cdc20 con 0x7f478c0793f0 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.643+0000 7f4789e00700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 0x7f47700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.643+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2813089701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 msgr2=0x55b458f93cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.643+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f93cd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f477c011820 tx=0x7f477c009260 comp rx=0 tx=0).stop 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.644+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2813089701 shutdown_connections 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.644+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f93cd0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.644+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.644+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2813089701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f99720 0x55b458f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.644+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2813089701 >> 172.21.15.46:0/2813089701 conn(0x55b458ef1210 msgr2=0x55b458f8ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.645+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2813089701 shutdown_connections 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.645+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2813089701 wait complete. 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.645+0000 7f4789e00700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 0x7f47700a37d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55b458f8fbd0 tx=0x7f477802b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.645+0000 7f47a4b8cb80 1 -- start start 2024-05-08T23:18:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.645+0000 7f47a4b8cb80 1 Processor -- start 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f9cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458f9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458f9d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458e7f560 con 0x55b458f98e00 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458e7f200 con 0x55b458f99720 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458d666a0 con 0x55b458f93890 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f9cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f47a4b8cb80 10 client.0 _ll_register_callbacks cb 0x55b458e88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f9cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49614/0 (socket says 172.21.15.46:49614) 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f4789e00700 1 -- 172.21.15.46:0/4005109555 learned_addr learned my addr 172.21.15.46:0/4005109555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.646+0000 7f4789400700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458f9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458f9d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 -- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 msgr2=0x55b458f9cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f9cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 -- 172.21.15.46:0/4005109555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 msgr2=0x55b458f9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458f9d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 -- 172.21.15.46:0/4005109555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c019040 con 0x55b458f99720 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f4789400700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458f9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f478a800700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458f9d5d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47840caef0 tx=0x7f478409a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d462fafc0cc154 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f476be00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47840b8ea0 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.647+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f478401b520 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b458e3c270 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47840b3250 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 10 client.29435 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b458d10b60 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b458e3c270 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4784004bd0 con 0x55b458f99720 2024-05-08T23:18:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 10 client.29435 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 10 client.29435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 20 client.29435 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.648+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b458fad160 0x55b458faf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.649+0000 7f47a4b8cb80 4 client.29435 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.649+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b458f62810 con 0x55b458fad160 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.649+0000 7f4789e00700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b458fad160 0x55b458faf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.650+0000 7f4789e00700 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b458fad160 0x55b458faf580 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ade4d2c7dc040560 server_cookie=6f8a71a8962fcedd in_seq=0 out_seq=0 2024-05-08T23:18:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.650+0000 7f476be00700 10 client.29435 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.652+0000 7f476be00700 1 -- 172.21.15.46:0/4005109555 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b458f62810 con 0x55b458fad160 2024-05-08T23:18:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.652+0000 7f476be00700 10 client.29435 handle_command_reply: tid=0 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.652+0000 7f47a4b8cb80 1 client.29435 shutdown 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.654+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 msgr2=0x55b458f9d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.654+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458f9d5d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47840caef0 tx=0x7f478409a2a0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 shutdown_connections 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b458fad160 0x55b458faf580 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458f9d5d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458f9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/4005109555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458f9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 >> 172.21.15.46:0/4005109555 conn(0x55b458ef1210 msgr2=0x55b458f8d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f476be00700 0 client.29435 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 shutdown_connections 2024-05-08T23:18:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.656+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/4005109555 wait complete. 2024-05-08T23:18:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.657+0000 7f47a4b8cb80 20 client.29435 trim_cache size 0 max 16384 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 Processor -- start 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 -- start start 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 0x55b458f93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f8dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 0x55b458f97c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458faa7f0 con 0x55b458f93890 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458f9e150 con 0x55b458f98e00 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.683+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458f8ee80 con 0x55b458f99720 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f4789e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 0x55b458f93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 0x55b458f97c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f4789e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 0x55b458f93350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58122/0 (socket says 172.21.15.46:58122) 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f4789e00700 1 -- 172.21.15.46:0/3492144753 learned_addr learned my addr 172.21.15.46:0/3492144753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f4789400700 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f8dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 -- 172.21.15.46:0/3492144753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 msgr2=0x55b458f8dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f8dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 -- 172.21.15.46:0/3492144753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 msgr2=0x55b458f93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 0x55b458f93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 -- 172.21.15.46:0/3492144753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b458ea9560 con 0x55b458f99720 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.684+0000 7f478a800700 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 0x55b458f97c20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f47840caef0 tx=0x7f47840046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f722e7a84ca5ffd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.685+0000 7f4777e00700 1 -- 172.21.15.46:0/3492144753 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47840b8ea0 con 0x55b458f99720 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.685+0000 7f4777e00700 1 -- 172.21.15.46:0/3492144753 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4784085040 con 0x55b458f99720 2024-05-08T23:18:11.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.685+0000 7f4777e00700 1 -- 172.21.15.46:0/3492144753 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47840a2b50 con 0x55b458f99720 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/3492144753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 msgr2=0x55b458f97c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 0x55b458f97c20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f47840caef0 tx=0x7f47840046a0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/3492144753 shutdown_connections 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f99720 0x55b458f97c20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f93890 0x55b458f93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/3492144753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f98e00 0x55b458f8dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.686+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/3492144753 >> 172.21.15.46:0/3492144753 conn(0x55b458ef1210 msgr2=0x55b458f96ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.687+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/3492144753 shutdown_connections 2024-05-08T23:18:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.687+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/3492144753 wait complete. 2024-05-08T23:18:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.687+0000 7f47a4b8cb80 1 -- start start 2024-05-08T23:18:11.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.688+0000 7f47a4b8cb80 1 Processor -- start 2024-05-08T23:18:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.688+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458fa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458fa2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f4789400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458fa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458fa2e00 con 0x55b458f98e00 2024-05-08T23:18:11.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458fa2f70 con 0x55b458f99720 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b459013630 con 0x55b458f93890 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f47a4b8cb80 10 client.0 _ll_register_callbacks cb 0x55b458e88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f4789400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458fa21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58126/0 (socket says 172.21.15.46:58126) 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f478a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458fa2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f478a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458fa2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43776/0 (socket says 172.21.15.46:43776) 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789400700 1 -- 172.21.15.46:0/2619583105 learned_addr learned my addr 172.21.15.46:0/2619583105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:11.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.689+0000 7f4789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:49634/0 (socket says 172.21.15.46:49634) 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789e00700 1 -- 172.21.15.46:0/2619583105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 msgr2=0x55b458fa2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789e00700 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458fa2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789e00700 1 -- 172.21.15.46:0/2619583105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 msgr2=0x55b458fa21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789e00700 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458fa21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.690+0000 7f4789e00700 1 -- 172.21.15.46:0/2619583105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b458d10b60 con 0x55b458f93890 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f4789e00700 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4778006010 tx=0x7f4778005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5abd9e57455637 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f476be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477803c040 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f477c004e50 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f4778003a90 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477802e040 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f47a4b8cb80 10 client.29533 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.691+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b458d68cd0 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c004e50 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4778036620 con 0x55b458f93890 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 10 client.29533 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 10 client.29533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 20 client.29533 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:11.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b459014d40 0x55b459017160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 4 client.29533 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:11.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b458edbec0 con 0x55b459014d40 2024-05-08T23:18:11.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.692+0000 7f4789400700 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b459014d40 0x55b459017160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.693+0000 7f4789400700 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b459014d40 0x55b459017160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=570cfa2acdc5b152 server_cookie=863e21bd193bcce0 in_seq=0 out_seq=0 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.693+0000 7f476be00700 10 client.29533 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.693+0000 7f476be00700 1 -- 172.21.15.46:0/2619583105 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b458edbec0 con 0x55b459014d40 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.693+0000 7f476be00700 10 client.29533 handle_command_reply: tid=0 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.693+0000 7f47a4b8cb80 1 client.29533 shutdown 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.694+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 msgr2=0x55b458fa44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.694+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4778006010 tx=0x7f4778005050 comp rx=0 tx=0).stop 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 shutdown_connections 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55b459014d40 0x55b459017160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b458f99720 0x55b458fa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f476be00700 0 client.29533 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b458f93890 0x55b458fa44f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 --2- 172.21.15.46:0/2619583105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b458f98e00 0x55b458fa21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 >> 172.21.15.46:0/2619583105 conn(0x55b458ef1210 msgr2=0x55b458f97c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 shutdown_connections 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 1 -- 172.21.15.46:0/2619583105 wait complete. 2024-05-08T23:18:11.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.695+0000 7f47a4b8cb80 20 client.29533 trim_cache size 0 max 16384 2024-05-08T23:18:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.696+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 msgr2=0x7f47700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.696+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 0x7f47700a37d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55b458f8fbd0 tx=0x7f477802b040 comp rx=0 tx=0).stop 2024-05-08T23:18:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.696+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 msgr2=0x7f478c0754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.696+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 0x7f478c0754d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f478400efb0 tx=0x7f478400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 shutdown_connections 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f478c0793f0 0x7f478c0754d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f478c1032b0 0x7f478c10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47700a1310 0x7f47700a37d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 --2- 172.21.15.46:0/3335887635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478c1083d0 0x7f478c10fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.698+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 >> 172.21.15.46:0/3335887635 conn(0x7f478c0feb10 msgr2=0x7f478c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.700+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 shutdown_connections 2024-05-08T23:18:11.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:11.700+0000 7f4791000700 1 -- 172.21.15.46:0/3335887635 wait complete. 2024-05-08T23:18:11.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:18:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 -- 172.21.15.46:0/2151780561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544101a50 msgr2=0x7f3544105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 --2- 172.21.15.46:0/2151780561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544101a50 0x7f3544105a90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3534011f90 tx=0x7f35340134c0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 -- 172.21.15.46:0/2151780561 shutdown_connections 2024-05-08T23:18:12.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 --2- 172.21.15.46:0/2151780561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 0x7f354410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 --2- 172.21.15.46:0/2151780561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35441010a0 0x7f3544101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 --2- 172.21.15.46:0/2151780561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544101a50 0x7f3544105a90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.066+0000 7f354aa00700 1 -- 172.21.15.46:0/2151780561 >> 172.21.15.46:0/2151780561 conn(0x7f35440fc960 msgr2=0x7f35440fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.067+0000 7f354aa00700 1 -- 172.21.15.46:0/2151780561 shutdown_connections 2024-05-08T23:18:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.067+0000 7f354aa00700 1 -- 172.21.15.46:0/2151780561 wait complete. 2024-05-08T23:18:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.068+0000 7f354aa00700 1 Processor -- start 2024-05-08T23:18:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.068+0000 7f354aa00700 1 -- start start 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35441010a0 0x7f3544195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 0x7f3544195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419a4b0 con 0x7f35441061c0 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419a620 con 0x7f35441010a0 2024-05-08T23:18:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f354aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354419a7e0 con 0x7f3544199a30 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f3548a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f3548a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47256/0 (socket says 172.21.15.46:47256) 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f3548a00700 1 -- 172.21.15.46:0/1665922907 learned_addr learned my addr 172.21.15.46:0/1665922907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.069+0000 7f3543e00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35441010a0 0x7f3544195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.070+0000 7f3548a00700 1 -- 172.21.15.46:0/1665922907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35441010a0 msgr2=0x7f3544195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.070+0000 7f3548a00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35441010a0 0x7f3544195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.070+0000 7f3548a00700 1 -- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 msgr2=0x7f3544195690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.070+0000 7f3543400700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 0x7f3544195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3548a00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 0x7f3544195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3548a00700 1 -- 172.21.15.46:0/1665922907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530009520 con 0x7f3544199a30 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3548a00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f353c007ee0 tx=0x7f353c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3540c00700 1 -- 172.21.15.46:0/1665922907 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c017070 con 0x7f3544199a30 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3540c00700 1 -- 172.21.15.46:0/1665922907 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f353c00f040 con 0x7f3544199a30 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3534011e00 con 0x7f3544199a30 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f3540c00700 1 -- 172.21.15.46:0/1665922907 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c005740 con 0x7f3544199a30 2024-05-08T23:18:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.071+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35441a20e0 con 0x7f3544199a30 2024-05-08T23:18:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.075+0000 7f3540c00700 1 -- 172.21.15.46:0/1665922907 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f353c0058a0 con 0x7f3544199a30 2024-05-08T23:18:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.077+0000 7f3540c00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 0x7f35280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.077+0000 7f3540c00700 1 -- 172.21.15.46:0/1665922907 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f353c0cd6f0 con 0x7f3544199a30 2024-05-08T23:18:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.078+0000 7f3543e00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 0x7f35280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.078+0000 7f3543e00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 0x7f35280a37d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3530009e40 tx=0x7f3530009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.079+0000 7f355e602b80 1 Processor -- start 2024-05-08T23:18:12.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.079+0000 7f355e602b80 1 -- start start 2024-05-08T23:18:12.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c916cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c86d150 con 0x558c1c91be00 2024-05-08T23:18:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c86ab10 con 0x558c1c91c720 2024-05-08T23:18:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f355e602b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c866400 con 0x558c1c916890 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f3543e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f3543400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f3543400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36218/0 (socket says 172.21.15.46:36218) 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f3543400700 1 -- 172.21.15.46:0/3186157192 learned_addr learned my addr 172.21.15.46:0/3186157192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.080+0000 7f3548a00700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c916cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 -- 172.21.15.46:0/3186157192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 msgr2=0x558c1c916cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c916cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 -- 172.21.15.46:0/3186157192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 msgr2=0x558c1c91c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 -- 172.21.15.46:0/3186157192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1c6e73b0 con 0x558c1c91c720 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543e00700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3548a00700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c916cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3543400700 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35340059d0 tx=0x7f3534005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0ed4681efe4fdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.081+0000 7f3539200700 1 -- 172.21.15.46:0/3186157192 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534011530 con 0x558c1c91c720 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.082+0000 7f3539200700 1 -- 172.21.15.46:0/3186157192 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3534005580 con 0x558c1c91c720 2024-05-08T23:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.082+0000 7f3539200700 1 -- 172.21.15.46:0/3186157192 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534003cc0 con 0x558c1c91c720 2024-05-08T23:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.082+0000 7f355e602b80 1 -- 172.21.15.46:0/3186157192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 msgr2=0x558c1c916350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.082+0000 7f355e602b80 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35340059d0 tx=0x7f3534005c00 comp rx=0 tx=0).stop 2024-05-08T23:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 -- 172.21.15.46:0/3186157192 shutdown_connections 2024-05-08T23:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c916cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 --2- 172.21.15.46:0/3186157192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c916350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 -- 172.21.15.46:0/3186157192 >> 172.21.15.46:0/3186157192 conn(0x558c1c874210 msgr2=0x558c1c910df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 -- 172.21.15.46:0/3186157192 shutdown_connections 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 -- 172.21.15.46:0/3186157192 wait complete. 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.083+0000 7f355e602b80 1 -- start start 2024-05-08T23:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.084+0000 7f355e602b80 1 Processor -- start 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.084+0000 7f355e602b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c91fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.084+0000 7f355e602b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.084+0000 7f355e602b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91c720 0x558c1c9204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f355e602b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c802560 con 0x558c1c91c720 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f355e602b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c802200 con 0x558c1c91be00 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f355e602b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c6e96a0 con 0x558c1c916890 2024-05-08T23:18:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36230/0 (socket says 172.21.15.46:36230) 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 -- 172.21.15.46:0/244752370 learned_addr learned my addr 172.21.15.46:0/244752370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543e00700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c91fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f355e602b80 10 client.0 _ll_register_callbacks cb 0x558c1c80bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 -- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 msgr2=0x558c1c91fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c91fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 -- 172.21.15.46:0/244752370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91c720 msgr2=0x558c1c9204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91c720 0x558c1c9204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 -- 172.21.15.46:0/244752370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544073550 con 0x558c1c91be00 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f3543400700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3534005d10 tx=0x7f3534005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cb206b6c4ec4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.085+0000 7f352d600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.086+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353403c070 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.086+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f3534003720 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.086+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c1c7bf270 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35340138a0 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 10 client.29453 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558c1c693b60 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c1c7bf270 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f353404b6e0 con 0x558c1c91be00 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 10 client.29453 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 10 client.29453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 20 client.29453 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c92fb90 0x558c1c931ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 4 client.29453 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.087+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c1c8e5810 con 0x558c1c92fb90 2024-05-08T23:18:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.088+0000 7f3543e00700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c92fb90 0x558c1c931ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.089+0000 7f3543e00700 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c92fb90 0x558c1c931ff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9480c06b4fb10572 server_cookie=e9da1b54350ba44a in_seq=0 out_seq=0 2024-05-08T23:18:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.089+0000 7f352d600700 10 client.29453 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.091+0000 7f352d600700 1 -- 172.21.15.46:0/244752370 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c1c8e5810 con 0x558c1c92fb90 2024-05-08T23:18:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.091+0000 7f352d600700 10 client.29453 handle_command_reply: tid=0 2024-05-08T23:18:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.091+0000 7f355e602b80 1 client.29453 shutdown 2024-05-08T23:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 msgr2=0x558c1c91ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3534005d10 tx=0x7f3534005280 comp rx=0 tx=0).stop 2024-05-08T23:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 shutdown_connections 2024-05-08T23:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c92fb90 0x558c1c931ff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91c720 0x558c1c9204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c916890 0x558c1c91fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 --2- 172.21.15.46:0/244752370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91be00 0x558c1c91ff90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 >> 172.21.15.46:0/244752370 conn(0x558c1c874210 msgr2=0x558c1c9105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.092+0000 7f352d600700 0 client.29453 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.093+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 shutdown_connections 2024-05-08T23:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.093+0000 7f355e602b80 1 -- 172.21.15.46:0/244752370 wait complete. 2024-05-08T23:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.093+0000 7f355e602b80 20 client.29453 trim_cache size 0 max 16384 2024-05-08T23:18:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.111+0000 7f355e602b80 1 Processor -- start 2024-05-08T23:18:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.111+0000 7f355e602b80 1 -- start start 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.111+0000 7f355e602b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 0x558c1c910d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.111+0000 7f355e602b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f355e602b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f355e602b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c911920 con 0x558c1c91be00 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f355e602b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c9210f0 con 0x558c1c916890 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f355e602b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c91bad0 con 0x558c1c91c720 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3543e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 0x558c1c910d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47278/0 (socket says 172.21.15.46:47278) 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 -- 172.21.15.46:0/1794855587 learned_addr learned my addr 172.21.15.46:0/1794855587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3543e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 0x558c1c910d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36238/0 (socket says 172.21.15.46:36238) 2024-05-08T23:18:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3543400700 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 -- 172.21.15.46:0/1794855587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 msgr2=0x558c1c910d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 0x558c1c910d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 -- 172.21.15.46:0/1794855587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 msgr2=0x558c1c91b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.112+0000 7f3548a00700 1 -- 172.21.15.46:0/1794855587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1c82c560 con 0x558c1c91c720 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.113+0000 7f3548a00700 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f353c009510 tx=0x7f353c007470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0bac21207866bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.113+0000 7f3539c00700 1 -- 172.21.15.46:0/1794855587 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c099ca0 con 0x558c1c91c720 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.113+0000 7f3539c00700 1 -- 172.21.15.46:0/1794855587 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f353c0cd070 con 0x558c1c91c720 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.113+0000 7f3539c00700 1 -- 172.21.15.46:0/1794855587 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0a27c0 con 0x558c1c91c720 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- 172.21.15.46:0/1794855587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 msgr2=0x558c1c9202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f353c009510 tx=0x7f353c007470 comp rx=0 tx=0).stop 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- 172.21.15.46:0/1794855587 shutdown_connections 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91c720 0x558c1c9202f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c916890 0x558c1c910d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 --2- 172.21.15.46:0/1794855587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c91be00 0x558c1c91b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- 172.21.15.46:0/1794855587 >> 172.21.15.46:0/1794855587 conn(0x558c1c874210 msgr2=0x558c1c91ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- 172.21.15.46:0/1794855587 shutdown_connections 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- 172.21.15.46:0/1794855587 wait complete. 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.114+0000 7f355e602b80 1 -- start start 2024-05-08T23:18:12.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.115+0000 7f355e602b80 1 Processor -- start 2024-05-08T23:18:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c916890 0x558c1c9262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 0x558c1c9267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c926d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c985120 con 0x558c1c916890 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c985260 con 0x558c1c91c720 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1c9853d0 con 0x558c1c91be00 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f355e602b80 10 client.0 _ll_register_callbacks cb 0x558c1c80bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f3543400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 0x558c1c9267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f3543400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 0x558c1c9267e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47282/0 (socket says 172.21.15.46:47282) 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f3543400700 1 -- 172.21.15.46:0/1233198664 learned_addr learned my addr 172.21.15.46:0/1233198664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.116+0000 7f3548a00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c926d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 -- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 msgr2=0x558c1c9267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 0x558c1c9267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 -- 172.21.15.46:0/1233198664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c916890 msgr2=0x558c1c9262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c916890 0x558c1c9262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 -- 172.21.15.46:0/1233198664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35441a20e0 con 0x558c1c91c720 2024-05-08T23:18:12.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f3548a00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c926d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f353c09a040 tx=0x7f353c019550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbdd5d4c94be2d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.117+0000 7f352d600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:12.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c07d070 con 0x558c1c91c720 2024-05-08T23:18:12.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f353c004ad0 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3534005580 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f353c0a27c0 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 10 client.29459 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558c1c6ebcd0 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3534005580 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f353c080020 con 0x558c1c91c720 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 10 client.29459 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 10 client.29459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 20 client.29459 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c986a70 0x558c1c988e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 4 client.29459 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.118+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c1c85eec0 con 0x558c1c986a70 2024-05-08T23:18:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.119+0000 7f3543e00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c986a70 0x558c1c988e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.119+0000 7f352d600700 10 client.29459 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:12.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.119+0000 7f3543e00700 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c986a70 0x558c1c988e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f92faa976c2ec30 server_cookie=91eef6f9f777b380 in_seq=0 out_seq=0 2024-05-08T23:18:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.122+0000 7f352d600700 1 -- 172.21.15.46:0/1233198664 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c1c85eec0 con 0x558c1c986a70 2024-05-08T23:18:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.122+0000 7f352d600700 10 client.29459 handle_command_reply: tid=0 2024-05-08T23:18:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.122+0000 7f355e602b80 1 client.29459 shutdown 2024-05-08T23:18:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 msgr2=0x558c1c926d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c926d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f353c09a040 tx=0x7f353c019550 comp rx=0 tx=0).stop 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 shutdown_connections 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c1c91c720 0x558c1c926d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x558c1c986a70 0x558c1c988e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1c916890 0x558c1c9262a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 --2- 172.21.15.46:0/1233198664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c1c91be00 0x558c1c9267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 >> 172.21.15.46:0/1233198664 conn(0x558c1c874210 msgr2=0x558c1c91e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f352d600700 0 client.29459 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 shutdown_connections 2024-05-08T23:18:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.123+0000 7f355e602b80 1 -- 172.21.15.46:0/1233198664 wait complete. 2024-05-08T23:18:12.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.124+0000 7f355e602b80 20 client.29459 trim_cache size 0 max 16384 2024-05-08T23:18:12.127 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 msgr2=0x7f35280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 0x7f35280a37d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3530009e40 tx=0x7f3530009260 comp rx=0 tx=0).stop 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 msgr2=0x7f3544199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f353c007ee0 tx=0x7f353c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 shutdown_connections 2024-05-08T23:18:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3544199a30 0x7f3544199eb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35441010a0 0x7f3544195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f35280a1310 0x7f35280a37d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 --2- 172.21.15.46:0/1665922907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35441061c0 0x7f3544195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 >> 172.21.15.46:0/1665922907 conn(0x7f35440fc960 msgr2=0x7f35440fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 shutdown_connections 2024-05-08T23:18:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.125+0000 7f354aa00700 1 -- 172.21.15.46:0/1665922907 wait complete. 2024-05-08T23:18:12.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:18:12.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14049 28.35 MB/sec warmup 33 sec latency 55.947 ms 2024-05-08T23:18:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:11 smithi053 ceph-mon[77587]: pgmap v448: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.8 MiB/s wr, 622 op/s 2024-05-08T23:18:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:11 smithi053 ceph-mon[77587]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:18:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:11 smithi046 ceph-mon[36898]: pgmap v448: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.8 MiB/s wr, 622 op/s 2024-05-08T23:18:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:11 smithi046 ceph-mon[36898]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:18:12.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 -- 172.21.15.46:0/1521878194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444104b40 msgr2=0x7f9444104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 --2- 172.21.15.46:0/1521878194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444104b40 0x7f9444104f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9434009990 tx=0x7f9434009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 -- 172.21.15.46:0/1521878194 shutdown_connections 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 --2- 172.21.15.46:0/1521878194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9444100730 0x7f944410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 --2- 172.21.15.46:0/1521878194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444104b40 0x7f9444104f20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 --2- 172.21.15.46:0/1521878194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94440ffd90 0x7f94441001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 -- 172.21.15.46:0/1521878194 >> 172.21.15.46:0/1521878194 conn(0x7f9444075710 msgr2=0x7f9444075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 -- 172.21.15.46:0/1521878194 shutdown_connections 2024-05-08T23:18:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.399+0000 7f9449000700 1 -- 172.21.15.46:0/1521878194 wait complete. 2024-05-08T23:18:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.400+0000 7f9449000700 1 Processor -- start 2024-05-08T23:18:12.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.400+0000 7f9449000700 1 -- start start 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94440ffd90 0x7f9444101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 0x7f94441022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a53b0 con 0x7f9444104b40 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a5520 con 0x7f9444100730 2024-05-08T23:18:12.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9449000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441a56e0 con 0x7f94440ffd90 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9443400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94440ffd90 0x7f9444101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36254/0 (socket says 172.21.15.46:36254) 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.401+0000 7f9442a00700 1 -- 172.21.15.46:0/1829976725 learned_addr learned my addr 172.21.15.46:0/1829976725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9443e00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 0x7f94441022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 -- 172.21.15.46:0/1829976725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94440ffd90 msgr2=0x7f9444101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94440ffd90 0x7f9444101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 -- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 msgr2=0x7f94441022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 0x7f94441022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 -- 172.21.15.46:0/1829976725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94340095f0 con 0x7f9444100730 2024-05-08T23:18:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9443e00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 0x7f94441022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f9442a00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9a0 tx=0x7f943000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.402+0000 7f943be00700 1 -- 172.21.15.46:0/1829976725 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430009790 con 0x7f9444100730 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.403+0000 7f943be00700 1 -- 172.21.15.46:0/1829976725 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943000cbd0 con 0x7f9444100730 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.403+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94441a59c0 con 0x7f9444100730 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.403+0000 7f943be00700 1 -- 172.21.15.46:0/1829976725 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430018680 con 0x7f9444100730 2024-05-08T23:18:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.403+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441a5f20 con 0x7f9444100730 2024-05-08T23:18:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.406+0000 7f943be00700 1 -- 172.21.15.46:0/1829976725 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f943000cd40 con 0x7f9444100730 2024-05-08T23:18:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.408+0000 7f943be00700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 0x7f94280a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.408+0000 7f943be00700 1 -- 172.21.15.46:0/1829976725 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f9430014070 con 0x7f9444100730 2024-05-08T23:18:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.408+0000 7f9443400700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 0x7f94280a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.409+0000 7f9443400700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 0x7f94280a3870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9434006010 tx=0x7f943400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.410+0000 7f945e031b80 1 Processor -- start 2024-05-08T23:18:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 -- start start 2024-05-08T23:18:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 0x55d09bf77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bec8150 con 0x55d09bf77970 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bec5b10 con 0x55d09bf71890 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f945e031b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bec1400 con 0x55d09bf77050 2024-05-08T23:18:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f9443400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 0x55d09bf77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f9442a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf71350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f9443e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.411+0000 7f9443400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 0x55d09bf77430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47312/0 (socket says 172.21.15.46:47312) 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443400700 1 -- 172.21.15.46:0/3005372685 learned_addr learned my addr 172.21.15.46:0/3005372685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443e00700 1 -- 172.21.15.46:0/3005372685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 msgr2=0x55d09bf77430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443e00700 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 0x55d09bf77430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443e00700 1 -- 172.21.15.46:0/3005372685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 msgr2=0x55d09bf71350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443e00700 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf71350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9443e00700 1 -- 172.21.15.46:0/3005372685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d09bd423b0 con 0x55d09bf71890 2024-05-08T23:18:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.412+0000 7f9442a00700 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf71350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.413+0000 7f9443e00700 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f943c0093a0 tx=0x7f943c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6caa2bae7db7355 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.413+0000 7f9438c00700 1 -- 172.21.15.46:0/3005372685 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00e070 con 0x55d09bf71890 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.413+0000 7f9438c00700 1 -- 172.21.15.46:0/3005372685 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943c00f040 con 0x55d09bf71890 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.413+0000 7f9438c00700 1 -- 172.21.15.46:0/3005372685 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c015e60 con 0x55d09bf71890 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 -- 172.21.15.46:0/3005372685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 msgr2=0x55d09bf71cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f943c0093a0 tx=0x7f943c009920 comp rx=0 tx=0).stop 2024-05-08T23:18:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 -- 172.21.15.46:0/3005372685 shutdown_connections 2024-05-08T23:18:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71cd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77050 0x55d09bf77430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 --2- 172.21.15.46:0/3005372685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf71350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 -- 172.21.15.46:0/3005372685 >> 172.21.15.46:0/3005372685 conn(0x55d09becf210 msgr2=0x55d09bf6c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 -- 172.21.15.46:0/3005372685 shutdown_connections 2024-05-08T23:18:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.414+0000 7f945e031b80 1 -- 172.21.15.46:0/3005372685 wait complete. 2024-05-08T23:18:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.415+0000 7f945e031b80 1 -- start start 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.415+0000 7f945e031b80 1 Processor -- start 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 0x55d09bf7d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 0x55d09bf7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf7b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09be5d560 con 0x55d09bf77970 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09be5d200 con 0x55d09bf77050 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bd446a0 con 0x55d09bf71890 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f945e031b80 10 client.0 _ll_register_callbacks cb 0x55d09be66b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f9443400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 0x55d09bf7d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f9442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 0x55d09bf7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f9443400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 0x55d09bf7d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47318/0 (socket says 172.21.15.46:47318) 2024-05-08T23:18:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.416+0000 7f9443400700 1 -- 172.21.15.46:0/3123228203 learned_addr learned my addr 172.21.15.46:0/3123228203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9443e00700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf7b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 -- 172.21.15.46:0/3123228203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 msgr2=0x55d09bf7d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 0x55d09bf7d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 -- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 msgr2=0x55d09bf7b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf7b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 -- 172.21.15.46:0/3123228203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441a5f20 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9443e00700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf7b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f9442a00700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 0x55d09bf7ab90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94300ca640 tx=0x7f943009a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b5791b8d70cca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f942d600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300a2480 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f94300994f0 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.417+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d09be1a270 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.418+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f943007b050 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.418+0000 7f945e031b80 10 client.29471 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.418+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f943c009050 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.418+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d09be1a270 con 0x55d09bf77050 2024-05-08T23:18:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.418+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f94300b2bb0 con 0x55d09bf77050 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 10 client.29471 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 10 client.29471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 20 client.29471 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bf8b380 0x55d09bf8d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 4 client.29471 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d09bf40810 con 0x55d09bf8b380 2024-05-08T23:18:12.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.419+0000 7f9443400700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bf8b380 0x55d09bf8d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.420+0000 7f942d600700 10 client.29471 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.420+0000 7f9443400700 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bf8b380 0x55d09bf8d7a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfc9afc1253fa45a server_cookie=cbfd8aad3543febd in_seq=0 out_seq=0 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.421+0000 7f942d600700 1 -- 172.21.15.46:0/3123228203 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d09bf40810 con 0x55d09bf8b380 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.421+0000 7f942d600700 10 client.29471 handle_command_reply: tid=0 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.421+0000 7f945e031b80 1 client.29471 shutdown 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.421+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 msgr2=0x55d09bf7ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.421+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 0x55d09bf7ab90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94300ca640 tx=0x7f943009a780 comp rx=0 tx=0).stop 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 shutdown_connections 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bf8b380 0x55d09bf8d7a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf7b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf71890 0x55d09bf7d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f942d600700 0 client.29471 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 --2- 172.21.15.46:0/3123228203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf77050 0x55d09bf7ab90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 >> 172.21.15.46:0/3123228203 conn(0x55d09becf210 msgr2=0x55d09bf6c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 shutdown_connections 2024-05-08T23:18:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.422+0000 7f945e031b80 1 -- 172.21.15.46:0/3123228203 wait complete. 2024-05-08T23:18:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.423+0000 7f945e031b80 20 client.29471 trim_cache size 0 max 16384 2024-05-08T23:18:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.441+0000 7f945e031b80 1 Processor -- start 2024-05-08T23:18:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.441+0000 7f945e031b80 1 -- start start 2024-05-08T23:18:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77970 0x55d09bf71c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf75370 0x55d09bf86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf77430 con 0x55d09bf75370 2024-05-08T23:18:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf790e0 con 0x55d09bf71890 2024-05-08T23:18:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f945e031b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf64040 con 0x55d09bf77970 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f9443400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f9443e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf75370 0x55d09bf86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f9443400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36274/0 (socket says 172.21.15.46:36274) 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.442+0000 7f9443400700 1 -- 172.21.15.46:0/2753825256 learned_addr learned my addr 172.21.15.46:0/2753825256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 -- 172.21.15.46:0/2753825256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77970 msgr2=0x55d09bf71c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77970 0x55d09bf71c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 -- 172.21.15.46:0/2753825256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf75370 msgr2=0x55d09bf86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf75370 0x55d09bf86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 -- 172.21.15.46:0/2753825256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d09bceeb60 con 0x55d09bf71890 2024-05-08T23:18:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9443400700 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94340098e0 tx=0x7f9434023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dee4ff359eb38f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.443+0000 7f9439600700 1 -- 172.21.15.46:0/2753825256 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434005380 con 0x55d09bf71890 2024-05-08T23:18:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f9439600700 1 -- 172.21.15.46:0/2753825256 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943401e040 con 0x55d09bf71890 2024-05-08T23:18:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f9439600700 1 -- 172.21.15.46:0/2753825256 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434036760 con 0x55d09bf71890 2024-05-08T23:18:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 -- 172.21.15.46:0/2753825256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 msgr2=0x55d09bf71350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94340098e0 tx=0x7f9434023000 comp rx=0 tx=0).stop 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 -- 172.21.15.46:0/2753825256 shutdown_connections 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf75370 0x55d09bf86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf71350 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 --2- 172.21.15.46:0/2753825256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf77970 0x55d09bf71c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.444+0000 7f945e031b80 1 -- 172.21.15.46:0/2753825256 >> 172.21.15.46:0/2753825256 conn(0x55d09becf210 msgr2=0x55d09bf74c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.445+0000 7f945e031b80 1 -- 172.21.15.46:0/2753825256 shutdown_connections 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.445+0000 7f945e031b80 1 -- 172.21.15.46:0/2753825256 wait complete. 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.445+0000 7f945e031b80 1 -- start start 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.445+0000 7f945e031b80 1 Processor -- start 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf72a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf73000 con 0x55d09bf77970 2024-05-08T23:18:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf73140 con 0x55d09bf71890 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09bf73300 con 0x55d09bf75370 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f945e031b80 10 client.0 _ll_register_callbacks cb 0x55d09be66b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9442a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9443400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9443400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36276/0 (socket says 172.21.15.46:36276) 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9442a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47334/0 (socket says 172.21.15.46:47334) 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9443400700 1 -- 172.21.15.46:0/3782655005 learned_addr learned my addr 172.21.15.46:0/3782655005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9443e00700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf72a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.446+0000 7f9442a00700 1 -- 172.21.15.46:0/3782655005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 msgr2=0x55d09bf82330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9442a00700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9442a00700 1 -- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 msgr2=0x55d09bf72a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9442a00700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf72a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9442a00700 1 -- 172.21.15.46:0/3782655005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944410d320 con 0x55d09bf75370 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9443400700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f9442a00700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x55d09bf7bc20 tx=0x7f9430099a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3429138d18e1868f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f942d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943007b070 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f94300a2480 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f943c00f040 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.447+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94300b80d0 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 10 client.29566 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d09be87560 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943c00f040 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f943007e060 con 0x55d09bf75370 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 10 client.29566 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 10 client.29566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 20 client.29566 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bfe2650 0x55d09bfe4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 4 client.29566 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.448+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d09beb9ec0 con 0x55d09bfe2650 2024-05-08T23:18:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.449+0000 7f9443400700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bfe2650 0x55d09bfe4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.450+0000 7f942d600700 10 client.29566 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.450+0000 7f9443400700 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bfe2650 0x55d09bfe4a70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d5e89f85def71b4 server_cookie=c27b08d52d18e48c in_seq=0 out_seq=0 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.450+0000 7f942d600700 1 -- 172.21.15.46:0/3782655005 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d09beb9ec0 con 0x55d09bfe2650 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.450+0000 7f942d600700 10 client.29566 handle_command_reply: tid=0 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.450+0000 7f945e031b80 1 client.29566 shutdown 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 msgr2=0x55d09bf82870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x55d09bf7bc20 tx=0x7f9430099a80 comp rx=0 tx=0).stop 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 shutdown_connections 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55d09bfe2650 0x55d09bfe4a70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d09bf77970 0x55d09bf72a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d09bf71890 0x55d09bf82330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 --2- 172.21.15.46:0/3782655005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d09bf75370 0x55d09bf82870 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 >> 172.21.15.46:0/3782655005 conn(0x55d09becf210 msgr2=0x55d09bf75c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f942d600700 0 client.29566 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 shutdown_connections 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.451+0000 7f945e031b80 1 -- 172.21.15.46:0/3782655005 wait complete. 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.452+0000 7f945e031b80 20 client.29566 trim_cache size 0 max 16384 2024-05-08T23:18:12.454 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 msgr2=0x7f94280a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 0x7f94280a3870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9434006010 tx=0x7f943400b260 comp rx=0 tx=0).stop 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 msgr2=0x7f9444101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9a0 tx=0x7f943000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 shutdown_connections 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94280a13b0 0x7f94280a3870 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9444104b40 0x7f94441022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94440ffd90 0x7f9444101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 --2- 172.21.15.46:0/1829976725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9444100730 0x7f9444101d70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.453+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 >> 172.21.15.46:0/1829976725 conn(0x7f9444075710 msgr2=0x7f94440683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.454+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 shutdown_connections 2024-05-08T23:18:12.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.454+0000 7f9449000700 1 -- 172.21.15.46:0/1829976725 wait complete. 2024-05-08T23:18:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:11 smithi086 ceph-mon[78610]: pgmap v448: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.8 MiB/s wr, 622 op/s 2024-05-08T23:18:12.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:11 smithi086 ceph-mon[78610]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:18:12.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:18:12.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.783+0000 7fc2f8a00700 1 -- 172.21.15.46:0/4277579400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f4107ea0 msgr2=0x7fc2f41142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.783+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/4277579400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41142d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009f90 tx=0x7fc2e400b330 comp rx=0 tx=0).stop 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 -- 172.21.15.46:0/4277579400 shutdown_connections 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/4277579400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41142d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/4277579400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/4277579400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f41074e0 0x7fc2f4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 -- 172.21.15.46:0/4277579400 >> 172.21.15.46:0/4277579400 conn(0x7fc2f4076de0 msgr2=0x7fc2f4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.784+0000 7fc2f8a00700 1 -- 172.21.15.46:0/4277579400 shutdown_connections 2024-05-08T23:18:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.785+0000 7fc2f8a00700 1 -- 172.21.15.46:0/4277579400 wait complete. 2024-05-08T23:18:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.787+0000 7fc2f8a00700 1 Processor -- start 2024-05-08T23:18:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.787+0000 7fc2f8a00700 1 -- start start 2024-05-08T23:18:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f41074e0 0x7fc2f41a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419de60 con 0x7fc2f41074e0 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419dfd0 con 0x7fc2f410d510 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.788+0000 7fc2f8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f419e190 con 0x7fc2f4107ea0 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f41074e0 0x7fc2f41a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36290/0 (socket says 172.21.15.46:36290) 2024-05-08T23:18:12.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f2800700 1 -- 172.21.15.46:0/3665742440 learned_addr learned my addr 172.21.15.46:0/3665742440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f1400700 1 -- 172.21.15.46:0/3665742440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 msgr2=0x7fc2f419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.789+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.790+0000 7fc2f1400700 1 -- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f41074e0 msgr2=0x7fc2f41a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.790+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f41074e0 0x7fc2f41a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.790+0000 7fc2f1400700 1 -- 172.21.15.46:0/3665742440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4009c40 con 0x7fc2f4107ea0 2024-05-08T23:18:12.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.792+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41a3c00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00b450 tx=0x7fc2dc00b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.795+0000 7fc2eaa00700 1 -- 172.21.15.46:0/3665742440 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc011830 con 0x7fc2f4107ea0 2024-05-08T23:18:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.796+0000 7fc2eaa00700 1 -- 172.21.15.46:0/3665742440 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc011e50 con 0x7fc2f4107ea0 2024-05-08T23:18:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.796+0000 7fc2eaa00700 1 -- 172.21.15.46:0/3665742440 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc010de0 con 0x7fc2f4107ea0 2024-05-08T23:18:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.796+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f419e470 con 0x7fc2f4107ea0 2024-05-08T23:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.796+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f41aa6b0 con 0x7fc2f4107ea0 2024-05-08T23:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.803+0000 7fc30c45db80 1 Processor -- start 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 -- start start 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e32751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e326f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e31c6150 con 0x5576e326f890 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e31c3b10 con 0x5576e3274e00 2024-05-08T23:18:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.804+0000 7fc30c45db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e31bf400 con 0x5576e3275720 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.805+0000 7fc2f1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.805+0000 7fc2f1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35710/0 (socket says 172.21.15.46:35710) 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.805+0000 7fc2f1400700 1 -- 172.21.15.46:0/2218604692 learned_addr learned my addr 172.21.15.46:0/2218604692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.805+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e326f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.805+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e32751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.806+0000 7fc2eaa00700 1 -- 172.21.15.46:0/3665742440 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc2dc010480 con 0x7fc2f4107ea0 2024-05-08T23:18:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.808+0000 7fc2f2800700 1 -- 172.21.15.46:0/2218604692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 msgr2=0x5576e326f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.808+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e326f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.808+0000 7fc2f2800700 1 -- 172.21.15.46:0/2218604692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 msgr2=0x5576e326fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.808+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.808+0000 7fc2f2800700 1 -- 172.21.15.46:0/2218604692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e30403b0 con 0x5576e3274e00 2024-05-08T23:18:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.812+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e32751e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b770 tx=0x7fc2e4007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4988633b8c3573b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.812+0000 7fc2e3000700 1 -- 172.21.15.46:0/2218604692 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400e040 con 0x5576e3274e00 2024-05-08T23:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.812+0000 7fc2e3000700 1 -- 172.21.15.46:0/2218604692 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e400bca0 con 0x5576e3274e00 2024-05-08T23:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.812+0000 7fc2e3000700 1 -- 172.21.15.46:0/2218604692 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e401c440 con 0x5576e3274e00 2024-05-08T23:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 -- 172.21.15.46:0/2218604692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 msgr2=0x5576e32751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e32751e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b770 tx=0x7fc2e4007c30 comp rx=0 tx=0).stop 2024-05-08T23:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 -- 172.21.15.46:0/2218604692 shutdown_connections 2024-05-08T23:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e32751e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2218604692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e326f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 -- 172.21.15.46:0/2218604692 >> 172.21.15.46:0/2218604692 conn(0x5576e31cd210 msgr2=0x5576e3269df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.814+0000 7fc30c45db80 1 -- 172.21.15.46:0/2218604692 shutdown_connections 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.815+0000 7fc30c45db80 1 -- 172.21.15.46:0/2218604692 wait complete. 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.815+0000 7fc2eaa00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 0x7fc2d40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.815+0000 7fc2eaa00700 1 -- 172.21.15.46:0/3665742440 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc2dc0cca10 con 0x7fc2f4107ea0 2024-05-08T23:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.815+0000 7fc30c45db80 1 -- start start 2024-05-08T23:18:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.817+0000 7fc30c45db80 1 Processor -- start 2024-05-08T23:18:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.818+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 0x7fc2d40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.819+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.819+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e3279200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.819+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e3279740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 0x7fc2d40a3830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5576e326bbd0 tx=0x7fc2ec009450 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc2f1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e3279200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc30c45db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e315b560 con 0x5576e3274e00 2024-05-08T23:18:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc30c45db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e315b200 con 0x5576e3275720 2024-05-08T23:18:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc30c45db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e30426a0 con 0x5576e326f890 2024-05-08T23:18:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc30c45db80 10 client.0 _ll_register_callbacks cb 0x5576e3164b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.820+0000 7fc2f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47358/0 (socket says 172.21.15.46:47358) 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 -- 172.21.15.46:0/3411150059 learned_addr learned my addr 172.21.15.46:0/3411150059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 -- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 msgr2=0x5576e3279740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e3279740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 -- 172.21.15.46:0/3411150059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 msgr2=0x5576e3279200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e3279200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.821+0000 7fc2f1e00700 1 -- 172.21.15.46:0/3411150059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f4114bc0 con 0x5576e326f890 2024-05-08T23:18:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.824+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00a010 tx=0x7fc2ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c4dc8effc8a80e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.824+0000 7fc2db400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.825+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01f420 con 0x5576e326f890 2024-05-08T23:18:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.825+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc2ec018070 con 0x5576e326f890 2024-05-08T23:18:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.825+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576e3118270 con 0x5576e326f890 2024-05-08T23:18:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.825+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2ec01f580 con 0x5576e326f890 2024-05-08T23:18:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.826+0000 7fc30c45db80 10 client.29575 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.826+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5576e2fecb60 con 0x5576e326f890 2024-05-08T23:18:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.828+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576e3118270 con 0x5576e326f890 2024-05-08T23:18:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.828+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc2ec025020 con 0x5576e326f890 2024-05-08T23:18:12.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 10 client.29575 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 10 client.29575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 20 client.29575 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e329a260 0x5576e329c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 4 client.29575 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576e323e810 con 0x5576e329a260 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.829+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e329a260 0x5576e329c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.830+0000 7fc2f1400700 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e329a260 0x5576e329c680 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c82524b2dc92db1 server_cookie=8e049fc76b1ba363 in_seq=0 out_seq=0 2024-05-08T23:18:12.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.830+0000 7fc2db400700 10 client.29575 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.831+0000 7fc2db400700 1 -- 172.21.15.46:0/3411150059 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576e323e810 con 0x5576e329a260 2024-05-08T23:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.831+0000 7fc2db400700 10 client.29575 handle_command_reply: tid=0 2024-05-08T23:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.832+0000 7fc30c45db80 1 client.29575 shutdown 2024-05-08T23:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.832+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 msgr2=0x5576e3278cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.832+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00a010 tx=0x7fc2ec004750 comp rx=0 tx=0).stop 2024-05-08T23:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 shutdown_connections 2024-05-08T23:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e3279740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e3278cc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e329a260 0x5576e329c680 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 --2- 172.21.15.46:0/3411150059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e3279200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 >> 172.21.15.46:0/3411150059 conn(0x5576e31cd210 msgr2=0x5576e32695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc2db400700 0 client.29575 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 shutdown_connections 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 1 -- 172.21.15.46:0/3411150059 wait complete. 2024-05-08T23:18:12.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.833+0000 7fc30c45db80 20 client.29575 trim_cache size 0 max 16384 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.861+0000 7fc30c45db80 1 Processor -- start 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 -- start start 2024-05-08T23:18:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e3272de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e327b6b0 con 0x5576e326f890 2024-05-08T23:18:12.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e326be50 con 0x5576e3274e00 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc30c45db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3264800 con 0x5576e3275720 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.862+0000 7fc2f1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36320/0 (socket says 172.21.15.46:36320) 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 -- 172.21.15.46:0/1782944827 learned_addr learned my addr 172.21.15.46:0/1782944827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 -- 172.21.15.46:0/1782944827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 msgr2=0x5576e3272de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e3272de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 -- 172.21.15.46:0/1782944827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 msgr2=0x5576e326f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.863+0000 7fc2f1400700 1 -- 172.21.15.46:0/1782944827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e3185560 con 0x5576e3274e00 2024-05-08T23:18:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.864+0000 7fc2f1400700 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc099470 tx=0x7fc2dc0b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f0304a1e2576dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.865+0000 7fc2e3a00700 1 -- 172.21.15.46:0/1782944827 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0990d0 con 0x5576e3274e00 2024-05-08T23:18:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.865+0000 7fc2e3a00700 1 -- 172.21.15.46:0/1782944827 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc009280 con 0x5576e3274e00 2024-05-08T23:18:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.865+0000 7fc2e3a00700 1 -- 172.21.15.46:0/1782944827 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0a1c80 con 0x5576e3274e00 2024-05-08T23:18:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 -- 172.21.15.46:0/1782944827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 msgr2=0x5576e3269d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc099470 tx=0x7fc2dc0b9040 comp rx=0 tx=0).stop 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 -- 172.21.15.46:0/1782944827 shutdown_connections 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e3275720 0x5576e3272de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e326f890 0x5576e326f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 --2- 172.21.15.46:0/1782944827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3274e00 0x5576e3269d20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.866+0000 7fc30c45db80 1 -- 172.21.15.46:0/1782944827 >> 172.21.15.46:0/1782944827 conn(0x5576e31cd210 msgr2=0x5576e3266210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.867+0000 7fc30c45db80 1 -- 172.21.15.46:0/1782944827 shutdown_connections 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.867+0000 7fc30c45db80 1 -- 172.21.15.46:0/1782944827 wait complete. 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.867+0000 7fc30c45db80 1 -- start start 2024-05-08T23:18:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.868+0000 7fc30c45db80 1 Processor -- start 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.868+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e328f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.868+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e328f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc30c45db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc30c45db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e32919d0 con 0x5576e3274e00 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc30c45db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e32ef740 con 0x5576e3275720 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc30c45db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e32ef880 con 0x5576e326f890 2024-05-08T23:18:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc30c45db80 10 client.0 _ll_register_callbacks cb 0x5576e3164b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc2f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e328f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc2f2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc2f2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36334/0 (socket says 172.21.15.46:36334) 2024-05-08T23:18:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc2f2800700 1 -- 172.21.15.46:0/2516991544 learned_addr learned my addr 172.21.15.46:0/2516991544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:12.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.869+0000 7fc2f2800700 1 -- 172.21.15.46:0/2516991544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 msgr2=0x5576e328f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f1400700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e328f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e328f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f2800700 1 -- 172.21.15.46:0/2516991544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 msgr2=0x5576e328f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e328f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f2800700 1 -- 172.21.15.46:0/2516991544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f41aa6b0 con 0x5576e3275720 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e328f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f1400700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e328f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2f2800700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4007640 tx=0x7fc2e4007720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61be56eea782d8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2db400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400e070 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc2e4007970 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.870+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2e400bca0 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e4003e70 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 10 client.29492 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5576e3044cd0 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e400bca0 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc2e4019030 con 0x5576e3275720 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 10 client.29492 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 10 client.29492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 20 client.29492 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e32f0c00 0x5576e32f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 4 client.29492 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.871+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5576e31b7ec0 con 0x5576e32f0c00 2024-05-08T23:18:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.872+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e32f0c00 0x5576e32f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.873+0000 7fc2db400700 10 client.29492 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.873+0000 7fc2f1e00700 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e32f0c00 0x5576e32f2ff0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd1dd9c5914e9cb server_cookie=42b4194d71030246 in_seq=0 out_seq=0 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.873+0000 7fc2db400700 1 -- 172.21.15.46:0/2516991544 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5576e31b7ec0 con 0x5576e32f0c00 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.873+0000 7fc2db400700 10 client.29492 handle_command_reply: tid=0 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.873+0000 7fc30c45db80 1 client.29492 shutdown 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 msgr2=0x5576e328fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4007640 tx=0x7fc2e4007720 comp rx=0 tx=0).stop 2024-05-08T23:18:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 shutdown_connections 2024-05-08T23:18:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5576e3275720 0x5576e328fb40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576e326f890 0x5576e328f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5576e32f0c00 0x5576e32f2ff0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 --2- 172.21.15.46:0/2516991544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3274e00 0x5576e328f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 >> 172.21.15.46:0/2516991544 conn(0x5576e31cd210 msgr2=0x5576e3272de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.874+0000 7fc2db400700 0 client.29492 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.875+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 shutdown_connections 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.875+0000 7fc30c45db80 1 -- 172.21.15.46:0/2516991544 wait complete. 2024-05-08T23:18:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.875+0000 7fc30c45db80 20 client.29492 trim_cache size 0 max 16384 2024-05-08T23:18:12.877 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 msgr2=0x7fc2d40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 0x7fc2d40a3830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5576e326bbd0 tx=0x7fc2ec009450 comp rx=0 tx=0).stop 2024-05-08T23:18:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 msgr2=0x7fc2f41a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41a3c00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00b450 tx=0x7fc2dc00b760 comp rx=0 tx=0).stop 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 shutdown_connections 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f410d510 0x7fc2f419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2d40a1370 0x7fc2d40a3830 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5576e326bbd0 tx=0x7fc2ec009450 comp rx=0 tx=0).stop 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2f41074e0 0x7fc2f41a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 --2- 172.21.15.46:0/3665742440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f4107ea0 0x7fc2f41a3c00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.877+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 >> 172.21.15.46:0/3665742440 conn(0x7fc2f4076de0 msgr2=0x7fc2f4078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.878+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 shutdown_connections 2024-05-08T23:18:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:12.878+0000 7fc2f8a00700 1 -- 172.21.15.46:0/3665742440 wait complete. 2024-05-08T23:18:13.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14439 28.32 MB/sec warmup 34 sec latency 109.260 ms 2024-05-08T23:18:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:12 smithi053 ceph-mon[77587]: pgmap v450: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.4 MiB/s wr, 565 op/s 2024-05-08T23:18:13.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:12 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:12 smithi046 ceph-mon[36898]: pgmap v450: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.4 MiB/s wr, 565 op/s 2024-05-08T23:18:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:12 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:12 smithi086 ceph-mon[78610]: pgmap v450: 129 pgs: 129 active+clean; 872 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.4 MiB/s wr, 565 op/s 2024-05-08T23:18:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:12 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:13.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:18:13.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:18:14.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.170+0000 7f66db200700 1 -- 172.21.15.46:0/3175405365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d4103c30 msgr2=0x7f66d4107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.170+0000 7f66db200700 1 --2- 172.21.15.46:0/3175405365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d4103c30 0x7f66d4107c70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009990 tx=0x7f66c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 -- 172.21.15.46:0/3175405365 shutdown_connections 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 --2- 172.21.15.46:0/3175405365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d41083a0 0x7f66d410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 --2- 172.21.15.46:0/3175405365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 --2- 172.21.15.46:0/3175405365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d4103c30 0x7f66d4107c70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 -- 172.21.15.46:0/3175405365 >> 172.21.15.46:0/3175405365 conn(0x7f66d40fea80 msgr2=0x7f66d4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 -- 172.21.15.46:0/3175405365 shutdown_connections 2024-05-08T23:18:14.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.171+0000 7f66db200700 1 -- 172.21.15.46:0/3175405365 wait complete. 2024-05-08T23:18:14.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.172+0000 7f66db200700 1 Processor -- start 2024-05-08T23:18:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.173+0000 7f66db200700 1 -- start start 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41083a0 0x7f66d4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d419df40 0x7f66d419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419e9c0 con 0x7f66d419df40 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419eb30 con 0x7f66d4103280 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66db200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419ecf0 con 0x7f66d41083a0 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36360/0 (socket says 172.21.15.46:36360) 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 -- 172.21.15.46:0/1404123862 learned_addr learned my addr 172.21.15.46:0/1404123862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3400700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41083a0 0x7f66d4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 -- 172.21.15.46:0/1404123862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41083a0 msgr2=0x7f66d4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41083a0 0x7f66d4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 -- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d419df40 msgr2=0x7f66d419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d419df40 0x7f66d419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.174+0000 7f66d3e00700 1 -- 172.21.15.46:0/1404123862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00095f0 con 0x7f66d4103280 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66d3e00700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66c400e9c0 tx=0x7f66c400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66d0c00700 1 -- 172.21.15.46:0/1404123862 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400cbd0 con 0x7f66d4103280 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66d0c00700 1 -- 172.21.15.46:0/1404123862 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c400cd30 con 0x7f66d4103280 2024-05-08T23:18:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66d0c00700 1 -- 172.21.15.46:0/1404123862 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4005690 con 0x7f66d4103280 2024-05-08T23:18:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d41a6250 con 0x7f66d4103280 2024-05-08T23:18:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.175+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d41a66e0 con 0x7f66d4103280 2024-05-08T23:18:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.179+0000 7f66d0c00700 1 -- 172.21.15.46:0/1404123862 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f66c4004750 con 0x7f66d4103280 2024-05-08T23:18:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.181+0000 7f66d0c00700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 0x7f66b80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.181+0000 7f66d0c00700 1 -- 172.21.15.46:0/1404123862 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66c4014070 con 0x7f66d4103280 2024-05-08T23:18:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.182+0000 7f66d3400700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 0x7f66b80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.184+0000 7f66d3400700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 0x7f66b80a3850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b2e0 tx=0x7f66c0005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.186+0000 7f66eed51b80 1 Processor -- start 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.186+0000 7f66eed51b80 1 -- start start 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b2043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b204910 0x55bc5b204d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b15b1d0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b158b90 con 0x55bc5b204910 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66eed51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b154480 con 0x55bc5b209e80 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47412/0 (socket says 172.21.15.46:47412) 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66d3e00700 1 -- 172.21.15.46:0/3175895016 learned_addr learned my addr 172.21.15.46:0/3175895016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.187+0000 7f66d3400700 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b2043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d9200700 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b204910 0x55bc5b204d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 -- 172.21.15.46:0/3175895016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b204910 msgr2=0x55bc5b204d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b204910 0x55bc5b204d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 -- 172.21.15.46:0/3175895016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 msgr2=0x55bc5b2043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b2043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 -- 172.21.15.46:0/3175895016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5afd5430 con 0x55bc5b209e80 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f66c4096800 tx=0x7f66c4004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33579dd4ad195d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.188+0000 7f66c9600700 1 -- 172.21.15.46:0/3175895016 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40cd070 con 0x55bc5b209e80 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.189+0000 7f66c9600700 1 -- 172.21.15.46:0/3175895016 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c409a300 con 0x55bc5b209e80 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.189+0000 7f66c9600700 1 -- 172.21.15.46:0/3175895016 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40987c0 con 0x55bc5b209e80 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 -- 172.21.15.46:0/3175895016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 msgr2=0x55bc5b20a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f66c4096800 tx=0x7f66c4004f90 comp rx=0 tx=0).stop 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 -- 172.21.15.46:0/3175895016 shutdown_connections 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b204910 0x55bc5b204d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b209e80 0x55bc5b20a260 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3175895016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b2043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 -- 172.21.15.46:0/3175895016 >> 172.21.15.46:0/3175895016 conn(0x55bc5b162290 msgr2=0x55bc5b1fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 -- 172.21.15.46:0/3175895016 shutdown_connections 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.190+0000 7f66eed51b80 1 -- 172.21.15.46:0/3175895016 wait complete. 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.191+0000 7f66eed51b80 1 -- start start 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.191+0000 7f66eed51b80 1 Processor -- start 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b20db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b20e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b0f05e0 con 0x55bc5b204910 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b0f0280 con 0x55bc5b209e80 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afd7720 con 0x55bc5b20a7a0 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66eed51b80 10 client.0 _ll_register_callbacks cb 0x55bc5b0f9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66d9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b20db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66d9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47422/0 (socket says 172.21.15.46:47422) 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.192+0000 7f66d9200700 1 -- 172.21.15.46:0/986876191 learned_addr learned my addr 172.21.15.46:0/986876191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d3400700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b20e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 -- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 msgr2=0x55bc5b20e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b20e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 -- 172.21.15.46:0/986876191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 msgr2=0x55bc5b20db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b20db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 -- 172.21.15.46:0/986876191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d41a66e0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d3e00700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b20db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66d9200700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009710 tx=0x7f66cc011e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7949cda3d3c4239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.193+0000 7f66be000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc010040 con 0x55bc5b20a7a0 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66cc003c00 con 0x55bc5b20a7a0 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc5b0ad2f0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66cc003de0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66eed51b80 10 client.29587 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.194+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc5af81be0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc5b0ad2f0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66cc004620 con 0x55bc5b20a7a0 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 10 client.29587 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 10 client.29587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 20 client.29587 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b215220 0x55bc5b217640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 4 client.29587 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5b1d3890 con 0x55bc5b215220 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.195+0000 7f66d3e00700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b215220 0x55bc5b217640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.196+0000 7f66d3e00700 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b215220 0x55bc5b217640 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2172b2b15c98dfc5 server_cookie=1534e922349862e1 in_seq=0 out_seq=0 2024-05-08T23:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.196+0000 7f66be000700 10 client.29587 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.197+0000 7f66be000700 1 -- 172.21.15.46:0/986876191 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc5b1d3890 con 0x55bc5b215220 2024-05-08T23:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.197+0000 7f66be000700 10 client.29587 handle_command_reply: tid=0 2024-05-08T23:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.197+0000 7f66eed51b80 1 client.29587 shutdown 2024-05-08T23:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.197+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 msgr2=0x55bc5b20e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.197+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009710 tx=0x7f66cc011e80 comp rx=0 tx=0).stop 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 shutdown_connections 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b20e590 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b215220 0x55bc5b217640 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b20db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 --2- 172.21.15.46:0/986876191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b20e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 >> 172.21.15.46:0/986876191 conn(0x55bc5b162290 msgr2=0x55bc5b1fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66be000700 0 client.29587 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 shutdown_connections 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 1 -- 172.21.15.46:0/986876191 wait complete. 2024-05-08T23:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.198+0000 7f66eed51b80 20 client.29587 trim_cache size 0 max 16384 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 Processor -- start 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 -- start start 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b204910 0x55bc5b2043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b209310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b211d70 con 0x55bc5b20a7a0 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b1f9880 con 0x55bc5b209e80 2024-05-08T23:18:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.222+0000 7f66eed51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b1ffb30 con 0x55bc5b204910 2024-05-08T23:18:14.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b209310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b204910 0x55bc5b2043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35784/0 (socket says 172.21.15.46:35784) 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 -- 172.21.15.46:0/3195827046 learned_addr learned my addr 172.21.15.46:0/3195827046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 -- 172.21.15.46:0/3195827046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b204910 msgr2=0x55bc5b2043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b204910 0x55bc5b2043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 -- 172.21.15.46:0/3195827046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 msgr2=0x55bc5b209310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b209310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.223+0000 7f66d9200700 1 -- 172.21.15.46:0/3195827046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5b204fc0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.224+0000 7f66d9200700 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f66cc004980 tx=0x7f66cc00f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ef7517ac2638c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.224+0000 7f66ca000700 1 -- 172.21.15.46:0/3195827046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004bb0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.224+0000 7f66ca000700 1 -- 172.21.15.46:0/3195827046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66cc004d10 con 0x55bc5b20a7a0 2024-05-08T23:18:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.224+0000 7f66ca000700 1 -- 172.21.15.46:0/3195827046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0168d0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 -- 172.21.15.46:0/3195827046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 msgr2=0x55bc5b1feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f66cc004980 tx=0x7f66cc00f410 comp rx=0 tx=0).stop 2024-05-08T23:18:14.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 -- 172.21.15.46:0/3195827046 shutdown_connections 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b20a7a0 0x55bc5b1feda0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b204910 0x55bc5b2043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3195827046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b209310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 -- 172.21.15.46:0/3195827046 >> 172.21.15.46:0/3195827046 conn(0x55bc5b162290 msgr2=0x55bc5b208c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 -- 172.21.15.46:0/3195827046 shutdown_connections 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.225+0000 7f66eed51b80 1 -- 172.21.15.46:0/3195827046 wait complete. 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.226+0000 7f66eed51b80 1 -- start start 2024-05-08T23:18:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.226+0000 7f66eed51b80 1 Processor -- start 2024-05-08T23:18:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b2067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b206d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b285250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b207620 con 0x55bc5b204910 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b285790 con 0x55bc5b209e80 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5b2858d0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66eed51b80 10 client.0 _ll_register_callbacks cb 0x55bc5b0f9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b2067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b206d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b285250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b206d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36414/0 (socket says 172.21.15.46:36414) 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d3400700 1 -- 172.21.15.46:0/3897989446 learned_addr learned my addr 172.21.15.46:0/3897989446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 -- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 msgr2=0x55bc5b206d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b206d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 -- 172.21.15.46:0/3897989446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 msgr2=0x55bc5b2067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b2067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d9200700 1 -- 172.21.15.46:0/3897989446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5af81be0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.227+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b2067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66d9200700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b285250 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0164c0 tx=0x7f66cc004e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c05bd68099fda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66be000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004bb0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66c409a300 con 0x55bc5b20a7a0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f66cc0168d0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66cc016a90 con 0x55bc5b20a7a0 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66eed51b80 10 client.29593 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.228+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc5b11a5e0 con 0x55bc5b20a7a0 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.229+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c409a300 con 0x55bc5b20a7a0 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66cc00f870 con 0x55bc5b20a7a0 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 10 client.29593 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 10 client.29593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 20 client.29593 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b286bd0 0x55bc5b288fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 4 client.29593 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bc5b14cf40 con 0x55bc5b286bd0 2024-05-08T23:18:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.230+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b286bd0 0x55bc5b288fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.231+0000 7f66d3e00700 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b286bd0 0x55bc5b288fc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05399a73fcee992 server_cookie=1e883fcc8c08b93b in_seq=0 out_seq=0 2024-05-08T23:18:14.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.231+0000 7f66be000700 10 client.29593 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66be000700 1 -- 172.21.15.46:0/3897989446 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bc5b14cf40 con 0x55bc5b286bd0 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66be000700 10 client.29593 handle_command_reply: tid=0 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 client.29593 shutdown 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 msgr2=0x55bc5b285250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b285250 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0164c0 tx=0x7f66cc004e80 comp rx=0 tx=0).stop 2024-05-08T23:18:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 shutdown_connections 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5b20a7a0 0x55bc5b285250 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bc5b286bd0 0x55bc5b288fc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc5b204910 0x55bc5b2067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 --2- 172.21.15.46:0/3897989446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc5b209e80 0x55bc5b206d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 >> 172.21.15.46:0/3897989446 conn(0x55bc5b162290 msgr2=0x55bc5b209490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.232+0000 7f66be000700 0 client.29593 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.233+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 shutdown_connections 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.233+0000 7f66eed51b80 1 -- 172.21.15.46:0/3897989446 wait complete. 2024-05-08T23:18:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.233+0000 7f66eed51b80 20 client.29593 trim_cache size 0 max 16384 2024-05-08T23:18:14.236 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:18:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "85fd783b-437b-48c9-8a31-8ecf0803cd84", 2024-05-08T23:18:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:18:14.236 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 msgr2=0x7f66b80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 0x7f66b80a3850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b2e0 tx=0x7f66c0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 msgr2=0x7f66d4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66c400e9c0 tx=0x7f66c400ed80 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 shutdown_connections 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f66b80a1390 0x7f66b80a3850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d419df40 0x7f66d419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d4103280 0x7f66d4199660 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 --2- 172.21.15.46:0/1404123862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41083a0 0x7f66d4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 >> 172.21.15.46:0/1404123862 conn(0x7f66d40fea80 msgr2=0x7f66d4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 shutdown_connections 2024-05-08T23:18:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.234+0000 7f66db200700 1 -- 172.21.15.46:0/1404123862 wait complete. 2024-05-08T23:18:14.247 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14736 27.95 MB/sec warmup 35 sec latency 181.116 ms 2024-05-08T23:18:14.256 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:13 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:14.256 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:13 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:14.256 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:13 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:18:14.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:18:14.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:13 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:13 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:13 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:18:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:13 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:14.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:13 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:14.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:13 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:18:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 -- 172.21.15.46:0/570841760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4106d30 msgr2=0x7efee4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 --2- 172.21.15.46:0/570841760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4106d30 0x7efee4107110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efed40098f0 tx=0x7efed4009c00 comp rx=0 tx=0).stop 2024-05-08T23:18:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 -- 172.21.15.46:0/570841760 shutdown_connections 2024-05-08T23:18:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 --2- 172.21.15.46:0/570841760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068b20 0x7efee410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 --2- 172.21.15.46:0/570841760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4106d30 0x7efee4107110 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 --2- 172.21.15.46:0/570841760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4068160 0x7efee40685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 -- 172.21.15.46:0/570841760 >> 172.21.15.46:0/570841760 conn(0x7efee4075710 msgr2=0x7efee4075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 -- 172.21.15.46:0/570841760 shutdown_connections 2024-05-08T23:18:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.497+0000 7efee3200700 1 -- 172.21.15.46:0/570841760 wait complete. 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 Processor -- start 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 -- start start 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 0x7efee4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 0x7efee419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419e130 con 0x7efee4068b20 2024-05-08T23:18:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419e2a0 con 0x7efee4068160 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.499+0000 7efee3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419e460 con 0x7efee4106d30 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 0x7efee419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36438/0 (socket says 172.21.15.46:36438) 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1e00700 1 -- 172.21.15.46:0/3073320996 learned_addr learned my addr 172.21.15.46:0/3073320996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1400700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 0x7efee4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 -- 172.21.15.46:0/3073320996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 msgr2=0x7efee41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 -- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 msgr2=0x7efee4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 0x7efee4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 -- 172.21.15.46:0/3073320996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00095f0 con 0x7efee4106d30 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1e00700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee1400700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 0x7efee4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.500+0000 7efee2800700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 0x7efee419daf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efedc00efc0 tx=0x7efedc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.501+0000 7efedaa00700 1 -- 172.21.15.46:0/3073320996 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc018070 con 0x7efee4106d30 2024-05-08T23:18:14.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.501+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed4009520 con 0x7efee4106d30 2024-05-08T23:18:14.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.501+0000 7efedaa00700 1 -- 172.21.15.46:0/3073320996 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efedc00f040 con 0x7efee4106d30 2024-05-08T23:18:14.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.501+0000 7efedaa00700 1 -- 172.21.15.46:0/3073320996 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00cdf0 con 0x7efee4106d30 2024-05-08T23:18:14.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.502+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee41a5dc0 con 0x7efee4106d30 2024-05-08T23:18:14.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.505+0000 7efedaa00700 1 -- 172.21.15.46:0/3073320996 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efedc003a00 con 0x7efee4106d30 2024-05-08T23:18:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.506+0000 7efedaa00700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 0x7efec80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.506+0000 7efedaa00700 1 -- 172.21.15.46:0/3073320996 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7efedc0cc990 con 0x7efee4106d30 2024-05-08T23:18:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.507+0000 7efee1e00700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 0x7efec80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.508+0000 7efee1e00700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 0x7efec80a3870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed4019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.508+0000 7efefc4c4b80 1 Processor -- start 2024-05-08T23:18:14.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 -- start start 2024-05-08T23:18:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af77254350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af77254cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af771ab150 con 0x55af7725a970 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af771a8b10 con 0x55af7725a050 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af771a4400 con 0x55af77254890 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efee1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efee1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af77254350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.509+0000 7efee1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36440/0 (socket says 172.21.15.46:36440) 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 -- 172.21.15.46:0/3808568201 learned_addr learned my addr 172.21.15.46:0/3808568201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee2800700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af77254cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 -- 172.21.15.46:0/3808568201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 msgr2=0x55af77254cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af77254cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 -- 172.21.15.46:0/3808568201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 msgr2=0x55af77254350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af77254350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee1e00700 1 -- 172.21.15.46:0/3808568201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af770253b0 con 0x55af7725a050 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.510+0000 7efee2800700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af77254cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.511+0000 7efee1400700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af77254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.511+0000 7efee1e00700 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efed4009230 tx=0x7efed401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639f39a7c6890f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.511+0000 7efecf000700 1 -- 172.21.15.46:0/3808568201 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400b860 con 0x55af7725a050 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.511+0000 7efecf000700 1 -- 172.21.15.46:0/3808568201 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed401d040 con 0x55af7725a050 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.511+0000 7efecf000700 1 -- 172.21.15.46:0/3808568201 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40039e0 con 0x55af7725a050 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3808568201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 msgr2=0x55af7725a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efed4009230 tx=0x7efed401f000 comp rx=0 tx=0).stop 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3808568201 shutdown_connections 2024-05-08T23:18:14.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af77254cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725a430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3808568201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af77254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3808568201 >> 172.21.15.46:0/3808568201 conn(0x55af771b2210 msgr2=0x55af7724f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.512+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3808568201 shutdown_connections 2024-05-08T23:18:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.513+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3808568201 wait complete. 2024-05-08T23:18:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.514+0000 7efefc4c4b80 1 -- start start 2024-05-08T23:18:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.514+0000 7efefc4c4b80 1 Processor -- start 2024-05-08T23:18:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af772608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af7725e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af77140560 con 0x55af7725a970 2024-05-08T23:18:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af77140200 con 0x55af7725a050 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af770276a0 con 0x55af77254890 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.515+0000 7efefc4c4b80 10 client.0 _ll_register_callbacks cb 0x55af77149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.516+0000 7efee1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.516+0000 7efee1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36444/0 (socket says 172.21.15.46:36444) 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.516+0000 7efee1400700 1 -- 172.21.15.46:0/3045839262 learned_addr learned my addr 172.21.15.46:0/3045839262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.516+0000 7efee1e00700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af772608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.516+0000 7efee2800700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af7725e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 -- 172.21.15.46:0/3045839262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 msgr2=0x55af772608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af772608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 -- 172.21.15.46:0/3045839262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 msgr2=0x55af7725e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af7725e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 -- 172.21.15.46:0/3045839262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed0009ba0 con 0x55af7725a050 2024-05-08T23:18:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee2800700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af7725e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.517+0000 7efee1400700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efed0013d00 tx=0x7efed00116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f3d30ebf7d0b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efec3400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0011e20 con 0x55af7725a050 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7efed0004830 con 0x55af7725a050 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af770fd270 con 0x55af7725a050 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed00049f0 con 0x55af7725a050 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efefc4c4b80 10 client.29522 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.518+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7efee410e390 con 0x55af7725a050 2024-05-08T23:18:14.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.519+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af770fd270 con 0x55af7725a050 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efed001b3f0 con 0x55af7725a050 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 10 client.29522 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 10 client.29522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 20 client.29522 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af7726e3b0 0x55af772707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 4 client.29522 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af77223810 con 0x55af7726e3b0 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.520+0000 7efee1e00700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af7726e3b0 0x55af772707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.521+0000 7efec3400700 10 client.29522 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.521+0000 7efee1e00700 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af7726e3b0 0x55af772707d0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b56e0dbdc36261 server_cookie=6f54a70e7ff67022 in_seq=0 out_seq=0 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.522+0000 7efec3400700 1 -- 172.21.15.46:0/3045839262 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af77223810 con 0x55af7726e3b0 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.522+0000 7efec3400700 10 client.29522 handle_command_reply: tid=0 2024-05-08T23:18:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.522+0000 7efefc4c4b80 1 client.29522 shutdown 2024-05-08T23:18:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.522+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 msgr2=0x55af7725dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.522+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efed0013d00 tx=0x7efed00116c0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 shutdown_connections 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af7726e3b0 0x55af772707d0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af7725a970 0x55af7725e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af77254890 0x55af772608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/3045839262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af7725dbf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 >> 172.21.15.46:0/3045839262 conn(0x55af771b2210 msgr2=0x55af7724f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efec3400700 0 client.29522 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 shutdown_connections 2024-05-08T23:18:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 1 -- 172.21.15.46:0/3045839262 wait complete. 2024-05-08T23:18:14.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.523+0000 7efefc4c4b80 20 client.29522 trim_cache size 0 max 16384 2024-05-08T23:18:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.544+0000 7efefc4c4b80 1 Processor -- start 2024-05-08T23:18:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.544+0000 7efefc4c4b80 1 -- start start 2024-05-08T23:18:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af77269d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af772509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7726ba20 con 0x55af77254890 2024-05-08T23:18:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af77254350 con 0x55af7725a050 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7724f7c0 con 0x55af7725a970 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efee2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efee2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47474/0 (socket says 172.21.15.46:47474) 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efee2800700 1 -- 172.21.15.46:0/1253512831 learned_addr learned my addr 172.21.15.46:0/1253512831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efee1e00700 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af77269d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.545+0000 7efee1400700 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af772509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 -- 172.21.15.46:0/1253512831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 msgr2=0x55af772509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af772509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 -- 172.21.15.46:0/1253512831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 msgr2=0x55af77269d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af77269d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 -- 172.21.15.46:0/1253512831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af76fd1b60 con 0x55af7725a970 2024-05-08T23:18:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.546+0000 7efee2800700 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efedc095bb0 tx=0x7efedc0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694fa9eeb4fddeb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.548+0000 7efecfa00700 1 -- 172.21.15.46:0/1253512831 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0b90b0 con 0x55af7725a970 2024-05-08T23:18:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.548+0000 7efecfa00700 1 -- 172.21.15.46:0/1253512831 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efedc099db0 con 0x55af7725a970 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.548+0000 7efecfa00700 1 -- 172.21.15.46:0/1253512831 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc098ba0 con 0x55af7725a970 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 -- 172.21.15.46:0/1253512831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 msgr2=0x55af772604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efedc095bb0 tx=0x7efedc0092e0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 -- 172.21.15.46:0/1253512831 shutdown_connections 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a970 0x55af772604e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af77269d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/1253512831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a050 0x55af772509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 -- 172.21.15.46:0/1253512831 >> 172.21.15.46:0/1253512831 conn(0x55af771b2210 msgr2=0x55af77275130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 -- 172.21.15.46:0/1253512831 shutdown_connections 2024-05-08T23:18:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.549+0000 7efefc4c4b80 1 -- 172.21.15.46:0/1253512831 wait complete. 2024-05-08T23:18:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 -- start start 2024-05-08T23:18:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 Processor -- start 2024-05-08T23:18:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af772656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a050 0x55af77255a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 0x55af77255fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af772566b0 con 0x55af77254890 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af77256820 con 0x55af7725a970 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.550+0000 7efefc4c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af772569e0 con 0x55af7725a050 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efefc4c4b80 10 client.0 _ll_register_callbacks cb 0x55af77149b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 0x55af77255fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af772656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 0x55af77255fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36464/0 (socket says 172.21.15.46:36464) 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee2800700 1 -- 172.21.15.46:0/88194565 learned_addr learned my addr 172.21.15.46:0/88194565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 -- 172.21.15.46:0/88194565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a050 msgr2=0x55af77255a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a050 0x55af77255a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 -- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 msgr2=0x55af77255fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 0x55af77255fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.551+0000 7efee1e00700 1 -- 172.21.15.46:0/88194565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc021060 con 0x55af77254890 2024-05-08T23:18:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efee1e00700 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af772656c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efed400fa00 tx=0x7efed400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674710ddb82deab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efec3400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:18:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4042070 con 0x55af77254890 2024-05-08T23:18:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9642+0+0 (secure 0 0 0) 0x7efed400b820 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efed401d040 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed403d070 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efefc4c4b80 10 client.19458 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.552+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55af7716a560 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.553+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed401d040 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.553+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efed4045030 con 0x55af77254890 2024-05-08T23:18:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.553+0000 7efefc4c4b80 10 client.19458 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.553+0000 7efefc4c4b80 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.553+0000 7efefc4c4b80 20 client.19458 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af772c4c10 0x55af772c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efefc4c4b80 4 client.19458 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af7719cec0 con 0x55af772c4c10 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efee1400700 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af772c4c10 0x55af772c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efee1400700 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af772c4c10 0x55af772c7000 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c926a9c8ee7e03 server_cookie=20474812ce7cf141 in_seq=0 out_seq=0 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.554+0000 7efec3400700 10 client.19458 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efec3400700 1 -- 172.21.15.46:0/88194565 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55af7719cec0 con 0x55af772c4c10 2024-05-08T23:18:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efec3400700 10 client.19458 handle_command_reply: tid=0 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 client.19458 shutdown 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 msgr2=0x55af772656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af772656c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efed400fa00 tx=0x7efed400bd10 comp rx=0 tx=0).stop 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 shutdown_connections 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7725a970 0x55af77255fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55af772c4c10 0x55af772c7000 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af77254890 0x55af772656c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 --2- 172.21.15.46:0/88194565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7725a050 0x55af77255a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 >> 172.21.15.46:0/88194565 conn(0x55af771b2210 msgr2=0x55af77257c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efec3400700 0 client.19458 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 shutdown_connections 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.555+0000 7efefc4c4b80 1 -- 172.21.15.46:0/88194565 wait complete. 2024-05-08T23:18:14.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efefc4c4b80 20 client.19458 trim_cache size 0 max 16384 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "85fd783b-437b-48c9-8a31-8ecf0803cd84": { 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "85fd783b-437b-48c9-8a31-8ecf0803cd84", 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:18:14.560 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 msgr2=0x7efec80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 0x7efec80a3870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed4019040 comp rx=0 tx=0).stop 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 msgr2=0x7efee419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 0x7efee419daf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efedc00efc0 tx=0x7efedc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 shutdown_connections 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4106d30 0x7efee419daf0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee4068160 0x7efee41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7efec80a13b0 0x7efec80a3870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 --2- 172.21.15.46:0/3073320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee4068b20 0x7efee4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.556+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 >> 172.21.15.46:0/3073320996 conn(0x7efee4075710 msgr2=0x7efee40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.557+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 shutdown_connections 2024-05-08T23:18:14.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:14.557+0000 7efee3200700 1 -- 172.21.15.46:0/3073320996 wait complete. 2024-05-08T23:18:14.587 INFO:tasks.cephfs.filesystem:scrub status for tag:85fd783b-437b-48c9-8a31-8ecf0803cd84 - {'path': '/', 'tag': '85fd783b-437b-48c9-8a31-8ecf0803cd84', 'options': 'recursive,force'} 2024-05-08T23:18:15.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:15.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:15.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: pgmap v451: 129 pgs: 129 active+clean; 860 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-08T23:18:15.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:15.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:15.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:15 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14834 27.19 MB/sec warmup 36 sec latency 708.433 ms 2024-05-08T23:18:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:15.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: pgmap v451: 129 pgs: 129 active+clean; 860 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-08T23:18:15.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:15.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:15.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:15 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: pgmap v451: 129 pgs: 129 active+clean; 860 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 513 op/s 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:15.514 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:15.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:15 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14970 26.58 MB/sec warmup 37 sec latency 1135.065 ms 2024-05-08T23:18:16.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:18:16.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:16.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:16 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:18:16.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:16.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:16 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:16 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:17.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 15264 26.27 MB/sec warmup 38 sec latency 251.553 ms 2024-05-08T23:18:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:17 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:17.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:17 smithi046 ceph-mon[36898]: pgmap v452: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:17.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:17 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:17.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:17 smithi086 ceph-mon[78610]: pgmap v452: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:17.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:17 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:17.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:17 smithi053 ceph-mon[77587]: pgmap v452: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:18.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 15688 26.16 MB/sec warmup 39 sec latency 259.543 ms 2024-05-08T23:18:19.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 16329 26.12 MB/sec warmup 40 sec latency 47.245 ms 2024-05-08T23:18:19.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:19 smithi086 ceph-mon[78610]: pgmap v453: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:19 smithi053 ceph-mon[77587]: pgmap v453: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:19 smithi046 ceph-mon[36898]: pgmap v453: 129 pgs: 129 active+clean; 843 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 483 op/s 2024-05-08T23:18:20.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 16998 26.44 MB/sec warmup 41 sec latency 43.879 ms 2024-05-08T23:18:21.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:20 smithi086 ceph-mon[78610]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:18:21.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 17542 26.73 MB/sec warmup 42 sec latency 156.739 ms 2024-05-08T23:18:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:20 smithi053 ceph-mon[77587]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:18:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:20 smithi046 ceph-mon[36898]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:18:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:21 smithi086 ceph-mon[78610]: pgmap v455: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-08T23:18:22.248 INFO:tasks.workunit.client.0.smithi046.stdout: 10 18061 26.95 MB/sec warmup 43 sec latency 203.832 ms 2024-05-08T23:18:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:21 smithi053 ceph-mon[77587]: pgmap v455: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-08T23:18:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:21 smithi046 ceph-mon[36898]: pgmap v455: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-08T23:18:23.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 18242 26.58 MB/sec warmup 44 sec latency 303.900 ms 2024-05-08T23:18:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:23 smithi086 ceph-mon[78610]: pgmap v456: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-08T23:18:24.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 18526 26.10 MB/sec warmup 45 sec latency 200.346 ms 2024-05-08T23:18:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:23 smithi053 ceph-mon[77587]: pgmap v456: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-08T23:18:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:23 smithi046 ceph-mon[36898]: pgmap v456: 129 pgs: 129 active+clean; 861 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-08T23:18:25.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 18909 25.85 MB/sec warmup 46 sec latency 269.134 ms 2024-05-08T23:18:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:25 smithi086 ceph-mon[78610]: pgmap v457: 129 pgs: 129 active+clean; 877 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.9 MiB/s wr, 527 op/s 2024-05-08T23:18:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:25 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:18:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:25 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:18:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:25 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:18:26.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 19476 25.81 MB/sec warmup 47 sec latency 180.044 ms 2024-05-08T23:18:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:25 smithi053 ceph-mon[77587]: pgmap v457: 129 pgs: 129 active+clean; 877 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.9 MiB/s wr, 527 op/s 2024-05-08T23:18:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:25 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:18:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:25 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:18:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:25 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:18:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:25 smithi046 ceph-mon[36898]: pgmap v457: 129 pgs: 129 active+clean; 877 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.9 MiB/s wr, 527 op/s 2024-05-08T23:18:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:25 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:18:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:25 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:18:26.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:25 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:18:27.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:26 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:18:27.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 20245 26.05 MB/sec warmup 48 sec latency 173.247 ms 2024-05-08T23:18:27.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:26 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:18:27.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:26 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:18:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:27 smithi086 ceph-mon[78610]: pgmap v458: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:28.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 20926 26.47 MB/sec warmup 49 sec latency 103.275 ms 2024-05-08T23:18:28.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:27 smithi053 ceph-mon[77587]: pgmap v458: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:27 smithi046 ceph-mon[36898]: pgmap v458: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:29.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 21508 26.92 MB/sec warmup 50 sec latency 75.341 ms 2024-05-08T23:18:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:29 smithi086 ceph-mon[78610]: pgmap v459: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:30.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 21880 26.70 MB/sec warmup 51 sec latency 155.613 ms 2024-05-08T23:18:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:29 smithi053 ceph-mon[77587]: pgmap v459: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:29 smithi046 ceph-mon[36898]: pgmap v459: 129 pgs: 129 active+clean; 883 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 533 op/s 2024-05-08T23:18:31.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 22162 26.44 MB/sec warmup 52 sec latency 238.849 ms 2024-05-08T23:18:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:31 smithi086 ceph-mon[78610]: pgmap v460: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-05-08T23:18:32.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 22510 26.20 MB/sec warmup 53 sec latency 230.070 ms 2024-05-08T23:18:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:31 smithi053 ceph-mon[77587]: pgmap v460: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-05-08T23:18:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:31 smithi046 ceph-mon[36898]: pgmap v460: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.8 MiB/s wr, 433 op/s 2024-05-08T23:18:33.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 23040 26.10 MB/sec warmup 54 sec latency 190.156 ms 2024-05-08T23:18:34.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:33 smithi086 ceph-mon[78610]: pgmap v461: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-08T23:18:34.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 23734 26.16 MB/sec warmup 55 sec latency 184.574 ms 2024-05-08T23:18:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:33 smithi053 ceph-mon[77587]: pgmap v461: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-08T23:18:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:33 smithi046 ceph-mon[36898]: pgmap v461: 129 pgs: 129 active+clean; 875 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-08T23:18:35.249 INFO:tasks.workunit.client.0.smithi046.stdout: 10 24438 26.57 MB/sec warmup 56 sec latency 155.407 ms 2024-05-08T23:18:36.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:35 smithi086 ceph-mon[78610]: pgmap v462: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.3 MiB/s wr, 469 op/s 2024-05-08T23:18:36.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 24945 26.76 MB/sec warmup 57 sec latency 154.027 ms 2024-05-08T23:18:36.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:35 smithi053 ceph-mon[77587]: pgmap v462: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.3 MiB/s wr, 469 op/s 2024-05-08T23:18:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:35 smithi046 ceph-mon[36898]: pgmap v462: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.3 MiB/s wr, 469 op/s 2024-05-08T23:18:37.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 25292 26.62 MB/sec warmup 58 sec latency 65.433 ms 2024-05-08T23:18:38.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:37 smithi086 ceph-mon[78610]: pgmap v463: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-05-08T23:18:38.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 25605 26.42 MB/sec warmup 59 sec latency 374.383 ms 2024-05-08T23:18:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:37 smithi053 ceph-mon[77587]: pgmap v463: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-05-08T23:18:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:37 smithi046 ceph-mon[36898]: pgmap v463: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-05-08T23:18:39.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 26052 26.32 MB/sec warmup 60 sec latency 187.232 ms 2024-05-08T23:18:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:39 smithi086 ceph-mon[78610]: pgmap v464: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-05-08T23:18:40.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 26595 26.22 MB/sec warmup 61 sec latency 209.718 ms 2024-05-08T23:18:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:39 smithi053 ceph-mon[77587]: pgmap v464: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-05-08T23:18:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:39 smithi046 ceph-mon[36898]: pgmap v464: 129 pgs: 129 active+clean; 880 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-05-08T23:18:41.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 26849 25.96 MB/sec warmup 62 sec latency 563.857 ms 2024-05-08T23:18:42.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:41 smithi086 ceph-mon[78610]: pgmap v465: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-05-08T23:18:42.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 26862 25.55 MB/sec warmup 63 sec latency 1563.945 ms 2024-05-08T23:18:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:41 smithi053 ceph-mon[77587]: pgmap v465: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-05-08T23:18:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:41 smithi046 ceph-mon[36898]: pgmap v465: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-05-08T23:18:43.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 27113 25.40 MB/sec warmup 64 sec latency 1853.203 ms 2024-05-08T23:18:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:43 smithi086 ceph-mon[78610]: pgmap v466: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-05-08T23:18:44.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 27517 25.35 MB/sec warmup 65 sec latency 218.466 ms 2024-05-08T23:18:44.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:43 smithi053 ceph-mon[77587]: pgmap v466: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-05-08T23:18:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:43 smithi046 ceph-mon[36898]: pgmap v466: 129 pgs: 129 active+clean; 888 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-05-08T23:18:44.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:18:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.926+0000 7fc919800700 1 -- 172.21.15.46:0/2132099501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9141051f0 msgr2=0x7fc9141056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.926+0000 7fc919800700 1 --2- 172.21.15.46:0/2132099501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9141051f0 0x7fc9141056a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00b0a0 tx=0x7fc90c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.928+0000 7fc919800700 1 -- 172.21.15.46:0/2132099501 shutdown_connections 2024-05-08T23:18:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.928+0000 7fc919800700 1 --2- 172.21.15.46:0/2132099501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9141051f0 0x7fc9141056a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.928+0000 7fc919800700 1 --2- 172.21.15.46:0/2132099501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9140ffdf0 0x7fc9141001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.928+0000 7fc919800700 1 --2- 172.21.15.46:0/2132099501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc914100710 0x7fc914104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.928+0000 7fc919800700 1 -- 172.21.15.46:0/2132099501 >> 172.21.15.46:0/2132099501 conn(0x7fc9140fb820 msgr2=0x7fc9140fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.929+0000 7fc919800700 1 -- 172.21.15.46:0/2132099501 shutdown_connections 2024-05-08T23:18:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.930+0000 7fc919800700 1 -- 172.21.15.46:0/2132099501 wait complete. 2024-05-08T23:18:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.932+0000 7fc919800700 1 Processor -- start 2024-05-08T23:18:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.932+0000 7fc919800700 1 -- start start 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9140ffdf0 0x7fc9141a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc914100710 0x7fc9141a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91419de40 con 0x7fc914100710 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91419dfb0 con 0x7fc9140ffdf0 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc919800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91419e170 con 0x7fc9141051f0 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc913400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc913400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:32962/0 (socket says 172.21.15.46:32962) 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.933+0000 7fc913400700 1 -- 172.21.15.46:0/2355827695 learned_addr learned my addr 172.21.15.46:0/2355827695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 -- 172.21.15.46:0/2355827695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9140ffdf0 msgr2=0x7fc9141a36a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9140ffdf0 0x7fc9141a36a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 -- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc914100710 msgr2=0x7fc9141a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc914100710 0x7fc9141a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 -- 172.21.15.46:0/2355827695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c009d00 con 0x7fc9141051f0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.934+0000 7fc913400700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc90c006070 tx=0x7fc90c005b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.935+0000 7fc90be00700 1 -- 172.21.15.46:0/2355827695 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00e040 con 0x7fc9141051f0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.935+0000 7fc90be00700 1 -- 172.21.15.46:0/2355827695 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc90c005530 con 0x7fc9141051f0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.935+0000 7fc90be00700 1 -- 172.21.15.46:0/2355827695 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c012bd0 con 0x7fc9141051f0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.935+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91419e3f0 con 0x7fc9141051f0 2024-05-08T23:18:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.935+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc914106380 con 0x7fc9141051f0 2024-05-08T23:18:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.940+0000 7fc90be00700 1 -- 172.21.15.46:0/2355827695 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc90c0056a0 con 0x7fc9141051f0 2024-05-08T23:18:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.943+0000 7fc90be00700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 0x7fc8f80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.943+0000 7fc90be00700 1 -- 172.21.15.46:0/2355827695 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc90c0df940 con 0x7fc9141051f0 2024-05-08T23:18:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.944+0000 7fc912a00700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 0x7fc8f80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.944+0000 7fc92d364b80 1 Processor -- start 2024-05-08T23:18:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.945+0000 7fc92d364b80 1 -- start start 2024-05-08T23:18:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.945+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f71501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.945+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.945+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f714acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc90a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc90a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55102/0 (socket says 172.21.15.46:55102) 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc913400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f714acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc913400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f714acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:32966/0 (socket says 172.21.15.46:32966) 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc92d364b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f70a1150 con 0x55f7f714fe00 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc92d364b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f709eb10 con 0x55f7f7150720 2024-05-08T23:18:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc92d364b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f709a400 con 0x55f7f714a890 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.946+0000 7fc90a000700 1 -- 172.21.15.46:0/149846822 learned_addr learned my addr 172.21.15.46:0/149846822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.947+0000 7fc90a000700 1 -- 172.21.15.46:0/149846822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 msgr2=0x55f7f714acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.947+0000 7fc90a000700 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f714acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.947+0000 7fc90a000700 1 -- 172.21.15.46:0/149846822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 msgr2=0x55f7f71501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.947+0000 7fc90a000700 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f71501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.947+0000 7fc90a000700 1 -- 172.21.15.46:0/149846822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7f6f1b3b0 con 0x55f7f7150720 2024-05-08T23:18:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.948+0000 7fc90a000700 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007b40 tx=0x7fc8fc007e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1460a413d7e60056 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.948+0000 7fc903e00700 1 -- 172.21.15.46:0/149846822 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc004830 con 0x55f7f7150720 2024-05-08T23:18:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.948+0000 7fc903e00700 1 -- 172.21.15.46:0/149846822 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8fc004e50 con 0x55f7f7150720 2024-05-08T23:18:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.948+0000 7fc903e00700 1 -- 172.21.15.46:0/149846822 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0052b0 con 0x55f7f7150720 2024-05-08T23:18:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.950+0000 7fc92d364b80 1 -- 172.21.15.46:0/149846822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 msgr2=0x55f7f714a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.950+0000 7fc912a00700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 0x7fc8f80a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097a0 tx=0x7fc904009b60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.950+0000 7fc92d364b80 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007b40 tx=0x7fc8fc007e50 comp rx=0 tx=0).stop 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.951+0000 7fc92d364b80 1 -- 172.21.15.46:0/149846822 shutdown_connections 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.951+0000 7fc92d364b80 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f714acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.951+0000 7fc92d364b80 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f71501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.951+0000 7fc92d364b80 1 --2- 172.21.15.46:0/149846822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f714a350 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.951+0000 7fc92d364b80 1 -- 172.21.15.46:0/149846822 >> 172.21.15.46:0/149846822 conn(0x55f7f70a8210 msgr2=0x55f7f7144df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.952+0000 7fc92d364b80 1 -- 172.21.15.46:0/149846822 shutdown_connections 2024-05-08T23:18:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.954+0000 7fc92d364b80 1 -- 172.21.15.46:0/149846822 wait complete. 2024-05-08T23:18:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.954+0000 7fc92d364b80 1 -- start start 2024-05-08T23:18:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.955+0000 7fc92d364b80 1 Processor -- start 2024-05-08T23:18:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714fe00 0x55f7f7153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 0x55f7f7154520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f7036560 con 0x55f7f714a890 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f7036200 con 0x55f7f714fe00 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f6f1d6a0 con 0x55f7f7150720 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc92d364b80 10 client.0 _ll_register_callbacks cb 0x55f7f703fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc912a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc912a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40590/0 (socket says 172.21.15.46:40590) 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.956+0000 7fc912a00700 1 -- 172.21.15.46:0/3567310875 learned_addr learned my addr 172.21.15.46:0/3567310875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 0x55f7f7154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc90a000700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714fe00 0x55f7f7153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 -- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714fe00 msgr2=0x55f7f7153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714fe00 0x55f7f7153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 -- 172.21.15.46:0/3567310875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 msgr2=0x55f7f7153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc913400700 1 -- 172.21.15.46:0/3567310875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc007770 con 0x55f7f7150720 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.957+0000 7fc912a00700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.958+0000 7fc913400700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 0x55f7f7154520 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0dcc80 tx=0x7fc90c007b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2761853e495fe99d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.958+0000 7fc900c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.959+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0b0320 con 0x55f7f7150720 2024-05-08T23:18:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.959+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc90c0b0500 con 0x55f7f7150720 2024-05-08T23:18:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.959+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7f6ff3270 con 0x55f7f7150720 2024-05-08T23:18:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.960+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90c0b4ac0 con 0x55f7f7150720 2024-05-08T23:18:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.961+0000 7fc92d364b80 10 client.29617 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.961+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7f6ec7b60 con 0x55f7f7150720 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7f6ff3270 con 0x55f7f7150720 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc90c0b4630 con 0x55f7f7150720 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 10 client.29617 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 10 client.29617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 20 client.29617 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f7163bd0 0x55f7f7165fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 4 client.29617 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.963+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7f7119810 con 0x55f7f7163bd0 2024-05-08T23:18:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.966+0000 7fc912a00700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f7163bd0 0x55f7f7165fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.966+0000 7fc912a00700 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f7163bd0 0x55f7f7165fc0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138c0c0ad0a571e2 server_cookie=2f2d7a14689e7b1e in_seq=0 out_seq=0 2024-05-08T23:18:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.966+0000 7fc900c00700 10 client.29617 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.969+0000 7fc900c00700 1 -- 172.21.15.46:0/3567310875 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7f7119810 con 0x55f7f7163bd0 2024-05-08T23:18:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.969+0000 7fc900c00700 10 client.29617 handle_command_reply: tid=0 2024-05-08T23:18:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.969+0000 7fc92d364b80 1 client.29617 shutdown 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.969+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 msgr2=0x55f7f7154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.969+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 0x55f7f7154520 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0dcc80 tx=0x7fc90c007b60 comp rx=0 tx=0).stop 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 shutdown_connections 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f7150720 0x55f7f7154520 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f7163bd0 0x55f7f7165fc0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714a890 0x55f7f7153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 --2- 172.21.15.46:0/3567310875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714fe00 0x55f7f7153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 >> 172.21.15.46:0/3567310875 conn(0x55f7f70a8210 msgr2=0x55f7f71445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.971+0000 7fc900c00700 0 client.29617 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.972+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 shutdown_connections 2024-05-08T23:18:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.972+0000 7fc92d364b80 1 -- 172.21.15.46:0/3567310875 wait complete. 2024-05-08T23:18:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:44.973+0000 7fc92d364b80 20 client.29617 trim_cache size 0 max 16384 2024-05-08T23:18:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.005+0000 7fc92d364b80 1 Processor -- start 2024-05-08T23:18:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.005+0000 7fc92d364b80 1 -- start start 2024-05-08T23:18:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 0x55f7f7144d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714fe00 0x55f7f7153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 0x55f7f7154530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f7145a90 con 0x55f7f7150720 2024-05-08T23:18:45.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f7156610 con 0x55f7f714a890 2024-05-08T23:18:45.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc92d364b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f7141dc0 con 0x55f7f714fe00 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc913400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 0x55f7f7154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc913400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 0x55f7f7154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40594/0 (socket says 172.21.15.46:40594) 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.006+0000 7fc913400700 1 -- 172.21.15.46:0/63089478 learned_addr learned my addr 172.21.15.46:0/63089478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.007+0000 7fc912a00700 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 0x55f7f7144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.008+0000 7fc912a00700 1 -- 172.21.15.46:0/63089478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714fe00 msgr2=0x55f7f7153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.008+0000 7fc912a00700 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714fe00 0x55f7f7153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.008+0000 7fc912a00700 1 -- 172.21.15.46:0/63089478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 msgr2=0x55f7f7154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.008+0000 7fc912a00700 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 0x55f7f7154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.008+0000 7fc912a00700 1 -- 172.21.15.46:0/63089478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7f7060560 con 0x55f7f714a890 2024-05-08T23:18:45.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.009+0000 7fc912a00700 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 0x55f7f7144d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0aaa20 tx=0x7fc904009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d637e291851b5ec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.009+0000 7fc908c00700 1 -- 172.21.15.46:0/63089478 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90401c040 con 0x55f7f714a890 2024-05-08T23:18:45.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.009+0000 7fc908c00700 1 -- 172.21.15.46:0/63089478 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc904021870 con 0x55f7f714a890 2024-05-08T23:18:45.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.009+0000 7fc908c00700 1 -- 172.21.15.46:0/63089478 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040156d0 con 0x55f7f714a890 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 -- 172.21.15.46:0/63089478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 msgr2=0x55f7f7144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 0x55f7f7144d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0aaa20 tx=0x7fc904009f90 comp rx=0 tx=0).stop 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 -- 172.21.15.46:0/63089478 shutdown_connections 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f7150720 0x55f7f7154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f714a890 0x55f7f7144d20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.010+0000 7fc92d364b80 1 --2- 172.21.15.46:0/63089478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714fe00 0x55f7f7153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.011+0000 7fc92d364b80 1 -- 172.21.15.46:0/63089478 >> 172.21.15.46:0/63089478 conn(0x55f7f70a8210 msgr2=0x55f7f7152370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.012+0000 7fc92d364b80 1 -- 172.21.15.46:0/63089478 shutdown_connections 2024-05-08T23:18:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.012+0000 7fc92d364b80 1 -- 172.21.15.46:0/63089478 wait complete. 2024-05-08T23:18:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.012+0000 7fc92d364b80 1 -- start start 2024-05-08T23:18:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.012+0000 7fc92d364b80 1 Processor -- start 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f7159340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f7159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f71b9570 con 0x55f7f714fe00 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f71b96b0 con 0x55f7f7150720 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f71b97f0 con 0x55f7f714a890 2024-05-08T23:18:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.013+0000 7fc92d364b80 10 client.0 _ll_register_callbacks cb 0x55f7f703fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.014+0000 7fc913400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.014+0000 7fc913400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55112/0 (socket says 172.21.15.46:55112) 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.014+0000 7fc913400700 1 -- 172.21.15.46:0/1518770024 learned_addr learned my addr 172.21.15.46:0/1518770024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.014+0000 7fc912a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f7159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.014+0000 7fc90a000700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f7159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 -- 172.21.15.46:0/1518770024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 msgr2=0x55f7f7159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f7159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 -- 172.21.15.46:0/1518770024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 msgr2=0x55f7f7159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f7159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 -- 172.21.15.46:0/1518770024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040126b0 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc913400700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0a9140 tx=0x7fc90c0b4860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc1303ee1888d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc900c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.015+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0c0510 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc90c0b0500 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8fc004e50 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90c0c0510 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc92d364b80 10 client.29543 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7f6f1fcd0 con 0x55f7f7150720 2024-05-08T23:18:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8fc004e50 con 0x55f7f7150720 2024-05-08T23:18:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.016+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc90c08dd20 con 0x55f7f7150720 2024-05-08T23:18:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 10 client.29543 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 10 client.29543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 20 client.29543 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f71baa80 0x55f7f71bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 4 client.29543 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f7f7092ec0 con 0x55f7f71baa80 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.017+0000 7fc912a00700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f71baa80 0x55f7f71bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc912a00700 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f71baa80 0x55f7f71bce70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270be0e8dd86a80e server_cookie=84973c7f37aecd42 in_seq=0 out_seq=0 2024-05-08T23:18:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc900c00700 10 client.29543 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc900c00700 1 -- 172.21.15.46:0/1518770024 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f7f7092ec0 con 0x55f7f71baa80 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc900c00700 10 client.29543 handle_command_reply: tid=0 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc92d364b80 1 client.29543 shutdown 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 msgr2=0x55f7f71b9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.018+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0a9140 tx=0x7fc90c0b4860 comp rx=0 tx=0).stop 2024-05-08T23:18:45.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 shutdown_connections 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f7f7150720 0x55f7f71b9030 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7f714a890 0x55f7f7159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f7f71baa80 0x55f7f71bce70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 --2- 172.21.15.46:0/1518770024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7f714fe00 0x55f7f7159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 >> 172.21.15.46:0/1518770024 conn(0x55f7f70a8210 msgr2=0x55f7f7155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc900c00700 0 client.29543 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 shutdown_connections 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 1 -- 172.21.15.46:0/1518770024 wait complete. 2024-05-08T23:18:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.019+0000 7fc92d364b80 20 client.29543 trim_cache size 0 max 16384 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.020+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 msgr2=0x7fc8f80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.020+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 0x7fc8f80a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097a0 tx=0x7fc904009b60 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.020+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 msgr2=0x7fc91419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.020+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc90c006070 tx=0x7fc90c005b80 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 shutdown_connections 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9141051f0 0x7fc91419d7d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9140ffdf0 0x7fc9141a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc8f80a1310 0x7fc8f80a37d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 --2- 172.21.15.46:0/2355827695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc914100710 0x7fc9141a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.022+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 >> 172.21.15.46:0/2355827695 conn(0x7fc9140fb820 msgr2=0x7fc914102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.023+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 shutdown_connections 2024-05-08T23:18:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.023+0000 7fc919800700 1 -- 172.21.15.46:0/2355827695 wait complete. 2024-05-08T23:18:45.060 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:18:45.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:18:45.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 28041 25.48 MB/sec warmup 66 sec latency 201.650 ms 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.411+0000 7f9c62a00700 1 -- 172.21.15.46:0/1755205054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 msgr2=0x7f9c5c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.411+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1755205054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c10d920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0098f0 tx=0x7f9c4c009c00 comp rx=0 tx=0).stop 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 -- 172.21.15.46:0/1755205054 shutdown_connections 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1755205054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c109a70 0x7f9c5c1153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1755205054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c10d920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1755205054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c1090b0 0x7f9c5c109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 -- 172.21.15.46:0/1755205054 >> 172.21.15.46:0/1755205054 conn(0x7f9c5c06c7d0 msgr2=0x7f9c5c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 -- 172.21.15.46:0/1755205054 shutdown_connections 2024-05-08T23:18:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.412+0000 7f9c62a00700 1 -- 172.21.15.46:0/1755205054 wait complete. 2024-05-08T23:18:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.416+0000 7f9c62a00700 1 Processor -- start 2024-05-08T23:18:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.416+0000 7f9c62a00700 1 -- start start 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.418+0000 7f9c62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c1090b0 0x7f9c5c1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.418+0000 7f9c62a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c109a70 0x7f9c5c1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.418+0000 7f9c62a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.418+0000 7f9c62a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19ffb0 con 0x7f9c5c109a70 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.418+0000 7f9c62a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1a0120 con 0x7f9c5c10d540 2024-05-08T23:18:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c62a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1a02e0 con 0x7f9c5c1090b0 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c5b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c109a70 0x7f9c5c1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55128/0 (socket says 172.21.15.46:55128) 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 -- 172.21.15.46:0/1744591585 learned_addr learned my addr 172.21.15.46:0/1744591585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 -- 172.21.15.46:0/1744591585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c1090b0 msgr2=0x7f9c5c1a5820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c1090b0 0x7f9c5c1a5820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 -- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c109a70 msgr2=0x7f9c5c1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c109a70 0x7f9c5c1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.419+0000 7f9c60a00700 1 -- 172.21.15.46:0/1744591585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c009520 con 0x7f9c5c10d540 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.420+0000 7f9c60a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007ee0 tx=0x7f9c5400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.420+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54017070 con 0x7f9c5c10d540 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.420+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c5400f040 con 0x7f9c5c10d540 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.420+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54005740 con 0x7f9c5c10d540 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.420+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c1a05c0 con 0x7f9c5c10d540 2024-05-08T23:18:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.421+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c116700 con 0x7f9c5c10d540 2024-05-08T23:18:45.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.424+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c110ab0 con 0x7f9c5c10d540 2024-05-08T23:18:45.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.426+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9c540058a0 con 0x7f9c5c10d540 2024-05-08T23:18:45.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.428+0000 7f9c58c00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 0x7f9c400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.428+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9c540cca00 con 0x7f9c5c10d540 2024-05-08T23:18:45.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.428+0000 7f9c5be00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 0x7f9c400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.429+0000 7f9c5be00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 0x7f9c400a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009230 tx=0x7f9c4c009f50 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.432+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9c540959e0 con 0x7f9c5c10d540 2024-05-08T23:18:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.883+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c5c1a07b0 con 0x7f9c5c10d540 2024-05-08T23:18:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.884+0000 7f9c58c00700 1 -- 172.21.15.46:0/1744591585 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10910 (secure 0 0 0) 0x7f9c54099000 con 0x7f9c5c10d540 2024-05-08T23:18:45.885 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:18:45.885 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:17:42.880411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:18:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.888+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 msgr2=0x7f9c400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 0x7f9c400a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009230 tx=0x7f9c4c009f50 comp rx=0 tx=0).stop 2024-05-08T23:18:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 msgr2=0x7f9c5c19f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007ee0 tx=0x7f9c5400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 shutdown_connections 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c5c10d540 0x7f9c5c19f950 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c1090b0 0x7f9c5c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9c400a1290 0x7f9c400a3750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 --2- 172.21.15.46:0/1744591585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c5c109a70 0x7f9c5c1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 >> 172.21.15.46:0/1744591585 conn(0x7f9c5c06c7d0 msgr2=0x7f9c5c070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 shutdown_connections 2024-05-08T23:18:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:45.889+0000 7f9c62a00700 1 -- 172.21.15.46:0/1744591585 wait complete. 2024-05-08T23:18:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-05-08T23:18:45.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:18:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:45 smithi086 ceph-mon[78610]: pgmap v467: 129 pgs: 129 active+clean; 901 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 495 op/s 2024-05-08T23:18:46.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:45 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1744591585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.246+0000 7f830d800700 1 -- 172.21.15.46:0/2528199490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 msgr2=0x7f8308102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.246+0000 7f830d800700 1 --2- 172.21.15.46:0/2528199490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 0x7f8308102380 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.246+0000 7f830d800700 1 -- 172.21.15.46:0/2528199490 shutdown_connections 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.246+0000 7f830d800700 1 --2- 172.21.15.46:0/2528199490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.247+0000 7f830d800700 1 --2- 172.21.15.46:0/2528199490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 0x7f8308102380 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.247+0000 7f830d800700 1 --2- 172.21.15.46:0/2528199490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 0x7f830810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.247+0000 7f830d800700 1 -- 172.21.15.46:0/2528199490 >> 172.21.15.46:0/2528199490 conn(0x7f83080fb760 msgr2=0x7f83080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.247+0000 7f830d800700 1 -- 172.21.15.46:0/2528199490 shutdown_connections 2024-05-08T23:18:46.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.247+0000 7f830d800700 1 -- 172.21.15.46:0/2528199490 wait complete. 2024-05-08T23:18:46.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.248+0000 7f830d800700 1 Processor -- start 2024-05-08T23:18:46.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.248+0000 7f830d800700 1 -- start start 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 0x7f83081972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 0x7f8308197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830819c1e0 con 0x7f8308101fa0 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830819c350 con 0x7f83081028c0 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f830d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830819c510 con 0x7f830810b3c0 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f8307400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f8307400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33032/0 (socket says 172.21.15.46:33032) 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f8307400700 1 -- 172.21.15.46:0/4135047261 learned_addr learned my addr 172.21.15.46:0/4135047261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.249+0000 7f8306000700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 0x7f8308197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 -- 172.21.15.46:0/4135047261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 msgr2=0x7f8308197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 0x7f8308197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 -- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 msgr2=0x7f83081972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 0x7f83081972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 -- 172.21.15.46:0/4135047261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80095f0 con 0x7f830810b3c0 2024-05-08T23:18:46.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 28504 25.57 MB/sec warmup 67 sec latency 60.478 ms 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f8307400700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f830000ef30 tx=0x7f830000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f82ff400700 1 -- 172.21.15.46:0/4135047261 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300018070 con 0x7f830810b3c0 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830819c790 con 0x7f830810b3c0 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f82ff400700 1 -- 172.21.15.46:0/4135047261 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830000f040 con 0x7f830810b3c0 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f82ff400700 1 -- 172.21.15.46:0/4135047261 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000cdb0 con 0x7f830810b3c0 2024-05-08T23:18:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.250+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83080ffd00 con 0x7f830810b3c0 2024-05-08T23:18:46.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.255+0000 7f82ff400700 1 -- 172.21.15.46:0/4135047261 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8300004000 con 0x7f830810b3c0 2024-05-08T23:18:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 Processor -- start 2024-05-08T23:18:46.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 -- start start 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f82ff400700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 0x7f82ec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f82ff400700 1 -- 172.21.15.46:0/4135047261 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83000cccf0 con 0x7f830810b3c0 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89390350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d89390cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d892e7150 con 0x557d89396720 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d892e4b10 con 0x557d89395e00 2024-05-08T23:18:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.257+0000 7f832133fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d892e0400 con 0x557d89390890 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55148/0 (socket says 172.21.15.46:55148) 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 -- 172.21.15.46:0/359086039 learned_addr learned my addr 172.21.15.46:0/359086039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89390350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306000700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 0x7f82ec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 -- 172.21.15.46:0/359086039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 msgr2=0x557d89390cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d89390cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 -- 172.21.15.46:0/359086039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 msgr2=0x557d89390350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.258+0000 7f8306a00700 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89390350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.259+0000 7f8306a00700 1 -- 172.21.15.46:0/359086039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d891613b0 con 0x557d89395e00 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.259+0000 7f8306a00700 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b2e0 tx=0x7f82f8005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409811a2fcfc8953 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.259+0000 7f82f3e00700 1 -- 172.21.15.46:0/359086039 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f801d070 con 0x557d89395e00 2024-05-08T23:18:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.260+0000 7f82f3e00700 1 -- 172.21.15.46:0/359086039 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f800bc10 con 0x557d89395e00 2024-05-08T23:18:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.260+0000 7f82f3e00700 1 -- 172.21.15.46:0/359086039 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800fbc0 con 0x557d89395e00 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- 172.21.15.46:0/359086039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 msgr2=0x557d893961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b2e0 tx=0x7f82f8005f00 comp rx=0 tx=0).stop 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f8306000700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 0x7f82ec0a37d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8308198810 tx=0x7f82f40077e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- 172.21.15.46:0/359086039 shutdown_connections 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d89390cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893961e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 --2- 172.21.15.46:0/359086039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89390350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- 172.21.15.46:0/359086039 >> 172.21.15.46:0/359086039 conn(0x557d892ee210 msgr2=0x557d8938adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- 172.21.15.46:0/359086039 shutdown_connections 2024-05-08T23:18:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- 172.21.15.46:0/359086039 wait complete. 2024-05-08T23:18:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.261+0000 7f832133fb80 1 -- start start 2024-05-08T23:18:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 Processor -- start 2024-05-08T23:18:46.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:45 smithi053 ceph-mon[77587]: pgmap v467: 129 pgs: 129 active+clean; 901 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 495 op/s 2024-05-08T23:18:46.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:45 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1744591585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d89399b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d8939a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8927c560 con 0x557d89390890 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8927c200 con 0x557d89395e00 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d891636a0 con 0x557d89396720 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f832133fb80 10 client.0 _ll_register_callbacks cb 0x557d89285b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f8306000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d8939a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f8307400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f8306a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d89399b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f8307400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33060/0 (socket says 172.21.15.46:33060) 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 -- 172.21.15.46:0/2287788144 learned_addr learned my addr 172.21.15.46:0/2287788144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.262+0000 7f8306000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d8939a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55160/0 (socket says 172.21.15.46:55160) 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 -- 172.21.15.46:0/2287788144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 msgr2=0x557d8939a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d8939a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 -- 172.21.15.46:0/2287788144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 msgr2=0x557d89399b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d89399b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 -- 172.21.15.46:0/2287788144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830810e540 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f8307400700 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f83000a18e0 tx=0x7f8300099560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f7a376e63def4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f82f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300005ac0 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d89239270 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.263+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8300004ad0 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830009a120 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 10 client.29632 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d8910db60 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d89239270 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f83000a1980 con 0x557d89396720 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 10 client.29632 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 10 client.29632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 20 client.29632 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d893aa260 0x557d893ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 4 client.29632 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.264+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d8935f810 con 0x557d893aa260 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.265+0000 7f8306a00700 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d893aa260 0x557d893ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.265+0000 7f8306a00700 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d893aa260 0x557d893ac680 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66808aa8e81e413d server_cookie=40247854b41f4e51 in_seq=0 out_seq=0 2024-05-08T23:18:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.265+0000 7f82f0c00700 10 client.29632 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f82f0c00700 1 -- 172.21.15.46:0/2287788144 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d8935f810 con 0x557d893aa260 2024-05-08T23:18:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f82f0c00700 10 client.29632 handle_command_reply: tid=0 2024-05-08T23:18:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 client.29632 shutdown 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 msgr2=0x557d8939a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f83000a18e0 tx=0x7f8300099560 comp rx=0 tx=0).stop 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 shutdown_connections 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d893aa260 0x557d893ac680 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89396720 0x557d8939a5a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d89399b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 --2- 172.21.15.46:0/2287788144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d8939a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 >> 172.21.15.46:0/2287788144 conn(0x557d892ee210 msgr2=0x557d8938a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f82f0c00700 0 client.29632 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 shutdown_connections 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 1 -- 172.21.15.46:0/2287788144 wait complete. 2024-05-08T23:18:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.266+0000 7f832133fb80 20 client.29632 trim_cache size 0 max 16384 2024-05-08T23:18:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:45 smithi046 ceph-mon[36898]: pgmap v467: 129 pgs: 129 active+clean; 901 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 495 op/s 2024-05-08T23:18:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:45 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1744591585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 Processor -- start 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 -- start start 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d893ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89387210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d89387dc0 con 0x557d89396720 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8938ce50 con 0x557d89395e00 2024-05-08T23:18:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.285+0000 7f832133fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d893906d0 con 0x557d89390890 2024-05-08T23:18:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d893ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55174/0 (socket says 172.21.15.46:55174) 2024-05-08T23:18:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306000700 1 -- 172.21.15.46:0/798562883 learned_addr learned my addr 172.21.15.46:0/798562883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8307400700 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89387210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 -- 172.21.15.46:0/798562883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 msgr2=0x557d893a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 -- 172.21.15.46:0/798562883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 msgr2=0x557d89387210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89387210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.286+0000 7f8306a00700 1 -- 172.21.15.46:0/798562883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d892a6560 con 0x557d89390890 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.287+0000 7f8306a00700 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d893ac650 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f800f760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2668e4cc476731aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.287+0000 7f8306000700 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.287+0000 7f82fcc00700 1 -- 172.21.15.46:0/798562883 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f801d040 con 0x557d89390890 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.287+0000 7f82fcc00700 1 -- 172.21.15.46:0/798562883 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f801e040 con 0x557d89390890 2024-05-08T23:18:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.287+0000 7f82fcc00700 1 -- 172.21.15.46:0/798562883 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f802f180 con 0x557d89390890 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 -- 172.21.15.46:0/798562883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 msgr2=0x557d893ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d893ac650 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f800f760 comp rx=0 tx=0).stop 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 -- 172.21.15.46:0/798562883 shutdown_connections 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89396720 0x557d89387210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89390890 0x557d893ac650 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 --2- 172.21.15.46:0/798562883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89395e00 0x557d893a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.288+0000 7f832133fb80 1 -- 172.21.15.46:0/798562883 >> 172.21.15.46:0/798562883 conn(0x557d892ee210 msgr2=0x557d893993b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.289+0000 7f832133fb80 1 -- 172.21.15.46:0/798562883 shutdown_connections 2024-05-08T23:18:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.289+0000 7f832133fb80 1 -- 172.21.15.46:0/798562883 wait complete. 2024-05-08T23:18:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.289+0000 7f832133fb80 1 -- start start 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.290+0000 7f832133fb80 1 Processor -- start 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.290+0000 7f832133fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d893a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.290+0000 7f832133fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89395e00 0x557d893a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f832133fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f832133fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8939d9f0 con 0x557d89390890 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f832133fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8939db60 con 0x557d89396720 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f832133fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d8939dd20 con 0x557d89395e00 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f832133fb80 10 client.0 _ll_register_callbacks cb 0x557d89285b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8306a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d893a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8306a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d893a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40684/0 (socket says 172.21.15.46:40684) 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55186/0 (socket says 172.21.15.46:55186) 2024-05-08T23:18:46.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8306a00700 1 -- 172.21.15.46:0/4227488411 learned_addr learned my addr 172.21.15.46:0/4227488411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 -- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89395e00 msgr2=0x557d893a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89395e00 0x557d893a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 -- 172.21.15.46:0/4227488411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 msgr2=0x557d893a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d893a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.291+0000 7f8307400700 1 -- 172.21.15.46:0/4227488411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f401a040 con 0x557d89396720 2024-05-08T23:18:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.292+0000 7f8307400700 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8300009f40 tx=0x7f83000998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22395f7d40733789 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.292+0000 7f82f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.292+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300005ac0 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.292+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8300004ad0 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.292+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f800bc10 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830009a120 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 10 client.29567 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d891613b0 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8300064030 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8300078be0 con 0x557d89396720 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 10 client.29567 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 10 client.29567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 20 client.29567 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d894005d0 0x557d894029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 4 client.29567 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.293+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d892d8ec0 con 0x557d894005d0 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f8306a00700 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d894005d0 0x557d894029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f82f0c00700 10 client.29567 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f8306a00700 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d894005d0 0x557d894029c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=660052c32673663d server_cookie=c611044ef623077e in_seq=0 out_seq=0 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f82f0c00700 1 -- 172.21.15.46:0/4227488411 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d892d8ec0 con 0x557d894005d0 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f82f0c00700 10 client.29567 handle_command_reply: tid=0 2024-05-08T23:18:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f832133fb80 1 client.29567 shutdown 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 msgr2=0x557d8939d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.294+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8300009f40 tx=0x7f83000998a0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 shutdown_connections 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x557d894005d0 0x557d894029c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d89396720 0x557d8939d2e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d89390890 0x557d893a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 --2- 172.21.15.46:0/4227488411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d89395e00 0x557d893a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 >> 172.21.15.46:0/4227488411 conn(0x557d892ee210 msgr2=0x557d89398390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f82f0c00700 0 client.29567 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:18:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 shutdown_connections 2024-05-08T23:18:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 1 -- 172.21.15.46:0/4227488411 wait complete. 2024-05-08T23:18:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.295+0000 7f832133fb80 20 client.29567 trim_cache size 0 max 16384 2024-05-08T23:18:46.298 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 msgr2=0x7f82ec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 0x7f82ec0a37d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8308198810 tx=0x7f82f40077e0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 msgr2=0x7f830819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f830000ef30 tx=0x7f830000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 shutdown_connections 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f830810b3c0 0x7f830819bba0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f82ec0a1310 0x7f82ec0a37d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308101fa0 0x7f83081972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 --2- 172.21.15.46:0/4135047261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83081028c0 0x7f8308197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 >> 172.21.15.46:0/4135047261 conn(0x7f83080fb760 msgr2=0x7f83080fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 shutdown_connections 2024-05-08T23:18:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.296+0000 7f830d800700 1 -- 172.21.15.46:0/4135047261 wait complete. 2024-05-08T23:18:46.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:18:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.730+0000 7f6aacc00700 1 -- 172.21.15.46:0/1390300236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068b20 msgr2=0x7f6aa810dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.730+0000 7f6aacc00700 1 --2- 172.21.15.46:0/1390300236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068b20 0x7f6aa810dad0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000f530 tx=0x7f6aa000f840 comp rx=0 tx=0).stop 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 -- 172.21.15.46:0/1390300236 shutdown_connections 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 --2- 172.21.15.46:0/1390300236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068b20 0x7f6aa810dad0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 --2- 172.21.15.46:0/1390300236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8106d30 0x7f6aa8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 --2- 172.21.15.46:0/1390300236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8068160 0x7f6aa80685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 -- 172.21.15.46:0/1390300236 >> 172.21.15.46:0/1390300236 conn(0x7f6aa8075710 msgr2=0x7f6aa8075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 -- 172.21.15.46:0/1390300236 shutdown_connections 2024-05-08T23:18:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.731+0000 7f6aacc00700 1 -- 172.21.15.46:0/1390300236 wait complete. 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.732+0000 7f6aacc00700 1 Processor -- start 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.732+0000 7f6aacc00700 1 -- start start 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.732+0000 7f6aacc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068160 0x7f6aa8194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aacc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aacc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 0x7f6aa81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aacc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8199e20 con 0x7f6aa8106d30 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aacc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8199f90 con 0x7f6aa8068b20 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aacc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa819a150 con 0x7f6aa8068160 2024-05-08T23:18:46.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068160 0x7f6aa8194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55202/0 (socket says 172.21.15.46:55202) 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 -- 172.21.15.46:0/3133075975 learned_addr learned my addr 172.21.15.46:0/3133075975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 0x7f6aa81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 -- 172.21.15.46:0/3133075975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068160 msgr2=0x7f6aa8194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068160 0x7f6aa8194f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 -- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 msgr2=0x7f6aa81997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 0x7f6aa81997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa6a00700 1 -- 172.21.15.46:0/3133075975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009d00 con 0x7f6aa8068b20 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.733+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 0x7f6aa81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.734+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d950 tx=0x7f6a9400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.734+0000 7f6a9fe00700 1 -- 172.21.15.46:0/3133075975 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94009940 con 0x7f6aa8068b20 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.734+0000 7f6a9fe00700 1 -- 172.21.15.46:0/3133075975 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a94010460 con 0x7f6aa8068b20 2024-05-08T23:18:46.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.734+0000 7f6a9fe00700 1 -- 172.21.15.46:0/3133075975 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400f620 con 0x7f6aa8068b20 2024-05-08T23:18:46.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.734+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa819a430 con 0x7f6aa8068b20 2024-05-08T23:18:46.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.735+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa81a1c70 con 0x7f6aa8068b20 2024-05-08T23:18:46.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.738+0000 7f6a9fe00700 1 -- 172.21.15.46:0/3133075975 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6a9400f780 con 0x7f6aa8068b20 2024-05-08T23:18:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.739+0000 7f6a9fe00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 0x7f6a8c0a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6a9fe00700 1 -- 172.21.15.46:0/3133075975 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6a940ccda0 con 0x7f6aa8068b20 2024-05-08T23:18:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 Processor -- start 2024-05-08T23:18:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6aa7400700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 0x7f6a8c0a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 -- start start 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x564306774b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x564306775520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643066c3150 con 0x56430677b230 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643066c0b10 con 0x5643067750a0 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.740+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643066bc400 con 0x56430677a910 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 0x7f6a8c0a3950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009c60 tx=0x7f6a98009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33118/0 (socket says 172.21.15.46:33118) 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 -- 172.21.15.46:0/3552128007 learned_addr learned my addr 172.21.15.46:0/3552128007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x564306774b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x564306775520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 -- 172.21.15.46:0/3552128007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 msgr2=0x564306775520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x564306775520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 -- 172.21.15.46:0/3552128007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 msgr2=0x564306774b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x564306774b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 -- 172.21.15.46:0/3552128007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56430653d3b0 con 0x56430677a910 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x564306774b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.741+0000 7f6aa7400700 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004140 tx=0x7f6a98014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de44511bb5443a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.742+0000 7f6a93e00700 1 -- 172.21.15.46:0/3552128007 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003b10 con 0x56430677a910 2024-05-08T23:18:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.742+0000 7f6a93e00700 1 -- 172.21.15.46:0/3552128007 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a98004cf0 con 0x56430677a910 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.742+0000 7f6a93e00700 1 -- 172.21.15.46:0/3552128007 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a980056c0 con 0x56430677a910 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3552128007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 msgr2=0x56430677acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004140 tx=0x7f6a98014000 comp rx=0 tx=0).stop 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3552128007 shutdown_connections 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x564306775520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677acf0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3552128007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x564306774b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3552128007 >> 172.21.15.46:0/3552128007 conn(0x5643066ca210 msgr2=0x5643067672d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3552128007 shutdown_connections 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.743+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3552128007 wait complete. 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.744+0000 7f6ac1dd1b80 1 -- start start 2024-05-08T23:18:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.744+0000 7f6ac1dd1b80 1 Processor -- start 2024-05-08T23:18:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x5643067810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x56430677eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564306658560 con 0x56430677b230 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564306658200 con 0x5643067750a0 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430653f6a0 con 0x56430677a910 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6ac1dd1b80 10 client.0 _ll_register_callbacks cb 0x564306661b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x5643067810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33132/0 (socket says 172.21.15.46:33132) 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6aa6a00700 1 -- 172.21.15.46:0/4167020818 learned_addr learned my addr 172.21.15.46:0/4167020818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.745+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x56430677eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 -- 172.21.15.46:0/4167020818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 msgr2=0x5643067810c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x5643067810c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 -- 172.21.15.46:0/4167020818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 msgr2=0x56430677eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x56430677eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 -- 172.21.15.46:0/4167020818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98017070 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6aa6a00700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6a940999d0 tx=0x7f6a94099b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9929da4cb0d0a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6a90c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94099f70 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6a94098f30 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564306615270 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a940b2400 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6ac1dd1b80 10 client.29647 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.746+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5643064e9b60 con 0x56430677a910 2024-05-08T23:18:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9407d020 con 0x56430677a910 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a94098aa0 con 0x56430677a910 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 10 client.29647 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 10 client.29647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 20 client.29647 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x564306790ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 4 client.29647 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56430673b810 con 0x56430678eab0 2024-05-08T23:18:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.747+0000 7f6aa7400700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x564306790ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.748+0000 7f6aa7400700 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x564306790ed0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e0b45d113fdda89 server_cookie=3f62ad97515b789c in_seq=0 out_seq=0 2024-05-08T23:18:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.748+0000 7f6a90c00700 10 client.29647 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.748+0000 7f6a90c00700 1 -- 172.21.15.46:0/4167020818 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56430673b810 con 0x56430678eab0 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.748+0000 7f6a90c00700 10 client.29647 handle_command_reply: tid=0 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 client.29647 shutdown 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 msgr2=0x56430677e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6a940999d0 tx=0x7f6a94099b80 comp rx=0 tx=0).stop 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 shutdown_connections 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x564306790ed0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6a90c00700 0 client.29647 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56430677b230 0x56430677eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5643067750a0 0x5643067810c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/4167020818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x56430677e3b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.749+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 >> 172.21.15.46:0/4167020818 conn(0x5643066ca210 msgr2=0x5643067670b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.750+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 shutdown_connections 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.750+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/4167020818 wait complete. 2024-05-08T23:18:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.750+0000 7f6ac1dd1b80 20 client.29647 trim_cache size 0 max 16384 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.769+0000 7f6ac1dd1b80 1 Processor -- start 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 -- start start 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x56430677ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306765d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x5643067784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430677fa20 con 0x5643067750a0 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643067960c0 con 0x56430677b230 2024-05-08T23:18:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643067677c0 con 0x56430677a910 2024-05-08T23:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x56430677ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306765d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6aa6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306765d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33140/0 (socket says 172.21.15.46:33140) 2024-05-08T23:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6aa6a00700 1 -- 172.21.15.46:0/3197739868 learned_addr learned my addr 172.21.15.46:0/3197739868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.770+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x5643067784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 -- 172.21.15.46:0/3197739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 msgr2=0x564306765d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306765d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 -- 172.21.15.46:0/3197739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 msgr2=0x56430677ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x56430677ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 -- 172.21.15.46:0/3197739868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564306682560 con 0x56430677b230 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.771+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x5643067784d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fd0 tx=0x7f6aa0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e48ebc7f1bddd4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.772+0000 7f6a9d200700 1 -- 172.21.15.46:0/3197739868 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000f2b0 con 0x56430677b230 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.772+0000 7f6a9d200700 1 -- 172.21.15.46:0/3197739868 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa00092e0 con 0x56430677b230 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.772+0000 7f6a9d200700 1 -- 172.21.15.46:0/3197739868 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0017790 con 0x56430677b230 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3197739868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 msgr2=0x5643067784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x5643067784d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fd0 tx=0x7f6aa0009b80 comp rx=0 tx=0).stop 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3197739868 shutdown_connections 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x5643067784d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x56430677ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/3197739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306765d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3197739868 >> 172.21.15.46:0/3197739868 conn(0x5643066ca210 msgr2=0x564306763210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3197739868 shutdown_connections 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.773+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/3197739868 wait complete. 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 -- start start 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 Processor -- start 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x5643067667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306766cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643067e4080 con 0x5643067750a0 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643067e41f0 con 0x56430677b230 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643067f53c0 con 0x56430677a910 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.774+0000 7f6ac1dd1b80 10 client.0 _ll_register_callbacks cb 0x564306661b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55228/0 (socket says 172.21.15.46:55228) 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 -- 172.21.15.46:0/405400163 learned_addr learned my addr 172.21.15.46:0/405400163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 -- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 msgr2=0x564306766cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306766cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.775+0000 7f6aa7e00700 1 -- 172.21.15.46:0/405400163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 msgr2=0x5643067667b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.776+0000 7f6aa7400700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x5643067667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.776+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x5643067667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.776+0000 7f6aa7e00700 1 -- 172.21.15.46:0/405400163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa810e390 con 0x56430677b230 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.776+0000 7f6aa7e00700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6aa001a040 tx=0x7f6aa0006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b0142927398a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.776+0000 7f6a90c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0009450 con 0x56430677b230 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6aa000e070 con 0x56430677b230 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a98004cf0 con 0x56430677b230 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aa0017760 con 0x56430677b230 2024-05-08T23:18:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6ac1dd1b80 10 client.29585 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.777+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564306541cd0 con 0x56430677b230 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a98004cf0 con 0x56430677b230 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6aa001e070 con 0x56430677b230 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 10 client.29585 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 10 client.29585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 20 client.29585 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x5643067f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 4 client.29585 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643066b4ec0 con 0x56430678eab0 2024-05-08T23:18:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.778+0000 7f6aa7400700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x5643067f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.779+0000 7f6a90c00700 10 client.29585 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.779+0000 7f6aa7400700 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x5643067f8830 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e289b2fef269b392 server_cookie=d829b5c9541816f7 in_seq=0 out_seq=0 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.779+0000 7f6a90c00700 1 -- 172.21.15.46:0/405400163 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643066b4ec0 con 0x56430678eab0 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.779+0000 7f6a90c00700 10 client.29585 handle_command_reply: tid=0 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.779+0000 7f6ac1dd1b80 1 client.29585 shutdown 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 msgr2=0x564306767230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6aa001a040 tx=0x7f6aa0006160 comp rx=0 tx=0).stop 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 shutdown_connections 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x56430678eab0 0x5643067f8830 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56430677b230 0x564306767230 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643067750a0 0x5643067667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 --2- 172.21.15.46:0/405400163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56430677a910 0x564306766cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 >> 172.21.15.46:0/405400163 conn(0x5643066ca210 msgr2=0x564306795830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6a90c00700 0 client.29585 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 shutdown_connections 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 1 -- 172.21.15.46:0/405400163 wait complete. 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.780+0000 7f6ac1dd1b80 20 client.29585 trim_cache size 0 max 16384 2024-05-08T23:18:46.783 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 msgr2=0x7f6a8c0a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 0x7f6a8c0a3950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009c60 tx=0x7f6a98009260 comp rx=0 tx=0).stop 2024-05-08T23:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 msgr2=0x7f6aa8195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d950 tx=0x7f6a9400dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 shutdown_connections 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6a8c0a1490 0x7f6a8c0a3950 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa8106d30 0x7f6aa81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa8068160 0x7f6aa8194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 --2- 172.21.15.46:0/3133075975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa8068b20 0x7f6aa8195490 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 >> 172.21.15.46:0/3133075975 conn(0x7f6aa8075710 msgr2=0x7f6aa80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 shutdown_connections 2024-05-08T23:18:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:46.781+0000 7f6aacc00700 1 -- 172.21.15.46:0/3133075975 wait complete. 2024-05-08T23:18:46.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:18:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.131+0000 7ff83ae00700 1 -- 172.21.15.46:0/559373896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8340ffd90 msgr2=0x7ff834100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.131+0000 7ff83ae00700 1 --2- 172.21.15.46:0/559373896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8340ffd90 0x7ff834100170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff824009990 tx=0x7ff824009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 -- 172.21.15.46:0/559373896 shutdown_connections 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 --2- 172.21.15.46:0/559373896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834105190 0x7ff834105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 --2- 172.21.15.46:0/559373896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8340ffd90 0x7ff834100170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 --2- 172.21.15.46:0/559373896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8341006b0 0x7ff834104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 -- 172.21.15.46:0/559373896 >> 172.21.15.46:0/559373896 conn(0x7ff8340fb760 msgr2=0x7ff8340fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.132+0000 7ff83ae00700 1 -- 172.21.15.46:0/559373896 shutdown_connections 2024-05-08T23:18:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.133+0000 7ff83ae00700 1 -- 172.21.15.46:0/559373896 wait complete. 2024-05-08T23:18:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.135+0000 7ff83ae00700 1 Processor -- start 2024-05-08T23:18:47.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 -- start start 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8341006b0 0x7ff834114350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834105190 0x7ff83410cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83410d7b0 con 0x7ff8340ffd90 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83410d920 con 0x7ff8341006b0 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.136+0000 7ff83ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83410dae0 con 0x7ff834105190 2024-05-08T23:18:47.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.137+0000 7ff833e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.137+0000 7ff833e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40766/0 (socket says 172.21.15.46:40766) 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.137+0000 7ff833e00700 1 -- 172.21.15.46:0/3013693436 learned_addr learned my addr 172.21.15.46:0/3013693436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.137+0000 7ff833400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8341006b0 0x7ff834114350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.138+0000 7ff833e00700 1 -- 172.21.15.46:0/3013693436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834105190 msgr2=0x7ff83410cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.138+0000 7ff833e00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834105190 0x7ff83410cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.138+0000 7ff833e00700 1 -- 172.21.15.46:0/3013693436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8341006b0 msgr2=0x7ff834114350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.138+0000 7ff833e00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8341006b0 0x7ff834114350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.138+0000 7ff833e00700 1 -- 172.21.15.46:0/3013693436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff820009520 con 0x7ff8340ffd90 2024-05-08T23:18:47.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.139+0000 7ff833e00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff824000c00 tx=0x7ff824005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.139+0000 7ff830c00700 1 -- 172.21.15.46:0/3013693436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82401d070 con 0x7ff8340ffd90 2024-05-08T23:18:47.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.139+0000 7ff830c00700 1 -- 172.21.15.46:0/3013693436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff824022470 con 0x7ff8340ffd90 2024-05-08T23:18:47.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.139+0000 7ff830c00700 1 -- 172.21.15.46:0/3013693436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824017650 con 0x7ff8340ffd90 2024-05-08T23:18:47.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.139+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240095f0 con 0x7ff8340ffd90 2024-05-08T23:18:47.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.140+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff834111b40 con 0x7ff8340ffd90 2024-05-08T23:18:47.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.145+0000 7ff830c00700 1 -- 172.21.15.46:0/3013693436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8240177b0 con 0x7ff8340ffd90 2024-05-08T23:18:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.149+0000 7ff830c00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 0x7ff8180a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.149+0000 7ff830c00700 1 -- 172.21.15.46:0/3013693436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff8240ce450 con 0x7ff8340ffd90 2024-05-08T23:18:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.150+0000 7ff833400700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 0x7ff8180a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.151+0000 7ff84e9c6b80 1 Processor -- start 2024-05-08T23:18:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.151+0000 7ff84e9c6b80 1 -- start start 2024-05-08T23:18:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d860350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d860890 0x55946d860cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d7b7150 con 0x55946d860890 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff833e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d7b4b10 con 0x55946d866720 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.152+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d7b0400 con 0x55946d865e00 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33184/0 (socket says 172.21.15.46:33184) 2024-05-08T23:18:47.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 -- 172.21.15.46:0/1085010239 learned_addr learned my addr 172.21.15.46:0/1085010239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff838e00700 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d860890 0x55946d860cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 -- 172.21.15.46:0/1085010239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 msgr2=0x55946d860350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d860350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 -- 172.21.15.46:0/1085010239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d860890 msgr2=0x55946d860cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d860890 0x55946d860cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.153+0000 7ff833e00700 1 -- 172.21.15.46:0/1085010239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55946d6313b0 con 0x55946d865e00 2024-05-08T23:18:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.154+0000 7ff833400700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 0x7ff8180a3850 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff83410e5d0 tx=0x7ff820009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.154+0000 7ff833e00700 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff8240981a0 tx=0x7ff824009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6745ec66a2654ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.154+0000 7ff829600700 1 -- 172.21.15.46:0/1085010239 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82409b090 con 0x55946d865e00 2024-05-08T23:18:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.154+0000 7ff829600700 1 -- 172.21.15.46:0/1085010239 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82409be30 con 0x55946d865e00 2024-05-08T23:18:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.154+0000 7ff829600700 1 -- 172.21.15.46:0/1085010239 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240ce130 con 0x55946d865e00 2024-05-08T23:18:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1085010239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 msgr2=0x55946d8661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff8240981a0 tx=0x7ff824009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1085010239 shutdown_connections 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d860890 0x55946d860cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d8661e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1085010239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.155+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1085010239 >> 172.21.15.46:0/1085010239 conn(0x55946d7be210 msgr2=0x55946d85adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.156+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1085010239 shutdown_connections 2024-05-08T23:18:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.156+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1085010239 wait complete. 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.156+0000 7ff84e9c6b80 1 -- start start 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.157+0000 7ff84e9c6b80 1 Processor -- start 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.157+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d860890 0x55946d869b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d865e00 0x55946d86a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d74c560 con 0x55946d865e00 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d74c200 con 0x55946d866720 2024-05-08T23:18:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d6336a0 con 0x55946d860890 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff84e9c6b80 10 client.0 _ll_register_callbacks cb 0x55946d755b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55270/0 (socket says 172.21.15.46:55270) 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 -- 172.21.15.46:0/3009990005 learned_addr learned my addr 172.21.15.46:0/3009990005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 -- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d860890 msgr2=0x55946d869b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d860890 0x55946d869b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 -- 172.21.15.46:0/3009990005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d865e00 msgr2=0x55946d86a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d865e00 0x55946d86a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.158+0000 7ff838e00700 1 -- 172.21.15.46:0/3009990005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff834111b40 con 0x55946d866720 2024-05-08T23:18:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.159+0000 7ff838e00700 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0078f0 tx=0x7ff82c007cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d0865d650a6d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.159+0000 7ff81e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.159+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00fa50 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.159+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff82c00fbb0 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.159+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55946d709270 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82c00fd70 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 10 client.29594 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55946d5ddb60 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55946d709270 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff82c0143d0 con 0x55946d866720 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 10 client.29594 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 10 client.29594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:18:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 20 client.29594 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d87a080 0x55946d87c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 4 client.29594 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.160+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55946d82f810 con 0x55946d87a080 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.161+0000 7ff833e00700 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d87a080 0x55946d87c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.162+0000 7ff833e00700 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d87a080 0x55946d87c4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11c1a1f537d983af server_cookie=cefcd0cfb3f1ef49 in_seq=0 out_seq=0 2024-05-08T23:18:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.162+0000 7ff81e000700 10 client.29594 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:47.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.173+0000 7ff81e000700 1 -- 172.21.15.46:0/3009990005 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55946d82f810 con 0x55946d87a080 2024-05-08T23:18:47.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.173+0000 7ff81e000700 10 client.29594 handle_command_reply: tid=0 2024-05-08T23:18:47.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.173+0000 7ff84e9c6b80 1 client.29594 shutdown 2024-05-08T23:18:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.173+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 msgr2=0x55946d86a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.173+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0078f0 tx=0x7ff82c007cb0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 shutdown_connections 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d866720 0x55946d86a5a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d860890 0x55946d869b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d87a080 0x55946d87c4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/3009990005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d865e00 0x55946d86a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 >> 172.21.15.46:0/3009990005 conn(0x55946d7be210 msgr2=0x55946d85a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.174+0000 7ff81e000700 0 client.29594 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.175+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 shutdown_connections 2024-05-08T23:18:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.175+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/3009990005 wait complete. 2024-05-08T23:18:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.175+0000 7ff84e9c6b80 20 client.29594 trim_cache size 0 max 16384 2024-05-08T23:18:47.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.200+0000 7ff84e9c6b80 1 Processor -- start 2024-05-08T23:18:47.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.200+0000 7ff84e9c6b80 1 -- start start 2024-05-08T23:18:47.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d87c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d860c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d876a70 con 0x55946d866720 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d86b190 con 0x55946d860890 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.201+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d86cb70 con 0x55946d865e00 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55274/0 (socket says 172.21.15.46:55274) 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 -- 172.21.15.46:0/2668500601 learned_addr learned my addr 172.21.15.46:0/2668500601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833400700 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d87c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff838e00700 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d860c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 -- 172.21.15.46:0/2668500601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 msgr2=0x55946d87c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d87c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 -- 172.21.15.46:0/2668500601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 msgr2=0x55946d860c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d860c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.202+0000 7ff833e00700 1 -- 172.21.15.46:0/2668500601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55946d776560 con 0x55946d860890 2024-05-08T23:18:47.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.203+0000 7ff833e00700 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff82409b6c0 tx=0x7ff82401ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd5e59f6a58c3ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.203+0000 7ff82a000700 1 -- 172.21.15.46:0/2668500601 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82409b7b0 con 0x55946d860890 2024-05-08T23:18:47.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.203+0000 7ff82a000700 1 -- 172.21.15.46:0/2668500601 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8240a3570 con 0x55946d860890 2024-05-08T23:18:47.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.203+0000 7ff82a000700 1 -- 172.21.15.46:0/2668500601 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240a10b0 con 0x55946d860890 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/2668500601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 msgr2=0x55946d860350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff82409b6c0 tx=0x7ff82401ffd0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/2668500601 shutdown_connections 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d860c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d860350 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/2668500601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d87c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/2668500601 >> 172.21.15.46:0/2668500601 conn(0x55946d7be210 msgr2=0x55946d857210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/2668500601 shutdown_connections 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.204+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/2668500601 wait complete. 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.205+0000 7ff84e9c6b80 1 -- start start 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.205+0000 7ff84e9c6b80 1 Processor -- start 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.206+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d870ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.206+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.206+0000 7ff84e9c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d86eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d86f5d0 con 0x55946d866720 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff84e9c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d86f740 con 0x55946d860890 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33224/0 (socket says 172.21.15.46:33224) 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 -- 172.21.15.46:0/1756148345 learned_addr learned my addr 172.21.15.46:0/1756148345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946d86f900 con 0x55946d865e00 2024-05-08T23:18:47.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833e00700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d870ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff84e9c6b80 10 client.0 _ll_register_callbacks cb 0x55946d755b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 -- 172.21.15.46:0/1756148345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 msgr2=0x55946d870ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d870ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 -- 172.21.15.46:0/1756148345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 msgr2=0x55946d86eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d86eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.207+0000 7ff833400700 1 -- 172.21.15.46:0/1756148345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55946d6313b0 con 0x55946d865e00 2024-05-08T23:18:47.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.208+0000 7ff833400700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x55946d866b80 tx=0x7ff8200047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f775d544d5b005 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.208+0000 7ff81e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:47.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.208+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82001e630 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.208+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff82001e790 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.208+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff82409be30 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.209+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82001e950 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.209+0000 7ff84e9c6b80 10 client.29662 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.209+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55946d635cd0 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.209+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82409be30 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.209+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff820031a10 con 0x55946d865e00 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 10 client.29662 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 10 client.29662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 20 client.29662 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d8d0860 0x55946d8d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 4 client.29662 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55946d7a8ec0 con 0x55946d8d0860 2024-05-08T23:18:47.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff833e00700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d8d0860 0x55946d8d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.210+0000 7ff81e000700 10 client.29662 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:47.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.211+0000 7ff833e00700 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d8d0860 0x55946d8d2c80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce6363f304e3a16f server_cookie=64e9ebb35f3c6a7f in_seq=0 out_seq=0 2024-05-08T23:18:47.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.212+0000 7ff81e000700 1 -- 172.21.15.46:0/1756148345 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55946d7a8ec0 con 0x55946d8d0860 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.212+0000 7ff81e000700 10 client.29662 handle_command_reply: tid=0 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.213+0000 7ff84e9c6b80 1 client.29662 shutdown 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 msgr2=0x55946d871420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x55946d866b80 tx=0x7ff8200047c0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 shutdown_connections 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55946d8d0860 0x55946d8d2c80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946d866720 0x55946d86eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55946d860890 0x55946d870ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff81e000700 0 client.29662 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 --2- 172.21.15.46:0/1756148345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946d865e00 0x55946d871420 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 >> 172.21.15.46:0/1756148345 conn(0x55946d7be210 msgr2=0x55946d860c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.214+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 shutdown_connections 2024-05-08T23:18:47.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.215+0000 7ff84e9c6b80 1 -- 172.21.15.46:0/1756148345 wait complete. 2024-05-08T23:18:47.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.216+0000 7ff84e9c6b80 20 client.29662 trim_cache size 0 max 16384 2024-05-08T23:18:47.218 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:47.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.217+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 msgr2=0x7ff8180a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.217+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 0x7ff8180a3850 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff83410e5d0 tx=0x7ff820009260 comp rx=0 tx=0).stop 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.217+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 msgr2=0x7ff834113dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.217+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff824000c00 tx=0x7ff824005740 comp rx=0 tx=0).stop 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 shutdown_connections 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834105190 0x7ff83410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8180a1390 0x7ff8180a3850 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340ffd90 0x7ff834113dd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 --2- 172.21.15.46:0/3013693436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8341006b0 0x7ff834114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.218+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 >> 172.21.15.46:0/3013693436 conn(0x7ff8340fb760 msgr2=0x7ff834102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.219+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 shutdown_connections 2024-05-08T23:18:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.219+0000 7ff83ae00700 1 -- 172.21.15.46:0/3013693436 wait complete. 2024-05-08T23:18:47.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 28706 25.35 MB/sec warmup 68 sec latency 301.339 ms 2024-05-08T23:18:47.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 -- 172.21.15.46:0/2967066117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f281061b0 msgr2=0x7f8f2810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 --2- 172.21.15.46:0/2967066117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f281061b0 0x7f8f2810dd30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009990 tx=0x7f8f18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 -- 172.21.15.46:0/2967066117 shutdown_connections 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 --2- 172.21.15.46:0/2967066117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f281061b0 0x7f8f2810dd30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 --2- 172.21.15.46:0/2967066117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f28101090 0x7f8f28101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 --2- 172.21.15.46:0/2967066117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28101a40 0x7f8f28105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 -- 172.21.15.46:0/2967066117 >> 172.21.15.46:0/2967066117 conn(0x7f8f280fc8f0 msgr2=0x7f8f280fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 -- 172.21.15.46:0/2967066117 shutdown_connections 2024-05-08T23:18:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.600+0000 7f8f30200700 1 -- 172.21.15.46:0/2967066117 wait complete. 2024-05-08T23:18:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.602+0000 7f8f30200700 1 Processor -- start 2024-05-08T23:18:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.602+0000 7f8f30200700 1 -- start start 2024-05-08T23:18:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 0x7f8f28195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f281999e0 0x7f8f28199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2819a460 con 0x7f8f281999e0 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2819a5d0 con 0x7f8f28101090 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f30200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2819a790 con 0x7f8f281061b0 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f2ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f2ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33238/0 (socket says 172.21.15.46:33238) 2024-05-08T23:18:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.604+0000 7f8f2ce00700 1 -- 172.21.15.46:0/4242721861 learned_addr learned my addr 172.21.15.46:0/4242721861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 0x7f8f28195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 -- 172.21.15.46:0/4242721861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 msgr2=0x7f8f28195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 -- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f281999e0 msgr2=0x7f8f28199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f281999e0 0x7f8f28199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2d800700 1 -- 172.21.15.46:0/4242721861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180095f0 con 0x7f8f28101090 2024-05-08T23:18:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.605+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.606+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 0x7f8f28195100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000bd70 tx=0x7f8f2000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.606+0000 7f8f26000700 1 -- 172.21.15.46:0/4242721861 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000cc90 con 0x7f8f28101090 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.606+0000 7f8f26000700 1 -- 172.21.15.46:0/4242721861 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f2000cdf0 con 0x7f8f28101090 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.606+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f281aa500 con 0x7f8f28101090 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.606+0000 7f8f26000700 1 -- 172.21.15.46:0/4242721861 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20012760 con 0x7f8f28101090 2024-05-08T23:18:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.607+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f281aaa30 con 0x7f8f28101090 2024-05-08T23:18:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.610+0000 7f8f26000700 1 -- 172.21.15.46:0/4242721861 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f20014440 con 0x7f8f28101090 2024-05-08T23:18:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.612+0000 7f8f26000700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 0x7f8f0c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.613+0000 7f8f26000700 1 -- 172.21.15.46:0/4242721861 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8f20006e40 con 0x7f8f28101090 2024-05-08T23:18:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.613+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 0x7f8f0c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.615+0000 7f8f43d66b80 1 Processor -- start 2024-05-08T23:18:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.615+0000 7f8f43d66b80 1 -- start start 2024-05-08T23:18:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.616+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 0x7f8f0c0a3830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f28196650 tx=0x7f8f1400b480 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.616+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.616+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b19350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.616+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b19890 0x556006b19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006a70150 con 0x556006b19890 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006a6db10 con 0x556006b1ee00 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006a69400 con 0x556006b1f720 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55318/0 (socket says 172.21.15.46:55318) 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 -- 172.21.15.46:0/2067756524 learned_addr learned my addr 172.21.15.46:0/2067756524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b19350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2e200700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b19890 0x556006b19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 -- 172.21.15.46:0/2067756524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 msgr2=0x556006b19350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b19350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 -- 172.21.15.46:0/2067756524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b19890 msgr2=0x556006b19cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b19890 0x556006b19cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 -- 172.21.15.46:0/2067756524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560068ea3b0 con 0x556006b1ee00 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b19350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.617+0000 7f8f2d800700 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8f200961a0 tx=0x7f8f20099340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c358e9ec776afcbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.618+0000 7f8f1ea00700 1 -- 172.21.15.46:0/2067756524 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2009d560 con 0x556006b1ee00 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.618+0000 7f8f1ea00700 1 -- 172.21.15.46:0/2067756524 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f200a1b20 con 0x556006b1ee00 2024-05-08T23:18:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.618+0000 7f8f1ea00700 1 -- 172.21.15.46:0/2067756524 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2009d160 con 0x556006b1ee00 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 -- 172.21.15.46:0/2067756524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 msgr2=0x556006b1f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8f200961a0 tx=0x7f8f20099340 comp rx=0 tx=0).stop 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 -- 172.21.15.46:0/2067756524 shutdown_connections 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b19890 0x556006b19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1ee00 0x556006b1f1e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/2067756524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b19350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 -- 172.21.15.46:0/2067756524 >> 172.21.15.46:0/2067756524 conn(0x556006a77210 msgr2=0x556006b13df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 -- 172.21.15.46:0/2067756524 shutdown_connections 2024-05-08T23:18:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.619+0000 7f8f43d66b80 1 -- 172.21.15.46:0/2067756524 wait complete. 2024-05-08T23:18:47.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.620+0000 7f8f43d66b80 1 -- start start 2024-05-08T23:18:47.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.620+0000 7f8f43d66b80 1 Processor -- start 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b23040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006a05560 con 0x556006b1ee00 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006a05200 con 0x556006b19890 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560068ec6a0 con 0x556006b1f720 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f43d66b80 10 client.0 _ll_register_callbacks cb 0x556006a0eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.621+0000 7f8f2d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55322/0 (socket says 172.21.15.46:55322) 2024-05-08T23:18:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 -- 172.21.15.46:0/4207838412 learned_addr learned my addr 172.21.15.46:0/4207838412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2e200700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b23040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 -- 172.21.15.46:0/4207838412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 msgr2=0x556006b23580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b23580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 -- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 msgr2=0x556006b23040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b23040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.622+0000 7f8f2d800700 1 -- 172.21.15.46:0/4207838412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f281aaa30 con 0x556006b19890 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8f20099430 tx=0x7f8f20095ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651d54e5bcedaacd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b23040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2009d360 con 0x556006b19890 2024-05-08T23:18:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8f20097b90 con 0x556006b19890 2024-05-08T23:18:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560069c2270 con 0x556006b19890 2024-05-08T23:18:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f200c9e50 con 0x556006b19890 2024-05-08T23:18:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f43d66b80 10 client.29612 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556006896b60 con 0x556006b19890 2024-05-08T23:18:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f20080030 con 0x556006b19890 2024-05-08T23:18:47.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.623+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8f200b4070 con 0x556006b19890 2024-05-08T23:18:47.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 10 client.29612 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 10 client.29612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 20 client.29612 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b32e30 0x556006b35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 4 client.29612 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556006ae8810 con 0x556006b32e30 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b32e30 0x556006b35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b32e30 0x556006b35250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e94679b47ecf3a8 server_cookie=6b0651f12bc3b8b0 in_seq=0 out_seq=0 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.624+0000 7f8f13400700 10 client.29612 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f13400700 1 -- 172.21.15.46:0/4207838412 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556006ae8810 con 0x556006b32e30 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f13400700 10 client.29612 handle_command_reply: tid=0 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 client.29612 shutdown 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 msgr2=0x556006b22b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8f20099430 tx=0x7f8f20095ba0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 shutdown_connections 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b23580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b22b00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b32e30 0x556006b35250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4207838412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b23040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f13400700 0 client.29612 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.625+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 >> 172.21.15.46:0/4207838412 conn(0x556006a77210 msgr2=0x556006b135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.626+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 shutdown_connections 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.626+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4207838412 wait complete. 2024-05-08T23:18:47.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.626+0000 7f8f43d66b80 20 client.29612 trim_cache size 0 max 16384 2024-05-08T23:18:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.640+0000 7f8f43d66b80 1 Processor -- start 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.640+0000 7f8f43d66b80 1 -- start start 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b19890 0x556006b19350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b13d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b25670 con 0x556006b1ee00 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b0ea90 con 0x556006b1f720 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b15e50 con 0x556006b19890 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f2e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f2ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b13d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f2e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55328/0 (socket says 172.21.15.46:55328) 2024-05-08T23:18:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f2e200700 1 -- 172.21.15.46:0/1532816551 learned_addr learned my addr 172.21.15.46:0/1532816551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.641+0000 7f8f2d800700 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b19890 0x556006b19350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 -- 172.21.15.46:0/1532816551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b19890 msgr2=0x556006b19350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b19890 0x556006b19350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 -- 172.21.15.46:0/1532816551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 msgr2=0x556006b13d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b13d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 -- 172.21.15.46:0/1532816551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556006b19f40 con 0x556006b1f720 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b13d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f2e200700 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009990 tx=0x7f8f18005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f08c7d716dde64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.642+0000 7f8f1f400700 1 -- 172.21.15.46:0/1532816551 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800b9d0 con 0x556006b1f720 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.643+0000 7f8f1f400700 1 -- 172.21.15.46:0/1532816551 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f1801e040 con 0x556006b1f720 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.643+0000 7f8f1f400700 1 -- 172.21.15.46:0/1532816551 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18003e60 con 0x556006b1f720 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.643+0000 7f8f43d66b80 1 -- 172.21.15.46:0/1532816551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 msgr2=0x556006b1e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.643+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009990 tx=0x7f8f18005c00 comp rx=0 tx=0).stop 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 -- 172.21.15.46:0/1532816551 shutdown_connections 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b1f720 0x556006b1e2e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b19890 0x556006b19350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/1532816551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b13d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 -- 172.21.15.46:0/1532816551 >> 172.21.15.46:0/1532816551 conn(0x556006a77210 msgr2=0x556006b1dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 -- 172.21.15.46:0/1532816551 shutdown_connections 2024-05-08T23:18:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 -- 172.21.15.46:0/1532816551 wait complete. 2024-05-08T23:18:47.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.644+0000 7f8f43d66b80 1 -- start start 2024-05-08T23:18:47.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 Processor -- start 2024-05-08T23:18:47.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b2a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b28aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b2aab0 con 0x556006b1ee00 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b28fe0 con 0x556006b19890 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556006b29170 con 0x556006b1f720 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.645+0000 7f8f43d66b80 10 client.0 _ll_register_callbacks cb 0x556006a0eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40874/0 (socket says 172.21.15.46:40874) 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2ce00700 1 -- 172.21.15.46:0/4077129003 learned_addr learned my addr 172.21.15.46:0/4077129003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b2a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2e200700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b28aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 -- 172.21.15.46:0/4077129003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 msgr2=0x556006b28aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b28aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 -- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 msgr2=0x556006b28560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 -- 172.21.15.46:0/4077129003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2810e5a0 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f2d800700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b2a1a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000cfd0 tx=0x7f8f20004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262ba0c85be37c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f13400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.646+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20099270 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8f20097b90 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f200a1b20 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f2009d560 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 10 client.29618 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556006a2f560 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f200a1b20 con 0x556006b19890 2024-05-08T23:18:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8f200a9070 con 0x556006b19890 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 10 client.29618 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 10 client.29618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 20 client.29618 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b840e0 0x556006b844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 4 client.29618 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.647+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556006a61ec0 con 0x556006b840e0 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b840e0 0x556006b844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f2ce00700 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b840e0 0x556006b844c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38a14ee2e9ffb401 server_cookie=f46b1b84760e8ea6 in_seq=0 out_seq=0 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f13400700 10 client.29618 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f13400700 1 -- 172.21.15.46:0/4077129003 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556006a61ec0 con 0x556006b840e0 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f13400700 10 client.29618 handle_command_reply: tid=0 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.648+0000 7f8f43d66b80 1 client.29618 shutdown 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 msgr2=0x556006b2a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b2a1a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000cfd0 tx=0x7f8f20004ad0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 shutdown_connections 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556006b1f720 0x556006b28aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556006b19890 0x556006b2a1a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x556006b840e0 0x556006b844c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 --2- 172.21.15.46:0/4077129003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556006b1ee00 0x556006b28560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f13400700 0 client.29618 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 >> 172.21.15.46:0/4077129003 conn(0x556006a77210 msgr2=0x556006b1e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 shutdown_connections 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 1 -- 172.21.15.46:0/4077129003 wait complete. 2024-05-08T23:18:47.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.649+0000 7f8f43d66b80 20 client.29618 trim_cache size 0 max 16384 2024-05-08T23:18:47.654 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:47.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 msgr2=0x7f8f0c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 0x7f8f0c0a3830 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f28196650 tx=0x7f8f1400b480 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 msgr2=0x7f8f28195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 0x7f8f28195100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000bd70 tx=0x7f8f2000bda0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 shutdown_connections 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8f0c0a1370 0x7f8f0c0a3830 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f281999e0 0x7f8f28199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f28101090 0x7f8f28195100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 --2- 172.21.15.46:0/4242721861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f281061b0 0x7f8f28195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 >> 172.21.15.46:0/4242721861 conn(0x7f8f280fc8f0 msgr2=0x7f8f280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 shutdown_connections 2024-05-08T23:18:47.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.650+0000 7f8f30200700 1 -- 172.21.15.46:0/4242721861 wait complete. 2024-05-08T23:18:47.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.965+0000 7f6c19000700 1 -- 172.21.15.46:0/982673207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073370 msgr2=0x7f6c14073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.965+0000 7f6c19000700 1 --2- 172.21.15.46:0/982673207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073370 0x7f6c14073750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009990 tx=0x7f6c04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.966+0000 7f6c19000700 1 -- 172.21.15.46:0/982673207 shutdown_connections 2024-05-08T23:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.966+0000 7f6c19000700 1 --2- 172.21.15.46:0/982673207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1410d6d0 0x7f6c1410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.966+0000 7f6c19000700 1 --2- 172.21.15.46:0/982673207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073370 0x7f6c14073750 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.966+0000 7f6c19000700 1 --2- 172.21.15.46:0/982673207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c14073c90 0x7f6c1410d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.966+0000 7f6c19000700 1 -- 172.21.15.46:0/982673207 >> 172.21.15.46:0/982673207 conn(0x7f6c140fc960 msgr2=0x7f6c140fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.967+0000 7f6c19000700 1 -- 172.21.15.46:0/982673207 shutdown_connections 2024-05-08T23:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.967+0000 7f6c19000700 1 -- 172.21.15.46:0/982673207 wait complete. 2024-05-08T23:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.968+0000 7f6c19000700 1 Processor -- start 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 -- start start 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c14073370 0x7f6c1410cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1410d6d0 0x7f6c1410a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1410ad20 con 0x7f6c14073370 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1410ae60 con 0x7f6c14073c90 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c19000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c14198700 con 0x7f6c1410d6d0 2024-05-08T23:18:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c11400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c11400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55356/0 (socket says 172.21.15.46:55356) 2024-05-08T23:18:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.969+0000 7f6c11400700 1 -- 172.21.15.46:0/927833756 learned_addr learned my addr 172.21.15.46:0/927833756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.970+0000 7f6c11400700 1 -- 172.21.15.46:0/927833756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1410d6d0 msgr2=0x7f6c1410a550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.970+0000 7f6c11400700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1410d6d0 0x7f6c1410a550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.970+0000 7f6c11400700 1 -- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c14073370 msgr2=0x7f6c1410cef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.970+0000 7f6c11e00700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c14073370 0x7f6c1410cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.971+0000 7f6c11400700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c14073370 0x7f6c1410cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.971+0000 7f6c11400700 1 -- 172.21.15.46:0/927833756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040095f0 con 0x7f6c14073c90 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.971+0000 7f6c11400700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00b550 tx=0x7f6bfc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.972+0000 7f6c0aa00700 1 -- 172.21.15.46:0/927833756 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc011830 con 0x7f6c14073c90 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.972+0000 7f6c0aa00700 1 -- 172.21.15.46:0/927833756 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc011e50 con 0x7f6c14073c90 2024-05-08T23:18:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.972+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c141989e0 con 0x7f6c14073c90 2024-05-08T23:18:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.972+0000 7f6c0aa00700 1 -- 172.21.15.46:0/927833756 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00f2e0 con 0x7f6c14073c90 2024-05-08T23:18:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.972+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14198ef0 con 0x7f6c14073c90 2024-05-08T23:18:47.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.978+0000 7f6c0aa00700 1 -- 172.21.15.46:0/927833756 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6bfc010580 con 0x7f6c14073c90 2024-05-08T23:18:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.980+0000 7f6c0aa00700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 0x7f6bf80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.980+0000 7f6c0aa00700 1 -- 172.21.15.46:0/927833756 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6bfc0cca00 con 0x7f6c14073c90 2024-05-08T23:18:47.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.981+0000 7f6c11e00700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 0x7f6bf80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.982+0000 7f6c11e00700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 0x7f6bf80a38a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b5c0 tx=0x7f6c04005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.982+0000 7f6c2caebb80 1 Processor -- start 2024-05-08T23:18:47.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.983+0000 7f6c2caebb80 1 -- start start 2024-05-08T23:18:47.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f3ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83e93150 con 0x561e83f42720 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83e90b10 con 0x561e83f3c890 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83e8c400 con 0x561e83f41e00 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33276/0 (socket says 172.21.15.46:33276) 2024-05-08T23:18:47.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.984+0000 7f6c11e00700 1 -- 172.21.15.46:0/2975890858 learned_addr learned my addr 172.21.15.46:0/2975890858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11e00700 1 -- 172.21.15.46:0/2975890858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 msgr2=0x561e83f3ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11e00700 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f3ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11e00700 1 -- 172.21.15.46:0/2975890858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 msgr2=0x561e83f3c350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11400700 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11e00700 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f3c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.985+0000 7f6c11e00700 1 -- 172.21.15.46:0/2975890858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e83d0d3b0 con 0x561e83f41e00 2024-05-08T23:18:47.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.986+0000 7f6c11e00700 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bf80 tx=0x7f6c0401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb127b99089ad8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.986+0000 7f6c03400700 1 -- 172.21.15.46:0/2975890858 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04029070 con 0x561e83f41e00 2024-05-08T23:18:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.988+0000 7f6c03400700 1 -- 172.21.15.46:0/2975890858 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c04005580 con 0x561e83f41e00 2024-05-08T23:18:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.988+0000 7f6c03400700 1 -- 172.21.15.46:0/2975890858 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401d070 con 0x561e83f41e00 2024-05-08T23:18:47.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.989+0000 7f6c2caebb80 1 -- 172.21.15.46:0/2975890858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 msgr2=0x561e83f421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:47.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.989+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bf80 tx=0x7f6c0401d000 comp rx=0 tx=0).stop 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.990+0000 7f6c2caebb80 1 -- 172.21.15.46:0/2975890858 shutdown_connections 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.990+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f3ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.990+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f41e00 0x561e83f421e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.990+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/2975890858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.990+0000 7f6c2caebb80 1 -- 172.21.15.46:0/2975890858 >> 172.21.15.46:0/2975890858 conn(0x561e83e9a210 msgr2=0x561e83f36df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.991+0000 7f6c2caebb80 1 -- 172.21.15.46:0/2975890858 shutdown_connections 2024-05-08T23:18:47.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.991+0000 7f6c2caebb80 1 -- 172.21.15.46:0/2975890858 wait complete. 2024-05-08T23:18:47.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.992+0000 7f6c2caebb80 1 -- start start 2024-05-08T23:18:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.992+0000 7f6c2caebb80 1 Processor -- start 2024-05-08T23:18:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83e28560 con 0x561e83f3c890 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83e28200 con 0x561e83f41e00 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83d0f6a0 con 0x561e83f42720 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.994+0000 7f6c2caebb80 10 client.0 _ll_register_callbacks cb 0x561e83e31b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.995+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.995+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f45b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40902/0 (socket says 172.21.15.46:40902) 2024-05-08T23:18:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.995+0000 7f6c11e00700 1 -- 172.21.15.46:0/520258075 learned_addr learned my addr 172.21.15.46:0/520258075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.995+0000 7f6c12800700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c11400700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 -- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 msgr2=0x561e83f460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 -- 172.21.15.46:0/520258075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 msgr2=0x561e83f45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f45b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 -- 172.21.15.46:0/520258075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14110810 con 0x561e83f42720 2024-05-08T23:18:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c11400700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.996+0000 7f6c12800700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f46600 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c14068d20 tx=0x7f6c0c007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54174d673b59b346 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6bf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00d950 con 0x561e83f42720 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6c0c00dab0 con 0x561e83f42720 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e83de5270 con 0x561e83f42720 2024-05-08T23:18:47.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0c00dc70 con 0x561e83f42720 2024-05-08T23:18:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6c2caebb80 10 client.29683 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.997+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e83cb9b60 con 0x561e83f42720 2024-05-08T23:18:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.998+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e83de5270 con 0x561e83f42720 2024-05-08T23:18:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c0c012070 con 0x561e83f42720 2024-05-08T23:18:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 10 client.29683 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 10 client.29683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 20 client.29683 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83f563a0 0x561e83f587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 4 client.29683 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:47.999+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e83f0b810 con 0x561e83f563a0 2024-05-08T23:18:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.000+0000 7f6c11e00700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83f563a0 0x561e83f587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.001+0000 7f6c11e00700 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83f563a0 0x561e83f587c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9677d336d23b8 server_cookie=9653294102289587 in_seq=0 out_seq=0 2024-05-08T23:18:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.001+0000 7f6bf7e00700 10 client.29683 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.001+0000 7f6bf7e00700 1 -- 172.21.15.46:0/520258075 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e83f0b810 con 0x561e83f563a0 2024-05-08T23:18:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.001+0000 7f6bf7e00700 10 client.29683 handle_command_reply: tid=0 2024-05-08T23:18:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 client.29683 shutdown 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 msgr2=0x561e83f46600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f46600 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c14068d20 tx=0x7f6c0c007dd0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 shutdown_connections 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f46600 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83f563a0 0x561e83f587c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f45b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/520258075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6bf7e00700 0 client.29683 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.002+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 >> 172.21.15.46:0/520258075 conn(0x561e83e9a210 msgr2=0x561e83f365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.003+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 shutdown_connections 2024-05-08T23:18:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.003+0000 7f6c2caebb80 1 -- 172.21.15.46:0/520258075 wait complete. 2024-05-08T23:18:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.003+0000 7f6c2caebb80 20 client.29683 trim_cache size 0 max 16384 2024-05-08T23:18:48.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:47 smithi046 ceph-mon[36898]: pgmap v468: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-05-08T23:18:48.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.033+0000 7f6c2caebb80 1 Processor -- start 2024-05-08T23:18:48.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.034+0000 7f6c2caebb80 1 -- start start 2024-05-08T23:18:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.035+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f33210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.035+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.035+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f40c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c11400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c11400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55388/0 (socket says 172.21.15.46:55388) 2024-05-08T23:18:48.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c11400700 1 -- 172.21.15.46:0/1255193171 learned_addr learned my addr 172.21.15.46:0/1255193171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:48.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83f387e0 con 0x561e83f3c890 2024-05-08T23:18:48.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83f46ed0 con 0x561e83f41e00 2024-05-08T23:18:48.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83f37ca0 con 0x561e83f42720 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.036+0000 7f6c12800700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f40c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11e00700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f33210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 -- 172.21.15.46:0/1255193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 msgr2=0x561e83f40c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f40c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 -- 172.21.15.46:0/1255193171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 msgr2=0x561e83f33210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f33210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 -- 172.21.15.46:0/1255193171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e83e52560 con 0x561e83f41e00 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c12800700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f40c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:48.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11e00700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f33210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:48.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.037+0000 7f6c11400700 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0997a0 tx=0x7f6bfc0991f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6b56776448f381 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.039+0000 7f6c03e00700 1 -- 172.21.15.46:0/1255193171 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0092a0 con 0x561e83f41e00 2024-05-08T23:18:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.040+0000 7f6c03e00700 1 -- 172.21.15.46:0/1255193171 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc099810 con 0x561e83f41e00 2024-05-08T23:18:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.040+0000 7f6c03e00700 1 -- 172.21.15.46:0/1255193171 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0172c0 con 0x561e83f41e00 2024-05-08T23:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 msgr2=0x561e83f36d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0997a0 tx=0x7f6bfc0991f0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 shutdown_connections 2024-05-08T23:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f42720 0x561e83f40c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f3c890 0x561e83f33210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/1255193171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f41e00 0x561e83f36d20 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0997a0 tx=0x7f6bfc0991f0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 >> 172.21.15.46:0/1255193171 conn(0x561e83e9a210 msgr2=0x561e83f45410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.043+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 shutdown_connections 2024-05-08T23:18:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.044+0000 7f6c2caebb80 1 -- 172.21.15.46:0/1255193171 wait complete. 2024-05-08T23:18:48.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.044+0000 7f6c2caebb80 1 -- start start 2024-05-08T23:18:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.045+0000 7f6c2caebb80 1 Processor -- start 2024-05-08T23:18:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.046+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.046+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f4b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c2caebb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f4b960 0x561e83fab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83fabe20 con 0x561e83f42720 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83fabf90 con 0x561e83f3c890 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c2caebb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e83fac150 con 0x561e83f4b960 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c2caebb80 10 client.0 _ll_register_callbacks cb 0x561e83e31b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c11e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55392/0 (socket says 172.21.15.46:55392) 2024-05-08T23:18:48.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c11e00700 1 -- 172.21.15.46:0/3891210152 learned_addr learned my addr 172.21.15.46:0/3891210152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.047+0000 7f6c12800700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f4b960 0x561e83fab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11e00700 1 -- 172.21.15.46:0/3891210152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f4b960 msgr2=0x561e83fab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11e00700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f4b960 0x561e83fab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11e00700 1 -- 172.21.15.46:0/3891210152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 msgr2=0x561e83f4b420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11400700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f4b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11e00700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f4b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.048+0000 7f6c11e00700 1 -- 172.21.15.46:0/3891210152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14198ef0 con 0x561e83f3c890 2024-05-08T23:18:48.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.049+0000 7f6c11e00700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6c0402e4d0 tx=0x7f6c04009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91a0b2c15422de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:48.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.049+0000 7f6bf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:48.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.050+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04032040 con 0x561e83f3c890 2024-05-08T23:18:48.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.050+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6c0403b030 con 0x561e83f3c890 2024-05-08T23:18:48.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.050+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c04005580 con 0x561e83f3c890 2024-05-08T23:18:48.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.051+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c04036a90 con 0x561e83f3c890 2024-05-08T23:18:48.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.052+0000 7f6c2caebb80 10 client.29630 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:48.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.052+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e83d11cd0 con 0x561e83f3c890 2024-05-08T23:18:48.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.052+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c04005580 con 0x561e83f3c890 2024-05-08T23:18:48.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c04035070 con 0x561e83f3c890 2024-05-08T23:18:48.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 10 client.29630 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 10 client.29630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 20 client.29630 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83fad4d0 0x561e83faf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 4 client.29630 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e83e84ec0 con 0x561e83fad4d0 2024-05-08T23:18:48.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.053+0000 7f6c11400700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83fad4d0 0x561e83faf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.054+0000 7f6c11400700 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83fad4d0 0x561e83faf8c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bea97039d29883 server_cookie=30edfd8ded8baa89 in_seq=0 out_seq=0 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.054+0000 7f6bf7e00700 10 client.29630 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.054+0000 7f6bf7e00700 1 -- 172.21.15.46:0/3891210152 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e83e84ec0 con 0x561e83fad4d0 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.054+0000 7f6bf7e00700 10 client.29630 handle_command_reply: tid=0 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.054+0000 7f6c2caebb80 1 client.29630 shutdown 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 msgr2=0x561e83f4aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6c0402e4d0 tx=0x7f6c04009950 comp rx=0 tx=0).stop 2024-05-08T23:18:48.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 shutdown_connections 2024-05-08T23:18:48.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e83f4b960 0x561e83fab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e83f3c890 0x561e83f4aee0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561e83fad4d0 0x561e83faf8c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 --2- 172.21.15.46:0/3891210152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e83f42720 0x561e83f4b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.055+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 >> 172.21.15.46:0/3891210152 conn(0x561e83e9a210 msgr2=0x561e83f46460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.056+0000 7f6bf7e00700 0 client.29630 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.057+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 shutdown_connections 2024-05-08T23:18:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.057+0000 7f6c2caebb80 1 -- 172.21.15.46:0/3891210152 wait complete. 2024-05-08T23:18:48.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.058+0000 7f6c2caebb80 20 client.29630 trim_cache size 0 max 16384 2024-05-08T23:18:48.058 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:18:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.059+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 msgr2=0x7f6bf80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.059+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 0x7f6bf80a38a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b5c0 tx=0x7f6c04005fd0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.059+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 msgr2=0x7f6c1410a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.059+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00b550 tx=0x7f6bfc00b910 comp rx=0 tx=0).stop 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 shutdown_connections 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1410d6d0 0x7f6c1410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6bf80a13e0 0x7f6bf80a38a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c14073370 0x7f6c1410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 --2- 172.21.15.46:0/927833756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c14073c90 0x7f6c1410a010 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.060+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 >> 172.21.15.46:0/927833756 conn(0x7f6c140fc960 msgr2=0x7f6c140fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.061+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 shutdown_connections 2024-05-08T23:18:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:48.061+0000 7f6c19000700 1 -- 172.21.15.46:0/927833756 wait complete. 2024-05-08T23:18:48.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:47 smithi086 ceph-mon[78610]: pgmap v468: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-05-08T23:18:48.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 29018 25.31 MB/sec warmup 69 sec latency 183.703 ms 2024-05-08T23:18:48.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:47 smithi053 ceph-mon[77587]: pgmap v468: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-05-08T23:18:49.096 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:18:49.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:18:49.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 29385 25.17 MB/sec warmup 70 sec latency 194.269 ms 2024-05-08T23:18:49.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 -- 172.21.15.46:0/1715448317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb41010a0 msgr2=0x7f9eb4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 --2- 172.21.15.46:0/1715448317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb41010a0 0x7f9eb4101480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00b0a0 tx=0x7f9eac00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 -- 172.21.15.46:0/1715448317 shutdown_connections 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 --2- 172.21.15.46:0/1715448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 --2- 172.21.15.46:0/1715448317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb41010a0 0x7f9eb4101480 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 --2- 172.21.15.46:0/1715448317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb4101a50 0x7f9eb4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.410+0000 7f9eba600700 1 -- 172.21.15.46:0/1715448317 >> 172.21.15.46:0/1715448317 conn(0x7f9eb40fc960 msgr2=0x7f9eb40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.411+0000 7f9eba600700 1 -- 172.21.15.46:0/1715448317 shutdown_connections 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.411+0000 7f9eba600700 1 -- 172.21.15.46:0/1715448317 wait complete. 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 Processor -- start 2024-05-08T23:18:49.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 -- start start 2024-05-08T23:18:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 0x7f9eb4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419e9f0 con 0x7f9eb41061c0 2024-05-08T23:18:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419eb60 con 0x7f9eb41010a0 2024-05-08T23:18:49.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.412+0000 7f9eba600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419ed20 con 0x7f9eb419df70 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 0x7f9eb4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33324/0 (socket says 172.21.15.46:33324) 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 -- 172.21.15.46:0/4265923101 learned_addr learned my addr 172.21.15.46:0/4265923101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 -- 172.21.15.46:0/4265923101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 msgr2=0x7f9eb4199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 0x7f9eb4199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 -- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 msgr2=0x7f9eb4199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb4199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.413+0000 7f9eb3e00700 1 -- 172.21.15.46:0/4265923101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac009d00 con 0x7f9eb419df70 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eb3400700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 0x7f9eb4199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e940 tx=0x7f9ea000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eabe00700 1 -- 172.21.15.46:0/4265923101 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cbd0 con 0x7f9eb419df70 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb4107c90 con 0x7f9eb419df70 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb41081f0 con 0x7f9eb419df70 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eabe00700 1 -- 172.21.15.46:0/4265923101 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea000ee90 con 0x7f9eb419df70 2024-05-08T23:18:49.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.414+0000 7f9eabe00700 1 -- 172.21.15.46:0/4265923101 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00055c0 con 0x7f9eb419df70 2024-05-08T23:18:49.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.419+0000 7f9eabe00700 1 -- 172.21.15.46:0/4265923101 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9ea000cd30 con 0x7f9eb419df70 2024-05-08T23:18:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.420+0000 7f9eabe00700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 0x7f9e980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.421+0000 7f9eabe00700 1 -- 172.21.15.46:0/4265923101 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9ea0014070 con 0x7f9eb419df70 2024-05-08T23:18:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.421+0000 7f9eb3400700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 0x7f9e980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.422+0000 7f9eb3400700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 0x7f9e980a37d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0062a0 tx=0x7f9eac006210 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.423+0000 7f9ece0fbb80 1 Processor -- start 2024-05-08T23:18:49.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.423+0000 7f9ece0fbb80 1 -- start start 2024-05-08T23:18:49.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a131260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a0821d0 con 0x555c7a130e80 2024-05-08T23:18:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a07fb90 con 0x555c7a12b910 2024-05-08T23:18:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a07b480 con 0x555c7a1317a0 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55414/0 (socket says 172.21.15.46:55414) 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2833858207 learned_addr learned my addr 172.21.15.46:0/2833858207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.424+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2833858207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 msgr2=0x555c7a12b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2833858207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 msgr2=0x555c7a131260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a131260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2833858207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c79efc430 con 0x555c7a12b910 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00964d0 tx=0x7f9ea0005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f9c9e0d6849263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9ea8c00700 1 -- 172.21.15.46:0/2833858207 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00adea0 con 0x555c7a12b910 2024-05-08T23:18:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9ea8c00700 1 -- 172.21.15.46:0/2833858207 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea009de00 con 0x555c7a12b910 2024-05-08T23:18:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.425+0000 7f9ea8c00700 1 -- 172.21.15.46:0/2833858207 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea009a230 con 0x555c7a12b910 2024-05-08T23:18:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2833858207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 msgr2=0x555c7a12bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00964d0 tx=0x7f9ea0005720 comp rx=0 tx=0).stop 2024-05-08T23:18:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2833858207 shutdown_connections 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a12bd50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a131260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2833858207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2833858207 >> 172.21.15.46:0/2833858207 conn(0x555c7a089290 msgr2=0x555c7a125e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2833858207 shutdown_connections 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2833858207 wait complete. 2024-05-08T23:18:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.427+0000 7f9ece0fbb80 1 -- start start 2024-05-08T23:18:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 Processor -- start 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a135110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a0175e0 con 0x555c7a130e80 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a017280 con 0x555c7a12b910 2024-05-08T23:18:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c79efe720 con 0x555c7a1317a0 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.429+0000 7f9ece0fbb80 10 client.0 _ll_register_callbacks cb 0x555c7a020bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.430+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.430+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33346/0 (socket says 172.21.15.46:33346) 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.430+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2622494867 learned_addr learned my addr 172.21.15.46:0/2622494867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.430+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a135110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.431+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 msgr2=0x555c7a134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.431+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.431+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2622494867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 msgr2=0x555c7a135110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.431+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a135110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.431+0000 7f9eb3e00700 1 -- 172.21.15.46:0/2622494867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb41081f0 con 0x555c7a1317a0 2024-05-08T23:18:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.432+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c5b0 tx=0x7f9ea0094ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108277da090b859a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.432+0000 7f9e9d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.432+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00adea0 con 0x555c7a1317a0 2024-05-08T23:18:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.432+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c79fd42f0 con 0x555c7a1317a0 2024-05-08T23:18:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.433+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9ea00a2450 con 0x555c7a1317a0 2024-05-08T23:18:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.435+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea00996d0 con 0x555c7a1317a0 2024-05-08T23:18:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.435+0000 7f9ece0fbb80 10 client.29701 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.435+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555c79ea8be0 con 0x555c7a1317a0 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.435+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c79fd42f0 con 0x555c7a1317a0 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.435+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ea00b80d0 con 0x555c7a1317a0 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 10 client.29701 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 10 client.29701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 20 client.29701 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a144e80 0x555c7a1472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 4 client.29701 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c7a0fa890 con 0x555c7a144e80 2024-05-08T23:18:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.436+0000 7f9eb3400700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a144e80 0x555c7a1472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.437+0000 7f9eb3400700 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a144e80 0x555c7a1472a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0d23478add3a59 server_cookie=e649b6aec9cdd84b in_seq=0 out_seq=0 2024-05-08T23:18:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.437+0000 7f9e9d600700 10 client.29701 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9e9d600700 1 -- 172.21.15.46:0/2622494867 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c7a0fa890 con 0x555c7a144e80 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9e9d600700 10 client.29701 handle_command_reply: tid=0 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 client.29701 shutdown 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 msgr2=0x555c7a135650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c5b0 tx=0x7f9ea0094ee0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 shutdown_connections 2024-05-08T23:18:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a135650 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a144e80 0x555c7a1472a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2622494867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a130e80 0x555c7a135110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 >> 172.21.15.46:0/2622494867 conn(0x555c7a089290 msgr2=0x555c7a125620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9e9d600700 0 client.29701 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.438+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 shutdown_connections 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.439+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2622494867 wait complete. 2024-05-08T23:18:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.439+0000 7f9ece0fbb80 20 client.29701 trim_cache size 0 max 16384 2024-05-08T23:18:49.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.466+0000 7f9ece0fbb80 1 Processor -- start 2024-05-08T23:18:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.467+0000 7f9ece0fbb80 1 -- start start 2024-05-08T23:18:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.467+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a12b910 0x555c7a12b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a1361d0 con 0x555c7a12b910 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a122e40 con 0x555c7a130e80 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a1349c0 con 0x555c7a1317a0 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55422/0 (socket says 172.21.15.46:55422) 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 -- 172.21.15.46:0/2504219150 learned_addr learned my addr 172.21.15.46:0/2504219150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb3400700 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a12b910 0x555c7a12b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 -- 172.21.15.46:0/2504219150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 msgr2=0x555c7a12fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 -- 172.21.15.46:0/2504219150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a12b910 msgr2=0x555c7a12b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.468+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a12b910 0x555c7a12b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9eb2a00700 1 -- 172.21.15.46:0/2504219150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c7a122560 con 0x555c7a130e80 2024-05-08T23:18:49.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9eb419abe0 tx=0x7f9ea4009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f9d11b5039fa3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9ea9600700 1 -- 172.21.15.46:0/2504219150 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4013ca0 con 0x555c7a130e80 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9ea9600700 1 -- 172.21.15.46:0/2504219150 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea40044e0 con 0x555c7a130e80 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.469+0000 7f9ea9600700 1 -- 172.21.15.46:0/2504219150 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4005280 con 0x555c7a130e80 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2504219150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 msgr2=0x555c7a125da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9eb419abe0 tx=0x7f9ea4009f20 comp rx=0 tx=0).stop 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2504219150 shutdown_connections 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a1317a0 0x555c7a12fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a12b910 0x555c7a12b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/2504219150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a130e80 0x555c7a125da0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2504219150 >> 172.21.15.46:0/2504219150 conn(0x555c7a089290 msgr2=0x555c7a1334a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2504219150 shutdown_connections 2024-05-08T23:18:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.470+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/2504219150 wait complete. 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.471+0000 7f9ece0fbb80 1 -- start start 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.471+0000 7f9ece0fbb80 1 Processor -- start 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a13ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a1317a0 0x555c7a13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a1ac3a0 con 0x555c7a1317a0 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a1ac4e0 con 0x555c7a12b910 2024-05-08T23:18:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a1ac6a0 con 0x555c7a130e80 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9ece0fbb80 10 client.0 _ll_register_callbacks cb 0x555c7a020bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9eb2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9eb2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33364/0 (socket says 172.21.15.46:33364) 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9eb2a00700 1 -- 172.21.15.46:0/748703887 learned_addr learned my addr 172.21.15.46:0/748703887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.472+0000 7f9eb3e00700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a1317a0 0x555c7a13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 -- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 msgr2=0x555c7a13ad50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a13ad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 -- 172.21.15.46:0/748703887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a1317a0 msgr2=0x555c7a13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a1317a0 0x555c7a13b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 -- 172.21.15.46:0/748703887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c79ea8be0 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9eb2a00700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4004d10 tx=0x7f9ea4004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf67505fd201c0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9e9d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.473+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4020040 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9ea4013aa0 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ea009de00 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea4013c60 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9ece0fbb80 10 client.29707 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f9ea4009920 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.474+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea009de00 con 0x555c7a130e80 2024-05-08T23:18:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ea4003880 con 0x555c7a130e80 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 10 client.29707 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 10 client.29707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 20 client.29707 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a1ade20 0x555c7a1b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 4 client.29707 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.475+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c7a073f40 con 0x555c7a1ade20 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.476+0000 7f9eb3400700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a1ade20 0x555c7a1b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.476+0000 7f9eb3400700 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a1ade20 0x555c7a1b0240 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f414537bf49c6258 server_cookie=43ea8bf6a76ce504 in_seq=0 out_seq=0 2024-05-08T23:18:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.476+0000 7f9e9d600700 10 client.29707 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.478+0000 7f9e9d600700 1 -- 172.21.15.46:0/748703887 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c7a073f40 con 0x555c7a1ade20 2024-05-08T23:18:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.478+0000 7f9e9d600700 10 client.29707 handle_command_reply: tid=0 2024-05-08T23:18:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.478+0000 7f9ece0fbb80 1 client.29707 shutdown 2024-05-08T23:18:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.479+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 msgr2=0x555c7a13b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.479+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4004d10 tx=0x7f9ea4004980 comp rx=0 tx=0).stop 2024-05-08T23:18:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 shutdown_connections 2024-05-08T23:18:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555c7a1ade20 0x555c7a1b0240 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c7a1317a0 0x555c7a13b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c7a12b910 0x555c7a13ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 --2- 172.21.15.46:0/748703887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7a130e80 0x555c7a13b290 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 >> 172.21.15.46:0/748703887 conn(0x555c7a089290 msgr2=0x555c7a12fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.480+0000 7f9e9d600700 0 client.29707 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.482+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 shutdown_connections 2024-05-08T23:18:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.483+0000 7f9ece0fbb80 1 -- 172.21.15.46:0/748703887 wait complete. 2024-05-08T23:18:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.483+0000 7f9ece0fbb80 20 client.29707 trim_cache size 0 max 16384 2024-05-08T23:18:49.484 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:18:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "476ec26d-c032-4f9f-a0e3-c9cc8d979dad", 2024-05-08T23:18:49.484 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:18:49.484 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 msgr2=0x7f9e980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 0x7f9e980a37d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0062a0 tx=0x7f9eac006210 comp rx=0 tx=0).stop 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 msgr2=0x7f9eb419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e940 tx=0x7f9ea000ec50 comp rx=0 tx=0).stop 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 shutdown_connections 2024-05-08T23:18:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb419df70 0x7f9eb419e3f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9eb41010a0 0x7f9eb4199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f9e980a1310 0x7f9e980a37d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 --2- 172.21.15.46:0/4265923101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eb41061c0 0x7f9eb4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.484+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 >> 172.21.15.46:0/4265923101 conn(0x7f9eb40fc960 msgr2=0x7f9eb40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.485+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 shutdown_connections 2024-05-08T23:18:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.485+0000 7f9eba600700 1 -- 172.21.15.46:0/4265923101 wait complete. 2024-05-08T23:18:49.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:18:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.791+0000 7ff017800700 1 -- 172.21.15.46:0/918439203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 msgr2=0x7ff0101023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.791+0000 7ff017800700 1 --2- 172.21.15.46:0/918439203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff0101023e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff004009990 tx=0x7ff004009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 -- 172.21.15.46:0/918439203 shutdown_connections 2024-05-08T23:18:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 --2- 172.21.15.46:0/918439203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01010b3d0 0x7ff01010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 --2- 172.21.15.46:0/918439203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff0101023e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 --2- 172.21.15.46:0/918439203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 0x7ff01010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 -- 172.21.15.46:0/918439203 >> 172.21.15.46:0/918439203 conn(0x7ff0100fb760 msgr2=0x7ff0100fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 -- 172.21.15.46:0/918439203 shutdown_connections 2024-05-08T23:18:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.793+0000 7ff017800700 1 -- 172.21.15.46:0/918439203 wait complete. 2024-05-08T23:18:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.795+0000 7ff017800700 1 Processor -- start 2024-05-08T23:18:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.796+0000 7ff017800700 1 -- start start 2024-05-08T23:18:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 0x7ff010199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01010b3d0 0x7ff01019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e4a0 con 0x7ff01010b3d0 2024-05-08T23:18:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e610 con 0x7ff010102920 2024-05-08T23:18:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff017800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e7d0 con 0x7ff010102000 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33394/0 (socket says 172.21.15.46:33394) 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.797+0000 7ff014e00700 1 -- 172.21.15.46:0/2884739069 learned_addr learned my addr 172.21.15.46:0/2884739069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff00fe00700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 0x7ff010199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 -- 172.21.15.46:0/2884739069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 msgr2=0x7ff010199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 0x7ff010199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 -- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01010b3d0 msgr2=0x7ff01019de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01010b3d0 0x7ff01019de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 -- 172.21.15.46:0/2884739069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0040095f0 con 0x7ff010102000 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.798+0000 7ff014e00700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c010 tx=0x7ff00400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.799+0000 7ff00d600700 1 -- 172.21.15.46:0/2884739069 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00401d070 con 0x7ff010102000 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.799+0000 7ff00d600700 1 -- 172.21.15.46:0/2884739069 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0040054f0 con 0x7ff010102000 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.800+0000 7ff00d600700 1 -- 172.21.15.46:0/2884739069 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004004bc0 con 0x7ff010102000 2024-05-08T23:18:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.800+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01019ea50 con 0x7ff010102000 2024-05-08T23:18:49.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.800+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01006c9d0 con 0x7ff010102000 2024-05-08T23:18:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.809+0000 7ff00d600700 1 -- 172.21.15.46:0/2884739069 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff004004d20 con 0x7ff010102000 2024-05-08T23:18:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.809+0000 7ff02b282b80 1 Processor -- start 2024-05-08T23:18:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.811+0000 7ff00d600700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 0x7feff80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.812+0000 7ff00d600700 1 -- 172.21.15.46:0/2884739069 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff0040cde00 con 0x7ff010102000 2024-05-08T23:18:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.812+0000 7ff015800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 0x7feff80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.813+0000 7ff02b282b80 1 -- start start 2024-05-08T23:18:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 0x558e33c201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 0x558e33c1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1a890 0x558e33c1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33b71150 con 0x558e33c1a890 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33b6eb10 con 0x558e33c1fe00 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.814+0000 7ff02b282b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33b6a400 con 0x558e33c20720 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff015800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1a890 0x558e33c1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 0x558e33c1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 0x558e33c1a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33404/0 (socket says 172.21.15.46:33404) 2024-05-08T23:18:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff00fe00700 1 -- 172.21.15.46:0/4157719729 learned_addr learned my addr 172.21.15.46:0/4157719729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 0x558e33c201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 -- 172.21.15.46:0/4157719729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 msgr2=0x558e33c1a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 0x558e33c1a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 -- 172.21.15.46:0/4157719729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1a890 msgr2=0x558e33c1acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1a890 0x558e33c1acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.815+0000 7ff014e00700 1 -- 172.21.15.46:0/4157719729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e339eb3b0 con 0x558e33c1fe00 2024-05-08T23:18:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.816+0000 7ff014e00700 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 0x558e33c201e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff0040cb0e0 tx=0x7ff004009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6367f67cf503a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.816+0000 7ff002000700 1 -- 172.21.15.46:0/4157719729 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00409ab30 con 0x558e33c1fe00 2024-05-08T23:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.817+0000 7ff002000700 1 -- 172.21.15.46:0/4157719729 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00409e960 con 0x558e33c1fe00 2024-05-08T23:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.817+0000 7ff002000700 1 -- 172.21.15.46:0/4157719729 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0040165e0 con 0x558e33c1fe00 2024-05-08T23:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.816+0000 7ff015800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 0x7feff80a3810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff008009fa0 tx=0x7ff008007890 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.817+0000 7ff02b282b80 1 -- 172.21.15.46:0/4157719729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 msgr2=0x558e33c201e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.817+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 0x558e33c201e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff0040cb0e0 tx=0x7ff004009310 comp rx=0 tx=0).stop 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.818+0000 7ff02b282b80 1 -- 172.21.15.46:0/4157719729 shutdown_connections 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.818+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1a890 0x558e33c1acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.818+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1fe00 0x558e33c201e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.818+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4157719729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c20720 0x558e33c1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.818+0000 7ff02b282b80 1 -- 172.21.15.46:0/4157719729 >> 172.21.15.46:0/4157719729 conn(0x558e33b78210 msgr2=0x558e33c14df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.819+0000 7ff02b282b80 1 -- 172.21.15.46:0/4157719729 shutdown_connections 2024-05-08T23:18:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.819+0000 7ff02b282b80 1 -- 172.21.15.46:0/4157719729 wait complete. 2024-05-08T23:18:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.820+0000 7ff02b282b80 1 -- start start 2024-05-08T23:18:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.820+0000 7ff02b282b80 1 Processor -- start 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1a890 0x558e33c23cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 0x558e33c24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33b06560 con 0x558e33c20720 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33b06200 con 0x558e33c1a890 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.823+0000 7ff02b282b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e339ed6a0 con 0x558e33c1fe00 2024-05-08T23:18:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff02b282b80 10 client.0 _ll_register_callbacks cb 0x558e33b0fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff015800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff015800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41026/0 (socket says 172.21.15.46:41026) 2024-05-08T23:18:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff015800700 1 -- 172.21.15.46:0/822161235 learned_addr learned my addr 172.21.15.46:0/822161235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff014e00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1a890 0x558e33c23cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.824+0000 7ff00fe00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 0x558e33c24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.825+0000 7ff00fe00700 1 -- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1a890 msgr2=0x558e33c23cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.826+0000 7ff00fe00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1a890 0x558e33c23cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.826+0000 7ff00fe00700 1 -- 172.21.15.46:0/822161235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 msgr2=0x558e33c24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.826+0000 7ff00fe00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.826+0000 7ff00fe00700 1 -- 172.21.15.46:0/822161235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01006c9d0 con 0x558e33c1fe00 2024-05-08T23:18:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.826+0000 7ff015800700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.827+0000 7ff00fe00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 0x558e33c24210 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x558e33c20b80 tx=0x7feffc010580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ff61b340d4835c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.827+0000 7feff6a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:18:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.828+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc010ea0 con 0x558e33c1fe00 2024-05-08T23:18:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.828+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feffc014430 con 0x558e33c1fe00 2024-05-08T23:18:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.828+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e33ac3270 con 0x558e33c1fe00 2024-05-08T23:18:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.829+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc0145f0 con 0x558e33c1fe00 2024-05-08T23:18:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.829+0000 7ff02b282b80 10 client.29719 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.829+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558e33997b60 con 0x558e33c1fe00 2024-05-08T23:18:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e33ac3270 con 0x558e33c1fe00 2024-05-08T23:18:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feffc01d980 con 0x558e33c1fe00 2024-05-08T23:18:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 10 client.29719 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 10 client.29719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 20 client.29719 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c33fa0 0x558e33c363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 4 client.29719 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:18:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.832+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e33be9810 con 0x558e33c33fa0 2024-05-08T23:18:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.833+0000 7ff014e00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c33fa0 0x558e33c363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.835+0000 7ff014e00700 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c33fa0 0x558e33c363c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c033b4e204a69b5 server_cookie=99d079a4a4850204 in_seq=0 out_seq=0 2024-05-08T23:18:49.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.835+0000 7feff6a00700 10 client.29719 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.836+0000 7feff6a00700 1 -- 172.21.15.46:0/822161235 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e33be9810 con 0x558e33c33fa0 2024-05-08T23:18:49.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.836+0000 7feff6a00700 10 client.29719 handle_command_reply: tid=0 2024-05-08T23:18:49.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.837+0000 7ff02b282b80 1 client.29719 shutdown 2024-05-08T23:18:49.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 msgr2=0x558e33c24210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 0x558e33c24210 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x558e33c20b80 tx=0x7feffc010580 comp rx=0 tx=0).stop 2024-05-08T23:18:49.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 shutdown_connections 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c33fa0 0x558e33c363c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c20720 0x558e33c24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c1a890 0x558e33c23cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 --2- 172.21.15.46:0/822161235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1fe00 0x558e33c24210 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.838+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 >> 172.21.15.46:0/822161235 conn(0x558e33b78210 msgr2=0x558e33c145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.839+0000 7feff6a00700 0 client.29719 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.839+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 shutdown_connections 2024-05-08T23:18:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.840+0000 7ff02b282b80 1 -- 172.21.15.46:0/822161235 wait complete. 2024-05-08T23:18:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.841+0000 7ff02b282b80 20 client.29719 trim_cache size 0 max 16384 2024-05-08T23:18:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.878+0000 7ff02b282b80 1 Processor -- start 2024-05-08T23:18:49.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 -- start start 2024-05-08T23:18:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c14d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c1dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c26ba0 con 0x558e33c1fe00 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c3b5b0 con 0x558e33c20720 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.881+0000 7ff02b282b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c16c40 con 0x558e33c1a890 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33420/0 (socket says 172.21.15.46:33420) 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 -- 172.21.15.46:0/2341647996 learned_addr learned my addr 172.21.15.46:0/2341647996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff015800700 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c1dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff00fe00700 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c14d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 -- 172.21.15.46:0/2341647996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 msgr2=0x558e33c1dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c1dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 -- 172.21.15.46:0/2341647996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 msgr2=0x558e33c14d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c14d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.882+0000 7ff014e00700 1 -- 172.21.15.46:0/2341647996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e33b30560 con 0x558e33c1a890 2024-05-08T23:18:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.883+0000 7ff014e00700 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff0040095c0 tx=0x7ff0040cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183eecedb56ea152 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.883+0000 7ff002a00700 1 -- 172.21.15.46:0/2341647996 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0040a2f20 con 0x558e33c1a890 2024-05-08T23:18:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.883+0000 7ff002a00700 1 -- 172.21.15.46:0/2341647996 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff004098ed0 con 0x558e33c1a890 2024-05-08T23:18:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.883+0000 7ff002a00700 1 -- 172.21.15.46:0/2341647996 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0040163a0 con 0x558e33c1a890 2024-05-08T23:18:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 -- 172.21.15.46:0/2341647996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 msgr2=0x558e33c1a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff0040095c0 tx=0x7ff0040cd040 comp rx=0 tx=0).stop 2024-05-08T23:18:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 -- 172.21.15.46:0/2341647996 shutdown_connections 2024-05-08T23:18:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c1dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c1a350 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 --2- 172.21.15.46:0/2341647996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c14d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.885+0000 7ff02b282b80 1 -- 172.21.15.46:0/2341647996 >> 172.21.15.46:0/2341647996 conn(0x558e33b78210 msgr2=0x558e33c231f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.886+0000 7ff02b282b80 1 -- 172.21.15.46:0/2341647996 shutdown_connections 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.886+0000 7ff02b282b80 1 -- 172.21.15.46:0/2341647996 wait complete. 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.887+0000 7ff02b282b80 1 -- start start 2024-05-08T23:18:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.887+0000 7ff02b282b80 1 Processor -- start 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c23ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c845b0 con 0x558e33c1fe00 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c84720 con 0x558e33c20720 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e33c848e0 con 0x558e33c1a890 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff02b282b80 10 client.0 _ll_register_callbacks cb 0x7feffc007950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:18:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff014e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:33428/0 (socket says 172.21.15.46:33428) 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff015800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff015800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55464/0 (socket says 172.21.15.46:55464) 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.889+0000 7ff015800700 1 -- 172.21.15.46:0/4292761504 learned_addr learned my addr 172.21.15.46:0/4292761504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.890+0000 7ff015800700 1 -- 172.21.15.46:0/4292761504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 msgr2=0x558e33c23960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.890+0000 7ff015800700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.890+0000 7ff015800700 1 -- 172.21.15.46:0/4292761504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 msgr2=0x558e33c23ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.890+0000 7ff00fe00700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c23ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.890+0000 7ff015800700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c23ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.891+0000 7ff015800700 1 -- 172.21.15.46:0/4292761504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0040b2040 con 0x558e33c20720 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.891+0000 7ff014e00700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.891+0000 7ff015800700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff008019e50 tx=0x7ff008004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816967e7d8a5cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.892+0000 7feff6a00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:18:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.892+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008003c00 con 0x558e33c20720 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.892+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff008003d60 con 0x558e33c20720 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.892+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff00409e960 con 0x558e33c20720 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.892+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff008004020 con 0x558e33c20720 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.893+0000 7ff02b282b80 10 client.29657 fetch_fsmap learned FSMap version 32 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.893+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558e339efcd0 con 0x558e33c20720 2024-05-08T23:18:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.893+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff008020040 con 0x558e33c20720 2024-05-08T23:18:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff008023070 con 0x558e33c20720 2024-05-08T23:18:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 10 client.29657 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:18:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 10 client.29657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:18:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 20 client.29657 populate_metadata read hostname 'smithi046' 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c92ca0 0x558e33c95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 4 client.29657 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.894+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7ff00409e960 con 0x558e33c92ca0 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.895+0000 7ff014e00700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c92ca0 0x558e33c95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.895+0000 7ff014e00700 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c92ca0 0x558e33c95090 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7556908e3700754c server_cookie=4f20e8d436a9210b in_seq=0 out_seq=0 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.895+0000 7feff6a00700 10 client.29657 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.896+0000 7feff6a00700 1 -- 172.21.15.46:0/4292761504 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7ff00409e960 con 0x558e33c92ca0 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.896+0000 7feff6a00700 10 client.29657 handle_command_reply: tid=0 2024-05-08T23:18:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 client.29657 shutdown 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 msgr2=0x558e33c243e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff008019e50 tx=0x7ff008004ad0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 shutdown_connections 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e33c20720 0x558e33c243e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e33c1a890 0x558e33c23960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558e33c92ca0 0x558e33c95090 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.897+0000 7ff02b282b80 1 --2- 172.21.15.46:0/4292761504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e33c1fe00 0x558e33c23ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.898+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 >> 172.21.15.46:0/4292761504 conn(0x558e33b78210 msgr2=0x558e33c22540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.898+0000 7feff6a00700 0 client.29657 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.898+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 shutdown_connections 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.898+0000 7ff02b282b80 1 -- 172.21.15.46:0/4292761504 wait complete. 2024-05-08T23:18:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.899+0000 7ff02b282b80 20 client.29657 trim_cache size 0 max 16384 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "476ec26d-c032-4f9f-a0e3-c9cc8d979dad": { 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "476ec26d-c032-4f9f-a0e3-c9cc8d979dad", 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:18:49.901 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 msgr2=0x7feff80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 0x7feff80a3810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff008009fa0 tx=0x7ff008007890 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 msgr2=0x7ff010199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c010 tx=0x7ff00400f740 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 shutdown_connections 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feff80a1350 0x7feff80a3810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01010b3d0 0x7ff01019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff010102000 0x7ff010199580 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 --2- 172.21.15.46:0/2884739069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff010102920 0x7ff010199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.900+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 >> 172.21.15.46:0/2884739069 conn(0x7ff0100fb760 msgr2=0x7ff0100fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.901+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 shutdown_connections 2024-05-08T23:18:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:18:49.901+0000 7ff017800700 1 -- 172.21.15.46:0/2884739069 wait complete. 2024-05-08T23:18:49.936 INFO:tasks.cephfs.filesystem:scrub status for tag:476ec26d-c032-4f9f-a0e3-c9cc8d979dad - {'path': '/', 'tag': '476ec26d-c032-4f9f-a0e3-c9cc8d979dad', 'options': 'recursive,force'} 2024-05-08T23:18:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: pgmap v469: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 485 op/s 2024-05-08T23:18:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:50.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:49 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:50.250 INFO:tasks.workunit.client.0.smithi046.stdout: 10 29826 25.10 MB/sec warmup 71 sec latency 220.513 ms 2024-05-08T23:18:50.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: pgmap v469: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 485 op/s 2024-05-08T23:18:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:49 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: pgmap v469: 129 pgs: 129 active+clean; 910 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 485 op/s 2024-05-08T23:18:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:50.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:50.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:50.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:49 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:18:51.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:50 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 30310 24.98 MB/sec warmup 72 sec latency 211.701 ms 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:51.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:18:51.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:50 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:18:51.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:51.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:50 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:18:52.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:51 smithi086 ceph-mon[78610]: pgmap v470: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 723 op/s 2024-05-08T23:18:52.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 30913 25.11 MB/sec warmup 73 sec latency 169.179 ms 2024-05-08T23:18:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:51 smithi053 ceph-mon[77587]: pgmap v470: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 723 op/s 2024-05-08T23:18:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:51 smithi046 ceph-mon[36898]: pgmap v470: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 723 op/s 2024-05-08T23:18:53.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 31476 25.33 MB/sec warmup 74 sec latency 152.016 ms 2024-05-08T23:18:54.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:53 smithi086 ceph-mon[78610]: pgmap v471: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 566 op/s 2024-05-08T23:18:54.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 31889 25.30 MB/sec warmup 75 sec latency 64.010 ms 2024-05-08T23:18:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:53 smithi053 ceph-mon[77587]: pgmap v471: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 566 op/s 2024-05-08T23:18:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:53 smithi046 ceph-mon[36898]: pgmap v471: 129 pgs: 129 active+clean; 873 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 566 op/s 2024-05-08T23:18:55.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 32305 25.19 MB/sec warmup 76 sec latency 180.908 ms 2024-05-08T23:18:56.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:55 smithi086 ceph-mon[78610]: pgmap v472: 129 pgs: 129 active+clean; 871 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 682 op/s 2024-05-08T23:18:56.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 32698 25.17 MB/sec warmup 77 sec latency 174.323 ms 2024-05-08T23:18:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:55 smithi053 ceph-mon[77587]: pgmap v472: 129 pgs: 129 active+clean; 871 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 682 op/s 2024-05-08T23:18:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:55 smithi046 ceph-mon[36898]: pgmap v472: 129 pgs: 129 active+clean; 871 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 682 op/s 2024-05-08T23:18:57.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 33215 25.26 MB/sec warmup 78 sec latency 71.641 ms 2024-05-08T23:18:58.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:57 smithi086 ceph-mon[78610]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 528 op/s 2024-05-08T23:18:58.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 33715 25.15 MB/sec warmup 79 sec latency 234.611 ms 2024-05-08T23:18:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:57 smithi053 ceph-mon[77587]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 528 op/s 2024-05-08T23:18:58.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:57 smithi046 ceph-mon[36898]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 528 op/s 2024-05-08T23:18:59.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 34246 25.16 MB/sec warmup 80 sec latency 204.892 ms 2024-05-08T23:19:00.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:18:59 smithi086 ceph-mon[78610]: pgmap v474: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-05-08T23:19:00.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 34800 25.33 MB/sec warmup 81 sec latency 204.764 ms 2024-05-08T23:19:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:18:59 smithi053 ceph-mon[77587]: pgmap v474: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-05-08T23:19:00.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:18:59 smithi046 ceph-mon[36898]: pgmap v474: 129 pgs: 129 active+clean; 875 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.4 MiB/s wr, 389 op/s 2024-05-08T23:19:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:01.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:01.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 34917 25.06 MB/sec warmup 82 sec latency 858.611 ms 2024-05-08T23:19:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:01.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:02.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:01 smithi086 ceph-mon[78610]: pgmap v475: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-05-08T23:19:02.251 INFO:tasks.workunit.client.0.smithi046.stdout: 10 34917 24.75 MB/sec warmup 83 sec latency 1858.740 ms 2024-05-08T23:19:02.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:01 smithi053 ceph-mon[77587]: pgmap v475: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-05-08T23:19:02.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:01 smithi046 ceph-mon[36898]: pgmap v475: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-05-08T23:19:03.253 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35110 24.55 MB/sec warmup 84 sec latency 2534.827 ms 2024-05-08T23:19:04.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35251 24.31 MB/sec warmup 85 sec latency 354.552 ms 2024-05-08T23:19:04.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:03 smithi053 ceph-mon[77587]: pgmap v476: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-05-08T23:19:04.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:03 smithi046 ceph-mon[36898]: pgmap v476: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-05-08T23:19:04.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:03 smithi086 ceph-mon[78610]: pgmap v476: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-05-08T23:19:05.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35284 24.07 MB/sec warmup 86 sec latency 280.919 ms 2024-05-08T23:19:06.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35384 23.91 MB/sec warmup 87 sec latency 228.517 ms 2024-05-08T23:19:06.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:05 smithi053 ceph-mon[77587]: pgmap v477: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-08T23:19:06.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:05 smithi046 ceph-mon[36898]: pgmap v477: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-08T23:19:06.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:05 smithi086 ceph-mon[78610]: pgmap v477: 129 pgs: 129 active+clean; 883 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-05-08T23:19:07.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35767 23.89 MB/sec warmup 88 sec latency 109.974 ms 2024-05-08T23:19:08.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 36164 23.82 MB/sec warmup 89 sec latency 103.086 ms 2024-05-08T23:19:08.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:07 smithi053 ceph-mon[77587]: pgmap v478: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-08T23:19:08.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:07 smithi046 ceph-mon[36898]: pgmap v478: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-08T23:19:08.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:07 smithi086 ceph-mon[78610]: pgmap v478: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-08T23:19:09.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 36690 23.90 MB/sec warmup 90 sec latency 160.529 ms 2024-05-08T23:19:10.252 INFO:tasks.workunit.client.0.smithi046.stdout: 10 37099 23.92 MB/sec warmup 91 sec latency 196.449 ms 2024-05-08T23:19:10.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:09 smithi053 ceph-mon[77587]: pgmap v479: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2024-05-08T23:19:10.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:09 smithi046 ceph-mon[36898]: pgmap v479: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2024-05-08T23:19:10.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:09 smithi086 ceph-mon[78610]: pgmap v479: 129 pgs: 129 active+clean; 880 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2024-05-08T23:19:11.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 37583 23.93 MB/sec warmup 92 sec latency 155.354 ms 2024-05-08T23:19:12.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 37927 23.84 MB/sec warmup 93 sec latency 195.914 ms 2024-05-08T23:19:12.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:12 smithi053 ceph-mon[77587]: pgmap v480: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-05-08T23:19:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:12 smithi053 ceph-mon[77587]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:19:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:12 smithi046 ceph-mon[36898]: pgmap v480: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-05-08T23:19:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:12 smithi046 ceph-mon[36898]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:19:12.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:12 smithi086 ceph-mon[78610]: pgmap v480: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-05-08T23:19:12.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:12 smithi086 ceph-mon[78610]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T23:19:13.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 37988 23.61 MB/sec warmup 94 sec latency 739.374 ms 2024-05-08T23:19:13.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:13 smithi053 ceph-mon[77587]: pgmap v482: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 420 op/s 2024-05-08T23:19:13.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:13 smithi046 ceph-mon[36898]: pgmap v482: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 420 op/s 2024-05-08T23:19:13.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:13 smithi086 ceph-mon[78610]: pgmap v482: 129 pgs: 129 active+clean; 887 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 420 op/s 2024-05-08T23:19:14.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 38226 23.46 MB/sec warmup 95 sec latency 348.329 ms 2024-05-08T23:19:14.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:14 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:14.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:14 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:14.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:14 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:15.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 38616 23.52 MB/sec warmup 96 sec latency 69.703 ms 2024-05-08T23:19:15.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:15 smithi086 ceph-mon[78610]: pgmap v483: 129 pgs: 129 active+clean; 907 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-05-08T23:19:15.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:15 smithi053 ceph-mon[77587]: pgmap v483: 129 pgs: 129 active+clean; 907 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-05-08T23:19:15.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:15 smithi046 ceph-mon[36898]: pgmap v483: 129 pgs: 129 active+clean; 907 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-05-08T23:19:16.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 39133 23.55 MB/sec warmup 97 sec latency 211.479 ms 2024-05-08T23:19:16.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:16 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:16.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:16 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:16.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:16 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:19:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:16 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:16 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:16.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:16 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:19:16.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:16 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:16.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:16 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:16.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:16 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:19:17.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 39588 23.56 MB/sec warmup 98 sec latency 155.928 ms 2024-05-08T23:19:17.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:17 smithi086 ceph-mon[78610]: pgmap v484: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:17.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:17 smithi053 ceph-mon[77587]: pgmap v484: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:17.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:17 smithi046 ceph-mon[36898]: pgmap v484: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:18.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 39756 23.39 MB/sec warmup 99 sec latency 431.474 ms 2024-05-08T23:19:19.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 40002 23.26 MB/sec warmup 100 sec latency 533.582 ms 2024-05-08T23:19:19.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:19 smithi086 ceph-mon[78610]: pgmap v485: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:19.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:19 smithi053 ceph-mon[77587]: pgmap v485: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:19.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:19 smithi046 ceph-mon[36898]: pgmap v485: 129 pgs: 129 active+clean; 917 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-08T23:19:19.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:19:20.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 40329 23.28 MB/sec warmup 101 sec latency 219.928 ms 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.284+0000 7f1ae7600700 1 -- 172.21.15.46:0/3069519867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae0105060 msgr2=0x7f1ae010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.284+0000 7f1ae7600700 1 --2- 172.21.15.46:0/3069519867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae0105060 0x7f1ae010cbe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009990 tx=0x7f1ac8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.285+0000 7f1ae7600700 1 -- 172.21.15.46:0/3069519867 shutdown_connections 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.285+0000 7f1ae7600700 1 --2- 172.21.15.46:0/3069519867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae0105060 0x7f1ae010cbe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.285+0000 7f1ae7600700 1 --2- 172.21.15.46:0/3069519867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae00fff40 0x7f1ae0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.285+0000 7f1ae7600700 1 --2- 172.21.15.46:0/3069519867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01008f0 0x7f1ae0104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.285+0000 7f1ae7600700 1 -- 172.21.15.46:0/3069519867 >> 172.21.15.46:0/3069519867 conn(0x7f1ae00fb760 msgr2=0x7f1ae00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.286+0000 7f1ae7600700 1 -- 172.21.15.46:0/3069519867 shutdown_connections 2024-05-08T23:19:20.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.286+0000 7f1ae7600700 1 -- 172.21.15.46:0/3069519867 wait complete. 2024-05-08T23:19:20.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.289+0000 7f1ae7600700 1 Processor -- start 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.289+0000 7f1ae7600700 1 -- start start 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae00fff40 0x7f1ae0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 0x7f1ae0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019c660 con 0x7f1ae00fff40 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019c7d0 con 0x7f1ae019bbe0 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae7600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019c990 con 0x7f1ae0105060 2024-05-08T23:19:20.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae5600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1adfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 0x7f1ae0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae5600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46308/0 (socket says 172.21.15.46:46308) 2024-05-08T23:19:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.290+0000 7f1ae5600700 1 -- 172.21.15.46:0/1764747463 learned_addr learned my addr 172.21.15.46:0/1764747463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae00fff40 0x7f1ae0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae5600700 1 -- 172.21.15.46:0/1764747463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 msgr2=0x7f1ae0197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae5600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 0x7f1ae0197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae5600700 1 -- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae00fff40 msgr2=0x7f1ae0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae5600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae00fff40 0x7f1ae0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1ae5600700 1 -- 172.21.15.46:0/1764747463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80095f0 con 0x7f1ae019bbe0 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.291+0000 7f1adfe00700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 0x7f1ae0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.292+0000 7f1ae5600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005910 tx=0x7f1ac800f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.293+0000 7f1add600700 1 -- 172.21.15.46:0/1764747463 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8005230 con 0x7f1ae019bbe0 2024-05-08T23:19:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.293+0000 7f1add600700 1 -- 172.21.15.46:0/1764747463 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac8023070 con 0x7f1ae019bbe0 2024-05-08T23:19:20.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.293+0000 7f1add600700 1 -- 172.21.15.46:0/1764747463 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800f9a0 con 0x7f1ae019bbe0 2024-05-08T23:19:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.294+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae010ed00 con 0x7f1ae019bbe0 2024-05-08T23:19:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.294+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae010f130 con 0x7f1ae019bbe0 2024-05-08T23:19:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.300+0000 7f1add600700 1 -- 172.21.15.46:0/1764747463 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1ac8025070 con 0x7f1ae019bbe0 2024-05-08T23:19:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.303+0000 7f1add600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 0x7f1ac40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.303+0000 7f1add600700 1 -- 172.21.15.46:0/1764747463 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ac80d16b0 con 0x7f1ae019bbe0 2024-05-08T23:19:20.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.304+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 0x7f1ac40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.308+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 0x7f1ac40a3960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e270 tx=0x7f1ad8009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.312+0000 7f1afb1f1b80 1 Processor -- start 2024-05-08T23:19:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 -- start start 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035223150 con 0x55f0352cc890 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035220b10 con 0x55f0352d1e00 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.313+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f03521c400 con 0x55f0352d2720 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.314+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.314+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46310/0 (socket says 172.21.15.46:46310) 2024-05-08T23:19:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.314+0000 7f1ae4c00700 1 -- 172.21.15.46:0/1683480159 learned_addr learned my addr 172.21.15.46:0/1683480159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.314+0000 7f1ae5600700 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.316+0000 7f1adfe00700 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 -- 172.21.15.46:0/1683480159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 msgr2=0x55f0352cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 -- 172.21.15.46:0/1683480159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 msgr2=0x55f0352cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 -- 172.21.15.46:0/1683480159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f03509d3b0 con 0x55f0352d1e00 2024-05-08T23:19:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.318+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800efb0 tx=0x7f1ad8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88385c9d37b173a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.319+0000 7f1ad5c00700 1 -- 172.21.15.46:0/1683480159 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8015db0 con 0x55f0352d1e00 2024-05-08T23:19:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.319+0000 7f1ad5c00700 1 -- 172.21.15.46:0/1683480159 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad80208c0 con 0x55f0352d1e00 2024-05-08T23:19:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.319+0000 7f1ad5c00700 1 -- 172.21.15.46:0/1683480159 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8013910 con 0x55f0352d1e00 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/1683480159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 msgr2=0x55f0352d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800efb0 tx=0x7f1ad8011000 comp rx=0 tx=0).stop 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/1683480159 shutdown_connections 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d21e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/1683480159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.322+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/1683480159 >> 172.21.15.46:0/1683480159 conn(0x55f03522a210 msgr2=0x55f0352c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.324+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/1683480159 shutdown_connections 2024-05-08T23:19:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.324+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/1683480159 wait complete. 2024-05-08T23:19:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.325+0000 7f1afb1f1b80 1 -- start start 2024-05-08T23:19:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.325+0000 7f1afb1f1b80 1 Processor -- start 2024-05-08T23:19:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0351b8560 con 0x55f0352cc890 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0351b8200 con 0x55f0352d1e00 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f03509f6a0 con 0x55f0352d2720 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.326+0000 7f1afb1f1b80 10 client.0 _ll_register_callbacks cb 0x55f0351c1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.327+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.327+0000 7f1adfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.327+0000 7f1ae5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.327+0000 7f1ae5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39716/0 (socket says 172.21.15.46:39716) 2024-05-08T23:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.327+0000 7f1ae5600700 1 -- 172.21.15.46:0/3566763132 learned_addr learned my addr 172.21.15.46:0/3566763132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.328+0000 7f1ae5600700 1 -- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 msgr2=0x55f0352d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.328+0000 7f1ae5600700 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.328+0000 7f1ae5600700 1 -- 172.21.15.46:0/3566763132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 msgr2=0x55f0352d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.328+0000 7f1ae5600700 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.328+0000 7f1ae5600700 1 -- 172.21.15.46:0/3566763132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad800f040 con 0x55f0352d2720 2024-05-08T23:19:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.329+0000 7f1ae5600700 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005000 tx=0x7f1ac80ceef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d245fcf699050a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.329+0000 7f1ace000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.329+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80a2170 con 0x55f0352d2720 2024-05-08T23:19:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.329+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ac809a040 con 0x55f0352d2720 2024-05-08T23:19:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.330+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f035175270 con 0x55f0352d2720 2024-05-08T23:19:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.330+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac80b9030 con 0x55f0352d2720 2024-05-08T23:19:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.330+0000 7f1afb1f1b80 10 client.29734 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.330+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f035049b60 con 0x55f0352d2720 2024-05-08T23:19:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.331+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f035175270 con 0x55f0352d2720 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.331+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1ac80bc020 con 0x55f0352d2720 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.331+0000 7f1afb1f1b80 10 client.29734 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.332+0000 7f1afb1f1b80 10 client.29734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.332+0000 7f1afb1f1b80 20 client.29734 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.333+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f0352f6de0 0x55f0352f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.333+0000 7f1afb1f1b80 4 client.29734 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.333+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f03529b810 con 0x55f0352f6de0 2024-05-08T23:19:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.333+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f0352f6de0 0x55f0352f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.334+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f0352f6de0 0x55f0352f9200 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ab980b9e687db6 server_cookie=1c195fed1b6ca1a1 in_seq=0 out_seq=0 2024-05-08T23:19:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.334+0000 7f1ace000700 10 client.29734 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1ace000700 1 -- 172.21.15.46:0/3566763132 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f03529b810 con 0x55f0352f6de0 2024-05-08T23:19:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1ace000700 10 client.29734 handle_command_reply: tid=0 2024-05-08T23:19:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1afb1f1b80 1 client.29734 shutdown 2024-05-08T23:19:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 msgr2=0x55f0352d6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005000 tx=0x7f1ac80ceef0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.335+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 shutdown_connections 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352d6630 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f0352f6de0 0x55f0352f9200 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3566763132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 >> 172.21.15.46:0/3566763132 conn(0x55f03522a210 msgr2=0x55f0352c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.336+0000 7f1ace000700 0 client.29734 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:20.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.338+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 shutdown_connections 2024-05-08T23:19:20.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.338+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3566763132 wait complete. 2024-05-08T23:19:20.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.338+0000 7f1afb1f1b80 20 client.29734 trim_cache size 0 max 16384 2024-05-08T23:19:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.373+0000 7f1afb1f1b80 1 Processor -- start 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.373+0000 7f1afb1f1b80 1 -- start start 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.373+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.373+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 0x55f0352d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352d8720 con 0x55f0352d2720 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352c3dc0 con 0x55f0352d1e00 2024-05-08T23:19:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352c8e50 con 0x55f0352cc890 2024-05-08T23:19:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1adfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39720/0 (socket says 172.21.15.46:39720) 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1ae4c00700 1 -- 172.21.15.46:0/604323089 learned_addr learned my addr 172.21.15.46:0/604323089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.374+0000 7f1ae5600700 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 0x55f0352d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 -- 172.21.15.46:0/604323089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 msgr2=0x55f0352c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 -- 172.21.15.46:0/604323089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 msgr2=0x55f0352d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 0x55f0352d44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 -- 172.21.15.46:0/604323089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0351e2560 con 0x55f0352cc890 2024-05-08T23:19:20.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae5600700 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 0x55f0352d44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:20.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.375+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80159f0 tx=0x7f1ad8013480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a3e10849871e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.376+0000 7f1ad6600700 1 -- 172.21.15.46:0/604323089 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8020400 con 0x55f0352cc890 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.376+0000 7f1ad6600700 1 -- 172.21.15.46:0/604323089 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad8020700 con 0x55f0352cc890 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.376+0000 7f1ad6600700 1 -- 172.21.15.46:0/604323089 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80312a0 con 0x55f0352cc890 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/604323089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 msgr2=0x55f0352cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80159f0 tx=0x7f1ad8013480 comp rx=0 tx=0).stop 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/604323089 shutdown_connections 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352d2720 0x55f0352d44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352cc890 0x55f0352cc350 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/604323089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/604323089 >> 172.21.15.46:0/604323089 conn(0x55f03522a210 msgr2=0x55f0352fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/604323089 shutdown_connections 2024-05-08T23:19:20.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.377+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/604323089 wait complete. 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 -- start start 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 Processor -- start 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352ed270 con 0x55f0352cc890 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352ed3b0 con 0x55f0352d1e00 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.378+0000 7f1afb1f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0352ed540 con 0x55f0352d2720 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.379+0000 7f1afb1f1b80 10 client.0 _ll_register_callbacks cb 0x55f0351c1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.379+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.379+0000 7f1ae4c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50852/0 (socket says 172.21.15.46:50852) 2024-05-08T23:19:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.379+0000 7f1ae4c00700 1 -- 172.21.15.46:0/3330475598 learned_addr learned my addr 172.21.15.46:0/3330475598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.379+0000 7f1ae5600700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.380+0000 7f1ae4c00700 1 -- 172.21.15.46:0/3330475598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 msgr2=0x55f0352ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.380+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.380+0000 7f1ae4c00700 1 -- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 msgr2=0x55f0352ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.380+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.380+0000 7f1ae4c00700 1 -- 172.21.15.46:0/3330475598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f035049b60 con 0x55f0352cc890 2024-05-08T23:19:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.382+0000 7f1ae4c00700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8020890 tx=0x7f1ad8007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1109f5310583965 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.382+0000 7f1ace000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:19:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.382+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8020d80 con 0x55f0352cc890 2024-05-08T23:19:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.382+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1ad8004a90 con 0x55f0352cc890 2024-05-08T23:19:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.383+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ad80208c0 con 0x55f0352cc890 2024-05-08T23:19:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.384+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad8004c50 con 0x55f0352cc890 2024-05-08T23:19:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.384+0000 7f1afb1f1b80 10 client.19578 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.384+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f03509d3b0 con 0x55f0352cc890 2024-05-08T23:19:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.385+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad80208c0 con 0x55f0352cc890 2024-05-08T23:19:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1ad80134f0 con 0x55f0352cc890 2024-05-08T23:19:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 10 client.19578 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 20 client.19578 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f035356d30 0x55f035359120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 4 client.19578 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f035214ec0 con 0x55f035356d30 2024-05-08T23:19:20.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.386+0000 7f1adfe00700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f035356d30 0x55f035359120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.388+0000 7f1adfe00700 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f035356d30 0x55f035359120 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3883c4fbc3e62d server_cookie=65392e91855fa7e in_seq=0 out_seq=0 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.388+0000 7f1ace000700 10 client.19578 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.388+0000 7f1ace000700 1 -- 172.21.15.46:0/3330475598 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f035214ec0 con 0x55f035356d30 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.388+0000 7f1ace000700 10 client.19578 handle_command_reply: tid=0 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.388+0000 7f1afb1f1b80 1 client.19578 shutdown 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 msgr2=0x55f0352ee370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8020890 tx=0x7f1ad8007cf0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 shutdown_connections 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0352d2720 0x55f0352ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f035356d30 0x55f035359120 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0352cc890 0x55f0352ee370 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 --2- 172.21.15.46:0/3330475598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0352d1e00 0x55f0352ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 >> 172.21.15.46:0/3330475598 conn(0x55f03522a210 msgr2=0x55f0352d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.389+0000 7f1ace000700 0 client.19578 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:20.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.390+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 shutdown_connections 2024-05-08T23:19:20.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.391+0000 7f1afb1f1b80 1 -- 172.21.15.46:0/3330475598 wait complete. 2024-05-08T23:19:20.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.391+0000 7f1afb1f1b80 20 client.19578 trim_cache size 0 max 16384 2024-05-08T23:19:20.392 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:19:20.392 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:19:20.392 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:19:20.392 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:19:20.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 msgr2=0x7f1ac40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 0x7f1ac40a3960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e270 tx=0x7f1ad8009040 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 msgr2=0x7f1ae019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005910 tx=0x7f1ac800f510 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 shutdown_connections 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ae019bbe0 0x7f1ae019c060 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1ac40a14a0 0x7f1ac40a3960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae00fff40 0x7f1ae0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 --2- 172.21.15.46:0/1764747463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0105060 0x7f1ae0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 >> 172.21.15.46:0/1764747463 conn(0x7f1ae00fb760 msgr2=0x7f1ae00fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.394+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 shutdown_connections 2024-05-08T23:19:20.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.395+0000 7f1ae7600700 1 -- 172.21.15.46:0/1764747463 wait complete. 2024-05-08T23:19:20.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:19:20.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 -- 172.21.15.46:0/980556736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101a20 msgr2=0x7f4260105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 --2- 172.21.15.46:0/980556736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101a20 0x7f4260105a60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4248009990 tx=0x7f4248009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 -- 172.21.15.46:0/980556736 shutdown_connections 2024-05-08T23:19:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 --2- 172.21.15.46:0/980556736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106190 0x7f426010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 --2- 172.21.15.46:0/980556736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260101070 0x7f4260101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 --2- 172.21.15.46:0/980556736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101a20 0x7f4260105a60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 -- 172.21.15.46:0/980556736 >> 172.21.15.46:0/980556736 conn(0x7f42600fc8d0 msgr2=0x7f42600fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 -- 172.21.15.46:0/980556736 shutdown_connections 2024-05-08T23:19:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.769+0000 7f4266400700 1 -- 172.21.15.46:0/980556736 wait complete. 2024-05-08T23:19:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.771+0000 7f4266400700 1 Processor -- start 2024-05-08T23:19:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.771+0000 7f4266400700 1 -- start start 2024-05-08T23:19:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.772+0000 7f4266400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101070 0x7f42601994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.772+0000 7f4266400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.772+0000 7f4266400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 0x7f426019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.772+0000 7f425ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39754/0 (socket says 172.21.15.46:39754) 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 0x7f426019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 -- 172.21.15.46:0/1460685780 learned_addr learned my addr 172.21.15.46:0/1460685780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426019e850 con 0x7f426019ddd0 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426019e9c0 con 0x7f4260101070 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425f400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101070 0x7f42601994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426019eb80 con 0x7f4260106190 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 -- 172.21.15.46:0/1460685780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101070 msgr2=0x7f42601994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101070 0x7f42601994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 -- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 msgr2=0x7f426019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 0x7f426019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425ea00700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480095f0 con 0x7f4260106190 2024-05-08T23:19:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.773+0000 7f425fe00700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 0x7f426019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f425ea00700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4248000c00 tx=0x7f4248005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424801d070 con 0x7f4260106190 2024-05-08T23:19:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4248022470 con 0x7f4260106190 2024-05-08T23:19:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248017650 con 0x7f4260106190 2024-05-08T23:19:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42601a6080 con 0x7f4260106190 2024-05-08T23:19:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.774+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42601a64b0 con 0x7f4260106190 2024-05-08T23:19:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.776+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c0052f0 con 0x7f4260106190 2024-05-08T23:19:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.781+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f424800fad0 con 0x7f4260106190 2024-05-08T23:19:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.783+0000 7f4257e00700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 0x7f42440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.783+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f42480ce340 con 0x7f4260106190 2024-05-08T23:19:20.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.784+0000 7f425f400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 0x7f42440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.786+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4248097410 con 0x7f4260106190 2024-05-08T23:19:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:20.787+0000 7f425f400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 0x7f42440a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4250009c40 tx=0x7f4250009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:20 smithi086 ceph-mon[78610]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:19:21.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 40771 23.24 MB/sec warmup 102 sec latency 142.832 ms 2024-05-08T23:19:21.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:20 smithi053 ceph-mon[77587]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:19:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.278+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f423c005160 con 0x7f4260106190 2024-05-08T23:19:21.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:20 smithi046 ceph-mon[36898]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T23:19:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.279+0000 7f4257e00700 1 -- 172.21.15.46:0/1460685780 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10910 (secure 0 0 0) 0x7f4248027090 con 0x7f4260106190 2024-05-08T23:19:21.280 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:19:21.280 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:17:42.880411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 msgr2=0x7f42440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 0x7f42440a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4250009c40 tx=0x7f4250009260 comp rx=0 tx=0).stop 2024-05-08T23:19:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 msgr2=0x7f4260199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4248000c00 tx=0x7f4248005740 comp rx=0 tx=0).stop 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 shutdown_connections 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42440a1310 0x7f42440a37d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f426019ddd0 0x7f426019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4260101070 0x7f42601994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 --2- 172.21.15.46:0/1460685780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4260106190 0x7f4260199a30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 >> 172.21.15.46:0/1460685780 conn(0x7f42600fc8d0 msgr2=0x7f42600fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 shutdown_connections 2024-05-08T23:19:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.284+0000 7f4266400700 1 -- 172.21.15.46:0/1460685780 wait complete. 2024-05-08T23:19:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-05-08T23:19:21.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:19:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.594+0000 7fb237e00700 1 -- 172.21.15.46:0/824125825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073350 msgr2=0x7fb238073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.594+0000 7fb237e00700 1 --2- 172.21.15.46:0/824125825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073350 0x7fb238073730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb228009990 tx=0x7fb228009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 -- 172.21.15.46:0/824125825 shutdown_connections 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 --2- 172.21.15.46:0/824125825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb23810d520 0x7fb23810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 --2- 172.21.15.46:0/824125825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073350 0x7fb238073730 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 --2- 172.21.15.46:0/824125825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238073c70 0x7fb23810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 -- 172.21.15.46:0/824125825 >> 172.21.15.46:0/824125825 conn(0x7fb2380fc770 msgr2=0x7fb2380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 -- 172.21.15.46:0/824125825 shutdown_connections 2024-05-08T23:19:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.595+0000 7fb237e00700 1 -- 172.21.15.46:0/824125825 wait complete. 2024-05-08T23:19:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.596+0000 7fb237e00700 1 Processor -- start 2024-05-08T23:19:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.596+0000 7fb237e00700 1 -- start start 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 0x7fb238100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 0x7fb2381011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23810d520 0x7fb2381016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2381a98b0 con 0x7fb238073350 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2381a99f0 con 0x7fb238073c70 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb237e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2381a9b60 con 0x7fb23810d520 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb236a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 0x7fb238100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb236000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 0x7fb2381011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb236a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 0x7fb238100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50874/0 (socket says 172.21.15.46:50874) 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.597+0000 7fb236a00700 1 -- 172.21.15.46:0/1563206338 learned_addr learned my addr 172.21.15.46:0/1563206338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb237400700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23810d520 0x7fb2381016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 -- 172.21.15.46:0/1563206338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23810d520 msgr2=0x7fb2381016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23810d520 0x7fb2381016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 -- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 msgr2=0x7fb238100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 0x7fb238100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 -- 172.21.15.46:0/1563206338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224009520 con 0x7fb238073c70 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb236000700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 0x7fb2381011a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb22400ea20 tx=0x7fb22400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.598+0000 7fb22f400700 1 -- 172.21.15.46:0/1563206338 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224004220 con 0x7fb238073c70 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.599+0000 7fb22f400700 1 -- 172.21.15.46:0/1563206338 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22400eec0 con 0x7fb238073c70 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.599+0000 7fb22f400700 1 -- 172.21.15.46:0/1563206338 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224003cf0 con 0x7fb238073c70 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.599+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2280095f0 con 0x7fb238073c70 2024-05-08T23:19:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.599+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2381aa170 con 0x7fb238073c70 2024-05-08T23:19:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.603+0000 7fb22f400700 1 -- 172.21.15.46:0/1563206338 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb22400c970 con 0x7fb238073c70 2024-05-08T23:19:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb22f400700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 0x7fb21c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb2516c7b80 1 Processor -- start 2024-05-08T23:19:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb22f400700 1 -- 172.21.15.46:0/1563206338 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb224014070 con 0x7fb238073c70 2024-05-08T23:19:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb2516c7b80 1 -- start start 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb236a00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 0x7fb21c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.605+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db6ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57dac3150 con 0x55b57db6c890 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57dac0b10 con 0x55b57db72970 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57dabc400 con 0x55b57db72050 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39782/0 (socket says 172.21.15.46:39782) 2024-05-08T23:19:21.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236a00700 1 -- 172.21.15.46:0/4043141603 learned_addr learned my addr 172.21.15.46:0/4043141603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb237400700 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db6ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236000700 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236a00700 1 -- 172.21.15.46:0/4043141603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 msgr2=0x55b57db6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.606+0000 7fb236a00700 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb236a00700 1 -- 172.21.15.46:0/4043141603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 msgr2=0x55b57db6ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb236a00700 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db6ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb236a00700 1 -- 172.21.15.46:0/4043141603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57d93d3b0 con 0x55b57db72050 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb236a00700 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb22800bae0 tx=0x7fb22800bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98365416ccbe7b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb236a00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 0x7fb21c0a38f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb22800be60 tx=0x7fb22801f000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb223e00700 1 -- 172.21.15.46:0/4043141603 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800fd30 con 0x55b57db72050 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb223e00700 1 -- 172.21.15.46:0/4043141603 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22801e040 con 0x55b57db72050 2024-05-08T23:19:21.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.607+0000 7fb223e00700 1 -- 172.21.15.46:0/4043141603 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280173f0 con 0x55b57db72050 2024-05-08T23:19:21.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.608+0000 7fb2516c7b80 1 -- 172.21.15.46:0/4043141603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 msgr2=0x55b57db72430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.608+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb22800bae0 tx=0x7fb22800bb10 comp rx=0 tx=0).stop 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 -- 172.21.15.46:0/4043141603 shutdown_connections 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db6ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db72430 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/4043141603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 -- 172.21.15.46:0/4043141603 >> 172.21.15.46:0/4043141603 conn(0x55b57daca210 msgr2=0x55b57db672d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 -- 172.21.15.46:0/4043141603 shutdown_connections 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.609+0000 7fb2516c7b80 1 -- 172.21.15.46:0/4043141603 wait complete. 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.610+0000 7fb2516c7b80 1 -- start start 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.610+0000 7fb2516c7b80 1 Processor -- start 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db76720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57da58560 con 0x55b57db6c890 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57da58200 con 0x55b57db72970 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57d93f6a0 con 0x55b57db72050 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb2516c7b80 10 client.0 _ll_register_callbacks cb 0x55b57da61b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39784/0 (socket says 172.21.15.46:39784) 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 -- 172.21.15.46:0/2798814543 learned_addr learned my addr 172.21.15.46:0/2798814543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb237400700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db76720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 -- 172.21.15.46:0/2798814543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 msgr2=0x55b57db76720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db76720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.611+0000 7fb236000700 1 -- 172.21.15.46:0/2798814543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 msgr2=0x55b57db75ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb236a00700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb236000700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db75ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb236000700 1 -- 172.21.15.46:0/2798814543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2381105d0 con 0x55b57db72050 2024-05-08T23:19:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb236a00700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db75ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb236000700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb224099d80 tx=0x7fb224003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530b8450660acc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb220c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22409a080 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.612+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b57da15270 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb22409deb0 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2240a2500 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb2516c7b80 10 client.29758 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b57d8e9b60 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b57da15270 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb22409a2e0 con 0x55b57db72050 2024-05-08T23:19:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb2516c7b80 10 client.29758 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb2516c7b80 10 client.29758 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.613+0000 7fb2516c7b80 20 client.29758 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57db86390 0x55b57db887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb2516c7b80 4 client.29758 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b57db3b810 con 0x55b57db86390 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb236a00700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57db86390 0x55b57db887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb236a00700 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57db86390 0x55b57db887b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8883c2bde2bfdc7f server_cookie=10a163ef64cac48 in_seq=0 out_seq=0 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.614+0000 7fb220c00700 10 client.29758 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.615+0000 7fb220c00700 1 -- 172.21.15.46:0/2798814543 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b57db3b810 con 0x55b57db86390 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.615+0000 7fb220c00700 10 client.29758 handle_command_reply: tid=0 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.615+0000 7fb2516c7b80 1 client.29758 shutdown 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 msgr2=0x55b57db761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb224099d80 tx=0x7fb224003f00 comp rx=0 tx=0).stop 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 shutdown_connections 2024-05-08T23:19:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57db86390 0x55b57db887b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db72970 0x55b57db76720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db6c890 0x55b57db75ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/2798814543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db761e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 >> 172.21.15.46:0/2798814543 conn(0x55b57daca210 msgr2=0x55b57db670b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb220c00700 0 client.29758 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 shutdown_connections 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 1 -- 172.21.15.46:0/2798814543 wait complete. 2024-05-08T23:19:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.616+0000 7fb2516c7b80 20 client.29758 trim_cache size 0 max 16384 2024-05-08T23:19:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.635+0000 7fb2516c7b80 1 Processor -- start 2024-05-08T23:19:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.635+0000 7fb2516c7b80 1 -- start start 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.635+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.635+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 0x55b57db7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.635+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72970 0x55b57db7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db67dc0 con 0x55b57db72050 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db78810 con 0x55b57db6c890 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db63dc0 con 0x55b57db72970 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 0x55b57db7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46382/0 (socket says 172.21.15.46:46382) 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236a00700 1 -- 172.21.15.46:0/415709558 learned_addr learned my addr 172.21.15.46:0/415709558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb237400700 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72970 0x55b57db7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 -- 172.21.15.46:0/415709558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72970 msgr2=0x55b57db7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72970 0x55b57db7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 -- 172.21.15.46:0/415709558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 msgr2=0x55b57db6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.636+0000 7fb236000700 1 -- 172.21.15.46:0/415709558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57da82560 con 0x55b57db72050 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.637+0000 7fb236a00700 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.637+0000 7fb236000700 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 0x55b57db7d2c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb224000c00 tx=0x7fb224003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cb98def415b85f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.637+0000 7fb22cc00700 1 -- 172.21.15.46:0/415709558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224099570 con 0x55b57db72050 2024-05-08T23:19:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.637+0000 7fb22cc00700 1 -- 172.21.15.46:0/415709558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2240a6070 con 0x55b57db72050 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.637+0000 7fb22cc00700 1 -- 172.21.15.46:0/415709558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240ca940 con 0x55b57db72050 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 -- 172.21.15.46:0/415709558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 msgr2=0x55b57db7d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 0x55b57db7d2c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb224000c00 tx=0x7fb224003680 comp rx=0 tx=0).stop 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 -- 172.21.15.46:0/415709558 shutdown_connections 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72970 0x55b57db7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/415709558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72050 0x55b57db7d2c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.638+0000 7fb2516c7b80 1 -- 172.21.15.46:0/415709558 >> 172.21.15.46:0/415709558 conn(0x55b57daca210 msgr2=0x55b57db65d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.639+0000 7fb2516c7b80 1 -- 172.21.15.46:0/415709558 shutdown_connections 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.639+0000 7fb2516c7b80 1 -- 172.21.15.46:0/415709558 wait complete. 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.639+0000 7fb2516c7b80 1 -- start start 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.639+0000 7fb2516c7b80 1 Processor -- start 2024-05-08T23:19:21.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 0x55b57db6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db6e000 con 0x55b57db72970 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db6e170 con 0x55b57db6c890 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57db6e330 con 0x55b57db72050 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.640+0000 7fb2516c7b80 10 client.0 _ll_register_callbacks cb 0x55b57da61b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46398/0 (socket says 172.21.15.46:46398) 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 -- 172.21.15.46:0/3723990581 learned_addr learned my addr 172.21.15.46:0/3723990581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb237400700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 0x55b57db6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236000700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 -- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 msgr2=0x55b57db7d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db7d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 -- 172.21.15.46:0/3723990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 msgr2=0x55b57db6da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 0x55b57db6da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 -- 172.21.15.46:0/3723990581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b57d93d3b0 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb237400700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 0x55b57db6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb236a00700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb228006010 tx=0x7fb22800ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445a10d458a6a5fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.641+0000 7fb220c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22803bb00 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fb22803bc60 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb22801e040 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22803be20 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb2516c7b80 10 client.29690 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.642+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b57d8e9b60 con 0x55b57db6c890 2024-05-08T23:19:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22801e040 con 0x55b57db6c890 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb228021070 con 0x55b57db6c890 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 10 client.29690 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 10 client.29690 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 20 client.29690 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57dbdd5c0 0x55b57dbdf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 4 client.29690 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b57dab4ec0 con 0x55b57dbdd5c0 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.643+0000 7fb236000700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57dbdd5c0 0x55b57dbdf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.644+0000 7fb236000700 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57dbdd5c0 0x55b57dbdf9e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a307ca811ce340b5 server_cookie=30513de80a2a51ec in_seq=0 out_seq=0 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.644+0000 7fb220c00700 10 client.29690 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.645+0000 7fb220c00700 1 -- 172.21.15.46:0/3723990581 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b57dab4ec0 con 0x55b57dbdd5c0 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.645+0000 7fb220c00700 10 client.29690 handle_command_reply: tid=0 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.645+0000 7fb2516c7b80 1 client.29690 shutdown 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.645+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 msgr2=0x55b57db7d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.645+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb228006010 tx=0x7fb22800ff40 comp rx=0 tx=0).stop 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 shutdown_connections 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55b57dbdd5c0 0x55b57dbdf9e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57db72970 0x55b57db6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b57db6c890 0x55b57db7d2d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 --2- 172.21.15.46:0/3723990581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57db72050 0x55b57db7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 >> 172.21.15.46:0/3723990581 conn(0x55b57daca210 msgr2=0x55b57db74510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb220c00700 0 client.29690 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 shutdown_connections 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.646+0000 7fb2516c7b80 1 -- 172.21.15.46:0/3723990581 wait complete. 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.647+0000 7fb2516c7b80 20 client.29690 trim_cache size 0 max 16384 2024-05-08T23:19:21.650 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 msgr2=0x7fb21c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 0x7fb21c0a38f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb22800be60 tx=0x7fb22801f000 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 msgr2=0x7fb2381011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 0x7fb2381011a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb22400ea20 tx=0x7fb22400ed30 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 shutdown_connections 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23810d520 0x7fb2381016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb21c0a1430 0x7fb21c0a38f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb238073350 0x7fb238100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 --2- 172.21.15.46:0/1563206338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb238073c70 0x7fb2381011a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.648+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 >> 172.21.15.46:0/1563206338 conn(0x7fb2380fc770 msgr2=0x7fb2380fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.649+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 shutdown_connections 2024-05-08T23:19:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.650+0000 7fb237e00700 1 -- 172.21.15.46:0/1563206338 wait complete. 2024-05-08T23:19:21.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:19:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.960+0000 7f1a61a00700 1 -- 172.21.15.46:0/797710536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c109940 msgr2=0x7f1a5c109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.960+0000 7f1a61a00700 1 --2- 172.21.15.46:0/797710536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c109940 0x7f1a5c109d20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098f0 tx=0x7f1a4c009c00 comp rx=0 tx=0).stop 2024-05-08T23:19:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 -- 172.21.15.46:0/797710536 shutdown_connections 2024-05-08T23:19:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 --2- 172.21.15.46:0/797710536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c112ef0 0x7f1a5c1152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 --2- 172.21.15.46:0/797710536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c109940 0x7f1a5c109d20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 --2- 172.21.15.46:0/797710536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c10a260 0x7f1a5c1129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 -- 172.21.15.46:0/797710536 >> 172.21.15.46:0/797710536 conn(0x7f1a5c06bf30 msgr2=0x7f1a5c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 -- 172.21.15.46:0/797710536 shutdown_connections 2024-05-08T23:19:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.961+0000 7f1a61a00700 1 -- 172.21.15.46:0/797710536 wait complete. 2024-05-08T23:19:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.962+0000 7f1a61a00700 1 Processor -- start 2024-05-08T23:19:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.963+0000 7f1a61a00700 1 -- start start 2024-05-08T23:19:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.964+0000 7f1a61a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a61a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c10a260 0x7f1a5c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a61a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c112ef0 0x7f1a5c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a61a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c19eef0 con 0x7f1a5c109940 2024-05-08T23:19:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a61a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c19f060 con 0x7f1a5c10a260 2024-05-08T23:19:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a61a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c19f220 con 0x7f1a5c112ef0 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.965+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c10a260 0x7f1a5c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35436/0 (socket says 172.21.15.46:35436) 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c112ef0 0x7f1a5c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 -- 172.21.15.46:0/83796358 learned_addr learned my addr 172.21.15.46:0/83796358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 -- 172.21.15.46:0/83796358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c112ef0 msgr2=0x7f1a5c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c112ef0 0x7f1a5c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 -- 172.21.15.46:0/83796358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c10a260 msgr2=0x7f1a5c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c10a260 0x7f1a5c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 -- 172.21.15.46:0/83796358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c009520 con 0x7f1a5c109940 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.966+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009230 tx=0x7f1a4c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.967+0000 7f1a53400700 1 -- 172.21.15.46:0/83796358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c01c070 con 0x7f1a5c109940 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.967+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c19f4a0 con 0x7f1a5c109940 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.967+0000 7f1a53400700 1 -- 172.21.15.46:0/83796358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4c00baa0 con 0x7f1a5c109940 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.967+0000 7f1a53400700 1 -- 172.21.15.46:0/83796358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c004930 con 0x7f1a5c109940 2024-05-08T23:19:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.967+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c108a10 con 0x7f1a5c109940 2024-05-08T23:19:21.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.971+0000 7f1a53400700 1 -- 172.21.15.46:0/83796358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1a4c020070 con 0x7f1a5c109940 2024-05-08T23:19:21.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.973+0000 7f1a754e3b80 1 Processor -- start 2024-05-08T23:19:21.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.973+0000 7f1a53400700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 0x7f1a400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.973+0000 7f1a754e3b80 1 -- start start 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a53400700 1 -- 172.21.15.46:0/83796358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1a4c0cddd0 con 0x7f1a5c109940 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a5b400700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 0x7f1a400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790bd7150 con 0x560790c80890 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790bd4b10 con 0x560790c85e00 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790bd0400 con 0x560790c86720 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50770/0 (socket says 172.21.15.46:50770) 2024-05-08T23:19:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.974+0000 7f1a5aa00700 1 -- 172.21.15.46:0/2342936977 learned_addr learned my addr 172.21.15.46:0/2342936977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5a000700 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 -- 172.21.15.46:0/2342936977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 msgr2=0x560790c80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 -- 172.21.15.46:0/2342936977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 msgr2=0x560790c80cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 -- 172.21.15.46:0/2342936977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560790a513b0 con 0x560790c85e00 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5a000700 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0a29c0 tx=0x7f1a4c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a2f7aeddda2745 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.975+0000 7f1a5b400700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 0x7f1a400a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c06a270 tx=0x7f1a540099e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.976+0000 7f1a47e00700 1 -- 172.21.15.46:0/2342936977 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c09ab00 con 0x560790c85e00 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.976+0000 7f1a47e00700 1 -- 172.21.15.46:0/2342936977 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4c09e930 con 0x560790c85e00 2024-05-08T23:19:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.976+0000 7f1a47e00700 1 -- 172.21.15.46:0/2342936977 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c09afc0 con 0x560790c85e00 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 -- 172.21.15.46:0/2342936977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 msgr2=0x560790c861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0a29c0 tx=0x7f1a4c009260 comp rx=0 tx=0).stop 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 -- 172.21.15.46:0/2342936977 shutdown_connections 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c85e00 0x560790c861e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/2342936977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 -- 172.21.15.46:0/2342936977 >> 172.21.15.46:0/2342936977 conn(0x560790bde210 msgr2=0x560790c7adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 -- 172.21.15.46:0/2342936977 shutdown_connections 2024-05-08T23:19:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.977+0000 7f1a754e3b80 1 -- 172.21.15.46:0/2342936977 wait complete. 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.978+0000 7f1a754e3b80 1 -- start start 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.978+0000 7f1a754e3b80 1 Processor -- start 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790b6c560 con 0x560790c80890 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790b6c200 con 0x560790c86720 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790a536a0 con 0x560790c85e00 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a754e3b80 10 client.0 _ll_register_callbacks cb 0x560790b75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.979+0000 7f1a5a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46794/0 (socket says 172.21.15.46:46794) 2024-05-08T23:19:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 -- 172.21.15.46:0/3983818877 learned_addr learned my addr 172.21.15.46:0/3983818877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5b400700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 -- 172.21.15.46:0/3983818877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 msgr2=0x560790c8a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c8a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 -- 172.21.15.46:0/3983818877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 msgr2=0x560790c89ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c89ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 -- 172.21.15.46:0/3983818877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c108a10 con 0x560790c85e00 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a5a000700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b9b0 tx=0x7f1a4800bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d70e9e6aa5a83e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.980+0000 7f1a44c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003d70 con 0x560790c85e00 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1a4801b470 con 0x560790c85e00 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560790b29270 con 0x560790c85e00 2024-05-08T23:19:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4801b630 con 0x560790c85e00 2024-05-08T23:19:21.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a754e3b80 10 client.29773 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:21.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.981+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5607909fdb60 con 0x560790c85e00 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560790b29270 con 0x560790c85e00 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a48012820 con 0x560790c85e00 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 10 client.29773 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 10 client.29773 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 20 client.29773 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790ca26a0 0x560790ca4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 4 client.29773 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560790c4f810 con 0x560790ca26a0 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.982+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790ca26a0 0x560790ca4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.983+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790ca26a0 0x560790ca4ac0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14c6df75c5d65113 server_cookie=cb74d1e8973207e4 in_seq=0 out_seq=0 2024-05-08T23:19:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.983+0000 7f1a44c00700 10 client.29773 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a44c00700 1 -- 172.21.15.46:0/3983818877 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560790c4f810 con 0x560790ca26a0 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a44c00700 10 client.29773 handle_command_reply: tid=0 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 client.29773 shutdown 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 msgr2=0x560790c8a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b9b0 tx=0x7f1a4800bcc0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 shutdown_connections 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790ca26a0 0x560790ca4ac0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a44c00700 0 client.29773 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3983818877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c8a010 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.984+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 >> 172.21.15.46:0/3983818877 conn(0x560790bde210 msgr2=0x560790c7a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.985+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 shutdown_connections 2024-05-08T23:19:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.985+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3983818877 wait complete. 2024-05-08T23:19:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:21.985+0000 7f1a754e3b80 20 client.29773 trim_cache size 0 max 16384 2024-05-08T23:19:22.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.004+0000 7f1a754e3b80 1 Processor -- start 2024-05-08T23:19:22.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 -- start start 2024-05-08T23:19:22.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c80c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790ca9cd0 con 0x560790c80890 2024-05-08T23:19:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790c7c020 con 0x560790c86720 2024-05-08T23:19:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790c88020 con 0x560790c85e00 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.005+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50792/0 (socket says 172.21.15.46:50792) 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 -- 172.21.15.46:0/3131528978 learned_addr learned my addr 172.21.15.46:0/3131528978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5a000700 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c80c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 -- 172.21.15.46:0/3131528978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 msgr2=0x560790c80c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c80c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 -- 172.21.15.46:0/3131528978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 msgr2=0x560790c80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 -- 172.21.15.46:0/3131528978 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560790b96560 con 0x560790c86720 2024-05-08T23:19:22.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.006+0000 7f1a5b400700 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1a54019850 tx=0x7f1a5401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d501fa87960571 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.007+0000 7f1a50c00700 1 -- 172.21.15.46:0/3131528978 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400e040 con 0x560790c86720 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.007+0000 7f1a50c00700 1 -- 172.21.15.46:0/3131528978 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a5400f040 con 0x560790c86720 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.007+0000 7f1a50c00700 1 -- 172.21.15.46:0/3131528978 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54017ce0 con 0x560790c86720 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3131528978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 msgr2=0x560790c85280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1a54019850 tx=0x7f1a5401a000 comp rx=0 tx=0).stop 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3131528978 shutdown_connections 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c86720 0x560790c85280 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c80890 0x560790c80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/3131528978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c85e00 0x560790c80c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3131528978 >> 172.21.15.46:0/3131528978 conn(0x560790bde210 msgr2=0x560790c84b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3131528978 shutdown_connections 2024-05-08T23:19:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.008+0000 7f1a754e3b80 1 -- 172.21.15.46:0/3131528978 wait complete. 2024-05-08T23:19:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.009+0000 7f1a754e3b80 1 -- start start 2024-05-08T23:19:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.009+0000 7f1a754e3b80 1 Processor -- start 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c85e00 0x560790c95c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c96140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790cf7d40 con 0x560790c85e00 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790cf7eb0 con 0x560790c80890 2024-05-08T23:19:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560790cf8070 con 0x560790c86720 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a754e3b80 10 client.0 _ll_register_callbacks cb 0x560790b75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a5aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50796/0 (socket says 172.21.15.46:50796) 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.010+0000 7f1a5aa00700 1 -- 172.21.15.46:0/1731513085 learned_addr learned my addr 172.21.15.46:0/1731513085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5b400700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c96140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 -- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 msgr2=0x560790c96140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c96140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5a000700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c85e00 0x560790c95c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 -- 172.21.15.46:0/1731513085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c85e00 msgr2=0x560790c95c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c85e00 0x560790c95c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 -- 172.21.15.46:0/1731513085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a54019500 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a5aa00700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c096f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adabd4da2199d459 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a44c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c099ff0 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.011+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1a4c09a150 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.012+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a4c09e930 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.012+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c09ad60 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.012+0000 7f1a754e3b80 10 client.29708 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.012+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560790a55cd0 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.012+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4c09e930 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a4c0a2a60 con 0x560790c80890 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 10 client.29708 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:22.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 10 client.29708 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 20 client.29708 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790cf93c0 0x560790cfb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 4 client.29708 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560790bc8ec0 con 0x560790cf93c0 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a5a000700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790cf93c0 0x560790cfb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a44c00700 10 client.29708 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:22.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.013+0000 7f1a5a000700 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790cf93c0 0x560790cfb7b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da985a9426679d24 server_cookie=ed93957fb22d3b2f in_seq=0 out_seq=0 2024-05-08T23:19:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a44c00700 1 -- 172.21.15.46:0/1731513085 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560790bc8ec0 con 0x560790cf93c0 2024-05-08T23:19:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a44c00700 10 client.29708 handle_command_reply: tid=0 2024-05-08T23:19:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 client.29708 shutdown 2024-05-08T23:19:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 msgr2=0x560790c956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c096f70 comp rx=0 tx=0).stop 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 shutdown_connections 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560790cf93c0 0x560790cfb7b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560790c86720 0x560790c96140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560790c80890 0x560790c956c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 --2- 172.21.15.46:0/1731513085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560790c85e00 0x560790c95c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 >> 172.21.15.46:0/1731513085 conn(0x560790bde210 msgr2=0x560790c84b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a44c00700 0 client.29708 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 shutdown_connections 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 1 -- 172.21.15.46:0/1731513085 wait complete. 2024-05-08T23:19:22.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.014+0000 7f1a754e3b80 20 client.29708 trim_cache size 0 max 16384 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 msgr2=0x7f1a400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 0x7f1a400a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c06a270 tx=0x7f1a540099e0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 msgr2=0x7f1a5c1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009230 tx=0x7f1a4c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:19:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 shutdown_connections 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a5c112ef0 0x7f1a5c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1a400a1310 0x7f1a400a37d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a5c109940 0x7f1a5c1a4700 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 --2- 172.21.15.46:0/83796358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c10a260 0x7f1a5c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 >> 172.21.15.46:0/83796358 conn(0x7f1a5c06bf30 msgr2=0x7f1a5c070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.015+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 shutdown_connections 2024-05-08T23:19:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.016+0000 7f1a61a00700 1 -- 172.21.15.46:0/83796358 wait complete. 2024-05-08T23:19:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:21 smithi046 ceph-mon[36898]: pgmap v487: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-05-08T23:19:22.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:21 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1460685780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:22.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:19:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:21 smithi086 ceph-mon[78610]: pgmap v487: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-05-08T23:19:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:21 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1460685780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:22.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 41300 23.27 MB/sec warmup 103 sec latency 150.802 ms 2024-05-08T23:19:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:21 smithi053 ceph-mon[77587]: pgmap v487: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-05-08T23:19:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:21 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1460685780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.304+0000 7ff00b600700 1 -- 172.21.15.46:0/717721817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 msgr2=0x7ff004102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.304+0000 7ff00b600700 1 --2- 172.21.15.46:0/717721817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004102380 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feff8009990 tx=0x7feff8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 -- 172.21.15.46:0/717721817 shutdown_connections 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 --2- 172.21.15.46:0/717721817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 0x7ff00410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 --2- 172.21.15.46:0/717721817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004102380 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 --2- 172.21.15.46:0/717721817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff00410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 -- 172.21.15.46:0/717721817 >> 172.21.15.46:0/717721817 conn(0x7ff0040fb760 msgr2=0x7ff0040fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.305+0000 7ff00b600700 1 -- 172.21.15.46:0/717721817 shutdown_connections 2024-05-08T23:19:22.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.306+0000 7ff00b600700 1 -- 172.21.15.46:0/717721817 wait complete. 2024-05-08T23:19:22.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.307+0000 7ff00b600700 1 Processor -- start 2024-05-08T23:19:22.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 -- start start 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 0x7ff00419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e380 con 0x7ff00410b3c0 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e4f0 con 0x7ff004101fa0 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff00b600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e6b0 con 0x7ff0041028c0 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff008c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff009600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 0x7ff00419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46832/0 (socket says 172.21.15.46:46832) 2024-05-08T23:19:22.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff003e00700 1 -- 172.21.15.46:0/2056903849 learned_addr learned my addr 172.21.15.46:0/2056903849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.308+0000 7ff008c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50820/0 (socket says 172.21.15.46:50820) 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 -- 172.21.15.46:0/2056903849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 msgr2=0x7ff004199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 -- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 msgr2=0x7ff00419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 0x7ff00419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 -- 172.21.15.46:0/2056903849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80095f0 con 0x7ff0041028c0 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.309+0000 7ff003e00700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.310+0000 7ff001600700 1 -- 172.21.15.46:0/2056903849 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000c9c0 con 0x7ff0041028c0 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.310+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00419e990 con 0x7ff0041028c0 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.310+0000 7ff001600700 1 -- 172.21.15.46:0/2056903849 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff0010430 con 0x7ff0041028c0 2024-05-08T23:19:22.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.310+0000 7ff001600700 1 -- 172.21.15.46:0/2056903849 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0018680 con 0x7ff0041028c0 2024-05-08T23:19:22.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.310+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a61d0 con 0x7ff0041028c0 2024-05-08T23:19:22.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.314+0000 7ff001600700 1 -- 172.21.15.46:0/2056903849 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feff0010860 con 0x7ff0041028c0 2024-05-08T23:19:22.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.315+0000 7ff01f16cb80 1 Processor -- start 2024-05-08T23:19:22.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.315+0000 7ff01f16cb80 1 -- start start 2024-05-08T23:19:22.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.315+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d533080350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532fd7150 con 0x55d533080890 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532fd4b10 con 0x55d533085e00 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532fd0400 con 0x55d533086720 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff001600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 0x7fefec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff003e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff009600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff008c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d533080350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff003e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35492/0 (socket says 172.21.15.46:35492) 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff003e00700 1 -- 172.21.15.46:0/2330209390 learned_addr learned my addr 172.21.15.46:0/2330209390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff009600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50822/0 (socket says 172.21.15.46:50822) 2024-05-08T23:19:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.316+0000 7ff001600700 1 -- 172.21.15.46:0/2056903849 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feff0014070 con 0x7ff0041028c0 2024-05-08T23:19:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff003e00700 1 -- 172.21.15.46:0/2330209390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 msgr2=0x55d533080350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff003e00700 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d533080350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff003e00700 1 -- 172.21.15.46:0/2330209390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 msgr2=0x55d5330861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff003e00700 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff003e00700 1 -- 172.21.15.46:0/2330209390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d532e513b0 con 0x55d533080890 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff009600700 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.317+0000 7ff008c00700 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d533080350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.318+0000 7ff009600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 0x7fefec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.318+0000 7ff003e00700 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feff00ca730 tx=0x7feff006d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619701767b6c59d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.319+0000 7feff5c00700 1 -- 172.21.15.46:0/2330209390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0065a30 con 0x55d533080890 2024-05-08T23:19:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.319+0000 7ff009600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 0x7fefec0a37d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feffc010480 tx=0x7feffc004ad0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.320+0000 7feff5c00700 1 -- 172.21.15.46:0/2330209390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff009a790 con 0x55d533080890 2024-05-08T23:19:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.320+0000 7feff5c00700 1 -- 172.21.15.46:0/2330209390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff00657d0 con 0x55d533080890 2024-05-08T23:19:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2330209390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 msgr2=0x55d533080cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feff00ca730 tx=0x7feff006d220 comp rx=0 tx=0).stop 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2330209390 shutdown_connections 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080cd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533085e00 0x55d5330861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2330209390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d533080350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2330209390 >> 172.21.15.46:0/2330209390 conn(0x55d532fde210 msgr2=0x55d53307adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2330209390 shutdown_connections 2024-05-08T23:19:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.321+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2330209390 wait complete. 2024-05-08T23:19:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.322+0000 7ff01f16cb80 1 -- start start 2024-05-08T23:19:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.322+0000 7ff01f16cb80 1 Processor -- start 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533085e00 0x55d533089ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d53308a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532f6c560 con 0x55d533085e00 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532f6c200 con 0x55d533080890 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d532e536a0 con 0x55d533086720 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff01f16cb80 10 client.0 _ll_register_callbacks cb 0x55d532f75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff008c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff009600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d53308a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff008c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50832/0 (socket says 172.21.15.46:50832) 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff008c00700 1 -- 172.21.15.46:0/2961830730 learned_addr learned my addr 172.21.15.46:0/2961830730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.323+0000 7ff003e00700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533085e00 0x55d533089ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 -- 172.21.15.46:0/2961830730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 msgr2=0x55d53308c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 -- 172.21.15.46:0/2961830730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533085e00 msgr2=0x55d533089ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533085e00 0x55d533089ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 -- 172.21.15.46:0/2961830730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a61d0 con 0x55d533086720 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff008c00700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff009600700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d53308a1f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feffc007fd0 tx=0x7feffc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311be8fdfa4c0918 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7fefea000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc014e30 con 0x55d533086720 2024-05-08T23:19:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feffc019030 con 0x55d533086720 2024-05-08T23:19:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.324+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d532f29270 con 0x55d533086720 2024-05-08T23:19:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc013a30 con 0x55d533086720 2024-05-08T23:19:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7ff01f16cb80 10 client.29788 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d532dfdb60 con 0x55d533086720 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d532f29270 con 0x55d533086720 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feffc020c70 con 0x55d533086720 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7ff01f16cb80 10 client.29788 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.325+0000 7ff01f16cb80 10 client.29788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff01f16cb80 20 client.29788 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d53309a190 0x55d53309c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff01f16cb80 4 client.29788 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d53304f810 con 0x55d53309a190 2024-05-08T23:19:22.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff008c00700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d53309a190 0x55d53309c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7fefea000700 10 client.29788 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:22.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.326+0000 7ff008c00700 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d53309a190 0x55d53309c5b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8cebb14dd7de86a4 server_cookie=cfa57e50357cd89b in_seq=0 out_seq=0 2024-05-08T23:19:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.330+0000 7fefea000700 1 -- 172.21.15.46:0/2961830730 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d53304f810 con 0x55d53309a190 2024-05-08T23:19:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.330+0000 7fefea000700 10 client.29788 handle_command_reply: tid=0 2024-05-08T23:19:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.330+0000 7ff01f16cb80 1 client.29788 shutdown 2024-05-08T23:19:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 msgr2=0x55d53308a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d53308a1f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feffc007fd0 tx=0x7feffc004b40 comp rx=0 tx=0).stop 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 shutdown_connections 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533086720 0x55d53308a1f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533080890 0x55d53308c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d53309a190 0x55d53309c5b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2961830730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533085e00 0x55d533089ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7fefea000700 0 client.29788 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:22.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 >> 172.21.15.46:0/2961830730 conn(0x55d532fde210 msgr2=0x55d53307a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 shutdown_connections 2024-05-08T23:19:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2961830730 wait complete. 2024-05-08T23:19:22.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.331+0000 7ff01f16cb80 20 client.29788 trim_cache size 0 max 16384 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 Processor -- start 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 -- start start 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d533075800 con 0x55d533080890 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53307c020 con 0x55d533086720 2024-05-08T23:19:22.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d533097740 con 0x55d533085e00 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46852/0 (socket says 172.21.15.46:46852) 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff003e00700 1 -- 172.21.15.46:0/3678768160 learned_addr learned my addr 172.21.15.46:0/3678768160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.351+0000 7ff008c00700 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff009600700 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 -- 172.21.15.46:0/3678768160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 msgr2=0x55d5330883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 -- 172.21.15.46:0/3678768160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 msgr2=0x55d533080350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 -- 172.21.15.46:0/3678768160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d532f96560 con 0x55d533085e00 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7ff003e00700 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7feff0096590 tx=0x7feff009a900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3271dd36db0a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7feff6600700 1 -- 172.21.15.46:0/3678768160 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff00a2510 con 0x55d533085e00 2024-05-08T23:19:22.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7feff6600700 1 -- 172.21.15.46:0/3678768160 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff009d070 con 0x55d533085e00 2024-05-08T23:19:22.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.352+0000 7feff6600700 1 -- 172.21.15.46:0/3678768160 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff009a550 con 0x55d533085e00 2024-05-08T23:19:22.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 -- 172.21.15.46:0/3678768160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 msgr2=0x55d53307ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7feff0096590 tx=0x7feff009a900 comp rx=0 tx=0).stop 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 -- 172.21.15.46:0/3678768160 shutdown_connections 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d533080350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/3678768160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53307ad20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 -- 172.21.15.46:0/3678768160 >> 172.21.15.46:0/3678768160 conn(0x55d532fde210 msgr2=0x55d533095b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.353+0000 7ff01f16cb80 1 -- 172.21.15.46:0/3678768160 shutdown_connections 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.354+0000 7ff01f16cb80 1 -- 172.21.15.46:0/3678768160 wait complete. 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.354+0000 7ff01f16cb80 1 -- start start 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.354+0000 7ff01f16cb80 1 Processor -- start 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d53308d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330ef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53308df10 con 0x55d533080890 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5330efbc0 con 0x55d533086720 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5330efd50 con 0x55d533085e00 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff01f16cb80 10 client.0 _ll_register_callbacks cb 0x55d532f75b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff009600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff008c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d53308d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46866/0 (socket says 172.21.15.46:46866) 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 -- 172.21.15.46:0/2769914777 learned_addr learned my addr 172.21.15.46:0/2769914777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff009600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330ef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50836/0 (socket says 172.21.15.46:50836) 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 -- 172.21.15.46:0/2769914777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 msgr2=0x55d5330ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330ef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 -- 172.21.15.46:0/2769914777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 msgr2=0x55d53308d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d53308d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.355+0000 7ff003e00700 1 -- 172.21.15.46:0/2769914777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d532e513b0 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7ff003e00700 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7feff00cd4e0 tx=0x7feff009a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d36687fe9693a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7fefea000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff009a4f0 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feff009dec0 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feff009a790 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff00a2510 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7ff01f16cb80 10 client.29794 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.356+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d532e55cd0 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff009a790 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feff0096020 con 0x55d533085e00 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 10 client.29794 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 10 client.29794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 20 client.29794 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d5330f1040 0x55d5330f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 4 client.29794 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d532fc8ec0 con 0x55d5330f1040 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.357+0000 7ff008c00700 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d5330f1040 0x55d5330f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.358+0000 7ff008c00700 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d5330f1040 0x55d5330f3460 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69c5c6c447e37565 server_cookie=303e7c526baedc22 in_seq=0 out_seq=0 2024-05-08T23:19:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.358+0000 7fefea000700 10 client.29794 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.361+0000 7fefea000700 1 -- 172.21.15.46:0/2769914777 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d532fc8ec0 con 0x55d5330f1040 2024-05-08T23:19:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.361+0000 7fefea000700 10 client.29794 handle_command_reply: tid=0 2024-05-08T23:19:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 client.29794 shutdown 2024-05-08T23:19:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 msgr2=0x55d53308d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7feff00cd4e0 tx=0x7feff009a210 comp rx=0 tx=0).stop 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 shutdown_connections 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d533086720 0x55d5330ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55d5330f1040 0x55d5330f3460 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d533080890 0x55d53308d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 --2- 172.21.15.46:0/2769914777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d533085e00 0x55d53308d7e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 >> 172.21.15.46:0/2769914777 conn(0x55d532fde210 msgr2=0x55d533095b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7fefea000700 0 client.29794 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 shutdown_connections 2024-05-08T23:19:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 1 -- 172.21.15.46:0/2769914777 wait complete. 2024-05-08T23:19:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.362+0000 7ff01f16cb80 20 client.29794 trim_cache size 0 max 16384 2024-05-08T23:19:22.363 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 msgr2=0x7fefec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 0x7fefec0a37d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feffc010480 tx=0x7feffc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 msgr2=0x7ff0041999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000ed00 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 shutdown_connections 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fefec0a1310 0x7fefec0a37d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff00410b3c0 0x7ff00419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004101fa0 0x7ff004199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 --2- 172.21.15.46:0/2056903849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0041028c0 0x7ff0041999a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.363+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 >> 172.21.15.46:0/2056903849 conn(0x7ff0040fb760 msgr2=0x7ff0040fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.364+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 shutdown_connections 2024-05-08T23:19:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.364+0000 7ff00b600700 1 -- 172.21.15.46:0/2056903849 wait complete. 2024-05-08T23:19:22.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:19:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.641+0000 7f3d09800700 1 -- 172.21.15.46:0/1696914899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d04068a80 msgr2=0x7f3d0410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.641+0000 7f3d09800700 1 --2- 172.21.15.46:0/1696914899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d04068a80 0x7f3d0410d400 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009990 tx=0x7f3cf4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 -- 172.21.15.46:0/1696914899 shutdown_connections 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 --2- 172.21.15.46:0/1696914899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d0410d940 0x7f3d0410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 --2- 172.21.15.46:0/1696914899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 0x7f3d04068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 --2- 172.21.15.46:0/1696914899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d04068a80 0x7f3d0410d400 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 -- 172.21.15.46:0/1696914899 >> 172.21.15.46:0/1696914899 conn(0x7f3d04075710 msgr2=0x7f3d04075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 -- 172.21.15.46:0/1696914899 shutdown_connections 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.642+0000 7f3d09800700 1 -- 172.21.15.46:0/1696914899 wait complete. 2024-05-08T23:19:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.643+0000 7f3d09800700 1 Processor -- start 2024-05-08T23:19:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.643+0000 7f3d09800700 1 -- start start 2024-05-08T23:19:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 0x7f3d04199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04068a80 0x7f3d04199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419e220 con 0x7f3d0410d940 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419e390 con 0x7f3d04068160 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d09800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419e550 con 0x7f3d04068a80 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35538/0 (socket says 172.21.15.46:35538) 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 -- 172.21.15.46:0/1566871288 learned_addr learned my addr 172.21.15.46:0/1566871288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d03400700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04068a80 0x7f3d04199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d03e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 0x7f3d04199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 -- 172.21.15.46:0/1566871288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04068a80 msgr2=0x7f3d04199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04068a80 0x7f3d04199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 -- 172.21.15.46:0/1566871288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 msgr2=0x7f3d04199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 0x7f3d04199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.644+0000 7f3d08e00700 1 -- 172.21.15.46:0/1566871288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40095f0 con 0x7f3d0410d940 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d08e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e970 tx=0x7f3cec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d00c00700 1 -- 172.21.15.46:0/1566871288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec009750 con 0x7f3d0410d940 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d00c00700 1 -- 172.21.15.46:0/1566871288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cec005020 con 0x7f3d0410d940 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0419e830 con 0x7f3d0410d940 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d00c00700 1 -- 172.21.15.46:0/1566871288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00cbf0 con 0x7f3d0410d940 2024-05-08T23:19:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.645+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d041a6070 con 0x7f3d0410d940 2024-05-08T23:19:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.649+0000 7f3d00c00700 1 -- 172.21.15.46:0/1566871288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3cec005470 con 0x7f3d0410d940 2024-05-08T23:19:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.650+0000 7f3d00c00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 0x7f3ce80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.650+0000 7f3d00c00700 1 -- 172.21.15.46:0/1566871288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3cec014070 con 0x7f3d0410d940 2024-05-08T23:19:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.651+0000 7f3d1e9c5b80 1 Processor -- start 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.651+0000 7f3d1e9c5b80 1 -- start start 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.651+0000 7f3d03e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 0x7f3ce80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15d03150 con 0x55bc15db2970 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15d00b10 con 0x55bc15db2050 2024-05-08T23:19:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15cfc400 con 0x55bc15dac890 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d03400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46908/0 (socket says 172.21.15.46:46908) 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.652+0000 7f3d08e00700 1 -- 172.21.15.46:0/3040826372 learned_addr learned my addr 172.21.15.46:0/3040826372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 -- 172.21.15.46:0/3040826372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 msgr2=0x55bc15db2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d03e00700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 -- 172.21.15.46:0/3040826372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 msgr2=0x55bc15dac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 -- 172.21.15.46:0/3040826372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc15b7d3b0 con 0x55bc15dac890 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d03e00700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d03e00700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 0x7f3ce80a3870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0099d0 tx=0x7f3cfc003b10 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d03400700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3d08e00700 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0a1d70 tx=0x7f3cec099810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a0848afc3c3492 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.653+0000 7f3cf9600700 1 -- 172.21.15.46:0/3040826372 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec099eb0 con 0x55bc15dac890 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.654+0000 7f3cf9600700 1 -- 172.21.15.46:0/3040826372 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cec09dce0 con 0x55bc15dac890 2024-05-08T23:19:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.654+0000 7f3cf9600700 1 -- 172.21.15.46:0/3040826372 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec016590 con 0x55bc15dac890 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.654+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/3040826372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 msgr2=0x55bc15daccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.654+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0a1d70 tx=0x7f3cec099810 comp rx=0 tx=0).stop 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/3040826372 shutdown_connections 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15daccd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/3040826372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/3040826372 >> 172.21.15.46:0/3040826372 conn(0x55bc15d0a210 msgr2=0x55bc15da72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/3040826372 shutdown_connections 2024-05-08T23:19:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.655+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/3040826372 wait complete. 2024-05-08T23:19:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.656+0000 7f3d1e9c5b80 1 -- start start 2024-05-08T23:19:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.656+0000 7f3d1e9c5b80 1 Processor -- start 2024-05-08T23:19:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15db5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15c98560 con 0x55bc15db2970 2024-05-08T23:19:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15c98200 con 0x55bc15db2050 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15b7f6a0 con 0x55bc15dac890 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d1e9c5b80 10 client.0 _ll_register_callbacks cb 0x55bc15ca1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.657+0000 7f3d03400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50876/0 (socket says 172.21.15.46:50876) 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03400700 1 -- 172.21.15.46:0/578255708 learned_addr learned my addr 172.21.15.46:0/578255708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15db5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 -- 172.21.15.46:0/578255708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 msgr2=0x55bc15db6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 -- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 msgr2=0x55bc15db67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.658+0000 7f3d03e00700 1 -- 172.21.15.46:0/578255708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d041a6070 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3d03e00700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15db5d20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009690 tx=0x7f3cfc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f58a85e6df68ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3d03400700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3cf2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc017460 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3cfc0175c0 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.659+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc15c55270 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.660+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cfc017780 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.660+0000 7f3d1e9c5b80 10 client.29806 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.660+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc15b29b60 con 0x55bc15dac890 2024-05-08T23:19:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.660+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cfc00f040 con 0x55bc15dac890 2024-05-08T23:19:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3cfc021cb0 con 0x55bc15dac890 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 10 client.29806 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 10 client.29806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 20 client.29806 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15dcebb0 0x55bc15dd0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 4 client.29806 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc15c55270 con 0x55bc15dcebb0 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.661+0000 7f3d03400700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15dcebb0 0x55bc15dd0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.662+0000 7f3d03400700 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15dcebb0 0x55bc15dd0fd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61e94262aa6744a6 server_cookie=8da01fcde091f85d in_seq=0 out_seq=0 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.662+0000 7f3cf2000700 10 client.29806 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.662+0000 7f3cf2000700 1 -- 172.21.15.46:0/578255708 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc15c55270 con 0x55bc15dcebb0 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.662+0000 7f3cf2000700 10 client.29806 handle_command_reply: tid=0 2024-05-08T23:19:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.662+0000 7f3d1e9c5b80 1 client.29806 shutdown 2024-05-08T23:19:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 msgr2=0x55bc15db5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15db5d20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009690 tx=0x7f3cfc004b40 comp rx=0 tx=0).stop 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 shutdown_connections 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15dcebb0 0x55bc15dd0fd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15db5d20 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/578255708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 >> 172.21.15.46:0/578255708 conn(0x55bc15d0a210 msgr2=0x55bc15da70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.663+0000 7f3cf2000700 0 client.29806 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.664+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 shutdown_connections 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.664+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/578255708 wait complete. 2024-05-08T23:19:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.664+0000 7f3d1e9c5b80 20 client.29806 trim_cache size 0 max 16384 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 Processor -- start 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 -- start start 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15db8cf0 con 0x55bc15db2970 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15dd61c0 con 0x55bc15db2050 2024-05-08T23:19:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.686+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15da6390 con 0x55bc15dac890 2024-05-08T23:19:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46924/0 (socket says 172.21.15.46:46924) 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 -- 172.21.15.46:0/473256489 learned_addr learned my addr 172.21.15.46:0/473256489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03400700 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d08e00700 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 -- 172.21.15.46:0/473256489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 msgr2=0x55bc15db1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 -- 172.21.15.46:0/473256489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 msgr2=0x55bc15db65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 -- 172.21.15.46:0/473256489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc15cc2560 con 0x55bc15dac890 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.687+0000 7f3d03e00700 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009470 tx=0x7f3cfc0185c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2e07cc894d9ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.688+0000 7f3cfa000700 1 -- 172.21.15.46:0/473256489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc009ea0 con 0x55bc15dac890 2024-05-08T23:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.688+0000 7f3cfa000700 1 -- 172.21.15.46:0/473256489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cfc017460 con 0x55bc15dac890 2024-05-08T23:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.688+0000 7f3cfa000700 1 -- 172.21.15.46:0/473256489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc034430 con 0x55bc15dac890 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/473256489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 msgr2=0x55bc15dac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009470 tx=0x7f3cfc0185c0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/473256489 shutdown_connections 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15db65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15dac350 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/473256489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15db1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/473256489 >> 172.21.15.46:0/473256489 conn(0x55bc15d0a210 msgr2=0x55bc15db0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/473256489 shutdown_connections 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.689+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/473256489 wait complete. 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.690+0000 7f3d1e9c5b80 1 -- start start 2024-05-08T23:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.690+0000 7f3d1e9c5b80 1 Processor -- start 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15e1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dc3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15dc4550 con 0x55bc15db2970 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15dc46c0 con 0x55bc15db2050 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc15dc4880 con 0x55bc15dac890 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d1e9c5b80 10 client.0 _ll_register_callbacks cb 0x55bc15ca1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d08e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dc3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46940/0 (socket says 172.21.15.46:46940) 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 -- 172.21.15.46:0/61989290 learned_addr learned my addr 172.21.15.46:0/61989290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 -- 172.21.15.46:0/61989290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 msgr2=0x55bc15e1e820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15e1e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.691+0000 7f3d03e00700 1 -- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 msgr2=0x55bc15dc3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3d03e00700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dc3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3d03e00700 1 -- 172.21.15.46:0/61989290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc15b29b60 con 0x55bc15dac890 2024-05-08T23:19:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3d08e00700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dc3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3d03e00700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0092a0 tx=0x7f3cfc0094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ee2ec716c9b5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3cf2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc009ea0 con 0x55bc15dac890 2024-05-08T23:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cec09dce0 con 0x55bc15dac890 2024-05-08T23:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.692+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3cfc017af0 con 0x55bc15dac890 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cfc017cb0 con 0x55bc15dac890 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 10 client.29812 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc15b81cd0 con 0x55bc15dac890 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cec09dce0 con 0x55bc15dac890 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3cfc045070 con 0x55bc15dac890 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 10 client.29812 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 10 client.29812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 20 client.29812 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15e25df0 0x55bc15e28210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 4 client.29812 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc15d7b810 con 0x55bc15e25df0 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.693+0000 7f3d03400700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15e25df0 0x55bc15e28210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.694+0000 7f3d03400700 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15e25df0 0x55bc15e28210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed9658c09ce3eefc server_cookie=9be18c9b7a28a2e3 in_seq=0 out_seq=0 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.694+0000 7f3cf2000700 10 client.29812 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3cf2000700 1 -- 172.21.15.46:0/61989290 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc15d7b810 con 0x55bc15e25df0 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3cf2000700 10 client.29812 handle_command_reply: tid=0 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 client.29812 shutdown 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 msgr2=0x55bc15e1e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0092a0 tx=0x7f3cfc0094e0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 shutdown_connections 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55bc15e25df0 0x55bc15e28210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc15db2970 0x55bc15dc3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc15dac890 0x55bc15e1e2e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 --2- 172.21.15.46:0/61989290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc15db2050 0x55bc15e1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3cf2000700 0 client.29812 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.695+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 >> 172.21.15.46:0/61989290 conn(0x55bc15d0a210 msgr2=0x55bc15db2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 shutdown_connections 2024-05-08T23:19:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d1e9c5b80 1 -- 172.21.15.46:0/61989290 wait complete. 2024-05-08T23:19:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d1e9c5b80 20 client.29812 trim_cache size 0 max 16384 2024-05-08T23:19:22.698 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 msgr2=0x7f3ce80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 0x7f3ce80a3870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0099d0 tx=0x7f3cfc003b10 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 msgr2=0x7f3d0419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e970 tx=0x7f3cec00ec80 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 shutdown_connections 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3ce80a13b0 0x7f3ce80a3870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d0410d940 0x7f3d0419dbe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04068160 0x7f3d04199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 --2- 172.21.15.46:0/1566871288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d04068a80 0x7f3d04199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.696+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 >> 172.21.15.46:0/1566871288 conn(0x7f3d04075710 msgr2=0x7f3d0410ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.697+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 shutdown_connections 2024-05-08T23:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:22.697+0000 7f3d09800700 1 -- 172.21.15.46:0/1566871288 wait complete. 2024-05-08T23:19:22.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:19:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 -- 172.21.15.46:0/4140157953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb08101fa0 msgr2=0x7fcb08102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 --2- 172.21.15.46:0/4140157953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb08101fa0 0x7fcb08102380 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009990 tx=0x7fcaf8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 -- 172.21.15.46:0/4140157953 shutdown_connections 2024-05-08T23:19:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 --2- 172.21.15.46:0/4140157953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb0810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 --2- 172.21.15.46:0/4140157953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb08101fa0 0x7fcb08102380 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 --2- 172.21.15.46:0/4140157953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb081028c0 0x7fcb0810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.022+0000 7fcb0f000700 1 -- 172.21.15.46:0/4140157953 >> 172.21.15.46:0/4140157953 conn(0x7fcb080fb760 msgr2=0x7fcb080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.023+0000 7fcb0f000700 1 -- 172.21.15.46:0/4140157953 shutdown_connections 2024-05-08T23:19:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.023+0000 7fcb0f000700 1 -- 172.21.15.46:0/4140157953 wait complete. 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.024+0000 7fcb0f000700 1 Processor -- start 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.024+0000 7fcb0f000700 1 -- start start 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 0x7fcb080ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 0x7fcb08100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081a56a0 con 0x7fcb08101fa0 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081a5810 con 0x7fcb081028c0 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081a59d0 con 0x7fcb0810b3c0 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46960/0 (socket says 172.21.15.46:46960) 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb0d000700 1 -- 172.21.15.46:0/3705820153 learned_addr learned my addr 172.21.15.46:0/3705820153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb07e00700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 0x7fcb080ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.025+0000 7fcb07400700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 0x7fcb08100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 -- 172.21.15.46:0/3705820153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 msgr2=0x7fcb08100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 0x7fcb08100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 -- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 msgr2=0x7fcb080ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 0x7fcb080ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 -- 172.21.15.46:0/3705820153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00009c40 con 0x7fcb0810b3c0 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb07e00700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 0x7fcb080ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb07400700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 0x7fcb08100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0d000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c370 tx=0x7fcb0000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb04c00700 1 -- 172.21.15.46:0/3705820153 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00017070 con 0x7fcb0810b3c0 2024-05-08T23:19:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.026+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf80095f0 con 0x7fcb0810b3c0 2024-05-08T23:19:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.027+0000 7fcb04c00700 1 -- 172.21.15.46:0/3705820153 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0000f040 con 0x7fcb0810b3c0 2024-05-08T23:19:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.027+0000 7fcb04c00700 1 -- 172.21.15.46:0/3705820153 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000e050 con 0x7fcb0810b3c0 2024-05-08T23:19:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.027+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb081a6020 con 0x7fcb0810b3c0 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.030+0000 7fcb04c00700 1 -- 172.21.15.46:0/3705820153 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb00004520 con 0x7fcb0810b3c0 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.031+0000 7fcb22b77b80 1 Processor -- start 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.031+0000 7fcb22b77b80 1 -- start start 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.031+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.031+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad800150 con 0x55b1ad8a9890 2024-05-08T23:19:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad7fdb10 con 0x55b1ad8af720 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad7f9400 con 0x55b1ad8aee00 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb04c00700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 0x7fcaec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb04c00700 1 -- 172.21.15.46:0/3705820153 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcb000cc9d0 con 0x7fcb0810b3c0 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46972/0 (socket says 172.21.15.46:46972) 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb0d000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 0x7fcaec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb07400700 1 -- 172.21.15.46:0/1739092177 learned_addr learned my addr 172.21.15.46:0/1739092177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb07e00700 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.032+0000 7fcb07400700 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 -- 172.21.15.46:0/1739092177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 msgr2=0x55b1ad8a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 -- 172.21.15.46:0/1739092177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 msgr2=0x55b1ad8a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 -- 172.21.15.46:0/1739092177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ad67a3b0 con 0x55b1ad8aee00 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb0d000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 0x7fcaec0a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb000960e0 tx=0x7fcb00009630 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcb07400700 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011c60 tx=0x7fcaf4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a7f43d7876b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcafd600700 1 -- 172.21.15.46:0/1739092177 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400f6d0 con 0x55b1ad8aee00 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcafd600700 1 -- 172.21.15.46:0/1739092177 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf400fcf0 con 0x55b1ad8aee00 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.033+0000 7fcafd600700 1 -- 172.21.15.46:0/1739092177 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401fc10 con 0x55b1ad8aee00 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 -- 172.21.15.46:0/1739092177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 msgr2=0x55b1ad8af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011c60 tx=0x7fcaf4009c20 comp rx=0 tx=0).stop 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 -- 172.21.15.46:0/1739092177 shutdown_connections 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8af1e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/1739092177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 -- 172.21.15.46:0/1739092177 >> 172.21.15.46:0/1739092177 conn(0x55b1ad807210 msgr2=0x55b1ad8a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 -- 172.21.15.46:0/1739092177 shutdown_connections 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.034+0000 7fcb22b77b80 1 -- 172.21.15.46:0/1739092177 wait complete. 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.035+0000 7fcb22b77b80 1 -- start start 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.035+0000 7fcb22b77b80 1 Processor -- start 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.035+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8af720 0x55b1ad8b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad795560 con 0x55b1ad8a9890 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad795200 con 0x55b1ad8aee00 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad67c6a0 con 0x55b1ad8af720 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb22b77b80 10 client.0 _ll_register_callbacks cb 0x55b1ad79eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50920/0 (socket says 172.21.15.46:50920) 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 -- 172.21.15.46:0/371286041 learned_addr learned my addr 172.21.15.46:0/371286041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07400700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8af720 0x55b1ad8b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 -- 172.21.15.46:0/371286041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8af720 msgr2=0x55b1ad8b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8af720 0x55b1ad8b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 -- 172.21.15.46:0/371286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 msgr2=0x55b1ad8b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.036+0000 7fcb07e00700 1 -- 172.21.15.46:0/371286041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf4019040 con 0x55b1ad8aee00 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.037+0000 7fcb07e00700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8015000 tx=0x7fcaf80056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2522879a675fa9e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.037+0000 7fcaf2000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.037+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8004500 con 0x55b1ad8aee00 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.037+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcaf8004660 con 0x55b1ad8aee00 2024-05-08T23:19:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.037+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1ad752270 con 0x55b1ad8aee00 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.038+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf8004820 con 0x55b1ad8aee00 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.038+0000 7fcb22b77b80 10 client.29747 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.038+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b1ad626b60 con 0x55b1ad8aee00 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1ad752270 con 0x55b1ad8aee00 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcaf802c070 con 0x55b1ad8aee00 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 10 client.29747 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 10 client.29747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 20 client.29747 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad8c31c0 0x55b1ad8c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 4 client.29747 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.039+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1ad878810 con 0x55b1ad8c31c0 2024-05-08T23:19:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.041+0000 7fcb07400700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad8c31c0 0x55b1ad8c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.042+0000 7fcb07400700 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad8c31c0 0x55b1ad8c55e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc41e0a3c101fbc4 server_cookie=1d3eac2306340ac4 in_seq=0 out_seq=0 2024-05-08T23:19:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.043+0000 7fcaf2000700 10 client.29747 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.043+0000 7fcaf2000700 1 -- 172.21.15.46:0/371286041 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1ad878810 con 0x55b1ad8c31c0 2024-05-08T23:19:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.043+0000 7fcaf2000700 10 client.29747 handle_command_reply: tid=0 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.044+0000 7fcb22b77b80 1 client.29747 shutdown 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 msgr2=0x55b1ad8b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8015000 tx=0x7fcaf80056a0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 shutdown_connections 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8af720 0x55b1ad8b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad8c31c0 0x55b1ad8c55e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/371286041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b3090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 >> 172.21.15.46:0/371286041 conn(0x55b1ad807210 msgr2=0x55b1ad8a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcaf2000700 0 client.29747 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 shutdown_connections 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.045+0000 7fcb22b77b80 1 -- 172.21.15.46:0/371286041 wait complete. 2024-05-08T23:19:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.046+0000 7fcb22b77b80 20 client.29747 trim_cache size 0 max 16384 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.075+0000 7fcb22b77b80 1 Processor -- start 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.075+0000 7fcb22b77b80 1 -- start start 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad8ca7f0 con 0x55b1ad8a9890 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad8b0b10 con 0x55b1ad8af720 2024-05-08T23:19:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad8a9690 con 0x55b1ad8aee00 2024-05-08T23:19:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb07400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb07400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35604/0 (socket says 172.21.15.46:35604) 2024-05-08T23:19:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb07400700 1 -- 172.21.15.46:0/3942512439 learned_addr learned my addr 172.21.15.46:0/3942512439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb07e00700 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.076+0000 7fcb07400700 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.077+0000 7fcb07400700 1 -- 172.21.15.46:0/3942512439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 msgr2=0x55b1ad8b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.077+0000 7fcb07400700 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.077+0000 7fcb07400700 1 -- 172.21.15.46:0/3942512439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 msgr2=0x55b1ad8adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.077+0000 7fcb07400700 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.077+0000 7fcb07400700 1 -- 172.21.15.46:0/3942512439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ad7bf560 con 0x55b1ad8a9890 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.078+0000 7fcb07400700 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011610 tx=0x7fcaf401f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732e8f041ea6218c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.078+0000 7fcafe000700 1 -- 172.21.15.46:0/3942512439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4023040 con 0x55b1ad8a9890 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.078+0000 7fcafe000700 1 -- 172.21.15.46:0/3942512439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf4035020 con 0x55b1ad8a9890 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.078+0000 7fcafe000700 1 -- 172.21.15.46:0/3942512439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401f8a0 con 0x55b1ad8a9890 2024-05-08T23:19:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.079+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3942512439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 msgr2=0x55b1ad8a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.079+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4011610 tx=0x7fcaf401f830 comp rx=0 tx=0).stop 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3942512439 shutdown_connections 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8a9890 0x55b1ad8a3d20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3942512439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3942512439 >> 172.21.15.46:0/3942512439 conn(0x55b1ad807210 msgr2=0x55b1ad8b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3942512439 shutdown_connections 2024-05-08T23:19:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.080+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3942512439 wait complete. 2024-05-08T23:19:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.081+0000 7fcb22b77b80 1 -- start start 2024-05-08T23:19:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.081+0000 7fcb22b77b80 1 Processor -- start 2024-05-08T23:19:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad921030 con 0x55b1ad8aee00 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad9211a0 con 0x55b1ad8af720 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ad921360 con 0x55b1ad8a9890 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb22b77b80 10 client.0 _ll_register_callbacks cb 0x55b1ad79eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35618/0 (socket says 172.21.15.46:35618) 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07e00700 1 -- 172.21.15.46:0/3138988265 learned_addr learned my addr 172.21.15.46:0/3138988265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 -- 172.21.15.46:0/3138988265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8a9890 msgr2=0x55b1ad8b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 -- 172.21.15.46:0/3138988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 msgr2=0x55b1ad8b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.082+0000 7fcb07400700 1 -- 172.21.15.46:0/3138988265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ad67a3b0 con 0x55b1ad8af720 2024-05-08T23:19:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.083+0000 7fcb07400700 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8b8bd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401f750 tx=0x7fcaf4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8aa06746296d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.083+0000 7fcaf2000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.083+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4023030 con 0x55b1ad8af720 2024-05-08T23:19:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.084+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcaf4036070 con 0x55b1ad8af720 2024-05-08T23:19:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.084+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf400fcf0 con 0x55b1ad8af720 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.084+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf402e430 con 0x55b1ad8af720 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.084+0000 7fcb22b77b80 10 client.29753 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b1ad67ecd0 con 0x55b1ad8af720 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf400fcf0 con 0x55b1ad8af720 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcaf4041020 con 0x55b1ad8af720 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 10 client.29753 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 10 client.29753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 20 client.29753 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad922770 0x55b1ad924b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 4 client.29753 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.085+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1ad7f1ec0 con 0x55b1ad922770 2024-05-08T23:19:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.086+0000 7fcb07e00700 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad922770 0x55b1ad924b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.087+0000 7fcb07e00700 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad922770 0x55b1ad924b90 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d94522492e91e1a server_cookie=99413c452eb56556 in_seq=0 out_seq=0 2024-05-08T23:19:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.088+0000 7fcaf2000700 10 client.29753 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.088+0000 7fcaf2000700 1 -- 172.21.15.46:0/3138988265 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1ad7f1ec0 con 0x55b1ad922770 2024-05-08T23:19:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.088+0000 7fcaf2000700 10 client.29753 handle_command_reply: tid=0 2024-05-08T23:19:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.088+0000 7fcb22b77b80 1 client.29753 shutdown 2024-05-08T23:19:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.089+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 msgr2=0x55b1ad8b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.089+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8b8bd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcaf401f750 tx=0x7fcaf4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 shutdown_connections 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1ad8af720 0x55b1ad8b8bd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1ad8a9890 0x55b1ad8b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b1ad922770 0x55b1ad924b90 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcaf2000700 0 client.29753 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 --2- 172.21.15.46:0/3138988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1ad8aee00 0x55b1ad8b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.090 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 >> 172.21.15.46:0/3138988265 conn(0x55b1ad807210 msgr2=0x55b1ad8adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 shutdown_connections 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 1 -- 172.21.15.46:0/3138988265 wait complete. 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.090+0000 7fcb22b77b80 20 client.29753 trim_cache size 0 max 16384 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 msgr2=0x7fcaec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 0x7fcaec0a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb000960e0 tx=0x7fcb00009630 comp rx=0 tx=0).stop 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 msgr2=0x7fcb08100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c370 tx=0x7fcb0000c680 comp rx=0 tx=0).stop 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 shutdown_connections 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb0810b3c0 0x7fcb08100670 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcaec0a1310 0x7fcaec0a37d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb08101fa0 0x7fcb080ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 --2- 172.21.15.46:0/3705820153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb081028c0 0x7fcb08100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.092+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 >> 172.21.15.46:0/3705820153 conn(0x7fcb080fb760 msgr2=0x7fcb080fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.093+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 shutdown_connections 2024-05-08T23:19:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:23.093+0000 7fcb0f000700 1 -- 172.21.15.46:0/3705820153 wait complete. 2024-05-08T23:19:23.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 41665 23.24 MB/sec warmup 104 sec latency 354.428 ms 2024-05-08T23:19:24.125 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:19:24.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:19:24.201 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:23 smithi046 ceph-mon[36898]: pgmap v488: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-05-08T23:19:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:23 smithi086 ceph-mon[78610]: pgmap v488: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-05-08T23:19:24.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 42166 23.31 MB/sec warmup 105 sec latency 219.567 ms 2024-05-08T23:19:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:23 smithi053 ceph-mon[77587]: pgmap v488: 129 pgs: 129 active+clean; 931 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 -- 172.21.15.46:0/924268137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4111e20 msgr2=0x7fcfe4114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 --2- 172.21.15.46:0/924268137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4111e20 0x7fcfe4114210 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009f90 tx=0x7fcfdc00b330 comp rx=0 tx=0).stop 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 -- 172.21.15.46:0/924268137 shutdown_connections 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 --2- 172.21.15.46:0/924268137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4111e20 0x7fcfe4114210 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 --2- 172.21.15.46:0/924268137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073340 0x7fcfe4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 --2- 172.21.15.46:0/924268137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4073c60 0x7fcfe41118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 -- 172.21.15.46:0/924268137 >> 172.21.15.46:0/924268137 conn(0x7fcfe40fc8e0 msgr2=0x7fcfe40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 -- 172.21.15.46:0/924268137 shutdown_connections 2024-05-08T23:19:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.388+0000 7fcfec400700 1 -- 172.21.15.46:0/924268137 wait complete. 2024-05-08T23:19:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.389+0000 7fcfec400700 1 Processor -- start 2024-05-08T23:19:24.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.389+0000 7fcfec400700 1 -- start start 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4073340 0x7fcfe41a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 0x7fcfe419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe419dd90 con 0x7fcfe4073c60 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe419df00 con 0x7fcfe4111e20 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfec400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe419e0c0 con 0x7fcfe4073340 2024-05-08T23:19:24.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35632/0 (socket says 172.21.15.46:35632) 2024-05-08T23:19:24.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfe9000700 1 -- 172.21.15.46:0/4072562924 learned_addr learned my addr 172.21.15.46:0/4072562924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfea400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 0x7fcfe419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.390+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4073340 0x7fcfe41a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 -- 172.21.15.46:0/4072562924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4073340 msgr2=0x7fcfe41a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4073340 0x7fcfe41a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 -- 172.21.15.46:0/4072562924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 msgr2=0x7fcfe419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 0x7fcfe419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 -- 172.21.15.46:0/4072562924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80095f0 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfea400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 0x7fcfe419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe9000700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000eb30 tx=0x7fcfd000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe2000700 1 -- 172.21.15.46:0/4072562924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0009880 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd0009520 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe2000700 1 -- 172.21.15.46:0/4072562924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfd00044e0 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfe2000700 1 -- 172.21.15.46:0/4072562924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0003d80 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.391+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc009c40 con 0x7fcfe4073c60 2024-05-08T23:19:24.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.394+0000 7fcfe2000700 1 -- 172.21.15.46:0/4072562924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcfd0004000 con 0x7fcfe4073c60 2024-05-08T23:19:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcfe2000700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 0x7fcfcc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcffffc0b80 1 Processor -- start 2024-05-08T23:19:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcfe2000700 1 -- 172.21.15.46:0/4072562924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcfd0014070 con 0x7fcfe4073c60 2024-05-08T23:19:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcffffc0b80 1 -- start start 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 0x7fcfcc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf000260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.396+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceffa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcef511d0 con 0x55ffcf0007a0 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcef4eb90 con 0x55ffcefffe80 2024-05-08T23:19:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcef4a480 con 0x55ffceffa910 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceffa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47014/0 (socket says 172.21.15.46:47014) 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 -- 172.21.15.46:0/2152327586 learned_addr learned my addr 172.21.15.46:0/2152327586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf000260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 -- 172.21.15.46:0/2152327586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 msgr2=0x55ffcf000260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf000260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 0x7fcfcc0a37d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd8011720 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 -- 172.21.15.46:0/2152327586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 msgr2=0x55ffceffa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceffa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfea400700 1 -- 172.21.15.46:0/2152327586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffcedcb430 con 0x55ffceffa910 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.397+0000 7fcfe9000700 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceffa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.398+0000 7fcfea400700 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc008f40 tx=0x7fcfdc008f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5aef982ee44e002 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.398+0000 7fcfd6a00700 1 -- 172.21.15.46:0/2152327586 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00e040 con 0x55ffceffa910 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.398+0000 7fcfd6a00700 1 -- 172.21.15.46:0/2152327586 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfdc003c80 con 0x55ffceffa910 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.398+0000 7fcfd6a00700 1 -- 172.21.15.46:0/2152327586 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0366c0 con 0x55ffceffa910 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 -- 172.21.15.46:0/2152327586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 msgr2=0x55ffceffad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc008f40 tx=0x7fcfdc008f70 comp rx=0 tx=0).stop 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 -- 172.21.15.46:0/2152327586 shutdown_connections 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffad50 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf000260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/2152327586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceffa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 -- 172.21.15.46:0/2152327586 >> 172.21.15.46:0/2152327586 conn(0x55ffcef58290 msgr2=0x55ffceff4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 -- 172.21.15.46:0/2152327586 shutdown_connections 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.399+0000 7fcffffc0b80 1 -- 172.21.15.46:0/2152327586 wait complete. 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.400+0000 7fcffffc0b80 1 -- start start 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.400+0000 7fcffffc0b80 1 Processor -- start 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 0x55ffcf004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffcf004660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffceee65e0 con 0x55ffcf0007a0 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffceee6280 con 0x55ffceffa910 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcedcd720 con 0x55ffcefffe80 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcffffc0b80 10 client.0 _ll_register_callbacks cb 0x55ffceeefbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.401+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50952/0 (socket says 172.21.15.46:50952) 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3339672412 learned_addr learned my addr 172.21.15.46:0/3339672412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9000700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 0x55ffcf004120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3339672412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 msgr2=0x55ffcf004120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 0x55ffcf004120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3339672412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 msgr2=0x55ffcf004660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffcf004660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3339672412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc009c40 con 0x55ffceffa910 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9000700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 0x55ffcf004120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800f9d0 tx=0x7fcfd80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab013f7911f3222 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.402+0000 7fcfcb400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd801ec70 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcfd801edd0 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffceea32f0 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfd8027440 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcffffc0b80 10 client.29765 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fcfe4114bc0 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.403+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffceea32f0 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcfd80264e0 con 0x55ffceffa910 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 10 client.29765 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 10 client.29765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 20 client.29765 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:24.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf014120 0x55ffcf016540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 4 client.29765 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffcefc9890 con 0x55ffcf014120 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.404+0000 7fcfe9000700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf014120 0x55ffcf016540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.405+0000 7fcfe9000700 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf014120 0x55ffcf016540 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3c2ffacbf4c426 server_cookie=68c8fd4d23b952fc in_seq=0 out_seq=0 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.405+0000 7fcfcb400700 10 client.29765 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.405+0000 7fcfcb400700 1 -- 172.21.15.46:0/3339672412 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffcefc9890 con 0x55ffcf014120 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.405+0000 7fcfcb400700 10 client.29765 handle_command_reply: tid=0 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.405+0000 7fcffffc0b80 1 client.29765 shutdown 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 msgr2=0x55ffcf003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800f9d0 tx=0x7fcfd80047c0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 shutdown_connections 2024-05-08T23:19:24.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf014120 0x55ffcf016540 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffcf004660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffceffa910 0x55ffcf003be0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3339672412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcefffe80 0x55ffcf004120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 >> 172.21.15.46:0/3339672412 conn(0x55ffcef58290 msgr2=0x55ffceff4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcfcb400700 0 client.29765 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 shutdown_connections 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.406+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3339672412 wait complete. 2024-05-08T23:19:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.407+0000 7fcffffc0b80 20 client.29765 trim_cache size 0 max 16384 2024-05-08T23:19:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.430+0000 7fcffffc0b80 1 Processor -- start 2024-05-08T23:19:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 -- start start 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcefff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceff4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffceff1e40 con 0x55ffcf0007a0 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcf006bb0 con 0x55ffcefffe80 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcefffad0 con 0x55ffceffa910 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47024/0 (socket says 172.21.15.46:47024) 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3496764356 learned_addr learned my addr 172.21.15.46:0/3496764356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcefff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.431+0000 7fcfea400700 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceff4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3496764356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 msgr2=0x55ffcefff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcefff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3496764356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 msgr2=0x55ffceff4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceff4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 -- 172.21.15.46:0/3496764356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffceff1560 con 0x55ffceffa910 2024-05-08T23:19:24.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.432+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8026840 tx=0x7fcfd8009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54b959c156be0d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.433+0000 7fcfd7400700 1 -- 172.21.15.46:0/3496764356 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8005d40 con 0x55ffceffa910 2024-05-08T23:19:24.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.433+0000 7fcfd7400700 1 -- 172.21.15.46:0/3496764356 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfd8026a60 con 0x55ffceffa910 2024-05-08T23:19:24.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.433+0000 7fcfd7400700 1 -- 172.21.15.46:0/3496764356 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80276a0 con 0x55ffceffa910 2024-05-08T23:19:24.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3496764356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 msgr2=0x55ffceffa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8026840 tx=0x7fcfd8009290 comp rx=0 tx=0).stop 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3496764356 shutdown_connections 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffcf0007a0 0x55ffceff4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffceffa910 0x55ffceffa3d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/3496764356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcefff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3496764356 >> 172.21.15.46:0/3496764356 conn(0x55ffcef58290 msgr2=0x55ffceffecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.434+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3496764356 shutdown_connections 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.435+0000 7fcffffc0b80 1 -- 172.21.15.46:0/3496764356 wait complete. 2024-05-08T23:19:24.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.435+0000 7fcffffc0b80 1 -- start start 2024-05-08T23:19:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.436+0000 7fcffffc0b80 1 Processor -- start 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.436+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 0x55ffcf05e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.436+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.436+0000 7fcffffc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcf0007a0 0x55ffcf062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcf0631b0 con 0x55ffceffa910 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcf063320 con 0x55ffcefffe80 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcffffc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffcf0634e0 con 0x55ffcf0007a0 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcffffc0b80 10 client.0 _ll_register_callbacks cb 0x55ffceeefbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfea400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcf0007a0 0x55ffcf062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50970/0 (socket says 172.21.15.46:50970) 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 -- 172.21.15.46:0/791823805 learned_addr learned my addr 172.21.15.46:0/791823805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 0x55ffcf05e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 -- 172.21.15.46:0/791823805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcf0007a0 msgr2=0x55ffcf062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcf0007a0 0x55ffcf062c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 -- 172.21.15.46:0/791823805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 msgr2=0x55ffcf05e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.437+0000 7fcfe9000700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 0x55ffcf05e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfe9000700 1 -- 172.21.15.46:0/791823805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4114bc0 con 0x55ffcefffe80 2024-05-08T23:19:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 0x55ffcf05e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfe9000700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009a1e0 tx=0x7fcfd0099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de18aebf5491f629 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfcb400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00b1040 con 0x55ffcefffe80 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fcfd00b0040 con 0x55ffcefffe80 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.438+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfdc003c80 con 0x55ffcefffe80 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfd000cdd0 con 0x55ffcefffe80 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 10 client.29771 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ffced77be0 con 0x55ffcefffe80 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfdc003c80 con 0x55ffcefffe80 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcfd0099740 con 0x55ffcefffe80 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 10 client.29771 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 10 client.29771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 20 client.29771 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf06bf30 0x55ffcf06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 4 client.29771 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.439+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ffcef42f40 con 0x55ffcf06bf30 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.440+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf06bf30 0x55ffcf06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.440+0000 7fcfe9a00700 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf06bf30 0x55ffcf06e350 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75658569e29f8f4e server_cookie=f52f7095265697b4 in_seq=0 out_seq=0 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.440+0000 7fcfcb400700 10 client.29771 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcfcb400700 1 -- 172.21.15.46:0/791823805 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ffcef42f40 con 0x55ffcf06bf30 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcfcb400700 10 client.29771 handle_command_reply: tid=0 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 client.29771 shutdown 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 msgr2=0x55ffcf05ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009a1e0 tx=0x7fcfd0099eb0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 shutdown_connections 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffcf0007a0 0x55ffcf062c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ffcf06bf30 0x55ffcf06e350 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffceffa910 0x55ffcf05e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 --2- 172.21.15.46:0/791823805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffcefffe80 0x55ffcf05ed20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 >> 172.21.15.46:0/791823805 conn(0x55ffcef58290 msgr2=0x55ffcf000b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.442+0000 7fcfcb400700 0 client.29771 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.443+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 shutdown_connections 2024-05-08T23:19:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.443+0000 7fcffffc0b80 1 -- 172.21.15.46:0/791823805 wait complete. 2024-05-08T23:19:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.443+0000 7fcffffc0b80 20 client.29771 trim_cache size 0 max 16384 2024-05-08T23:19:24.465 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:19:24.466 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:19:24.466 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "8ff2e465-51ac-4918-8e5b-da73b1d03449", 2024-05-08T23:19:24.466 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:19:24.466 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 msgr2=0x7fcfcc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 0x7fcfcc0a37d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd8011720 comp rx=0 tx=0).stop 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 msgr2=0x7fcfe41a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000eb30 tx=0x7fcfd000ee40 comp rx=0 tx=0).stop 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 shutdown_connections 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe4111e20 0x7fcfe419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe4073340 0x7fcfe41a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcfcc0a1310 0x7fcfcc0a37d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 --2- 172.21.15.46:0/4072562924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe4073c60 0x7fcfe41a3b40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 >> 172.21.15.46:0/4072562924 conn(0x7fcfe40fc8e0 msgr2=0x7fcfe40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 shutdown_connections 2024-05-08T23:19:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.444+0000 7fcfec400700 1 -- 172.21.15.46:0/4072562924 wait complete. 2024-05-08T23:19:24.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 -- 172.21.15.46:0/3390359316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc101a30 msgr2=0x7faecc105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 --2- 172.21.15.46:0/3390359316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc101a30 0x7faecc105a70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009990 tx=0x7faeb8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 -- 172.21.15.46:0/3390359316 shutdown_connections 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 --2- 172.21.15.46:0/3390359316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 --2- 172.21.15.46:0/3390359316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 --2- 172.21.15.46:0/3390359316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc101a30 0x7faecc105a70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 -- 172.21.15.46:0/3390359316 >> 172.21.15.46:0/3390359316 conn(0x7faecc0fc8e0 msgr2=0x7faecc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 -- 172.21.15.46:0/3390359316 shutdown_connections 2024-05-08T23:19:24.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.737+0000 7faed1e00700 1 -- 172.21.15.46:0/3390359316 wait complete. 2024-05-08T23:19:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.738+0000 7faed1e00700 1 Processor -- start 2024-05-08T23:19:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.738+0000 7faed1e00700 1 -- start start 2024-05-08T23:19:24.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc19de10 0x7faecc19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc19e890 con 0x7faecc101080 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc19ea00 con 0x7faecc1061a0 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faed1e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc19ebc0 con 0x7faecc19de10 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faecaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faeca000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faeca000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50986/0 (socket says 172.21.15.46:50986) 2024-05-08T23:19:24.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faeca000700 1 -- 172.21.15.46:0/46449944 learned_addr learned my addr 172.21.15.46:0/46449944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.739+0000 7faecb400700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc19de10 0x7faecc19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faeca000700 1 -- 172.21.15.46:0/46449944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc19de10 msgr2=0x7faecc19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faeca000700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc19de10 0x7faecc19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faeca000700 1 -- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 msgr2=0x7faecc199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faeca000700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faeca000700 1 -- 172.21.15.46:0/46449944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80095f0 con 0x7faecc1061a0 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.740+0000 7faecaa00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.742+0000 7faeca000700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.743+0000 7faed0a00700 1 -- 172.21.15.46:0/46449944 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb801d070 con 0x7faecc1061a0 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.743+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc1a60c0 con 0x7faecc1061a0 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.743+0000 7faed0a00700 1 -- 172.21.15.46:0/46449944 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeb8022470 con 0x7faecc1061a0 2024-05-08T23:19:24.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.743+0000 7faed0a00700 1 -- 172.21.15.46:0/46449944 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8017650 con 0x7faecc1061a0 2024-05-08T23:19:24.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.743+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc1a6540 con 0x7faecc1061a0 2024-05-08T23:19:24.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.748+0000 7faed0a00700 1 -- 172.21.15.46:0/46449944 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faeb80177b0 con 0x7faecc1061a0 2024-05-08T23:19:24.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.749+0000 7faee5874b80 1 Processor -- start 2024-05-08T23:19:24.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.749+0000 7faee5874b80 1 -- start start 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c039150 con 0x561b7c0f0610 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c036b10 con 0x561b7c0eb0a0 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faee5874b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c032400 con 0x561b7c0f0f30 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faed0a00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 0x7faeb00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.750+0000 7faed0a00700 1 -- 172.21.15.46:0/46449944 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faeb80ce3a0 con 0x7faecc1061a0 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35692/0 (socket says 172.21.15.46:35692) 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecb400700 1 -- 172.21.15.46:0/3525557496 learned_addr learned my addr 172.21.15.46:0/3525557496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecb400700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 0x7faeb00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faeca000700 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 -- 172.21.15.46:0/3525557496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 msgr2=0x561b7c0eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 -- 172.21.15.46:0/3525557496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 msgr2=0x561b7c0f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.751+0000 7faecaa00700 1 -- 172.21.15.46:0/3525557496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b7beb33b0 con 0x561b7c0f0f30 2024-05-08T23:19:24.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.752+0000 7faecaa00700 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0eab60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faebc0114f0 tx=0x7faebc011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c585db95981254d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.752+0000 7faeb7e00700 1 -- 172.21.15.46:0/3525557496 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc004230 con 0x561b7c0f0f30 2024-05-08T23:19:24.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.752+0000 7faeb7e00700 1 -- 172.21.15.46:0/3525557496 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faebc004390 con 0x561b7c0f0f30 2024-05-08T23:19:24.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.752+0000 7faeb7e00700 1 -- 172.21.15.46:0/3525557496 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc00e850 con 0x561b7c0f0f30 2024-05-08T23:19:24.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 -- 172.21.15.46:0/3525557496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 msgr2=0x561b7c0eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0eab60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faebc0114f0 tx=0x7faebc011800 comp rx=0 tx=0).stop 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faecb400700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 0x7faeb00a37d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faec4009fd0 tx=0x7faec400b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 -- 172.21.15.46:0/3525557496 shutdown_connections 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 --2- 172.21.15.46:0/3525557496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0eab60 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.753+0000 7faee5874b80 1 -- 172.21.15.46:0/3525557496 >> 172.21.15.46:0/3525557496 conn(0x561b7c040210 msgr2=0x561b7c0dcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.754+0000 7faee5874b80 1 -- 172.21.15.46:0/3525557496 shutdown_connections 2024-05-08T23:19:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.754+0000 7faee5874b80 1 -- 172.21.15.46:0/3525557496 wait complete. 2024-05-08T23:19:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.755+0000 7faee5874b80 1 -- start start 2024-05-08T23:19:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.756+0000 7faee5874b80 1 Processor -- start 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7bfce560 con 0x561b7c0f0610 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7bfce200 con 0x561b7c0eb0a0 2024-05-08T23:19:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7beb56a0 con 0x561b7c0f0f30 2024-05-08T23:19:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faee5874b80 10 client.0 _ll_register_callbacks cb 0x561b7bfd7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faecb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faecb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47080/0 (socket says 172.21.15.46:47080) 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faecb400700 1 -- 172.21.15.46:0/2577604988 learned_addr learned my addr 172.21.15.46:0/2577604988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.758+0000 7faeca000700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 -- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 msgr2=0x561b7c0f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 -- 172.21.15.46:0/2577604988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 msgr2=0x561b7c0f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 -- 172.21.15.46:0/2577604988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc009c90 con 0x561b7c0f0f30 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faecb400700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faec4018c80 tx=0x7faec4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e64f2a25192d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.759+0000 7faeb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4020070 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faec4003a00 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b7bf8b270 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faec400e030 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faee5874b80 10 client.29857 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.760+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b7be5fb60 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b7bf8b270 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faec4004150 con 0x561b7c0f0f30 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 10 client.29857 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 10 client.29857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 20 client.29857 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c104a40 0x561b7c106e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 4 client.29857 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b7c0b1810 con 0x561b7c104a40 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.761+0000 7faecaa00700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c104a40 0x561b7c106e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.762+0000 7faecaa00700 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c104a40 0x561b7c106e60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb448c1215a7ac5 server_cookie=5c88f73cd61d978 in_seq=0 out_seq=0 2024-05-08T23:19:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.762+0000 7faeb4c00700 10 client.29857 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.763+0000 7faeb4c00700 1 -- 172.21.15.46:0/2577604988 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b7c0b1810 con 0x561b7c104a40 2024-05-08T23:19:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.763+0000 7faeb4c00700 10 client.29857 handle_command_reply: tid=0 2024-05-08T23:19:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.763+0000 7faee5874b80 1 client.29857 shutdown 2024-05-08T23:19:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.764+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 msgr2=0x561b7c0f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.764+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faec4018c80 tx=0x7faec4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 shutdown_connections 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0f4db0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c104a40 0x561b7c106e60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 --2- 172.21.15.46:0/2577604988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0610 0x561b7c0f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 >> 172.21.15.46:0/2577604988 conn(0x561b7c040210 msgr2=0x561b7c0dc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faeb4c00700 0 client.29857 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.765+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 shutdown_connections 2024-05-08T23:19:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.766+0000 7faee5874b80 1 -- 172.21.15.46:0/2577604988 wait complete. 2024-05-08T23:19:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.766+0000 7faee5874b80 20 client.29857 trim_cache size 0 max 16384 2024-05-08T23:19:24.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.792+0000 7faee5874b80 1 Processor -- start 2024-05-08T23:19:24.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 -- start start 2024-05-08T23:19:24.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0610 0x561b7c0dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 0x561b7c0f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c0f5790 con 0x561b7c0f0f30 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c0f5930 con 0x561b7c0eb0a0 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.793+0000 7faee5874b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c0d9dc0 con 0x561b7c0f0610 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecaa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 0x561b7c0f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecaa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51008/0 (socket says 172.21.15.46:51008) 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faeca000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0610 0x561b7c0dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecaa00700 1 -- 172.21.15.46:0/3287911552 learned_addr learned my addr 172.21.15.46:0/3287911552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 -- 172.21.15.46:0/3287911552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0610 msgr2=0x561b7c0dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0610 0x561b7c0dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 -- 172.21.15.46:0/3287911552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 msgr2=0x561b7c0eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecb400700 1 -- 172.21.15.46:0/3287911552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b7bff8560 con 0x561b7c0f0f30 2024-05-08T23:19:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.794+0000 7faecaa00700 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.795+0000 7faecb400700 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 0x561b7c0f2c00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faec40045b0 tx=0x7faec4009140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3618cecfd500f2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.795+0000 7faec1200700 1 -- 172.21.15.46:0/3287911552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4020070 con 0x561b7c0f0f30 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.795+0000 7faec1200700 1 -- 172.21.15.46:0/3287911552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faec400f040 con 0x561b7c0f0f30 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.795+0000 7faec1200700 1 -- 172.21.15.46:0/3287911552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4004bb0 con 0x561b7c0f0f30 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.796+0000 7faee5874b80 1 -- 172.21.15.46:0/3287911552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 msgr2=0x561b7c0f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.796+0000 7faee5874b80 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 0x561b7c0f2c00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faec40045b0 tx=0x7faec4009140 comp rx=0 tx=0).stop 2024-05-08T23:19:24.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 -- 172.21.15.46:0/3287911552 shutdown_connections 2024-05-08T23:19:24.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0f0f30 0x561b7c0f2c00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0eb0a0 0x561b7c0eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 --2- 172.21.15.46:0/3287911552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0610 0x561b7c0dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 -- 172.21.15.46:0/3287911552 >> 172.21.15.46:0/3287911552 conn(0x561b7c040210 msgr2=0x561b7c0ee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 -- 172.21.15.46:0/3287911552 shutdown_connections 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 -- 172.21.15.46:0/3287911552 wait complete. 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 -- start start 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.797+0000 7faee5874b80 1 Processor -- start 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 0x561b7c0f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c15a010 con 0x561b7c0eb0a0 2024-05-08T23:19:24.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c15a180 con 0x561b7c0f0610 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b7c15a340 con 0x561b7c0f0f30 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faee5874b80 10 client.0 _ll_register_callbacks cb 0x561b7bfd7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faeca000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 0x561b7c0f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47096/0 (socket says 172.21.15.46:47096) 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faeca000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 0x561b7c0f9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51016/0 (socket says 172.21.15.46:51016) 2024-05-08T23:19:24.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 -- 172.21.15.46:0/3118985530 learned_addr learned my addr 172.21.15.46:0/3118985530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:24.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecaa00700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 -- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 msgr2=0x561b7c0f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 0x561b7c0f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 -- 172.21.15.46:0/3118985530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 msgr2=0x561b7c0f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecb400700 1 -- 172.21.15.46:0/3118985530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b7beb33b0 con 0x561b7c0f0f30 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.798+0000 7faecaa00700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faecb400700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faec4004a30 tx=0x7faec4009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43de01f23966237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faeb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4020040 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faebc004390 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faec401b970 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faec401bb30 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faee5874b80 10 client.29863 fetch_fsmap learned FSMap version 32 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.799+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b7be5fb60 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faebc004390 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faec4030420 con 0x561b7c0f0f30 2024-05-08T23:19:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 10 client.29863 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 10 client.29863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 20 client.29863 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c15b790 0x561b7c15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 4 client.29863 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b7c02aec0 con 0x561b7c15b790 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.800+0000 7faecaa00700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c15b790 0x561b7c15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faeb4c00700 10 client.29863 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faecaa00700 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c15b790 0x561b7c15dbb0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e7ef392676ad1d server_cookie=77ae4173c3df0a8a in_seq=0 out_seq=0 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faeb4c00700 1 -- 172.21.15.46:0/3118985530 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561b7c02aec0 con 0x561b7c15b790 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faeb4c00700 10 client.29863 handle_command_reply: tid=0 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 client.29863 shutdown 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 msgr2=0x561b7c0fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faec4004a30 tx=0x7faec4009490 comp rx=0 tx=0).stop 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 shutdown_connections 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7c0f0f30 0x561b7c0fa440 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561b7c15b790 0x561b7c15dbb0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b7c0eb0a0 0x561b7c0f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 --2- 172.21.15.46:0/3118985530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561b7c0f0610 0x561b7c0f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 >> 172.21.15.46:0/3118985530 conn(0x561b7c040210 msgr2=0x561b7c0f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.801+0000 7faeb4c00700 0 client.29863 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.802+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 shutdown_connections 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.802+0000 7faee5874b80 1 -- 172.21.15.46:0/3118985530 wait complete. 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.802+0000 7faee5874b80 20 client.29863 trim_cache size 0 max 16384 2024-05-08T23:19:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "8ff2e465-51ac-4918-8e5b-da73b1d03449": { 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "8ff2e465-51ac-4918-8e5b-da73b1d03449", 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:19:24.808 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.803+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 msgr2=0x7faeb00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.803+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 0x7faeb00a37d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faec4009fd0 tx=0x7faec400b040 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.803+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 msgr2=0x7faecc199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.803+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb8005740 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 shutdown_connections 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faecc19de10 0x7faecc19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faeb00a1310 0x7faeb00a37d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faecc101080 0x7faecc199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 --2- 172.21.15.46:0/46449944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faecc1061a0 0x7faecc199a70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 >> 172.21.15.46:0/46449944 conn(0x7faecc0fc8e0 msgr2=0x7faecc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 shutdown_connections 2024-05-08T23:19:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:24.804+0000 7faed1e00700 1 -- 172.21.15.46:0/46449944 wait complete. 2024-05-08T23:19:24.824 INFO:tasks.cephfs.filesystem:scrub status for tag:8ff2e465-51ac-4918-8e5b-da73b1d03449 - {'path': '/', 'tag': '8ff2e465-51ac-4918-8e5b-da73b1d03449', 'options': 'recursive,force'} 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:25.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 42608 23.31 MB/sec warmup 106 sec latency 71.045 ms 2024-05-08T23:19:25.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: pgmap v489: 129 pgs: 129 active+clean; 925 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:25 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.254 INFO:tasks.workunit.client.0.smithi046.stdout: 10 43118 23.34 MB/sec warmup 107 sec latency 52.834 ms 2024-05-08T23:19:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: pgmap v489: 129 pgs: 129 active+clean; 925 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:19:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:25 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: pgmap v489: 129 pgs: 129 active+clean; 925 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 542 op/s 2024-05-08T23:19:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:19:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:19:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:25 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:27.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 43567 23.36 MB/sec warmup 108 sec latency 224.497 ms 2024-05-08T23:19:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:27 smithi086 ceph-mon[78610]: pgmap v490: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:28.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 43970 23.34 MB/sec warmup 109 sec latency 247.472 ms 2024-05-08T23:19:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:27 smithi053 ceph-mon[77587]: pgmap v490: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:28.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:27 smithi046 ceph-mon[36898]: pgmap v490: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:29.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 44542 23.42 MB/sec warmup 110 sec latency 154.535 ms 2024-05-08T23:19:30.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:29 smithi086 ceph-mon[78610]: pgmap v491: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:30.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 45027 23.41 MB/sec warmup 111 sec latency 190.278 ms 2024-05-08T23:19:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:29 smithi053 ceph-mon[77587]: pgmap v491: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:29 smithi046 ceph-mon[36898]: pgmap v491: 129 pgs: 129 active+clean; 916 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-05-08T23:19:31.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 45525 23.46 MB/sec warmup 112 sec latency 196.596 ms 2024-05-08T23:19:32.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:31 smithi086 ceph-mon[78610]: pgmap v492: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 446 op/s 2024-05-08T23:19:32.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:31 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:32.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 45938 23.51 MB/sec warmup 113 sec latency 280.980 ms 2024-05-08T23:19:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:31 smithi053 ceph-mon[77587]: pgmap v492: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 446 op/s 2024-05-08T23:19:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:31 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:31 smithi046 ceph-mon[36898]: pgmap v492: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 446 op/s 2024-05-08T23:19:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:31 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:33.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 46363 23.44 MB/sec warmup 114 sec latency 141.850 ms 2024-05-08T23:19:34.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:33 smithi086 ceph-mon[78610]: pgmap v493: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-08T23:19:34.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:33 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:34.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 46797 23.42 MB/sec warmup 115 sec latency 170.925 ms 2024-05-08T23:19:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:33 smithi053 ceph-mon[77587]: pgmap v493: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-08T23:19:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:33 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:33 smithi046 ceph-mon[36898]: pgmap v493: 129 pgs: 129 active+clean; 910 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-08T23:19:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:33 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:35.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 47263 23.40 MB/sec warmup 116 sec latency 183.577 ms 2024-05-08T23:19:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:35 smithi086 ceph-mon[78610]: pgmap v494: 129 pgs: 129 active+clean; 918 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-05-08T23:19:36.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 47721 23.49 MB/sec warmup 117 sec latency 215.733 ms 2024-05-08T23:19:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:35 smithi053 ceph-mon[77587]: pgmap v494: 129 pgs: 129 active+clean; 918 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-05-08T23:19:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:35 smithi046 ceph-mon[36898]: pgmap v494: 129 pgs: 129 active+clean; 918 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-05-08T23:19:37.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 48316 23.50 MB/sec warmup 118 sec latency 148.495 ms 2024-05-08T23:19:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:37 smithi086 ceph-mon[78610]: pgmap v495: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 350 op/s 2024-05-08T23:19:38.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 48760 23.56 MB/sec warmup 119 sec latency 212.526 ms 2024-05-08T23:19:38.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:37 smithi053 ceph-mon[77587]: pgmap v495: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 350 op/s 2024-05-08T23:19:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:37 smithi046 ceph-mon[36898]: pgmap v495: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 350 op/s 2024-05-08T23:19:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:39 smithi086 ceph-mon[78610]: pgmap v496: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-08T23:19:40.255 INFO:tasks.workunit.client.0.smithi046.stdout: 10 49710 29.59 MB/sec execute 1 sec latency 257.168 ms 2024-05-08T23:19:40.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:39 smithi053 ceph-mon[77587]: pgmap v496: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-08T23:19:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:39 smithi046 ceph-mon[36898]: pgmap v496: 129 pgs: 129 active+clean; 920 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-08T23:19:41.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:41.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:41.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:41.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:40 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:19:41.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 50132 26.49 MB/sec execute 2 sec latency 179.432 ms 2024-05-08T23:19:41.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:41.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:41.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:41.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:40 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:19:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:40 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:19:42.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:41 smithi086 ceph-mon[78610]: pgmap v497: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 6.4 MiB/s wr, 481 op/s 2024-05-08T23:19:42.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 50509 22.30 MB/sec execute 3 sec latency 233.055 ms 2024-05-08T23:19:42.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:41 smithi053 ceph-mon[77587]: pgmap v497: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 6.4 MiB/s wr, 481 op/s 2024-05-08T23:19:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:41 smithi046 ceph-mon[36898]: pgmap v497: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 6.4 MiB/s wr, 481 op/s 2024-05-08T23:19:43.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 50995 23.21 MB/sec execute 4 sec latency 290.938 ms 2024-05-08T23:19:44.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:43 smithi086 ceph-mon[78610]: pgmap v498: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-08T23:19:44.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 51505 24.65 MB/sec execute 5 sec latency 98.284 ms 2024-05-08T23:19:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:43 smithi053 ceph-mon[77587]: pgmap v498: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-08T23:19:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:43 smithi046 ceph-mon[36898]: pgmap v498: 129 pgs: 129 active+clean; 931 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-08T23:19:45.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 52171 26.37 MB/sec execute 6 sec latency 200.890 ms 2024-05-08T23:19:46.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:45 smithi086 ceph-mon[78610]: pgmap v499: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.6 MiB/s wr, 430 op/s 2024-05-08T23:19:46.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 52571 26.73 MB/sec execute 7 sec latency 162.790 ms 2024-05-08T23:19:46.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:45 smithi053 ceph-mon[77587]: pgmap v499: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.6 MiB/s wr, 430 op/s 2024-05-08T23:19:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:45 smithi046 ceph-mon[36898]: pgmap v499: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.6 MiB/s wr, 430 op/s 2024-05-08T23:19:47.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53001 26.44 MB/sec execute 8 sec latency 195.923 ms 2024-05-08T23:19:48.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:47 smithi086 ceph-mon[78610]: pgmap v500: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-05-08T23:19:48.256 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53407 25.73 MB/sec execute 9 sec latency 213.132 ms 2024-05-08T23:19:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:47 smithi053 ceph-mon[77587]: pgmap v500: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-05-08T23:19:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:47 smithi046 ceph-mon[36898]: pgmap v500: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-05-08T23:19:49.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:48 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:49.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53663 24.22 MB/sec execute 10 sec latency 467.281 ms 2024-05-08T23:19:49.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:48 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:49.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:48 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:50.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:49 smithi086 ceph-mon[78610]: pgmap v501: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-05-08T23:19:50.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53681 22.02 MB/sec execute 11 sec latency 1157.710 ms 2024-05-08T23:19:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:49 smithi053 ceph-mon[77587]: pgmap v501: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-05-08T23:19:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:49 smithi046 ceph-mon[36898]: pgmap v501: 129 pgs: 129 active+clean; 938 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-05-08T23:19:51.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53840 21.01 MB/sec execute 12 sec latency 770.638 ms 2024-05-08T23:19:52.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 54161 20.39 MB/sec execute 13 sec latency 159.516 ms 2024-05-08T23:19:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:51 smithi053 ceph-mon[77587]: pgmap v502: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-05-08T23:19:52.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:51 smithi046 ceph-mon[36898]: pgmap v502: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-05-08T23:19:52.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:51 smithi086 ceph-mon[78610]: pgmap v502: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 534 op/s 2024-05-08T23:19:53.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 54463 20.32 MB/sec execute 14 sec latency 77.935 ms 2024-05-08T23:19:54.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 54892 20.33 MB/sec execute 15 sec latency 189.744 ms 2024-05-08T23:19:54.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:53 smithi053 ceph-mon[77587]: pgmap v503: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 364 op/s 2024-05-08T23:19:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:53 smithi046 ceph-mon[36898]: pgmap v503: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 364 op/s 2024-05-08T23:19:54.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:53 smithi086 ceph-mon[78610]: pgmap v503: 129 pgs: 129 active+clean; 994 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 364 op/s 2024-05-08T23:19:54.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:19:55.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 55010 19.36 MB/sec execute 16 sec latency 487.968 ms 2024-05-08T23:19:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 -- 172.21.15.46:0/593962669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781032a0 msgr2=0x7f5c78103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 --2- 172.21.15.46:0/593962669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781032a0 0x7f5c78103680 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b0a0 tx=0x7f5c6800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 -- 172.21.15.46:0/593962669 shutdown_connections 2024-05-08T23:19:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 --2- 172.21.15.46:0/593962669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c781083c0 0x7f5c7810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 --2- 172.21.15.46:0/593962669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781032a0 0x7f5c78103680 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 --2- 172.21.15.46:0/593962669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c78103c50 0x7f5c78107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.308+0000 7f5c7da00700 1 -- 172.21.15.46:0/593962669 >> 172.21.15.46:0/593962669 conn(0x7f5c780feae0 msgr2=0x7f5c78100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.309+0000 7f5c7da00700 1 -- 172.21.15.46:0/593962669 shutdown_connections 2024-05-08T23:19:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.309+0000 7f5c7da00700 1 -- 172.21.15.46:0/593962669 wait complete. 2024-05-08T23:19:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.310+0000 7f5c7da00700 1 Processor -- start 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 -- start start 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781083c0 0x7f5c780759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 0x7f5c781a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780796b0 con 0x7f5c78075f30 2024-05-08T23:19:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78199780 con 0x7f5c781032a0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c7da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78199940 con 0x7f5c781083c0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c76a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c76a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35710/0 (socket says 172.21.15.46:35710) 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.313+0000 7f5c76a00700 1 -- 172.21.15.46:0/747184606 learned_addr learned my addr 172.21.15.46:0/747184606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.314+0000 7f5c6e000700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781083c0 0x7f5c780759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.314+0000 7f5c76a00700 1 -- 172.21.15.46:0/747184606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781083c0 msgr2=0x7f5c780759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.314+0000 7f5c76a00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781083c0 0x7f5c780759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.314+0000 7f5c76a00700 1 -- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 msgr2=0x7f5c781a9fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.316+0000 7f5c77400700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 0x7f5c781a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.316+0000 7f5c76a00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 0x7f5c781a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.316+0000 7f5c76a00700 1 -- 172.21.15.46:0/747184606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009d00 con 0x7f5c781032a0 2024-05-08T23:19:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.316+0000 7f5c77400700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 0x7f5c781a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.317+0000 7f5c76a00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d010 tx=0x7f5c6800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.319+0000 7f5c6fe00700 1 -- 172.21.15.46:0/747184606 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800e040 con 0x7f5c781032a0 2024-05-08T23:19:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.319+0000 7f5c6fe00700 1 -- 172.21.15.46:0/747184606 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c68003c00 con 0x7f5c781032a0 2024-05-08T23:19:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.319+0000 7f5c6fe00700 1 -- 172.21.15.46:0/747184606 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6803c660 con 0x7f5c781032a0 2024-05-08T23:19:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.320+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c78199bc0 con 0x7f5c781032a0 2024-05-08T23:19:55.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.320+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7819a080 con 0x7f5c781032a0 2024-05-08T23:19:55.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.330+0000 7f5c6fe00700 1 -- 172.21.15.46:0/747184606 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c68005040 con 0x7f5c781032a0 2024-05-08T23:19:55.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.333+0000 7f5c91431b80 1 Processor -- start 2024-05-08T23:19:55.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 -- start start 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b0583d150 con 0x555b058ec720 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b0583ab10 con 0x555b058e6890 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c91431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b05836400 con 0x555b058ebe00 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c6e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c6e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35714/0 (socket says 172.21.15.46:35714) 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.334+0000 7f5c6e000700 1 -- 172.21.15.46:0/1082534336 learned_addr learned my addr 172.21.15.46:0/1082534336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c76a00700 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 -- 172.21.15.46:0/1082534336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 msgr2=0x555b058ec1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058ec1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 -- 172.21.15.46:0/1082534336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 msgr2=0x555b058e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058e6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 -- 172.21.15.46:0/1082534336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b056b73b0 con 0x555b058e6890 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.335+0000 7f5c6e000700 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009cb0 tx=0x7f5c60011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68e1455597d1e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.336+0000 7f5c67e00700 1 -- 172.21.15.46:0/1082534336 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60011e30 con 0x555b058e6890 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.336+0000 7f5c67e00700 1 -- 172.21.15.46:0/1082534336 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60004510 con 0x555b058e6890 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.336+0000 7f5c67e00700 1 -- 172.21.15.46:0/1082534336 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60005310 con 0x555b058e6890 2024-05-08T23:19:55.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.337+0000 7f5c6fe00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 0x7f5c5c0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.337+0000 7f5c6fe00700 1 -- 172.21.15.46:0/747184606 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5c680efa80 con 0x7f5c781032a0 2024-05-08T23:19:55.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.338+0000 7f5c77400700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 0x7f5c5c0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.339+0000 7f5c91431b80 1 -- 172.21.15.46:0/1082534336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 msgr2=0x555b058e6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.339+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009cb0 tx=0x7f5c60011440 comp rx=0 tx=0).stop 2024-05-08T23:19:55.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.341+0000 7f5c77400700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 0x7f5c5c0a38a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b980 tx=0x7f5c70005c60 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 -- 172.21.15.46:0/1082534336 shutdown_connections 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6cd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1082534336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 -- 172.21.15.46:0/1082534336 >> 172.21.15.46:0/1082534336 conn(0x555b05844210 msgr2=0x555b058e0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 -- 172.21.15.46:0/1082534336 shutdown_connections 2024-05-08T23:19:55.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 -- 172.21.15.46:0/1082534336 wait complete. 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.342+0000 7f5c91431b80 1 -- start start 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 Processor -- start 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058e6890 0x555b058efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b057d2560 con 0x555b058e6890 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b057d2200 con 0x555b058ec720 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.343+0000 7f5c91431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b056b96a0 con 0x555b058ebe00 2024-05-08T23:19:55.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c91431b80 10 client.0 _ll_register_callbacks cb 0x555b057dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:55.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35730/0 (socket says 172.21.15.46:35730) 2024-05-08T23:19:55.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 -- 172.21.15.46:0/2543507853 learned_addr learned my addr 172.21.15.46:0/2543507853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c6e000700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 -- 172.21.15.46:0/2543507853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 msgr2=0x555b058f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 -- 172.21.15.46:0/2543507853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058e6890 msgr2=0x555b058efb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058e6890 0x555b058efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.344+0000 7f5c77400700 1 -- 172.21.15.46:0/2543507853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c600098e0 con 0x555b058ec720 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.345+0000 7f5c77400700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000f6d0 tx=0x7f5c70004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb8b6dd46f98be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.345+0000 7f5c64c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70024660 con 0x555b058ec720 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5c700247c0 con 0x555b058ec720 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b0578f270 con 0x555b058ec720 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c700249a0 con 0x555b058ec720 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c91431b80 10 client.29798 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.346+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555b05663b60 con 0x555b058ec720 2024-05-08T23:19:55.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.347+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b0578f270 con 0x555b058ec720 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.347+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c70017070 con 0x555b058ec720 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 10 client.29798 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 10 client.29798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 20 client.29798 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b059002d0 0x555b059026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 4 client.29798 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:55.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.348+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b058b5810 con 0x555b059002d0 2024-05-08T23:19:55.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.350+0000 7f5c76a00700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b059002d0 0x555b059026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.351+0000 7f5c64c00700 10 client.29798 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:55.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.351+0000 7f5c76a00700 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b059002d0 0x555b059026f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8a33744249ea51 server_cookie=5cd50947d955eca2 in_seq=0 out_seq=0 2024-05-08T23:19:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.352+0000 7f5c64c00700 1 -- 172.21.15.46:0/2543507853 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b058b5810 con 0x555b059002d0 2024-05-08T23:19:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.352+0000 7f5c64c00700 10 client.29798 handle_command_reply: tid=0 2024-05-08T23:19:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.352+0000 7f5c91431b80 1 client.29798 shutdown 2024-05-08T23:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.352+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 msgr2=0x555b058f0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000f6d0 tx=0x7f5c70004750 comp rx=0 tx=0).stop 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 shutdown_connections 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058ec720 0x555b058f0610 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b059002d0 0x555b059026f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058e6890 0x555b058efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 --2- 172.21.15.46:0/2543507853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 >> 172.21.15.46:0/2543507853 conn(0x555b05844210 msgr2=0x555b058e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c64c00700 0 client.29798 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 shutdown_connections 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.353+0000 7f5c91431b80 1 -- 172.21.15.46:0/2543507853 wait complete. 2024-05-08T23:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.354+0000 7f5c91431b80 20 client.29798 trim_cache size 0 max 16384 2024-05-08T23:19:55.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.394+0000 7f5c91431b80 1 Processor -- start 2024-05-08T23:19:55.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.394+0000 7f5c91431b80 1 -- start start 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b058dddc0 con 0x555b058ec720 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b058f1160 con 0x555b058e6890 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.395+0000 7f5c91431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b05907900 con 0x555b058ebe00 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c77400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40096/0 (socket says 172.21.15.46:40096) 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 -- 172.21.15.46:0/414867433 learned_addr learned my addr 172.21.15.46:0/414867433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 -- 172.21.15.46:0/414867433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 msgr2=0x555b058e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 -- 172.21.15.46:0/414867433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 msgr2=0x555b058ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.396+0000 7f5c6e000700 1 -- 172.21.15.46:0/414867433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b057fc560 con 0x555b058ebe00 2024-05-08T23:19:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.397+0000 7f5c6e000700 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009cb0 tx=0x7f5c60005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eef64e011d382c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.397+0000 7f5c6cc00700 1 -- 172.21.15.46:0/414867433 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004200 con 0x555b058ebe00 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.397+0000 7f5c6cc00700 1 -- 172.21.15.46:0/414867433 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60004360 con 0x555b058ebe00 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.397+0000 7f5c6cc00700 1 -- 172.21.15.46:0/414867433 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004200 con 0x555b058ebe00 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.398+0000 7f5c91431b80 1 -- 172.21.15.46:0/414867433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 msgr2=0x555b058e0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.398+0000 7f5c91431b80 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009cb0 tx=0x7f5c60005e10 comp rx=0 tx=0).stop 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 -- 172.21.15.46:0/414867433 shutdown_connections 2024-05-08T23:19:55.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ec720 0x555b058ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 --2- 172.21.15.46:0/414867433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ebe00 0x555b058e0d20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 -- 172.21.15.46:0/414867433 >> 172.21.15.46:0/414867433 conn(0x555b05844210 msgr2=0x555b058e9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 -- 172.21.15.46:0/414867433 shutdown_connections 2024-05-08T23:19:55.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.399+0000 7f5c91431b80 1 -- 172.21.15.46:0/414867433 wait complete. 2024-05-08T23:19:55.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.401+0000 7f5c91431b80 1 -- start start 2024-05-08T23:19:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.401+0000 7f5c91431b80 1 Processor -- start 2024-05-08T23:19:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 0x555b058f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b058f6020 con 0x555b058ebe00 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b05955d60 con 0x555b058e6890 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b05955f20 con 0x555b058ec720 2024-05-08T23:19:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c76a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c91431b80 10 client.0 _ll_register_callbacks cb 0x555b057dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40104/0 (socket says 172.21.15.46:40104) 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.403+0000 7f5c77400700 1 -- 172.21.15.46:0/1817972537 learned_addr learned my addr 172.21.15.46:0/1817972537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c6e000700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 0x555b058f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 -- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 msgr2=0x555b058f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 -- 172.21.15.46:0/1817972537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 msgr2=0x555b058f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 0x555b058f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 -- 172.21.15.46:0/1817972537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6001d040 con 0x555b058ec720 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c6e000700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 0x555b058f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c77400700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5c70019430 tx=0x7f5c700197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5882d0d086ac6f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.404+0000 7f5c64c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70017ed0 con 0x555b058ec720 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5c70017330 con 0x555b058ec720 2024-05-08T23:19:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c60004510 con 0x555b058ec720 2024-05-08T23:19:55.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c700174f0 con 0x555b058ec720 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c91431b80 10 client.29878 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.405+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555b056bbcd0 con 0x555b058ec720 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60004510 con 0x555b058ec720 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c70037020 con 0x555b058ec720 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 10 client.29878 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 10 client.29878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 20 client.29878 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b05957750 0x555b05959bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 4 client.29878 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:19:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.406+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b0582eec0 con 0x555b05957750 2024-05-08T23:19:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.408+0000 7f5c76a00700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.408+0000 7f5c76a00700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b05957750 0x555b05959bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.409+0000 7f5c64c00700 10 client.29878 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.409+0000 7f5c76a00700 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b05957750 0x555b05959bd0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8df870512b80564 server_cookie=7cdd949d7b1b565a in_seq=0 out_seq=0 2024-05-08T23:19:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c64c00700 1 -- 172.21.15.46:0/1817972537 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555b0582eec0 con 0x555b05957750 2024-05-08T23:19:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c64c00700 10 client.29878 handle_command_reply: tid=0 2024-05-08T23:19:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 client.29878 shutdown 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 msgr2=0x555b059557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5c70019430 tx=0x7f5c700197d0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 shutdown_connections 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b058ec720 0x555b059557f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b058e6890 0x555b058f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555b05957750 0x555b05959bd0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 --2- 172.21.15.46:0/1817972537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b058ebe00 0x555b058f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 >> 172.21.15.46:0/1817972537 conn(0x555b05844210 msgr2=0x555b058ee420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.410+0000 7f5c64c00700 0 client.29878 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.411+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 shutdown_connections 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.411+0000 7f5c91431b80 1 -- 172.21.15.46:0/1817972537 wait complete. 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.411+0000 7f5c91431b80 20 client.29878 trim_cache size 0 max 16384 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:19:55.413 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 msgr2=0x7f5c5c0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 0x7f5c5c0a38a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b980 tx=0x7f5c70005c60 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 msgr2=0x7f5c780754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d010 tx=0x7f5c6800be60 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 shutdown_connections 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5c5c0a13e0 0x7f5c5c0a38a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c78075f30 0x7f5c781a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781032a0 0x7f5c780754b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 --2- 172.21.15.46:0/747184606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c781083c0 0x7f5c780759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.412+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 >> 172.21.15.46:0/747184606 conn(0x7f5c780feae0 msgr2=0x7f5c780ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.413+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 shutdown_connections 2024-05-08T23:19:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.413+0000 7f5c7da00700 1 -- 172.21.15.46:0/747184606 wait complete. 2024-05-08T23:19:55.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:19:55.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 -- 172.21.15.46:0/330401195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff94075df0 msgr2=0x7eff94111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 --2- 172.21.15.46:0/330401195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff94075df0 0x7eff94111770 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7eff80009990 tx=0x7eff80009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 -- 172.21.15.46:0/330401195 shutdown_connections 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 --2- 172.21.15.46:0/330401195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94111cb0 0x7eff941140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 --2- 172.21.15.46:0/330401195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff940754d0 0x7eff940758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 --2- 172.21.15.46:0/330401195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff94075df0 0x7eff94111770 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 -- 172.21.15.46:0/330401195 >> 172.21.15.46:0/330401195 conn(0x7eff940feac0 msgr2=0x7eff94100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.747+0000 7eff9b400700 1 -- 172.21.15.46:0/330401195 shutdown_connections 2024-05-08T23:19:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.748+0000 7eff9b400700 1 -- 172.21.15.46:0/330401195 wait complete. 2024-05-08T23:19:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.749+0000 7eff9b400700 1 Processor -- start 2024-05-08T23:19:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.749+0000 7eff9b400700 1 -- start start 2024-05-08T23:19:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff940754d0 0x7eff94111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 0x7eff941116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9410bb50 con 0x7eff94075df0 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9410bcc0 con 0x7eff94111cb0 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.750+0000 7eff9b400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9410be80 con 0x7eff940754d0 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.752+0000 7eff99400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.752+0000 7eff98a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff940754d0 0x7eff94111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.752+0000 7eff99400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35756/0 (socket says 172.21.15.46:35756) 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.752+0000 7eff99400700 1 -- 172.21.15.46:0/1787339794 learned_addr learned my addr 172.21.15.46:0/1787339794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.752+0000 7eff93e00700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 0x7eff941116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff99400700 1 -- 172.21.15.46:0/1787339794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff940754d0 msgr2=0x7eff94111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff99400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff940754d0 0x7eff94111170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff99400700 1 -- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 msgr2=0x7eff941116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff99400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 0x7eff941116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff99400700 1 -- 172.21.15.46:0/1787339794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800095f0 con 0x7eff94111cb0 2024-05-08T23:19:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.753+0000 7eff93e00700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 0x7eff941116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.754+0000 7eff99400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7eff8c007d40 tx=0x7eff8c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.755+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00ad40 con 0x7eff94111cb0 2024-05-08T23:19:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.755+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c007e90 con 0x7eff94111cb0 2024-05-08T23:19:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.755+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0116f0 con 0x7eff94111cb0 2024-05-08T23:19:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.755+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff941b2a00 con 0x7eff94111cb0 2024-05-08T23:19:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.755+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941b2e30 con 0x7eff94111cb0 2024-05-08T23:19:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.759+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94105760 con 0x7eff94111cb0 2024-05-08T23:19:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.761+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff8c012420 con 0x7eff94111cb0 2024-05-08T23:19:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.764+0000 7eff91600700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 0x7eff7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.765+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7eff8c0cd1d0 con 0x7eff94111cb0 2024-05-08T23:19:55.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.765+0000 7eff98a00700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 0x7eff7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.767+0000 7eff98a00700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 0x7eff7c0a37d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff88004630 tx=0x7eff8800b3c0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:55.768+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7eff8c096320 con 0x7eff94111cb0 2024-05-08T23:19:56.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.179+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff94067d40 con 0x7eff94111cb0 2024-05-08T23:19:56.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.181+0000 7eff91600700 1 -- 172.21.15.46:0/1787339794 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10907 (secure 0 0 0) 0x7eff8c099940 con 0x7eff94111cb0 2024-05-08T23:19:56.181 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:19:56.181 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:19:47.919560+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 msgr2=0x7eff7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 0x7eff7c0a37d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff88004630 tx=0x7eff8800b3c0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 msgr2=0x7eff9410b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7eff8c007d40 tx=0x7eff8c00a340 comp rx=0 tx=0).stop 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 shutdown_connections 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff94111cb0 0x7eff9410b380 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff940754d0 0x7eff94111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7eff7c0a1310 0x7eff7c0a37d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 --2- 172.21.15.46:0/1787339794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff94075df0 0x7eff941116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 >> 172.21.15.46:0/1787339794 conn(0x7eff940feac0 msgr2=0x7eff94112920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.185+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 shutdown_connections 2024-05-08T23:19:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.186+0000 7eff9b400700 1 -- 172.21.15.46:0/1787339794 wait complete. 2024-05-08T23:19:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-05-08T23:19:56.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:19:56.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 55397 19.54 MB/sec execute 17 sec latency 341.318 ms 2024-05-08T23:19:56.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:55 smithi053 ceph-mon[77587]: pgmap v504: 129 pgs: 129 active+clean; 972 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-05-08T23:19:56.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:55 smithi046 ceph-mon[36898]: pgmap v504: 129 pgs: 129 active+clean; 972 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-05-08T23:19:56.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:55 smithi086 ceph-mon[78610]: pgmap v504: 129 pgs: 129 active+clean; 972 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 433 op/s 2024-05-08T23:19:56.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.586+0000 7feed5800700 1 -- 172.21.15.46:0/2795696114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed0103c50 msgr2=0x7feed0107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.586+0000 7feed5800700 1 --2- 172.21.15.46:0/2795696114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed0103c50 0x7feed0107c90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feebc009990 tx=0x7feebc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.586+0000 7feed5800700 1 -- 172.21.15.46:0/2795696114 shutdown_connections 2024-05-08T23:19:56.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.587+0000 7feed5800700 1 --2- 172.21.15.46:0/2795696114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed01083c0 0x7feed010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.587+0000 7feed5800700 1 --2- 172.21.15.46:0/2795696114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01032a0 0x7feed0103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.587+0000 7feed5800700 1 --2- 172.21.15.46:0/2795696114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed0103c50 0x7feed0107c90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.587+0000 7feed5800700 1 -- 172.21.15.46:0/2795696114 >> 172.21.15.46:0/2795696114 conn(0x7feed00feb20 msgr2=0x7feed0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.587+0000 7feed5800700 1 -- 172.21.15.46:0/2795696114 shutdown_connections 2024-05-08T23:19:56.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.588+0000 7feed5800700 1 -- 172.21.15.46:0/2795696114 wait complete. 2024-05-08T23:19:56.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.589+0000 7feed5800700 1 Processor -- start 2024-05-08T23:19:56.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 -- start start 2024-05-08T23:19:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed01032a0 0x7feed010e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01a75d0 0x7feed01a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01a9f30 con 0x7feed01a75d0 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01aa0a0 con 0x7feed01083c0 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.590+0000 7feed5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01aa260 con 0x7feed01032a0 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35786/0 (socket says 172.21.15.46:35786) 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 -- 172.21.15.46:0/3102556416 learned_addr learned my addr 172.21.15.46:0/3102556416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feecf400700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01a75d0 0x7feed01a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 -- 172.21.15.46:0/3102556416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed01032a0 msgr2=0x7feed010e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed01032a0 0x7feed010e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 -- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01a75d0 msgr2=0x7feed01a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01a75d0 0x7feed01a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 -- 172.21.15.46:0/3102556416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0095f0 con 0x7feed01083c0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.591+0000 7feece000700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b5c0 tx=0x7feebc00f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.592+0000 7feec7400700 1 -- 172.21.15.46:0/3102556416 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc01d070 con 0x7feed01083c0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.592+0000 7feec7400700 1 -- 172.21.15.46:0/3102556416 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feebc00fbe0 con 0x7feed01083c0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.592+0000 7feec7400700 1 -- 172.21.15.46:0/3102556416 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc017650 con 0x7feed01083c0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.592+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed01aa4e0 con 0x7feed01083c0 2024-05-08T23:19:56.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.592+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed01aa9a0 con 0x7feed01083c0 2024-05-08T23:19:56.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.597+0000 7feec7400700 1 -- 172.21.15.46:0/3102556416 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feebc00fd50 con 0x7feed01083c0 2024-05-08T23:19:56.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.598+0000 7feee9351b80 1 Processor -- start 2024-05-08T23:19:56.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 -- start start 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 0x55c2f5b76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b76890 0x55c2f5b76cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5acd150 con 0x55c2f5b76890 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5acab10 con 0x55c2f5b7c720 2024-05-08T23:19:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.599+0000 7feee9351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5ac6400 con 0x55c2f5b7be00 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 0x55c2f5b76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 0x55c2f5b76350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35800/0 (socket says 172.21.15.46:35800) 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecea00700 1 -- 172.21.15.46:0/2681333545 learned_addr learned my addr 172.21.15.46:0/2681333545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feece000700 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b76890 0x55c2f5b76cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 -- 172.21.15.46:0/2681333545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 msgr2=0x55c2f5b76350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 0x55c2f5b76350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 -- 172.21.15.46:0/2681333545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b76890 msgr2=0x55c2f5b76cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b76890 0x55c2f5b76cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.600+0000 7feecf400700 1 -- 172.21.15.46:0/2681333545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f59473b0 con 0x55c2f5b7be00 2024-05-08T23:19:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.601+0000 7feecf400700 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b7c1e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feec8009e60 tx=0x7feec8009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=422b803be332ce2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.602+0000 7feec7400700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 0x7feeb40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.602+0000 7feec7400700 1 -- 172.21.15.46:0/3102556416 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feebc0cdfa0 con 0x7feed01083c0 2024-05-08T23:19:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.602+0000 7feebbe00700 1 -- 172.21.15.46:0/2681333545 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011070 con 0x55c2f5b7be00 2024-05-08T23:19:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.602+0000 7feebbe00700 1 -- 172.21.15.46:0/2681333545 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec800ce90 con 0x55c2f5b7be00 2024-05-08T23:19:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.602+0000 7feebbe00700 1 -- 172.21.15.46:0/2681333545 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8015d70 con 0x55c2f5b7be00 2024-05-08T23:19:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.603+0000 7feecea00700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 0x7feeb40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.604+0000 7feee9351b80 1 -- 172.21.15.46:0/2681333545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 msgr2=0x55c2f5b7c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.604+0000 7feecea00700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 0x7feeb40a3810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feec000a640 tx=0x7feec000a5d0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.604+0000 7feee9351b80 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b7c1e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feec8009e60 tx=0x7feec8009e90 comp rx=0 tx=0).stop 2024-05-08T23:19:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.607+0000 7feee9351b80 1 -- 172.21.15.46:0/2681333545 shutdown_connections 2024-05-08T23:19:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b76890 0x55c2f5b76cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b7c1e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 --2- 172.21.15.46:0/2681333545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b7c720 0x55c2f5b76350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 -- 172.21.15.46:0/2681333545 >> 172.21.15.46:0/2681333545 conn(0x55c2f5ad4210 msgr2=0x55c2f5b70df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 -- 172.21.15.46:0/2681333545 shutdown_connections 2024-05-08T23:19:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.608+0000 7feee9351b80 1 -- 172.21.15.46:0/2681333545 wait complete. 2024-05-08T23:19:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.610+0000 7feee9351b80 1 -- start start 2024-05-08T23:19:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 Processor -- start 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b80670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5a62560 con 0x55c2f5b7c720 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5a62200 con 0x55c2f5b76890 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f59496a0 con 0x55c2f5b7be00 2024-05-08T23:19:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.611+0000 7feee9351b80 10 client.0 _ll_register_callbacks cb 0x55c2f5a6bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.612+0000 7feecf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b80670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.612+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b7fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35814/0 (socket says 172.21.15.46:35814) 2024-05-08T23:19:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feecea00700 1 -- 172.21.15.46:0/1917355798 learned_addr learned my addr 172.21.15.46:0/1917355798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feece000700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feece000700 1 -- 172.21.15.46:0/1917355798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 msgr2=0x55c2f5b7fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feece000700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b7fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.613+0000 7feece000700 1 -- 172.21.15.46:0/1917355798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 msgr2=0x55c2f5b80670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.614+0000 7feece000700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b80670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.614+0000 7feece000700 1 -- 172.21.15.46:0/1917355798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec8009b10 con 0x55c2f5b7be00 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.614+0000 7feecf400700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b80670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.614+0000 7feece000700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b80130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feebc0cb280 tx=0x7feebc09abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebfef5712f4f89df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.615+0000 7feeb8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.615+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc09acd0 con 0x55c2f5b7be00 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.615+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feebc09eb00 con 0x55c2f5b7be00 2024-05-08T23:19:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.615+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2f5a1f270 con 0x55c2f5b7be00 2024-05-08T23:19:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.616+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7feebc0a30c0 con 0x55c2f5b7be00 2024-05-08T23:19:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.616+0000 7feee9351b80 10 client.29890 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.616+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c2f58f3b60 con 0x55c2f5b7be00 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.616+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2f5a1f270 con 0x55c2f5b7be00 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feebc0d1b80 con 0x55c2f5b7be00 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 10 client.29890 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 10 client.29890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 20 client.29890 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5b90050 0x55c2f5b92470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 4 client.29890 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.617+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2f5b45810 con 0x55c2f5b90050 2024-05-08T23:19:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.618+0000 7feecea00700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5b90050 0x55c2f5b92470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.619+0000 7feecea00700 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5b90050 0x55c2f5b92470 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3d205cb53af42e8 server_cookie=b3d9ec8f810e95a7 in_seq=0 out_seq=0 2024-05-08T23:19:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.619+0000 7feeb8c00700 10 client.29890 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.621+0000 7feeb8c00700 1 -- 172.21.15.46:0/1917355798 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2f5b45810 con 0x55c2f5b90050 2024-05-08T23:19:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.621+0000 7feeb8c00700 10 client.29890 handle_command_reply: tid=0 2024-05-08T23:19:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.621+0000 7feee9351b80 1 client.29890 shutdown 2024-05-08T23:19:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.622+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 msgr2=0x55c2f5b80130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.622+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b80130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feebc0cb280 tx=0x7feebc09abd0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 shutdown_connections 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5b90050 0x55c2f5b92470 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b80670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b7fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 --2- 172.21.15.46:0/1917355798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b80130 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 >> 172.21.15.46:0/1917355798 conn(0x55c2f5ad4210 msgr2=0x55c2f5b705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feeb8c00700 0 client.29890 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 shutdown_connections 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.623+0000 7feee9351b80 1 -- 172.21.15.46:0/1917355798 wait complete. 2024-05-08T23:19:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.624+0000 7feee9351b80 20 client.29890 trim_cache size 0 max 16384 2024-05-08T23:19:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.650+0000 7feee9351b80 1 Processor -- start 2024-05-08T23:19:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.650+0000 7feee9351b80 1 -- start start 2024-05-08T23:19:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b8c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b70d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5b69110 con 0x55c2f5b7c720 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5b7c430 con 0x55c2f5b76890 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feee9351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5b811f0 con 0x55c2f5b7be00 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35830/0 (socket says 172.21.15.46:35830) 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feecea00700 1 -- 172.21.15.46:0/2370602767 learned_addr learned my addr 172.21.15.46:0/2370602767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feece000700 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b8c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.651+0000 7feecf400700 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 -- 172.21.15.46:0/2370602767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 msgr2=0x55c2f5b8c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b8c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 -- 172.21.15.46:0/2370602767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 msgr2=0x55c2f5b70d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b70d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 -- 172.21.15.46:0/2370602767 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f5a8c560 con 0x55c2f5b76890 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.652+0000 7feecea00700 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c2f5b92d50 tx=0x7feec0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fbf86d5ef5d408c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.653+0000 7feec4c00700 1 -- 172.21.15.46:0/2370602767 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00244d0 con 0x55c2f5b76890 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.653+0000 7feec4c00700 1 -- 172.21.15.46:0/2370602767 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec0022040 con 0x55c2f5b76890 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.653+0000 7feec4c00700 1 -- 172.21.15.46:0/2370602767 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0010c60 con 0x55c2f5b76890 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.653+0000 7feee9351b80 1 -- 172.21.15.46:0/2370602767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 msgr2=0x55c2f5b76350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.653+0000 7feee9351b80 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c2f5b92d50 tx=0x7feec0012000 comp rx=0 tx=0).stop 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 -- 172.21.15.46:0/2370602767 shutdown_connections 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5b70d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b76350 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 --2- 172.21.15.46:0/2370602767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b8c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 -- 172.21.15.46:0/2370602767 >> 172.21.15.46:0/2370602767 conn(0x55c2f5ad4210 msgr2=0x55c2f5b6d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 -- 172.21.15.46:0/2370602767 shutdown_connections 2024-05-08T23:19:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.654+0000 7feee9351b80 1 -- 172.21.15.46:0/2370602767 wait complete. 2024-05-08T23:19:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.655+0000 7feee9351b80 1 -- start start 2024-05-08T23:19:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.655+0000 7feee9351b80 1 Processor -- start 2024-05-08T23:19:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.658+0000 7feee9351b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.658+0000 7feee9351b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feee9351b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5be5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feee9351b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5be5a90 con 0x55c2f5b7c720 2024-05-08T23:19:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feee9351b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5be5bd0 con 0x55c2f5b76890 2024-05-08T23:19:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feee9351b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f5be5d10 con 0x55c2f5b7be00 2024-05-08T23:19:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feee9351b80 10 client.0 _ll_register_callbacks cb 0x55c2f5a6bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feece000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feece000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40186/0 (socket says 172.21.15.46:40186) 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feece000700 1 -- 172.21.15.46:0/571735379 learned_addr learned my addr 172.21.15.46:0/571735379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.659+0000 7feecf400700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5be5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 -- 172.21.15.46:0/571735379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 msgr2=0x55c2f5b85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 -- 172.21.15.46:0/571735379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 msgr2=0x55c2f5be5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5be5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 -- 172.21.15.46:0/571735379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec000fd90 con 0x55c2f5b7be00 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feece000700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7feebc09a9d0 tx=0x7feebc0a2e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819629954cb19489 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feeb8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.660+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc096d90 con 0x55c2f5b7be00 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.661+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7feebc09eb00 con 0x55c2f5b7be00 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.661+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec800ce90 con 0x55c2f5b7be00 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.661+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7feebc09acd0 con 0x55c2f5b7be00 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.661+0000 7feee9351b80 10 client.29896 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.661+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c2f594bcd0 con 0x55c2f5b7be00 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec800ce90 con 0x55c2f5b7be00 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feebc0b3270 con 0x55c2f5b7be00 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 10 client.29896 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 10 client.29896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 20 client.29896 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5be6f90 0x55c2f5be9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 4 client.29896 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.664+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2f5abeec0 con 0x55c2f5be6f90 2024-05-08T23:19:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.665+0000 7feecea00700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5be6f90 0x55c2f5be9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.666+0000 7feecea00700 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5be6f90 0x55c2f5be9380 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bcddf4582c4d16a server_cookie=1402a8e4a53215ba in_seq=0 out_seq=0 2024-05-08T23:19:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.667+0000 7feeb8c00700 10 client.29896 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.667+0000 7feeb8c00700 1 -- 172.21.15.46:0/571735379 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2f5abeec0 con 0x55c2f5be6f90 2024-05-08T23:19:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.667+0000 7feeb8c00700 10 client.29896 handle_command_reply: tid=0 2024-05-08T23:19:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.667+0000 7feee9351b80 1 client.29896 shutdown 2024-05-08T23:19:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 msgr2=0x55c2f5b85880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7feebc09a9d0 tx=0x7feebc0a2e40 comp rx=0 tx=0).stop 2024-05-08T23:19:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 shutdown_connections 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55c2f5be6f90 0x55c2f5be9380 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2f5b7c720 0x55c2f5be5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feeb8c00700 0 client.29896 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2f5b76890 0x55c2f5b85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 --2- 172.21.15.46:0/571735379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f5b7be00 0x55c2f5b85880 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.669+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 >> 172.21.15.46:0/571735379 conn(0x55c2f5ad4210 msgr2=0x55c2f5b7e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.670+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 shutdown_connections 2024-05-08T23:19:56.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.674+0000 7feee9351b80 1 -- 172.21.15.46:0/571735379 wait complete. 2024-05-08T23:19:56.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.675+0000 7feee9351b80 20 client.29896 trim_cache size 0 max 16384 2024-05-08T23:19:56.675 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:56.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 msgr2=0x7feeb40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 0x7feeb40a3810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feec000a640 tx=0x7feec000a5d0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 msgr2=0x7feed010edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b5c0 tx=0x7feebc00f7b0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 shutdown_connections 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7feeb40a1350 0x7feeb40a3810 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed01a75d0 0x7feed01a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed01032a0 0x7feed010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 --2- 172.21.15.46:0/3102556416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed01083c0 0x7feed010edf0 secure :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b5c0 tx=0x7feebc00f7b0 comp rx=0 tx=0).stop 2024-05-08T23:19:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.676+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 >> 172.21.15.46:0/3102556416 conn(0x7feed00feb20 msgr2=0x7feed00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:56.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.677+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 shutdown_connections 2024-05-08T23:19:56.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:56.678+0000 7feed5800700 1 -- 172.21.15.46:0/3102556416 wait complete. 2024-05-08T23:19:56.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:19:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 -- 172.21.15.46:0/2846545095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 msgr2=0x7f655410bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 --2- 172.21.15.46:0/2846545095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 0x7f655410bc80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f65400098f0 tx=0x7f6540009c00 comp rx=0 tx=0).stop 2024-05-08T23:19:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 -- 172.21.15.46:0/2846545095 shutdown_connections 2024-05-08T23:19:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 --2- 172.21.15.46:0/2846545095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f65541186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 --2- 172.21.15.46:0/2846545095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f6554111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 --2- 172.21.15.46:0/2846545095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 0x7f655410bc80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 -- 172.21.15.46:0/2846545095 >> 172.21.15.46:0/2846545095 conn(0x7f655406bde0 msgr2=0x7f6554109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 -- 172.21.15.46:0/2846545095 shutdown_connections 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.086+0000 7f6559400700 1 -- 172.21.15.46:0/2846545095 wait complete. 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.088+0000 7f6559400700 1 Processor -- start 2024-05-08T23:19:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.088+0000 7f6559400700 1 -- start start 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 0x7f6554117100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f6554117640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541b2330 con 0x7f655410c1c0 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541b24a0 con 0x7f6554111810 2024-05-08T23:19:57.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.089+0000 7f6559400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541b2660 con 0x7f655410b800 2024-05-08T23:19:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6551e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f6554117640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35858/0 (socket says 172.21.15.46:35858) 2024-05-08T23:19:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 -- 172.21.15.46:0/775089579 learned_addr learned my addr 172.21.15.46:0/775089579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 -- 172.21.15.46:0/775089579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 msgr2=0x7f6554117100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 0x7f6554117100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 -- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 msgr2=0x7f6554117640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f6554117640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 -- 172.21.15.46:0/775089579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540009520 con 0x7f6554111810 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6551e00700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f6554117640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.090+0000 7f6553200700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fd0 tx=0x7f654c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.091+0000 7f654b400700 1 -- 172.21.15.46:0/775089579 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00ddd0 con 0x7f6554111810 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.091+0000 7f654b400700 1 -- 172.21.15.46:0/775089579 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654c00f040 con 0x7f6554111810 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.091+0000 7f654b400700 1 -- 172.21.15.46:0/775089579 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c004760 con 0x7f6554111810 2024-05-08T23:19:57.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.091+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65541b2940 con 0x7f6554111810 2024-05-08T23:19:57.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.091+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65541b2dd0 con 0x7f6554111810 2024-05-08T23:19:57.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.095+0000 7f654b400700 1 -- 172.21.15.46:0/775089579 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f654c003a00 con 0x7f6554111810 2024-05-08T23:19:57.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.097+0000 7f656cf4eb80 1 Processor -- start 2024-05-08T23:19:57.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.097+0000 7f654b400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 0x7f65380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 -- start start 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f654b400700 1 -- 172.21.15.46:0/775089579 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f654c0cc9f0 con 0x7f6554111810 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8ce00 0x562006b8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006ade150 con 0x562006b8ce00 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006adbb10 con 0x562006b8d720 2024-05-08T23:19:57.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.098+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006ad7400 con 0x562006b87890 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40210/0 (socket says 172.21.15.46:40210) 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 -- 172.21.15.46:0/3601920953 learned_addr learned my addr 172.21.15.46:0/3601920953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 -- 172.21.15.46:0/3601920953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 msgr2=0x562006b87350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b87350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 -- 172.21.15.46:0/3601920953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8ce00 msgr2=0x562006b8d1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8ce00 0x562006b8d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 -- 172.21.15.46:0/3601920953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620069583b0 con 0x562006b87890 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6551e00700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 0x7f65380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.099+0000 7f6553200700 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f654c095b50 tx=0x7f654c095a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37fd8dd0ef13051 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.100+0000 7f653f400700 1 -- 172.21.15.46:0/3601920953 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c098fb0 con 0x562006b87890 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.100+0000 7f653f400700 1 -- 172.21.15.46:0/3601920953 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654c004020 con 0x562006b87890 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.100+0000 7f653f400700 1 -- 172.21.15.46:0/3601920953 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a1b60 con 0x562006b87890 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.100+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3601920953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 msgr2=0x562006b87cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.100+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f654c095b50 tx=0x7f654c095a10 comp rx=0 tx=0).stop 2024-05-08T23:19:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3601920953 shutdown_connections 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87cd0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8ce00 0x562006b8d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3601920953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3601920953 >> 172.21.15.46:0/3601920953 conn(0x562006ae5210 msgr2=0x562006b81df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f6551e00700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 0x7f65380a37d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65400098c0 tx=0x7f6540005dc0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3601920953 shutdown_connections 2024-05-08T23:19:57.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3601920953 wait complete. 2024-05-08T23:19:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.101+0000 7f656cf4eb80 1 -- start start 2024-05-08T23:19:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 Processor -- start 2024-05-08T23:19:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b91040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006a73560 con 0x562006b87890 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006a73200 con 0x562006b8ce00 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200695a6a0 con 0x562006b8d720 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f656cf4eb80 10 client.0 _ll_register_callbacks cb 0x562006a7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.102+0000 7f6553200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40212/0 (socket says 172.21.15.46:40212) 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 -- 172.21.15.46:0/3653719001 learned_addr learned my addr 172.21.15.46:0/3653719001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6552800700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 -- 172.21.15.46:0/3653719001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 msgr2=0x562006b91040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:19:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b91040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 -- 172.21.15.46:0/3653719001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 msgr2=0x562006b90b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b90b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 -- 172.21.15.46:0/3653719001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65541b2dd0 con 0x562006b8d720 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6553200700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f654c0a15a0 tx=0x7f654c0990d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162d2f7f885e51c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6533e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0842f0 con 0x562006b8d720 2024-05-08T23:19:57.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562006a30270 con 0x562006b8d720 2024-05-08T23:19:57.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.103+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f654c09d480 con 0x562006b8d720 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f654c0a1ad0 con 0x562006b8d720 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 10 client.29905 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562006904b60 con 0x562006b8d720 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562006a30270 con 0x562006b8d720 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f654c0b5070 con 0x562006b8d720 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 10 client.29905 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 10 client.29905 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 20 client.29905 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006ba0e30 0x562006ba3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 4 client.29905 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562006b56810 con 0x562006ba0e30 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.104+0000 7f6552800700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006ba0e30 0x562006ba3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.105+0000 7f6552800700 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006ba0e30 0x562006ba3220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=440ed2f1423b9912 server_cookie=25b3b2d5da8ae6 in_seq=0 out_seq=0 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.105+0000 7f6533e00700 10 client.29905 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.106+0000 7f6533e00700 1 -- 172.21.15.46:0/3653719001 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562006b56810 con 0x562006ba0e30 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.106+0000 7f6533e00700 10 client.29905 handle_command_reply: tid=0 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.106+0000 7f656cf4eb80 1 client.29905 shutdown 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.106+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 msgr2=0x562006b91580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.106+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f654c0a15a0 tx=0x7f654c0990d0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 shutdown_connections 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006ba0e30 0x562006ba3220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8d720 0x562006b91580 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b90b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3653719001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b91040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 >> 172.21.15.46:0/3653719001 conn(0x562006ae5210 msgr2=0x562006b815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f6533e00700 0 client.29905 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 shutdown_connections 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3653719001 wait complete. 2024-05-08T23:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.107+0000 7f656cf4eb80 20 client.29905 trim_cache size 0 max 16384 2024-05-08T23:19:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 Processor -- start 2024-05-08T23:19:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 -- start start 2024-05-08T23:19:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b87c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 0x562006b9cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b8d430 con 0x562006b8d720 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b7a0a0 con 0x562006b8ce00 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.125+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b8b5b0 con 0x562006b87890 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40216/0 (socket says 172.21.15.46:40216) 2024-05-08T23:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 -- 172.21.15.46:0/3541332366 learned_addr learned my addr 172.21.15.46:0/3541332366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6553200700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 0x562006b9cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 -- 172.21.15.46:0/3541332366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 msgr2=0x562006b87c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6551e00700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b87c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b87c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 -- 172.21.15.46:0/3541332366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 msgr2=0x562006b9cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 0x562006b9cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.126+0000 7f6552800700 1 -- 172.21.15.46:0/3541332366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562006a9d560 con 0x562006b87890 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.127+0000 7f6551e00700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b87c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.127+0000 7f6553200700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 0x562006b9cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.135+0000 7f6552800700 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x562006ba3b30 tx=0x7f6544005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c0a3c7ab7e97f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.136+0000 7f653fe00700 1 -- 172.21.15.46:0/3541332366 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544017070 con 0x562006b87890 2024-05-08T23:19:57.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.136+0000 7f653fe00700 1 -- 172.21.15.46:0/3541332366 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6544017e90 con 0x562006b87890 2024-05-08T23:19:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.136+0000 7f653fe00700 1 -- 172.21.15.46:0/3541332366 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544012a70 con 0x562006b87890 2024-05-08T23:19:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3541332366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 msgr2=0x562006b87350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x562006ba3b30 tx=0x7f6544005d80 comp rx=0 tx=0).stop 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3541332366 shutdown_connections 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b8d720 0x562006b9cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b87890 0x562006b87350 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/3541332366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8ce00 0x562006b87c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.137+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3541332366 >> 172.21.15.46:0/3541332366 conn(0x562006ae5210 msgr2=0x562006b7e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.138+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3541332366 shutdown_connections 2024-05-08T23:19:57.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.138+0000 7f656cf4eb80 1 -- 172.21.15.46:0/3541332366 wait complete. 2024-05-08T23:19:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.138+0000 7f656cf4eb80 1 -- start start 2024-05-08T23:19:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.139+0000 7f656cf4eb80 1 Processor -- start 2024-05-08T23:19:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b97e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 0x562006b98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b96be0 con 0x562006b87890 2024-05-08T23:19:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b96d50 con 0x562006b8d720 2024-05-08T23:19:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562006b96f10 con 0x562006b8ce00 2024-05-08T23:19:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f656cf4eb80 10 client.0 _ll_register_callbacks cb 0x562006a7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6553200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6551e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 0x562006b98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6553200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35890/0 (socket says 172.21.15.46:35890) 2024-05-08T23:19:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6551e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 0x562006b98380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40226/0 (socket says 172.21.15.46:40226) 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6553200700 1 -- 172.21.15.46:0/1423245394 learned_addr learned my addr 172.21.15.46:0/1423245394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.140+0000 7f6552800700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b97e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 -- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 msgr2=0x562006b98380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 0x562006b98380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 -- 172.21.15.46:0/1423245394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 msgr2=0x562006b97e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b97e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 -- 172.21.15.46:0/1423245394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654400bbb0 con 0x562006b8d720 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6553200700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f654c099350 tx=0x7f654c0074e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a79833bd56a9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6533e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.141+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c099b70 con 0x562006b8d720 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.142+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f654c0842f0 con 0x562006b8d720 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.142+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f654c004020 con 0x562006b8d720 2024-05-08T23:19:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.142+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f654c09d480 con 0x562006b8d720 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.142+0000 7f656cf4eb80 10 client.29831 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.142+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56200695ccd0 con 0x562006b8d720 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654c004020 con 0x562006b8d720 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f654c0a1d20 con 0x562006b8d720 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 10 client.29831 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 10 client.29831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 20 client.29831 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006bf8030 0x562006bfa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 4 client.29831 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562006acfec0 con 0x562006bf8030 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.143+0000 7f6552800700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006bf8030 0x562006bfa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.144+0000 7f6533e00700 10 client.29831 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.144+0000 7f6552800700 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006bf8030 0x562006bfa450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f045d4e3b023bcdb server_cookie=49c31398a56c05d7 in_seq=0 out_seq=0 2024-05-08T23:19:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.145+0000 7f6533e00700 1 -- 172.21.15.46:0/1423245394 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562006acfec0 con 0x562006bf8030 2024-05-08T23:19:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.145+0000 7f6533e00700 10 client.29831 handle_command_reply: tid=0 2024-05-08T23:19:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.145+0000 7f656cf4eb80 1 client.29831 shutdown 2024-05-08T23:19:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.147+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 msgr2=0x562006b96500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.147+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f654c099350 tx=0x7f654c0074e0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 shutdown_connections 2024-05-08T23:19:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x562006bf8030 0x562006bfa450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562006b8d720 0x562006b96500 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562006b87890 0x562006b97e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 --2- 172.21.15.46:0/1423245394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562006b8ce00 0x562006b98380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 >> 172.21.15.46:0/1423245394 conn(0x562006ae5210 msgr2=0x562006b8f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f6533e00700 0 client.29831 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.148+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 shutdown_connections 2024-05-08T23:19:57.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.149+0000 7f656cf4eb80 1 -- 172.21.15.46:0/1423245394 wait complete. 2024-05-08T23:19:57.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.149+0000 7f656cf4eb80 20 client.29831 trim_cache size 0 max 16384 2024-05-08T23:19:57.150 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.150+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 msgr2=0x7f65380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.150+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 0x7f65380a37d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65400098c0 tx=0x7f6540005dc0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.150+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 msgr2=0x7f65541b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.150+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fd0 tx=0x7f654c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 shutdown_connections 2024-05-08T23:19:57.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6554111810 0x7f65541b1ce0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655410b800 0x7f6554117100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f65380a1310 0x7f65380a37d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 --2- 172.21.15.46:0/775089579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f655410c1c0 0x7f6554117640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 >> 172.21.15.46:0/775089579 conn(0x7f655406bde0 msgr2=0x7f655410fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 shutdown_connections 2024-05-08T23:19:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.151+0000 7f6559400700 1 -- 172.21.15.46:0/775089579 wait complete. 2024-05-08T23:19:57.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:19:57.257 INFO:tasks.workunit.client.0.smithi046.stdout: 10 55804 19.45 MB/sec execute 18 sec latency 158.700 ms 2024-05-08T23:19:57.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:56 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1787339794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:57.271 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:56 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1787339794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:57.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:56 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1787339794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.513+0000 7f4865600700 1 -- 172.21.15.46:0/228264445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107ea0 msgr2=0x7f4860114300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.513+0000 7f4865600700 1 --2- 172.21.15.46:0/228264445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107ea0 0x7f4860114300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f484c01a9d0 tx=0x7f484c01ace0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 -- 172.21.15.46:0/228264445 shutdown_connections 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 --2- 172.21.15.46:0/228264445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107ea0 0x7f4860114300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 --2- 172.21.15.46:0/228264445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f486010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 --2- 172.21.15.46:0/228264445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48601074e0 0x7f4860107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 -- 172.21.15.46:0/228264445 >> 172.21.15.46:0/228264445 conn(0x7f4860076d70 msgr2=0x7f4860105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 -- 172.21.15.46:0/228264445 shutdown_connections 2024-05-08T23:19:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.514+0000 7f4865600700 1 -- 172.21.15.46:0/228264445 wait complete. 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.515+0000 7f4865600700 1 Processor -- start 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.515+0000 7f4865600700 1 -- start start 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601074e0 0x7f486010fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 0x7f4860110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601c3180 con 0x7f4860107ea0 2024-05-08T23:19:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601c32c0 con 0x7f48601074e0 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f4865600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601c3480 con 0x7f486010d540 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.516+0000 7f485f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 0x7f4860110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40254/0 (socket says 172.21.15.46:40254) 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 -- 172.21.15.46:0/2358443548 learned_addr learned my addr 172.21.15.46:0/2358443548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485ea00700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601074e0 0x7f486010fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 -- 172.21.15.46:0/2358443548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601074e0 msgr2=0x7f486010fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601074e0 0x7f486010fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 -- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 msgr2=0x7f4860110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 0x7f4860110200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485f400700 1 -- 172.21.15.46:0/2358443548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c01a600 con 0x7f486010d540 2024-05-08T23:19:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.517+0000 7f485e000700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 0x7f4860110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.518+0000 7f485f400700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f484c005440 tx=0x7f484c005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.520+0000 7f4857400700 1 -- 172.21.15.46:0/2358443548 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c003a60 con 0x7f486010d540 2024-05-08T23:19:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.520+0000 7f4857400700 1 -- 172.21.15.46:0/2358443548 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c003bc0 con 0x7f486010d540 2024-05-08T23:19:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.520+0000 7f4857400700 1 -- 172.21.15.46:0/2358443548 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c01f650 con 0x7f486010d540 2024-05-08T23:19:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.521+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48601c3700 con 0x7f486010d540 2024-05-08T23:19:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.521+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601c3b60 con 0x7f486010d540 2024-05-08T23:19:57.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.526+0000 7f4857400700 1 -- 172.21.15.46:0/2358443548 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f484c003d30 con 0x7f486010d540 2024-05-08T23:19:57.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.529+0000 7f4857400700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 0x7f48440a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.529+0000 7f4857400700 1 -- 172.21.15.46:0/2358443548 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f484c0ce690 con 0x7f486010d540 2024-05-08T23:19:57.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.532+0000 7f485ea00700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 0x7f48440a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.533+0000 7f485ea00700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 0x7f48440a38a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4858007fd0 tx=0x7f48580079e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.533+0000 7f4879019b80 1 Processor -- start 2024-05-08T23:19:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.533+0000 7f4879019b80 1 -- start start 2024-05-08T23:19:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7ce00 0x557acbf7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf77cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbece150 con 0x557acbf7ce00 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbecbb10 con 0x557acbf77890 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.535+0000 7f4879019b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbec7400 con 0x557acbf7d720 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40266/0 (socket says 172.21.15.46:40266) 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 -- 172.21.15.46:0/4106594242 learned_addr learned my addr 172.21.15.46:0/4106594242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485ea00700 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7ce00 0x557acbf7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 -- 172.21.15.46:0/4106594242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 msgr2=0x557acbf77cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf77cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 -- 172.21.15.46:0/4106594242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7ce00 msgr2=0x557acbf7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7ce00 0x557acbf7d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.536+0000 7f485e000700 1 -- 172.21.15.46:0/4106594242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557acbd483b0 con 0x557acbf7d720 2024-05-08T23:19:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.537+0000 7f485e000700 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4850008cd0 tx=0x7f485000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17bef36b93ecc7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.537+0000 7f484b400700 1 -- 172.21.15.46:0/4106594242 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000ee90 con 0x557acbf7d720 2024-05-08T23:19:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.537+0000 7f484b400700 1 -- 172.21.15.46:0/4106594242 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4850004cf0 con 0x557acbf7d720 2024-05-08T23:19:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.537+0000 7f484b400700 1 -- 172.21.15.46:0/4106594242 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850010690 con 0x557acbf7d720 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.538+0000 7f4879019b80 1 -- 172.21.15.46:0/4106594242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 msgr2=0x557acbf77350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.538+0000 7f4879019b80 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4850008cd0 tx=0x7f485000e350 comp rx=0 tx=0).stop 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.539+0000 7f4879019b80 1 -- 172.21.15.46:0/4106594242 shutdown_connections 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.539+0000 7f4879019b80 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf77cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.539+0000 7f4879019b80 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7ce00 0x557acbf7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.539+0000 7f4879019b80 1 --2- 172.21.15.46:0/4106594242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf77350 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.540+0000 7f4879019b80 1 -- 172.21.15.46:0/4106594242 >> 172.21.15.46:0/4106594242 conn(0x557acbed5210 msgr2=0x557acbf71df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.540+0000 7f4879019b80 1 -- 172.21.15.46:0/4106594242 shutdown_connections 2024-05-08T23:19:57.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.540+0000 7f4879019b80 1 -- 172.21.15.46:0/4106594242 wait complete. 2024-05-08T23:19:57.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.541+0000 7f4879019b80 1 -- start start 2024-05-08T23:19:57.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.542+0000 7f4879019b80 1 Processor -- start 2024-05-08T23:19:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.542+0000 7f4879019b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf80bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbe63560 con 0x557acbf77890 2024-05-08T23:19:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbe63200 con 0x557acbf7ce00 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbd4a6a0 con 0x557acbf7d720 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f4879019b80 10 client.0 _ll_register_callbacks cb 0x557acbe6cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f485f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f485f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40276/0 (socket says 172.21.15.46:40276) 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f485f400700 1 -- 172.21.15.46:0/2804260767 learned_addr learned my addr 172.21.15.46:0/2804260767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.543+0000 7f485e000700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.544+0000 7f485f400700 1 -- 172.21.15.46:0/2804260767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 msgr2=0x557acbf810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.544+0000 7f485f400700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.544+0000 7f485f400700 1 -- 172.21.15.46:0/2804260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 msgr2=0x557acbf80bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.544+0000 7f485f400700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf80bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.544+0000 7f485f400700 1 -- 172.21.15.46:0/2804260767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4850008930 con 0x557acbf7d720 2024-05-08T23:19:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.546+0000 7f483fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.546+0000 7f485f400700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f484c09b440 tx=0x7f484c09b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feedcf20e1bf9a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.549+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c09b4b0 con 0x557acbf7d720 2024-05-08T23:19:57.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.549+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557acbe20270 con 0x557acbf7d720 2024-05-08T23:19:57.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.549+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f484c09f2e0 con 0x557acbf7d720 2024-05-08T23:19:57.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.549+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f484c09b4b0 con 0x557acbf7d720 2024-05-08T23:19:57.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.551+0000 7f4879019b80 10 client.29920 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:57.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.551+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557acbcf4b60 con 0x557acbf7d720 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557acbe20270 con 0x557acbf7d720 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f484c0a3410 con 0x557acbf7d720 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 10 client.29920 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 10 client.29920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 20 client.29920 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbf90e40 0x557acbf93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 4 client.29920 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557acbf46810 con 0x557acbf90e40 2024-05-08T23:19:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.553+0000 7f485ea00700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbf90e40 0x557acbf93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.554+0000 7f485ea00700 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbf90e40 0x557acbf93230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9485b404d9c6fc56 server_cookie=3e4a42080141f181 in_seq=0 out_seq=0 2024-05-08T23:19:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.554+0000 7f483fe00700 10 client.29920 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:57.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.560+0000 7f483fe00700 1 -- 172.21.15.46:0/2804260767 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557acbf46810 con 0x557acbf90e40 2024-05-08T23:19:57.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.560+0000 7f483fe00700 10 client.29920 handle_command_reply: tid=0 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.560+0000 7f4879019b80 1 client.29920 shutdown 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.560+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 msgr2=0x557acbf81630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.560+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f484c09b440 tx=0x7f484c09b070 comp rx=0 tx=0).stop 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 shutdown_connections 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbf81630 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbf90e40 0x557acbf93230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 --2- 172.21.15.46:0/2804260767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 >> 172.21.15.46:0/2804260767 conn(0x557acbed5210 msgr2=0x557acbf715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f483fe00700 0 client.29920 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:57.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 shutdown_connections 2024-05-08T23:19:57.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.561+0000 7f4879019b80 1 -- 172.21.15.46:0/2804260767 wait complete. 2024-05-08T23:19:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.564+0000 7f4879019b80 20 client.29920 trim_cache size 0 max 16384 2024-05-08T23:19:57.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.604+0000 7f4879019b80 1 Processor -- start 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 -- start start 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7d720 0x557acbf7f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbf82220 con 0x557acbf7d720 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbf73800 con 0x557acbf77890 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.606+0000 7f4879019b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbf982b0 con 0x557acbf7ce00 2024-05-08T23:19:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40282/0 (socket says 172.21.15.46:40282) 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 -- 172.21.15.46:0/2268317098 learned_addr learned my addr 172.21.15.46:0/2268317098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 -- 172.21.15.46:0/2268317098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 msgr2=0x557acbf71d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf71d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 -- 172.21.15.46:0/2268317098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7d720 msgr2=0x557acbf7f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7d720 0x557acbf7f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.607+0000 7f485e000700 1 -- 172.21.15.46:0/2268317098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557acbe8d560 con 0x557acbf7ce00 2024-05-08T23:19:57.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.608+0000 7f485e000700 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4850008cd0 tx=0x7f4850005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6873515933f7c21c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.611+0000 7f484be00700 1 -- 172.21.15.46:0/2268317098 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000ec90 con 0x557acbf7ce00 2024-05-08T23:19:57.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.611+0000 7f484be00700 1 -- 172.21.15.46:0/2268317098 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f485001d040 con 0x557acbf7ce00 2024-05-08T23:19:57.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.611+0000 7f484be00700 1 -- 172.21.15.46:0/2268317098 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850003e10 con 0x557acbf7ce00 2024-05-08T23:19:57.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 -- 172.21.15.46:0/2268317098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 msgr2=0x557acbf81450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4850008cd0 tx=0x7f4850005d20 comp rx=0 tx=0).stop 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 -- 172.21.15.46:0/2268317098 shutdown_connections 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf7d720 0x557acbf7f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf77890 0x557acbf71d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 --2- 172.21.15.46:0/2268317098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7ce00 0x557acbf81450 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.614+0000 7f4879019b80 1 -- 172.21.15.46:0/2268317098 >> 172.21.15.46:0/2268317098 conn(0x557acbed5210 msgr2=0x557acbf77c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.615+0000 7f4879019b80 1 -- 172.21.15.46:0/2268317098 shutdown_connections 2024-05-08T23:19:57.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.616+0000 7f4879019b80 1 -- 172.21.15.46:0/2268317098 wait complete. 2024-05-08T23:19:57.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.617+0000 7f4879019b80 1 -- start start 2024-05-08T23:19:57.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.617+0000 7f4879019b80 1 Processor -- start 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.618+0000 7f4879019b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.618+0000 7f4879019b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.618+0000 7f4879019b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbfe6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35942/0 (socket says 172.21.15.46:35942) 2024-05-08T23:19:57.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 -- 172.21.15.46:0/827974150 learned_addr learned my addr 172.21.15.46:0/827974150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbf86fa0 con 0x557acbf77890 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbfe66c0 con 0x557acbf7ce00 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acbfe6800 con 0x557acbf7d720 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485f400700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbfe6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485ea00700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 -- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 msgr2=0x557acbfe6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbfe6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 -- 172.21.15.46:0/827974150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 msgr2=0x557acbf86150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf86150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.619+0000 7f485e000700 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557acbd483b0 con 0x557acbf7ce00 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.620+0000 7f4879019b80 10 client.0 _ll_register_callbacks cb 0x557acbe6cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.620+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4850004cf0 con 0x557acbf7ce00 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.620+0000 7f485e000700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4850000c00 tx=0x7f4850005070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaee4c41d9ea8d9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.620+0000 7f483fe00700 10 client.29846 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f485ea00700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf86150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:57.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f483fe00700 1 -- 172.21.15.46:0/827974150 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850003a90 con 0x557acbf7ce00 2024-05-08T23:19:57.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f483fe00700 1 -- 172.21.15.46:0/827974150 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4850003bf0 con 0x557acbf7ce00 2024-05-08T23:19:57.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f483fe00700 1 -- 172.21.15.46:0/827974150 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4850003a90 con 0x557acbf7ce00 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f4879019b80 10 client.29846 fetch_fsmap learned FSMap version 34 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.621+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x557acbd4ccd0 con 0x557acbf7ce00 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f483fe00700 1 -- 172.21.15.46:0/827974150 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4850033050 con 0x557acbf7ce00 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 10 client.29846 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 10 client.29846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 20 client.29846 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbfe78c0 0x557acbfe9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 4 client.29846 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557acbebfec0 con 0x557acbfe78c0 2024-05-08T23:19:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.622+0000 7f485ea00700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbfe78c0 0x557acbfe9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.625+0000 7f485ea00700 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbfe78c0 0x557acbfe9ce0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8658b78ebb95500a server_cookie=34d0196e412b9eea in_seq=0 out_seq=0 2024-05-08T23:19:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.625+0000 7f483fe00700 10 client.29846 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.626+0000 7f483fe00700 1 -- 172.21.15.46:0/827974150 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557acbebfec0 con 0x557acbfe78c0 2024-05-08T23:19:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.626+0000 7f483fe00700 10 client.29846 handle_command_reply: tid=0 2024-05-08T23:19:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.626+0000 7f4879019b80 1 client.29846 shutdown 2024-05-08T23:19:57.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.626+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 msgr2=0x557acbf86690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.626+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4850000c00 tx=0x7f4850005070 comp rx=0 tx=0).stop 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 shutdown_connections 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557acbf7d720 0x557acbfe6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x557acbfe78c0 0x557acbfe9ce0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557acbf77890 0x557acbf86150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 --2- 172.21.15.46:0/827974150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557acbf7ce00 0x557acbf86690 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f483fe00700 0 client.29846 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.627+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 >> 172.21.15.46:0/827974150 conn(0x557acbed5210 msgr2=0x557acbf7f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.628+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 shutdown_connections 2024-05-08T23:19:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.629+0000 7f4879019b80 1 -- 172.21.15.46:0/827974150 wait complete. 2024-05-08T23:19:57.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.629+0000 7f4879019b80 20 client.29846 trim_cache size 0 max 16384 2024-05-08T23:19:57.631 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 msgr2=0x7f48440a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 0x7f48440a38a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4858007fd0 tx=0x7f48580079e0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 msgr2=0x7f48601c2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f484c005440 tx=0x7f484c005470 comp rx=0 tx=0).stop 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 shutdown_connections 2024-05-08T23:19:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486010d540 0x7f48601c2bb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601074e0 0x7f486010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48440a13e0 0x7f48440a38a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 --2- 172.21.15.46:0/2358443548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4860107ea0 0x7f4860110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.631+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 >> 172.21.15.46:0/2358443548 conn(0x7f4860076d70 msgr2=0x7f48600772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.632+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 shutdown_connections 2024-05-08T23:19:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:57.632+0000 7f4865600700 1 -- 172.21.15.46:0/2358443548 wait complete. 2024-05-08T23:19:57.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:19:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.041+0000 7f85eae00700 1 -- 172.21.15.46:0/206293588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e4103c50 msgr2=0x7f85e4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.041+0000 7f85eae00700 1 --2- 172.21.15.46:0/206293588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e4103c50 0x7f85e4107c90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.042+0000 7f85eae00700 1 -- 172.21.15.46:0/206293588 shutdown_connections 2024-05-08T23:19:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.042+0000 7f85eae00700 1 --2- 172.21.15.46:0/206293588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 0x7f85e410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.042+0000 7f85eae00700 1 --2- 172.21.15.46:0/206293588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e41032a0 0x7f85e4103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.042+0000 7f85eae00700 1 --2- 172.21.15.46:0/206293588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e4103c50 0x7f85e4107c90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.042+0000 7f85eae00700 1 -- 172.21.15.46:0/206293588 >> 172.21.15.46:0/206293588 conn(0x7f85e40feae0 msgr2=0x7f85e4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.043+0000 7f85eae00700 1 -- 172.21.15.46:0/206293588 shutdown_connections 2024-05-08T23:19:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.043+0000 7f85eae00700 1 -- 172.21.15.46:0/206293588 wait complete. 2024-05-08T23:19:58.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.045+0000 7f85eae00700 1 Processor -- start 2024-05-08T23:19:58.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.045+0000 7f85eae00700 1 -- start start 2024-05-08T23:19:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 0x7f85e410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 0x7f85e410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41aa0d0 con 0x7f85e41083c0 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41aa240 con 0x7f85e41032a0 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85eae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41aa400 con 0x7f85e410d4f0 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35964/0 (socket says 172.21.15.46:35964) 2024-05-08T23:19:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85e3e00700 1 -- 172.21.15.46:0/2635094607 learned_addr learned my addr 172.21.15.46:0/2635094607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85e3400700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 0x7f85e410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.046+0000 7f85e8e00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 0x7f85e410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 -- 172.21.15.46:0/2635094607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 msgr2=0x7f85e410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 0x7f85e410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 -- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 msgr2=0x7f85e410cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 0x7f85e410cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 -- 172.21.15.46:0/2635094607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00095f0 con 0x7f85e41032a0 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e8e00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 0x7f85e410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.047+0000 7f85e3e00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9a0 tx=0x7f85d400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.048+0000 7f85e0c00700 1 -- 172.21.15.46:0/2635094607 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400cbd0 con 0x7f85e41032a0 2024-05-08T23:19:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.048+0000 7f85e0c00700 1 -- 172.21.15.46:0/2635094607 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85d400cd30 con 0x7f85e41032a0 2024-05-08T23:19:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.048+0000 7f85e0c00700 1 -- 172.21.15.46:0/2635094607 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018680 con 0x7f85e41032a0 2024-05-08T23:19:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.048+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e41aa6e0 con 0x7f85e41032a0 2024-05-08T23:19:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.048+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41aaba0 con 0x7f85e41032a0 2024-05-08T23:19:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.056+0000 7f85e0c00700 1 -- 172.21.15.46:0/2635094607 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f85d4010430 con 0x7f85e41032a0 2024-05-08T23:19:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.057+0000 7f85e0c00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 0x7f85c80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.057+0000 7f85e0c00700 1 -- 172.21.15.46:0/2635094607 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f85d4014070 con 0x7f85e41032a0 2024-05-08T23:19:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.058+0000 7f85e3400700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 0x7f85c80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.059+0000 7f85fe935b80 1 Processor -- start 2024-05-08T23:19:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.059+0000 7f85e3400700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 0x7f85c80a38a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85d000b610 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 -- start start 2024-05-08T23:19:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3bfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3b4f150 con 0x5626d3bf8890 2024-05-08T23:19:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3b4cb10 con 0x5626d3bfde00 2024-05-08T23:19:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.061+0000 7f85fe935b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3b48400 con 0x5626d3bfe720 2024-05-08T23:19:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45480/0 (socket says 172.21.15.46:45480) 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 -- 172.21.15.46:0/1326865685 learned_addr learned my addr 172.21.15.46:0/1326865685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e3400700 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 -- 172.21.15.46:0/1326865685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 msgr2=0x5626d3bf8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 -- 172.21.15.46:0/1326865685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 msgr2=0x5626d3bfe1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3bfe1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.062+0000 7f85e8e00700 1 -- 172.21.15.46:0/1326865685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d39c93b0 con 0x5626d3bf8890 2024-05-08T23:19:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.066+0000 7f85e8e00700 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009c10 tx=0x7f85dc009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3b2429f69cc16f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.066+0000 7f85d9600700 1 -- 172.21.15.46:0/1326865685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc014400 con 0x5626d3bf8890 2024-05-08T23:19:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.067+0000 7f85d9600700 1 -- 172.21.15.46:0/1326865685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85dc014a20 con 0x5626d3bf8890 2024-05-08T23:19:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.067+0000 7f85d9600700 1 -- 172.21.15.46:0/1326865685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc014400 con 0x5626d3bf8890 2024-05-08T23:19:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.068+0000 7f85fe935b80 1 -- 172.21.15.46:0/1326865685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 msgr2=0x5626d3bf8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.068+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009c10 tx=0x7f85dc009f20 comp rx=0 tx=0).stop 2024-05-08T23:19:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 -- 172.21.15.46:0/1326865685 shutdown_connections 2024-05-08T23:19:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8cd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3bfe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1326865685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 -- 172.21.15.46:0/1326865685 >> 172.21.15.46:0/1326865685 conn(0x5626d3b56210 msgr2=0x5626d3bf2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.069+0000 7f85fe935b80 1 -- 172.21.15.46:0/1326865685 shutdown_connections 2024-05-08T23:19:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.070+0000 7f85fe935b80 1 -- 172.21.15.46:0/1326865685 wait complete. 2024-05-08T23:19:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.070+0000 7f85fe935b80 1 -- start start 2024-05-08T23:19:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.071+0000 7f85fe935b80 1 Processor -- start 2024-05-08T23:19:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.072+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3c01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.072+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.072+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3c02670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85fe935b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3ae4560 con 0x5626d3bf8890 2024-05-08T23:19:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85fe935b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3ae4200 con 0x5626d3bfde00 2024-05-08T23:19:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85fe935b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d39cb6a0 con 0x5626d3bfe720 2024-05-08T23:19:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85fe935b80 10 client.0 _ll_register_callbacks cb 0x5626d3aedb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3c01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85e8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3c02670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85e8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3c02670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40326/0 (socket says 172.21.15.46:40326) 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.073+0000 7f85e8e00700 1 -- 172.21.15.46:0/1815662980 learned_addr learned my addr 172.21.15.46:0/1815662980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.074+0000 7f85e3e00700 1 -- 172.21.15.46:0/1815662980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 msgr2=0x5626d3c02670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.074+0000 7f85e3e00700 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3c02670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.074+0000 7f85e3e00700 1 -- 172.21.15.46:0/1815662980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 msgr2=0x5626d3c02130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.074+0000 7f85e3e00700 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.074+0000 7f85e3e00700 1 -- 172.21.15.46:0/1815662980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc009840 con 0x5626d3bf8890 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.075+0000 7f85e3e00700 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3c01bf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d40ca690 tx=0x7f85d4099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7c87ba2b7c7fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.075+0000 7f85ce000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:19:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.077+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40b1070 con 0x5626d3bf8890 2024-05-08T23:19:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.077+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f85d409a080 con 0x5626d3bf8890 2024-05-08T23:19:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.077+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626d3aa1270 con 0x5626d3bf8890 2024-05-08T23:19:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.078+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d409d030 con 0x5626d3bf8890 2024-05-08T23:19:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.078+0000 7f85fe935b80 10 client.19737 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.078+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5626d3975b60 con 0x5626d3bf8890 2024-05-08T23:19:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.079+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626d3aa1270 con 0x5626d3bf8890 2024-05-08T23:19:58.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.080+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f85d4099690 con 0x5626d3bf8890 2024-05-08T23:19:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.081+0000 7f85fe935b80 10 client.19737 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.081+0000 7f85fe935b80 10 client.19737 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:19:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.082+0000 7f85fe935b80 20 client.19737 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.082+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c122a0 0x5626d3c14690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.082+0000 7f85fe935b80 4 client.19737 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.082+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626d3bc7810 con 0x5626d3c122a0 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.082+0000 7f85e3400700 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c122a0 0x5626d3c14690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.083+0000 7f85e3400700 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c122a0 0x5626d3c14690 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=319d2dbf45d8ee0c server_cookie=c81cdf2f1e90ee57 in_seq=0 out_seq=0 2024-05-08T23:19:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.084+0000 7f85ce000700 10 client.19737 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.089+0000 7f85ce000700 1 -- 172.21.15.46:0/1815662980 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626d3bc7810 con 0x5626d3c122a0 2024-05-08T23:19:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.089+0000 7f85ce000700 10 client.19737 handle_command_reply: tid=0 2024-05-08T23:19:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.090+0000 7f85fe935b80 1 client.19737 shutdown 2024-05-08T23:19:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.090+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 msgr2=0x5626d3c01bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.090+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3c01bf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d40ca690 tx=0x7f85d4099a60 comp rx=0 tx=0).stop 2024-05-08T23:19:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.091+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 shutdown_connections 2024-05-08T23:19:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3c02670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c122a0 0x5626d3c14690 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3c01bf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 --2- 172.21.15.46:0/1815662980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c02130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 >> 172.21.15.46:0/1815662980 conn(0x5626d3b56210 msgr2=0x5626d3bf25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85ce000700 0 client.19737 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.092+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 shutdown_connections 2024-05-08T23:19:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.093+0000 7f85fe935b80 1 -- 172.21.15.46:0/1815662980 wait complete. 2024-05-08T23:19:58.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.093+0000 7f85fe935b80 20 client.19737 trim_cache size 0 max 16384 2024-05-08T23:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.124+0000 7f85fe935b80 1 Processor -- start 2024-05-08T23:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.124+0000 7f85fe935b80 1 -- start start 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3bf3930 con 0x5626d3bf8890 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3bfe430 con 0x5626d3bfde00 2024-05-08T23:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85fe935b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3bffb10 con 0x5626d3bfe720 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.125+0000 7f85e3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.126+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.126+0000 7f85e3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35992/0 (socket says 172.21.15.46:35992) 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.126+0000 7f85e3400700 1 -- 172.21.15.46:0/3626746892 learned_addr learned my addr 172.21.15.46:0/3626746892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.126+0000 7f85e3400700 1 -- 172.21.15.46:0/3626746892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 msgr2=0x5626d3bf2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.126+0000 7f85e3400700 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.128+0000 7f85e3400700 1 -- 172.21.15.46:0/3626746892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 msgr2=0x5626d3bf8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.128+0000 7f85e3400700 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.128+0000 7f85e3400700 1 -- 172.21.15.46:0/3626746892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d3bef4e0 con 0x5626d3bfde00 2024-05-08T23:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.129+0000 7f85e3400700 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721b5914fa7f7e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.129+0000 7f85da000700 1 -- 172.21.15.46:0/3626746892 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d000bac0 con 0x5626d3bfde00 2024-05-08T23:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.129+0000 7f85da000700 1 -- 172.21.15.46:0/3626746892 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85d002c040 con 0x5626d3bfde00 2024-05-08T23:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.129+0000 7f85da000700 1 -- 172.21.15.46:0/3626746892 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0004760 con 0x5626d3bfde00 2024-05-08T23:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 -- 172.21.15.46:0/3626746892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 msgr2=0x5626d3c0e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d001f000 comp rx=0 tx=0).stop 2024-05-08T23:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 -- 172.21.15.46:0/3626746892 shutdown_connections 2024-05-08T23:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bfe720 0x5626d3bf2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bf8890 0x5626d3bf8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 --2- 172.21.15.46:0/3626746892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3bfde00 0x5626d3c0e340 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d001f000 comp rx=0 tx=0).stop 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 -- 172.21.15.46:0/3626746892 >> 172.21.15.46:0/3626746892 conn(0x5626d3b56210 msgr2=0x5626d3c004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.131+0000 7f85fe935b80 1 -- 172.21.15.46:0/3626746892 shutdown_connections 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.132+0000 7f85fe935b80 1 -- 172.21.15.46:0/3626746892 wait complete. 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.132+0000 7f85fe935b80 1 -- start start 2024-05-08T23:19:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.133+0000 7f85fe935b80 1 Processor -- start 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bfe720 0x5626d3c078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3c65730 0x5626d3c67b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3c68060 con 0x5626d3bfe720 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3c681a0 con 0x5626d3c65730 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d3c682e0 con 0x5626d3bf8890 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85fe935b80 10 client.0 _ll_register_callbacks cb 0x5626d3aedb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.134+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.135+0000 7f85e3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40334/0 (socket says 172.21.15.46:40334) 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.135+0000 7f85e3e00700 1 -- 172.21.15.46:0/812771819 learned_addr learned my addr 172.21.15.46:0/812771819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.135+0000 7f85e8e00700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3c65730 0x5626d3c67b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.135+0000 7f85e3400700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bfe720 0x5626d3c078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 -- 172.21.15.46:0/812771819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3c65730 msgr2=0x5626d3c67b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3c65730 0x5626d3c67b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 -- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bfe720 msgr2=0x5626d3c078c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bfe720 0x5626d3c078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 -- 172.21.15.46:0/812771819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41107b0 con 0x5626d3bf8890 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85e3e00700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f85d4096af0 tx=0x7f85d40ca6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f073d430c9b9f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.136+0000 7f85ce000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.138+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40b1060 con 0x5626d3bf8890 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.138+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f85d409a080 con 0x5626d3bf8890 2024-05-08T23:19:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.138+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85dc014a20 con 0x5626d3bf8890 2024-05-08T23:19:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.141+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d409d070 con 0x5626d3bf8890 2024-05-08T23:19:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.141+0000 7f85fe935b80 10 client.29938 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.141+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5626d3b0e560 con 0x5626d3bf8890 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.141+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85dc014a20 con 0x5626d3bf8890 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.141+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f85d40106e0 con 0x5626d3bf8890 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 10 client.29938 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 10 client.29938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 20 client.29938 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c7a840 0x5626d3c7cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 4 client.29938 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.142+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626d3b40ec0 con 0x5626d3c7a840 2024-05-08T23:19:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.143+0000 7f85e3400700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c7a840 0x5626d3c7cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.146+0000 7f85e3400700 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c7a840 0x5626d3c7cc60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a28f63f45d9b7477 server_cookie=abd0f1f8163a971e in_seq=0 out_seq=0 2024-05-08T23:19:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.146+0000 7f85ce000700 10 client.29938 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.146+0000 7f85ce000700 1 -- 172.21.15.46:0/812771819 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626d3b40ec0 con 0x5626d3c7a840 2024-05-08T23:19:58.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.146+0000 7f85ce000700 10 client.29938 handle_command_reply: tid=0 2024-05-08T23:19:58.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.147+0000 7f85fe935b80 1 client.29938 shutdown 2024-05-08T23:19:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 msgr2=0x5626d3c07380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f85d4096af0 tx=0x7f85d40ca6c0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 shutdown_connections 2024-05-08T23:19:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626d3c65730 0x5626d3c67b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3bf8890 0x5626d3c07380 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5626d3c7a840 0x5626d3c7cc60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 --2- 172.21.15.46:0/812771819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626d3bfe720 0x5626d3c078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 >> 172.21.15.46:0/812771819 conn(0x5626d3b56210 msgr2=0x5626d3bfcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85ce000700 0 client.29938 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.148+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 shutdown_connections 2024-05-08T23:19:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.149+0000 7f85fe935b80 1 -- 172.21.15.46:0/812771819 wait complete. 2024-05-08T23:19:58.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.149+0000 7f85fe935b80 20 client.29938 trim_cache size 0 max 16384 2024-05-08T23:19:58.150 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 msgr2=0x7f85c80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 0x7f85c80a38a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85d0000c00 tx=0x7f85d000b610 comp rx=0 tx=0).stop 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 msgr2=0x7f85e410fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9a0 tx=0x7f85d400ed60 comp rx=0 tx=0).stop 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 shutdown_connections 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e410d4f0 0x7f85e410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85e41032a0 0x7f85e410fe90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f85c80a13e0 0x7f85c80a38a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 --2- 172.21.15.46:0/2635094607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e41083c0 0x7f85e410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.151+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 >> 172.21.15.46:0/2635094607 conn(0x7f85e40feae0 msgr2=0x7f85e40ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.152+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 shutdown_connections 2024-05-08T23:19:58.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.152+0000 7f85eae00700 1 -- 172.21.15.46:0/2635094607 wait complete. 2024-05-08T23:19:58.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:19:58.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 56178 19.94 MB/sec execute 19 sec latency 127.069 ms 2024-05-08T23:19:58.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:57 smithi053 ceph-mon[77587]: pgmap v505: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 549 op/s 2024-05-08T23:19:58.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:57 smithi046 ceph-mon[36898]: pgmap v505: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 549 op/s 2024-05-08T23:19:58.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:57 smithi086 ceph-mon[78610]: pgmap v505: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 549 op/s 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 -- 172.21.15.46:0/4101550064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47301083d0 msgr2=0x7f473010ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 --2- 172.21.15.46:0/4101550064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47301083d0 0x7f473010ff50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4728009f90 tx=0x7f472800b330 comp rx=0 tx=0).stop 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 -- 172.21.15.46:0/4101550064 shutdown_connections 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 --2- 172.21.15.46:0/4101550064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47301083d0 0x7f473010ff50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 --2- 172.21.15.46:0/4101550064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301032b0 0x7f4730103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 --2- 172.21.15.46:0/4101550064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730103c60 0x7f4730107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 -- 172.21.15.46:0/4101550064 >> 172.21.15.46:0/4101550064 conn(0x7f47300feb10 msgr2=0x7f4730100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 -- 172.21.15.46:0/4101550064 shutdown_connections 2024-05-08T23:19:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.521+0000 7f4735600700 1 -- 172.21.15.46:0/4101550064 wait complete. 2024-05-08T23:19:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 Processor -- start 2024-05-08T23:19:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 -- start start 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301083d0 0x7f4730199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 0x7f473019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473019e9b0 con 0x7f473019df30 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473019eb20 con 0x7f47301083d0 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.523+0000 7f4735600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473019ece0 con 0x7f47301032b0 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40356/0 (socket says 172.21.15.46:40356) 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 -- 172.21.15.46:0/732212874 learned_addr learned my addr 172.21.15.46:0/732212874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472e000700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301083d0 0x7f4730199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472f400700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 0x7f473019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 -- 172.21.15.46:0/732212874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301083d0 msgr2=0x7f4730199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301083d0 0x7f4730199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 -- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 msgr2=0x7f473019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 0x7f473019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 -- 172.21.15.46:0/732212874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200095f0 con 0x7f47301032b0 2024-05-08T23:19:58.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472f400700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 0x7f473019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:58.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.524+0000 7f472ea00700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f472000ec10 tx=0x7f472000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.525+0000 7f4727400700 1 -- 172.21.15.46:0/732212874 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000cd40 con 0x7f47301032b0 2024-05-08T23:19:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.525+0000 7f4727400700 1 -- 172.21.15.46:0/732212874 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47200044e0 con 0x7f47301032b0 2024-05-08T23:19:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.525+0000 7f4727400700 1 -- 172.21.15.46:0/732212874 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720010690 con 0x7f47301032b0 2024-05-08T23:19:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.525+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728009c40 con 0x7f47301032b0 2024-05-08T23:19:58.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.525+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a6590 con 0x7f47301032b0 2024-05-08T23:19:58.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.529+0000 7f4727400700 1 -- 172.21.15.46:0/732212874 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4720003710 con 0x7f47301032b0 2024-05-08T23:19:58.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 Processor -- start 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 -- start start 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f17ff150 con 0x5646f18b6f80 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.530+0000 7f47491d4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f17fcb10 con 0x5646f18b10f0 2024-05-08T23:19:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f47491d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f17f8400 con 0x5646f18b6660 2024-05-08T23:19:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40372/0 (socket says 172.21.15.46:40372) 2024-05-08T23:19:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 -- 172.21.15.46:0/4203449089 learned_addr learned my addr 172.21.15.46:0/4203449089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472e000700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472ea00700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 -- 172.21.15.46:0/4203449089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 msgr2=0x5646f18b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.531+0000 7f472f400700 1 -- 172.21.15.46:0/4203449089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 msgr2=0x5646f18b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f472f400700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f472f400700 1 -- 172.21.15.46:0/4203449089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f16793b0 con 0x5646f18b6660 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f472ea00700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f4727400700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f4727400700 1 -- 172.21.15.46:0/732212874 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4720014070 con 0x7f47301032b0 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f472f400700 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4728009350 tx=0x7f4728009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7499be2b60c2faec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.532+0000 7f472ea00700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.533+0000 7f471b400700 1 -- 172.21.15.46:0/4203449089 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800e040 con 0x5646f18b6660 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.533+0000 7f471b400700 1 -- 172.21.15.46:0/4203449089 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4728004a50 con 0x5646f18b6660 2024-05-08T23:19:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.533+0000 7f471b400700 1 -- 172.21.15.46:0/4203449089 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800bbf0 con 0x5646f18b6660 2024-05-08T23:19:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 -- 172.21.15.46:0/4203449089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 msgr2=0x5646f18b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4728009350 tx=0x7f4728009380 comp rx=0 tx=0).stop 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f472ea00700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 0x7f47140a37d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5646f18a4a60 tx=0x7f4720000bc0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 -- 172.21.15.46:0/4203449089 shutdown_connections 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18b6a40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 --2- 172.21.15.46:0/4203449089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 -- 172.21.15.46:0/4203449089 >> 172.21.15.46:0/4203449089 conn(0x5646f1806210 msgr2=0x5646f18a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 -- 172.21.15.46:0/4203449089 shutdown_connections 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.534+0000 7f47491d4b80 1 -- 172.21.15.46:0/4203449089 wait complete. 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 -- start start 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 Processor -- start 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 0x5646f18bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6f80 0x5646f18af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f1794560 con 0x5646f18b6660 2024-05-08T23:19:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f1794200 con 0x5646f18b6f80 2024-05-08T23:19:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f167b6a0 con 0x5646f18b10f0 2024-05-08T23:19:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.535+0000 7f47491d4b80 10 client.0 _ll_register_callbacks cb 0x5646f179db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40376/0 (socket says 172.21.15.46:40376) 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 -- 172.21.15.46:0/2436489313 learned_addr learned my addr 172.21.15.46:0/2436489313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6f80 0x5646f18af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472e000700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 0x5646f18bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 -- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6f80 msgr2=0x5646f18af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6f80 0x5646f18af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 -- 172.21.15.46:0/2436489313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 msgr2=0x5646f18bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 0x5646f18bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472f400700 1 -- 172.21.15.46:0/2436489313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301107c0 con 0x5646f18b10f0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.536+0000 7f472e000700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 0x5646f18bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f472f400700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4728009c10 tx=0x7f4728007c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebdbe30c32c28989 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f470fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800e070 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f47280074a0 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646f1751270 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4728007660 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f47491d4b80 10 client.29950 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5646f1625b60 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.537+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4728034020 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4728019040 con 0x5646f18b10f0 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 10 client.29950 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 10 client.29950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 20 client.29950 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f18d2e90 0x5646f18d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 4 client.29950 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646f1877810 con 0x5646f18d2e90 2024-05-08T23:19:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.538+0000 7f472e000700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f18d2e90 0x5646f18d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.539+0000 7f470fe00700 10 client.29950 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.539+0000 7f472e000700 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f18d2e90 0x5646f18d52b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1315ba2622899636 server_cookie=5467d87e09ce6b99 in_seq=0 out_seq=0 2024-05-08T23:19:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f470fe00700 1 -- 172.21.15.46:0/2436489313 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646f1877810 con 0x5646f18d2e90 2024-05-08T23:19:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f470fe00700 10 client.29950 handle_command_reply: tid=0 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 client.29950 shutdown 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 msgr2=0x5646f18baa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4728009c10 tx=0x7f4728007c40 comp rx=0 tx=0).stop 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 shutdown_connections 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6f80 0x5646f18af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18baa60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f18d2e90 0x5646f18d52b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 --2- 172.21.15.46:0/2436489313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6660 0x5646f18bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f470fe00700 0 client.29950 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.540+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 >> 172.21.15.46:0/2436489313 conn(0x5646f1806210 msgr2=0x5646f18a25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.541+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 shutdown_connections 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.541+0000 7f47491d4b80 1 -- 172.21.15.46:0/2436489313 wait complete. 2024-05-08T23:19:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.541+0000 7f47491d4b80 20 client.29950 trim_cache size 0 max 16384 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 Processor -- start 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 -- start start 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f18bb980 con 0x5646f18b6f80 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f18b88e0 con 0x5646f18b6660 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f47491d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f18b8d80 con 0x5646f18b10f0 2024-05-08T23:19:58.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.557+0000 7f472e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36038/0 (socket says 172.21.15.46:36038) 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 -- 172.21.15.46:0/1708636022 learned_addr learned my addr 172.21.15.46:0/1708636022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472f400700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472f400700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 -- 172.21.15.46:0/1708636022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 msgr2=0x5646f18a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 -- 172.21.15.46:0/1708636022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 msgr2=0x5646f18a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 -- 172.21.15.46:0/1708636022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f17be560 con 0x5646f18b6660 2024-05-08T23:19:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472f400700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f472e000700 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f473019aba0 tx=0x7f471c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8092fd6be22d9300 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f471be00700 1 -- 172.21.15.46:0/1708636022 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c017040 con 0x5646f18b6660 2024-05-08T23:19:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f471be00700 1 -- 172.21.15.46:0/1708636022 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f471c020070 con 0x5646f18b6660 2024-05-08T23:19:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.558+0000 7f471be00700 1 -- 172.21.15.46:0/1708636022 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c005440 con 0x5646f18b6660 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 -- 172.21.15.46:0/1708636022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 msgr2=0x5646f18b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f473019aba0 tx=0x7f471c009ba0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 -- 172.21.15.46:0/1708636022 shutdown_connections 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f18a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b10f0 0x5646f18a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 --2- 172.21.15.46:0/1708636022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b6660 0x5646f18b8370 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 -- 172.21.15.46:0/1708636022 >> 172.21.15.46:0/1708636022 conn(0x5646f1806210 msgr2=0x5646f18b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 -- 172.21.15.46:0/1708636022 shutdown_connections 2024-05-08T23:19:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.559+0000 7f47491d4b80 1 -- 172.21.15.46:0/1708636022 wait complete. 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 -- start start 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 Processor -- start 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f1928350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f18ab610 con 0x5646f18b6f80 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f1928890 con 0x5646f18b10f0 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.560+0000 7f47491d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f1928a20 con 0x5646f18b6660 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f47491d4b80 10 client.0 _ll_register_callbacks cb 0x5646f179db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40390/0 (socket says 172.21.15.46:40390) 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 -- 172.21.15.46:0/872177371 learned_addr learned my addr 172.21.15.46:0/872177371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472f400700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472f400700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f1928350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 -- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 msgr2=0x5646f18aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 -- 172.21.15.46:0/872177371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 msgr2=0x5646f1928350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f1928350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 -- 172.21.15.46:0/872177371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646f167dcd0 con 0x5646f18b6660 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472f400700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f1928350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f472e000700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f473019aba0 tx=0x7f471c01c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52542e696016c20b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.561+0000 7f470fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c017040 con 0x5646f18b6660 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f471c0052c0 con 0x5646f18b6660 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4728004a50 con 0x5646f18b6660 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471c005480 con 0x5646f18b6660 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 10 client.29956 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5646f1624030 con 0x5646f18b6660 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4728004a50 con 0x5646f18b6660 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f471c003a00 con 0x5646f18b6660 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 10 client.29956 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 10 client.29956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 20 client.29956 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f192a1c0 0x5646f192c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 4 client.29956 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.562+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5646f17f0ec0 con 0x5646f192a1c0 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f472f400700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f192a1c0 0x5646f192c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f472f400700 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f192a1c0 0x5646f192c5e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733407c66d06914b server_cookie=14c385520dbdd6a7 in_seq=0 out_seq=0 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f470fe00700 10 client.29956 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f470fe00700 1 -- 172.21.15.46:0/872177371 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5646f17f0ec0 con 0x5646f192a1c0 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f470fe00700 10 client.29956 handle_command_reply: tid=0 2024-05-08T23:19:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.563+0000 7f47491d4b80 1 client.29956 shutdown 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 msgr2=0x5646f18aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f473019aba0 tx=0x7f471c01c670 comp rx=0 tx=0).stop 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 shutdown_connections 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5646f192a1c0 0x5646f192c5e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646f18b6f80 0x5646f1928350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646f18b10f0 0x5646f18aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 --2- 172.21.15.46:0/872177371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646f18b6660 0x5646f18aaee0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f470fe00700 0 client.29956 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 >> 172.21.15.46:0/872177371 conn(0x5646f1806210 msgr2=0x5646f18b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 shutdown_connections 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 1 -- 172.21.15.46:0/872177371 wait complete. 2024-05-08T23:19:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f47491d4b80 20 client.29956 trim_cache size 0 max 16384 2024-05-08T23:19:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:19:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 msgr2=0x7f47140a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 0x7f47140a37d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5646f18a4a60 tx=0x7f4720000bc0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 msgr2=0x7f4730199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.564+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f472000ec10 tx=0x7f472000ef20 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 shutdown_connections 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473019df30 0x7f473019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47301032b0 0x7f4730199650 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 --2- 172.21.15.46:0/732212874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301083d0 0x7f4730199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 >> 172.21.15.46:0/732212874 conn(0x7f47300feb10 msgr2=0x7f4730100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 shutdown_connections 2024-05-08T23:19:58.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:58.565+0000 7f4735600700 1 -- 172.21.15.46:0/732212874 wait complete. 2024-05-08T23:19:59.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 56589 20.29 MB/sec execute 20 sec latency 189.412 ms 2024-05-08T23:19:59.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:58 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:59.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:19:58 smithi053 ceph-mon[77587]: pgmap v506: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 498 op/s 2024-05-08T23:19:59.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:58 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:59.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:19:58 smithi046 ceph-mon[36898]: pgmap v506: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 498 op/s 2024-05-08T23:19:59.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:58 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:19:59.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:19:58 smithi086 ceph-mon[78610]: pgmap v506: 129 pgs: 129 active+clean; 954 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 498 op/s 2024-05-08T23:19:59.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:19:59.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:19:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 -- 172.21.15.46:0/2190687914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab01028c0 msgr2=0x7fdab010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 --2- 172.21.15.46:0/2190687914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab01028c0 0x7fdab010ada0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009990 tx=0x7fdaa0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 -- 172.21.15.46:0/2190687914 shutdown_connections 2024-05-08T23:19:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 --2- 172.21.15.46:0/2190687914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab010b3c0 0x7fdab010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 --2- 172.21.15.46:0/2190687914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab0101fa0 0x7fdab0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.826+0000 7fdab6e00700 1 --2- 172.21.15.46:0/2190687914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab01028c0 0x7fdab010ada0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.827+0000 7fdab6e00700 1 -- 172.21.15.46:0/2190687914 >> 172.21.15.46:0/2190687914 conn(0x7fdab00fb760 msgr2=0x7fdab00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.827+0000 7fdab6e00700 1 -- 172.21.15.46:0/2190687914 shutdown_connections 2024-05-08T23:19:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.827+0000 7fdab6e00700 1 -- 172.21.15.46:0/2190687914 wait complete. 2024-05-08T23:19:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.828+0000 7fdab6e00700 1 Processor -- start 2024-05-08T23:19:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.828+0000 7fdab6e00700 1 -- start start 2024-05-08T23:19:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab0101fa0 0x7fdab01a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 0x7fdab01a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab010a4c0 con 0x7fdab01028c0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab010a630 con 0x7fdab0101fa0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab010a7f0 con 0x7fdab010b3c0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40420/0 (socket says 172.21.15.46:40420) 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdab4e00700 1 -- 172.21.15.46:0/1616315185 learned_addr learned my addr 172.21.15.46:0/1616315185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.829+0000 7fdaaf400700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 0x7fdab01a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab0101fa0 0x7fdab01a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdab4e00700 1 -- 172.21.15.46:0/1616315185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab0101fa0 msgr2=0x7fdab01a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdab4e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab0101fa0 0x7fdab01a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdab4e00700 1 -- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 msgr2=0x7fdab01a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdab4e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 0x7fdab01a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdab4e00700 1 -- 172.21.15.46:0/1616315185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa00095f0 con 0x7fdab010b3c0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.830+0000 7fdaaf400700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 0x7fdab01a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdab4e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d980 tx=0x7fda9800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdaacc00700 1 -- 172.21.15.46:0/1616315185 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98004a30 con 0x7fdab010b3c0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdaacc00700 1 -- 172.21.15.46:0/1616315185 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9800b5c0 con 0x7fdab010b3c0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdaacc00700 1 -- 172.21.15.46:0/1616315185 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98009b50 con 0x7fdab010b3c0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab010aad0 con 0x7fdab010b3c0 2024-05-08T23:19:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.831+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab006aba0 con 0x7fdab010b3c0 2024-05-08T23:19:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.835+0000 7fdaacc00700 1 -- 172.21.15.46:0/1616315185 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fda98009cb0 con 0x7fdab010b3c0 2024-05-08T23:19:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.837+0000 7fdaacc00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 0x7fda940a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.837+0000 7fdaacc00700 1 -- 172.21.15.46:0/1616315185 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fda980ccfa0 con 0x7fdab010b3c0 2024-05-08T23:19:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.837+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 0x7fda940a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.838+0000 7fdaca948b80 1 Processor -- start 2024-05-08T23:19:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.838+0000 7fdaca948b80 1 -- start start 2024-05-08T23:19:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb68e80 0x55702fb69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb63910 0x55702fb63d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702faba1d0 con 0x55702fb68e80 2024-05-08T23:19:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fab7b90 con 0x55702fb697a0 2024-05-08T23:19:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaca948b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fab3480 con 0x55702fb63910 2024-05-08T23:19:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36076/0 (socket says 172.21.15.46:36076) 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaaf400700 1 -- 172.21.15.46:0/2308701233 learned_addr learned my addr 172.21.15.46:0/2308701233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdab4e00700 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb63910 0x55702fb63d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.839+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 0x7fda940a37d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009550 tx=0x7fdaa8009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb68e80 0x55702fb69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 -- 172.21.15.46:0/2308701233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb63910 msgr2=0x55702fb63d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb63910 0x55702fb63d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 -- 172.21.15.46:0/2308701233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb68e80 msgr2=0x55702fb69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb68e80 0x55702fb69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 -- 172.21.15.46:0/2308701233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702f934430 con 0x55702fb697a0 2024-05-08T23:19:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.840+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0015000 tx=0x7fdaa0005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21a583416c3729f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.841+0000 7fdaa5600700 1 -- 172.21.15.46:0/2308701233 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa001d050 con 0x55702fb697a0 2024-05-08T23:19:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.841+0000 7fdaa5600700 1 -- 172.21.15.46:0/2308701233 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa001e040 con 0x55702fb697a0 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.841+0000 7fdaa5600700 1 -- 172.21.15.46:0/2308701233 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00055a0 con 0x55702fb697a0 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 -- 172.21.15.46:0/2308701233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 msgr2=0x55702fb633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0015000 tx=0x7fdaa0005ee0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 -- 172.21.15.46:0/2308701233 shutdown_connections 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb63910 0x55702fb63d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb68e80 0x55702fb69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2308701233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb633d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 -- 172.21.15.46:0/2308701233 >> 172.21.15.46:0/2308701233 conn(0x55702fac1290 msgr2=0x55702fb5de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.842+0000 7fdaca948b80 1 -- 172.21.15.46:0/2308701233 shutdown_connections 2024-05-08T23:19:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.843+0000 7fdaca948b80 1 -- 172.21.15.46:0/2308701233 wait complete. 2024-05-08T23:19:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.843+0000 7fdaca948b80 1 -- start start 2024-05-08T23:19:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.844+0000 7fdaca948b80 1 Processor -- start 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.844+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb63910 0x55702fb6cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 0x55702fb6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fa4f5e0 con 0x55702fb697a0 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fa4f280 con 0x55702fb63910 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702f936720 con 0x55702fb68e80 2024-05-08T23:19:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaca948b80 10 client.0 _ll_register_callbacks cb 0x55702fa58bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40444/0 (socket says 172.21.15.46:40444) 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 -- 172.21.15.46:0/1506195498 learned_addr learned my addr 172.21.15.46:0/1506195498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb63910 0x55702fb6cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdab4e00700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 0x55702fb6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 -- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb63910 msgr2=0x55702fb6cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb63910 0x55702fb6cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 -- 172.21.15.46:0/1506195498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 msgr2=0x55702fb6d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 0x55702fb6d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdaaf400700 1 -- 172.21.15.46:0/1506195498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab010e540 con 0x55702fb68e80 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.845+0000 7fdab4e00700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 0x55702fb6d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fdaaf400700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00098e0 tx=0x7fdaa0003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8b9ed2c8a2f46f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fda9e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:19:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0005630 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55702fa0c2f0 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdaa000f460 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa0003680 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fdaca948b80 10 client.29968 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.846+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55702f8e0be0 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55702fa0c2f0 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdaa0023460 con 0x55702fb68e80 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 10 client.29968 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 10 client.29968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 20 client.29968 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fb7cd90 0x55702fb7f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 4 client.29968 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55702fb32890 con 0x55702fb7cd90 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.847+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fb7cd90 0x55702fb7f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.848+0000 7fdaafe00700 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fb7cd90 0x55702fb7f1b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6262973c63e51881 server_cookie=49721cd41fbe2337 in_seq=0 out_seq=0 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.848+0000 7fda9e000700 10 client.29968 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fda9e000700 1 -- 172.21.15.46:0/1506195498 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55702fb32890 con 0x55702fb7cd90 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fda9e000700 10 client.29968 handle_command_reply: tid=0 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 client.29968 shutdown 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 msgr2=0x55702fb6d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00098e0 tx=0x7fdaa0003ed0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 shutdown_connections 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fb7cd90 0x55702fb7f1b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb697a0 0x55702fb6d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb63910 0x55702fb6cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 --2- 172.21.15.46:0/1506195498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb6d120 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 >> 172.21.15.46:0/1506195498 conn(0x55702fac1290 msgr2=0x55702fb5d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fda9e000700 0 client.29968 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 shutdown_connections 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.849+0000 7fdaca948b80 1 -- 172.21.15.46:0/1506195498 wait complete. 2024-05-08T23:19:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.850+0000 7fdaca948b80 20 client.29968 trim_cache size 0 max 16384 2024-05-08T23:19:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.874+0000 7fdaca948b80 1 Processor -- start 2024-05-08T23:19:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.874+0000 7fdaca948b80 1 -- start start 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb68550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fb6e250 con 0x55702fb63910 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fb63770 con 0x55702fb697a0 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaca948b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fb5ae40 con 0x55702fb68e80 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40446/0 (socket says 172.21.15.46:40446) 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdaaf400700 1 -- 172.21.15.46:0/2241517406 learned_addr learned my addr 172.21.15.46:0/2241517406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.875+0000 7fdab4e00700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 -- 172.21.15.46:0/2241517406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 msgr2=0x55702fb6bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb6bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 -- 172.21.15.46:0/2241517406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 msgr2=0x55702fb68550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb68550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 -- 172.21.15.46:0/2241517406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702fb6c740 con 0x55702fb68e80 2024-05-08T23:19:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb68550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.876+0000 7fdaaf400700 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009990 tx=0x7fdaa0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1daa16649cde559 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.877+0000 7fdaa6000700 1 -- 172.21.15.46:0/2241517406 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa001d030 con 0x55702fb68e80 2024-05-08T23:19:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.877+0000 7fdaa6000700 1 -- 172.21.15.46:0/2241517406 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa0005980 con 0x55702fb68e80 2024-05-08T23:19:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.877+0000 7fdaa6000700 1 -- 172.21.15.46:0/2241517406 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0016c00 con 0x55702fb68e80 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.877+0000 7fdaca948b80 1 -- 172.21.15.46:0/2241517406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 msgr2=0x55702fb5dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.877+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009990 tx=0x7fdaa0009310 comp rx=0 tx=0).stop 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 -- 172.21.15.46:0/2241517406 shutdown_connections 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb6bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb68550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2241517406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb5dda0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 -- 172.21.15.46:0/2241517406 >> 172.21.15.46:0/2241517406 conn(0x55702fac1290 msgr2=0x55702fb6b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 -- 172.21.15.46:0/2241517406 shutdown_connections 2024-05-08T23:19:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.878+0000 7fdaca948b80 1 -- 172.21.15.46:0/2241517406 wait complete. 2024-05-08T23:19:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.879+0000 7fdaca948b80 1 -- start start 2024-05-08T23:19:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.879+0000 7fdaca948b80 1 Processor -- start 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fb73670 con 0x55702fb63910 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fb737e0 con 0x55702fb697a0 2024-05-08T23:19:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702fbd32a0 con 0x55702fb68e80 2024-05-08T23:19:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.881+0000 7fdaca948b80 10 client.0 _ll_register_callbacks cb 0x55702fa58bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36092/0 (socket says 172.21.15.46:36092) 2024-05-08T23:19:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 -- 172.21.15.46:0/2695509589 learned_addr learned my addr 172.21.15.46:0/2695509589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:19:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 -- 172.21.15.46:0/2695509589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 msgr2=0x55702fb72a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.882+0000 7fdab4e00700 1 -- 172.21.15.46:0/2695509589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 msgr2=0x55702fb74560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.885+0000 7fdab4e00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb74560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.885+0000 7fdab4e00700 1 -- 172.21.15.46:0/2695509589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702f8e0be0 con 0x55702fb697a0 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.885+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.885+0000 7fdab4e00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda980966b0 tx=0x7fda9809a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec187b304620f02f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.885+0000 7fda9e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:19:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.886+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980a20c0 con 0x55702fb697a0 2024-05-08T23:19:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.886+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fda9809a0b0 con 0x55702fb697a0 2024-05-08T23:19:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.886+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaa001e040 con 0x55702fb697a0 2024-05-08T23:19:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda98099130 con 0x55702fb697a0 2024-05-08T23:19:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fdaca948b80 10 client.29894 fetch_fsmap learned FSMap version 35 2024-05-08T23:19:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55702fa795e0 con 0x55702fb697a0 2024-05-08T23:19:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa001e040 con 0x55702fb697a0 2024-05-08T23:19:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fda9805faf0 con 0x55702fb697a0 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.887+0000 7fdaca948b80 10 client.29894 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.888+0000 7fdaca948b80 10 client.29894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.888+0000 7fdaca948b80 20 client.29894 populate_metadata read hostname 'smithi046' 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.888+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fbd4940 0x55702fbd6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.888+0000 7fdaca948b80 4 client.29894 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.888+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55702faabf40 con 0x55702fbd4940 2024-05-08T23:19:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.889+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fbd4940 0x55702fbd6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:59.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.890+0000 7fdaafe00700 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fbd4940 0x55702fbd6d60 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b34d135a454e23c server_cookie=1933359c7d3e7a3 in_seq=0 out_seq=0 2024-05-08T23:19:59.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.891+0000 7fda9e000700 10 client.29894 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.892+0000 7fda9e000700 1 -- 172.21.15.46:0/2695509589 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55702faabf40 con 0x55702fbd4940 2024-05-08T23:19:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.892+0000 7fda9e000700 10 client.29894 handle_command_reply: tid=0 2024-05-08T23:19:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.892+0000 7fdaca948b80 1 client.29894 shutdown 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 msgr2=0x55702fb72f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda980966b0 tx=0x7fda9809a3f0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 shutdown_connections 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55702fb697a0 0x55702fb72f80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55702fbd4940 0x55702fbd6d60 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55702fb63910 0x55702fb74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 --2- 172.21.15.46:0/2695509589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702fb68e80 0x55702fb72a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.893+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 >> 172.21.15.46:0/2695509589 conn(0x55702fac1290 msgr2=0x55702fb6cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.894+0000 7fda9e000700 0 client.29894 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:19:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.894+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 shutdown_connections 2024-05-08T23:19:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.894+0000 7fdaca948b80 1 -- 172.21.15.46:0/2695509589 wait complete. 2024-05-08T23:19:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.895+0000 7fdaca948b80 20 client.29894 trim_cache size 0 max 16384 2024-05-08T23:19:59.895 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:19:59.895 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:19:59.896 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "15ea237f-8cf3-4742-9fae-3fece178fa0d", 2024-05-08T23:19:59.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:19:59.896 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:19:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 msgr2=0x7fda940a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 0x7fda940a37d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009550 tx=0x7fdaa8009040 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 msgr2=0x7fdab0109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d980 tx=0x7fda9800dc90 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 shutdown_connections 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdab010b3c0 0x7fdab0109e60 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab0101fa0 0x7fdab01a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda940a1310 0x7fda940a37d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 --2- 172.21.15.46:0/1616315185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab01028c0 0x7fdab01a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 >> 172.21.15.46:0/1616315185 conn(0x7fdab00fb760 msgr2=0x7fdab0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 shutdown_connections 2024-05-08T23:19:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:19:59.896+0000 7fdab6e00700 1 -- 172.21.15.46:0/1616315185 wait complete. 2024-05-08T23:19:59.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 -- 172.21.15.46:0/3310281853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338106d30 msgr2=0x7f7338107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 --2- 172.21.15.46:0/3310281853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338106d30 0x7f7338107110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f732c0098f0 tx=0x7f732c009c00 comp rx=0 tx=0).stop 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 -- 172.21.15.46:0/3310281853 shutdown_connections 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 --2- 172.21.15.46:0/3310281853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338068b20 0x7f733810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 --2- 172.21.15.46:0/3310281853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338106d30 0x7f7338107110 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 --2- 172.21.15.46:0/3310281853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f73380685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 -- 172.21.15.46:0/3310281853 >> 172.21.15.46:0/3310281853 conn(0x7f7338075710 msgr2=0x7f7338075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 -- 172.21.15.46:0/3310281853 shutdown_connections 2024-05-08T23:20:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.146+0000 7f733e200700 1 -- 172.21.15.46:0/3310281853 wait complete. 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.147+0000 7f733e200700 1 Processor -- start 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 -- start start 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338068b20 0x7f7338197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 0x7f733819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733819bf10 con 0x7f7338106d30 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733819c080 con 0x7f7338068b20 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733819c240 con 0x7f7338068160 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40470/0 (socket says 172.21.15.46:40470) 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733ce00700 1 -- 172.21.15.46:0/2249691083 learned_addr learned my addr 172.21.15.46:0/2249691083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f7337e00700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338068b20 0x7f7338197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.148+0000 7f733d800700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 0x7f733819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 -- 172.21.15.46:0/2249691083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338068b20 msgr2=0x7f7338197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338068b20 0x7f7338197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 -- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 msgr2=0x7f733819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 0x7f733819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 -- 172.21.15.46:0/2249691083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330009d00 con 0x7f7338068160 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733d800700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 0x7f733819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733ce00700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f732c005950 tx=0x7f732c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f7335600700 1 -- 172.21.15.46:0/2249691083 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01c070 con 0x7f7338068160 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.149+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c009520 con 0x7f7338068160 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.150+0000 7f7335600700 1 -- 172.21.15.46:0/2249691083 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c00bbd0 con 0x7f7338068160 2024-05-08T23:20:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.150+0000 7f7335600700 1 -- 172.21.15.46:0/2249691083 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c003a00 con 0x7f7338068160 2024-05-08T23:20:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.150+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381a3b80 con 0x7f7338068160 2024-05-08T23:20:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.154+0000 7f7335600700 1 -- 172.21.15.46:0/2249691083 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f732c020030 con 0x7f7338068160 2024-05-08T23:20:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.155+0000 7f735340ab80 1 Processor -- start 2024-05-08T23:20:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.155+0000 7f735340ab80 1 -- start start 2024-05-08T23:20:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5ac3150 con 0x55bbb5b6c890 2024-05-08T23:20:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5ac0b10 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f735340ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5abc400 con 0x55bbb5b7b1d0 2024-05-08T23:20:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f7335600700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 0x7f73200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f733d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f733ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f733d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36120/0 (socket says 172.21.15.46:36120) 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.156+0000 7f733d800700 1 -- 172.21.15.46:0/252709323 learned_addr learned my addr 172.21.15.46:0/252709323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f7337e00700 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 0x7f73200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f7335600700 1 -- 172.21.15.46:0/2249691083 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f732c0ce070 con 0x7f7338068160 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 -- 172.21.15.46:0/252709323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7b1d0 msgr2=0x55bbb5b6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 -- 172.21.15.46:0/252709323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 msgr2=0x55bbb5b6ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f733d800700 1 -- 172.21.15.46:0/252709323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb593d3b0 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.157+0000 7f7337e00700 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.158+0000 7f733d800700 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7330019690 tx=0x7f73300199a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2746982ffa493976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.158+0000 7f733d800700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 0x7f73200a37f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7330019cf0 tx=0x7f7330008000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.158+0000 7f732a000700 1 -- 172.21.15.46:0/252709323 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000e040 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.158+0000 7f732a000700 1 -- 172.21.15.46:0/252709323 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f733001e070 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.158+0000 7f732a000700 1 -- 172.21.15.46:0/252709323 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330003e80 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 -- 172.21.15.46:0/252709323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 msgr2=0x55bbb5b7ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7330019690 tx=0x7f73300199a0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 -- 172.21.15.46:0/252709323 shutdown_connections 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ac90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 --2- 172.21.15.46:0/252709323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 -- 172.21.15.46:0/252709323 >> 172.21.15.46:0/252709323 conn(0x55bbb5aca210 msgr2=0x55bbb5b673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.159+0000 7f735340ab80 1 -- 172.21.15.46:0/252709323 shutdown_connections 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.160+0000 7f735340ab80 1 -- 172.21.15.46:0/252709323 wait complete. 2024-05-08T23:20:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.160+0000 7f735340ab80 1 -- start start 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 Processor -- start 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5a58560 con 0x55bbb5b7b1d0 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5a58200 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.161+0000 7f735340ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb593f6a0 con 0x55bbb5b6c890 2024-05-08T23:20:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f735340ab80 10 client.0 _ll_register_callbacks cb 0x55bbb5a61b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36136/0 (socket says 172.21.15.46:36136) 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 -- 172.21.15.46:0/1297583783 learned_addr learned my addr 172.21.15.46:0/1297583783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f733d800700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f733ce00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 -- 172.21.15.46:0/1297583783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 msgr2=0x55bbb5b7e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b7e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 -- 172.21.15.46:0/1297583783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7b1d0 msgr2=0x55bbb5b7ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b7ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.162+0000 7f7337e00700 1 -- 172.21.15.46:0/1297583783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73240078a0 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.163+0000 7f7337e00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f73240116f0 tx=0x7f7324011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8770b06e50dda044 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.163+0000 7f731ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.163+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324016070 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.163+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbb5a15270 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.163+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7324010640 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7324010800 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 10 client.29906 fetch_fsmap learned FSMap version 35 2024-05-08T23:20:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bbb58e9b60 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbb5a15270 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f732401b9c0 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 10 client.29906 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 10 client.29906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 20 client.29906 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5b8ebf0 0x55bbb5b91010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 4 client.29906 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.164+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbb5b3b810 con 0x55bbb5b8ebf0 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.165+0000 7f733ce00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5b8ebf0 0x55bbb5b91010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.165+0000 7f733ce00700 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5b8ebf0 0x55bbb5b91010 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5709f81142d40970 server_cookie=b2cda85bec9c8331 in_seq=0 out_seq=0 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.165+0000 7f731ea00700 10 client.29906 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f731ea00700 1 -- 172.21.15.46:0/1297583783 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bbb5b3b810 con 0x55bbb5b8ebf0 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f731ea00700 10 client.29906 handle_command_reply: tid=0 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 client.29906 shutdown 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 msgr2=0x55bbb5b7ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f73240116f0 tx=0x7f7324011a00 comp rx=0 tx=0).stop 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 shutdown_connections 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5b8ebf0 0x55bbb5b91010 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b7ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 --2- 172.21.15.46:0/1297583783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b7ea20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f731ea00700 0 client.29906 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 >> 172.21.15.46:0/1297583783 conn(0x55bbb5aca210 msgr2=0x55bbb5b67040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 shutdown_connections 2024-05-08T23:20:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 1 -- 172.21.15.46:0/1297583783 wait complete. 2024-05-08T23:20:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.166+0000 7f735340ab80 20 client.29906 trim_cache size 0 max 16384 2024-05-08T23:20:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.188+0000 7f735340ab80 1 Processor -- start 2024-05-08T23:20:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.188+0000 7f735340ab80 1 -- start start 2024-05-08T23:20:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5b7fb50 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5b63dc0 con 0x55bbb5b7b1d0 2024-05-08T23:20:00.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f735340ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5b5f0d0 con 0x55bbb5b6c890 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f733ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f7337e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f733ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:40490/0 (socket says 172.21.15.46:40490) 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f733ce00700 1 -- 172.21.15.46:0/2540770040 learned_addr learned my addr 172.21.15.46:0/2540770040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.189+0000 7f733d800700 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 -- 172.21.15.46:0/2540770040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 msgr2=0x55bbb5b79ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b79ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 -- 172.21.15.46:0/2540770040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7a8b0 msgr2=0x55bbb5b79580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b79580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 -- 172.21.15.46:0/2540770040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbb5a82560 con 0x55bbb5b6c890 2024-05-08T23:20:00.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f733ce00700 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f732c0058b0 tx=0x7f732c019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28e41b688c2ba4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f732aa00700 1 -- 172.21.15.46:0/2540770040 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c09b4a0 con 0x55bbb5b6c890 2024-05-08T23:20:00.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f732aa00700 1 -- 172.21.15.46:0/2540770040 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c09ada0 con 0x55bbb5b6c890 2024-05-08T23:20:00.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.190+0000 7f732aa00700 1 -- 172.21.15.46:0/2540770040 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c099140 con 0x55bbb5b6c890 2024-05-08T23:20:00.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 -- 172.21.15.46:0/2540770040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 msgr2=0x55bbb5b6c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f732c0058b0 tx=0x7f732c019040 comp rx=0 tx=0).stop 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 -- 172.21.15.46:0/2540770040 shutdown_connections 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5b79ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b6c890 0x55bbb5b6c350 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 --2- 172.21.15.46:0/2540770040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 -- 172.21.15.46:0/2540770040 >> 172.21.15.46:0/2540770040 conn(0x55bbb5aca210 msgr2=0x55bbb5b65d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 -- 172.21.15.46:0/2540770040 shutdown_connections 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.191+0000 7f735340ab80 1 -- 172.21.15.46:0/2540770040 wait complete. 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.192+0000 7f735340ab80 1 -- start start 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.192+0000 7f735340ab80 1 Processor -- start 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.192+0000 7f735340ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.192+0000 7f735340ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f735340ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5be40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f735340ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5b84900 con 0x55bbb5b6c890 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f735340ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5be4630 con 0x55bbb5b7b1d0 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f735340ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbb5be4770 con 0x55bbb5b7a8b0 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f735340ab80 10 client.0 _ll_register_callbacks cb 0x55bbb5a61b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45656/0 (socket says 172.21.15.46:45656) 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5be40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 -- 172.21.15.46:0/2976474443 learned_addr learned my addr 172.21.15.46:0/2976474443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 -- 172.21.15.46:0/2976474443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7a8b0 msgr2=0x55bbb5b84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f7337e00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b84020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 -- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 msgr2=0x55bbb5be40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.193+0000 7f733ce00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5be40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f733ce00700 1 -- 172.21.15.46:0/2976474443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381a3b80 con 0x55bbb5b6c890 2024-05-08T23:20:00.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f733ce00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f732c009fd0 tx=0x7f732c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b8f11a62deda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f731ea00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0b9120 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f732c09b4a0 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.194+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f733001e070 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c0a3190 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 10 client.19773 fetch_fsmap learned FSMap version 35 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bbb5941cd0 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f733001e070 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f732c0cb640 con 0x55bbb5b6c890 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 10 client.19773 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 10 client.19773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 20 client.19773 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:00.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5be5a50 0x55bbb5be7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 4 client.19773 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbb5ab4ec0 con 0x55bbb5be5a50 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.195+0000 7f7337e00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5be5a50 0x55bbb5be7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f7337e00700 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5be5a50 0x55bbb5be7e70 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309b670b92a760fb server_cookie=212f0b222a4c432a in_seq=0 out_seq=0 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f731ea00700 10 client.19773 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f731ea00700 1 -- 172.21.15.46:0/2976474443 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bbb5ab4ec0 con 0x55bbb5be5a50 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f731ea00700 10 client.19773 handle_command_reply: tid=0 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f735340ab80 1 client.19773 shutdown 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 msgr2=0x55bbb5b83ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.196+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f732c009fd0 tx=0x7f732c00fa70 comp rx=0 tx=0).stop 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 shutdown_connections 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bbb5b7b1d0 0x55bbb5be40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55bbb5be5a50 0x55bbb5be7e70 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbb5b6c890 0x55bbb5b83ae0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 --2- 172.21.15.46:0/2976474443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbb5b7a8b0 0x55bbb5b84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 >> 172.21.15.46:0/2976474443 conn(0x55bbb5aca210 msgr2=0x55bbb5b79580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f731ea00700 0 client.19773 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 shutdown_connections 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 1 -- 172.21.15.46:0/2976474443 wait complete. 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.197+0000 7f735340ab80 20 client.19773 trim_cache size 0 max 16384 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:20:00.201 INFO:teuthology.orchestra.run.smithi046.stdout: "15ea237f-8cf3-4742-9fae-3fece178fa0d": { 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "15ea237f-8cf3-4742-9fae-3fece178fa0d", 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:20:00.202 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 msgr2=0x7f73200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 0x7f73200a37f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7330019cf0 tx=0x7f7330008000 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 msgr2=0x7f7338196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f732c005950 tx=0x7f732c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 shutdown_connections 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f73200a1330 0x7f73200a37f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7338106d30 0x7f733819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7338068160 0x7f7338196ff0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 --2- 172.21.15.46:0/2249691083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7338068b20 0x7f7338197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 >> 172.21.15.46:0/2249691083 conn(0x7f7338075710 msgr2=0x7f73380fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 shutdown_connections 2024-05-08T23:20:00.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:00.198+0000 7f733e200700 1 -- 172.21.15.46:0/2249691083 wait complete. 2024-05-08T23:20:00.216 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:00 smithi046 ceph-mon[36898]: overall HEALTH_OK 2024-05-08T23:20:00.219 INFO:tasks.cephfs.filesystem:scrub status for tag:15ea237f-8cf3-4742-9fae-3fece178fa0d - {'path': '/', 'tag': '15ea237f-8cf3-4742-9fae-3fece178fa0d', 'options': 'recursive,force'} 2024-05-08T23:20:00.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 56919 20.07 MB/sec execute 21 sec latency 229.037 ms 2024-05-08T23:20:00.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:00 smithi053 ceph-mon[77587]: overall HEALTH_OK 2024-05-08T23:20:00.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:00 smithi086 ceph-mon[78610]: overall HEALTH_OK 2024-05-08T23:20:01.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57136 19.40 MB/sec execute 22 sec latency 388.749 ms 2024-05-08T23:20:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:20:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:01.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: pgmap v507: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 727 op/s 2024-05-08T23:20:01.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: pgmap v507: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 727 op/s 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: pgmap v507: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 727 op/s 2024-05-08T23:20:01.513 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:02.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57136 18.56 MB/sec execute 23 sec latency 1388.833 ms 2024-05-08T23:20:03.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57192 18.07 MB/sec execute 24 sec latency 2108.203 ms 2024-05-08T23:20:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:03 smithi086 ceph-mon[78610]: pgmap v508: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-05-08T23:20:03.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:03 smithi053 ceph-mon[77587]: pgmap v508: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-05-08T23:20:03.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:03 smithi046 ceph-mon[36898]: pgmap v508: 129 pgs: 129 active+clean; 934 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-05-08T23:20:04.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57411 17.58 MB/sec execute 25 sec latency 796.796 ms 2024-05-08T23:20:05.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57701 17.17 MB/sec execute 26 sec latency 237.260 ms 2024-05-08T23:20:05.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:05 smithi086 ceph-mon[78610]: pgmap v509: 129 pgs: 129 active+clean; 925 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 536 op/s 2024-05-08T23:20:05.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:05 smithi053 ceph-mon[77587]: pgmap v509: 129 pgs: 129 active+clean; 925 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 536 op/s 2024-05-08T23:20:05.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:05 smithi046 ceph-mon[36898]: pgmap v509: 129 pgs: 129 active+clean; 925 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 536 op/s 2024-05-08T23:20:06.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57821 16.69 MB/sec execute 27 sec latency 282.305 ms 2024-05-08T23:20:07.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57908 16.40 MB/sec execute 28 sec latency 285.533 ms 2024-05-08T23:20:07.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:07 smithi086 ceph-mon[78610]: pgmap v510: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 535 op/s 2024-05-08T23:20:07.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:07 smithi053 ceph-mon[77587]: pgmap v510: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 535 op/s 2024-05-08T23:20:07.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:07 smithi046 ceph-mon[36898]: pgmap v510: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 535 op/s 2024-05-08T23:20:08.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 58152 16.46 MB/sec execute 29 sec latency 248.927 ms 2024-05-08T23:20:09.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 58653 16.80 MB/sec execute 30 sec latency 183.349 ms 2024-05-08T23:20:09.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:09 smithi086 ceph-mon[78610]: pgmap v511: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-05-08T23:20:09.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:09 smithi086 ceph-mon[78610]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:20:09.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:09 smithi053 ceph-mon[77587]: pgmap v511: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-05-08T23:20:09.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:09 smithi053 ceph-mon[77587]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:20:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:09 smithi046 ceph-mon[36898]: pgmap v511: 129 pgs: 129 active+clean; 920 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-05-08T23:20:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:09 smithi046 ceph-mon[36898]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T23:20:10.258 INFO:tasks.workunit.client.0.smithi046.stdout: 10 59221 17.28 MB/sec execute 31 sec latency 68.521 ms 2024-05-08T23:20:11.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 59746 17.70 MB/sec execute 32 sec latency 131.121 ms 2024-05-08T23:20:11.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:11 smithi086 ceph-mon[78610]: pgmap v512: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.2 MiB/s wr, 587 op/s 2024-05-08T23:20:11.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:11 smithi086 ceph-mon[78610]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:20:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:11 smithi053 ceph-mon[77587]: pgmap v512: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.2 MiB/s wr, 587 op/s 2024-05-08T23:20:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:11 smithi053 ceph-mon[77587]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:20:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:11 smithi046 ceph-mon[36898]: pgmap v512: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.2 MiB/s wr, 587 op/s 2024-05-08T23:20:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:11 smithi046 ceph-mon[36898]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T23:20:12.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 60333 18.22 MB/sec execute 33 sec latency 165.052 ms 2024-05-08T23:20:13.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 60726 18.52 MB/sec execute 34 sec latency 212.982 ms 2024-05-08T23:20:13.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:13 smithi086 ceph-mon[78610]: pgmap v514: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-05-08T23:20:13.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:13 smithi053 ceph-mon[77587]: pgmap v514: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-05-08T23:20:13.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:13 smithi046 ceph-mon[36898]: pgmap v514: 129 pgs: 129 active+clean; 928 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-05-08T23:20:14.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 60996 18.25 MB/sec execute 35 sec latency 251.032 ms 2024-05-08T23:20:15.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 61416 18.35 MB/sec execute 36 sec latency 181.565 ms 2024-05-08T23:20:15.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:15 smithi086 ceph-mon[78610]: pgmap v515: 129 pgs: 129 active+clean; 913 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-08T23:20:15.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:15.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:15.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:15 smithi053 ceph-mon[77587]: pgmap v515: 129 pgs: 129 active+clean; 913 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-08T23:20:15.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:15.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:15.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:15 smithi046 ceph-mon[36898]: pgmap v515: 129 pgs: 129 active+clean; 913 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-08T23:20:15.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:15.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:16.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 61913 18.41 MB/sec execute 37 sec latency 113.708 ms 2024-05-08T23:20:16.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:16.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:16 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:16.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:16.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:16 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:16.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:16.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:16.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:16 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:17.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 62446 18.77 MB/sec execute 38 sec latency 72.913 ms 2024-05-08T23:20:17.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:17 smithi086 ceph-mon[78610]: pgmap v516: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:17.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:17 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:17.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:17 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:17.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:17 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:20:17.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:17 smithi053 ceph-mon[77587]: pgmap v516: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:17.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:17 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:17.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:17 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:17.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:17 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:20:17.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:17 smithi046 ceph-mon[36898]: pgmap v516: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:17.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:17 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:17.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:17 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:17.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:17 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:20:18.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 62828 18.71 MB/sec execute 39 sec latency 370.751 ms 2024-05-08T23:20:19.259 INFO:tasks.workunit.client.0.smithi046.stdout: 10 63338 19.04 MB/sec execute 40 sec latency 123.292 ms 2024-05-08T23:20:19.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:19 smithi053 ceph-mon[77587]: pgmap v517: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:19.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:19 smithi046 ceph-mon[36898]: pgmap v517: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:19 smithi086 ceph-mon[78610]: pgmap v517: 129 pgs: 129 active+clean; 914 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.2 MiB/s wr, 508 op/s 2024-05-08T23:20:20.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 63876 19.40 MB/sec execute 41 sec latency 152.616 ms 2024-05-08T23:20:21.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:20 smithi086 ceph-mon[78610]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:20:21.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 64067 19.23 MB/sec execute 42 sec latency 393.068 ms 2024-05-08T23:20:21.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:20 smithi053 ceph-mon[77587]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:20:21.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:20 smithi046 ceph-mon[36898]: osdmap e107: 12 total, 12 up, 12 in 2024-05-08T23:20:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:21 smithi086 ceph-mon[78610]: pgmap v519: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 382 op/s 2024-05-08T23:20:22.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 64323 18.98 MB/sec execute 43 sec latency 229.167 ms 2024-05-08T23:20:22.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:21 smithi053 ceph-mon[77587]: pgmap v519: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 382 op/s 2024-05-08T23:20:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:21 smithi046 ceph-mon[36898]: pgmap v519: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 382 op/s 2024-05-08T23:20:23.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 64677 18.98 MB/sec execute 44 sec latency 203.910 ms 2024-05-08T23:20:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:23 smithi086 ceph-mon[78610]: pgmap v520: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2024-05-08T23:20:24.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 65044 18.98 MB/sec execute 45 sec latency 223.083 ms 2024-05-08T23:20:24.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:23 smithi053 ceph-mon[77587]: pgmap v520: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2024-05-08T23:20:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:23 smithi046 ceph-mon[36898]: pgmap v520: 129 pgs: 129 active+clean; 931 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2024-05-08T23:20:25.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 65582 19.17 MB/sec execute 46 sec latency 190.185 ms 2024-05-08T23:20:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:25 smithi086 ceph-mon[78610]: pgmap v521: 129 pgs: 129 active+clean; 936 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2024-05-08T23:20:26.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 65818 18.99 MB/sec execute 47 sec latency 697.500 ms 2024-05-08T23:20:26.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:25 smithi053 ceph-mon[77587]: pgmap v521: 129 pgs: 129 active+clean; 936 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2024-05-08T23:20:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:25 smithi046 ceph-mon[36898]: pgmap v521: 129 pgs: 129 active+clean; 936 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2024-05-08T23:20:27.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 66106 18.85 MB/sec execute 48 sec latency 1032.917 ms 2024-05-08T23:20:28.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:27 smithi086 ceph-mon[78610]: pgmap v522: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:28.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 66533 18.84 MB/sec execute 49 sec latency 217.700 ms 2024-05-08T23:20:28.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:27 smithi053 ceph-mon[77587]: pgmap v522: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:27 smithi046 ceph-mon[36898]: pgmap v522: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:29.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 67012 19.24 MB/sec execute 50 sec latency 70.741 ms 2024-05-08T23:20:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:29 smithi086 ceph-mon[78610]: pgmap v523: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:30.220 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:30.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 67554 19.56 MB/sec execute 51 sec latency 58.869 ms 2024-05-08T23:20:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:29 smithi053 ceph-mon[77587]: pgmap v523: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:29 smithi046 ceph-mon[36898]: pgmap v523: 129 pgs: 129 active+clean; 955 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 426 op/s 2024-05-08T23:20:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.515+0000 7f1b76a00700 1 -- 172.21.15.46:0/205333556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 msgr2=0x7f1b701084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.515+0000 7f1b76a00700 1 --2- 172.21.15.46:0/205333556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b701084b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801c0d0 tx=0x7f1b5801c3e0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 -- 172.21.15.46:0/205333556 shutdown_connections 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 --2- 172.21.15.46:0/205333556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b7011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 --2- 172.21.15.46:0/205333556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7010f5e0 0x7f1b7010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 --2- 172.21.15.46:0/205333556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b701084b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 -- 172.21.15.46:0/205333556 >> 172.21.15.46:0/205333556 conn(0x7f1b7006bf30 msgr2=0x7f1b7006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 -- 172.21.15.46:0/205333556 shutdown_connections 2024-05-08T23:20:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.516+0000 7f1b76a00700 1 -- 172.21.15.46:0/205333556 wait complete. 2024-05-08T23:20:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.518+0000 7f1b76a00700 1 Processor -- start 2024-05-08T23:20:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.518+0000 7f1b76a00700 1 -- start start 2024-05-08T23:20:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.518+0000 7f1b76a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.518+0000 7f1b76a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7010f5e0 0x7f1b70115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b76a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b70116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b76a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b701168d0 con 0x7f1b7011a780 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b76a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70116a40 con 0x7f1b70108030 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b76a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70116c00 con 0x7f1b7010f5e0 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59364/0 (socket says 172.21.15.46:59364) 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 -- 172.21.15.46:0/344364877 learned_addr learned my addr 172.21.15.46:0/344364877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b74a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b70116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 -- 172.21.15.46:0/344364877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7010f5e0 msgr2=0x7f1b70115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7010f5e0 0x7f1b70115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.519+0000 7f1b6fe00700 1 -- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 msgr2=0x7f1b70116390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.520+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b70116390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.520+0000 7f1b6fe00700 1 -- 172.21.15.46:0/344364877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5801ad10 con 0x7f1b70108030 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.520+0000 7f1b74a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b70116390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.521+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c920 tx=0x7f1b6000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.522+0000 7f1b6cc00700 1 -- 172.21.15.46:0/344364877 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007b00 con 0x7f1b70108030 2024-05-08T23:20:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.522+0000 7f1b6cc00700 1 -- 172.21.15.46:0/344364877 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b60004cf0 con 0x7f1b70108030 2024-05-08T23:20:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.522+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b701aa3f0 con 0x7f1b70108030 2024-05-08T23:20:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.523+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b701aa920 con 0x7f1b70108030 2024-05-08T23:20:30.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.523+0000 7f1b6cc00700 1 -- 172.21.15.46:0/344364877 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60005610 con 0x7f1b70108030 2024-05-08T23:20:30.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.527+0000 7f1b6cc00700 1 -- 172.21.15.46:0/344364877 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1b6000f450 con 0x7f1b70108030 2024-05-08T23:20:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.530+0000 7f1b6cc00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 0x7f1b540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.530+0000 7f1b6cc00700 1 -- 172.21.15.46:0/344364877 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1b600ce040 con 0x7f1b70108030 2024-05-08T23:20:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.531+0000 7f1b6f400700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 0x7f1b540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.532+0000 7f1b6f400700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 0x7f1b540a37d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801c860 tx=0x7f1b580061f0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 Processor -- start 2024-05-08T23:20:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 -- start start 2024-05-08T23:20:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac63cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8abba150 con 0x564a8ac68e00 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8abb7b10 con 0x564a8ac63890 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.534+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8abb3400 con 0x564a8ac69720 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51170/0 (socket says 172.21.15.46:51170) 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 -- 172.21.15.46:0/2226617794 learned_addr learned my addr 172.21.15.46:0/2226617794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b74a00700 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac63cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 -- 172.21.15.46:0/2226617794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 msgr2=0x564a8ac63cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac63cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 -- 172.21.15.46:0/2226617794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 msgr2=0x564a8ac691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.535+0000 7f1b6f400700 1 -- 172.21.15.46:0/2226617794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8aa343b0 con 0x564a8ac69720 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.536+0000 7f1b74a00700 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.536+0000 7f1b6f400700 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801a230 tx=0x7f1b5802b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5426733c00166042 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.536+0000 7f1b65200700 1 -- 172.21.15.46:0/2226617794 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5801f040 con 0x564a8ac69720 2024-05-08T23:20:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.536+0000 7f1b65200700 1 -- 172.21.15.46:0/2226617794 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b580189b0 con 0x564a8ac69720 2024-05-08T23:20:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.536+0000 7f1b65200700 1 -- 172.21.15.46:0/2226617794 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5802c420 con 0x564a8ac69720 2024-05-08T23:20:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.537+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/2226617794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 msgr2=0x564a8ac63350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.537+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801a230 tx=0x7f1b5802b000 comp rx=0 tx=0).stop 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/2226617794 shutdown_connections 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/2226617794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac63350 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/2226617794 >> 172.21.15.46:0/2226617794 conn(0x564a8abc1210 msgr2=0x564a8ac5ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/2226617794 shutdown_connections 2024-05-08T23:20:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.538+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/2226617794 wait complete. 2024-05-08T23:20:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.539+0000 7f1b8a4fdb80 1 -- start start 2024-05-08T23:20:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.539+0000 7f1b8a4fdb80 1 Processor -- start 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.540+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac6d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59376/0 (socket says 172.21.15.46:59376) 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b6fe00700 1 -- 172.21.15.46:0/229635128 learned_addr learned my addr 172.21.15.46:0/229635128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ab4f560 con 0x564a8ac68e00 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ab4f200 con 0x564a8ac63890 2024-05-08T23:20:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.541+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8aa366a0 con 0x564a8ac69720 2024-05-08T23:20:30.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b74a00700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6f400700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac6d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b8a4fdb80 10 client.0 _ll_register_callbacks cb 0x564a8ab58b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6fe00700 1 -- 172.21.15.46:0/229635128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 msgr2=0x564a8ac6d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6fe00700 1 -- 172.21.15.46:0/229635128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 msgr2=0x564a8ac6d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac6d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6fe00700 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7011d3e0 con 0x564a8ac63890 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.542+0000 7f1b6f400700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c8f0 tx=0x7f1b6009b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9b850a64099707 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a8ab0c270 con 0x564a8ac63890 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b5d600700 10 client.29924 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b5d600700 1 -- 172.21.15.46:0/229635128 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b600a31c0 con 0x564a8ac63890 2024-05-08T23:20:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b5d600700 1 -- 172.21.15.46:0/229635128 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1b60007c60 con 0x564a8ac63890 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b5d600700 1 -- 172.21.15.46:0/229635128 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b60007e40 con 0x564a8ac63890 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b8a4fdb80 10 client.29924 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.543+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=36}) v3 -- 0x564a8a9e0b60 con 0x564a8ac63890 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b5d600700 1 -- 172.21.15.46:0/229635128 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b6009afa0 con 0x564a8ac63890 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 10 client.29924 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:30.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 10 client.29924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 20 client.29924 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8ac7cf70 0x564a8ac7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 4 client.29924 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a8ac32810 con 0x564a8ac7cf70 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.544+0000 7f1b6f400700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8ac7cf70 0x564a8ac7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.545+0000 7f1b6f400700 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8ac7cf70 0x564a8ac7f390 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caecee916694cb33 server_cookie=eb73db0f589a04a2 in_seq=0 out_seq=0 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.545+0000 7f1b5d600700 10 client.29924 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:30.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.546+0000 7f1b5d600700 1 -- 172.21.15.46:0/229635128 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a8ac32810 con 0x564a8ac7cf70 2024-05-08T23:20:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.546+0000 7f1b5d600700 10 client.29924 handle_command_reply: tid=0 2024-05-08T23:20:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.546+0000 7f1b8a4fdb80 1 client.29924 shutdown 2024-05-08T23:20:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.546+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 msgr2=0x564a8ac6cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.546+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c8f0 tx=0x7f1b6009b4f0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 shutdown_connections 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac6cbb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8ac7cf70 0x564a8ac7f390 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/229635128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 >> 172.21.15.46:0/229635128 conn(0x564a8abc1210 msgr2=0x564a8ac5d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b5d600700 0 client.29924 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 shutdown_connections 2024-05-08T23:20:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/229635128 wait complete. 2024-05-08T23:20:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.547+0000 7f1b8a4fdb80 20 client.29924 trim_cache size 0 max 16384 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.584+0000 7f1b8a4fdb80 1 Processor -- start 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.584+0000 7f1b8a4fdb80 1 -- start start 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac63890 0x564a8ac5dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ac6f720 con 0x564a8ac63890 2024-05-08T23:20:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ac5adc0 con 0x564a8ac68e00 2024-05-08T23:20:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8ac84560 con 0x564a8ac69720 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b6f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b6f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59378/0 (socket says 172.21.15.46:59378) 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b6f400700 1 -- 172.21.15.46:0/3047450634 learned_addr learned my addr 172.21.15.46:0/3047450634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.585+0000 7f1b74a00700 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac63890 0x564a8ac5dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 -- 172.21.15.46:0/3047450634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 msgr2=0x564a8ac6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 -- 172.21.15.46:0/3047450634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac63890 msgr2=0x564a8ac5dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac63890 0x564a8ac5dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 -- 172.21.15.46:0/3047450634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8ab79560 con 0x564a8ac68e00 2024-05-08T23:20:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.586+0000 7f1b6f400700 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801a040 tx=0x7f1b5801aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0dacfd6179d5875 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.587+0000 7f1b65c00700 1 -- 172.21.15.46:0/3047450634 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5804b070 con 0x564a8ac68e00 2024-05-08T23:20:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.587+0000 7f1b65c00700 1 -- 172.21.15.46:0/3047450634 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b58037440 con 0x564a8ac68e00 2024-05-08T23:20:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.587+0000 7f1b65c00700 1 -- 172.21.15.46:0/3047450634 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5801f040 con 0x564a8ac68e00 2024-05-08T23:20:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/3047450634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 msgr2=0x564a8ac6d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801a040 tx=0x7f1b5801aca0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/3047450634 shutdown_connections 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac63890 0x564a8ac5dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/3047450634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac68e00 0x564a8ac6d100 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.588+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/3047450634 >> 172.21.15.46:0/3047450634 conn(0x564a8abc1210 msgr2=0x564a8ac6b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.589+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/3047450634 shutdown_connections 2024-05-08T23:20:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.589+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/3047450634 wait complete. 2024-05-08T23:20:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.589+0000 7f1b8a4fdb80 1 -- start start 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 Processor -- start 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8acd2590 con 0x564a8ac68e00 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8acd2700 con 0x564a8ac63890 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.590+0000 7f1b8a4fdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8acd28c0 con 0x564a8ac69720 2024-05-08T23:20:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b8a4fdb80 10 client.0 _ll_register_callbacks cb 0x564a8ab58b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b74a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b6fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59388/0 (socket says 172.21.15.46:59388) 2024-05-08T23:20:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b74a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51198/0 (socket says 172.21.15.46:51198) 2024-05-08T23:20:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.591+0000 7f1b74a00700 1 -- 172.21.15.46:0/1214840096 learned_addr learned my addr 172.21.15.46:0/1214840096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b6fe00700 1 -- 172.21.15.46:0/1214840096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 msgr2=0x564a8ac72c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b6fe00700 1 -- 172.21.15.46:0/1214840096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 msgr2=0x564a8ac726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b6fe00700 1 -- 172.21.15.46:0/1214840096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a8aa343b0 con 0x564a8ac63890 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.592+0000 7f1b74a00700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b6fe00700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b600977b0 tx=0x7f1b600cd070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a217a353736b6272 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b5d600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b600a31c0 con 0x564a8ac63890 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1b600a2d30 con 0x564a8ac63890 2024-05-08T23:20:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b580189b0 con 0x564a8ac63890 2024-05-08T23:20:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b600a2f30 con 0x564a8ac63890 2024-05-08T23:20:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b8a4fdb80 10 client.29930 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.593+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x564a8aa38cd0 con 0x564a8ac63890 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b60080080 con 0x564a8ac63890 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b6007bd90 con 0x564a8ac63890 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 10 client.29930 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 10 client.29930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 20 client.29930 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8acd3c40 0x564a8acd6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 4 client.29930 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a8ababec0 con 0x564a8acd3c40 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.594+0000 7f1b6f400700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8acd3c40 0x564a8acd6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.595+0000 7f1b6f400700 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8acd3c40 0x564a8acd6060 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7988b5541650f03 server_cookie=a2e743f1eea0ec98 in_seq=0 out_seq=0 2024-05-08T23:20:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.595+0000 7f1b5d600700 10 client.29930 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.595+0000 7f1b5d600700 1 -- 172.21.15.46:0/1214840096 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564a8ababec0 con 0x564a8acd3c40 2024-05-08T23:20:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.595+0000 7f1b5d600700 10 client.29930 handle_command_reply: tid=0 2024-05-08T23:20:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.595+0000 7f1b8a4fdb80 1 client.29930 shutdown 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 msgr2=0x564a8ac72180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b600977b0 tx=0x7f1b600cd070 comp rx=0 tx=0).stop 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 shutdown_connections 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a8ac69720 0x564a8ac72c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a8ac63890 0x564a8ac72180 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x564a8acd3c40 0x564a8acd6060 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 --2- 172.21.15.46:0/1214840096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a8ac68e00 0x564a8ac726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 >> 172.21.15.46:0/1214840096 conn(0x564a8abc1210 msgr2=0x564a8ac67c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b5d600700 0 client.29930 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.596+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 shutdown_connections 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.597+0000 7f1b8a4fdb80 1 -- 172.21.15.46:0/1214840096 wait complete. 2024-05-08T23:20:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.597+0000 7f1b8a4fdb80 20 client.29930 trim_cache size 0 max 16384 2024-05-08T23:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:20:30.598 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.599+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 msgr2=0x7f1b540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.599+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 0x7f1b540a37d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b5801c860 tx=0x7f1b580061f0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.599+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 msgr2=0x7f1b70119d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.599+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c920 tx=0x7f1b6000cce0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.599+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 shutdown_connections 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1b540a1310 0x7f1b540a37d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b7011a780 0x7f1b70116390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b70108030 0x7f1b70119d40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 --2- 172.21.15.46:0/344364877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7010f5e0 0x7f1b70115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 >> 172.21.15.46:0/344364877 conn(0x7f1b7006bf30 msgr2=0x7f1b7010c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 shutdown_connections 2024-05-08T23:20:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.600+0000 7f1b76a00700 1 -- 172.21.15.46:0/344364877 wait complete. 2024-05-08T23:20:30.632 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:20:30.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:20:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.896+0000 7f1730600700 1 -- 172.21.15.46:0/3018851708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1728101fc0 msgr2=0x7f17281023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.896+0000 7f1730600700 1 --2- 172.21.15.46:0/3018851708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1728101fc0 0x7f17281023a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f171c009990 tx=0x7f171c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 -- 172.21.15.46:0/3018851708 shutdown_connections 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 --2- 172.21.15.46:0/3018851708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f172810b3e0 0x7f172810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 --2- 172.21.15.46:0/3018851708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1728101fc0 0x7f17281023a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 --2- 172.21.15.46:0/3018851708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 0x7f172810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 -- 172.21.15.46:0/3018851708 >> 172.21.15.46:0/3018851708 conn(0x7f17280fb820 msgr2=0x7f17280fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 -- 172.21.15.46:0/3018851708 shutdown_connections 2024-05-08T23:20:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.897+0000 7f1730600700 1 -- 172.21.15.46:0/3018851708 wait complete. 2024-05-08T23:20:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.898+0000 7f1730600700 1 Processor -- start 2024-05-08T23:20:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 -- start start 2024-05-08T23:20:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 0x7f1728199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 0x7f1728199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819e440 con 0x7f17281028e0 2024-05-08T23:20:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819e5b0 con 0x7f1728101fc0 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.899+0000 7f1730600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819e770 con 0x7f172810b3e0 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 0x7f1728199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172dc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 0x7f1728199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59398/0 (socket says 172.21.15.46:59398) 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51220/0 (socket says 172.21.15.46:51220) 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172dc00700 1 -- 172.21.15.46:0/2990760136 learned_addr learned my addr 172.21.15.46:0/2990760136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172d200700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 0x7f1728199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 -- 172.21.15.46:0/2990760136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 msgr2=0x7f1728199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 0x7f1728199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 -- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 msgr2=0x7f1728199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 0x7f1728199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 -- 172.21.15.46:0/2990760136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0095f0 con 0x7f172810b3e0 2024-05-08T23:20:30.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.900+0000 7f172e600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f172000efc0 tx=0x7f172000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.901+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720018070 con 0x7f172810b3e0 2024-05-08T23:20:30.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.901+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172000f040 con 0x7f172810b3e0 2024-05-08T23:20:30.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.901+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000cd40 con 0x7f172810b3e0 2024-05-08T23:20:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.901+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172819e9f0 con 0x7f172810b3e0 2024-05-08T23:20:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.901+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281a6290 con 0x7f172810b3e0 2024-05-08T23:20:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.902+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728108910 con 0x7f172810b3e0 2024-05-08T23:20:30.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.906+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1720003a00 con 0x7f172810b3e0 2024-05-08T23:20:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.907+0000 7f1726000700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 0x7f17100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.907+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f17200ccc60 con 0x7f172810b3e0 2024-05-08T23:20:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.908+0000 7f172dc00700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 0x7f17100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.908+0000 7f172dc00700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 0x7f17100a37d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f171c006010 tx=0x7f171c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:30.911+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1720095cd0 con 0x7f172810b3e0 2024-05-08T23:20:31.260 INFO:tasks.workunit.client.0.smithi046.stdout: 10 67835 19.44 MB/sec execute 52 sec latency 190.102 ms 2024-05-08T23:20:31.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.284+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172819a8e0 con 0x7f172810b3e0 2024-05-08T23:20:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.285+0000 7f1726000700 1 -- 172.21.15.46:0/2990760136 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f17200992f0 con 0x7f172810b3e0 2024-05-08T23:20:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:20:31.286 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:20:31.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 msgr2=0x7f17100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 0x7f17100a37d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f171c006010 tx=0x7f171c00b260 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 msgr2=0x7f172819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f172000efc0 tx=0x7f172000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 shutdown_connections 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172810b3e0 0x7f172819de00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1728101fc0 0x7f1728199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f17100a1310 0x7f17100a37d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 --2- 172.21.15.46:0/2990760136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17281028e0 0x7f1728199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.290+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 >> 172.21.15.46:0/2990760136 conn(0x7f17280fb820 msgr2=0x7f17281003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.291+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 shutdown_connections 2024-05-08T23:20:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.291+0000 7f1730600700 1 -- 172.21.15.46:0/2990760136 wait complete. 2024-05-08T23:20:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:20:31.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 -- 172.21.15.46:0/938088378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec101870 msgr2=0x7f49ec1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 --2- 172.21.15.46:0/938088378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec101870 0x7f49ec1058b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009990 tx=0x7f49d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 -- 172.21.15.46:0/938088378 shutdown_connections 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 --2- 172.21.15.46:0/938088378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 0x7f49ec10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 --2- 172.21.15.46:0/938088378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 --2- 172.21.15.46:0/938088378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec101870 0x7f49ec1058b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 -- 172.21.15.46:0/938088378 >> 172.21.15.46:0/938088378 conn(0x7f49ec0fc740 msgr2=0x7f49ec0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 -- 172.21.15.46:0/938088378 shutdown_connections 2024-05-08T23:20:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.619+0000 7f49f1200700 1 -- 172.21.15.46:0/938088378 wait complete. 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.620+0000 7f49f1200700 1 Processor -- start 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.620+0000 7f49f1200700 1 -- start start 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 0x7f49ec0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec073d20 0x7f49ec1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1a5b90 con 0x7f49ec105fe0 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1a5d00 con 0x7f49ec100ec0 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49f1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1a5ec0 con 0x7f49ec073d20 2024-05-08T23:20:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec073d20 0x7f49ec1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59418/0 (socket says 172.21.15.46:59418) 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 -- 172.21.15.46:0/2900178388 learned_addr learned my addr 172.21.15.46:0/2900178388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 0x7f49ec0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 -- 172.21.15.46:0/2900178388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec073d20 msgr2=0x7f49ec1a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec073d20 0x7f49ec1a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 -- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 msgr2=0x7f49ec0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 0x7f49ec0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.621+0000 7f49eb400700 1 -- 172.21.15.46:0/2900178388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4009c40 con 0x7f49ec100ec0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49eb400700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00e9c0 tx=0x7f49dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49e3e00700 1 -- 172.21.15.46:0/2900178388 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0097b0 con 0x7f49ec100ec0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49e3e00700 1 -- 172.21.15.46:0/2900178388 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49dc00ee50 con 0x7f49ec100ec0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80095f0 con 0x7f49ec100ec0 2024-05-08T23:20:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49e3e00700 1 -- 172.21.15.46:0/2900178388 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0054f0 con 0x7f49ec100ec0 2024-05-08T23:20:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.622+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec1a64d0 con 0x7f49ec100ec0 2024-05-08T23:20:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.626+0000 7f49e3e00700 1 -- 172.21.15.46:0/2900178388 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f49dc004750 con 0x7f49ec100ec0 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.627+0000 7f49e3e00700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 0x7f49d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 Processor -- start 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f49e3e00700 1 -- 172.21.15.46:0/2900178388 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f49dc014070 con 0x7f49ec100ec0 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 -- start start 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 0x7f49d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c510b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c5105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c505c150 con 0x5620c510b050 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c5059b10 con 0x5620c510b970 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.628+0000 7f4a06285b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c5055400 con 0x5620c5105890 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51230/0 (socket says 172.21.15.46:51230) 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 -- 172.21.15.46:0/93820022 learned_addr learned my addr 172.21.15.46:0/93820022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49eaa00700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c5105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49eb400700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c510b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 -- 172.21.15.46:0/93820022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 msgr2=0x5620c5105350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c5105350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 -- 172.21.15.46:0/93820022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 msgr2=0x5620c510b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 0x7f49d00a37f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0749d0 tx=0x7f49d800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c510b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 -- 172.21.15.46:0/93820022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c4ed63b0 con 0x5620c5105890 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49eb400700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c510b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.629+0000 7f49ebe00700 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e730 tx=0x7f49e4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3897e2e21501f8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f49e0c00700 1 -- 172.21.15.46:0/93820022 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e400e040 con 0x5620c5105890 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f49e0c00700 1 -- 172.21.15.46:0/93820022 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e400f040 con 0x5620c5105890 2024-05-08T23:20:31.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f49e0c00700 1 -- 172.21.15.46:0/93820022 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4007760 con 0x5620c5105890 2024-05-08T23:20:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 -- 172.21.15.46:0/93820022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 msgr2=0x5620c5105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e730 tx=0x7f49e4009920 comp rx=0 tx=0).stop 2024-05-08T23:20:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 -- 172.21.15.46:0/93820022 shutdown_connections 2024-05-08T23:20:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105cd0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c510b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 --2- 172.21.15.46:0/93820022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c5105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.630+0000 7f4a06285b80 1 -- 172.21.15.46:0/93820022 >> 172.21.15.46:0/93820022 conn(0x5620c5063210 msgr2=0x5620c51002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.631+0000 7f4a06285b80 1 -- 172.21.15.46:0/93820022 shutdown_connections 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.631+0000 7f4a06285b80 1 -- 172.21.15.46:0/93820022 wait complete. 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.631+0000 7f4a06285b80 1 -- start start 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.631+0000 7f4a06285b80 1 Processor -- start 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c5105890 0x5620c5111890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c4ff1560 con 0x5620c5105890 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c4ff1200 con 0x5620c510b970 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c4ed86a0 con 0x5620c510b050 2024-05-08T23:20:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f4a06285b80 10 client.0 _ll_register_callbacks cb 0x5620c4ffab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f49eaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f49eaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51242/0 (socket says 172.21.15.46:51242) 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f49eaa00700 1 -- 172.21.15.46:0/2424049301 learned_addr learned my addr 172.21.15.46:0/2424049301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.632+0000 7f49ebe00700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 -- 172.21.15.46:0/2424049301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 msgr2=0x5620c510f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 -- 172.21.15.46:0/2424049301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c5105890 msgr2=0x5620c5111890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c5105890 0x5620c5111890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 -- 172.21.15.46:0/2424049301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e400e410 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49ec06a260 tx=0x7f49d8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891868921e93485f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49d5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800bee0 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f49d8004cf0 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.633+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620c4fae270 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d8025470 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 10 client.30016 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5620c4e82b60 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620c4fae270 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49d800f560 con 0x5620c510b050 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 10 client.30016 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 10 client.30016 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 20 client.30016 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.634+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5121760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.635+0000 7f4a06285b80 4 client.30016 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.635+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620c50d4810 con 0x5620c511f340 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.635+0000 7f49eb400700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.635+0000 7f49eb400700 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5121760 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=362c687ce24ee600 server_cookie=ea8cbb94d0aef926 in_seq=0 out_seq=0 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.635+0000 7f49d5600700 10 client.30016 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f49d5600700 1 -- 172.21.15.46:0/2424049301 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620c50d4810 con 0x5620c511f340 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f49d5600700 10 client.30016 handle_command_reply: tid=0 2024-05-08T23:20:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 client.30016 shutdown 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 msgr2=0x5620c510eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49ec06a260 tx=0x7f49d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 shutdown_connections 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5121760 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f49d5600700 0 client.30016 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:20:31.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c5105890 0x5620c5111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2424049301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b050 0x5620c510eb80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.636+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 >> 172.21.15.46:0/2424049301 conn(0x5620c5063210 msgr2=0x5620c51000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.637+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 shutdown_connections 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.637+0000 7f4a06285b80 1 -- 172.21.15.46:0/2424049301 wait complete. 2024-05-08T23:20:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.637+0000 7f4a06285b80 20 client.30016 trim_cache size 0 max 16384 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 Processor -- start 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 -- start start 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5121760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c511bdf0 con 0x5620c510b050 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c510b430 con 0x5620c510b970 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f4a06285b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c511ad80 con 0x5620c5105890 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.652+0000 7f49eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51256/0 (socket says 172.21.15.46:51256) 2024-05-08T23:20:31.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eb400700 1 -- 172.21.15.46:0/3965045382 learned_addr learned my addr 172.21.15.46:0/3965045382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49ebe00700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 -- 172.21.15.46:0/3965045382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 msgr2=0x5620c5105350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 -- 172.21.15.46:0/3965045382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 msgr2=0x5620c510cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 -- 172.21.15.46:0/3965045382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c501b560 con 0x5620c510b050 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49ebe00700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eb400700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49eaa00700 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5121760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f49d800f8c0 tx=0x7f49d800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa6b222d358bf6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.653+0000 7f49e1600700 1 -- 172.21.15.46:0/3965045382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80054d0 con 0x5620c510b050 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f49e1600700 1 -- 172.21.15.46:0/3965045382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d801e040 con 0x5620c510b050 2024-05-08T23:20:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f49e1600700 1 -- 172.21.15.46:0/3965045382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80256d0 con 0x5620c510b050 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 -- 172.21.15.46:0/3965045382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 msgr2=0x5620c5121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5121760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f49d800f8c0 tx=0x7f49d800bf60 comp rx=0 tx=0).stop 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 -- 172.21.15.46:0/3965045382 shutdown_connections 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c510b970 0x5620c510cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c5105890 0x5620c5105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 --2- 172.21.15.46:0/3965045382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5121760 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.654+0000 7f4a06285b80 1 -- 172.21.15.46:0/3965045382 >> 172.21.15.46:0/3965045382 conn(0x5620c5063210 msgr2=0x5620c50fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.655+0000 7f4a06285b80 1 -- 172.21.15.46:0/3965045382 shutdown_connections 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.655+0000 7f4a06285b80 1 -- 172.21.15.46:0/3965045382 wait complete. 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.655+0000 7f4a06285b80 1 -- start start 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.655+0000 7f4a06285b80 1 Processor -- start 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 0x5620c5116600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 0x5620c5106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c5107650 con 0x5620c510b050 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c51077c0 con 0x5620c5105890 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c5107980 con 0x5620c510b970 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f4a06285b80 10 client.0 _ll_register_callbacks cb 0x5620c4ffab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49eb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 0x5620c5116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 0x5620c5106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49eb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 0x5620c5116600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59458/0 (socket says 172.21.15.46:59458) 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49eb400700 1 -- 172.21.15.46:0/2924050922 learned_addr learned my addr 172.21.15.46:0/2924050922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49eaa00700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 -- 172.21.15.46:0/2924050922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 msgr2=0x5620c5116600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 0x5620c5116600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 -- 172.21.15.46:0/2924050922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 msgr2=0x5620c5106a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5106a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.656+0000 7f49ebe00700 1 -- 172.21.15.46:0/2924050922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec1a64d0 con 0x5620c510b970 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49ebe00700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 0x5620c5106f40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e700 tx=0x7f49e4009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf4c829c130d080 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4024070 con 0x5620c510b970 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49e400f040 con 0x5620c510b970 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f49e40052d0 con 0x5620c510b970 2024-05-08T23:20:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49e400e040 con 0x5620c510b970 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f4a06285b80 10 client.30022 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5620c4ed63b0 con 0x5620c510b970 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e4031020 con 0x5620c510b970 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.657+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49e4015a70 con 0x5620c510b970 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 10 client.30022 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 10 client.30022 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 20 client.30022 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5177f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 4 client.30022 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620c504dec0 con 0x5620c511f340 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f49eb400700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5177f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f49eb400700 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5177f10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d353d7c6fdb3d92 server_cookie=2ba69c2c60149a70 in_seq=0 out_seq=0 2024-05-08T23:20:31.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f49d5600700 10 client.30022 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f49d5600700 1 -- 172.21.15.46:0/2924050922 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620c504dec0 con 0x5620c511f340 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f49d5600700 10 client.30022 handle_command_reply: tid=0 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.658+0000 7f4a06285b80 1 client.30022 shutdown 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 msgr2=0x5620c5106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 0x5620c5106f40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e700 tx=0x7f49e4009e90 comp rx=0 tx=0).stop 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 shutdown_connections 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5620c511f340 0x5620c5177f10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c510b970 0x5620c5106f40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620c5105890 0x5620c5116600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 --2- 172.21.15.46:0/2924050922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620c510b050 0x5620c5106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f49d5600700 0 client.30022 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 >> 172.21.15.46:0/2924050922 conn(0x5620c5063210 msgr2=0x5620c5108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 shutdown_connections 2024-05-08T23:20:31.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 1 -- 172.21.15.46:0/2924050922 wait complete. 2024-05-08T23:20:31.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.659+0000 7f4a06285b80 20 client.30022 trim_cache size 0 max 16384 2024-05-08T23:20:31.664 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 msgr2=0x7f49d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 0x7f49d00a37f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0749d0 tx=0x7f49d800b260 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 msgr2=0x7f49ec0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00e9c0 tx=0x7f49dc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 shutdown_connections 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49ec073d20 0x7f49ec1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49ec100ec0 0x7f49ec0732a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f49d00a1330 0x7f49d00a37f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 --2- 172.21.15.46:0/2900178388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec105fe0 0x7f49ec0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 >> 172.21.15.46:0/2900178388 conn(0x7f49ec0fc740 msgr2=0x7f49ec0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 shutdown_connections 2024-05-08T23:20:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.660+0000 7f49f1200700 1 -- 172.21.15.46:0/2900178388 wait complete. 2024-05-08T23:20:31.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:20:31.928 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:31 smithi046 ceph-mon[36898]: pgmap v524: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-05-08T23:20:31.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 -- 172.21.15.46:0/2943680103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c1032b0 msgr2=0x7fc34c103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 --2- 172.21.15.46:0/2943680103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c1032b0 0x7fc34c103690 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009990 tx=0x7fc33c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 -- 172.21.15.46:0/2943680103 shutdown_connections 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 --2- 172.21.15.46:0/2943680103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1083d0 0x7fc34c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 --2- 172.21.15.46:0/2943680103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c1032b0 0x7fc34c103690 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 --2- 172.21.15.46:0/2943680103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c103c60 0x7fc34c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.933+0000 7fc352200700 1 -- 172.21.15.46:0/2943680103 >> 172.21.15.46:0/2943680103 conn(0x7fc34c0feb10 msgr2=0x7fc34c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.934+0000 7fc352200700 1 -- 172.21.15.46:0/2943680103 shutdown_connections 2024-05-08T23:20:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.934+0000 7fc352200700 1 -- 172.21.15.46:0/2943680103 wait complete. 2024-05-08T23:20:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 Processor -- start 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 -- start start 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 0x7fc34c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c19de40 0x7fc34c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19e8c0 con 0x7fc34c1083d0 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19ea30 con 0x7fc34c1032b0 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.935+0000 7fc352200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19ebf0 con 0x7fc34c19de40 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41770/0 (socket says 172.21.15.46:41770) 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 -- 172.21.15.46:0/2861186456 learned_addr learned my addr 172.21.15.46:0/2861186456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34aa00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 0x7fc34c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34be00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c19de40 0x7fc34c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 -- 172.21.15.46:0/2861186456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c19de40 msgr2=0x7fc34c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c19de40 0x7fc34c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 -- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 msgr2=0x7fc34c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 0x7fc34c199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34b400700 1 -- 172.21.15.46:0/2861186456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0095f0 con 0x7fc34c1032b0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.936+0000 7fc34aa00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 0x7fc34c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc34b400700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc33c005910 tx=0x7fc33c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc343e00700 1 -- 172.21.15.46:0/2861186456 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c01d070 con 0x7fc34c1032b0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc343e00700 1 -- 172.21.15.46:0/2861186456 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc33c022470 con 0x7fc34c1032b0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c1a60f0 con 0x7fc34c1032b0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc343e00700 1 -- 172.21.15.46:0/2861186456 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c017650 con 0x7fc34c1032b0 2024-05-08T23:20:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.937+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c1a6510 con 0x7fc34c1032b0 2024-05-08T23:20:31.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.940+0000 7fc343e00700 1 -- 172.21.15.46:0/2861186456 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc33c0225e0 con 0x7fc34c1032b0 2024-05-08T23:20:31.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.942+0000 7fc343e00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 0x7fc3300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.942+0000 7fc343e00700 1 -- 172.21.15.46:0/2861186456 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc33c0ce310 con 0x7fc34c1032b0 2024-05-08T23:20:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.943+0000 7fc34aa00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 0x7fc3300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.943+0000 7fc365d3cb80 1 Processor -- start 2024-05-08T23:20:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.943+0000 7fc365d3cb80 1 -- start start 2024-05-08T23:20:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad825ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 0x55cad8256b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad8257570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad81a5150 con 0x55cad825c660 2024-05-08T23:20:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad81a2b10 con 0x55cad82570f0 2024-05-08T23:20:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad819e400 con 0x55cad825cf80 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc34aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 0x55cad8256b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc34aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 0x55cad8256b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58844/0 (socket says 172.21.15.46:58844) 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.944+0000 7fc34aa00700 1 -- 172.21.15.46:0/2911920051 learned_addr learned my addr 172.21.15.46:0/2911920051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad8257570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 -- 172.21.15.46:0/2911920051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 msgr2=0x55cad8256b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 0x55cad8256b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 -- 172.21.15.46:0/2911920051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 msgr2=0x55cad825ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad825ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 -- 172.21.15.46:0/2911920051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad801f3b0 con 0x55cad82570f0 2024-05-08T23:20:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34aa00700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 0x7fc3300a3750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc34c19aab0 tx=0x7fc338009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.945+0000 7fc34be00700 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad8257570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc344011580 tx=0x7fc344009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4aa851b5ba74ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.946+0000 7fc337e00700 1 -- 172.21.15.46:0/2911920051 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400e040 con 0x55cad82570f0 2024-05-08T23:20:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.946+0000 7fc337e00700 1 -- 172.21.15.46:0/2911920051 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc34400f040 con 0x55cad82570f0 2024-05-08T23:20:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.947+0000 7fc365d3cb80 1 -- 172.21.15.46:0/2911920051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 msgr2=0x55cad8257570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.947+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad8257570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc344011580 tx=0x7fc344009920 comp rx=0 tx=0).stop 2024-05-08T23:20:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 -- 172.21.15.46:0/2911920051 shutdown_connections 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad8257570 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad825ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/2911920051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825cf80 0x55cad8256b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 -- 172.21.15.46:0/2911920051 >> 172.21.15.46:0/2911920051 conn(0x55cad81ac210 msgr2=0x55cad8248df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 -- 172.21.15.46:0/2911920051 shutdown_connections 2024-05-08T23:20:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.948+0000 7fc365d3cb80 1 -- 172.21.15.46:0/2911920051 wait complete. 2024-05-08T23:20:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.949+0000 7fc365d3cb80 1 -- start start 2024-05-08T23:20:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.950+0000 7fc365d3cb80 1 Processor -- start 2024-05-08T23:20:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.950+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad82602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.950+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad8260830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.951+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.951+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad813a560 con 0x55cad825c660 2024-05-08T23:20:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.951+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad813a200 con 0x55cad825cf80 2024-05-08T23:20:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.951+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad80216a0 con 0x55cad82570f0 2024-05-08T23:20:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.951+0000 7fc365d3cb80 10 client.0 _ll_register_callbacks cb 0x55cad8143b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41784/0 (socket says 172.21.15.46:41784) 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34be00700 1 -- 172.21.15.46:0/1639728484 learned_addr learned my addr 172.21.15.46:0/1639728484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34b400700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad82602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34b400700 1 -- 172.21.15.46:0/1639728484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 msgr2=0x55cad8260d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34b400700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.952+0000 7fc34b400700 1 -- 172.21.15.46:0/1639728484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 msgr2=0x55cad8260830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc34aa00700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad8260830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc34b400700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad8260830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc34b400700 1 -- 172.21.15.46:0/1639728484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344011230 con 0x55cad82570f0 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc34be00700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc34b400700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad82602f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0cb650 tx=0x7fc33c09ae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca87e52e15c5f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.953+0000 7fc334c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c09b7a0 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc33c09b0a0 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cad80f7270 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33c09eed0 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc365d3cb80 10 client.30034 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.954+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cad7fcbb60 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cad80f7270 con 0x55cad82570f0 2024-05-08T23:20:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc33c09b300 con 0x55cad82570f0 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 10 client.30034 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 10 client.30034 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 20 client.30034 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad8270710 0x55cad8272b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 4 client.30034 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.955+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cad821d810 con 0x55cad8270710 2024-05-08T23:20:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.956+0000 7fc34aa00700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad8270710 0x55cad8272b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.957+0000 7fc34aa00700 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad8270710 0x55cad8272b00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad4aaf5fcf5790b2 server_cookie=f5f5162f833686a8 in_seq=0 out_seq=0 2024-05-08T23:20:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.958+0000 7fc334c00700 10 client.30034 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:20:31.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.959+0000 7fc334c00700 1 -- 172.21.15.46:0/1639728484 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cad821d810 con 0x55cad8270710 2024-05-08T23:20:31.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.959+0000 7fc334c00700 10 client.30034 handle_command_reply: tid=0 2024-05-08T23:20:31.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.959+0000 7fc365d3cb80 1 client.30034 shutdown 2024-05-08T23:20:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.959+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 msgr2=0x55cad82602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.959+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad82602f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0cb650 tx=0x7fc33c09ae70 comp rx=0 tx=0).stop 2024-05-08T23:20:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 shutdown_connections 2024-05-08T23:20:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad8270710 0x55cad8272b00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825cf80 0x55cad8260d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad82602f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/1639728484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825c660 0x55cad8260830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc334c00700 0 client.30034 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 >> 172.21.15.46:0/1639728484 conn(0x55cad81ac210 msgr2=0x55cad82485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 shutdown_connections 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.960+0000 7fc365d3cb80 1 -- 172.21.15.46:0/1639728484 wait complete. 2024-05-08T23:20:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.961+0000 7fc365d3cb80 20 client.30034 trim_cache size 0 max 16384 2024-05-08T23:20:31.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.976+0000 7fc365d3cb80 1 Processor -- start 2024-05-08T23:20:31.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 -- start start 2024-05-08T23:20:31.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 0x55cad826c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad8248d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad8261960 con 0x55cad825cf80 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad8245dc0 con 0x55cad82570f0 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad824ae00 con 0x55cad825c660 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc34b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc34be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad8248d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc34aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 0x55cad826c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc34b400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41792/0 (socket says 172.21.15.46:41792) 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.977+0000 7fc34b400700 1 -- 172.21.15.46:0/4196145996 learned_addr learned my addr 172.21.15.46:0/4196145996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 -- 172.21.15.46:0/4196145996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 msgr2=0x55cad826c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 0x55cad826c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 -- 172.21.15.46:0/4196145996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 msgr2=0x55cad8248d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad8248d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 -- 172.21.15.46:0/4196145996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad8164560 con 0x55cad82570f0 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34aa00700 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 0x55cad826c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34be00700 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad8248d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc34b400700 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097b10 tx=0x7fc33c099650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a219761362ca8df4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc341200700 1 -- 172.21.15.46:0/4196145996 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c09a630 con 0x55cad82570f0 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc341200700 1 -- 172.21.15.46:0/4196145996 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc33c09a820 con 0x55cad82570f0 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.978+0000 7fc341200700 1 -- 172.21.15.46:0/4196145996 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c0a3490 con 0x55cad82570f0 2024-05-08T23:20:31.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 -- 172.21.15.46:0/4196145996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 msgr2=0x55cad825ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097b10 tx=0x7fc33c099650 comp rx=0 tx=0).stop 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 -- 172.21.15.46:0/4196145996 shutdown_connections 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad8248d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad82570f0 0x55cad825ca40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/4196145996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad825c660 0x55cad826c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 -- 172.21.15.46:0/4196145996 >> 172.21.15.46:0/4196145996 conn(0x55cad81ac210 msgr2=0x55cad825ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 -- 172.21.15.46:0/4196145996 shutdown_connections 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.979+0000 7fc365d3cb80 1 -- 172.21.15.46:0/4196145996 wait complete. 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.980+0000 7fc365d3cb80 1 -- start start 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.980+0000 7fc365d3cb80 1 Processor -- start 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825c660 0x55cad8258060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad82585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad8258ae0 con 0x55cad825cf80 2024-05-08T23:20:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad8258c50 con 0x55cad825c660 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad8258e10 con 0x55cad82570f0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc365d3cb80 10 client.0 _ll_register_callbacks cb 0x55cad8143b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58862/0 (socket says 172.21.15.46:58862) 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 -- 172.21.15.46:0/467538856 learned_addr learned my addr 172.21.15.46:0/467538856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34aa00700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825c660 0x55cad8258060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34be00700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad82585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 -- 172.21.15.46:0/467538856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825c660 msgr2=0x55cad8258060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825c660 0x55cad8258060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 -- 172.21.15.46:0/467538856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 msgr2=0x55cad82585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad82585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34b400700 1 -- 172.21.15.46:0/467538856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338017070 con 0x55cad82570f0 2024-05-08T23:20:31.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.981+0000 7fc34be00700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad82585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc34b400700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0993f0 tx=0x7fc33c096de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7c180b902ee175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc334c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c09a630 con 0x55cad82570f0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc33c09b0a0 con 0x55cad82570f0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc34400f040 con 0x55cad82570f0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33c09b7a0 con 0x55cad82570f0 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc365d3cb80 10 client.30040 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cad801f3b0 con 0x55cad82570f0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.982+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc34400f040 con 0x55cad82570f0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc33c0b3bc0 con 0x55cad82570f0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 10 client.30040 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 10 client.30040 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 20 client.30040 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad82c6ba0 0x55cad82c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 4 client.30040 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cad8196ec0 con 0x55cad82c6ba0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.983+0000 7fc34aa00700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad82c6ba0 0x55cad82c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc34aa00700 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad82c6ba0 0x55cad82c8f90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a62acc81ef0e82d server_cookie=6167301b757364f5 in_seq=0 out_seq=0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc334c00700 10 client.30040 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc334c00700 1 -- 172.21.15.46:0/467538856 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cad8196ec0 con 0x55cad82c6ba0 2024-05-08T23:20:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc334c00700 10 client.30040 handle_command_reply: tid=0 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 client.30040 shutdown 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 msgr2=0x55cad8267c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0993f0 tx=0x7fc33c096de0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 shutdown_connections 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55cad82c6ba0 0x55cad82c8f90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cad825cf80 0x55cad82585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cad82570f0 0x55cad8267c90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 --2- 172.21.15.46:0/467538856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cad825c660 0x55cad8258060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 >> 172.21.15.46:0/467538856 conn(0x55cad81ac210 msgr2=0x55cad825b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc334c00700 0 client.30040 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 shutdown_connections 2024-05-08T23:20:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.984+0000 7fc365d3cb80 1 -- 172.21.15.46:0/467538856 wait complete. 2024-05-08T23:20:31.992 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.985+0000 7fc365d3cb80 20 client.30040 trim_cache size 0 max 16384 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 msgr2=0x7fc3300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 0x7fc3300a3750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc34c19aab0 tx=0x7fc338009260 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 msgr2=0x7fc34c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc33c005910 tx=0x7fc33c004730 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 shutdown_connections 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc34c19de40 0x7fc34c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc34c1032b0 0x7fc34c199560 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc3300a1290 0x7fc3300a3750 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 --2- 172.21.15.46:0/2861186456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34c1083d0 0x7fc34c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 >> 172.21.15.46:0/2861186456 conn(0x7fc34c0feb10 msgr2=0x7fc34c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 shutdown_connections 2024-05-08T23:20:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:31.986+0000 7fc352200700 1 -- 172.21.15.46:0/2861186456 wait complete. 2024-05-08T23:20:32.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:20:32.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:31 smithi086 ceph-mon[78610]: pgmap v524: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-05-08T23:20:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:31 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2990760136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 -- 172.21.15.46:0/613220144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c102900 msgr2=0x7fda2c10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 --2- 172.21.15.46:0/613220144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c102900 0x7fda2c10ade0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda18009990 tx=0x7fda18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 -- 172.21.15.46:0/613220144 shutdown_connections 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 --2- 172.21.15.46:0/613220144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c10b400 0x7fda2c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 --2- 172.21.15.46:0/613220144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c101fe0 0x7fda2c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 --2- 172.21.15.46:0/613220144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c102900 0x7fda2c10ade0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 -- 172.21.15.46:0/613220144 >> 172.21.15.46:0/613220144 conn(0x7fda2c0fb760 msgr2=0x7fda2c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 -- 172.21.15.46:0/613220144 shutdown_connections 2024-05-08T23:20:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.257+0000 7fda31400700 1 -- 172.21.15.46:0/613220144 wait complete. 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.258+0000 7fda31400700 1 Processor -- start 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.258+0000 7fda31400700 1 -- start start 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c101fe0 0x7fda2c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 0x7fda2c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19e400 con 0x7fda2c101fe0 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19e570 con 0x7fda2c10b400 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda31400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19e730 con 0x7fda2c102900 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58870/0 (socket says 172.21.15.46:58870) 2024-05-08T23:20:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda2b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 0x7fda2c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda29e00700 1 -- 172.21.15.46:0/2516735158 learned_addr learned my addr 172.21.15.46:0/2516735158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda2a800700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c101fe0 0x7fda2c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.259+0000 7fda29e00700 1 -- 172.21.15.46:0/2516735158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 msgr2=0x7fda2c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda29e00700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 0x7fda2c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda29e00700 1 -- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c101fe0 msgr2=0x7fda2c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda29e00700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c101fe0 0x7fda2c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda29e00700 1 -- 172.21.15.46:0/2516735158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180095f0 con 0x7fda2c102900 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda2b200700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 0x7fda2c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda29e00700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fda18000c00 tx=0x7fda18005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda23400700 1 -- 172.21.15.46:0/2516735158 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1801d070 con 0x7fda2c102900 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda23400700 1 -- 172.21.15.46:0/2516735158 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda18022470 con 0x7fda2c102900 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda23400700 1 -- 172.21.15.46:0/2516735158 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18017650 con 0x7fda2c102900 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c19e9b0 con 0x7fda2c102900 2024-05-08T23:20:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.260+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c19ec30 con 0x7fda2c102900 2024-05-08T23:20:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:31 smithi053 ceph-mon[77587]: pgmap v524: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-05-08T23:20:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:31 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2990760136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:32.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 68204 19.41 MB/sec execute 53 sec latency 204.064 ms 2024-05-08T23:20:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.264+0000 7fda23400700 1 -- 172.21.15.46:0/2516735158 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fda180225e0 con 0x7fda2c102900 2024-05-08T23:20:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.265+0000 7fda23400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 0x7fda100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.265+0000 7fda44f12b80 1 Processor -- start 2024-05-08T23:20:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.265+0000 7fda23400700 1 -- 172.21.15.46:0/2516735158 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fda180ce250 con 0x7fda2c102900 2024-05-08T23:20:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.265+0000 7fda44f12b80 1 -- start start 2024-05-08T23:20:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.265+0000 7fda2a800700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 0x7fda100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584def0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584de47150 con 0x55584def0890 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584de44b10 con 0x55584def6720 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda44f12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584de40400 con 0x55584def5e00 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda2a800700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 0x7fda100a3750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009c60 tx=0x7fda1c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda29e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda2a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda29e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41818/0 (socket says 172.21.15.46:41818) 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda29e00700 1 -- 172.21.15.46:0/1311058385 learned_addr learned my addr 172.21.15.46:0/1311058385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.266+0000 7fda2b200700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584def0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 -- 172.21.15.46:0/1311058385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 msgr2=0x55584def61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 -- 172.21.15.46:0/1311058385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 msgr2=0x55584def0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584def0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 -- 172.21.15.46:0/1311058385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55584dcc13b0 con 0x55584def6720 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda2a800700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda2b200700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584def0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda29e00700 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda180975a0 tx=0x7fda1809ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd78ddf99e8aee67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda17400700 1 -- 172.21.15.46:0/1311058385 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1809b7a0 con 0x55584def6720 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda17400700 1 -- 172.21.15.46:0/1311058385 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1809b0a0 con 0x55584def6720 2024-05-08T23:20:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.267+0000 7fda17400700 1 -- 172.21.15.46:0/1311058385 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda180977b0 con 0x55584def6720 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 -- 172.21.15.46:0/1311058385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 msgr2=0x55584def0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda180975a0 tx=0x7fda1809ae70 comp rx=0 tx=0).stop 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 -- 172.21.15.46:0/1311058385 shutdown_connections 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584def0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1311058385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584def0350 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 -- 172.21.15.46:0/1311058385 >> 172.21.15.46:0/1311058385 conn(0x55584de4e210 msgr2=0x55584deeadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 -- 172.21.15.46:0/1311058385 shutdown_connections 2024-05-08T23:20:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.268+0000 7fda44f12b80 1 -- 172.21.15.46:0/1311058385 wait complete. 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.269+0000 7fda44f12b80 1 -- start start 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.269+0000 7fda44f12b80 1 Processor -- start 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584defa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584dddc560 con 0x55584def6720 2024-05-08T23:20:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584dddc200 con 0x55584def0890 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584dcc36a0 con 0x55584def5e00 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda44f12b80 10 client.0 _ll_register_callbacks cb 0x55584dde5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58882/0 (socket says 172.21.15.46:58882) 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda29e00700 1 -- 172.21.15.46:0/2320023418 learned_addr learned my addr 172.21.15.46:0/2320023418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.270+0000 7fda2a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda2b200700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584defa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 -- 172.21.15.46:0/2320023418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 msgr2=0x55584def9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 -- 172.21.15.46:0/2320023418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 msgr2=0x55584defa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584defa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 -- 172.21.15.46:0/2320023418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c017070 con 0x55584def5e00 2024-05-08T23:20:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda2a800700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda29e00700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fda180cbab0 tx=0x7fda1800b310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f60613cc3a50d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda0be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.271+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda180a3490 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fda18064070 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55584dd99270 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda180b92c0 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 10 client.30052 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fda2c19e9b0 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda18084020 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda1809a690 con 0x55584def5e00 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 10 client.30052 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 10 client.30052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:20:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 20 client.30052 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df09990 0x55584df0bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 4 client.30052 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.272+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55584debf810 con 0x55584df09990 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.273+0000 7fda2a800700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df09990 0x55584df0bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.273+0000 7fda2a800700 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df09990 0x55584df0bd80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa6f16409c822c28 server_cookie=bc18d5671464748d in_seq=0 out_seq=0 2024-05-08T23:20:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.273+0000 7fda0be00700 10 client.30052 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda0be00700 1 -- 172.21.15.46:0/2320023418 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55584debf810 con 0x55584df09990 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda0be00700 10 client.30052 handle_command_reply: tid=0 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 client.30052 shutdown 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 msgr2=0x55584defa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fda180cbab0 tx=0x7fda1800b310 comp rx=0 tx=0).stop 2024-05-08T23:20:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 shutdown_connections 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df09990 0x55584df0bd80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584defa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 --2- 172.21.15.46:0/2320023418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584defa0c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda0be00700 0 client.30052 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 >> 172.21.15.46:0/2320023418 conn(0x55584de4e210 msgr2=0x55584deea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 shutdown_connections 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.275+0000 7fda44f12b80 1 -- 172.21.15.46:0/2320023418 wait complete. 2024-05-08T23:20:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.276+0000 7fda44f12b80 20 client.30052 trim_cache size 0 max 16384 2024-05-08T23:20:32.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:31 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2990760136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.289+0000 7fda44f12b80 1 Processor -- start 2024-05-08T23:20:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 -- start start 2024-05-08T23:20:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584deead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584df10eb0 con 0x55584def6720 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584dee5800 con 0x55584def0890 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda44f12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584defcae0 con 0x55584def5e00 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda2a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda2a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41842/0 (socket says 172.21.15.46:41842) 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda2a800700 1 -- 172.21.15.46:0/4131398668 learned_addr learned my addr 172.21.15.46:0/4131398668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda29e00700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.290+0000 7fda2b200700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584deead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 -- 172.21.15.46:0/4131398668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 msgr2=0x55584def0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 -- 172.21.15.46:0/4131398668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 msgr2=0x55584deead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584deead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 -- 172.21.15.46:0/4131398668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55584dc6db60 con 0x55584def0890 2024-05-08T23:20:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda29e00700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda2a800700 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009b20 tx=0x7fda1c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ac27e6da2f2925 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda17e00700 1 -- 172.21.15.46:0/4131398668 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c022040 con 0x55584def0890 2024-05-08T23:20:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda17e00700 1 -- 172.21.15.46:0/4131398668 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1c023040 con 0x55584def0890 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.291+0000 7fda17e00700 1 -- 172.21.15.46:0/4131398668 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c003b80 con 0x55584def0890 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- 172.21.15.46:0/4131398668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 msgr2=0x55584def0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009b20 tx=0x7fda1c014000 comp rx=0 tx=0).stop 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- 172.21.15.46:0/4131398668 shutdown_connections 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def6720 0x55584deead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def0890 0x55584def0350 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 --2- 172.21.15.46:0/4131398668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584def0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- 172.21.15.46:0/4131398668 >> 172.21.15.46:0/4131398668 conn(0x55584de4e210 msgr2=0x55584def8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- 172.21.15.46:0/4131398668 shutdown_connections 2024-05-08T23:20:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- 172.21.15.46:0/4131398668 wait complete. 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.292+0000 7fda44f12b80 1 -- start start 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 Processor -- start 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584defeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584df5ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584deffd10 con 0x55584def0890 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584df5f2c0 con 0x55584def6720 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.293+0000 7fda44f12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584df5f400 con 0x55584def5e00 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda44f12b80 10 client.0 _ll_register_callbacks cb 0x55584dde5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58888/0 (socket says 172.21.15.46:58888) 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 -- 172.21.15.46:0/1251998538 learned_addr learned my addr 172.21.15.46:0/1251998538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda2b200700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584df5ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda2a800700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584defeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 -- 172.21.15.46:0/1251998538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 msgr2=0x55584df5ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584df5ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 -- 172.21.15.46:0/1251998538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 msgr2=0x55584defeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584defeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.294+0000 7fda29e00700 1 -- 172.21.15.46:0/1251998538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c19e9b0 con 0x55584def5e00 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda29e00700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fda18006010 tx=0x7fda18097490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7c60439f37b624 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1809a790 con 0x55584def5e00 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fda18064040 con 0x55584def5e00 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1809b0a0 con 0x55584def5e00 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1809b7a0 con 0x55584def5e00 2024-05-08T23:20:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda44f12b80 10 client.30058 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fda1c01da20 con 0x55584def5e00 2024-05-08T23:20:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1809b0a0 con 0x55584def5e00 2024-05-08T23:20:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.295+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda1809b300 con 0x55584def5e00 2024-05-08T23:20:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 10 client.30058 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 10 client.30058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 20 client.30058 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df60600 0x55584df62a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 4 client.30058 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55584de38ec0 con 0x55584df60600 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.296+0000 7fda2a800700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df60600 0x55584df62a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.297+0000 7fda0be00700 10 client.30058 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.297+0000 7fda2a800700 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df60600 0x55584df62a20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eacf2625e06a3bfa server_cookie=ce6a2babff0d07f0 in_seq=0 out_seq=0 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.299+0000 7fda0be00700 1 -- 172.21.15.46:0/1251998538 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55584de38ec0 con 0x55584df60600 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.299+0000 7fda0be00700 10 client.30058 handle_command_reply: tid=0 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.299+0000 7fda44f12b80 1 client.30058 shutdown 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.299+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 msgr2=0x55584deff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.299+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fda18006010 tx=0x7fda18097490 comp rx=0 tx=0).stop 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 shutdown_connections 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55584def6720 0x55584df5ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55584df60600 0x55584df62a20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55584def0890 0x55584defeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 --2- 172.21.15.46:0/1251998538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584def5e00 0x55584deff400 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda0be00700 0 client.30058 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 >> 172.21.15.46:0/1251998538 conn(0x55584de4e210 msgr2=0x55584def5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 shutdown_connections 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 1 -- 172.21.15.46:0/1251998538 wait complete. 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.300+0000 7fda44f12b80 20 client.30058 trim_cache size 0 max 16384 2024-05-08T23:20:32.301 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 msgr2=0x7fda100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 0x7fda100a3750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009c60 tx=0x7fda1c009260 comp rx=0 tx=0).stop 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 msgr2=0x7fda2c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fda18000c00 tx=0x7fda18005740 comp rx=0 tx=0).stop 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 shutdown_connections 2024-05-08T23:20:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c10b400 0x7fda2c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fda100a1290 0x7fda100a3750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda2c101fe0 0x7fda2c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 --2- 172.21.15.46:0/2516735158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda2c102900 0x7fda2c199a20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 >> 172.21.15.46:0/2516735158 conn(0x7fda2c0fb760 msgr2=0x7fda2c10c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 shutdown_connections 2024-05-08T23:20:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.301+0000 7fda31400700 1 -- 172.21.15.46:0/2516735158 wait complete. 2024-05-08T23:20:32.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.547+0000 7f0148600700 1 -- 172.21.15.46:0/4199141999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 msgr2=0x7f014010cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.547+0000 7f0148600700 1 --2- 172.21.15.46:0/4199141999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f014010cc40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f013800b0a0 tx=0x7f013800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 -- 172.21.15.46:0/4199141999 shutdown_connections 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 --2- 172.21.15.46:0/4199141999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f014010cc40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 --2- 172.21.15.46:0/4199141999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 --2- 172.21.15.46:0/4199141999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0140100950 0x7f0140104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 -- 172.21.15.46:0/4199141999 >> 172.21.15.46:0/4199141999 conn(0x7f01400fb820 msgr2=0x7f01400fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 -- 172.21.15.46:0/4199141999 shutdown_connections 2024-05-08T23:20:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.548+0000 7f0148600700 1 -- 172.21.15.46:0/4199141999 wait complete. 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 Processor -- start 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 -- start start 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f014019df20 0x7f014019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014019e9a0 con 0x7f01401050c0 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014019eb10 con 0x7f014019df20 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.549+0000 7f0148600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014019ecd0 con 0x7f01400fffa0 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52786/0 (socket says 172.21.15.46:52786) 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145200700 1 -- 172.21.15.46:0/1686703136 learned_addr learned my addr 172.21.15.46:0/1686703136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0146600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f014019df20 0x7f014019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 -- 172.21.15.46:0/1686703136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f014019df20 msgr2=0x7f014019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f014019df20 0x7f014019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 -- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 msgr2=0x7f0140199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 -- 172.21.15.46:0/1686703136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009d00 con 0x7f01400fffa0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145200700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.550+0000 7f0145c00700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140199640 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f013400d950 tx=0x7f013400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.551+0000 7f013e000700 1 -- 172.21.15.46:0/1686703136 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400f890 con 0x7f01400fffa0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.551+0000 7f013e000700 1 -- 172.21.15.46:0/1686703136 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0134010460 con 0x7f01400fffa0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.551+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01401a6230 con 0x7f01400fffa0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.551+0000 7f013e000700 1 -- 172.21.15.46:0/1686703136 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400e610 con 0x7f01400fffa0 2024-05-08T23:20:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.551+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01401a6760 con 0x7f01400fffa0 2024-05-08T23:20:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.555+0000 7f013e000700 1 -- 172.21.15.46:0/1686703136 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f01340108d0 con 0x7f01400fffa0 2024-05-08T23:20:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f013e000700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 0x7f01280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f015c214b80 1 Processor -- start 2024-05-08T23:20:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f013e000700 1 -- 172.21.15.46:0/1686703136 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f01340ccd10 con 0x7f01400fffa0 2024-05-08T23:20:32.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f015c214b80 1 -- start start 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f0145200700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 0x7f01280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.556+0000 7f015c214b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f015c214b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a78d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f015c214b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a78dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f015c214b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a6e4150 con 0x55c93a793720 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f015c214b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a6e1b10 con 0x55c93a78d890 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f015c214b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a6dd400 con 0x55c93a792e00 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58926/0 (socket says 172.21.15.46:58926) 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 -- 172.21.15.46:0/1232501685 learned_addr learned my addr 172.21.15.46:0/1232501685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145200700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a78d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0146600700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a78dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 -- 172.21.15.46:0/1232501685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 msgr2=0x55c93a78dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a78dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 -- 172.21.15.46:0/1232501685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 msgr2=0x55c93a78d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a78d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145c00700 1 -- 172.21.15.46:0/1232501685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93a55e3b0 con 0x55c93a792e00 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145200700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a78d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.557+0000 7f0145200700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 0x7f01280a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f014019ab90 tx=0x7f012c010760 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.558+0000 7f0145c00700 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f01340a1900 tx=0x7f013409a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6644a6fab14c250e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.558+0000 7f0132a00700 1 -- 172.21.15.46:0/1232501685 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013409d7e0 con 0x55c93a792e00 2024-05-08T23:20:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.558+0000 7f0132a00700 1 -- 172.21.15.46:0/1232501685 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01340a1e30 con 0x55c93a792e00 2024-05-08T23:20:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.558+0000 7f0132a00700 1 -- 172.21.15.46:0/1232501685 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134099100 con 0x55c93a792e00 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.558+0000 7f015c214b80 1 -- 172.21.15.46:0/1232501685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 msgr2=0x55c93a7931e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f01340a1900 tx=0x7f013409a0b0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 -- 172.21.15.46:0/1232501685 shutdown_connections 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a78dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a792e00 0x55c93a7931e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 --2- 172.21.15.46:0/1232501685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a78d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 -- 172.21.15.46:0/1232501685 >> 172.21.15.46:0/1232501685 conn(0x55c93a6eb210 msgr2=0x55c93a787df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 -- 172.21.15.46:0/1232501685 shutdown_connections 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 -- 172.21.15.46:0/1232501685 wait complete. 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.559+0000 7f015c214b80 1 -- start start 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 Processor -- start 2024-05-08T23:20:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a796ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a792e00 0x55c93a797010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a679560 con 0x55c93a792e00 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a679200 con 0x55c93a78d890 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a5606a0 con 0x55c93a793720 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.560+0000 7f015c214b80 10 client.0 _ll_register_callbacks cb 0x55c93a682b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0145c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a796ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58938/0 (socket says 172.21.15.46:58938) 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 -- 172.21.15.46:0/623900634 learned_addr learned my addr 172.21.15.46:0/623900634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0145200700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a792e00 0x55c93a797010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 -- 172.21.15.46:0/623900634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 msgr2=0x55c93a796ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a796ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 -- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a792e00 msgr2=0x55c93a797010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a792e00 0x55c93a797010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 -- 172.21.15.46:0/623900634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01401a6760 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0145c00700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a796ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0146600700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0138015000 tx=0x7f01380094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4377af4b3f4d587b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.561+0000 7f0127400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800e040 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c93a636270 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f01380074e0 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01380076e0 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f015c214b80 10 client.30070 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c93a50ab60 con 0x55c93a793720 2024-05-08T23:20:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.562+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c93a636270 con 0x55c93a793720 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0138025030 con 0x55c93a793720 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 10 client.30070 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 10 client.30070 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 20 client.30070 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7a71e0 0x55c93a7a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 4 client.30070 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c93a75c810 con 0x55c93a7a71e0 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f0145c00700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7a71e0 0x55c93a7a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f0145c00700 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7a71e0 0x55c93a7a9600 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c84c5f2afacf026 server_cookie=88735dea7a6b837 in_seq=0 out_seq=0 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.563+0000 7f0127400700 10 client.30070 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f0127400700 1 -- 172.21.15.46:0/623900634 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c93a75c810 con 0x55c93a7a71e0 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f0127400700 10 client.30070 handle_command_reply: tid=0 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 client.30070 shutdown 2024-05-08T23:20:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 msgr2=0x55c93a797550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0138015000 tx=0x7f01380094d0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 shutdown_connections 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a793720 0x55c93a797550 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a78d890 0x55c93a796ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7a71e0 0x55c93a7a9600 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f015c214b80 1 --2- 172.21.15.46:0/623900634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a792e00 0x55c93a797010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.564+0000 7f0127400700 0 client.30070 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.565+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 >> 172.21.15.46:0/623900634 conn(0x55c93a6eb210 msgr2=0x55c93a7875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.565+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 shutdown_connections 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.565+0000 7f015c214b80 1 -- 172.21.15.46:0/623900634 wait complete. 2024-05-08T23:20:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.565+0000 7f015c214b80 20 client.30070 trim_cache size 0 max 16384 2024-05-08T23:20:32.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 Processor -- start 2024-05-08T23:20:32.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 -- start start 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a787d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a7897e0 con 0x55c93a793720 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a789e50 con 0x55c93a792e00 2024-05-08T23:20:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.580+0000 7f015c214b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a793560 con 0x55c93a78d890 2024-05-08T23:20:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a787d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58942/0 (socket says 172.21.15.46:58942) 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a787d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41902/0 (socket says 172.21.15.46:41902) 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 -- 172.21.15.46:0/3754631024 learned_addr learned my addr 172.21.15.46:0/3754631024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0146600700 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 -- 172.21.15.46:0/3754631024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 msgr2=0x55c93a787d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a787d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 -- 172.21.15.46:0/3754631024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 msgr2=0x55c93a7922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.581+0000 7f0145c00700 1 -- 172.21.15.46:0/3754631024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93a6a3560 con 0x55c93a78d890 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.582+0000 7f0145c00700 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0134010fd0 tx=0x7f013409a220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243f938002dbcb7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.582+0000 7f0133400700 1 -- 172.21.15.46:0/3754631024 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340b2170 con 0x55c93a78d890 2024-05-08T23:20:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.582+0000 7f0133400700 1 -- 172.21.15.46:0/3754631024 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01340b22d0 con 0x55c93a78d890 2024-05-08T23:20:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.582+0000 7f0133400700 1 -- 172.21.15.46:0/3754631024 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134099e70 con 0x55c93a78d890 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- 172.21.15.46:0/3754631024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 msgr2=0x55c93a78d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0134010fd0 tx=0x7f013409a220 comp rx=0 tx=0).stop 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- 172.21.15.46:0/3754631024 shutdown_connections 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a78d350 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 --2- 172.21.15.46:0/3754631024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a787d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- 172.21.15.46:0/3754631024 >> 172.21.15.46:0/3754631024 conn(0x55c93a6eb210 msgr2=0x55c93a791bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- 172.21.15.46:0/3754631024 shutdown_connections 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- 172.21.15.46:0/3754631024 wait complete. 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 -- start start 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.583+0000 7f015c214b80 1 Processor -- start 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a79c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a79cf20 con 0x55c93a793720 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a7fcbe0 con 0x55c93a792e00 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93a7fcd20 con 0x55c93a78d890 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f015c214b80 10 client.0 _ll_register_callbacks cb 0x55c93a682b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f0145c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:58952/0 (socket says 172.21.15.46:58952) 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f0145c00700 1 -- 172.21.15.46:0/2842732077 learned_addr learned my addr 172.21.15.46:0/2842732077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f0145200700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a79c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.584+0000 7f0146600700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 -- 172.21.15.46:0/2842732077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 msgr2=0x55c93a79c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a79c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 -- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 msgr2=0x55c93a7fc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7fc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 -- 172.21.15.46:0/2842732077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c93a55e3b0 con 0x55c93a78d890 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145200700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a79c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0145c00700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f01340998f0 tx=0x7f0134010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce3ceff54c6b6181 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0127400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340b2170 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0134099e10 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01340a1e30 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0134098ea0 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f015c214b80 10 client.30076 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c93a562cd0 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.585+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01340a1e30 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01340b90b0 con 0x55c93a78d890 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 10 client.30076 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 10 client.30076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 20 client.30076 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7fdf50 0x55c93a800340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 4 client.30076 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:32.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c93a6d5ec0 con 0x55c93a7fdf50 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.586+0000 7f0145200700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7fdf50 0x55c93a800340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f0127400700 10 client.30076 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f0145200700 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7fdf50 0x55c93a800340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5aab87c0fc0e6464 server_cookie=bba9233bcbeeb107 in_seq=0 out_seq=0 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f0127400700 1 -- 172.21.15.46:0/2842732077 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c93a6d5ec0 con 0x55c93a7fdf50 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f0127400700 10 client.30076 handle_command_reply: tid=0 2024-05-08T23:20:32.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 client.30076 shutdown 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 msgr2=0x55c93a79c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f01340998f0 tx=0x7f0134010cd0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 shutdown_connections 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55c93a7fdf50 0x55c93a800340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93a793720 0x55c93a7fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c93a78d890 0x55c93a79c0f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f0127400700 0 client.30076 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 --2- 172.21.15.46:0/2842732077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c93a792e00 0x55c93a79c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 >> 172.21.15.46:0/2842732077 conn(0x55c93a6eb210 msgr2=0x55c93a792450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 shutdown_connections 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.587+0000 7f015c214b80 1 -- 172.21.15.46:0/2842732077 wait complete. 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f015c214b80 20 client.30076 trim_cache size 0 max 16384 2024-05-08T23:20:32.592 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:20:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 msgr2=0x7f01280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 0x7f01280a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f014019ab90 tx=0x7f012c010760 comp rx=0 tx=0).stop 2024-05-08T23:20:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 msgr2=0x7f0140199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140199640 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f013400d950 tx=0x7f013400dc60 comp rx=0 tx=0).stop 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 shutdown_connections 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f014019df20 0x7f014019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01400fffa0 0x7f0140199640 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f01280a1310 0x7f01280a37d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 --2- 172.21.15.46:0/1686703136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01401050c0 0x7f0140199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 >> 172.21.15.46:0/1686703136 conn(0x7f01400fb820 msgr2=0x7f01400fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 shutdown_connections 2024-05-08T23:20:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.588+0000 7f0148600700 1 -- 172.21.15.46:0/1686703136 wait complete. 2024-05-08T23:20:32.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:20:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.890+0000 7f6204e00700 1 -- 172.21.15.46:0/1076842239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200111e00 msgr2=0x7f62001141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.890+0000 7f6204e00700 1 --2- 172.21.15.46:0/1076842239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200111e00 0x7f62001141f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009f90 tx=0x7f61f800b330 comp rx=0 tx=0).stop 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 -- 172.21.15.46:0/1076842239 shutdown_connections 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 --2- 172.21.15.46:0/1076842239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200111e00 0x7f62001141f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 --2- 172.21.15.46:0/1076842239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073340 0x7f6200073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 --2- 172.21.15.46:0/1076842239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200073c60 0x7f62001118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 -- 172.21.15.46:0/1076842239 >> 172.21.15.46:0/1076842239 conn(0x7f62000fc900 msgr2=0x7f62000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.891+0000 7f6204e00700 1 -- 172.21.15.46:0/1076842239 shutdown_connections 2024-05-08T23:20:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.892+0000 7f6204e00700 1 -- 172.21.15.46:0/1076842239 wait complete. 2024-05-08T23:20:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.894+0000 7f6204e00700 1 Processor -- start 2024-05-08T23:20:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.894+0000 7f6204e00700 1 -- start start 2024-05-08T23:20:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 0x7f620019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073c60 0x7f620019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 0x7f62001a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001a2950 con 0x7f6200073340 2024-05-08T23:20:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001a2ac0 con 0x7f6200111e00 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.896+0000 7f6204e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001a2c80 con 0x7f6200073c60 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.897+0000 7f61fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073c60 0x7f620019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.897+0000 7f61fe800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 0x7f62001a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.897+0000 7f61fe800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 0x7f62001a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41926/0 (socket says 172.21.15.46:41926) 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.897+0000 7f61fe800700 1 -- 172.21.15.46:0/2426542257 learned_addr learned my addr 172.21.15.46:0/2426542257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.897+0000 7f61fde00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 0x7f620019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.899+0000 7f61fde00700 1 -- 172.21.15.46:0/2426542257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073c60 msgr2=0x7f620019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.899+0000 7f61fde00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073c60 0x7f620019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.899+0000 7f61fde00700 1 -- 172.21.15.46:0/2426542257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 msgr2=0x7f62001a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.899+0000 7f61fde00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 0x7f62001a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.899+0000 7f61fde00700 1 -- 172.21.15.46:0/2426542257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8009c40 con 0x7f6200073340 2024-05-08T23:20:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.901+0000 7f61fde00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 0x7f620019da70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9c0 tx=0x7f61f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.902+0000 7f61f6a00700 1 -- 172.21.15.46:0/2426542257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0004220 con 0x7f6200073340 2024-05-08T23:20:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.902+0000 7f61f6a00700 1 -- 172.21.15.46:0/2426542257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61f0004cf0 con 0x7f6200073340 2024-05-08T23:20:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.905+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62001a2f60 con 0x7f6200073340 2024-05-08T23:20:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.905+0000 7f61f6a00700 1 -- 172.21.15.46:0/2426542257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000ce20 con 0x7f6200073340 2024-05-08T23:20:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.905+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62001aa810 con 0x7f6200073340 2024-05-08T23:20:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.911+0000 7f61f6a00700 1 -- 172.21.15.46:0/2426542257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f61f0004380 con 0x7f6200073340 2024-05-08T23:20:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.913+0000 7f6218894b80 1 Processor -- start 2024-05-08T23:20:32.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.914+0000 7f6218894b80 1 -- start start 2024-05-08T23:20:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.914+0000 7f6218894b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36ae00 0x55b9fc36b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.914+0000 7f6218894b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc365350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.914+0000 7f6218894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f6218894b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc2bc150 con 0x55b9fc36ae00 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f6218894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc2b9b10 con 0x55b9fc365890 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f6218894b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc2b5400 con 0x55b9fc36b720 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41936/0 (socket says 172.21.15.46:41936) 2024-05-08T23:20:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 -- 172.21.15.46:0/107076886 learned_addr learned my addr 172.21.15.46:0/107076886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 -- 172.21.15.46:0/107076886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 msgr2=0x55b9fc365350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc365350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 -- 172.21.15.46:0/107076886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36ae00 msgr2=0x55b9fc36b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36ae00 0x55b9fc36b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.915+0000 7f61fd400700 1 -- 172.21.15.46:0/107076886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fc1363b0 con 0x55b9fc365890 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.916+0000 7f61f6a00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 0x7f61e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.916+0000 7f61f6a00700 1 -- 172.21.15.46:0/2426542257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f61f0014070 con 0x7f6200073340 2024-05-08T23:20:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.916+0000 7f61fe800700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 0x7f61e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.917+0000 7f61fe800700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 0x7f61e40a37d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61f80093f0 tx=0x7f61f8009380 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.925+0000 7f61fd400700 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009b30 tx=0x7f61ec009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141963354d1b643c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.937+0000 7f61eb400700 1 -- 172.21.15.46:0/107076886 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00eaa0 con 0x55b9fc365890 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.937+0000 7f61eb400700 1 -- 172.21.15.46:0/107076886 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ec004cf0 con 0x55b9fc365890 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.937+0000 7f61eb400700 1 -- 172.21.15.46:0/107076886 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec005580 con 0x55b9fc365890 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.938+0000 7f6218894b80 1 -- 172.21.15.46:0/107076886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 msgr2=0x55b9fc365cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.938+0000 7f6218894b80 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009b30 tx=0x7f61ec009e40 comp rx=0 tx=0).stop 2024-05-08T23:20:32.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 -- 172.21.15.46:0/107076886 shutdown_connections 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc365cd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36ae00 0x55b9fc36b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 --2- 172.21.15.46:0/107076886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc365350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 -- 172.21.15.46:0/107076886 >> 172.21.15.46:0/107076886 conn(0x55b9fc2c3210 msgr2=0x55b9fc35fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.939+0000 7f6218894b80 1 -- 172.21.15.46:0/107076886 shutdown_connections 2024-05-08T23:20:32.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.940+0000 7f6218894b80 1 -- 172.21.15.46:0/107076886 wait complete. 2024-05-08T23:20:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.941+0000 7f6218894b80 1 -- start start 2024-05-08T23:20:32.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.941+0000 7f6218894b80 1 Processor -- start 2024-05-08T23:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.942+0000 7f6218894b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.942+0000 7f6218894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36ae00 0x55b9fc36f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.942+0000 7f6218894b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc36f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.943+0000 7f6218894b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc251560 con 0x55b9fc365890 2024-05-08T23:20:32.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.943+0000 7f6218894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc251200 con 0x55b9fc36ae00 2024-05-08T23:20:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.943+0000 7f6218894b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc1386a0 con 0x55b9fc36b720 2024-05-08T23:20:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.943+0000 7f6218894b80 10 client.0 _ll_register_callbacks cb 0x55b9fc25ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:32.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.944+0000 7f61fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.944+0000 7f61fde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52870/0 (socket says 172.21.15.46:52870) 2024-05-08T23:20:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.944+0000 7f61fde00700 1 -- 172.21.15.46:0/2507496903 learned_addr learned my addr 172.21.15.46:0/2507496903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.945+0000 7f61fde00700 1 -- 172.21.15.46:0/2507496903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 msgr2=0x55b9fc36f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.945+0000 7f61fde00700 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc36f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.945+0000 7f61fde00700 1 -- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36ae00 msgr2=0x55b9fc36f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.945+0000 7f61fde00700 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36ae00 0x55b9fc36f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.945+0000 7f61fde00700 1 -- 172.21.15.46:0/2507496903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec009790 con 0x55b9fc365890 2024-05-08T23:20:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.946+0000 7f61fde00700 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c5b0 tx=0x7f61f0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb875ee5fab4f4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.946+0000 7f61dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:20:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.947+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0099ae0 con 0x55b9fc365890 2024-05-08T23:20:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.947+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f61f009d910 con 0x55b9fc365890 2024-05-08T23:20:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.947+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9fc20e270 con 0x55b9fc365890 2024-05-08T23:20:32.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.949+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f00a1f60 con 0x55b9fc365890 2024-05-08T23:20:32.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.949+0000 7f6218894b80 10 client.19854 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.949+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b9fc0e2b60 con 0x55b9fc365890 2024-05-08T23:20:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.950+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9fc20e270 con 0x55b9fc365890 2024-05-08T23:20:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61f00958d0 con 0x55b9fc365890 2024-05-08T23:20:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 10 client.19854 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 10 client.19854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 20 client.19854 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3900e0 0x55b9fc392500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 4 client.19854 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.951+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9fc334810 con 0x55b9fc3900e0 2024-05-08T23:20:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.952+0000 7f61fd400700 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3900e0 0x55b9fc392500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.960+0000 7f61dfe00700 10 client.19854 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.960+0000 7f61fd400700 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3900e0 0x55b9fc392500 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadb46a1b88b2448 server_cookie=19cca3592cbe607d in_seq=0 out_seq=0 2024-05-08T23:20:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f61dfe00700 1 -- 172.21.15.46:0/2507496903 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9fc334810 con 0x55b9fc3900e0 2024-05-08T23:20:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f61dfe00700 10 client.19854 handle_command_reply: tid=0 2024-05-08T23:20:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 client.19854 shutdown 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 msgr2=0x55b9fc36eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c5b0 tx=0x7f61f0004f90 comp rx=0 tx=0).stop 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 shutdown_connections 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36b720 0x55b9fc36f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3900e0 0x55b9fc392500 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc36eaf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 --2- 172.21.15.46:0/2507496903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36ae00 0x55b9fc36f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f61dfe00700 0 client.19854 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.972+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 >> 172.21.15.46:0/2507496903 conn(0x55b9fc2c3210 msgr2=0x55b9fc35f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:32.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.975+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 shutdown_connections 2024-05-08T23:20:32.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.975+0000 7f6218894b80 1 -- 172.21.15.46:0/2507496903 wait complete. 2024-05-08T23:20:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:32.975+0000 7f6218894b80 20 client.19854 trim_cache size 0 max 16384 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.006+0000 7f6218894b80 1 Processor -- start 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.006+0000 7f6218894b80 1 -- start start 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc365350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 0x55b9fc365c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc360b30 con 0x55b9fc365890 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc370160 con 0x55b9fc36b720 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f6218894b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc36b580 con 0x55b9fc36ae00 2024-05-08T23:20:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f61fe800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 0x55b9fc365c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f61fe800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 0x55b9fc365c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41950/0 (socket says 172.21.15.46:41950) 2024-05-08T23:20:33.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.007+0000 7f61fe800700 1 -- 172.21.15.46:0/3732991785 learned_addr learned my addr 172.21.15.46:0/3732991785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:33.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.008+0000 7f61fde00700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc365350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.008+0000 7f61fd400700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.008+0000 7f61fde00700 1 -- 172.21.15.46:0/3732991785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 msgr2=0x55b9fc3924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fde00700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fde00700 1 -- 172.21.15.46:0/3732991785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 msgr2=0x55b9fc365c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fde00700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 0x55b9fc365c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fde00700 1 -- 172.21.15.46:0/3732991785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fc27b560 con 0x55b9fc365890 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fd400700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.009+0000 7f61fde00700 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc365350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f61f0098a40 tx=0x7f61f0009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ab62f7cb3146af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.010+0000 7f61ebe00700 1 -- 172.21.15.46:0/3732991785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f009d910 con 0x55b9fc365890 2024-05-08T23:20:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.010+0000 7f61ebe00700 1 -- 172.21.15.46:0/3732991785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61f0099ae0 con 0x55b9fc365890 2024-05-08T23:20:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.010+0000 7f61ebe00700 1 -- 172.21.15.46:0/3732991785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a1d30 con 0x55b9fc365890 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 -- 172.21.15.46:0/3732991785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 msgr2=0x55b9fc365350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc365350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f61f0098a40 tx=0x7f61f0009690 comp rx=0 tx=0).stop 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 -- 172.21.15.46:0/3732991785 shutdown_connections 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc36b720 0x55b9fc365c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc365890 0x55b9fc365350 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 --2- 172.21.15.46:0/3732991785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 -- 172.21.15.46:0/3732991785 >> 172.21.15.46:0/3732991785 conn(0x55b9fc2c3210 msgr2=0x55b9fc35c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 -- 172.21.15.46:0/3732991785 shutdown_connections 2024-05-08T23:20:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.011+0000 7f6218894b80 1 -- 172.21.15.46:0/3732991785 wait complete. 2024-05-08T23:20:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.012+0000 7f6218894b80 1 -- start start 2024-05-08T23:20:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.012+0000 7f6218894b80 1 Processor -- start 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc387190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 0x55b9fc3851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc3858c0 con 0x55b9fc36b720 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc385a30 con 0x55b9fc365890 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc385bf0 con 0x55b9fc36ae00 2024-05-08T23:20:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.013+0000 7f6218894b80 10 client.0 _ll_register_callbacks cb 0x55b9fc25ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.014+0000 7f61fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.014+0000 7f61fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3876d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59014/0 (socket says 172.21.15.46:59014) 2024-05-08T23:20:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.014+0000 7f61fd400700 1 -- 172.21.15.46:0/883677988 learned_addr learned my addr 172.21.15.46:0/883677988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.015+0000 7f61fe800700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 0x55b9fc3851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.016+0000 7f61fe800700 1 -- 172.21.15.46:0/883677988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 msgr2=0x55b9fc3876d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.016+0000 7f61fe800700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3876d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.016+0000 7f61fe800700 1 -- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 msgr2=0x55b9fc387190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.016+0000 7f61fe800700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc387190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.016+0000 7f61fe800700 1 -- 172.21.15.46:0/883677988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9fc0e2b60 con 0x55b9fc36b720 2024-05-08T23:20:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.017+0000 7f61fe800700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 0x55b9fc3851b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009f60 tx=0x7f61f800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9865fa8e3cb3c08f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.017+0000 7f61dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:20:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.019+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8023040 con 0x55b9fc36b720 2024-05-08T23:20:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.020+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f61f802a450 con 0x55b9fc36b720 2024-05-08T23:20:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.020+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61ec004cf0 con 0x55b9fc36b720 2024-05-08T23:20:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.021+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f800e040 con 0x55b9fc36b720 2024-05-08T23:20:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f6218894b80 10 client.19860 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b9fc13acd0 con 0x55b9fc36b720 2024-05-08T23:20:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ec004cf0 con 0x55b9fc36b720 2024-05-08T23:20:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61f801c760 con 0x55b9fc36b720 2024-05-08T23:20:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f6218894b80 10 client.19860 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f6218894b80 10 client.19860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.022+0000 7f6218894b80 20 client.19860 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.023+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3e6860 0x55b9fc3e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.023+0000 7f6218894b80 4 client.19860 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:33.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.023+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9fc2adec0 con 0x55b9fc3e6860 2024-05-08T23:20:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.024+0000 7f61fde00700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3e6860 0x55b9fc3e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.025+0000 7f61fde00700 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3e6860 0x55b9fc3e8c80 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e244f72cb26ef298 server_cookie=750b6f322f36bc67 in_seq=0 out_seq=0 2024-05-08T23:20:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.025+0000 7f61dfe00700 10 client.19860 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:33.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.033+0000 7f61dfe00700 1 -- 172.21.15.46:0/883677988 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9fc2adec0 con 0x55b9fc3e6860 2024-05-08T23:20:33.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.033+0000 7f61dfe00700 10 client.19860 handle_command_reply: tid=0 2024-05-08T23:20:33.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.033+0000 7f6218894b80 1 client.19860 shutdown 2024-05-08T23:20:33.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.034+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 msgr2=0x55b9fc3851b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:33.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.034+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 0x55b9fc3851b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009f60 tx=0x7f61f800bd80 comp rx=0 tx=0).stop 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.034+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 shutdown_connections 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.034+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b9fc3e6860 0x55b9fc3e8c80 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9fc36b720 0x55b9fc3851b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b9fc365890 0x55b9fc387190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 --2- 172.21.15.46:0/883677988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9fc36ae00 0x55b9fc3876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 >> 172.21.15.46:0/883677988 conn(0x55b9fc2c3210 msgr2=0x55b9fc365c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f61dfe00700 0 client.19860 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 shutdown_connections 2024-05-08T23:20:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.035+0000 7f6218894b80 1 -- 172.21.15.46:0/883677988 wait complete. 2024-05-08T23:20:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.037+0000 7f6218894b80 20 client.19860 trim_cache size 0 max 16384 2024-05-08T23:20:33.037 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:20:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.038+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 msgr2=0x7f61e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.038+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 0x7f61e40a37d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61f80093f0 tx=0x7f61f8009380 comp rx=0 tx=0).stop 2024-05-08T23:20:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.038+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 msgr2=0x7f620019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.038+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 0x7f620019da70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9c0 tx=0x7f61f000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 shutdown_connections 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6200111e00 0x7f62001a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f61e40a1310 0x7f61e40a37d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200073340 0x7f620019da70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 --2- 172.21.15.46:0/2426542257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6200073c60 0x7f620019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 >> 172.21.15.46:0/2426542257 conn(0x7f62000fc900 msgr2=0x7f62000fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 shutdown_connections 2024-05-08T23:20:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:33.039+0000 7f6204e00700 1 -- 172.21.15.46:0/2426542257 wait complete. 2024-05-08T23:20:33.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 68476 19.26 MB/sec execute 54 sec latency 529.544 ms 2024-05-08T23:20:34.079 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:20:34.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:20:34.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:33 smithi086 ceph-mon[78610]: pgmap v525: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 460 op/s 2024-05-08T23:20:34.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 68720 19.08 MB/sec execute 55 sec latency 648.439 ms 2024-05-08T23:20:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:33 smithi053 ceph-mon[77587]: pgmap v525: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 460 op/s 2024-05-08T23:20:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:33 smithi046 ceph-mon[36898]: pgmap v525: 129 pgs: 129 active+clean; 933 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 460 op/s 2024-05-08T23:20:34.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 -- 172.21.15.46:0/3931928057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640fef20 msgr2=0x7f1e640ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 --2- 172.21.15.46:0/3931928057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640fef20 0x7f1e640ff3a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009990 tx=0x7f1e50009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 -- 172.21.15.46:0/3931928057 shutdown_connections 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 --2- 172.21.15.46:0/3931928057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e640ff8e0 0x7f1e6410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 --2- 172.21.15.46:0/3931928057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e64105f50 0x7f1e64106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 --2- 172.21.15.46:0/3931928057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640fef20 0x7f1e640ff3a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 -- 172.21.15.46:0/3931928057 >> 172.21.15.46:0/3931928057 conn(0x7f1e64075eb0 msgr2=0x7f1e640fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 -- 172.21.15.46:0/3931928057 shutdown_connections 2024-05-08T23:20:34.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.347+0000 7f1e6a800700 1 -- 172.21.15.46:0/3931928057 wait complete. 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.348+0000 7f1e6a800700 1 Processor -- start 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 -- start start 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640ff8e0 0x7f1e64199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 0x7f1e6419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419e330 con 0x7f1e64105f50 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419e4a0 con 0x7f1e640ff8e0 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e6a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419e660 con 0x7f1e640fef20 2024-05-08T23:20:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e62a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640ff8e0 0x7f1e64199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59030/0 (socket says 172.21.15.46:59030) 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e63400700 1 -- 172.21.15.46:0/2197082234 learned_addr learned my addr 172.21.15.46:0/2197082234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.349+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 0x7f1e6419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 -- 172.21.15.46:0/2197082234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640ff8e0 msgr2=0x7f1e64199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640ff8e0 0x7f1e64199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 -- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 msgr2=0x7f1e6419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 0x7f1e6419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 -- 172.21.15.46:0/2197082234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500095f0 con 0x7f1e640fef20 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 0x7f1e6419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e63400700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e9c0 tx=0x7f1e5400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e5be00700 1 -- 172.21.15.46:0/2197082234 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540097b0 con 0x7f1e640fef20 2024-05-08T23:20:34.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e5be00700 1 -- 172.21.15.46:0/2197082234 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e54004cf0 con 0x7f1e640fef20 2024-05-08T23:20:34.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6419e940 con 0x7f1e640fef20 2024-05-08T23:20:34.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e5be00700 1 -- 172.21.15.46:0/2197082234 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54005600 con 0x7f1e640fef20 2024-05-08T23:20:34.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.350+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e641a6180 con 0x7f1e640fef20 2024-05-08T23:20:34.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.354+0000 7f1e5be00700 1 -- 172.21.15.46:0/2197082234 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1e5400cc60 con 0x7f1e640fef20 2024-05-08T23:20:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 Processor -- start 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 -- start start 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e5be00700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 0x7f1e480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e5be00700 1 -- 172.21.15.46:0/2197082234 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1e54014070 con 0x7f1e640fef20 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.356+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ef461d0 con 0x55a84eff57a0 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ef43b90 con 0x55a84eff4e80 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ef3f480 con 0x55a84efef910 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 0x7f1e480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e62a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41990/0 (socket says 172.21.15.46:41990) 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 -- 172.21.15.46:0/2614309088 learned_addr learned my addr 172.21.15.46:0/2614309088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 -- 172.21.15.46:0/2614309088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 msgr2=0x55a84efefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efefd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 -- 172.21.15.46:0/2614309088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 msgr2=0x55a84efef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63400700 1 -- 172.21.15.46:0/2614309088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84edc0430 con 0x55a84eff4e80 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.357+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e63400700 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1e540a2000 tx=0x7f1e54099a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47962a4c890eafa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e63e00700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 0x7f1e480a3750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009d60 tx=0x7f1e5c009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e58c00700 1 -- 172.21.15.46:0/2614309088 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b1070 con 0x55a84eff4e80 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e58c00700 1 -- 172.21.15.46:0/2614309088 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5409a0b0 con 0x55a84eff4e80 2024-05-08T23:20:34.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e58c00700 1 -- 172.21.15.46:0/2614309088 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540967c0 con 0x55a84eff4e80 2024-05-08T23:20:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e7e237b80 1 -- 172.21.15.46:0/2614309088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 msgr2=0x55a84eff5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.358+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1e540a2000 tx=0x7f1e54099a90 comp rx=0 tx=0).stop 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 -- 172.21.15.46:0/2614309088 shutdown_connections 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff5260 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/2614309088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 -- 172.21.15.46:0/2614309088 >> 172.21.15.46:0/2614309088 conn(0x55a84ef4d290 msgr2=0x55a84efe9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 -- 172.21.15.46:0/2614309088 shutdown_connections 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 -- 172.21.15.46:0/2614309088 wait complete. 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.359+0000 7f1e7e237b80 1 -- start start 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 Processor -- start 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84eff9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84eff9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84eedb5e0 con 0x55a84eff57a0 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84eedb280 con 0x55a84efef910 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84edc2720 con 0x55a84eff4e80 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e7e237b80 10 client.0 _ll_register_callbacks cb 0x55a84eee4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e63400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41994/0 (socket says 172.21.15.46:41994) 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e63400700 1 -- 172.21.15.46:0/3358189919 learned_addr learned my addr 172.21.15.46:0/3358189919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.360+0000 7f1e62a00700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84eff9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63e00700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84eff9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 -- 172.21.15.46:0/3358189919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 msgr2=0x55a84eff9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84eff9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 -- 172.21.15.46:0/3358189919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 msgr2=0x55a84eff9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84eff9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 -- 172.21.15.46:0/3358189919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e641a6180 con 0x55a84efef910 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e63400700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1e54096b20 tx=0x7f1e5409df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64c3908b33adfc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e4d600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.361+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b1070 con 0x55a84efef910 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.362+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1e540cae40 con 0x55a84efef910 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.362+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a84ee982f0 con 0x55a84efef910 2024-05-08T23:20:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.362+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e540984d0 con 0x55a84efef910 2024-05-08T23:20:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.362+0000 7f1e7e237b80 10 client.30029 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.362+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a84ed6cbe0 con 0x55a84efef910 2024-05-08T23:20:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a84ee982f0 con 0x55a84efef910 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1e540995a0 con 0x55a84efef910 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 10 client.30029 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 10 client.30029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 20 client.30029 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f008e10 0x55a84f00b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 4 client.30029 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a84efbe890 con 0x55a84f008e10 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.363+0000 7f1e62a00700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f008e10 0x55a84f00b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.364+0000 7f1e62a00700 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f008e10 0x55a84f00b230 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76d0c6d32e3943 server_cookie=9cbc67b7378b4ddc in_seq=0 out_seq=0 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.364+0000 7f1e4d600700 10 client.30029 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e4d600700 1 -- 172.21.15.46:0/3358189919 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a84efbe890 con 0x55a84f008e10 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e4d600700 10 client.30029 handle_command_reply: tid=0 2024-05-08T23:20:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 client.30029 shutdown 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 msgr2=0x55a84eff8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1e54096b20 tx=0x7f1e5409df50 comp rx=0 tx=0).stop 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 shutdown_connections 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f008e10 0x55a84f00b230 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84eff9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84efef910 0x55a84eff8c00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3358189919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84eff9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 >> 172.21.15.46:0/3358189919 conn(0x55a84ef4d290 msgr2=0x55a84efe9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.365+0000 7f1e4d600700 0 client.30029 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.366+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 shutdown_connections 2024-05-08T23:20:34.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.366+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3358189919 wait complete. 2024-05-08T23:20:34.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.366+0000 7f1e7e237b80 20 client.30029 trim_cache size 0 max 16384 2024-05-08T23:20:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.391+0000 7f1e7e237b80 1 Processor -- start 2024-05-08T23:20:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.391+0000 7f1e7e237b80 1 -- start start 2024-05-08T23:20:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efe9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84effa1d0 con 0x55a84eff57a0 2024-05-08T23:20:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84eff7150 con 0x55a84eff4e80 2024-05-08T23:20:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84efe20c0 con 0x55a84efef910 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59058/0 (socket says 172.21.15.46:59058) 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.392+0000 7f1e63400700 1 -- 172.21.15.46:0/3539960115 learned_addr learned my addr 172.21.15.46:0/3539960115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e62a00700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 -- 172.21.15.46:0/3539960115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 msgr2=0x55a84eff4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63e00700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efe9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 -- 172.21.15.46:0/3539960115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 msgr2=0x55a84efe9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efe9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 -- 172.21.15.46:0/3539960115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84efeffc0 con 0x55a84efef910 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e62a00700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e63400700 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1e5409a040 tx=0x7f1e54099a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=300478d3ab1fdc30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.393+0000 7f1e59600700 1 -- 172.21.15.46:0/3539960115 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b1070 con 0x55a84efef910 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.394+0000 7f1e59600700 1 -- 172.21.15.46:0/3539960115 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e54098450 con 0x55a84efef910 2024-05-08T23:20:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.394+0000 7f1e59600700 1 -- 172.21.15.46:0/3539960115 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54099760 con 0x55a84efef910 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3539960115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 msgr2=0x55a84efef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1e5409a040 tx=0x7f1e54099a90 comp rx=0 tx=0).stop 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3539960115 shutdown_connections 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84eff57a0 0x55a84efe9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84efef910 0x55a84efef3d0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/3539960115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff4e80 0x55a84eff4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3539960115 >> 172.21.15.46:0/3539960115 conn(0x55a84ef4d290 msgr2=0x55a84eff73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3539960115 shutdown_connections 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- 172.21.15.46:0/3539960115 wait complete. 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 -- start start 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.395+0000 7f1e7e237b80 1 Processor -- start 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84effea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 0x55a84efff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84efff780 con 0x55a84efef910 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84efff8f0 con 0x55a84eff57a0 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84f05f350 con 0x55a84eff4e80 2024-05-08T23:20:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e7e237b80 10 client.0 _ll_register_callbacks cb 0x55a84eee4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84effea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e63400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52916/0 (socket says 172.21.15.46:52916) 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e62a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84effea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59074/0 (socket says 172.21.15.46:59074) 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e63400700 1 -- 172.21.15.46:0/1482069709 learned_addr learned my addr 172.21.15.46:0/1482069709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.396+0000 7f1e63e00700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 0x55a84efff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 -- 172.21.15.46:0/1482069709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 msgr2=0x55a84effea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84effea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 -- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 msgr2=0x55a84efff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 0x55a84efff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 -- 172.21.15.46:0/1482069709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a84ed6cbe0 con 0x55a84efef910 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63e00700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 0x55a84efff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e63400700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cc040 tx=0x7f1e54099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe1e7b970e19c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e4d600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b1070 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1e540cae40 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e5409a0b0 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5409dee0 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e7e237b80 10 client.19875 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.397+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a84ef055e0 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5409a0b0 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1e540b4d90 con 0x55a84efef910 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 10 client.19875 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 10 client.19875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 20 client.19875 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f060680 0x55a84f062a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 4 client.19875 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.398+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a84ef37f40 con 0x55a84f060680 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.399+0000 7f1e62a00700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f060680 0x55a84f062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.399+0000 7f1e62a00700 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f060680 0x55a84f062a70 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b43c684f02fcbd7 server_cookie=201f51e54ff67f71 in_seq=0 out_seq=0 2024-05-08T23:20:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.399+0000 7f1e4d600700 10 client.19875 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e4d600700 1 -- 172.21.15.46:0/1482069709 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a84ef37f40 con 0x55a84f060680 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e4d600700 10 client.19875 handle_command_reply: tid=0 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 client.19875 shutdown 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 msgr2=0x55a84f000680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cc040 tx=0x7f1e54099870 comp rx=0 tx=0).stop 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 shutdown_connections 2024-05-08T23:20:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a84eff57a0 0x55a84efff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.401+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55a84f060680 0x55a84f062a70 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a84efef910 0x55a84f000680 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e4d600700 0 client.19875 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 1 --2- 172.21.15.46:0/1482069709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a84eff4e80 0x55a84effea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 >> 172.21.15.46:0/1482069709 conn(0x55a84ef4d290 msgr2=0x55a84eff7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 shutdown_connections 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 1 -- 172.21.15.46:0/1482069709 wait complete. 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.402+0000 7f1e7e237b80 20 client.19875 trim_cache size 0 max 16384 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:20:34.403 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "24ff49d6-085b-4031-9ae0-c89c7682797f", 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 msgr2=0x7f1e480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 0x7f1e480a3750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009d60 tx=0x7f1e5c009920 comp rx=0 tx=0).stop 2024-05-08T23:20:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 msgr2=0x7f1e64199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e9c0 tx=0x7f1e5400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 shutdown_connections 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1e480a1290 0x7f1e480a3750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e64105f50 0x7f1e6419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e640fef20 0x7f1e64199410 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 --2- 172.21.15.46:0/2197082234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e640ff8e0 0x7f1e64199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 >> 172.21.15.46:0/2197082234 conn(0x7f1e64075eb0 msgr2=0x7f1e64100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 shutdown_connections 2024-05-08T23:20:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.404+0000 7f1e6a800700 1 -- 172.21.15.46:0/2197082234 wait complete. 2024-05-08T23:20:34.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 -- 172.21.15.46:0/2195550150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc0ffd90 msgr2=0x7f67cc100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2195550150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc0ffd90 0x7f67cc100170 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009990 tx=0x7f67bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 -- 172.21.15.46:0/2195550150 shutdown_connections 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2195550150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc105190 0x7f67cc105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2195550150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc0ffd90 0x7f67cc100170 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2195550150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc1006b0 0x7f67cc104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 -- 172.21.15.46:0/2195550150 >> 172.21.15.46:0/2195550150 conn(0x7f67cc0fb760 msgr2=0x7f67cc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 -- 172.21.15.46:0/2195550150 shutdown_connections 2024-05-08T23:20:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.669+0000 7f67d0c00700 1 -- 172.21.15.46:0/2195550150 wait complete. 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 Processor -- start 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 -- start start 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc1006b0 0x7f67cc1a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc105190 0x7f67cc19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc19dd90 con 0x7f67cc105190 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc19df00 con 0x7f67cc1006b0 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67d0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc19e0c0 con 0x7f67cc0ffd90 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.670+0000 7f67c9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc1006b0 0x7f67cc1a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59096/0 (socket says 172.21.15.46:59096) 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 -- 172.21.15.46:0/2522101416 learned_addr learned my addr 172.21.15.46:0/2522101416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67ca800700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc105190 0x7f67cc19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 -- 172.21.15.46:0/2522101416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc1006b0 msgr2=0x7f67cc1a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc1006b0 0x7f67cc1a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 -- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc105190 msgr2=0x7f67cc19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc105190 0x7f67cc19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 -- 172.21.15.46:0/2522101416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0095f0 con 0x7f67cc0ffd90 2024-05-08T23:20:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.671+0000 7f67c9e00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005910 tx=0x7f67bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.672+0000 7f67c2a00700 1 -- 172.21.15.46:0/2522101416 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc01d070 con 0x7f67cc0ffd90 2024-05-08T23:20:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.672+0000 7f67c2a00700 1 -- 172.21.15.46:0/2522101416 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc022470 con 0x7f67cc0ffd90 2024-05-08T23:20:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.672+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc19e340 con 0x7f67cc0ffd90 2024-05-08T23:20:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.672+0000 7f67c2a00700 1 -- 172.21.15.46:0/2522101416 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc017650 con 0x7f67cc0ffd90 2024-05-08T23:20:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.672+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc1aa5f0 con 0x7f67cc0ffd90 2024-05-08T23:20:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.676+0000 7f67c2a00700 1 -- 172.21.15.46:0/2522101416 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f67bc0225e0 con 0x7f67cc0ffd90 2024-05-08T23:20:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 Processor -- start 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 -- start start 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563300329150 con 0x5633003d8720 2024-05-08T23:20:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563300326b10 con 0x5633003d2890 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67c2a00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 0x7f67b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67e4797b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563300322400 con 0x5633003d7e00 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.677+0000 7f67ca800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59098/0 (socket says 172.21.15.46:59098) 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 -- 172.21.15.46:0/1907321960 learned_addr learned my addr 172.21.15.46:0/1907321960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67c2a00700 1 -- 172.21.15.46:0/2522101416 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67bc0ce2b0 con 0x7f67cc0ffd90 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 0x7f67b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67c9400700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67c9e00700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 -- 172.21.15.46:0/1907321960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 msgr2=0x5633003d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 -- 172.21.15.46:0/1907321960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 msgr2=0x5633003d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67ca800700 1 -- 172.21.15.46:0/1907321960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633001a33b0 con 0x5633003d7e00 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.678+0000 7f67c9400700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.679+0000 7f67ca800700 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f67c4008f70 tx=0x7f67c401a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899ba745ed9984ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.679+0000 7f67ca800700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 0x7f67b00a37d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f67c401aa10 tx=0x7f67c400e000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.679+0000 7f67b7000700 1 -- 172.21.15.46:0/1907321960 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4027070 con 0x5633003d7e00 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.679+0000 7f67b7000700 1 -- 172.21.15.46:0/1907321960 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c401f070 con 0x5633003d7e00 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.679+0000 7f67b7000700 1 -- 172.21.15.46:0/1907321960 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400e070 con 0x5633003d7e00 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 -- 172.21.15.46:0/1907321960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 msgr2=0x5633003d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f67c4008f70 tx=0x7f67c401a6c0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 -- 172.21.15.46:0/1907321960 shutdown_connections 2024-05-08T23:20:34.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003d81e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 --2- 172.21.15.46:0/1907321960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 -- 172.21.15.46:0/1907321960 >> 172.21.15.46:0/1907321960 conn(0x563300330210 msgr2=0x5633003ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 -- 172.21.15.46:0/1907321960 shutdown_connections 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.680+0000 7f67e4797b80 1 -- 172.21.15.46:0/1907321960 wait complete. 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.681+0000 7f67e4797b80 1 -- start start 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.681+0000 7f67e4797b80 1 Processor -- start 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 0x5633003dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633002be560 con 0x5633003d8720 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633002be200 con 0x5633003d7e00 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633001a56a0 con 0x5633003d2890 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67e4797b80 10 client.0 _ll_register_callbacks cb 0x5633002c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59112/0 (socket says 172.21.15.46:59112) 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 -- 172.21.15.46:0/3348805456 learned_addr learned my addr 172.21.15.46:0/3348805456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67ca800700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9400700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 0x5633003dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 -- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 msgr2=0x5633003dc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 0x5633003dc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 -- 172.21.15.46:0/3348805456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 msgr2=0x5633003dc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003dc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9e00700 1 -- 172.21.15.46:0/3348805456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b80098b0 con 0x5633003d2890 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.682+0000 7f67c9400700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 0x5633003dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67c9e00700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0cbbf0 tx=0x7f67bc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e65da816843dbf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc09aca0 con 0x5633003d2890 2024-05-08T23:20:34.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67bc09b860 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56330027b270 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67bc09aac0 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 10 client.30109 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f67c4008b50 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc0b7020 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67bc09b3c0 con 0x5633003d2890 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 10 client.30109 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 10 client.30109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.683+0000 7f67e4797b80 20 client.30109 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.684+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5633003ebe50 0x5633003ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.684+0000 7f67e4797b80 4 client.30109 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.684+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633003a1810 con 0x5633003ebe50 2024-05-08T23:20:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.684+0000 7f67c9400700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5633003ebe50 0x5633003ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.685+0000 7f67c9400700 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5633003ebe50 0x5633003ee270 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7995b4fac0e5ced server_cookie=e9428982f99d8a45 in_seq=0 out_seq=0 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.685+0000 7f67ab400700 10 client.30109 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.685+0000 7f67ab400700 1 -- 172.21.15.46:0/3348805456 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633003a1810 con 0x5633003ebe50 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.685+0000 7f67ab400700 10 client.30109 handle_command_reply: tid=0 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.685+0000 7f67e4797b80 1 client.30109 shutdown 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 msgr2=0x5633003dbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0cbbf0 tx=0x7f67bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 shutdown_connections 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5633003ebe50 0x5633003ee270 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x5633003dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d2890 0x5633003dbb70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67ab400700 0 client.30109 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3348805456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d7e00 0x5633003dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 >> 172.21.15.46:0/3348805456 conn(0x563300330210 msgr2=0x5633003cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 shutdown_connections 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 1 -- 172.21.15.46:0/3348805456 wait complete. 2024-05-08T23:20:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.686+0000 7f67e4797b80 20 client.30109 trim_cache size 0 max 16384 2024-05-08T23:20:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.707+0000 7f67e4797b80 1 Processor -- start 2024-05-08T23:20:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.707+0000 7f67e4797b80 1 -- start start 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d2890 0x5633003ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d8720 0x5633003d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633003c5040 con 0x5633003d2890 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633003d26d0 con 0x5633003d8720 2024-05-08T23:20:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67e4797b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633003f3480 con 0x5633003d7e00 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59114/0 (socket says 172.21.15.46:59114) 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67c9400700 1 -- 172.21.15.46:0/2657954748 learned_addr learned my addr 172.21.15.46:0/2657954748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.708+0000 7f67ca800700 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d8720 0x5633003d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9e00700 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d2890 0x5633003ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 -- 172.21.15.46:0/2657954748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d8720 msgr2=0x5633003d6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d8720 0x5633003d6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 -- 172.21.15.46:0/2657954748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d2890 msgr2=0x5633003ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d2890 0x5633003ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 -- 172.21.15.46:0/2657954748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330014fb60 con 0x5633003d7e00 2024-05-08T23:20:34.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67c9400700 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x5633003eeb80 tx=0x7f67b8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82170baae5a3efc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.709+0000 7f67b7a00700 1 -- 172.21.15.46:0/2657954748 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8003ca0 con 0x5633003d7e00 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67b7a00700 1 -- 172.21.15.46:0/2657954748 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b8017040 con 0x5633003d7e00 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67b7a00700 1 -- 172.21.15.46:0/2657954748 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80055e0 con 0x5633003d7e00 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 -- 172.21.15.46:0/2657954748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 msgr2=0x5633003dc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x5633003eeb80 tx=0x7f67b8009260 comp rx=0 tx=0).stop 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 -- 172.21.15.46:0/2657954748 shutdown_connections 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d8720 0x5633003d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d2890 0x5633003ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 --2- 172.21.15.46:0/2657954748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003dc410 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 -- 172.21.15.46:0/2657954748 >> 172.21.15.46:0/2657954748 conn(0x563300330210 msgr2=0x5633003da460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.710+0000 7f67e4797b80 1 -- 172.21.15.46:0/2657954748 shutdown_connections 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.711+0000 7f67e4797b80 1 -- 172.21.15.46:0/2657954748 wait complete. 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.711+0000 7f67e4797b80 1 -- start start 2024-05-08T23:20:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.711+0000 7f67e4797b80 1 Processor -- start 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x563300441350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633003e1eb0 con 0x5633003d8720 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563300441890 con 0x5633003d2890 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633004419d0 con 0x5633003d7e00 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67e4797b80 10 client.0 _ll_register_callbacks cb 0x5633002c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:59124/0 (socket says 172.21.15.46:59124) 2024-05-08T23:20:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 -- 172.21.15.46:0/3024255088 learned_addr learned my addr 172.21.15.46:0/3024255088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67ca800700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x563300441350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 -- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 msgr2=0x5633003e10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003e10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 -- 172.21.15.46:0/3024255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 msgr2=0x563300441350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x563300441350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.712+0000 7f67c9400700 1 -- 172.21.15.46:0/3024255088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b8009d80 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67c9400700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000c00 tx=0x7f67b8003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c834dec0ed4fb2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67ab400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80053c0 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67b8005520 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67c401f070 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b80056e0 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67e4797b80 10 client.30115 fetch_fsmap learned FSMap version 36 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.713+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5633002e8560 con 0x5633003d7e00 2024-05-08T23:20:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c401f070 con 0x5633003d7e00 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67b801e470 con 0x5633003d7e00 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 10 client.30115 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 10 client.30115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 20 client.30115 populate_metadata read hostname 'smithi046' 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563300443070 0x563300445490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 4 client.30115 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56330031aec0 con 0x563300443070 2024-05-08T23:20:34.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.714+0000 7f67c9e00700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563300443070 0x563300445490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.715+0000 7f67c9e00700 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563300443070 0x563300445490 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503e5a5c1a448b80 server_cookie=1ea269f8f72d6e0d in_seq=0 out_seq=0 2024-05-08T23:20:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.715+0000 7f67ab400700 10 client.30115 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.715+0000 7f67ab400700 1 -- 172.21.15.46:0/3024255088 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56330031aec0 con 0x563300443070 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.715+0000 7f67ab400700 10 client.30115 handle_command_reply: tid=0 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.715+0000 7f67e4797b80 1 client.30115 shutdown 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 msgr2=0x5633003e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000c00 tx=0x7f67b8003b30 comp rx=0 tx=0).stop 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 shutdown_connections 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x563300443070 0x563300445490 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633003d8720 0x563300441350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633003d2890 0x5633003e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 --2- 172.21.15.46:0/3024255088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633003d7e00 0x5633003e1630 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 >> 172.21.15.46:0/3024255088 conn(0x563300330210 msgr2=0x5633003d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67ab400700 0 client.30115 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 shutdown_connections 2024-05-08T23:20:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 1 -- 172.21.15.46:0/3024255088 wait complete. 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.716+0000 7f67e4797b80 20 client.30115 trim_cache size 0 max 16384 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "24ff49d6-085b-4031-9ae0-c89c7682797f": { 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "24ff49d6-085b-4031-9ae0-c89c7682797f", 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:20:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 msgr2=0x7f67b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 0x7f67b00a37d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f67c401aa10 tx=0x7f67c400e000 comp rx=0 tx=0).stop 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 msgr2=0x7f67cc1a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005910 tx=0x7f67bc004730 comp rx=0 tx=0).stop 2024-05-08T23:20:34.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 shutdown_connections 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f67b00a1310 0x7f67b00a37d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67cc105190 0x7f67cc19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67cc0ffd90 0x7f67cc1a3600 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 --2- 172.21.15.46:0/2522101416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc1006b0 0x7f67cc1a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 >> 172.21.15.46:0/2522101416 conn(0x7f67cc0fb760 msgr2=0x7f67cc102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 shutdown_connections 2024-05-08T23:20:34.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:20:34.717+0000 7f67d0c00700 1 -- 172.21.15.46:0/2522101416 wait complete. 2024-05-08T23:20:34.738 INFO:tasks.cephfs.filesystem:scrub status for tag:24ff49d6-085b-4031-9ae0-c89c7682797f - {'path': '/', 'tag': '24ff49d6-085b-4031-9ae0-c89c7682797f', 'options': 'recursive,force'} 2024-05-08T23:20:35.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:34 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:35.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:34 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:35.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:34 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:35.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:34 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:35.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:34 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:35.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:34 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:35.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:34 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:35.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:34 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:35.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:34 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:35.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:34 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:35.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 69211 19.23 MB/sec execute 56 sec latency 201.107 ms 2024-05-08T23:20:35.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:34 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:34 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:34 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:34 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:34 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: pgmap v526: 129 pgs: 129 active+clean; 941 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:20:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:35 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 69752 19.37 MB/sec execute 57 sec latency 203.491 ms 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: pgmap v526: 129 pgs: 129 active+clean; 941 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:35 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: pgmap v526: 129 pgs: 129 active+clean; 941 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 586 op/s 2024-05-08T23:20:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:36.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:35 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:37.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 70331 19.68 MB/sec execute 58 sec latency 70.843 ms 2024-05-08T23:20:38.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 70884 19.85 MB/sec execute 59 sec latency 122.151 ms 2024-05-08T23:20:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:37 smithi053 ceph-mon[77587]: pgmap v527: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T23:20:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:37 smithi046 ceph-mon[36898]: pgmap v527: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T23:20:38.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:37 smithi086 ceph-mon[78610]: pgmap v527: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 704 op/s 2024-05-08T23:20:39.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 71265 19.96 MB/sec execute 60 sec latency 213.501 ms 2024-05-08T23:20:40.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 71599 19.91 MB/sec execute 61 sec latency 159.353 ms 2024-05-08T23:20:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:39 smithi053 ceph-mon[77587]: pgmap v528: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 569 op/s 2024-05-08T23:20:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:39 smithi046 ceph-mon[36898]: pgmap v528: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 569 op/s 2024-05-08T23:20:40.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:39 smithi086 ceph-mon[78610]: pgmap v528: 129 pgs: 129 active+clean; 946 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 569 op/s 2024-05-08T23:20:41.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 71978 19.77 MB/sec execute 62 sec latency 159.871 ms 2024-05-08T23:20:42.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 72431 19.72 MB/sec execute 63 sec latency 167.240 ms 2024-05-08T23:20:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:41 smithi053 ceph-mon[77587]: pgmap v529: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-08T23:20:42.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:41 smithi046 ceph-mon[36898]: pgmap v529: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-08T23:20:42.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:41 smithi086 ceph-mon[78610]: pgmap v529: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-05-08T23:20:43.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 72989 20.00 MB/sec execute 64 sec latency 177.765 ms 2024-05-08T23:20:44.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 73594 20.14 MB/sec execute 65 sec latency 229.066 ms 2024-05-08T23:20:44.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:43 smithi053 ceph-mon[77587]: pgmap v530: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-05-08T23:20:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:43 smithi046 ceph-mon[36898]: pgmap v530: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-05-08T23:20:44.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:43 smithi086 ceph-mon[78610]: pgmap v530: 129 pgs: 129 active+clean; 963 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-05-08T23:20:45.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 74140 20.41 MB/sec execute 66 sec latency 156.776 ms 2024-05-08T23:20:46.261 INFO:tasks.workunit.client.0.smithi046.stdout: 10 74658 20.54 MB/sec execute 67 sec latency 159.397 ms 2024-05-08T23:20:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:45 smithi053 ceph-mon[77587]: pgmap v531: 129 pgs: 129 active+clean; 965 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 520 op/s 2024-05-08T23:20:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:45 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:20:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:45 smithi046 ceph-mon[36898]: pgmap v531: 129 pgs: 129 active+clean; 965 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 520 op/s 2024-05-08T23:20:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:45 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:20:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:45 smithi086 ceph-mon[78610]: pgmap v531: 129 pgs: 129 active+clean; 965 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.3 MiB/s wr, 520 op/s 2024-05-08T23:20:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:46.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:45 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:20:47.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 75013 20.46 MB/sec execute 68 sec latency 198.469 ms 2024-05-08T23:20:48.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 75425 20.58 MB/sec execute 69 sec latency 56.394 ms 2024-05-08T23:20:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:47 smithi053 ceph-mon[77587]: pgmap v532: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-05-08T23:20:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:47 smithi046 ceph-mon[36898]: pgmap v532: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-05-08T23:20:48.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:47 smithi086 ceph-mon[78610]: pgmap v532: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-05-08T23:20:49.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 75868 20.50 MB/sec execute 70 sec latency 72.536 ms 2024-05-08T23:20:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:49 smithi053 ceph-mon[77587]: pgmap v533: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-05-08T23:20:50.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 76394 20.61 MB/sec execute 71 sec latency 167.374 ms 2024-05-08T23:20:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:49 smithi046 ceph-mon[36898]: pgmap v533: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-05-08T23:20:50.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:49 smithi086 ceph-mon[78610]: pgmap v533: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-05-08T23:20:51.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 76990 20.72 MB/sec execute 72 sec latency 178.454 ms 2024-05-08T23:20:52.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:51 smithi053 ceph-mon[77587]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 487 op/s 2024-05-08T23:20:52.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 77472 20.84 MB/sec execute 73 sec latency 193.470 ms 2024-05-08T23:20:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:51 smithi046 ceph-mon[36898]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 487 op/s 2024-05-08T23:20:52.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:51 smithi086 ceph-mon[78610]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 487 op/s 2024-05-08T23:20:53.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 77974 21.03 MB/sec execute 74 sec latency 185.442 ms 2024-05-08T23:20:54.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:53 smithi053 ceph-mon[77587]: pgmap v535: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-05-08T23:20:54.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 78366 21.05 MB/sec execute 75 sec latency 199.912 ms 2024-05-08T23:20:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:53 smithi046 ceph-mon[36898]: pgmap v535: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-05-08T23:20:54.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:53 smithi086 ceph-mon[78610]: pgmap v535: 129 pgs: 129 active+clean; 971 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-05-08T23:20:55.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 78678 20.98 MB/sec execute 76 sec latency 109.675 ms 2024-05-08T23:20:56.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:55 smithi053 ceph-mon[77587]: pgmap v536: 129 pgs: 129 active+clean; 979 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 377 op/s 2024-05-08T23:20:56.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 79077 20.91 MB/sec execute 77 sec latency 66.895 ms 2024-05-08T23:20:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:55 smithi046 ceph-mon[36898]: pgmap v536: 129 pgs: 129 active+clean; 979 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 377 op/s 2024-05-08T23:20:56.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:55 smithi086 ceph-mon[78610]: pgmap v536: 129 pgs: 129 active+clean; 979 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 377 op/s 2024-05-08T23:20:57.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 79352 20.81 MB/sec execute 78 sec latency 234.060 ms 2024-05-08T23:20:57.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:57 smithi086 ceph-mon[78610]: pgmap v537: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.3 MiB/s wr, 417 op/s 2024-05-08T23:20:57.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:57 smithi053 ceph-mon[77587]: pgmap v537: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.3 MiB/s wr, 417 op/s 2024-05-08T23:20:57.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:57 smithi046 ceph-mon[36898]: pgmap v537: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.3 MiB/s wr, 417 op/s 2024-05-08T23:20:58.262 INFO:tasks.workunit.client.0.smithi046.stdout: 10 79487 20.59 MB/sec execute 79 sec latency 487.516 ms 2024-05-08T23:20:59.263 INFO:tasks.workunit.client.0.smithi046.stdout: 10 79789 20.51 MB/sec execute 80 sec latency 199.451 ms 2024-05-08T23:20:59.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:20:59 smithi086 ceph-mon[78610]: pgmap v538: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-05-08T23:20:59.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:20:59 smithi053 ceph-mon[77587]: pgmap v538: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-05-08T23:20:59.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:20:59 smithi046 ceph-mon[36898]: pgmap v538: 129 pgs: 129 active+clean; 977 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-05-08T23:21:00.263 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80244 20.59 MB/sec execute 81 sec latency 153.473 ms 2024-05-08T23:21:01.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80712 20.65 MB/sec execute 82 sec latency 137.751 ms 2024-05-08T23:21:01.713 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:01 smithi086 ceph-mon[78610]: pgmap v539: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 531 op/s 2024-05-08T23:21:01.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:01 smithi053 ceph-mon[77587]: pgmap v539: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 531 op/s 2024-05-08T23:21:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:01 smithi046 ceph-mon[36898]: pgmap v539: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 531 op/s 2024-05-08T23:21:02.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80712 20.40 MB/sec execute 83 sec latency 1133.394 ms 2024-05-08T23:21:03.265 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80712 20.16 MB/sec execute 84 sec latency 2133.477 ms 2024-05-08T23:21:03.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:03 smithi086 ceph-mon[78610]: pgmap v540: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.5 MiB/s wr, 385 op/s 2024-05-08T23:21:03.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:03 smithi053 ceph-mon[77587]: pgmap v540: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.5 MiB/s wr, 385 op/s 2024-05-08T23:21:03.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:03 smithi046 ceph-mon[36898]: pgmap v540: 129 pgs: 129 active+clean; 990 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.5 MiB/s wr, 385 op/s 2024-05-08T23:21:04.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80811 19.98 MB/sec execute 85 sec latency 2537.721 ms 2024-05-08T23:21:04.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:05.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.110+0000 7f2ddd400700 1 -- 172.21.15.46:0/3407943114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd8101a30 msgr2=0x7f2dd8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.110+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3407943114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd8101a30 0x7f2dd8105a70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00098f0 tx=0x7f2dd0009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 -- 172.21.15.46:0/3407943114 shutdown_connections 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3407943114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd81061a0 0x7f2dd810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3407943114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd8101080 0x7f2dd8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3407943114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd8101a30 0x7f2dd8105a70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 -- 172.21.15.46:0/3407943114 >> 172.21.15.46:0/3407943114 conn(0x7f2dd80fc8e0 msgr2=0x7f2dd80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 -- 172.21.15.46:0/3407943114 shutdown_connections 2024-05-08T23:21:05.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.111+0000 7f2ddd400700 1 -- 172.21.15.46:0/3407943114 wait complete. 2024-05-08T23:21:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.114+0000 7f2ddd400700 1 Processor -- start 2024-05-08T23:21:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.114+0000 7f2ddd400700 1 -- start start 2024-05-08T23:21:05.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd81a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 0x7f2dd819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819e1d0 con 0x7f2dd8101080 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819e310 con 0x7f2dd819d750 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.116+0000 7f2ddd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819e4d0 con 0x7f2dd81061a0 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.117+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 0x7f2dd819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.117+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 0x7f2dd819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37700/0 (socket says 172.21.15.46:37700) 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.117+0000 7f2dd7200700 1 -- 172.21.15.46:0/3026875808 learned_addr learned my addr 172.21.15.46:0/3026875808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.117+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.117+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd81a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 -- 172.21.15.46:0/3026875808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 msgr2=0x7f2dd819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 0x7f2dd819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 -- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 msgr2=0x7f2dd81a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd81a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 -- 172.21.15.46:0/3026875808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0009520 con 0x7f2dd81061a0 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd81a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.118+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd81a3b60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000b2e0 tx=0x7f2dd000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.119+0000 7f2dcf400700 1 -- 172.21.15.46:0/3026875808 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd001c070 con 0x7f2dd81061a0 2024-05-08T23:21:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.119+0000 7f2dcf400700 1 -- 172.21.15.46:0/3026875808 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd0005530 con 0x7f2dd81061a0 2024-05-08T23:21:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.119+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd81941f0 con 0x7f2dd81061a0 2024-05-08T23:21:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.119+0000 7f2dcf400700 1 -- 172.21.15.46:0/3026875808 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003900 con 0x7f2dd81061a0 2024-05-08T23:21:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.119+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd8194720 con 0x7f2dd81061a0 2024-05-08T23:21:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.129+0000 7f2dcf400700 1 -- 172.21.15.46:0/3026875808 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2dd0020070 con 0x7f2dd81061a0 2024-05-08T23:21:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.131+0000 7f2dcf400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 0x7f2dbc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.131+0000 7f2dcf400700 1 -- 172.21.15.46:0/3026875808 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dd00ce360 con 0x7f2dd81061a0 2024-05-08T23:21:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.132+0000 7f2df0febb80 1 Processor -- start 2024-05-08T23:21:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.132+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 0x7f2dbc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.132+0000 7f2df0febb80 1 -- start start 2024-05-08T23:21:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94874720 0x55ef9486e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef9486ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef947c5150 con 0x55ef94874720 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef947c2b10 con 0x55ef9486e890 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2df0febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef947be400 con 0x55ef94873e00 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2dd6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2dd6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38702/0 (socket says 172.21.15.46:38702) 2024-05-08T23:21:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.133+0000 7f2dd6800700 1 -- 172.21.15.46:0/250319458 learned_addr learned my addr 172.21.15.46:0/250319458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94874720 0x55ef9486e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd6800700 1 -- 172.21.15.46:0/250319458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 msgr2=0x55ef9486ecd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd6800700 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef9486ecd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd6800700 1 -- 172.21.15.46:0/250319458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94874720 msgr2=0x55ef9486e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd6800700 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94874720 0x55ef9486e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.134+0000 7f2dd6800700 1 -- 172.21.15.46:0/250319458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef9463f3b0 con 0x55ef94873e00 2024-05-08T23:21:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.135+0000 7f2dd6800700 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802a750 tx=0x7f2dc802aa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc230b3cf10ee6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.137+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 0x7f2dbc0a3890 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8025000 tx=0x7f2dc8024000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.138+0000 7f2dc7400700 1 -- 172.21.15.46:0/250319458 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8006040 con 0x55ef94873e00 2024-05-08T23:21:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.139+0000 7f2dc7400700 1 -- 172.21.15.46:0/250319458 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc802bde0 con 0x55ef94873e00 2024-05-08T23:21:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.139+0000 7f2dc7400700 1 -- 172.21.15.46:0/250319458 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8005860 con 0x55ef94873e00 2024-05-08T23:21:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.139+0000 7f2df0febb80 1 -- 172.21.15.46:0/250319458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 msgr2=0x55ef948741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.139+0000 7f2df0febb80 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802a750 tx=0x7f2dc802aa60 comp rx=0 tx=0).stop 2024-05-08T23:21:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 -- 172.21.15.46:0/250319458 shutdown_connections 2024-05-08T23:21:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef9486ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94873e00 0x55ef948741e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 --2- 172.21.15.46:0/250319458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94874720 0x55ef9486e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 -- 172.21.15.46:0/250319458 >> 172.21.15.46:0/250319458 conn(0x55ef947cc210 msgr2=0x55ef94868df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.141+0000 7f2df0febb80 1 -- 172.21.15.46:0/250319458 shutdown_connections 2024-05-08T23:21:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.142+0000 7f2df0febb80 1 -- 172.21.15.46:0/250319458 wait complete. 2024-05-08T23:21:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.145+0000 7f2df0febb80 1 -- start start 2024-05-08T23:21:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.146+0000 7f2df0febb80 1 Processor -- start 2024-05-08T23:21:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef94877a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94877fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9475a560 con 0x55ef94873e00 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9475a200 con 0x55ef9486e890 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef946416a0 con 0x55ef94874720 2024-05-08T23:21:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.147+0000 7f2df0febb80 10 client.0 _ll_register_callbacks cb 0x55ef94763b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef94877a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38708/0 (socket says 172.21.15.46:38708) 2024-05-08T23:21:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 -- 172.21.15.46:0/2534340284 learned_addr learned my addr 172.21.15.46:0/2534340284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94877fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 -- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 msgr2=0x55ef94877a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef94877a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 -- 172.21.15.46:0/2534340284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 msgr2=0x55ef94877fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94877fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.148+0000 7f2dd7200700 1 -- 172.21.15.46:0/2534340284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc802a380 con 0x55ef94874720 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.149+0000 7f2dd7200700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b8b0 tx=0x7f2dc000bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93dbffbfd17990b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.149+0000 7f2dbbe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.149+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0012040 con 0x55ef94874720 2024-05-08T23:21:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.149+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef94717270 con 0x55ef94874720 2024-05-08T23:21:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.150+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dc0016800 con 0x55ef94874720 2024-05-08T23:21:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc0012040 con 0x55ef94874720 2024-05-08T23:21:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 10 client.30127 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ef945ebb60 con 0x55ef94874720 2024-05-08T23:21:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef94717270 con 0x55ef94874720 2024-05-08T23:21:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dc0015070 con 0x55ef94874720 2024-05-08T23:21:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 10 client.30127 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 10 client.30127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 20 client.30127 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef94888220 0x55ef9488a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 4 client.30127 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.151+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef9483d810 con 0x55ef94888220 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.152+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef94888220 0x55ef9488a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.153+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef94888220 0x55ef9488a640 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf567ac58fd5a0b1 server_cookie=7f5fbb86cea414ec in_seq=0 out_seq=0 2024-05-08T23:21:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.153+0000 7f2dbbe00700 10 client.30127 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.154+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2534340284 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef9483d810 con 0x55ef94888220 2024-05-08T23:21:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.154+0000 7f2dbbe00700 10 client.30127 handle_command_reply: tid=0 2024-05-08T23:21:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.154+0000 7f2df0febb80 1 client.30127 shutdown 2024-05-08T23:21:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 msgr2=0x55ef948784f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b8b0 tx=0x7f2dc000bbc0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 shutdown_connections 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef948784f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef9486e890 0x55ef94877a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef94888220 0x55ef9488a640 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2534340284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94877fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 >> 172.21.15.46:0/2534340284 conn(0x55ef947cc210 msgr2=0x55ef948685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.157+0000 7f2dbbe00700 0 client.30127 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.158+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 shutdown_connections 2024-05-08T23:21:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.158+0000 7f2df0febb80 1 -- 172.21.15.46:0/2534340284 wait complete. 2024-05-08T23:21:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.158+0000 7f2df0febb80 20 client.30127 trim_cache size 0 max 16384 2024-05-08T23:21:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.197+0000 7f2df0febb80 1 Processor -- start 2024-05-08T23:21:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.197+0000 7f2df0febb80 1 -- start start 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 0x55ef9486e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94868d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 0x55ef94877fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9488f810 con 0x55ef94873e00 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef94863a90 con 0x55ef94874720 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2df0febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef94863800 con 0x55ef9486e890 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2dd5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94868d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 0x55ef94877fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2dd7200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 0x55ef94877fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37724/0 (socket says 172.21.15.46:37724) 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.198+0000 7f2dd7200700 1 -- 172.21.15.46:0/3289973880 learned_addr learned my addr 172.21.15.46:0/3289973880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.199+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 0x55ef9486e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.199+0000 7f2dd6800700 1 -- 172.21.15.46:0/3289973880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 msgr2=0x55ef94877fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.199+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 0x55ef94877fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.199+0000 7f2dd6800700 1 -- 172.21.15.46:0/3289973880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 msgr2=0x55ef94868d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.200+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94868d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.200+0000 7f2dd6800700 1 -- 172.21.15.46:0/3289973880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef94784560 con 0x55ef9486e890 2024-05-08T23:21:05.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.201+0000 7f2dd6800700 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 0x55ef9486e350 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80091f0 tx=0x7f2dc802d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c11bc41d079722 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.202+0000 7f2dc7e00700 1 -- 172.21.15.46:0/3289973880 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8043070 con 0x55ef9486e890 2024-05-08T23:21:05.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.202+0000 7f2dc7e00700 1 -- 172.21.15.46:0/3289973880 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc802ae90 con 0x55ef9486e890 2024-05-08T23:21:05.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.202+0000 7f2dc7e00700 1 -- 172.21.15.46:0/3289973880 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8005600 con 0x55ef9486e890 2024-05-08T23:21:05.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.203+0000 7f2df0febb80 1 -- 172.21.15.46:0/3289973880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 msgr2=0x55ef9486e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.203+0000 7f2df0febb80 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 0x55ef9486e350 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80091f0 tx=0x7f2dc802d040 comp rx=0 tx=0).stop 2024-05-08T23:21:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 -- 172.21.15.46:0/3289973880 shutdown_connections 2024-05-08T23:21:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94874720 0x55ef94877fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef9486e890 0x55ef9486e350 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 --2- 172.21.15.46:0/3289973880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef94873e00 0x55ef94868d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 -- 172.21.15.46:0/3289973880 >> 172.21.15.46:0/3289973880 conn(0x55ef947cc210 msgr2=0x55ef94876340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 -- 172.21.15.46:0/3289973880 shutdown_connections 2024-05-08T23:21:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.204+0000 7f2df0febb80 1 -- 172.21.15.46:0/3289973880 wait complete. 2024-05-08T23:21:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.205+0000 7f2df0febb80 1 -- start start 2024-05-08T23:21:05.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.206+0000 7f2df0febb80 1 Processor -- start 2024-05-08T23:21:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.208+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 0x55ef9487f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.208+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 0x55ef9487d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2df0febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef9487dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2dd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 0x55ef9487d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2dd5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 0x55ef9487d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37732/0 (socket says 172.21.15.46:37732) 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2df0febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9487e1d0 con 0x55ef9486e890 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2df0febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9487e340 con 0x55ef94873e00 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2df0febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9487e500 con 0x55ef94874720 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2df0febb80 10 client.0 _ll_register_callbacks cb 0x55ef94763b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.209+0000 7f2dd5e00700 1 -- 172.21.15.46:0/2297458907 learned_addr learned my addr 172.21.15.46:0/2297458907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 0x55ef9487f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 -- 172.21.15.46:0/2297458907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 msgr2=0x55ef9487dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef9487dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 -- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 msgr2=0x55ef9487d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 0x55ef9487d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.210+0000 7f2dd6800700 1 -- 172.21.15.46:0/2297458907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef9463f3b0 con 0x55ef9486e890 2024-05-08T23:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.211+0000 7f2dd6800700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 0x55ef9487f2b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005400 tx=0x7f2dc8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6138a1ce952d1d29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.211+0000 7f2dbbe00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.211+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8043040 con 0x55ef9486e890 2024-05-08T23:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.212+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dc8005190 con 0x55ef9486e890 2024-05-08T23:21:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.212+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dc802bde0 con 0x55ef9486e890 2024-05-08T23:21:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.212+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc8004ca0 con 0x55ef9486e890 2024-05-08T23:21:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.212+0000 7f2df0febb80 10 client.19905 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.212+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ef94643cd0 con 0x55ef9486e890 2024-05-08T23:21:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.213+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc802bde0 con 0x55ef9486e890 2024-05-08T23:21:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.213+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dc8048740 con 0x55ef9486e890 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 10 client.19905 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 10 client.19905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 20 client.19905 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef948df820 0x55ef948e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 4 client.19905 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.214+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef947b6ec0 con 0x55ef948df820 2024-05-08T23:21:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.215+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef948df820 0x55ef948e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.217+0000 7f2dd5e00700 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef948df820 0x55ef948e1c40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64aa8df85ff7736 server_cookie=9af6d139186cdb9 in_seq=0 out_seq=0 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.217+0000 7f2dbbe00700 10 client.19905 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.217+0000 7f2dbbe00700 1 -- 172.21.15.46:0/2297458907 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ef947b6ec0 con 0x55ef948df820 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.217+0000 7f2dbbe00700 10 client.19905 handle_command_reply: tid=0 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.217+0000 7f2df0febb80 1 client.19905 shutdown 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 msgr2=0x55ef9487f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 0x55ef9487f2b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005400 tx=0x7f2dc8007a50 comp rx=0 tx=0).stop 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 shutdown_connections 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef94874720 0x55ef9487dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55ef948df820 0x55ef948e1c40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ef9486e890 0x55ef9487f2b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 --2- 172.21.15.46:0/2297458907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef94873e00 0x55ef9487d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.218+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 >> 172.21.15.46:0/2297458907 conn(0x55ef947cc210 msgr2=0x55ef94879310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.219+0000 7f2dbbe00700 0 client.19905 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:05.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.219+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 shutdown_connections 2024-05-08T23:21:05.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.219+0000 7f2df0febb80 1 -- 172.21.15.46:0/2297458907 wait complete. 2024-05-08T23:21:05.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.220+0000 7f2df0febb80 20 client.19905 trim_cache size 0 max 16384 2024-05-08T23:21:05.220 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:21:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:21:05.220 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 msgr2=0x7f2dbc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 0x7f2dbc0a3890 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8025000 tx=0x7f2dc8024000 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 msgr2=0x7f2dd81a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd81a3b60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000b2e0 tx=0x7f2dd000fa70 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 shutdown_connections 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd819d750 0x7f2dd819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2dbc0a13d0 0x7f2dbc0a3890 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dd8101080 0x7f2dd81a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 --2- 172.21.15.46:0/3026875808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81061a0 0x7f2dd81a3b60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 >> 172.21.15.46:0/3026875808 conn(0x7f2dd80fc8e0 msgr2=0x7f2dd80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 shutdown_connections 2024-05-08T23:21:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.221+0000 7f2ddd400700 1 -- 172.21.15.46:0/3026875808 wait complete. 2024-05-08T23:21:05.258 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:21:05.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:05.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81056 19.92 MB/sec execute 86 sec latency 441.644 ms 2024-05-08T23:21:05.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:05 smithi046 ceph-mon[36898]: pgmap v541: 129 pgs: 129 active+clean; 1004 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-05-08T23:21:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 -- 172.21.15.46:0/2291080391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b780fff60 msgr2=0x7f6b78100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/2291080391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b780fff60 0x7f6b78100340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009990 tx=0x7f6b68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 -- 172.21.15.46:0/2291080391 shutdown_connections 2024-05-08T23:21:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/2291080391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b78105080 0x7f6b78068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/2291080391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b780fff60 0x7f6b78100340 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/2291080391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78100910 0x7f6b78104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 -- 172.21.15.46:0/2291080391 >> 172.21.15.46:0/2291080391 conn(0x7f6b780fb820 msgr2=0x7f6b780fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 -- 172.21.15.46:0/2291080391 shutdown_connections 2024-05-08T23:21:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.619+0000 7f6b7cc00700 1 -- 172.21.15.46:0/2291080391 wait complete. 2024-05-08T23:21:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 Processor -- start 2024-05-08T23:21:05.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 -- start start 2024-05-08T23:21:05.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b780fff60 0x7f6b78115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7811adf0 0x7f6b7811b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78110220 con 0x7f6b7811adf0 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78110390 con 0x7f6b78105080 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b7cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78110550 con 0x7f6b780fff60 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b75400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.621+0000 7f6b75400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37748/0 (socket says 172.21.15.46:37748) 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 -- 172.21.15.46:0/1561307477 learned_addr learned my addr 172.21.15.46:0/1561307477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b76800700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7811adf0 0x7f6b7811b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75e00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b780fff60 0x7f6b78115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 -- 172.21.15.46:0/1561307477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b780fff60 msgr2=0x7f6b78115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b780fff60 0x7f6b78115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 -- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7811adf0 msgr2=0x7f6b7811b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7811adf0 0x7f6b7811b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.622+0000 7f6b75400700 1 -- 172.21.15.46:0/1561307477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680095f0 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b75400700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c9e0 tx=0x7f6b6400ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007aa0 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b6400ce80 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64018660 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78110830 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.623+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78110cf0 con 0x7f6b78105080 2024-05-08T23:21:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.624+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78101430 con 0x7f6b78105080 2024-05-08T23:21:05.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.627+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6b6400f450 con 0x7f6b78105080 2024-05-08T23:21:05.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.629+0000 7f6b6ea00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 0x7f6b5c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:05.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.629+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6b640cd290 con 0x7f6b78105080 2024-05-08T23:21:05.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.630+0000 7f6b75e00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 0x7f6b5c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.631+0000 7f6b75e00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 0x7f6b5c0a37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006010 tx=0x7f6b6800b540 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:05.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:05.633+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b640963e0 con 0x7f6b78105080 2024-05-08T23:21:05.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:05 smithi053 ceph-mon[77587]: pgmap v541: 129 pgs: 129 active+clean; 1004 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-05-08T23:21:05.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:05 smithi086 ceph-mon[78610]: pgmap v541: 129 pgs: 129 active+clean; 1004 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-05-08T23:21:06.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.183+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b781116b0 con 0x7f6b78105080 2024-05-08T23:21:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.186+0000 7f6b6ea00700 1 -- 172.21.15.46:0/1561307477 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f6b64099a00 con 0x7f6b78105080 2024-05-08T23:21:06.186 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:21:06.186 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.191+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 msgr2=0x7f6b5c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.191+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 0x7f6b5c0a37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006010 tx=0x7f6b6800b540 comp rx=0 tx=0).stop 2024-05-08T23:21:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.191+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 msgr2=0x7f6b781155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.191+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c9e0 tx=0x7f6b6400ccf0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 shutdown_connections 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6b5c0a1310 0x7f6b5c0a37d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7811adf0 0x7f6b7811b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b780fff60 0x7f6b78115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 --2- 172.21.15.46:0/1561307477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b78105080 0x7f6b781155d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 >> 172.21.15.46:0/1561307477 conn(0x7f6b780fb820 msgr2=0x7f6b7810cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 shutdown_connections 2024-05-08T23:21:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.193+0000 7f6b7cc00700 1 -- 172.21.15.46:0/1561307477 wait complete. 2024-05-08T23:21:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:21:06.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:21:06.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81384 19.84 MB/sec execute 87 sec latency 456.554 ms 2024-05-08T23:21:06.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:06 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1561307477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 -- 172.21.15.46:0/873518395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8750105190 msgr2=0x7f8750105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 --2- 172.21.15.46:0/873518395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8750105190 0x7f8750105640 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f873c0098f0 tx=0x7f873c009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 -- 172.21.15.46:0/873518395 shutdown_connections 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 --2- 172.21.15.46:0/873518395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8750105190 0x7f8750105640 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 --2- 172.21.15.46:0/873518395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87500ffd90 0x7f8750100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 --2- 172.21.15.46:0/873518395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87501006b0 0x7f8750104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.597+0000 7f8758400700 1 -- 172.21.15.46:0/873518395 >> 172.21.15.46:0/873518395 conn(0x7f87500fb760 msgr2=0x7f87500fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.598+0000 7f8758400700 1 -- 172.21.15.46:0/873518395 shutdown_connections 2024-05-08T23:21:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.598+0000 7f8758400700 1 -- 172.21.15.46:0/873518395 wait complete. 2024-05-08T23:21:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.599+0000 7f8758400700 1 Processor -- start 2024-05-08T23:21:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.600+0000 7f8758400700 1 -- start start 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.600+0000 7f8758400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.600+0000 7f8758400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87501006b0 0x7f87501a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8758400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8750105190 0x7f875019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8758400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019dd70 con 0x7f87501006b0 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8758400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019dee0 con 0x7f87500ffd90 2024-05-08T23:21:06.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8758400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019e0a0 con 0x7f8750105190 2024-05-08T23:21:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37766/0 (socket says 172.21.15.46:37766) 2024-05-08T23:21:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 -- 172.21.15.46:0/3262535963 learned_addr learned my addr 172.21.15.46:0/3262535963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8756400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8750105190 0x7f875019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 -- 172.21.15.46:0/3262535963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8750105190 msgr2=0x7f875019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8750105190 0x7f875019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.601+0000 7f8755a00700 1 -- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87501006b0 msgr2=0x7f87501a3b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.602+0000 7f8755a00700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87501006b0 0x7f87501a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.602+0000 7f8755a00700 1 -- 172.21.15.46:0/3262535963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c009520 con 0x7f87500ffd90 2024-05-08T23:21:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.603+0000 7f8755a00700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f874800c0c0 tx=0x7f874800c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.604+0000 7f874e000700 1 -- 172.21.15.46:0/3262535963 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748018070 con 0x7f87500ffd90 2024-05-08T23:21:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.604+0000 7f874e000700 1 -- 172.21.15.46:0/3262535963 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f874800f040 con 0x7f87500ffd90 2024-05-08T23:21:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.604+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875019e320 con 0x7f87500ffd90 2024-05-08T23:21:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.604+0000 7f874e000700 1 -- 172.21.15.46:0/3262535963 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800e020 con 0x7f87500ffd90 2024-05-08T23:21:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.604+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87501aa5d0 con 0x7f87500ffd90 2024-05-08T23:21:06.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.613+0000 7f874e000700 1 -- 172.21.15.46:0/3262535963 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f87480075c0 con 0x7f87500ffd90 2024-05-08T23:21:06.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.614+0000 7f876bee3b80 1 Processor -- start 2024-05-08T23:21:06.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.614+0000 7f876bee3b80 1 -- start start 2024-05-08T23:21:06.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e72c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e726350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e67d150 con 0x562f0e726890 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e67ab10 con 0x562f0e72be00 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.615+0000 7f876bee3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e676400 con 0x562f0e72c720 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f874e000700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 0x7f87380a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57632/0 (socket says 172.21.15.46:57632) 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 -- 172.21.15.46:0/2802186249 learned_addr learned my addr 172.21.15.46:0/2802186249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f874e000700 1 -- 172.21.15.46:0/3262535963 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f87480cced0 con 0x7f87500ffd90 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8756400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 0x7f87380a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 -- 172.21.15.46:0/2802186249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 msgr2=0x562f0e726350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8756400700 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e72c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e726350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 -- 172.21.15.46:0/2802186249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 msgr2=0x562f0e72c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e72c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.616+0000 7f8755000700 1 -- 172.21.15.46:0/2802186249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f0e4f73b0 con 0x562f0e726890 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.617+0000 7f8755000700 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ae0 tx=0x7f8744007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b9c95dadc218d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.617+0000 7f8756400700 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e72c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.617+0000 7f8742a00700 1 -- 172.21.15.46:0/2802186249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400d890 con 0x562f0e726890 2024-05-08T23:21:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.618+0000 7f8742a00700 1 -- 172.21.15.46:0/2802186249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8744004830 con 0x562f0e726890 2024-05-08T23:21:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.618+0000 7f8742a00700 1 -- 172.21.15.46:0/2802186249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744005260 con 0x562f0e726890 2024-05-08T23:21:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.619+0000 7f876bee3b80 1 -- 172.21.15.46:0/2802186249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 msgr2=0x562f0e726cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.619+0000 7f876bee3b80 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ae0 tx=0x7f8744007df0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.620+0000 7f876bee3b80 1 -- 172.21.15.46:0/2802186249 shutdown_connections 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.620+0000 7f876bee3b80 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726cd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.620+0000 7f876bee3b80 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e72c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.620+0000 7f876bee3b80 1 --2- 172.21.15.46:0/2802186249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e726350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.620+0000 7f876bee3b80 1 -- 172.21.15.46:0/2802186249 >> 172.21.15.46:0/2802186249 conn(0x562f0e684210 msgr2=0x562f0e720df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.621+0000 7f8756400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 0x7f87380a3920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f873c005e90 tx=0x7f873c005b40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.621+0000 7f876bee3b80 1 -- 172.21.15.46:0/2802186249 shutdown_connections 2024-05-08T23:21:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.622+0000 7f876bee3b80 1 -- 172.21.15.46:0/2802186249 wait complete. 2024-05-08T23:21:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.622+0000 7f876bee3b80 1 -- start start 2024-05-08T23:21:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.623+0000 7f876bee3b80 1 Processor -- start 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e72fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e730130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e612560 con 0x562f0e726890 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e612200 con 0x562f0e72be00 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e4f96a0 con 0x562f0e72c720 2024-05-08T23:21:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.624+0000 7f876bee3b80 10 client.0 _ll_register_callbacks cb 0x562f0e61bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.625+0000 7f8756400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.625+0000 7f8756400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38778/0 (socket says 172.21.15.46:38778) 2024-05-08T23:21:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.625+0000 7f8756400700 1 -- 172.21.15.46:0/508737489 learned_addr learned my addr 172.21.15.46:0/508737489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.625+0000 7f8755a00700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e72fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8755000700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e730130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 -- 172.21.15.46:0/508737489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 msgr2=0x562f0e730130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e730130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 -- 172.21.15.46:0/508737489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 msgr2=0x562f0e72fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e72fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 -- 172.21.15.46:0/508737489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8744007710 con 0x562f0e72c720 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8756400700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f873c00f6e0 tx=0x7f873c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c0e935a31b2572 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.626+0000 7f8737400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.627+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c00b830 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.627+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f873c00b990 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.628+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f0e5cf270 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.628+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f873c00bb50 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.628+0000 7f876bee3b80 10 client.30142 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.628+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562f0e4a3b60 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.629+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f0e5cf270 con 0x562f0e72c720 2024-05-08T23:21:06.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.629+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f873c03e070 con 0x562f0e72c720 2024-05-08T23:21:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 10 client.30142 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 10 client.30142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 20 client.30142 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e740250 0x562f0e742670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 4 client.30142 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f0e6f5810 con 0x562f0e740250 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.631+0000 7f8755a00700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e740250 0x562f0e742670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.632+0000 7f8755a00700 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e740250 0x562f0e742670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b361ef3ef7bf2a67 server_cookie=6ab0cd87504c893 in_seq=0 out_seq=0 2024-05-08T23:21:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.632+0000 7f8737400700 10 client.30142 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f8737400700 1 -- 172.21.15.46:0/508737489 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f0e6f5810 con 0x562f0e740250 2024-05-08T23:21:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f8737400700 10 client.30142 handle_command_reply: tid=0 2024-05-08T23:21:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f876bee3b80 1 client.30142 shutdown 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 msgr2=0x562f0e730670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f873c00f6e0 tx=0x7f873c009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 shutdown_connections 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.633+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e740250 0x562f0e742670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72c720 0x562f0e730670 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e72fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 --2- 172.21.15.46:0/508737489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e730130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 >> 172.21.15.46:0/508737489 conn(0x562f0e684210 msgr2=0x562f0e7205a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f8737400700 0 client.30142 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 shutdown_connections 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 1 -- 172.21.15.46:0/508737489 wait complete. 2024-05-08T23:21:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.634+0000 7f876bee3b80 20 client.30142 trim_cache size 0 max 16384 2024-05-08T23:21:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.667+0000 7f876bee3b80 1 Processor -- start 2024-05-08T23:21:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 -- start start 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72be00 0x562f0e720d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 0x562f0e730490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e7311f0 con 0x562f0e726890 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e7227f0 con 0x562f0e72c720 2024-05-08T23:21:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.668+0000 7f876bee3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e72c430 con 0x562f0e72be00 2024-05-08T23:21:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57642/0 (socket says 172.21.15.46:57642) 2024-05-08T23:21:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 -- 172.21.15.46:0/1048163742 learned_addr learned my addr 172.21.15.46:0/1048163742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 -- 172.21.15.46:0/1048163742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72be00 msgr2=0x562f0e720d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72be00 0x562f0e720d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 -- 172.21.15.46:0/1048163742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 msgr2=0x562f0e730490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8756400700 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 0x562f0e730490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 0x562f0e730490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8755a00700 1 -- 172.21.15.46:0/1048163742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f0e63c560 con 0x562f0e726890 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.669+0000 7f8756400700 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 0x562f0e730490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.670+0000 7f8755a00700 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8748099870 tx=0x7f87480078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611b01de63c4d005 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.670+0000 7f8743400700 1 -- 172.21.15.46:0/1048163742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748099b70 con 0x562f0e726890 2024-05-08T23:21:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.670+0000 7f8743400700 1 -- 172.21.15.46:0/1048163742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f874809d9a0 con 0x562f0e726890 2024-05-08T23:21:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.670+0000 7f8743400700 1 -- 172.21.15.46:0/1048163742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748099060 con 0x562f0e726890 2024-05-08T23:21:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.672+0000 7f876bee3b80 1 -- 172.21.15.46:0/1048163742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 msgr2=0x562f0e726350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.672+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8748099870 tx=0x7f87480078f0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 -- 172.21.15.46:0/1048163742 shutdown_connections 2024-05-08T23:21:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72c720 0x562f0e730490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e726890 0x562f0e726350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1048163742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e72be00 0x562f0e720d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 -- 172.21.15.46:0/1048163742 >> 172.21.15.46:0/1048163742 conn(0x562f0e684210 msgr2=0x562f0e72e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.674+0000 7f876bee3b80 1 -- 172.21.15.46:0/1048163742 shutdown_connections 2024-05-08T23:21:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.675+0000 7f876bee3b80 1 -- 172.21.15.46:0/1048163742 wait complete. 2024-05-08T23:21:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.675+0000 7f876bee3b80 1 -- start start 2024-05-08T23:21:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.676+0000 7f876bee3b80 1 Processor -- start 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e735920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e72c720 0x562f0e78f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8755a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8755a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38800/0 (socket says 172.21.15.46:38800) 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8755000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e735920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8755000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e735920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37800/0 (socket says 172.21.15.46:37800) 2024-05-08T23:21:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8755a00700 1 -- 172.21.15.46:0/1450417195 learned_addr learned my addr 172.21.15.46:0/1450417195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e78fc60 con 0x562f0e72c720 2024-05-08T23:21:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e78fdd0 con 0x562f0e72be00 2024-05-08T23:21:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0e78ff90 con 0x562f0e726890 2024-05-08T23:21:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f876bee3b80 10 client.0 _ll_register_callbacks cb 0x562f0e61bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.677+0000 7f8756400700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e72c720 0x562f0e78f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 -- 172.21.15.46:0/1450417195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 msgr2=0x562f0e735920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e735920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 -- 172.21.15.46:0/1450417195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e72c720 msgr2=0x562f0e78f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e72c720 0x562f0e78f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87480b9040 con 0x562f0e726890 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8755a00700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f87480062a0 tx=0x7f87480995c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b38c008c8ad2fb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.678+0000 7f8737400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748099260 con 0x562f0e726890 2024-05-08T23:21:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f87480a1ff0 con 0x562f0e726890 2024-05-08T23:21:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8744004830 con 0x562f0e726890 2024-05-08T23:21:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8748099b70 con 0x562f0e726890 2024-05-08T23:21:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f876bee3b80 10 client.30145 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.679+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562f0e4fbcd0 con 0x562f0e726890 2024-05-08T23:21:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.681+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8744004830 con 0x562f0e726890 2024-05-08T23:21:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f874809ab80 con 0x562f0e726890 2024-05-08T23:21:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 10 client.30145 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 10 client.30145 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:21:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 20 client.30145 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e797240 0x562f0e799630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 4 client.30145 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562f0e66eec0 con 0x562f0e797240 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.682+0000 7f8755000700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e797240 0x562f0e799630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.683+0000 7f8755000700 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e797240 0x562f0e799630 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57294357cf64df71 server_cookie=67c88b1e1f2a435f in_seq=0 out_seq=0 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.683+0000 7f8737400700 10 client.30145 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f8737400700 1 -- 172.21.15.46:0/1450417195 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562f0e66eec0 con 0x562f0e797240 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f8737400700 10 client.30145 handle_command_reply: tid=0 2024-05-08T23:21:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f876bee3b80 1 client.30145 shutdown 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 msgr2=0x562f0e7353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f87480062a0 tx=0x7f87480995c0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 shutdown_connections 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.684+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x562f0e797240 0x562f0e799630 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.685+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f0e72c720 0x562f0e78f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.685+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f0e726890 0x562f0e7353e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.685+0000 7f876bee3b80 1 --2- 172.21.15.46:0/1450417195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f0e72be00 0x562f0e735920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.685+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 >> 172.21.15.46:0/1450417195 conn(0x562f0e684210 msgr2=0x562f0e72ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.685+0000 7f8737400700 0 client.30145 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.686+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 shutdown_connections 2024-05-08T23:21:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.687+0000 7f876bee3b80 1 -- 172.21.15.46:0/1450417195 wait complete. 2024-05-08T23:21:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.689+0000 7f876bee3b80 20 client.30145 trim_cache size 0 max 16384 2024-05-08T23:21:06.690 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.691+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 msgr2=0x7f87380a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.692+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 0x7f87380a3920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f873c005e90 tx=0x7f873c005b40 comp rx=0 tx=0).stop 2024-05-08T23:21:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.692+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 msgr2=0x7f87501a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.692+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f874800c0c0 tx=0x7f874800c480 comp rx=0 tx=0).stop 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 shutdown_connections 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8750105190 0x7f875019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87500ffd90 0x7f87501a35e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f87380a1460 0x7f87380a3920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 --2- 172.21.15.46:0/3262535963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87501006b0 0x7f87501a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 >> 172.21.15.46:0/3262535963 conn(0x7f87500fb760 msgr2=0x7f8750102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 shutdown_connections 2024-05-08T23:21:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:06.694+0000 7f8758400700 1 -- 172.21.15.46:0/3262535963 wait complete. 2024-05-08T23:21:06.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:21:06.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:06 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1561307477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:06.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:06 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1561307477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.151+0000 7fdeec600700 1 -- 172.21.15.46:0/3576664831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee410b3c0 msgr2=0x7fdee410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.151+0000 7fdeec600700 1 --2- 172.21.15.46:0/3576664831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee410b3c0 0x7fdee410d880 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009f90 tx=0x7fdedc00b330 comp rx=0 tx=0).stop 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.152+0000 7fdeec600700 1 -- 172.21.15.46:0/3576664831 shutdown_connections 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.152+0000 7fdeec600700 1 --2- 172.21.15.46:0/3576664831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee410b3c0 0x7fdee410d880 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.152+0000 7fdeec600700 1 --2- 172.21.15.46:0/3576664831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.152+0000 7fdeec600700 1 --2- 172.21.15.46:0/3576664831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee41028c0 0x7fdee410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.152+0000 7fdeec600700 1 -- 172.21.15.46:0/3576664831 >> 172.21.15.46:0/3576664831 conn(0x7fdee40fb760 msgr2=0x7fdee40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.153+0000 7fdeec600700 1 -- 172.21.15.46:0/3576664831 shutdown_connections 2024-05-08T23:21:07.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.153+0000 7fdeec600700 1 -- 172.21.15.46:0/3576664831 wait complete. 2024-05-08T23:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.155+0000 7fdeec600700 1 Processor -- start 2024-05-08T23:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.156+0000 7fdeec600700 1 -- start start 2024-05-08T23:21:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.156+0000 7fdeec600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.156+0000 7fdeec600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee41028c0 0x7fdee41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdeec600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 0x7fdee419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdeec600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee419c2d0 con 0x7fdee41028c0 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdeec600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee419c440 con 0x7fdee4101fa0 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdeec600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee419c600 con 0x7fdee410b3c0 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee41973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37832/0 (socket says 172.21.15.46:37832) 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdee9c00700 1 -- 172.21.15.46:0/929698410 learned_addr learned my addr 172.21.15.46:0/929698410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdeea600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 0x7fdee419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.157+0000 7fdee9200700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee41028c0 0x7fdee41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 -- 172.21.15.46:0/929698410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 msgr2=0x7fdee41973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee41973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 -- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee41028c0 msgr2=0x7fdee41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee41028c0 0x7fdee41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 -- 172.21.15.46:0/929698410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80095f0 con 0x7fdee410b3c0 2024-05-08T23:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.158+0000 7fdeea600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 0x7fdee419bc90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b6c0 tx=0x7fdedc0079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.159+0000 7fdee2000700 1 -- 172.21.15.46:0/929698410 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00e040 con 0x7fdee410b3c0 2024-05-08T23:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.159+0000 7fdee2000700 1 -- 172.21.15.46:0/929698410 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdedc007e80 con 0x7fdee410b3c0 2024-05-08T23:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.159+0000 7fdee2000700 1 -- 172.21.15.46:0/929698410 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc005330 con 0x7fdee410b3c0 2024-05-08T23:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.159+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc009c40 con 0x7fdee410b3c0 2024-05-08T23:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.159+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee40ffb20 con 0x7fdee410b3c0 2024-05-08T23:21:07.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.168+0000 7fdee2000700 1 -- 172.21.15.46:0/929698410 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdedc01c6c0 con 0x7fdee410b3c0 2024-05-08T23:21:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.171+0000 7fdee2000700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 0x7fdecc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.172+0000 7fdee2000700 1 -- 172.21.15.46:0/929698410 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fdedc0d1df0 con 0x7fdee410b3c0 2024-05-08T23:21:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.175+0000 7fdee9c00700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 0x7fdecc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.176+0000 7fdee9c00700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 0x7fdecc0a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fded8005d10 tx=0x7fded8009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.177+0000 7fdf001f4b80 1 Processor -- start 2024-05-08T23:21:07.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.178+0000 7fdf001f4b80 1 -- start start 2024-05-08T23:21:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.178+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf2f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf2f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf249150 con 0x558aaf300f90 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf246b10 con 0x558aaf300670 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf242400 con 0x558aaf2f2890 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdee9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf2f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37834/0 (socket says 172.21.15.46:37834) 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.179+0000 7fdee9c00700 1 -- 172.21.15.46:0/2038354985 learned_addr learned my addr 172.21.15.46:0/2038354985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdeea600700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf2f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9c00700 1 -- 172.21.15.46:0/2038354985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 msgr2=0x558aaf2f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9c00700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf2f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9c00700 1 -- 172.21.15.46:0/2038354985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 msgr2=0x558aaf2f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9c00700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf2f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9c00700 1 -- 172.21.15.46:0/2038354985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aaf0c33b0 con 0x558aaf300670 2024-05-08T23:21:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdeea600700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf2f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.180+0000 7fdee9200700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.182+0000 7fdee9c00700 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fded8012580 tx=0x7fded8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c72b1eeaa40010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.182+0000 7fded6a00700 1 -- 172.21.15.46:0/2038354985 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded801f620 con 0x558aaf300670 2024-05-08T23:21:07.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.182+0000 7fded6a00700 1 -- 172.21.15.46:0/2038354985 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded8016070 con 0x558aaf300670 2024-05-08T23:21:07.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.182+0000 7fded6a00700 1 -- 172.21.15.46:0/2038354985 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded801db30 con 0x558aaf300670 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.183+0000 7fdf001f4b80 1 -- 172.21.15.46:0/2038354985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 msgr2=0x558aaf300a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.183+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fded8012580 tx=0x7fded8014000 comp rx=0 tx=0).stop 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 -- 172.21.15.46:0/2038354985 shutdown_connections 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf2f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf300a50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/2038354985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf2f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 -- 172.21.15.46:0/2038354985 >> 172.21.15.46:0/2038354985 conn(0x558aaf250210 msgr2=0x558aaf2ecf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 -- 172.21.15.46:0/2038354985 shutdown_connections 2024-05-08T23:21:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.184+0000 7fdf001f4b80 1 -- 172.21.15.46:0/2038354985 wait complete. 2024-05-08T23:21:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.186+0000 7fdf001f4b80 1 -- start start 2024-05-08T23:21:07.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.187+0000 7fdf001f4b80 1 Processor -- start 2024-05-08T23:21:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf304870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300f90 0x558aaf304db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf1de560 con 0x558aaf2f2890 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf1de200 con 0x558aaf300670 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf0c56a0 con 0x558aaf300f90 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.189+0000 7fdf001f4b80 10 client.0 _ll_register_callbacks cb 0x558aaf1e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57686/0 (socket says 172.21.15.46:57686) 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdee9c00700 1 -- 172.21.15.46:0/4127380165 learned_addr learned my addr 172.21.15.46:0/4127380165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdeea600700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300f90 0x558aaf304db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdee9c00700 1 -- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300f90 msgr2=0x558aaf304db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.190+0000 7fdee9c00700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300f90 0x558aaf304db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.191+0000 7fdee9c00700 1 -- 172.21.15.46:0/4127380165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 msgr2=0x558aaf304870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.191+0000 7fdee9c00700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf304870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.191+0000 7fdee9c00700 1 -- 172.21.15.46:0/4127380165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee40ffb20 con 0x558aaf2f2890 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.192+0000 7fdee9c00700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fded801d3f0 tx=0x7fded801de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c8a3badfcbd768 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.192+0000 7fdecb400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.192+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded802d070 con 0x558aaf2f2890 2024-05-08T23:21:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.192+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fded8018070 con 0x558aaf2f2890 2024-05-08T23:21:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.193+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558aaf19b270 con 0x558aaf2f2890 2024-05-08T23:21:07.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.195+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded801d450 con 0x558aaf2f2890 2024-05-08T23:21:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.197+0000 7fdf001f4b80 10 client.19938 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.197+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fded8011e00 con 0x558aaf2f2890 2024-05-08T23:21:07.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.198+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558aaf19b270 con 0x558aaf2f2890 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.198+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fded803a070 con 0x558aaf2f2890 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 10 client.19938 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 10 client.19938 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 20 client.19938 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf3258d0 0x558aaf327cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 4 client.19938 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558aaf2c1810 con 0x558aaf3258d0 2024-05-08T23:21:07.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.199+0000 7fdee9200700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf3258d0 0x558aaf327cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.201+0000 7fdecb400700 10 client.19938 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:07.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.201+0000 7fdee9200700 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf3258d0 0x558aaf327cf0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49e14651a21b5a70 server_cookie=e37b441fd5818b14 in_seq=0 out_seq=0 2024-05-08T23:21:07.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.202+0000 7fdecb400700 1 -- 172.21.15.46:0/4127380165 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558aaf2c1810 con 0x558aaf3258d0 2024-05-08T23:21:07.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.202+0000 7fdecb400700 10 client.19938 handle_command_reply: tid=0 2024-05-08T23:21:07.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.203+0000 7fdf001f4b80 1 client.19938 shutdown 2024-05-08T23:21:07.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.203+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 msgr2=0x558aaf304330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.203+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fded801d3f0 tx=0x7fded801de50 comp rx=0 tx=0).stop 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 shutdown_connections 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf3258d0 0x558aaf327cf0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300f90 0x558aaf304db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf2f2890 0x558aaf304330 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/4127380165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300670 0x558aaf304870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 >> 172.21.15.46:0/4127380165 conn(0x558aaf250210 msgr2=0x558aaf2ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.206+0000 7fdecb400700 0 client.19938 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.207+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 shutdown_connections 2024-05-08T23:21:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.207+0000 7fdf001f4b80 1 -- 172.21.15.46:0/4127380165 wait complete. 2024-05-08T23:21:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.208+0000 7fdf001f4b80 20 client.19938 trim_cache size 0 max 16384 2024-05-08T23:21:07.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.237+0000 7fdf001f4b80 1 Processor -- start 2024-05-08T23:21:07.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.237+0000 7fdf001f4b80 1 -- start start 2024-05-08T23:21:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 0x558aaf2ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf304bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf2edc70 con 0x558aaf300f90 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf302880 con 0x558aaf2f2890 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf300030 con 0x558aaf300670 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdee9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 0x558aaf2ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37846/0 (socket says 172.21.15.46:37846) 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.238+0000 7fdee9c00700 1 -- 172.21.15.46:0/658266553 learned_addr learned my addr 172.21.15.46:0/658266553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdeea600700 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf304bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9c00700 1 -- 172.21.15.46:0/658266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 msgr2=0x558aaf2ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9c00700 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 0x558aaf2ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9c00700 1 -- 172.21.15.46:0/658266553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 msgr2=0x558aaf304bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9c00700 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf304bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9c00700 1 -- 172.21.15.46:0/658266553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aaf06fb60 con 0x558aaf2f2890 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.239+0000 7fdee9200700 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 0x558aaf2ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.240+0000 7fdee9c00700 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fded8000c00 tx=0x7fded8027990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b661f8d2daae529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.243+0000 7fded7400700 1 -- 172.21.15.46:0/658266553 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded802d030 con 0x558aaf2f2890 2024-05-08T23:21:07.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.243+0000 7fded7400700 1 -- 172.21.15.46:0/658266553 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded801faf0 con 0x558aaf2f2890 2024-05-08T23:21:07.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.243+0000 7fded7400700 1 -- 172.21.15.46:0/658266553 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80053b0 con 0x558aaf2f2890 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 -- 172.21.15.46:0/658266553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 msgr2=0x558aaf2f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fded8000c00 tx=0x7fded8027990 comp rx=0 tx=0).stop 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 -- 172.21.15.46:0/658266553 shutdown_connections 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300f90 0x558aaf304bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf2f2890 0x558aaf2f2350 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/658266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf300670 0x558aaf2ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.245+0000 7fdf001f4b80 1 -- 172.21.15.46:0/658266553 >> 172.21.15.46:0/658266553 conn(0x558aaf250210 msgr2=0x558aaf2ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.246+0000 7fdf001f4b80 1 -- 172.21.15.46:0/658266553 shutdown_connections 2024-05-08T23:21:07.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.246+0000 7fdf001f4b80 1 -- 172.21.15.46:0/658266553 wait complete. 2024-05-08T23:21:07.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.247+0000 7fdf001f4b80 1 -- start start 2024-05-08T23:21:07.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.249+0000 7fdf001f4b80 1 Processor -- start 2024-05-08T23:21:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.252+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.252+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300670 0x558aaf376db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.253+0000 7fdf001f4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300f90 0x558aaf3773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.253+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf377af0 con 0x558aaf300670 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.253+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf377c60 con 0x558aaf300f90 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.253+0000 7fdf001f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aaf309200 con 0x558aaf2f2890 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.253+0000 7fdf001f4b80 10 client.0 _ll_register_callbacks cb 0x558aaf1e7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.254+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.254+0000 7fdee9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38858/0 (socket says 172.21.15.46:38858) 2024-05-08T23:21:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.254+0000 7fdee9c00700 1 -- 172.21.15.46:0/197036588 learned_addr learned my addr 172.21.15.46:0/197036588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.254+0000 7fdeea600700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300f90 0x558aaf3773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.254+0000 7fdee9200700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300670 0x558aaf376db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.255+0000 7fdee9c00700 1 -- 172.21.15.46:0/197036588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300f90 msgr2=0x558aaf3773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.255+0000 7fdee9c00700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300f90 0x558aaf3773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.255+0000 7fdee9c00700 1 -- 172.21.15.46:0/197036588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300670 msgr2=0x558aaf376db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.255+0000 7fdee9c00700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300670 0x558aaf376db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.255+0000 7fdee9c00700 1 -- 172.21.15.46:0/197036588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded8011e00 con 0x558aaf2f2890 2024-05-08T23:21:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.256+0000 7fdee9c00700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fded8027960 tx=0x7fded8027c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c9c2cb0902ea01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.258+0000 7fdecb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.258+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8005040 con 0x558aaf2f2890 2024-05-08T23:21:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.258+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fded802e070 con 0x558aaf2f2890 2024-05-08T23:21:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.258+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fded8016070 con 0x558aaf2f2890 2024-05-08T23:21:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.259+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded80051a0 con 0x558aaf2f2890 2024-05-08T23:21:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.259+0000 7fdf001f4b80 10 client.30163 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.259+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558aaf208560 con 0x558aaf2f2890 2024-05-08T23:21:07.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.259+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded8016070 con 0x558aaf2f2890 2024-05-08T23:21:07.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fded8027d20 con 0x558aaf2f2890 2024-05-08T23:21:07.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 10 client.30163 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:07.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 10 client.30163 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:21:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 20 client.30163 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf2f76f0 0x558aaf2f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 4 client.30163 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558aaf23aec0 con 0x558aaf2f76f0 2024-05-08T23:21:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.261+0000 7fdee9200700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf2f76f0 0x558aaf2f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.262+0000 7fdee9200700 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf2f76f0 0x558aaf2f9ae0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88b82f58c57a50dd server_cookie=7a95e54c23ac339a in_seq=0 out_seq=0 2024-05-08T23:21:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.262+0000 7fdecb400700 10 client.30163 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.262+0000 7fdecb400700 1 -- 172.21.15.46:0/197036588 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558aaf23aec0 con 0x558aaf2f76f0 2024-05-08T23:21:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.262+0000 7fdecb400700 10 client.30163 handle_command_reply: tid=0 2024-05-08T23:21:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.262+0000 7fdf001f4b80 1 client.30163 shutdown 2024-05-08T23:21:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.263+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 msgr2=0x558aaf30bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.263+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fded8027960 tx=0x7fded8027c30 comp rx=0 tx=0).stop 2024-05-08T23:21:07.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81591 19.80 MB/sec execute 88 sec latency 527.108 ms 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 shutdown_connections 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x558aaf2f76f0 0x558aaf2f9ae0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558aaf300f90 0x558aaf3773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aaf2f2890 0x558aaf30bdf0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 --2- 172.21.15.46:0/197036588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aaf300670 0x558aaf376db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 >> 172.21.15.46:0/197036588 conn(0x558aaf250210 msgr2=0x558aaf304c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdecb400700 0 client.30163 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 shutdown_connections 2024-05-08T23:21:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.265+0000 7fdf001f4b80 1 -- 172.21.15.46:0/197036588 wait complete. 2024-05-08T23:21:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.266+0000 7fdf001f4b80 20 client.30163 trim_cache size 0 max 16384 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 msgr2=0x7fdecc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 0x7fdecc0a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fded8005d10 tx=0x7fded8009310 comp rx=0 tx=0).stop 2024-05-08T23:21:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 msgr2=0x7fdee419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 0x7fdee419bc90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b6c0 tx=0x7fdedc0079d0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 shutdown_connections 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdee410b3c0 0x7fdee419bc90 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee4101fa0 0x7fdee41973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdecc0a1310 0x7fdecc0a37d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 --2- 172.21.15.46:0/929698410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee41028c0 0x7fdee41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.267+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 >> 172.21.15.46:0/929698410 conn(0x7fdee40fb760 msgr2=0x7fdee40fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.268+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 shutdown_connections 2024-05-08T23:21:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.269+0000 7fdeec600700 1 -- 172.21.15.46:0/929698410 wait complete. 2024-05-08T23:21:07.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 -- 172.21.15.46:0/3947482569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c1008f0 msgr2=0x7f836c104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 --2- 172.21.15.46:0/3947482569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c1008f0 0x7f836c104930 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8360009990 tx=0x7f8360009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 -- 172.21.15.46:0/3947482569 shutdown_connections 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 --2- 172.21.15.46:0/3947482569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c105060 0x7f836c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 --2- 172.21.15.46:0/3947482569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 0x7f836c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 --2- 172.21.15.46:0/3947482569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c1008f0 0x7f836c104930 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 -- 172.21.15.46:0/3947482569 >> 172.21.15.46:0/3947482569 conn(0x7f836c0fb760 msgr2=0x7f836c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 -- 172.21.15.46:0/3947482569 shutdown_connections 2024-05-08T23:21:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.680+0000 7f8374400700 1 -- 172.21.15.46:0/3947482569 wait complete. 2024-05-08T23:21:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.683+0000 7f8374400700 1 Processor -- start 2024-05-08T23:21:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.683+0000 7f8374400700 1 -- start start 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 0x7f836c1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c10a4b0 0x7f836c10a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c10afc0 con 0x7f836c10a4b0 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c10b130 con 0x7f836c0fff40 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8374400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c10b2f0 con 0x7f836c105060 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8371000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8371000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38872/0 (socket says 172.21.15.46:38872) 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8371000700 1 -- 172.21.15.46:0/4074446150 learned_addr learned my addr 172.21.15.46:0/4074446150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8371a00700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 0x7f836c1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.684+0000 7f8372400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c10a4b0 0x7f836c10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.685+0000 7f8371000700 1 -- 172.21.15.46:0/4074446150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 msgr2=0x7f836c1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.685+0000 7f8371000700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 0x7f836c1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.685+0000 7f8371000700 1 -- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c10a4b0 msgr2=0x7f836c10a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.685+0000 7f8371000700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c10a4b0 0x7f836c10a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.685+0000 7f8371000700 1 -- 172.21.15.46:0/4074446150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83600095f0 con 0x7f836c105060 2024-05-08T23:21:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.686+0000 7f8371000700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8360005770 tx=0x7f83600057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.687+0000 7f836a000700 1 -- 172.21.15.46:0/4074446150 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836001d070 con 0x7f836c105060 2024-05-08T23:21:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.687+0000 7f836a000700 1 -- 172.21.15.46:0/4074446150 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8360003f30 con 0x7f836c105060 2024-05-08T23:21:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.687+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c1aa800 con 0x7f836c105060 2024-05-08T23:21:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.687+0000 7f836a000700 1 -- 172.21.15.46:0/4074446150 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360022cc0 con 0x7f836c105060 2024-05-08T23:21:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.687+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c1aac80 con 0x7f836c105060 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.697+0000 7f8387ea7b80 1 Processor -- start 2024-05-08T23:21:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.697+0000 7f8387ea7b80 1 -- start start 2024-05-08T23:21:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.698+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8372400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e320150 con 0x55e33e3cee00 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8372400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57742/0 (socket says 172.21.15.46:57742) 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8372400700 1 -- 172.21.15.46:0/439835117 learned_addr learned my addr 172.21.15.46:0/439835117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e31db10 con 0x55e33e3c9890 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e319400 con 0x55e33e3cf720 2024-05-08T23:21:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f8371a00700 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.699+0000 7f836a000700 1 -- 172.21.15.46:0/4074446150 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8360003a70 con 0x7f836c105060 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 -- 172.21.15.46:0/439835117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 msgr2=0x55e33e3c9cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 -- 172.21.15.46:0/439835117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 msgr2=0x55e33e3cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 -- 172.21.15.46:0/439835117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33e19a3b0 con 0x55e33e3cf720 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.700+0000 7f8371a00700 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3c9350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f836400a430 tx=0x7f83640096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51901ebad363ed51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.701+0000 7f835aa00700 1 -- 172.21.15.46:0/439835117 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364010040 con 0x55e33e3cf720 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.701+0000 7f835aa00700 1 -- 172.21.15.46:0/439835117 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83640148e0 con 0x55e33e3cf720 2024-05-08T23:21:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.701+0000 7f835aa00700 1 -- 172.21.15.46:0/439835117 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83640138c0 con 0x55e33e3cf720 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.702+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 msgr2=0x55e33e3c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.702+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3c9350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f836400a430 tx=0x7f83640096d0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.702+0000 7f836a000700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 0x7f83540a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.703+0000 7f836a000700 1 -- 172.21.15.46:0/4074446150 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83600ce540 con 0x7f836c105060 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8372400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 0x7f83540a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 shutdown_connections 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/439835117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3c9350 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.704+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 >> 172.21.15.46:0/439835117 conn(0x55e33e327210 msgr2=0x55e33e3c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.705+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 shutdown_connections 2024-05-08T23:21:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.705+0000 7f8387ea7b80 1 -- 172.21.15.46:0/439835117 wait complete. 2024-05-08T23:21:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.705+0000 7f8387ea7b80 1 -- start start 2024-05-08T23:21:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.706+0000 7f8387ea7b80 1 Processor -- start 2024-05-08T23:21:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3c9890 0x55e33e3d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e2b5560 con 0x55e33e3c9890 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e2b5200 con 0x55e33e3cf720 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e19c6a0 con 0x55e33e3cee00 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8372400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 0x7f83540a3850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f835c006c80 tx=0x7f835c008040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8387ea7b80 10 client.0 _ll_register_callbacks cb 0x55e33e2beb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8372400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8372400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37888/0 (socket says 172.21.15.46:37888) 2024-05-08T23:21:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.707+0000 7f8372400700 1 -- 172.21.15.46:0/3779234511 learned_addr learned my addr 172.21.15.46:0/3779234511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 -- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 msgr2=0x55e33e3d3030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3d3030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 -- 172.21.15.46:0/3779234511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3c9890 msgr2=0x55e33e3d2af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3c9890 0x55e33e3d2af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 -- 172.21.15.46:0/3779234511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364009c60 con 0x55e33e3cf720 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f8372400700 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f835c015700 tx=0x7f835c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8064ba552879aad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f834f400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c013620 con 0x55e33e3cf720 2024-05-08T23:21:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.708+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f835c01a040 con 0x55e33e3cf720 2024-05-08T23:21:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.709+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e33e272270 con 0x55e33e3cf720 2024-05-08T23:21:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.709+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c013780 con 0x55e33e3cf720 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.709+0000 7f8387ea7b80 10 client.30104 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.709+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e33e146b60 con 0x55e33e3cf720 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e33e272270 con 0x55e33e3cf720 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f835c028970 con 0x55e33e3cf720 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 10 client.30104 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 10 client.30104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 20 client.30104 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e3f4420 0x55e33e3f6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 4 client.30104 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e33e398810 con 0x55e33e3f4420 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.710+0000 7f8371a00700 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e3f4420 0x55e33e3f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.712+0000 7f834f400700 10 client.30104 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.712+0000 7f8371a00700 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e3f4420 0x55e33e3f6840 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72c9f56606f5df79 server_cookie=44d8c853d52defda in_seq=0 out_seq=0 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.712+0000 7f834f400700 1 -- 172.21.15.46:0/3779234511 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e33e398810 con 0x55e33e3f4420 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.712+0000 7f834f400700 10 client.30104 handle_command_reply: tid=0 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 client.30104 shutdown 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 msgr2=0x55e33e3d3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f835c015700 tx=0x7f835c004750 comp rx=0 tx=0).stop 2024-05-08T23:21:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 shutdown_connections 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3cf720 0x55e33e3d3570 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e3f4420 0x55e33e3f6840 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3c9890 0x55e33e3d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/3779234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 >> 172.21.15.46:0/3779234511 conn(0x55e33e327210 msgr2=0x55e33e3c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.713+0000 7f834f400700 0 client.30104 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.714+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 shutdown_connections 2024-05-08T23:21:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.714+0000 7f8387ea7b80 1 -- 172.21.15.46:0/3779234511 wait complete. 2024-05-08T23:21:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.714+0000 7f8387ea7b80 20 client.30104 trim_cache size 0 max 16384 2024-05-08T23:21:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.740+0000 7f8387ea7b80 1 Processor -- start 2024-05-08T23:21:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 -- start start 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cf720 0x55e33e3dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3c4b30 con 0x55e33e3cf720 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3f1a10 con 0x55e33e3c9890 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.741+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3f0e00 con 0x55e33e3cee00 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.742+0000 7f8371000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.742+0000 7f8371000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38892/0 (socket says 172.21.15.46:38892) 2024-05-08T23:21:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.742+0000 7f8371000700 1 -- 172.21.15.46:0/2197058964 learned_addr learned my addr 172.21.15.46:0/2197058964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.742+0000 7f8372400700 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cf720 0x55e33e3dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371a00700 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371000700 1 -- 172.21.15.46:0/2197058964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 msgr2=0x55e33e3c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371000700 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371000700 1 -- 172.21.15.46:0/2197058964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cf720 msgr2=0x55e33e3dec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371000700 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cf720 0x55e33e3dec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.743+0000 7f8371000700 1 -- 172.21.15.46:0/2197058964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33e2df560 con 0x55e33e3cee00 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.744+0000 7f8371000700 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8360005e50 tx=0x7f8360005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aacc6d0bfae765db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.744+0000 7f835b400700 1 -- 172.21.15.46:0/2197058964 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360022470 con 0x55e33e3cee00 2024-05-08T23:21:07.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.744+0000 7f835b400700 1 -- 172.21.15.46:0/2197058964 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f836009b970 con 0x55e33e3cee00 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.744+0000 7f835b400700 1 -- 172.21.15.46:0/2197058964 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836009b4d0 con 0x55e33e3cee00 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 -- 172.21.15.46:0/2197058964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 msgr2=0x55e33e3f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8360005e50 tx=0x7f8360005940 comp rx=0 tx=0).stop 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 -- 172.21.15.46:0/2197058964 shutdown_connections 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cf720 0x55e33e3dec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.745+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/2197058964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cee00 0x55e33e3f6810 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.747+0000 7f8387ea7b80 1 -- 172.21.15.46:0/2197058964 >> 172.21.15.46:0/2197058964 conn(0x55e33e327210 msgr2=0x55e33e3d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.748+0000 7f8387ea7b80 1 -- 172.21.15.46:0/2197058964 shutdown_connections 2024-05-08T23:21:07.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.749+0000 7f8387ea7b80 1 -- 172.21.15.46:0/2197058964 wait complete. 2024-05-08T23:21:07.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.750+0000 7f8387ea7b80 1 -- start start 2024-05-08T23:21:07.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.750+0000 7f8387ea7b80 1 Processor -- start 2024-05-08T23:21:07.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.751+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.751+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8387ea7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3d8890 con 0x55e33e3cee00 2024-05-08T23:21:07.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3d8a00 con 0x55e33e3c9890 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8387ea7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33e3d8bc0 con 0x55e33e3cf720 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8387ea7b80 10 client.0 _ll_register_callbacks cb 0x55e33e2beb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.752+0000 7f8371a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3da1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.753+0000 7f8371a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3da1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37912/0 (socket says 172.21.15.46:37912) 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.753+0000 7f8371a00700 1 -- 172.21.15.46:0/135393196 learned_addr learned my addr 172.21.15.46:0/135393196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.753+0000 7f8371000700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.753+0000 7f8372400700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.754+0000 7f8371000700 1 -- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 msgr2=0x55e33e3d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.754+0000 7f8371000700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.754+0000 7f8371000700 1 -- 172.21.15.46:0/135393196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 msgr2=0x55e33e3da1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.754+0000 7f8371000700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3da1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.754+0000 7f8371000700 1 -- 172.21.15.46:0/135393196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c1aac80 con 0x55e33e3cee00 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.755+0000 7f8371000700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3da6e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8360005e50 tx=0x7f8360022760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785ab1dc333d02f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.755+0000 7f834f400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:07.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.756+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836009f0a0 con 0x55e33e3cee00 2024-05-08T23:21:07.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.756+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83640148e0 con 0x55e33e3cee00 2024-05-08T23:21:07.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.756+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83600a3660 con 0x55e33e3cee00 2024-05-08T23:21:07.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.757+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f836009b270 con 0x55e33e3cee00 2024-05-08T23:21:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.757+0000 7f8387ea7b80 10 client.19953 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.757+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e33e19ecd0 con 0x55e33e3cee00 2024-05-08T23:21:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83640148e0 con 0x55e33e3cee00 2024-05-08T23:21:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f83600a10b0 con 0x55e33e3cee00 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 10 client.19953 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 10 client.19953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 20 client.19953 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e44a710 0x55e33e44cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 4 client.19953 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.758+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e33e311ec0 con 0x55e33e44a710 2024-05-08T23:21:07.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.760+0000 7f8371a00700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e44a710 0x55e33e44cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.761+0000 7f8371a00700 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e44a710 0x55e33e44cb30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7282eb5f184b642a server_cookie=3b7f239ec1fb694a in_seq=0 out_seq=0 2024-05-08T23:21:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.761+0000 7f834f400700 10 client.19953 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.762+0000 7f834f400700 1 -- 172.21.15.46:0/135393196 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e33e311ec0 con 0x55e33e44a710 2024-05-08T23:21:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.762+0000 7f834f400700 10 client.19953 handle_command_reply: tid=0 2024-05-08T23:21:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.762+0000 7f8387ea7b80 1 client.19953 shutdown 2024-05-08T23:21:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.762+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 msgr2=0x55e33e3da6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.762+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3da6e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8360005e50 tx=0x7f8360022760 comp rx=0 tx=0).stop 2024-05-08T23:21:07.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:07 smithi053 ceph-mon[77587]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 515 op/s 2024-05-08T23:21:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 shutdown_connections 2024-05-08T23:21:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e33e3cf720 0x55e33e3d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e33e3c9890 0x55e33e3da1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55e33e44a710 0x55e33e44cb30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 --2- 172.21.15.46:0/135393196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e33e3cee00 0x55e33e3da6e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.764+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 >> 172.21.15.46:0/135393196 conn(0x55e33e327210 msgr2=0x55e33e3d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.765+0000 7f834f400700 0 client.19953 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:07.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.765+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 shutdown_connections 2024-05-08T23:21:07.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.766+0000 7f8387ea7b80 1 -- 172.21.15.46:0/135393196 wait complete. 2024-05-08T23:21:07.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.766+0000 7f8387ea7b80 20 client.19953 trim_cache size 0 max 16384 2024-05-08T23:21:07.766 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 msgr2=0x7f83540a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 0x7f83540a3850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f835c006c80 tx=0x7f835c008040 comp rx=0 tx=0).stop 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 msgr2=0x7f836c1a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8360005770 tx=0x7f83600057a0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 shutdown_connections 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83540a1390 0x7f83540a3850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836c10a4b0 0x7f836c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f836c0fff40 0x7f836c1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 --2- 172.21.15.46:0/4074446150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f836c105060 0x7f836c1a3e70 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:07.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 >> 172.21.15.46:0/4074446150 conn(0x7f836c0fb760 msgr2=0x7f836c10edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:07.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 shutdown_connections 2024-05-08T23:21:07.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:07.768+0000 7f8374400700 1 -- 172.21.15.46:0/4074446150 wait complete. 2024-05-08T23:21:07.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:07 smithi046 ceph-mon[36898]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 515 op/s 2024-05-08T23:21:07.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:21:07.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:07 smithi086 ceph-mon[78610]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 515 op/s 2024-05-08T23:21:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.172+0000 7f0a3e200700 1 -- 172.21.15.46:0/1420534148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 msgr2=0x7f0a38108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.172+0000 7f0a3e200700 1 --2- 172.21.15.46:0/1420534148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a38108380 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009990 tx=0x7f0a28009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 -- 172.21.15.46:0/1420534148 shutdown_connections 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 --2- 172.21.15.46:0/1420534148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a381140f0 0x7f0a381164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 --2- 172.21.15.46:0/1420534148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a38108380 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 --2- 172.21.15.46:0/1420534148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381088c0 0x7f0a38113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 -- 172.21.15.46:0/1420534148 >> 172.21.15.46:0/1420534148 conn(0x7f0a3806bf30 msgr2=0x7f0a3806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 -- 172.21.15.46:0/1420534148 shutdown_connections 2024-05-08T23:21:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.174+0000 7f0a3e200700 1 -- 172.21.15.46:0/1420534148 wait complete. 2024-05-08T23:21:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.178+0000 7f0a3e200700 1 Processor -- start 2024-05-08T23:21:08.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.179+0000 7f0a3e200700 1 -- start start 2024-05-08T23:21:08.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a381088c0 0x7f0a381a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381140f0 0x7f0a381a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381a2110 con 0x7f0a381140f0 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381a2280 con 0x7f0a381088c0 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a3e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381a2440 con 0x7f0a38107fa0 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38918/0 (socket says 172.21.15.46:38918) 2024-05-08T23:21:08.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.181+0000 7f0a37400700 1 -- 172.21.15.46:0/594699377 learned_addr learned my addr 172.21.15.46:0/594699377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37e00700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381140f0 0x7f0a381a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 -- 172.21.15.46:0/594699377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a381088c0 msgr2=0x7f0a381a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a381088c0 0x7f0a381a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 -- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381140f0 msgr2=0x7f0a381a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381140f0 0x7f0a381a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 -- 172.21.15.46:0/594699377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280095f0 con 0x7f0a38107fa0 2024-05-08T23:21:08.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.182+0000 7f0a37400700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0a28003b60 tx=0x7f0a28003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.183+0000 7f0a2fe00700 1 -- 172.21.15.46:0/594699377 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2801d070 con 0x7f0a38107fa0 2024-05-08T23:21:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.183+0000 7f0a2fe00700 1 -- 172.21.15.46:0/594699377 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a28005580 con 0x7f0a38107fa0 2024-05-08T23:21:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.183+0000 7f0a2fe00700 1 -- 172.21.15.46:0/594699377 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800fe60 con 0x7f0a38107fa0 2024-05-08T23:21:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.183+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a381a26c0 con 0x7f0a38107fa0 2024-05-08T23:21:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.183+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381ae970 con 0x7f0a38107fa0 2024-05-08T23:21:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.191+0000 7f0a2fe00700 1 -- 172.21.15.46:0/594699377 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0a2802b440 con 0x7f0a38107fa0 2024-05-08T23:21:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.197+0000 7f0a51e0cb80 1 Processor -- start 2024-05-08T23:21:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 -- start start 2024-05-08T23:21:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a2fe00700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 0x7f0a1c0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12dba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a2fe00700 1 -- 172.21.15.46:0/594699377 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0a280cf160 con 0x7f0a38107fa0 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d1224150 con 0x5610d12d60f0 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d1221b10 con 0x5610d12db660 2024-05-08T23:21:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.200+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d121d400 con 0x5610d12dbf80 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38932/0 (socket says 172.21.15.46:38932) 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 -- 172.21.15.46:0/1434039995 learned_addr learned my addr 172.21.15.46:0/1434039995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 -- 172.21.15.46:0/1434039995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 msgr2=0x5610d12dba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.201+0000 7f0a36a00700 1 -- 172.21.15.46:0/1434039995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 msgr2=0x5610d12d6570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.202+0000 7f0a37e00700 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.203+0000 7f0a36a00700 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12d6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.203+0000 7f0a36a00700 1 -- 172.21.15.46:0/1434039995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d109e3b0 con 0x5610d12dbf80 2024-05-08T23:21:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.203+0000 7f0a36a00700 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400aaf0 tx=0x7f0a2400ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd2bd0bc3a7ded49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.203+0000 7f0a37e00700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 0x7f0a1c0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.203+0000 7f0a23e00700 1 -- 172.21.15.46:0/1434039995 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400ccd0 con 0x5610d12dbf80 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.204+0000 7f0a23e00700 1 -- 172.21.15.46:0/1434039995 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2400ce30 con 0x5610d12dbf80 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.204+0000 7f0a23e00700 1 -- 172.21.15.46:0/1434039995 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400f450 con 0x5610d12dbf80 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1434039995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 msgr2=0x5610d12d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400aaf0 tx=0x7f0a2400ae00 comp rx=0 tx=0).stop 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1434039995 shutdown_connections 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12dba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1434039995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12d5b60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.205+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1434039995 >> 172.21.15.46:0/1434039995 conn(0x5610d122b210 msgr2=0x5610d12c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.206+0000 7f0a37e00700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 0x7f0a1c0a3960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806a500 tx=0x7f0a30004ad0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.206+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1434039995 shutdown_connections 2024-05-08T23:21:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.206+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1434039995 wait complete. 2024-05-08T23:21:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.207+0000 7f0a51e0cb80 1 -- start start 2024-05-08T23:21:08.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.208+0000 7f0a51e0cb80 1 Processor -- start 2024-05-08T23:21:08.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12df910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d11b9560 con 0x5610d12d60f0 2024-05-08T23:21:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d11b9200 con 0x5610d12db660 2024-05-08T23:21:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d10a06a0 con 0x5610d12dbf80 2024-05-08T23:21:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a51e0cb80 10 client.0 _ll_register_callbacks cb 0x5610d11c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a37400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a37400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57786/0 (socket says 172.21.15.46:57786) 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.209+0000 7f0a37400700 1 -- 172.21.15.46:0/324012967 learned_addr learned my addr 172.21.15.46:0/324012967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37e00700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37400700 1 -- 172.21.15.46:0/324012967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 msgr2=0x5610d12dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37400700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37400700 1 -- 172.21.15.46:0/324012967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 msgr2=0x5610d12df910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37400700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.210+0000 7f0a37400700 1 -- 172.21.15.46:0/324012967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2400a720 con 0x5610d12d60f0 2024-05-08T23:21:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.211+0000 7f0a37400700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a28011040 tx=0x7f0a2809b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99263250d0f633c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.211+0000 7f0a20c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.211+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2809cea0 con 0x5610d12d60f0 2024-05-08T23:21:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.211+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0a2809be90 con 0x5610d12d60f0 2024-05-08T23:21:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.212+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610d1176270 con 0x5610d12d60f0 2024-05-08T23:21:08.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.215+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a280cc8d0 con 0x5610d12d60f0 2024-05-08T23:21:08.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.216+0000 7f0a51e0cb80 10 client.19956 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:08.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.216+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5610d104ab60 con 0x5610d12d60f0 2024-05-08T23:21:08.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.216+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610d1176270 con 0x5610d12d60f0 2024-05-08T23:21:08.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.216+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a2809b380 con 0x5610d12d60f0 2024-05-08T23:21:08.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 10 client.19956 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 10 client.19956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 20 client.19956 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d12ef650 0x5610d12f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 4 client.19956 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.218+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610d129c810 con 0x5610d12ef650 2024-05-08T23:21:08.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.220+0000 7f0a36a00700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d12ef650 0x5610d12f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.221+0000 7f0a20c00700 10 client.19956 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:08.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.221+0000 7f0a36a00700 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d12ef650 0x5610d12f1a40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8b9f0ccebfc431d server_cookie=369a04e99909a34d in_seq=0 out_seq=0 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.222+0000 7f0a20c00700 1 -- 172.21.15.46:0/324012967 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610d129c810 con 0x5610d12ef650 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.222+0000 7f0a20c00700 10 client.19956 handle_command_reply: tid=0 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 client.19956 shutdown 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 msgr2=0x5610d12df3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a28011040 tx=0x7f0a2809b870 comp rx=0 tx=0).stop 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 shutdown_connections 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d12ef650 0x5610d12f1a40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12d60f0 0x5610d12df3d0 secure :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a28011040 tx=0x7f0a2809b870 comp rx=0 tx=0).stop 2024-05-08T23:21:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a20c00700 0 client.19956 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/324012967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12db660 0x5610d12df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.223+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 >> 172.21.15.46:0/324012967 conn(0x5610d122b210 msgr2=0x5610d12c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.224+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 shutdown_connections 2024-05-08T23:21:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.224+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/324012967 wait complete. 2024-05-08T23:21:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.225+0000 7f0a51e0cb80 20 client.19956 trim_cache size 0 max 16384 2024-05-08T23:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.255+0000 7f0a51e0cb80 1 Processor -- start 2024-05-08T23:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 -- start start 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12dbf80 0x5610d12dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d12e09d0 con 0x5610d12dbf80 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d12c4dc0 con 0x5610d12c4210 2024-05-08T23:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.256+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d12f6c50 con 0x5610d12db660 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37964/0 (socket says 172.21.15.46:37964) 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37e00700 1 -- 172.21.15.46:0/3685100494 learned_addr learned my addr 172.21.15.46:0/3685100494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38946/0 (socket says 172.21.15.46:38946) 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a36a00700 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12dbf80 0x5610d12dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 -- 172.21.15.46:0/3685100494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 msgr2=0x5610d12da320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 -- 172.21.15.46:0/3685100494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12dbf80 msgr2=0x5610d12dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12dbf80 0x5610d12dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.257+0000 7f0a37400700 1 -- 172.21.15.46:0/3685100494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d11e3560 con 0x5610d12db660 2024-05-08T23:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.258+0000 7f0a37e00700 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.258+0000 7f0a37400700 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0a28098900 tx=0x7f0a28098300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6657c0807f4910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.258+0000 7f0a2d200700 1 -- 172.21.15.46:0/3685100494 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2809cea0 con 0x5610d12db660 2024-05-08T23:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.258+0000 7f0a2d200700 1 -- 172.21.15.46:0/3685100494 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2809be90 con 0x5610d12db660 2024-05-08T23:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.259+0000 7f0a2d200700 1 -- 172.21.15.46:0/3685100494 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280cc8d0 con 0x5610d12db660 2024-05-08T23:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/3685100494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 msgr2=0x5610d12c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0a28098900 tx=0x7f0a28098300 comp rx=0 tx=0).stop 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/3685100494 shutdown_connections 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12db660 0x5610d12c7d20 secure :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0a28098900 tx=0x7f0a28098300 comp rx=0 tx=0).stop 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/3685100494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12dbf80 0x5610d12dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/3685100494 >> 172.21.15.46:0/3685100494 conn(0x5610d122b210 msgr2=0x5610d12ddca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.260+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/3685100494 shutdown_connections 2024-05-08T23:21:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.262+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/3685100494 wait complete. 2024-05-08T23:21:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.262+0000 7f0a51e0cb80 1 -- start start 2024-05-08T23:21:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.263+0000 7f0a51e0cb80 1 Processor -- start 2024-05-08T23:21:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.264+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.264+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a51e0cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12e54b0 0x5610d12e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.264 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81731 19.69 MB/sec execute 89 sec latency 537.406 ms 2024-05-08T23:21:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d12e7110 con 0x5610d12e54b0 2024-05-08T23:21:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d12e7280 con 0x5610d12c4210 2024-05-08T23:21:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a51e0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d1345b80 con 0x5610d12dbf80 2024-05-08T23:21:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a51e0cb80 10 client.0 _ll_register_callbacks cb 0x5610d11c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.265+0000 7f0a37400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.266+0000 7f0a37400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37970/0 (socket says 172.21.15.46:37970) 2024-05-08T23:21:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.266+0000 7f0a37400700 1 -- 172.21.15.46:0/1447187230 learned_addr learned my addr 172.21.15.46:0/1447187230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.266+0000 7f0a36a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 -- 172.21.15.46:0/1447187230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 msgr2=0x5610d12e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 -- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12e54b0 msgr2=0x5610d12e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12e54b0 0x5610d12e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 -- 172.21.15.46:0/1447187230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280b3070 con 0x5610d12c4210 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a37400700 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a28005f50 tx=0x7f0a2800f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc602924e9012d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a20c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2809cea0 con 0x5610d12c4210 2024-05-08T23:21:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.267+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0a280a4280 con 0x5610d12c4210 2024-05-08T23:21:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.268+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a2400ce30 con 0x5610d12c4210 2024-05-08T23:21:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.268+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a280cc8d0 con 0x5610d12c4210 2024-05-08T23:21:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.268+0000 7f0a51e0cb80 10 client.30116 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.268+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5610d10a2cd0 con 0x5610d12c4210 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2400ce30 con 0x5610d12c4210 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a28097c80 con 0x5610d12c4210 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 10 client.30116 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 10 client.30116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 20 client.30116 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d1346e30 0x5610d1349220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 4 client.30116 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.269+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610d1215ec0 con 0x5610d1346e30 2024-05-08T23:21:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.270+0000 7f0a36a00700 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d1346e30 0x5610d1349220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.275+0000 7f0a20c00700 10 client.30116 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.275+0000 7f0a36a00700 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d1346e30 0x5610d1349220 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=463b889fc4206b3e server_cookie=60bf16623fb249cf in_seq=0 out_seq=0 2024-05-08T23:21:08.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.278+0000 7f0a20c00700 1 -- 172.21.15.46:0/1447187230 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610d1215ec0 con 0x5610d1346e30 2024-05-08T23:21:08.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.278+0000 7f0a20c00700 10 client.30116 handle_command_reply: tid=0 2024-05-08T23:21:08.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.278+0000 7f0a51e0cb80 1 client.30116 shutdown 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 msgr2=0x5610d12e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a28005f50 tx=0x7f0a2800f840 comp rx=0 tx=0).stop 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 shutdown_connections 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5610d1346e30 0x5610d1349220 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d12e54b0 0x5610d12e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610d12c4210 0x5610d12e69e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 --2- 172.21.15.46:0/1447187230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610d12dbf80 0x5610d12e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a20c00700 0 client.30116 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:08.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.279+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 >> 172.21.15.46:0/1447187230 conn(0x5610d122b210 msgr2=0x5610d12cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.280+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 shutdown_connections 2024-05-08T23:21:08.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.280+0000 7f0a51e0cb80 1 -- 172.21.15.46:0/1447187230 wait complete. 2024-05-08T23:21:08.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.281+0000 7f0a51e0cb80 20 client.30116 trim_cache size 0 max 16384 2024-05-08T23:21:08.281 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:08.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.282+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 msgr2=0x7f0a1c0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.282+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 0x7f0a1c0a3960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806a500 tx=0x7f0a30004ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.282+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 msgr2=0x7f0a381a7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.282+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0a28003b60 tx=0x7f0a28003c40 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 shutdown_connections 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0a1c0a14a0 0x7f0a1c0a3960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a381140f0 0x7f0a381a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38107fa0 0x7f0a381a7980 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 --2- 172.21.15.46:0/594699377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a381088c0 0x7f0a381a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 >> 172.21.15.46:0/594699377 conn(0x7f0a3806bf30 msgr2=0x7f0a38113150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 shutdown_connections 2024-05-08T23:21:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.283+0000 7f0a3e200700 1 -- 172.21.15.46:0/594699377 wait complete. 2024-05-08T23:21:08.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.698+0000 7ff550a00700 1 -- 172.21.15.46:0/2738462503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c10d270 msgr2=0x7ff54c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.698+0000 7ff550a00700 1 --2- 172.21.15.46:0/2738462503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c10d270 0x7ff54c10d650 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.699+0000 7ff550a00700 1 -- 172.21.15.46:0/2738462503 shutdown_connections 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.699+0000 7ff550a00700 1 --2- 172.21.15.46:0/2738462503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c1084e0 0x7ff54c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.699+0000 7ff550a00700 1 --2- 172.21.15.46:0/2738462503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c10d270 0x7ff54c10d650 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.699+0000 7ff550a00700 1 --2- 172.21.15.46:0/2738462503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10db90 0x7ff54c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.699+0000 7ff550a00700 1 -- 172.21.15.46:0/2738462503 >> 172.21.15.46:0/2738462503 conn(0x7ff54c072c90 msgr2=0x7ff54c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.701+0000 7ff550a00700 1 -- 172.21.15.46:0/2738462503 shutdown_connections 2024-05-08T23:21:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.702+0000 7ff550a00700 1 -- 172.21.15.46:0/2738462503 wait complete. 2024-05-08T23:21:08.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.704+0000 7ff550a00700 1 Processor -- start 2024-05-08T23:21:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.704+0000 7ff550a00700 1 -- start start 2024-05-08T23:21:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c1084e0 0x7ff54c1a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 0x7ff54c1aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1ab050 con 0x7ff54c10db90 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1ab1c0 con 0x7ff54c10d270 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.708+0000 7ff550a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1ab380 con 0x7ff54c1084e0 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c1084e0 0x7ff54c1a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37982/0 (socket says 172.21.15.46:37982) 2024-05-08T23:21:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 -- 172.21.15.46:0/3359206269 learned_addr learned my addr 172.21.15.46:0/3359206269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff54a800700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 0x7ff54c1aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 -- 172.21.15.46:0/3359206269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c1084e0 msgr2=0x7ff54c1a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c1084e0 0x7ff54c1a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 -- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 msgr2=0x7ff54c1aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 0x7ff54c1aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff549400700 1 -- 172.21.15.46:0/3359206269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff54c10d270 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.709+0000 7ff54a800700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 0x7ff54c1aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff549400700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d950 tx=0x7ff53800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff542a00700 1 -- 172.21.15.46:0/3359206269 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538009940 con 0x7ff54c10d270 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff542a00700 1 -- 172.21.15.46:0/3359206269 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff538010460 con 0x7ff54c10d270 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff542a00700 1 -- 172.21.15.46:0/3359206269 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800f620 con 0x7ff54c10d270 2024-05-08T23:21:08.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1ab660 con 0x7ff54c10d270 2024-05-08T23:21:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.710+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1b2ea0 con 0x7ff54c10d270 2024-05-08T23:21:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.719+0000 7ff564431b80 1 Processor -- start 2024-05-08T23:21:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.719+0000 7ff564431b80 1 -- start start 2024-05-08T23:21:08.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.721+0000 7ff542a00700 1 -- 172.21.15.46:0/3359206269 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff5380108d0 con 0x7ff54c10d270 2024-05-08T23:21:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.722+0000 7ff564431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff564431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b501492350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff564431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b501492cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff564431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5013e9150 con 0x55b501498720 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff564431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5013e6b10 con 0x55b501497e00 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff564431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5013e2400 con 0x55b501492890 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.723+0000 7ff54a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.724+0000 7ff542a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 0x7ff5300a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.724+0000 7ff54a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:37998/0 (socket says 172.21.15.46:37998) 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.724+0000 7ff54a800700 1 -- 172.21.15.46:0/2765527654 learned_addr learned my addr 172.21.15.46:0/2765527654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.724+0000 7ff542a00700 1 -- 172.21.15.46:0/3359206269 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff5380cd0f0 con 0x7ff54c10d270 2024-05-08T23:21:08.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.724+0000 7ff549e00700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b501492350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 0x7ff5300a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 -- 172.21.15.46:0/2765527654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 msgr2=0x55b501492cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff549400700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b501492cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b501492cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 -- 172.21.15.46:0/2765527654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 msgr2=0x55b501492350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b501492350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.726+0000 7ff54a800700 1 -- 172.21.15.46:0/2765527654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5012633b0 con 0x55b501497e00 2024-05-08T23:21:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.727+0000 7ff549400700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b501492cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.727+0000 7ff549e00700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b501492350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.728+0000 7ff54a800700 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff544010a00 tx=0x7ff544010d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326e8cc3a5fd720a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.728+0000 7ff54a800700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 0x7ff5300a3850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e660 tx=0x7ff544018000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.729+0000 7ff537000700 1 -- 172.21.15.46:0/2765527654 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544010050 con 0x55b501497e00 2024-05-08T23:21:08.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.729+0000 7ff537000700 1 -- 172.21.15.46:0/2765527654 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544016070 con 0x55b501497e00 2024-05-08T23:21:08.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.729+0000 7ff537000700 1 -- 172.21.15.46:0/2765527654 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401a6a0 con 0x55b501497e00 2024-05-08T23:21:08.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.730+0000 7ff564431b80 1 -- 172.21.15.46:0/2765527654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 msgr2=0x55b5014981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.730+0000 7ff564431b80 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff544010a00 tx=0x7ff544010d10 comp rx=0 tx=0).stop 2024-05-08T23:21:08.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.731+0000 7ff564431b80 1 -- 172.21.15.46:0/2765527654 shutdown_connections 2024-05-08T23:21:08.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.731+0000 7ff564431b80 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b501492cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.731+0000 7ff564431b80 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501497e00 0x55b5014981e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.731+0000 7ff564431b80 1 --2- 172.21.15.46:0/2765527654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b501492350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.731+0000 7ff564431b80 1 -- 172.21.15.46:0/2765527654 >> 172.21.15.46:0/2765527654 conn(0x55b5013f0210 msgr2=0x55b50148cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.732+0000 7ff564431b80 1 -- 172.21.15.46:0/2765527654 shutdown_connections 2024-05-08T23:21:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.732+0000 7ff564431b80 1 -- 172.21.15.46:0/2765527654 wait complete. 2024-05-08T23:21:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.733+0000 7ff564431b80 1 -- start start 2024-05-08T23:21:08.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.733+0000 7ff564431b80 1 Processor -- start 2024-05-08T23:21:08.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.735+0000 7ff564431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.735+0000 7ff564431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.735+0000 7ff564431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501498720 0x55b50149c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff564431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50137e560 con 0x55b501497e00 2024-05-08T23:21:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff564431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50137e200 con 0x55b501492890 2024-05-08T23:21:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff564431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5012656a0 con 0x55b501498720 2024-05-08T23:21:08.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff564431b80 10 client.0 _ll_register_callbacks cb 0x55b501387b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:08.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff549e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff549e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:38004/0 (socket says 172.21.15.46:38004) 2024-05-08T23:21:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.736+0000 7ff549e00700 1 -- 172.21.15.46:0/3711465559 learned_addr learned my addr 172.21.15.46:0/3711465559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.737+0000 7ff549400700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.737+0000 7ff549e00700 1 -- 172.21.15.46:0/3711465559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501498720 msgr2=0x55b50149c550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.737+0000 7ff549e00700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501498720 0x55b50149c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.737+0000 7ff549e00700 1 -- 172.21.15.46:0/3711465559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 msgr2=0x55b50149c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.740+0000 7ff549e00700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.740+0000 7ff549e00700 1 -- 172.21.15.46:0/3711465559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c10c520 con 0x55b501492890 2024-05-08T23:21:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.740+0000 7ff549400700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.745+0000 7ff549e00700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000c00 tx=0x7ff53c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8a25b03138f7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.745+0000 7ff52b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.745+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c01d070 con 0x55b501492890 2024-05-08T23:21:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.746+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff53c0037c0 con 0x55b501492890 2024-05-08T23:21:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.746+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50133b270 con 0x55b501492890 2024-05-08T23:21:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.746+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c0056d0 con 0x55b501492890 2024-05-08T23:21:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.746+0000 7ff564431b80 10 client.30128 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.746+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7ff544010640 con 0x55b501492890 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b50133b270 con 0x55b501492890 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff53c00faf0 con 0x55b501492890 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 10 client.30128 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 10 client.30128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 20 client.30128 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b5014ac100 0x55b5014ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 4 client.30128 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b501461810 con 0x55b5014ac100 2024-05-08T23:21:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.752+0000 7ff549400700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b5014ac100 0x55b5014ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.753+0000 7ff549400700 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b5014ac100 0x55b5014ae520 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeecff005702bedc server_cookie=539a5da2a88a94d3 in_seq=0 out_seq=0 2024-05-08T23:21:08.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.753+0000 7ff52b400700 10 client.30128 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:08.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.754+0000 7ff52b400700 1 -- 172.21.15.46:0/3711465559 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b501461810 con 0x55b5014ac100 2024-05-08T23:21:08.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.754+0000 7ff52b400700 10 client.30128 handle_command_reply: tid=0 2024-05-08T23:21:08.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 client.30128 shutdown 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 msgr2=0x55b50149bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000c00 tx=0x7ff53c005f00 comp rx=0 tx=0).stop 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 shutdown_connections 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501498720 0x55b50149c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b50149bad0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b5014ac100 0x55b5014ae520 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 --2- 172.21.15.46:0/3711465559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 >> 172.21.15.46:0/3711465559 conn(0x55b5013f0210 msgr2=0x55b50148c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.755+0000 7ff52b400700 0 client.30128 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.756+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 shutdown_connections 2024-05-08T23:21:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.756+0000 7ff564431b80 1 -- 172.21.15.46:0/3711465559 wait complete. 2024-05-08T23:21:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.757+0000 7ff564431b80 20 client.30128 trim_cache size 0 max 16384 2024-05-08T23:21:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 Processor -- start 2024-05-08T23:21:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 -- start start 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b5014a8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501497e00 0x55b50148cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50148de80 con 0x55b501498720 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50148e070 con 0x55b501492890 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.786+0000 7ff564431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5014b3750 con 0x55b501497e00 2024-05-08T23:21:08.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.787+0000 7ff54a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.787+0000 7ff54a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57844/0 (socket says 172.21.15.46:57844) 2024-05-08T23:21:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.787+0000 7ff54a800700 1 -- 172.21.15.46:0/3176389300 learned_addr learned my addr 172.21.15.46:0/3176389300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.788+0000 7ff54a800700 1 -- 172.21.15.46:0/3176389300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501497e00 msgr2=0x55b50148cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.788+0000 7ff54a800700 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501497e00 0x55b50148cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.788+0000 7ff54a800700 1 -- 172.21.15.46:0/3176389300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 msgr2=0x55b5014a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.788+0000 7ff54a800700 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b5014a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.788+0000 7ff54a800700 1 -- 172.21.15.46:0/3176389300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b50120fb60 con 0x55b501498720 2024-05-08T23:21:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.789+0000 7ff54a800700 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff544004b10 tx=0x7ff544010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe330abf9968544 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.789+0000 7ff537a00700 1 -- 172.21.15.46:0/3176389300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544028070 con 0x55b501498720 2024-05-08T23:21:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.790+0000 7ff537a00700 1 -- 172.21.15.46:0/3176389300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544007400 con 0x55b501498720 2024-05-08T23:21:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.790+0000 7ff537a00700 1 -- 172.21.15.46:0/3176389300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544008b70 con 0x55b501498720 2024-05-08T23:21:08.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 -- 172.21.15.46:0/3176389300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 msgr2=0x55b50149c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff544004b10 tx=0x7ff544010590 comp rx=0 tx=0).stop 2024-05-08T23:21:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 -- 172.21.15.46:0/3176389300 shutdown_connections 2024-05-08T23:21:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501498720 0x55b50149c370 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501492890 0x55b5014a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 --2- 172.21.15.46:0/3176389300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501497e00 0x55b50148cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.791+0000 7ff564431b80 1 -- 172.21.15.46:0/3176389300 >> 172.21.15.46:0/3176389300 conn(0x55b5013f0210 msgr2=0x55b501492c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.792+0000 7ff564431b80 1 -- 172.21.15.46:0/3176389300 shutdown_connections 2024-05-08T23:21:08.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.793+0000 7ff564431b80 1 -- 172.21.15.46:0/3176389300 wait complete. 2024-05-08T23:21:08.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.793+0000 7ff564431b80 1 -- start start 2024-05-08T23:21:08.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.794+0000 7ff564431b80 1 Processor -- start 2024-05-08T23:21:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.795+0000 7ff564431b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501498720 0x55b50149f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50149fda0 con 0x55b501497e00 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50149ff10 con 0x55b501498720 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b501501610 con 0x55b501492890 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff564431b80 10 client.0 _ll_register_callbacks cb 0x55b501387b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff549e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff549e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:38988/0 (socket says 172.21.15.46:38988) 2024-05-08T23:21:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.796+0000 7ff549e00700 1 -- 172.21.15.46:0/958066317 learned_addr learned my addr 172.21.15.46:0/958066317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549400700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549e00700 1 -- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501498720 msgr2=0x55b50149f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549e00700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501498720 0x55b50149f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549e00700 1 -- 172.21.15.46:0/958066317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 msgr2=0x55b50149f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549e00700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.797+0000 7ff549e00700 1 -- 172.21.15.46:0/958066317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5012633b0 con 0x55b501492890 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.798+0000 7ff549e00700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff53c011040 tx=0x7ff53c009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4548cc544a7090 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.798+0000 7ff52b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.798+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c01d040 con 0x55b501492890 2024-05-08T23:21:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.798+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff53c000e40 con 0x55b501492890 2024-05-08T23:21:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.799+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff544016070 con 0x55b501492890 2024-05-08T23:21:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.801+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c0246a0 con 0x55b501492890 2024-05-08T23:21:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.801+0000 7ff564431b80 10 client.30205 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.801+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b5013a8560 con 0x55b501492890 2024-05-08T23:21:08.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544016070 con 0x55b501492890 2024-05-08T23:21:08.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff53c02e070 con 0x55b501492890 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 10 client.30205 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 10 client.30205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 20 client.30205 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b501502920 0x55b501504d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 4 client.30205 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.802+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5013daec0 con 0x55b501502920 2024-05-08T23:21:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.803+0000 7ff549400700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b501502920 0x55b501504d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.807+0000 7ff52b400700 10 client.30205 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.807+0000 7ff549400700 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b501502920 0x55b501504d40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440eccd1bf17b577 server_cookie=34872441d69efa42 in_seq=0 out_seq=0 2024-05-08T23:21:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.807+0000 7ff52b400700 1 -- 172.21.15.46:0/958066317 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5013daec0 con 0x55b501502920 2024-05-08T23:21:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.807+0000 7ff52b400700 10 client.30205 handle_command_reply: tid=0 2024-05-08T23:21:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.807+0000 7ff564431b80 1 client.30205 shutdown 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 msgr2=0x55b5014a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff53c011040 tx=0x7ff53c009f50 comp rx=0 tx=0).stop 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 shutdown_connections 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b501498720 0x55b50149f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b501492890 0x55b5014a34f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b501502920 0x55b501504d40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 --2- 172.21.15.46:0/958066317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b501497e00 0x55b50149f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 >> 172.21.15.46:0/958066317 conn(0x55b5013f0210 msgr2=0x55b50149c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff52b400700 0 client.30205 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 shutdown_connections 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.808+0000 7ff564431b80 1 -- 172.21.15.46:0/958066317 wait complete. 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.809+0000 7ff564431b80 20 client.30205 trim_cache size 0 max 16384 2024-05-08T23:21:08.809 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 msgr2=0x7ff5300a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 0x7ff5300a3850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e660 tx=0x7ff544018000 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 msgr2=0x7ff54c1a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d950 tx=0x7ff53800dc60 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 shutdown_connections 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff5300a1390 0x7ff5300a3850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c10db90 0x7ff54c1aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54c1084e0 0x7ff54c1a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 --2- 172.21.15.46:0/3359206269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff54c10d270 0x7ff54c1a6670 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 >> 172.21.15.46:0/3359206269 conn(0x7ff54c072c90 msgr2=0x7ff54c109ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 shutdown_connections 2024-05-08T23:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:08.811+0000 7ff550a00700 1 -- 172.21.15.46:0/3359206269 wait complete. 2024-05-08T23:21:09.265 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81911 19.60 MB/sec execute 90 sec latency 493.044 ms 2024-05-08T23:21:09.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:09 smithi053 ceph-mon[77587]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-05-08T23:21:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:09 smithi046 ceph-mon[36898]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-05-08T23:21:09.847 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:21:09.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:21:09.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:09 smithi086 ceph-mon[78610]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-05-08T23:21:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.207+0000 7f2630200700 1 -- 172.21.15.46:0/1035268643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 msgr2=0x7f2628114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.207+0000 7f2630200700 1 --2- 172.21.15.46:0/1035268643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628114030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2620009990 tx=0x7f2620009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 -- 172.21.15.46:0/1035268643 shutdown_connections 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 --2- 172.21.15.46:0/1035268643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628114030 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 --2- 172.21.15.46:0/1035268643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280fef20 0x7f26280ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 --2- 172.21.15.46:0/1035268643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280ff840 0x7f2628111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 -- 172.21.15.46:0/1035268643 >> 172.21.15.46:0/1035268643 conn(0x7f2628075eb0 msgr2=0x7f26280fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.208+0000 7f2630200700 1 -- 172.21.15.46:0/1035268643 shutdown_connections 2024-05-08T23:21:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.209+0000 7f2630200700 1 -- 172.21.15.46:0/1035268643 wait complete. 2024-05-08T23:21:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.212+0000 7f2630200700 1 Processor -- start 2024-05-08T23:21:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.212+0000 7f2630200700 1 -- start start 2024-05-08T23:21:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.214+0000 7f2630200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f2630200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280ff840 0x7f262810ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f2630200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f262d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f262d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39008/0 (socket says 172.21.15.46:39008) 2024-05-08T23:21:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f2630200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26281082a0 con 0x7f26280ff840 2024-05-08T23:21:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f2630200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2628108410 con 0x7f2628111c40 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f2630200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26281085d0 con 0x7f26280fef20 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f262d800700 1 -- 172.21.15.46:0/40368180 learned_addr learned my addr 172.21.15.46:0/40368180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.215+0000 7f262ce00700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280ff840 0x7f262810ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262e200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262d800700 1 -- 172.21.15.46:0/40368180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 msgr2=0x7f2628107ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262d800700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628107ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262d800700 1 -- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280ff840 msgr2=0x7f262810ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262d800700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280ff840 0x7f262810ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262d800700 1 -- 172.21.15.46:0/40368180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200095f0 con 0x7f26280fef20 2024-05-08T23:21:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.216+0000 7f262e200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.217+0000 7f262d800700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b530 tx=0x7f261c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.217+0000 7f2626000700 1 -- 172.21.15.46:0/40368180 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c011830 con 0x7f26280fef20 2024-05-08T23:21:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.217+0000 7f2626000700 1 -- 172.21.15.46:0/40368180 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f261c011e50 con 0x7f26280fef20 2024-05-08T23:21:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.217+0000 7f2626000700 1 -- 172.21.15.46:0/40368180 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00f2e0 con 0x7f26280fef20 2024-05-08T23:21:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.218+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26281088b0 con 0x7f26280fef20 2024-05-08T23:21:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.219+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262810c830 con 0x7f26280fef20 2024-05-08T23:21:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.229+0000 7f2626000700 1 -- 172.21.15.46:0/40368180 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f261c010580 con 0x7f26280fef20 2024-05-08T23:21:10.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.231+0000 7f2626000700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 0x7f26100a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.231+0000 7f2626000700 1 -- 172.21.15.46:0/40368180 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f261c0ccae0 con 0x7f26280fef20 2024-05-08T23:21:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.231+0000 7f2643c7bb80 1 Processor -- start 2024-05-08T23:21:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 -- start start 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f262ce00700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 0x7f26100a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bededc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556beded63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 0x556beded6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bede2d1d0 con 0x556bededbe80 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bede2ab90 con 0x556beded6910 2024-05-08T23:21:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.232+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bede26480 con 0x556bededc7a0 2024-05-08T23:21:10.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.233+0000 7f262d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bededc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.233+0000 7f262d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bededc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57890/0 (socket says 172.21.15.46:57890) 2024-05-08T23:21:10.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.233+0000 7f262d800700 1 -- 172.21.15.46:0/1071756867 learned_addr learned my addr 172.21.15.46:0/1071756867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.233+0000 7f262e200700 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 0x556beded6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262ce00700 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556beded63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262e200700 1 -- 172.21.15.46:0/1071756867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 msgr2=0x556beded63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262e200700 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556beded63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262e200700 1 -- 172.21.15.46:0/1071756867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 msgr2=0x556bededc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262e200700 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bededc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.234+0000 7f262e200700 1 -- 172.21.15.46:0/1071756867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bedca7430 con 0x556beded6910 2024-05-08T23:21:10.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.238+0000 7f262ce00700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 0x7f26100a3850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2614007fd0 tx=0x7f2614012b50 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.240+0000 7f262e200700 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 0x556beded6d50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f26200098e0 tx=0x7f2620009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f0f8a78ee5c60ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.241+0000 7f261aa00700 1 -- 172.21.15.46:0/1071756867 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262001d040 con 0x556beded6910 2024-05-08T23:21:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.241+0000 7f261aa00700 1 -- 172.21.15.46:0/1071756867 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f262000b9d0 con 0x556beded6910 2024-05-08T23:21:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.241+0000 7f261aa00700 1 -- 172.21.15.46:0/1071756867 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620003cf0 con 0x556beded6910 2024-05-08T23:21:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 -- 172.21.15.46:0/1071756867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 msgr2=0x556beded6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 0x556beded6d50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f26200098e0 tx=0x7f2620009ee0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 -- 172.21.15.46:0/1071756867 shutdown_connections 2024-05-08T23:21:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded6910 0x556beded6d50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bededc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/1071756867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556beded63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.242+0000 7f2643c7bb80 1 -- 172.21.15.46:0/1071756867 >> 172.21.15.46:0/1071756867 conn(0x556bede34290 msgr2=0x556beded0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.243+0000 7f2643c7bb80 1 -- 172.21.15.46:0/1071756867 shutdown_connections 2024-05-08T23:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.243+0000 7f2643c7bb80 1 -- 172.21.15.46:0/1071756867 wait complete. 2024-05-08T23:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.243+0000 7f2643c7bb80 1 -- start start 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.244+0000 7f2643c7bb80 1 Processor -- start 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededbe80 0x556bededfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 0x556bedee0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556beddc25e0 con 0x556beded6910 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556beddc2280 con 0x556bededc7a0 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedca9720 con 0x556bededbe80 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f2643c7bb80 10 client.0 _ll_register_callbacks cb 0x556beddcbbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f262d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f262d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57898/0 (socket says 172.21.15.46:57898) 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.245+0000 7f262d800700 1 -- 172.21.15.46:0/431617056 learned_addr learned my addr 172.21.15.46:0/431617056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.246+0000 7f262e200700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 0x556bedee0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.246+0000 7f262ce00700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededbe80 0x556bededfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 -- 172.21.15.46:0/431617056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededbe80 msgr2=0x556bededfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededbe80 0x556bededfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 -- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 msgr2=0x556bedee0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 0x556bedee0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 -- 172.21.15.46:0/431617056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2628114b20 con 0x556beded6910 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262e200700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 0x556bedee0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f262d800700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f261c099590 tx=0x7f261c099450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89017101d740c702 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f260f400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.247+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c010380 con 0x556beded6910 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.248+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f261c099f90 con 0x556beded6910 2024-05-08T23:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.248+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bedd7f2f0 con 0x556beded6910 2024-05-08T23:21:10.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f261c0b2200 con 0x556beded6910 2024-05-08T23:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 10 client.19974 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556bedc53be0 con 0x556beded6910 2024-05-08T23:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bedd7f2f0 con 0x556beded6910 2024-05-08T23:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f261c0b90b0 con 0x556beded6910 2024-05-08T23:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 10 client.19974 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 10 client.19974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 20 client.19974 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedef00c0 0x556bedef24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 4 client.19974 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.249+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bedea5890 con 0x556bedef00c0 2024-05-08T23:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.250+0000 7f262ce00700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedef00c0 0x556bedef24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.251+0000 7f262ce00700 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedef00c0 0x556bedef24e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85156ac46e0d227 server_cookie=46807c1fa0a94e85 in_seq=0 out_seq=0 2024-05-08T23:21:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.251+0000 7f260f400700 10 client.19974 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.251+0000 7f260f400700 1 -- 172.21.15.46:0/431617056 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bedea5890 con 0x556bedef00c0 2024-05-08T23:21:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.251+0000 7f260f400700 10 client.19974 handle_command_reply: tid=0 2024-05-08T23:21:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.251+0000 7f2643c7bb80 1 client.19974 shutdown 2024-05-08T23:21:10.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 msgr2=0x556bededfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f261c099590 tx=0x7f261c099450 comp rx=0 tx=0).stop 2024-05-08T23:21:10.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 shutdown_connections 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556bededc7a0 0x556bedee0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedef00c0 0x556bedef24e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556beded6910 0x556bededfab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/431617056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededbe80 0x556bededfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 >> 172.21.15.46:0/431617056 conn(0x556bede34290 msgr2=0x556beded0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f260f400700 0 client.19974 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 shutdown_connections 2024-05-08T23:21:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.252+0000 7f2643c7bb80 1 -- 172.21.15.46:0/431617056 wait complete. 2024-05-08T23:21:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.253+0000 7f2643c7bb80 20 client.19974 trim_cache size 0 max 16384 2024-05-08T23:21:10.265 INFO:tasks.workunit.client.0.smithi046.stdout: 10 82152 19.54 MB/sec execute 91 sec latency 353.257 ms 2024-05-08T23:21:10.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 Processor -- start 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 -- start start 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556beded63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedef7750 con 0x556bededbe80 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556beded2860 con 0x556beded0da0 2024-05-08T23:21:10.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.283+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedecb880 con 0x556bededc7a0 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39036/0 (socket says 172.21.15.46:39036) 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 -- 172.21.15.46:0/327579436 learned_addr learned my addr 172.21.15.46:0/327579436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262e200700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262d800700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556beded63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 -- 172.21.15.46:0/327579436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 msgr2=0x556bedee0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 -- 172.21.15.46:0/327579436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 msgr2=0x556beded63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556beded63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262ce00700 1 -- 172.21.15.46:0/327579436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556beddec5e0 con 0x556bededc7a0 2024-05-08T23:21:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.284+0000 7f262d800700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556beded63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.285+0000 7f262ce00700 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2628067860 tx=0x7f2614012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68661d856ea18c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.286+0000 7f261b400700 1 -- 172.21.15.46:0/327579436 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614015d60 con 0x556bededc7a0 2024-05-08T23:21:10.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.286+0000 7f261b400700 1 -- 172.21.15.46:0/327579436 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2614015ec0 con 0x556bededc7a0 2024-05-08T23:21:10.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.286+0000 7f261b400700 1 -- 172.21.15.46:0/327579436 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614015d60 con 0x556bededc7a0 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.287+0000 7f2643c7bb80 1 -- 172.21.15.46:0/327579436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 msgr2=0x556bededea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.287+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2628067860 tx=0x7f2614012f90 comp rx=0 tx=0).stop 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.288+0000 7f2643c7bb80 1 -- 172.21.15.46:0/327579436 shutdown_connections 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.288+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.288+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556beded63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.288+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/327579436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bededea00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.288+0000 7f2643c7bb80 1 -- 172.21.15.46:0/327579436 >> 172.21.15.46:0/327579436 conn(0x556bede34290 msgr2=0x556bededd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.289+0000 7f2643c7bb80 1 -- 172.21.15.46:0/327579436 shutdown_connections 2024-05-08T23:21:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.289+0000 7f2643c7bb80 1 -- 172.21.15.46:0/327579436 wait complete. 2024-05-08T23:21:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.289+0000 7f2643c7bb80 1 -- start start 2024-05-08T23:21:10.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.290+0000 7f2643c7bb80 1 Processor -- start 2024-05-08T23:21:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.291+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.291+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bedee5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.291+0000 7f2643c7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedee5de0 con 0x556bededbe80 2024-05-08T23:21:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedee5f50 con 0x556beded0da0 2024-05-08T23:21:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f2643c7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bedf475f0 con 0x556bededc7a0 2024-05-08T23:21:10.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f2643c7bb80 10 client.0 _ll_register_callbacks cb 0x556beddcbbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39046/0 (socket says 172.21.15.46:39046) 2024-05-08T23:21:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 -- 172.21.15.46:0/4185805204 learned_addr learned my addr 172.21.15.46:0/4185805204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262d800700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262ce00700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bedee5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 -- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 msgr2=0x556bedee7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 -- 172.21.15.46:0/4185805204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 msgr2=0x556bedee5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bedee5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.292+0000 7f262e200700 1 -- 172.21.15.46:0/4185805204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261401f040 con 0x556bededc7a0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f262d800700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f262ce00700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bedee5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f262e200700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2620003940 tx=0x7f2620022760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4383ce177690e525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f260f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262001d040 con 0x556bededc7a0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2620022b90 con 0x556bededc7a0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bede1ef40 con 0x556bededc7a0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2620003790 con 0x556bededc7a0 2024-05-08T23:21:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f2643c7bb80 10 client.30220 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.293+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556bedcabd50 con 0x556bededc7a0 2024-05-08T23:21:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bede1ef40 con 0x556bededc7a0 2024-05-08T23:21:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f262000f920 con 0x556bededc7a0 2024-05-08T23:21:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 10 client.30220 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 10 client.30220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 20 client.30220 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedf48850 0x556bedf4ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 4 client.30220 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.294+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556bedc74d70 con 0x556bedf48850 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.295+0000 7f262d800700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedf48850 0x556bedf4ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.296+0000 7f262d800700 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedf48850 0x556bedf4ac40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec40d70bf8cd11f1 server_cookie=9787c25fc5b7b6ba in_seq=0 out_seq=0 2024-05-08T23:21:10.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.296+0000 7f260f400700 10 client.30220 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.297+0000 7f260f400700 1 -- 172.21.15.46:0/4185805204 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556bedc74d70 con 0x556bedf48850 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.297+0000 7f260f400700 10 client.30220 handle_command_reply: tid=0 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.297+0000 7f2643c7bb80 1 client.30220 shutdown 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.297+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 msgr2=0x556bedee56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.297+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2620003940 tx=0x7f2620022760 comp rx=0 tx=0).stop 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 shutdown_connections 2024-05-08T23:21:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bededc7a0 0x556bedee56d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556beded0da0 0x556bedee7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556bedf48850 0x556bedf4ac40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 --2- 172.21.15.46:0/4185805204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bededbe80 0x556bedee5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 >> 172.21.15.46:0/4185805204 conn(0x556bede34290 msgr2=0x556bededab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f260f400700 0 client.30220 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.298+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 shutdown_connections 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.299+0000 7f2643c7bb80 1 -- 172.21.15.46:0/4185805204 wait complete. 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.299+0000 7f2643c7bb80 20 client.30220 trim_cache size 0 max 16384 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5035d945-a8a2-44f8-b5f5-0e1887dfc288", 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:21:10.300 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.300+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 msgr2=0x7f26100a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.300+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 0x7f26100a3850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2614007fd0 tx=0x7f2614012b50 comp rx=0 tx=0).stop 2024-05-08T23:21:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 msgr2=0x7f262810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b530 tx=0x7f261c00b8f0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 shutdown_connections 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2628111c40 0x7f2628107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26280fef20 0x7f262810ea00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f26100a1390 0x7f26100a3850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 --2- 172.21.15.46:0/40368180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26280ff840 0x7f262810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 >> 172.21.15.46:0/40368180 conn(0x7f2628075eb0 msgr2=0x7f2628106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 shutdown_connections 2024-05-08T23:21:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.301+0000 7f2630200700 1 -- 172.21.15.46:0/40368180 wait complete. 2024-05-08T23:21:10.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 -- 172.21.15.46:0/1052788298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba01008f0 msgr2=0x7f8ba0104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1052788298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba01008f0 0x7f8ba0104930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009990 tx=0x7f8b8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 -- 172.21.15.46:0/1052788298 shutdown_connections 2024-05-08T23:21:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1052788298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1052788298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1052788298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba01008f0 0x7f8ba0104930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 -- 172.21.15.46:0/1052788298 >> 172.21.15.46:0/1052788298 conn(0x7f8ba00fb760 msgr2=0x7f8ba00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 -- 172.21.15.46:0/1052788298 shutdown_connections 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.746+0000 7f8ba7000700 1 -- 172.21.15.46:0/1052788298 wait complete. 2024-05-08T23:21:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.748+0000 7f8ba7000700 1 Processor -- start 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.748+0000 7f8ba7000700 1 -- start start 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba01955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 0x7f8ba0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba019a3f0 con 0x7f8ba00fff40 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba019a560 con 0x7f8ba0199970 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba019a720 con 0x7f8ba0105060 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8b9fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba01955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8b9fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57932/0 (socket says 172.21.15.46:57932) 2024-05-08T23:21:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8b9fe00700 1 -- 172.21.15.46:0/1293300577 learned_addr learned my addr 172.21.15.46:0/1293300577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.749+0000 7f8ba5000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 0x7f8ba0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 -- 172.21.15.46:0/1293300577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 msgr2=0x7f8ba0199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 0x7f8ba0199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 -- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 msgr2=0x7f8ba0195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 -- 172.21.15.46:0/1293300577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9000d930 con 0x7f8ba0105060 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8ba5000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 0x7f8ba0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.750+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba01955d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.751+0000 7f8b9cc00700 1 -- 172.21.15.46:0/1293300577 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01d070 con 0x7f8ba0105060 2024-05-08T23:21:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.751+0000 7f8b9cc00700 1 -- 172.21.15.46:0/1293300577 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8c021410 con 0x7f8ba0105060 2024-05-08T23:21:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.751+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0095f0 con 0x7f8ba0105060 2024-05-08T23:21:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.751+0000 7f8b9cc00700 1 -- 172.21.15.46:0/1293300577 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c017650 con 0x7f8ba0105060 2024-05-08T23:21:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.751+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba01a1f10 con 0x7f8ba0105060 2024-05-08T23:21:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.754+0000 7f8b9cc00700 1 -- 172.21.15.46:0/1293300577 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8b8c021580 con 0x7f8ba0105060 2024-05-08T23:21:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.755+0000 7f8b9cc00700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 0x7f8b840a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.756+0000 7f8b9cc00700 1 -- 172.21.15.46:0/1293300577 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f8b8c0ce280 con 0x7f8ba0105060 2024-05-08T23:21:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.756+0000 7f8bbac07b80 1 Processor -- start 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.756+0000 7f8bbac07b80 1 -- start start 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.756+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 0x7f8b840a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557879b9150 con 0x555787a68720 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557879b6b10 con 0x555787a67e00 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557879b2400 con 0x555787a62890 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57938/0 (socket says 172.21.15.46:57938) 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 -- 172.21.15.46:0/1380192457 learned_addr learned my addr 172.21.15.46:0/1380192457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8ba5000700 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 -- 172.21.15.46:0/1380192457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 msgr2=0x555787a62cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a62cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 -- 172.21.15.46:0/1380192457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 msgr2=0x555787a681e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a681e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.757+0000 7f8b9f400700 1 -- 172.21.15.46:0/1380192457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557878333b0 con 0x555787a68720 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.758+0000 7f8b9f400700 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0a2e80 tx=0x7f8b8c09ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b51e7d9cf8ba3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.758+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 0x7f8b840a38a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000dfd0 tx=0x7f8b9000f040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.758+0000 7f8b95600700 1 -- 172.21.15.46:0/1380192457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a10c0 con 0x555787a68720 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.758+0000 7f8b95600700 1 -- 172.21.15.46:0/1380192457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8c09afc0 con 0x555787a68720 2024-05-08T23:21:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.758+0000 7f8b95600700 1 -- 172.21.15.46:0/1380192457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c021860 con 0x555787a68720 2024-05-08T23:21:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 -- 172.21.15.46:0/1380192457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 msgr2=0x555787a62350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0a2e80 tx=0x7f8b8c09ae60 comp rx=0 tx=0).stop 2024-05-08T23:21:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 -- 172.21.15.46:0/1380192457 shutdown_connections 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a62cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/1380192457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a62350 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 -- 172.21.15.46:0/1380192457 >> 172.21.15.46:0/1380192457 conn(0x5557879c0210 msgr2=0x555787a5cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 -- 172.21.15.46:0/1380192457 shutdown_connections 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.759+0000 7f8bbac07b80 1 -- 172.21.15.46:0/1380192457 wait complete. 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.760+0000 7f8bbac07b80 1 -- start start 2024-05-08T23:21:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.760+0000 7f8bbac07b80 1 Processor -- start 2024-05-08T23:21:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.761+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a6e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.761+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.761+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a6bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578794e560 con 0x555787a68720 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578794e200 con 0x555787a67e00 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557878356a0 con 0x555787a62890 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8bbac07b80 10 client.0 _ll_register_callbacks cb 0x555787957b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a6e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:38132/0 (socket says 172.21.15.46:38132) 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 -- 172.21.15.46:0/2085738094 learned_addr learned my addr 172.21.15.46:0/2085738094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8ba5000700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a6bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 -- 172.21.15.46:0/2085738094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 msgr2=0x555787a6e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a6e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 -- 172.21.15.46:0/2085738094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 msgr2=0x555787a6bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a6bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9f400700 1 -- 172.21.15.46:0/2085738094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba01a1f10 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.762+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a6e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8b9f400700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0cb620 tx=0x7f8b8c0cb560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c38ab9a88340589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8b8a000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a3430 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55578790b270 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f8b8c0ce040 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b8c09b220 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8bbac07b80 10 client.30155 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.763+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5557877dfb60 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55578790b270 con 0x555787a67e00 2024-05-08T23:21:10.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b8c0a2f20 con 0x555787a67e00 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 10 client.30155 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 10 client.30155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 20 client.30155 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787a840c0 0x555787a864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 4 client.30155 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555787a31810 con 0x555787a840c0 2024-05-08T23:21:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.764+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787a840c0 0x555787a864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.765+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787a840c0 0x555787a864b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52abb91f455ffd01 server_cookie=5071a675e909d939 in_seq=0 out_seq=0 2024-05-08T23:21:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.765+0000 7f8b8a000700 10 client.30155 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8b8a000700 1 -- 172.21.15.46:0/2085738094 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555787a31810 con 0x555787a840c0 2024-05-08T23:21:10.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8b8a000700 10 client.30155 handle_command_reply: tid=0 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 client.30155 shutdown 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 msgr2=0x555787a6b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0cb620 tx=0x7f8b8c0cb560 comp rx=0 tx=0).stop 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 shutdown_connections 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787a840c0 0x555787a864b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a68720 0x555787a6bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a62890 0x555787a6e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/2085738094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a6b950 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 >> 172.21.15.46:0/2085738094 conn(0x5557879c0210 msgr2=0x555787a5c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8b8a000700 0 client.30155 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.766+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 shutdown_connections 2024-05-08T23:21:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.767+0000 7f8bbac07b80 1 -- 172.21.15.46:0/2085738094 wait complete. 2024-05-08T23:21:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.767+0000 7f8bbac07b80 20 client.30155 trim_cache size 0 max 16384 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.789+0000 7f8bbac07b80 1 Processor -- start 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.789+0000 7f8bbac07b80 1 -- start start 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.789+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a5cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.789+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.789+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a68720 0x555787a649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a6c560 con 0x555787a62890 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a5de80 con 0x555787a68720 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a8b650 con 0x555787a67e00 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a5cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39094/0 (socket says 172.21.15.46:39094) 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9f400700 1 -- 172.21.15.46:0/519878411 learned_addr learned my addr 172.21.15.46:0/519878411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8ba5000700 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a68720 0x555787a649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 -- 172.21.15.46:0/519878411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 msgr2=0x555787a6e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 -- 172.21.15.46:0/519878411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a68720 msgr2=0x555787a649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a68720 0x555787a649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9fe00700 1 -- 172.21.15.46:0/519878411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555787a5ebf0 con 0x555787a62890 2024-05-08T23:21:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.790+0000 7f8b9f400700 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.791+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a5cd20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x555787a86cf0 tx=0x7f8b9000d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3faaac62bdc6b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.791+0000 7f8b96000700 1 -- 172.21.15.46:0/519878411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90021040 con 0x555787a62890 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.791+0000 7f8b96000700 1 -- 172.21.15.46:0/519878411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b90013040 con 0x555787a62890 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.791+0000 7f8b96000700 1 -- 172.21.15.46:0/519878411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90012ab0 con 0x555787a62890 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 -- 172.21.15.46:0/519878411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 msgr2=0x555787a5cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a5cd20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x555787a86cf0 tx=0x7f8b9000d050 comp rx=0 tx=0).stop 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 -- 172.21.15.46:0/519878411 shutdown_connections 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a68720 0x555787a649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a5cd20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/519878411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a67e00 0x555787a6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 -- 172.21.15.46:0/519878411 >> 172.21.15.46:0/519878411 conn(0x5557879c0210 msgr2=0x555787a65ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 -- 172.21.15.46:0/519878411 shutdown_connections 2024-05-08T23:21:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.792+0000 7f8bbac07b80 1 -- 172.21.15.46:0/519878411 wait complete. 2024-05-08T23:21:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.793+0000 7f8bbac07b80 1 -- start start 2024-05-08T23:21:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.793+0000 7f8bbac07b80 1 Processor -- start 2024-05-08T23:21:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a68720 0x555787a63860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a65660 con 0x555787a62890 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a63da0 con 0x555787a67e00 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555787a63f60 con 0x555787a68720 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8bbac07b80 10 client.0 _ll_register_callbacks cb 0x555787a6be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8b9f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:38162/0 (socket says 172.21.15.46:38162) 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8b9f400700 1 -- 172.21.15.46:0/3055928760 learned_addr learned my addr 172.21.15.46:0/3055928760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.794+0000 7f8ba5000700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a68720 0x555787a63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 -- 172.21.15.46:0/3055928760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a68720 msgr2=0x555787a63860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a68720 0x555787a63860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 -- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 msgr2=0x555787a64d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9fe00700 1 -- 172.21.15.46:0/3055928760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557877dfb60 con 0x555787a62890 2024-05-08T23:21:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.795+0000 7f8b9f400700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.796+0000 7f8b9fe00700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a7bc30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8b90029040 tx=0x7f8b900040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f6c4bd800532d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.796+0000 7f8b8a000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.796+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90021040 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.796+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f8b900124f0 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.796+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b8c09afc0 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.797+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b900268f0 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.797+0000 7f8bbac07b80 10 client.19995 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.797+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f8b900042b0 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.797+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8c09afc0 con 0x555787a62890 2024-05-08T23:21:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b9001b700 con 0x555787a62890 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 10 client.19995 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 20 client.19995 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787adb1e0 0x555787add5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 4 client.19995 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5557879aaec0 con 0x555787adb1e0 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.798+0000 7f8b9f400700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787adb1e0 0x555787add5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.799+0000 7f8b9f400700 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787adb1e0 0x555787add5d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad2874abc1842e9 server_cookie=c8bf67a8f962d042 in_seq=0 out_seq=0 2024-05-08T23:21:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.799+0000 7f8b8a000700 10 client.19995 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.799+0000 7f8b8a000700 1 -- 172.21.15.46:0/3055928760 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5557879aaec0 con 0x555787adb1e0 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.799+0000 7f8b8a000700 10 client.19995 handle_command_reply: tid=0 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.799+0000 7f8bbac07b80 1 client.19995 shutdown 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 msgr2=0x555787a7bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a7bc30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8b90029040 tx=0x7f8b900040a0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 shutdown_connections 2024-05-08T23:21:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555787a68720 0x555787a63860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555787adb1e0 0x555787add5d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555787a62890 0x555787a7bc30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 --2- 172.21.15.46:0/3055928760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555787a67e00 0x555787a64d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 >> 172.21.15.46:0/3055928760 conn(0x5557879c0210 msgr2=0x555787a66a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8b8a000700 0 client.19995 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 shutdown_connections 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 1 -- 172.21.15.46:0/3055928760 wait complete. 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.800+0000 7f8bbac07b80 20 client.19995 trim_cache size 0 max 16384 2024-05-08T23:21:10.802 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "5035d945-a8a2-44f8-b5f5-0e1887dfc288": { 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "5035d945-a8a2-44f8-b5f5-0e1887dfc288", 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:21:10.803 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 msgr2=0x7f8b840a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 0x7f8b840a38a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000dfd0 tx=0x7f8b9000f040 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 msgr2=0x7f8ba01955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba01955d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c009580 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 shutdown_connections 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba0199970 0x7f8ba0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8b840a13e0 0x7f8b840a38a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba00fff40 0x7f8ba0195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 --2- 172.21.15.46:0/1293300577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0105060 0x7f8ba01955d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 >> 172.21.15.46:0/1293300577 conn(0x7f8ba00fb760 msgr2=0x7f8ba01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 shutdown_connections 2024-05-08T23:21:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:10.801+0000 7f8ba7000700 1 -- 172.21.15.46:0/1293300577 wait complete. 2024-05-08T23:21:10.824 INFO:tasks.cephfs.filesystem:scrub status for tag:5035d945-a8a2-44f8-b5f5-0e1887dfc288 - {'path': '/', 'tag': '5035d945-a8a2-44f8-b5f5-0e1887dfc288', 'options': 'recursive,force'} 2024-05-08T23:21:11.265 INFO:tasks.workunit.client.0.smithi046.stdout: 10 82391 19.51 MB/sec execute 92 sec latency 919.343 ms 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: pgmap v544: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:21:11.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:11.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:11 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: pgmap v544: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:11.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:11 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: pgmap v544: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:11.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: osdmap e108: 12 total, 12 up, 12 in 2024-05-08T23:21:11.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:11.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:11 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:12.265 INFO:tasks.workunit.client.0.smithi046.stdout: 10 82711 19.50 MB/sec execute 93 sec latency 1919.393 ms 2024-05-08T23:21:12.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:12 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:12.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:12 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:12.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:12 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:13.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 82960 19.40 MB/sec execute 94 sec latency 2919.464 ms 2024-05-08T23:21:13.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:13 smithi053 ceph-mon[77587]: pgmap v546: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 601 op/s 2024-05-08T23:21:13.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:13 smithi046 ceph-mon[36898]: pgmap v546: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 601 op/s 2024-05-08T23:21:13.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:13 smithi086 ceph-mon[78610]: pgmap v546: 129 pgs: 129 active+clean; 969 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 601 op/s 2024-05-08T23:21:14.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 83242 19.34 MB/sec execute 95 sec latency 3919.510 ms 2024-05-08T23:21:15.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 83641 19.24 MB/sec execute 96 sec latency 4919.604 ms 2024-05-08T23:21:15.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:15 smithi053 ceph-mon[77587]: pgmap v547: 129 pgs: 129 active+clean; 980 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-08T23:21:15.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:15 smithi046 ceph-mon[36898]: pgmap v547: 129 pgs: 129 active+clean; 980 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-08T23:21:15.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:15 smithi086 ceph-mon[78610]: pgmap v547: 129 pgs: 129 active+clean; 980 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-08T23:21:16.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 84119 19.33 MB/sec execute 97 sec latency 5919.675 ms 2024-05-08T23:21:17.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 84689 19.50 MB/sec execute 98 sec latency 6919.713 ms 2024-05-08T23:21:17.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:17 smithi086 ceph-mon[78610]: pgmap v548: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 546 op/s 2024-05-08T23:21:17.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:17 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:18.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:17 smithi053 ceph-mon[77587]: pgmap v548: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 546 op/s 2024-05-08T23:21:18.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:17 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:18.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 84785 19.38 MB/sec execute 99 sec latency 7919.767 ms 2024-05-08T23:21:19.072 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:17 smithi046 ceph-mon[36898]: pgmap v548: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 546 op/s 2024-05-08T23:21:19.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:17 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:19.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 85001 19.30 MB/sec execute 100 sec latency 8919.827 ms 2024-05-08T23:21:19.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:19 smithi086 ceph-mon[78610]: pgmap v549: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-05-08T23:21:20.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:19 smithi053 ceph-mon[77587]: pgmap v549: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-05-08T23:21:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:19 smithi046 ceph-mon[36898]: pgmap v549: 129 pgs: 129 active+clean; 971 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-05-08T23:21:20.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 85298 19.29 MB/sec execute 101 sec latency 9919.886 ms 2024-05-08T23:21:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:20 smithi086 ceph-mon[78610]: osdmap e109: 12 total, 12 up, 12 in 2024-05-08T23:21:21.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:20 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:20 smithi053 ceph-mon[77587]: osdmap e109: 12 total, 12 up, 12 in 2024-05-08T23:21:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:20 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:21.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 85621 19.29 MB/sec execute 102 sec latency 10269.759 ms 2024-05-08T23:21:21.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:20 smithi046 ceph-mon[36898]: osdmap e109: 12 total, 12 up, 12 in 2024-05-08T23:21:21.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:20 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:22.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:21 smithi086 ceph-mon[78610]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 452 op/s 2024-05-08T23:21:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:21 smithi053 ceph-mon[77587]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 452 op/s 2024-05-08T23:21:22.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 85930 19.25 MB/sec execute 103 sec latency 214.103 ms 2024-05-08T23:21:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:21 smithi046 ceph-mon[36898]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 452 op/s 2024-05-08T23:21:23.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 86283 19.22 MB/sec execute 104 sec latency 200.821 ms 2024-05-08T23:21:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:23 smithi086 ceph-mon[78610]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-05-08T23:21:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:23 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:23 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:23 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:21:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:23 smithi053 ceph-mon[77587]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-05-08T23:21:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:23 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:23 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:23 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:21:24.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 86729 19.23 MB/sec execute 105 sec latency 84.044 ms 2024-05-08T23:21:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:23 smithi046 ceph-mon[36898]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 446 op/s 2024-05-08T23:21:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:23 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:23 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:23 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:21:25.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 87283 19.31 MB/sec execute 106 sec latency 92.434 ms 2024-05-08T23:21:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:25 smithi086 ceph-mon[78610]: pgmap v553: 129 pgs: 129 active+clean; 991 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2024-05-08T23:21:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:25 smithi053 ceph-mon[77587]: pgmap v553: 129 pgs: 129 active+clean; 991 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2024-05-08T23:21:26.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 87744 19.36 MB/sec execute 107 sec latency 223.464 ms 2024-05-08T23:21:26.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:25 smithi046 ceph-mon[36898]: pgmap v553: 129 pgs: 129 active+clean; 991 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2024-05-08T23:21:27.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 88280 19.49 MB/sec execute 108 sec latency 177.530 ms 2024-05-08T23:21:28.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:27 smithi086 ceph-mon[78610]: pgmap v554: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:28.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:27 smithi053 ceph-mon[77587]: pgmap v554: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:28.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 88689 19.55 MB/sec execute 109 sec latency 304.910 ms 2024-05-08T23:21:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:27 smithi046 ceph-mon[36898]: pgmap v554: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:29.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 89149 19.65 MB/sec execute 110 sec latency 167.498 ms 2024-05-08T23:21:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:29 smithi086 ceph-mon[78610]: pgmap v555: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:30.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:29 smithi053 ceph-mon[77587]: pgmap v555: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:30.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 89435 19.58 MB/sec execute 111 sec latency 164.652 ms 2024-05-08T23:21:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:29 smithi046 ceph-mon[36898]: pgmap v555: 129 pgs: 129 active+clean; 989 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 447 op/s 2024-05-08T23:21:31.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:21:31.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:21:31.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:21:31.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:30 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:21:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:21:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:21:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:21:31.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:30 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:21:31.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 89828 19.60 MB/sec execute 112 sec latency 518.298 ms 2024-05-08T23:21:31.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:21:31.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:21:31.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:21:31.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:30 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:21:32.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:31 smithi086 ceph-mon[78610]: pgmap v556: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 460 op/s 2024-05-08T23:21:32.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:31 smithi053 ceph-mon[77587]: pgmap v556: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 460 op/s 2024-05-08T23:21:32.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 90225 19.54 MB/sec execute 113 sec latency 1518.364 ms 2024-05-08T23:21:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:31 smithi046 ceph-mon[36898]: pgmap v556: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.6 MiB/s wr, 460 op/s 2024-05-08T23:21:33.267 INFO:tasks.workunit.client.0.smithi046.stdout: 10 90615 19.54 MB/sec execute 114 sec latency 2518.440 ms 2024-05-08T23:21:34.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:33 smithi086 ceph-mon[78610]: pgmap v557: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-05-08T23:21:34.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:33 smithi053 ceph-mon[77587]: pgmap v557: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-05-08T23:21:34.266 INFO:tasks.workunit.client.0.smithi046.stdout: 10 91137 19.61 MB/sec execute 115 sec latency 2533.713 ms 2024-05-08T23:21:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:33 smithi046 ceph-mon[36898]: pgmap v557: 129 pgs: 129 active+clean; 999 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 307 op/s 2024-05-08T23:21:35.267 INFO:tasks.workunit.client.0.smithi046.stdout: 10 91709 19.81 MB/sec execute 116 sec latency 144.948 ms 2024-05-08T23:21:36.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:35 smithi086 ceph-mon[78610]: pgmap v558: 129 pgs: 129 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 410 op/s 2024-05-08T23:21:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:35 smithi053 ceph-mon[77587]: pgmap v558: 129 pgs: 129 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 410 op/s 2024-05-08T23:21:36.267 INFO:tasks.workunit.client.0.smithi046.stdout: 10 92170 19.80 MB/sec execute 117 sec latency 156.840 ms 2024-05-08T23:21:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:35 smithi046 ceph-mon[36898]: pgmap v558: 129 pgs: 129 active+clean; 1010 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 410 op/s 2024-05-08T23:21:37.267 INFO:tasks.workunit.client.0.smithi046.stdout: 10 92567 19.82 MB/sec execute 118 sec latency 172.967 ms 2024-05-08T23:21:38.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:37 smithi086 ceph-mon[78610]: pgmap v559: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 411 op/s 2024-05-08T23:21:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:37 smithi053 ceph-mon[77587]: pgmap v559: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 411 op/s 2024-05-08T23:21:38.267 INFO:tasks.workunit.client.0.smithi046.stdout: 10 92932 19.80 MB/sec execute 119 sec latency 234.781 ms 2024-05-08T23:21:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:37 smithi046 ceph-mon[36898]: pgmap v559: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 411 op/s 2024-05-08T23:21:39.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93409 19.86 MB/sec execute 120 sec latency 203.087 ms 2024-05-08T23:21:40.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:39 smithi086 ceph-mon[78610]: pgmap v560: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 318 op/s 2024-05-08T23:21:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:39 smithi053 ceph-mon[77587]: pgmap v560: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 318 op/s 2024-05-08T23:21:40.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93846 19.81 MB/sec execute 121 sec latency 187.963 ms 2024-05-08T23:21:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:39 smithi046 ceph-mon[36898]: pgmap v560: 129 pgs: 129 active+clean; 1012 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 318 op/s 2024-05-08T23:21:40.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:41.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 94290 19.87 MB/sec execute 122 sec latency 136.906 ms 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 -- 172.21.15.46:0/954465893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c0733d0 msgr2=0x7f6c1c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 --2- 172.21.15.46:0/954465893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c0737b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098f0 tx=0x7f6c0c009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 -- 172.21.15.46:0/954465893 shutdown_connections 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 --2- 172.21.15.46:0/954465893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c10d490 0x7f6c1c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 --2- 172.21.15.46:0/954465893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c0737b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 --2- 172.21.15.46:0/954465893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c073cf0 0x7f6c1c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 -- 172.21.15.46:0/954465893 >> 172.21.15.46:0/954465893 conn(0x7f6c1c0fc740 msgr2=0x7f6c1c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 -- 172.21.15.46:0/954465893 shutdown_connections 2024-05-08T23:21:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.296+0000 7f6c21600700 1 -- 172.21.15.46:0/954465893 wait complete. 2024-05-08T23:21:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.297+0000 7f6c21600700 1 Processor -- start 2024-05-08T23:21:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.297+0000 7f6c21600700 1 -- start start 2024-05-08T23:21:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c073cf0 0x7f6c1c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c10d490 0x7f6c1c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19e500 con 0x7f6c1c0733d0 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19e670 con 0x7f6c1c10d490 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c21600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19e830 con 0x7f6c1c073cf0 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55928/0 (socket says 172.21.15.46:55928) 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 -- 172.21.15.46:0/2907505852 learned_addr learned my addr 172.21.15.46:0/2907505852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1b400700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c073cf0 0x7f6c1c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 -- 172.21.15.46:0/2907505852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c073cf0 msgr2=0x7f6c1c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c073cf0 0x7f6c1c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 -- 172.21.15.46:0/2907505852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c10d490 msgr2=0x7f6c1c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c10d490 0x7f6c1c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.298+0000 7f6c1be00700 1 -- 172.21.15.46:0/2907505852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c009520 con 0x7f6c1c0733d0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c1be00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005950 tx=0x7f6c0c010ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c18c00700 1 -- 172.21.15.46:0/2907505852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c01c070 con 0x7f6c1c0733d0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c18c00700 1 -- 172.21.15.46:0/2907505852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c0c005500 con 0x7f6c1c0733d0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c18c00700 1 -- 172.21.15.46:0/2907505852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c003b40 con 0x7f6c1c0733d0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c19eab0 con 0x7f6c1c0733d0 2024-05-08T23:21:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.299+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c19ed30 con 0x7f6c1c0733d0 2024-05-08T23:21:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.304+0000 7f6c18c00700 1 -- 172.21.15.46:0/2907505852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6c0c003ca0 con 0x7f6c1c0733d0 2024-05-08T23:21:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.305+0000 7f6c3678fb80 1 Processor -- start 2024-05-08T23:21:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.305+0000 7f6c3678fb80 1 -- start start 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921b49150 con 0x558921bf8050 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921b46b10 con 0x558921bf2890 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921b42400 con 0x558921bf8970 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55936/0 (socket says 172.21.15.46:55936) 2024-05-08T23:21:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41566/0 (socket says 172.21.15.46:41566) 2024-05-08T23:21:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c20c00700 1 -- 172.21.15.46:0/1020804948 learned_addr learned my addr 172.21.15.46:0/1020804948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c1b400700 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.306+0000 7f6c18c00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 0x7f6c000a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c18c00700 1 -- 172.21.15.46:0/2907505852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6c0c0cde30 con 0x7f6c1c0733d0 2024-05-08T23:21:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 -- 172.21.15.46:0/1020804948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 msgr2=0x558921bf2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 -- 172.21.15.46:0/1020804948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 msgr2=0x558921bf8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 -- 172.21.15.46:0/1020804948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589219c33b0 con 0x558921bf8970 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c20c00700 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c20c00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 0x7f6c000a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c1be00700 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cb110 tx=0x7f6c0c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eeb183c9d5e5521 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c11200700 1 -- 172.21.15.46:0/1020804948 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c09a4c0 con 0x558921bf8970 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.307+0000 7f6c11200700 1 -- 172.21.15.46:0/1020804948 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c0c09b260 con 0x558921bf8970 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c11200700 1 -- 172.21.15.46:0/1020804948 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00b8f0 con 0x558921bf8970 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 -- 172.21.15.46:0/1020804948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 msgr2=0x558921bf2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cb110 tx=0x7f6c0c009260 comp rx=0 tx=0).stop 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c20c00700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 0x7f6c000a3870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009c10 tx=0x7f6c14009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 -- 172.21.15.46:0/1020804948 shutdown_connections 2024-05-08T23:21:41.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bf8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/1020804948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf2350 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 -- 172.21.15.46:0/1020804948 >> 172.21.15.46:0/1020804948 conn(0x558921b50210 msgr2=0x558921bed2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 -- 172.21.15.46:0/1020804948 shutdown_connections 2024-05-08T23:21:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.308+0000 7f6c3678fb80 1 -- 172.21.15.46:0/1020804948 wait complete. 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.309+0000 7f6c3678fb80 1 -- start start 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.309+0000 7f6c3678fb80 1 Processor -- start 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bfbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921bfc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921ade560 con 0x558921bf8050 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921ade200 con 0x558921bf8970 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589219c56a0 con 0x558921bf2890 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c3678fb80 10 client.0 _ll_register_callbacks cb 0x558921ae7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:41.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921bfc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41582/0 (socket says 172.21.15.46:41582) 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c1be00700 1 -- 172.21.15.46:0/3187577294 learned_addr learned my addr 172.21.15.46:0/3187577294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.310+0000 7f6c1b400700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bfbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 -- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 msgr2=0x558921bfc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921bfc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 -- 172.21.15.46:0/3187577294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 msgr2=0x558921bfbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bfbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 -- 172.21.15.46:0/3187577294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1400e0c0 con 0x558921bf2890 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c20c00700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921bfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c1be00700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c097060 tx=0x7f6c0c09b3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1c8698cf204539 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c05600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:41.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00bb00 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6c0c09ab60 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558921a9b270 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0c09e990 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c3678fb80 10 client.30250 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.311+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f6c1c19eab0 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558921a9b270 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c0c09a1a0 con 0x558921bf2890 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 10 client.30250 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 10 client.30250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 20 client.30250 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c14260 0x558921c16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 4 client.30250 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558921bc1810 con 0x558921c14260 2024-05-08T23:21:41.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.312+0000 7f6c1b400700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c14260 0x558921c16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.313+0000 7f6c1b400700 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c14260 0x558921c16650 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508c34278738a323 server_cookie=94a913ad7df3dcaf in_seq=0 out_seq=0 2024-05-08T23:21:41.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.313+0000 7f6c05600700 10 client.30250 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c05600700 1 -- 172.21.15.46:0/3187577294 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558921bc1810 con 0x558921c14260 2024-05-08T23:21:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c05600700 10 client.30250 handle_command_reply: tid=0 2024-05-08T23:21:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 client.30250 shutdown 2024-05-08T23:21:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 msgr2=0x558921bfe7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c097060 tx=0x7f6c0c09b3d0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 shutdown_connections 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921bfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf2890 0x558921bfe7d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c14260 0x558921c16650 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/3187577294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921bfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c05600700 0 client.30250 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 >> 172.21.15.46:0/3187577294 conn(0x558921b50210 msgr2=0x558921bed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 shutdown_connections 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.314+0000 7f6c3678fb80 1 -- 172.21.15.46:0/3187577294 wait complete. 2024-05-08T23:21:41.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.315+0000 7f6c3678fb80 20 client.30250 trim_cache size 0 max 16384 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 Processor -- start 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 -- start start 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf2890 0x558921bf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921c1af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921bec390 con 0x558921bf2890 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921bfc9d0 con 0x558921bf8970 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.334+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921bf8740 con 0x558921bf8050 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf2890 0x558921bf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41588/0 (socket says 172.21.15.46:41588) 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 -- 172.21.15.46:0/2647347545 learned_addr learned my addr 172.21.15.46:0/2647347545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c20c00700 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921c1af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 -- 172.21.15.46:0/2647347545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 msgr2=0x558921c1af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921c1af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 -- 172.21.15.46:0/2647347545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf2890 msgr2=0x558921bf2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf2890 0x558921bf2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 -- 172.21.15.46:0/2647347545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55892196fb60 con 0x558921bf8050 2024-05-08T23:21:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c20c00700 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921c1af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.335+0000 7f6c1b400700 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x558921bfd000 tx=0x7f6c08009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669013115f68e75c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.336+0000 7f6c11c00700 1 -- 172.21.15.46:0/2647347545 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08017040 con 0x558921bf8050 2024-05-08T23:21:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.336+0000 7f6c11c00700 1 -- 172.21.15.46:0/2647347545 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c08018040 con 0x558921bf8050 2024-05-08T23:21:41.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.336+0000 7f6c11c00700 1 -- 172.21.15.46:0/2647347545 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0801e840 con 0x558921bf8050 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- 172.21.15.46:0/2647347545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 msgr2=0x558921bed060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x558921bfd000 tx=0x7f6c08009310 comp rx=0 tx=0).stop 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- 172.21.15.46:0/2647347545 shutdown_connections 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf8970 0x558921c1af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf2890 0x558921bf2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/2647347545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8050 0x558921bed060 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- 172.21.15.46:0/2647347545 >> 172.21.15.46:0/2647347545 conn(0x558921b50210 msgr2=0x558921bf2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- 172.21.15.46:0/2647347545 shutdown_connections 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- 172.21.15.46:0/2647347545 wait complete. 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 -- start start 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.337+0000 7f6c3678fb80 1 Processor -- start 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921c03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921c03950 con 0x558921bf8050 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921bf3fb0 con 0x558921bf2890 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558921bf4170 con 0x558921bf8970 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c3678fb80 10 client.0 _ll_register_callbacks cb 0x558921ae7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c1b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921c03040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c1be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:41338/0 (socket says 172.21.15.46:41338) 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c1be00700 1 -- 172.21.15.46:0/934454444 learned_addr learned my addr 172.21.15.46:0/934454444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.338+0000 7f6c20c00700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 -- 172.21.15.46:0/934454444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 msgr2=0x558921bf3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 -- 172.21.15.46:0/934454444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 msgr2=0x558921c03040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921c03040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 -- 172.21.15.46:0/934454444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c08011440 con 0x558921bf2890 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c1be00700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009810 tx=0x7f6c0c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=729b368f0dcab023 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c05600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c09a4c0 con 0x558921bf2890 2024-05-08T23:21:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6c0c00bb00 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c0c09b260 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0c084170 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.339+0000 7f6c3678fb80 10 client.30176 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558921b08560 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c0c09b260 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c0c09bbc0 con 0x558921bf2890 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 10 client.30176 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 10 client.30176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 20 client.30176 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c6b040 0x558921c6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 4 client.30176 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.340+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558921b3aec0 con 0x558921c6b040 2024-05-08T23:21:41.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.341+0000 7f6c1b400700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c6b040 0x558921c6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.341+0000 7f6c05600700 10 client.30176 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.341+0000 7f6c1b400700 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c6b040 0x558921c6d430 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb784712bd52b0d server_cookie=8067be43b45bd944 in_seq=0 out_seq=0 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c05600700 1 -- 172.21.15.46:0/934454444 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558921b3aec0 con 0x558921c6b040 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c05600700 10 client.30176 handle_command_reply: tid=0 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 client.30176 shutdown 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 msgr2=0x558921bf52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009810 tx=0x7f6c0c009260 comp rx=0 tx=0).stop 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 shutdown_connections 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558921bf8970 0x558921bf3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558921bf2890 0x558921bf52b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x558921c6b040 0x558921c6d430 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 --2- 172.21.15.46:0/934454444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558921bf8050 0x558921c03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 >> 172.21.15.46:0/934454444 conn(0x558921b50210 msgr2=0x558921bf5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.342+0000 7f6c05600700 0 client.30176 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.343+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 shutdown_connections 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.343+0000 7f6c3678fb80 1 -- 172.21.15.46:0/934454444 wait complete. 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.343+0000 7f6c3678fb80 20 client.30176 trim_cache size 0 max 16384 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:21:41.347 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 msgr2=0x7f6c000a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 0x7f6c000a3870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009c10 tx=0x7f6c14009040 comp rx=0 tx=0).stop 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 msgr2=0x7f6c1c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005950 tx=0x7f6c0c010ff0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 shutdown_connections 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c10d490 0x7f6c1c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f6c000a13b0 0x7f6c000a3870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c0733d0 0x7f6c1c199630 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 --2- 172.21.15.46:0/2907505852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c073cf0 0x7f6c1c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 >> 172.21.15.46:0/2907505852 conn(0x7f6c1c0fc740 msgr2=0x7f6c1c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 shutdown_connections 2024-05-08T23:21:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.344+0000 7f6c21600700 1 -- 172.21.15.46:0/2907505852 wait complete. 2024-05-08T23:21:41.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:21:41.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 -- 172.21.15.46:0/2467328414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7400ffda0 msgr2=0x7fe740100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 --2- 172.21.15.46:0/2467328414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7400ffda0 0x7fe740100180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe734009990 tx=0x7fe734009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 -- 172.21.15.46:0/2467328414 shutdown_connections 2024-05-08T23:21:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 --2- 172.21.15.46:0/2467328414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7401051a0 0x7fe740105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 --2- 172.21.15.46:0/2467328414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7400ffda0 0x7fe740100180 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 --2- 172.21.15.46:0/2467328414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401006c0 0x7fe740104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 -- 172.21.15.46:0/2467328414 >> 172.21.15.46:0/2467328414 conn(0x7fe7400fb7f0 msgr2=0x7fe7400fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 -- 172.21.15.46:0/2467328414 shutdown_connections 2024-05-08T23:21:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.676+0000 7fe748800700 1 -- 172.21.15.46:0/2467328414 wait complete. 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.677+0000 7fe748800700 1 Processor -- start 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.678+0000 7fe748800700 1 -- start start 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.678+0000 7fe748800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 0x7fe740072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe748800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7401006c0 0x7fe740072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe745e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 0x7fe740072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe745e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 0x7fe740072630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55992/0 (socket says 172.21.15.46:55992) 2024-05-08T23:21:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe748800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 0x7fe74006b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe745e00700 1 -- 172.21.15.46:0/2050018751 learned_addr learned my addr 172.21.15.46:0/2050018751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74006c070 con 0x7fe7400ffda0 2024-05-08T23:21:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74006c1e0 con 0x7fe7401006c0 2024-05-08T23:21:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74006c3a0 con 0x7fe7401051a0 2024-05-08T23:21:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe746800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 0x7fe74006b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.679+0000 7fe745400700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7401006c0 0x7fe740072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 -- 172.21.15.46:0/2050018751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7401006c0 msgr2=0x7fe740072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7401006c0 0x7fe740072bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 -- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 msgr2=0x7fe740072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 0x7fe740072630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340095f0 con 0x7fe7401051a0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.680+0000 7fe746800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 0x7fe74006b8a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe73c007f10 tx=0x7fe73c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.681+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c004db0 con 0x7fe7401051a0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.681+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe73c00f040 con 0x7fe7401051a0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.681+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c005990 con 0x7fe7401051a0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.681+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7400701c0 con 0x7fe7401051a0 2024-05-08T23:21:41.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.681+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe740070680 con 0x7fe7401051a0 2024-05-08T23:21:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.682+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe740069d70 con 0x7fe7401051a0 2024-05-08T23:21:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.684+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe73c00db20 con 0x7fe7401051a0 2024-05-08T23:21:41.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.686+0000 7fe73aa00700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 0x7fe7280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.686+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe73c0ccb00 con 0x7fe7401051a0 2024-05-08T23:21:41.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.687+0000 7fe745e00700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 0x7fe7280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.687+0000 7fe745e00700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 0x7fe7280a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe73400b5c0 tx=0x7fe734005ab0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:41.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:41.688+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe73c095ae0 con 0x7fe7401051a0 2024-05-08T23:21:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.101+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe74006cc10 con 0x7fe7401051a0 2024-05-08T23:21:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.102+0000 7fe73aa00700 1 -- 172.21.15.46:0/2050018751 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fe73c099100 con 0x7fe7401051a0 2024-05-08T23:21:42.103 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:21:42.103 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:42.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.106+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 msgr2=0x7fe7280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.106+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 0x7fe7280a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe73400b5c0 tx=0x7fe734005ab0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.106+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 msgr2=0x7fe74006b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.106+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 0x7fe74006b8a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe73c007f10 tx=0x7fe73c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 shutdown_connections 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7401051a0 0x7fe74006b8a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe7280a1310 0x7fe7280a37d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7400ffda0 0x7fe740072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 --2- 172.21.15.46:0/2050018751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7401006c0 0x7fe740072bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.109+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 >> 172.21.15.46:0/2050018751 conn(0x7fe7400fb7f0 msgr2=0x7fe7400fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.110+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 shutdown_connections 2024-05-08T23:21:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.111+0000 7fe748800700 1 -- 172.21.15.46:0/2050018751 wait complete. 2024-05-08T23:21:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:21:42.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:21:42.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:41 smithi086 ceph-mon[78610]: pgmap v561: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.4 MiB/s wr, 460 op/s 2024-05-08T23:21:42.220 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:41 smithi046 ceph-mon[36898]: pgmap v561: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.4 MiB/s wr, 460 op/s 2024-05-08T23:21:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:41 smithi053 ceph-mon[77587]: pgmap v561: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.4 MiB/s wr, 460 op/s 2024-05-08T23:21:42.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 94950 20.02 MB/sec execute 123 sec latency 58.984 ms 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.609+0000 7f05bde00700 1 -- 172.21.15.46:0/2985024289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b00a91f0 msgr2=0x7f05b00a95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.609+0000 7f05bde00700 1 --2- 172.21.15.46:0/2985024289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b00a91f0 0x7f05b00a95d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0098f0 tx=0x7f05ac009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 -- 172.21.15.46:0/2985024289 shutdown_connections 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 --2- 172.21.15.46:0/2985024289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b00ae310 0x7f05b00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 --2- 172.21.15.46:0/2985024289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b00a91f0 0x7f05b00a95d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 --2- 172.21.15.46:0/2985024289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00a9ba0 0x7f05b00adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 -- 172.21.15.46:0/2985024289 >> 172.21.15.46:0/2985024289 conn(0x7f05b00a4a50 msgr2=0x7f05b00a6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 -- 172.21.15.46:0/2985024289 shutdown_connections 2024-05-08T23:21:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.610+0000 7f05bde00700 1 -- 172.21.15.46:0/2985024289 wait complete. 2024-05-08T23:21:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.612+0000 7f05bde00700 1 Processor -- start 2024-05-08T23:21:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.612+0000 7f05bde00700 1 -- start start 2024-05-08T23:21:42.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b00a91f0 0x7f05b013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 0x7f05b013fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0144920 con 0x7f05b0143ea0 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0144a90 con 0x7f05b00a91f0 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bde00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0144c50 con 0x7f05b00ae310 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.613+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38486/0 (socket says 172.21.15.46:38486) 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 -- 172.21.15.46:0/2573361915 learned_addr learned my addr 172.21.15.46:0/2573361915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 0x7f05b013fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 -- 172.21.15.46:0/2573361915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 msgr2=0x7f05b013fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 0x7f05b013fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 -- 172.21.15.46:0/2573361915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b00a91f0 msgr2=0x7f05b013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.614+0000 7f05bd400700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b00a91f0 0x7f05b013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.615+0000 7f05bd400700 1 -- 172.21.15.46:0/2573361915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009520 con 0x7f05b0143ea0 2024-05-08T23:21:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.615+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 0x7f05b013fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.615+0000 7f05bd400700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f05b80669e0 tx=0x7f05b8069f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.616+0000 7f05b5600700 1 -- 172.21.15.46:0/2573361915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80692d0 con 0x7f05b0143ea0 2024-05-08T23:21:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.616+0000 7f05b5600700 1 -- 172.21.15.46:0/2573361915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b8067940 con 0x7f05b0143ea0 2024-05-08T23:21:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.616+0000 7f05b5600700 1 -- 172.21.15.46:0/2573361915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8074910 con 0x7f05b0143ea0 2024-05-08T23:21:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.616+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b001adb0 con 0x7f05b0143ea0 2024-05-08T23:21:42.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.616+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b001b240 con 0x7f05b0143ea0 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.625+0000 7f05b5600700 1 -- 172.21.15.46:0/2573361915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f05b8067480 con 0x7f05b0143ea0 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.627+0000 7f05b5600700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 0x7f05a00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.631+0000 7f05b5600700 1 -- 172.21.15.46:0/2573361915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f05b80fd140 con 0x7f05b0143ea0 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.632+0000 7f05bca00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 0x7f05a00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.632+0000 7f05bca00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 0x7f05a00a38e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00b2e0 tx=0x7f05ac010dc0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.633+0000 7f05d2f7fb80 1 Processor -- start 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.633+0000 7f05d2f7fb80 1 -- start start 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559407422150 con 0x5594074d1970 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940741fb10 con 0x5594074cb890 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.634+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940741b400 con 0x5594074d1050 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.635+0000 7f05bca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.635+0000 7f05bca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45802/0 (socket says 172.21.15.46:45802) 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.635+0000 7f05bca00700 1 -- 172.21.15.46:0/2599018961 learned_addr learned my addr 172.21.15.46:0/2599018961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.636+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.638+0000 7f05b7e00700 1 -- 172.21.15.46:0/2599018961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 msgr2=0x5594074d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.638+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.638+0000 7f05b7e00700 1 -- 172.21.15.46:0/2599018961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 msgr2=0x5594074cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.638+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.638+0000 7f05b7e00700 1 -- 172.21.15.46:0/2599018961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55940729c3b0 con 0x5594074d1970 2024-05-08T23:21:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.641+0000 7f05b7e00700 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074cb350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05b0018ba0 tx=0x7f05a400ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b9774303dd6cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.642+0000 7f05a9c00700 1 -- 172.21.15.46:0/2599018961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400c8a0 con 0x5594074d1970 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.642+0000 7f05a9c00700 1 -- 172.21.15.46:0/2599018961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a40113f0 con 0x5594074d1970 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.642+0000 7f05a9c00700 1 -- 172.21.15.46:0/2599018961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40185c0 con 0x5594074d1970 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/2599018961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 msgr2=0x5594074cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074cb350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05b0018ba0 tx=0x7f05a400ac80 comp rx=0 tx=0).stop 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/2599018961 shutdown_connections 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/2599018961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074cb350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.643+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/2599018961 >> 172.21.15.46:0/2599018961 conn(0x559407429210 msgr2=0x5594074c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.644+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/2599018961 shutdown_connections 2024-05-08T23:21:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.644+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/2599018961 wait complete. 2024-05-08T23:21:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.644+0000 7f05d2f7fb80 1 -- start start 2024-05-08T23:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.649+0000 7f05d2f7fb80 1 Processor -- start 2024-05-08T23:21:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594073b7560 con 0x5594074d1970 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594073b7200 con 0x5594074cb890 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940729e6a0 con 0x5594074d1050 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05d2f7fb80 10 client.0 _ll_register_callbacks cb 0x5594073c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.650+0000 7f05bca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.654+0000 7f05bca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49230/0 (socket says 172.21.15.46:49230) 2024-05-08T23:21:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.654+0000 7f05bca00700 1 -- 172.21.15.46:0/4093522279 learned_addr learned my addr 172.21.15.46:0/4093522279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.654+0000 7f05bd400700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.654+0000 7f05bd400700 1 -- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 msgr2=0x5594074d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.654+0000 7f05bd400700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.655+0000 7f05bd400700 1 -- 172.21.15.46:0/4093522279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 msgr2=0x5594074d4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.655+0000 7f05bd400700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.655+0000 7f05bd400700 1 -- 172.21.15.46:0/4093522279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a400a600 con 0x5594074d1970 2024-05-08T23:21:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.655+0000 7f05bd400700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d56a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05b812d080 tx=0x7f05b80f75e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189598ba2865f048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.655+0000 7f059e000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.657+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b812d500 con 0x5594074d1970 2024-05-08T23:21:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.657+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559407374270 con 0x5594074d1970 2024-05-08T23:21:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.657+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f05b8100870 con 0x5594074d1970 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b8067ab0 con 0x5594074d1970 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f05d2f7fb80 10 client.20031 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559407248b60 con 0x5594074d1970 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b8119070 con 0x5594074d1970 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05b80fc3a0 con 0x5594074d1970 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.658+0000 7f05d2f7fb80 10 client.20031 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05d2f7fb80 10 client.20031 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:21:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05d2f7fb80 20 client.20031 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5594074e53b0 0x5594074e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05d2f7fb80 4 client.20031 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55940749a810 con 0x5594074e53b0 2024-05-08T23:21:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.659+0000 7f05bca00700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5594074e53b0 0x5594074e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.674+0000 7f059e000700 10 client.20031 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.674+0000 7f05bca00700 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5594074e53b0 0x5594074e77d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e49f038174cba8e server_cookie=291e99c08b86a09f in_seq=0 out_seq=0 2024-05-08T23:21:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.684+0000 7f059e000700 1 -- 172.21.15.46:0/4093522279 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55940749a810 con 0x5594074e53b0 2024-05-08T23:21:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.684+0000 7f059e000700 10 client.20031 handle_command_reply: tid=0 2024-05-08T23:21:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.684+0000 7f05d2f7fb80 1 client.20031 shutdown 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.684+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 msgr2=0x5594074d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.684+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d56a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05b812d080 tx=0x7f05b80f75e0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 shutdown_connections 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5594074e53b0 0x5594074e77d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d56a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/4093522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 >> 172.21.15.46:0/4093522279 conn(0x559407429210 msgr2=0x5594074c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f059e000700 0 client.20031 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 shutdown_connections 2024-05-08T23:21:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.685+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/4093522279 wait complete. 2024-05-08T23:21:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.686+0000 7f05d2f7fb80 20 client.20031 trim_cache size 0 max 16384 2024-05-08T23:21:42.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.719+0000 7f05d2f7fb80 1 Processor -- start 2024-05-08T23:21:42.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.720+0000 7f05d2f7fb80 1 -- start start 2024-05-08T23:21:42.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594074d0c50 con 0x5594074d1970 2024-05-08T23:21:42.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594074e1d70 con 0x5594074cb890 2024-05-08T23:21:42.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.721+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594074d5a00 con 0x5594074d1050 2024-05-08T23:21:42.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.722+0000 7f05bca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.722+0000 7f05bca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49234/0 (socket says 172.21.15.46:49234) 2024-05-08T23:21:42.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.722+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.722+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38522/0 (socket says 172.21.15.46:38522) 2024-05-08T23:21:42.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.722+0000 7f05bd400700 1 -- 172.21.15.46:0/3251119337 learned_addr learned my addr 172.21.15.46:0/3251119337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:42.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.723+0000 7f05bca00700 1 -- 172.21.15.46:0/3251119337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 msgr2=0x5594074e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.723+0000 7f05bca00700 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.723+0000 7f05bca00700 1 -- 172.21.15.46:0/3251119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 msgr2=0x5594074d54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.724+0000 7f05bca00700 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.724+0000 7f05bca00700 1 -- 172.21.15.46:0/3251119337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594073e1560 con 0x5594074cb890 2024-05-08T23:21:42.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.724+0000 7f05bd400700 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.728+0000 7f05bca00700 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5594074e7db0 tx=0x7f05ac01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44c024aa32822e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.729+0000 7f05aa600700 1 -- 172.21.15.46:0/3251119337 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac027040 con 0x5594074cb890 2024-05-08T23:21:42.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.729+0000 7f05aa600700 1 -- 172.21.15.46:0/3251119337 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05ac028040 con 0x5594074cb890 2024-05-08T23:21:42.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.729+0000 7f05aa600700 1 -- 172.21.15.46:0/3251119337 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac01c070 con 0x5594074cb890 2024-05-08T23:21:42.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/3251119337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 msgr2=0x5594074cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5594074e7db0 tx=0x7f05ac01a000 comp rx=0 tx=0).stop 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/3251119337 shutdown_connections 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074cbc70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/3251119337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/3251119337 >> 172.21.15.46:0/3251119337 conn(0x559407429210 msgr2=0x5594074c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/3251119337 shutdown_connections 2024-05-08T23:21:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.730+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/3251119337 wait complete. 2024-05-08T23:21:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.731+0000 7f05d2f7fb80 1 -- start start 2024-05-08T23:21:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.734+0000 7f05d2f7fb80 1 Processor -- start 2024-05-08T23:21:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940753aa20 con 0x5594074d1970 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940753ab90 con 0x5594074cb890 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940753ad50 con 0x5594074d1050 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.736+0000 7f05d2f7fb80 10 client.0 _ll_register_callbacks cb 0x5594073c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.737+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.737+0000 7f05bd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38528/0 (socket says 172.21.15.46:38528) 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.737+0000 7f05bd400700 1 -- 172.21.15.46:0/1180929831 learned_addr learned my addr 172.21.15.46:0/1180929831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 -- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 msgr2=0x5594074d8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 -- 172.21.15.46:0/1180929831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 msgr2=0x5594074d8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 -- 172.21.15.46:0/1180929831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559407248b60 con 0x5594074d1970 2024-05-08T23:21:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f05bd400700 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f05b80690a0 tx=0x7f05b80845c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c265859c24bacafc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.738+0000 7f059e000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.739+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8115430 con 0x5594074d1970 2024-05-08T23:21:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.739+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f05b8104ec0 con 0x5594074d1970 2024-05-08T23:21:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.739+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05a40113f0 con 0x5594074d1970 2024-05-08T23:21:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.740+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b80f9150 con 0x5594074d1970 2024-05-08T23:21:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.740+0000 7f05d2f7fb80 10 client.20037 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.740+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f05b00b6700 con 0x5594074d1970 2024-05-08T23:21:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a40113f0 con 0x5594074d1970 2024-05-08T23:21:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05b8104ad0 con 0x5594074d1970 2024-05-08T23:21:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 10 client.20037 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 10 client.20037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:21:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 20 client.20037 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55940753c180 0x55940753e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 4 client.20037 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.741+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559407413ec0 con 0x55940753c180 2024-05-08T23:21:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.745+0000 7f05bca00700 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55940753c180 0x55940753e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.747+0000 7f059e000700 10 client.20037 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.747+0000 7f05bca00700 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55940753c180 0x55940753e570 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1228ede031928fd server_cookie=2a2cb9d114076bd in_seq=0 out_seq=0 2024-05-08T23:21:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.748+0000 7f059e000700 1 -- 172.21.15.46:0/1180929831 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559407413ec0 con 0x55940753c180 2024-05-08T23:21:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.748+0000 7f059e000700 10 client.20037 handle_command_reply: tid=0 2024-05-08T23:21:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.748+0000 7f05d2f7fb80 1 client.20037 shutdown 2024-05-08T23:21:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 msgr2=0x5594074d8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f05b80690a0 tx=0x7f05b80845c0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 shutdown_connections 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55940753c180 0x55940753e570 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5594074d1970 0x5594074d8db0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594074cb890 0x5594074d8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 --2- 172.21.15.46:0/1180929831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594074d1050 0x5594074d8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 >> 172.21.15.46:0/1180929831 conn(0x559407429210 msgr2=0x5594074ec130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f059e000700 0 client.20037 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:21:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.749+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 shutdown_connections 2024-05-08T23:21:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.750+0000 7f05d2f7fb80 1 -- 172.21.15.46:0/1180929831 wait complete. 2024-05-08T23:21:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.750+0000 7f05d2f7fb80 20 client.20037 trim_cache size 0 max 16384 2024-05-08T23:21:42.751 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.751+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 msgr2=0x7f05a00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.751+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 0x7f05a00a38e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00b2e0 tx=0x7f05ac010dc0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.751+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 msgr2=0x7f05b0144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.751+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f05b80669e0 tx=0x7f05b8069f90 comp rx=0 tx=0).stop 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 shutdown_connections 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f05a00a1420 0x7f05a00a38e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05b0143ea0 0x7f05b0144320 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b00a91f0 0x7f05b013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 --2- 172.21.15.46:0/2573361915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b00ae310 0x7f05b013fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 >> 172.21.15.46:0/2573361915 conn(0x7f05b00a4a50 msgr2=0x7f05b00a5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 shutdown_connections 2024-05-08T23:21:42.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:42.752+0000 7f05bde00700 1 -- 172.21.15.46:0/2573361915 wait complete. 2024-05-08T23:21:42.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:21:43.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:43 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2050018751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:43.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95085 19.95 MB/sec execute 124 sec latency 893.970 ms 2024-05-08T23:21:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:43 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2050018751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:43.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:43 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2050018751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:43.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.772+0000 7f3a85e00700 1 -- 172.21.15.46:0/912319313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 msgr2=0x7f3a8010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.772+0000 7f3a85e00700 1 --2- 172.21.15.46:0/912319313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8010d870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009f90 tx=0x7f3a7800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.776+0000 7f3a85e00700 1 -- 172.21.15.46:0/912319313 shutdown_connections 2024-05-08T23:21:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.776+0000 7f3a85e00700 1 --2- 172.21.15.46:0/912319313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8010d870 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.776+0000 7f3a85e00700 1 --2- 172.21.15.46:0/912319313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80101fe0 0x7f3a801023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.776+0000 7f3a85e00700 1 --2- 172.21.15.46:0/912319313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 0x7f3a8010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.776+0000 7f3a85e00700 1 -- 172.21.15.46:0/912319313 >> 172.21.15.46:0/912319313 conn(0x7f3a800fb760 msgr2=0x7f3a800fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.780+0000 7f3a85e00700 1 -- 172.21.15.46:0/912319313 shutdown_connections 2024-05-08T23:21:43.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.780+0000 7f3a85e00700 1 -- 172.21.15.46:0/912319313 wait complete. 2024-05-08T23:21:43.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.781+0000 7f3a85e00700 1 Processor -- start 2024-05-08T23:21:43.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 -- start start 2024-05-08T23:21:43.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80101fe0 0x7f3a801995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 0x7f3a80199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e480 con 0x7f3a80102900 2024-05-08T23:21:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e5f0 con 0x7f3a8010b400 2024-05-08T23:21:43.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.782+0000 7f3a85e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e7b0 con 0x7f3a80101fe0 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49258/0 (socket says 172.21.15.46:49258) 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 -- 172.21.15.46:0/775116915 learned_addr learned my addr 172.21.15.46:0/775116915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7e000700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 0x7f3a80199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 -- 172.21.15.46:0/775116915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80101fe0 msgr2=0x7f3a801995d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80101fe0 0x7f3a801995d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 -- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 msgr2=0x7f3a80199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 0x7f3a80199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.783+0000 7f3a7f400700 1 -- 172.21.15.46:0/775116915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78009c40 con 0x7f3a8010b400 2024-05-08T23:21:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.784+0000 7f3a7f400700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a78004c80 tx=0x7f3a78004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.784+0000 7f3a84a00700 1 -- 172.21.15.46:0/775116915 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800e040 con 0x7f3a8010b400 2024-05-08T23:21:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.785+0000 7f3a84a00700 1 -- 172.21.15.46:0/775116915 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a78008ed0 con 0x7f3a8010b400 2024-05-08T23:21:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.785+0000 7f3a84a00700 1 -- 172.21.15.46:0/775116915 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800e040 con 0x7f3a8010b400 2024-05-08T23:21:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.786+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8019ea30 con 0x7f3a8010b400 2024-05-08T23:21:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.786+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a801a60f0 con 0x7f3a8010b400 2024-05-08T23:21:43.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.795+0000 7f3a84a00700 1 -- 172.21.15.46:0/775116915 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3a7801d9c0 con 0x7f3a8010b400 2024-05-08T23:21:43.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.798+0000 7f3a84a00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 0x7f3a640a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.799+0000 7f3a84a00700 1 -- 172.21.15.46:0/775116915 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3a780cdca0 con 0x7f3a8010b400 2024-05-08T23:21:43.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.799+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 0x7f3a640a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.800+0000 7f3a998adb80 1 Processor -- start 2024-05-08T23:21:43.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.801+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 0x7f3a640a3830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005e50 tx=0x7f3a70005de0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.801+0000 7f3a998adb80 1 -- start start 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.801+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a998adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a87150 con 0x565167b36720 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a998adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a84b10 con 0x565167b35e00 2024-05-08T23:21:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a998adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a80400 con 0x565167b30890 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49260/0 (socket says 172.21.15.46:49260) 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 -- 172.21.15.46:0/1032575276 learned_addr learned my addr 172.21.15.46:0/1032575276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 -- 172.21.15.46:0/1032575276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 msgr2=0x565167b30cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 -- 172.21.15.46:0/1032575276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 msgr2=0x565167b30350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.802+0000 7f3a7ea00700 1 -- 172.21.15.46:0/1032575276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651679013b0 con 0x565167b35e00 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.803+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000dbd0 tx=0x7f3a70010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f3d307ad7805cd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.803+0000 7f3a6be00700 1 -- 172.21.15.46:0/1032575276 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70010af0 con 0x565167b35e00 2024-05-08T23:21:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.803+0000 7f3a6be00700 1 -- 172.21.15.46:0/1032575276 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a70010c50 con 0x565167b35e00 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.804+0000 7f3a998adb80 1 -- 172.21.15.46:0/1032575276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 msgr2=0x565167b361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.804+0000 7f3a998adb80 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000dbd0 tx=0x7f3a70010000 comp rx=0 tx=0).stop 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.805+0000 7f3a998adb80 1 -- 172.21.15.46:0/1032575276 shutdown_connections 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.805+0000 7f3a998adb80 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.805+0000 7f3a998adb80 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b35e00 0x565167b361e0 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000dbd0 tx=0x7f3a70010000 comp rx=0 tx=0).stop 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.805+0000 7f3a998adb80 1 --2- 172.21.15.46:0/1032575276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.805+0000 7f3a998adb80 1 -- 172.21.15.46:0/1032575276 >> 172.21.15.46:0/1032575276 conn(0x565167a8e210 msgr2=0x565167b2adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.806+0000 7f3a998adb80 1 -- 172.21.15.46:0/1032575276 shutdown_connections 2024-05-08T23:21:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.806+0000 7f3a998adb80 1 -- 172.21.15.46:0/1032575276 wait complete. 2024-05-08T23:21:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.806+0000 7f3a998adb80 1 -- start start 2024-05-08T23:21:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.807+0000 7f3a998adb80 1 Processor -- start 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.809+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b3c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 0x565167b39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b3a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a1c560 con 0x565167b39dd0 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a1c200 con 0x565167b36720 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651679036a0 con 0x565167b30890 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a998adb80 10 client.0 _ll_register_callbacks cb 0x565167a25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b3c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b3c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45858/0 (socket says 172.21.15.46:45858) 2024-05-08T23:21:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a7ea00700 1 -- 172.21.15.46:0/2865986055 learned_addr learned my addr 172.21.15.46:0/2865986055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.810+0000 7f3a7e000700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 0x565167b39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7f400700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b3a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7e000700 1 -- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 msgr2=0x565167b3c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7e000700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b3c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7e000700 1 -- 172.21.15.46:0/2865986055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 msgr2=0x565167b3a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7e000700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b3a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.811+0000 7f3a7e000700 1 -- 172.21.15.46:0/2865986055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7000d8b0 con 0x565167b36720 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.812+0000 7f3a7e000700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 0x565167b39890 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x565167b36b80 tx=0x7f3a6c00d6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b902006d80d8702 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.812+0000 7f3a68c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.812+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c017620 con 0x565167b36720 2024-05-08T23:21:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.812+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3a6c013070 con 0x565167b36720 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.812+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565167aff810 con 0x565167b36720 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.813+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6c017780 con 0x565167b36720 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.813+0000 7f3a998adb80 10 client.30203 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.813+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5651678adb60 con 0x565167b36720 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.813+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565167aff810 con 0x565167b36720 2024-05-08T23:21:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a6c00fa60 con 0x565167b36720 2024-05-08T23:21:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 10 client.30203 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 10 client.30203 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:21:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 20 client.30203 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b41540 0x565167b43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 4 client.30203 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565167a78ec0 con 0x565167b41540 2024-05-08T23:21:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.814+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b41540 0x565167b43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.818+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b41540 0x565167b43960 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26e0cfe9c761412c server_cookie=4739c7c7a7f025d3 in_seq=0 out_seq=0 2024-05-08T23:21:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.818+0000 7f3a68c00700 10 client.30203 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.819+0000 7f3a68c00700 1 -- 172.21.15.46:0/2865986055 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565167a78ec0 con 0x565167b41540 2024-05-08T23:21:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.819+0000 7f3a68c00700 10 client.30203 handle_command_reply: tid=0 2024-05-08T23:21:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.819+0000 7f3a998adb80 1 client.30203 shutdown 2024-05-08T23:21:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.820+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 msgr2=0x565167b39890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.820+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 0x565167b39890 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x565167b36b80 tx=0x7f3a6c00d6a0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 shutdown_connections 2024-05-08T23:21:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b41540 0x565167b43960 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b3a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b3c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2865986055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b36720 0x565167b39890 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a68c00700 0 client.30203 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.821+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 >> 172.21.15.46:0/2865986055 conn(0x565167a8e210 msgr2=0x565167b29d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.822+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 shutdown_connections 2024-05-08T23:21:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.822+0000 7f3a998adb80 1 -- 172.21.15.46:0/2865986055 wait complete. 2024-05-08T23:21:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.823+0000 7f3a998adb80 20 client.30203 trim_cache size 0 max 16384 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.854+0000 7f3a998adb80 1 Processor -- start 2024-05-08T23:21:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.854+0000 7f3a998adb80 1 -- start start 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.854+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b2ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a998adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b3dd80 con 0x565167b36720 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a998adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b37e00 con 0x565167b39dd0 2024-05-08T23:21:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a998adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b43fc0 con 0x565167b30890 2024-05-08T23:21:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a7f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a7f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49274/0 (socket says 172.21.15.46:49274) 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7f400700 1 -- 172.21.15.46:0/345030587 learned_addr learned my addr 172.21.15.46:0/345030587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.855+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7e000700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b2ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7ea00700 1 -- 172.21.15.46:0/345030587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 msgr2=0x565167b2ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7ea00700 1 -- 172.21.15.46:0/345030587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 msgr2=0x565167b2ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b2ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.856+0000 7f3a7ea00700 1 -- 172.21.15.46:0/345030587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167a46560 con 0x565167b30890 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.857+0000 7f3a7e000700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b2ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.857+0000 7f3a7f400700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.857+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30350 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3a70010740 tx=0x7f3a7000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb262a7a80eeae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.858+0000 7f3a75200700 1 -- 172.21.15.46:0/345030587 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7002d040 con 0x565167b30890 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.858+0000 7f3a75200700 1 -- 172.21.15.46:0/345030587 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a700108f0 con 0x565167b30890 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.858+0000 7f3a75200700 1 -- 172.21.15.46:0/345030587 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700052a0 con 0x565167b30890 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 -- 172.21.15.46:0/345030587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 msgr2=0x565167b30350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30350 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3a70010740 tx=0x7f3a7000bf60 comp rx=0 tx=0).stop 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 -- 172.21.15.46:0/345030587 shutdown_connections 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b39dd0 0x565167b2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b30890 0x565167b30350 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 --2- 172.21.15.46:0/345030587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b36720 0x565167b2ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 -- 172.21.15.46:0/345030587 >> 172.21.15.46:0/345030587 conn(0x565167a8e210 msgr2=0x565167b3a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 -- 172.21.15.46:0/345030587 shutdown_connections 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.859+0000 7f3a998adb80 1 -- 172.21.15.46:0/345030587 wait complete. 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.860+0000 7f3a998adb80 1 -- start start 2024-05-08T23:21:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.860+0000 7f3a998adb80 1 Processor -- start 2024-05-08T23:21:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.862+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.862+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 0x565167b4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.862+0000 7f3a998adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b4e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.863+0000 7f3a7e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 0x565167b4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.863+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.863+0000 7f3a7ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49278/0 (socket says 172.21.15.46:49278) 2024-05-08T23:21:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a7f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b4e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a7e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 0x565167b4dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45868/0 (socket says 172.21.15.46:45868) 2024-05-08T23:21:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b4e9a0 con 0x565167b39dd0 2024-05-08T23:21:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a7ea00700 1 -- 172.21.15.46:0/2890654297 learned_addr learned my addr 172.21.15.46:0/2890654297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b31860 con 0x565167b30890 2024-05-08T23:21:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b31a20 con 0x565167b36720 2024-05-08T23:21:43.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.864+0000 7f3a998adb80 10 client.0 _ll_register_callbacks cb 0x565167a25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7f400700 1 -- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 msgr2=0x565167b4dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7f400700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 0x565167b4dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7f400700 1 -- 172.21.15.46:0/2890654297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 msgr2=0x565167b48630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7f400700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7f400700 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a801a60f0 con 0x565167b39dd0 2024-05-08T23:21:43.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.865+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:43.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.866+0000 7f3a7f400700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b4e290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a78096f30 tx=0x7f3a7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41170a3f9b247e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.866+0000 7f3a68c00700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.866+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7809ae90 con 0x565167b39dd0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.866+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3a7809b130 con 0x565167b39dd0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.867+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651679d9270 con 0x565167b39dd0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.867+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a7809aa30 con 0x565167b39dd0 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.867+0000 7f3a998adb80 10 client.20043 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:43.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.867+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5651679013b0 con 0x565167b39dd0 2024-05-08T23:21:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651679d9270 con 0x565167b39dd0 2024-05-08T23:21:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a780a2990 con 0x565167b39dd0 2024-05-08T23:21:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 10 client.20043 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 10 client.20043 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:21:43.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 20 client.20043 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b32f80 0x565167bb4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 4 client.20043 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651678cecf0 con 0x565167b32f80 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.868+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b32f80 0x565167bb4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.869+0000 7f3a7ea00700 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b32f80 0x565167bb4300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7002e0b359d28987 server_cookie=911c66eb7ae3265 in_seq=0 out_seq=0 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.869+0000 7f3a68c00700 10 client.20043 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.869+0000 7f3a68c00700 1 -- 172.21.15.46:0/2890654297 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651678cecf0 con 0x565167b32f80 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.869+0000 7f3a68c00700 10 client.20043 handle_command_reply: tid=0 2024-05-08T23:21:43.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.869+0000 7f3a998adb80 1 client.20043 shutdown 2024-05-08T23:21:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.870+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 msgr2=0x565167b4e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.870+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b4e290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a78096f30 tx=0x7f3a7800b330 comp rx=0 tx=0).stop 2024-05-08T23:21:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 shutdown_connections 2024-05-08T23:21:43.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x565167b32f80 0x565167bb4300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565167b39dd0 0x565167b4e290 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565167b30890 0x565167b48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 --2- 172.21.15.46:0/2890654297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565167b36720 0x565167b4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a68c00700 0 client.20043 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 >> 172.21.15.46:0/2890654297 conn(0x565167a8e210 msgr2=0x565167b38df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 shutdown_connections 2024-05-08T23:21:43.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.871+0000 7f3a998adb80 1 -- 172.21.15.46:0/2890654297 wait complete. 2024-05-08T23:21:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.872+0000 7f3a998adb80 20 client.20043 trim_cache size 0 max 16384 2024-05-08T23:21:43.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 msgr2=0x7f3a640a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 0x7f3a640a3830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005e50 tx=0x7f3a70005de0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 msgr2=0x7f3a8019de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a78004c80 tx=0x7f3a78004d60 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 shutdown_connections 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a8010b400 0x7f3a8019de40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80101fe0 0x7f3a801995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3a640a1370 0x7f3a640a3830 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 --2- 172.21.15.46:0/775116915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a80102900 0x7f3a80199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 >> 172.21.15.46:0/775116915 conn(0x7f3a800fb760 msgr2=0x7f3a800fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 shutdown_connections 2024-05-08T23:21:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:43.873+0000 7f3a85e00700 1 -- 172.21.15.46:0/775116915 wait complete. 2024-05-08T23:21:43.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:21:44.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:44 smithi053 ceph-mon[77587]: pgmap v562: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.1 MiB/s wr, 320 op/s 2024-05-08T23:21:44.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95106 19.80 MB/sec execute 125 sec latency 1635.780 ms 2024-05-08T23:21:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:44 smithi046 ceph-mon[36898]: pgmap v562: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.1 MiB/s wr, 320 op/s 2024-05-08T23:21:44.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:44 smithi086 ceph-mon[78610]: pgmap v562: 129 pgs: 129 active+clean; 1019 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.1 MiB/s wr, 320 op/s 2024-05-08T23:21:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 -- 172.21.15.46:0/1125279478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 msgr2=0x7f42d4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 --2- 172.21.15.46:0/1125279478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4102380 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009990 tx=0x7f42c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 -- 172.21.15.46:0/1125279478 shutdown_connections 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 --2- 172.21.15.46:0/1125279478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d410b3c0 0x7f42d410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 --2- 172.21.15.46:0/1125279478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4102380 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 --2- 172.21.15.46:0/1125279478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d41028c0 0x7f42d410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 -- 172.21.15.46:0/1125279478 >> 172.21.15.46:0/1125279478 conn(0x7f42d40fb760 msgr2=0x7f42d40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 -- 172.21.15.46:0/1125279478 shutdown_connections 2024-05-08T23:21:44.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.590+0000 7f42dc800700 1 -- 172.21.15.46:0/1125279478 wait complete. 2024-05-08T23:21:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.592+0000 7f42dc800700 1 Processor -- start 2024-05-08T23:21:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.592+0000 7f42dc800700 1 -- start start 2024-05-08T23:21:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42dc800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42dc800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42dc800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d410b3c0 0x7f42d4108990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45884/0 (socket says 172.21.15.46:45884) 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.593+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49302/0 (socket says 172.21.15.46:49302) 2024-05-08T23:21:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42da800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d410b3c0 0x7f42d4108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42d9e00700 1 -- 172.21.15.46:0/2195669238 learned_addr learned my addr 172.21.15.46:0/2195669238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d41a9b80 con 0x7f42d410b3c0 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d41a9cc0 con 0x7f42d4101fa0 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d41a9e00 con 0x7f42d41028c0 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.594+0000 7f42d9e00700 1 -- 172.21.15.46:0/2195669238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 msgr2=0x7f42d4108450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9e00700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9e00700 1 -- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d410b3c0 msgr2=0x7f42d4108990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9e00700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d410b3c0 0x7f42d4108990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9e00700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80095f0 con 0x7f42d4101fa0 2024-05-08T23:21:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9400700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:44.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.595+0000 7f42d9e00700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42c80095c0 tx=0x7f42c800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.596+0000 7f42cea00700 1 -- 172.21.15.46:0/2195669238 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c801d070 con 0x7f42d4101fa0 2024-05-08T23:21:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.596+0000 7f42cea00700 1 -- 172.21.15.46:0/2195669238 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c80054f0 con 0x7f42d4101fa0 2024-05-08T23:21:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.596+0000 7f42cea00700 1 -- 172.21.15.46:0/2195669238 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8004bd0 con 0x7f42d4101fa0 2024-05-08T23:21:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.596+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d41aa080 con 0x7f42d4101fa0 2024-05-08T23:21:44.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.596+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d41aa4e0 con 0x7f42d4101fa0 2024-05-08T23:21:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.603+0000 7f42f03c6b80 1 Processor -- start 2024-05-08T23:21:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.603+0000 7f42f03c6b80 1 -- start start 2024-05-08T23:21:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.603+0000 7f42cea00700 1 -- 172.21.15.46:0/2195669238 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f42c8004d30 con 0x7f42d4101fa0 2024-05-08T23:21:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.603+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb223331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.603+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb2232dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22284150 con 0x55fb2232d890 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22281b10 con 0x55fb22332e00 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2227d400 con 0x55fb22333720 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42da800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb223331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45890/0 (socket says 172.21.15.46:45890) 2024-05-08T23:21:44.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 -- 172.21.15.46:0/1527858539 learned_addr learned my addr 172.21.15.46:0/1527858539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9400700 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb2232dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 -- 172.21.15.46:0/1527858539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 msgr2=0x55fb223331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb223331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 -- 172.21.15.46:0/1527858539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 msgr2=0x55fb2232dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb2232dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.604+0000 7f42d9e00700 1 -- 172.21.15.46:0/1527858539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb220fe3b0 con 0x55fb22333720 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.605+0000 7f42da800700 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb223331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.605+0000 7f42d9e00700 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f42c80cb000 tx=0x7f42c80d4400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f07fc4836681e157 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.605+0000 7f42c3000700 1 -- 172.21.15.46:0/1527858539 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80d9070 con 0x55fb22333720 2024-05-08T23:21:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.605+0000 7f42c3000700 1 -- 172.21.15.46:0/1527858539 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c80d4d90 con 0x55fb22333720 2024-05-08T23:21:44.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.605+0000 7f42c3000700 1 -- 172.21.15.46:0/1527858539 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80db590 con 0x55fb22333720 2024-05-08T23:21:44.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.606+0000 7f42f03c6b80 1 -- 172.21.15.46:0/1527858539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 msgr2=0x55fb2232d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.606+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f42c80cb000 tx=0x7f42c80d4400 comp rx=0 tx=0).stop 2024-05-08T23:21:44.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 -- 172.21.15.46:0/1527858539 shutdown_connections 2024-05-08T23:21:44.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb2232dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb223331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/1527858539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2232d350 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 -- 172.21.15.46:0/1527858539 >> 172.21.15.46:0/1527858539 conn(0x55fb2228b210 msgr2=0x55fb22327df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.608+0000 7f42f03c6b80 1 -- 172.21.15.46:0/1527858539 shutdown_connections 2024-05-08T23:21:44.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.609+0000 7f42f03c6b80 1 -- 172.21.15.46:0/1527858539 wait complete. 2024-05-08T23:21:44.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.609+0000 7f42cea00700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 0x7f42bc0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.609+0000 7f42cea00700 1 -- 172.21.15.46:0/2195669238 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f42c80cdd50 con 0x7f42d4101fa0 2024-05-08T23:21:44.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.610+0000 7f42d9400700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 0x7f42bc0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.610+0000 7f42f03c6b80 1 -- start start 2024-05-08T23:21:44.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.611+0000 7f42f03c6b80 1 Processor -- start 2024-05-08T23:21:44.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.611+0000 7f42d9400700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 0x7f42bc0a36e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42d4109640 tx=0x7f42c400b360 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb22339750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb22336a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb22337180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22219560 con 0x55fb2232d890 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22219200 con 0x55fb22332e00 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.612+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb221006a0 con 0x55fb22333720 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.613+0000 7f42d9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb22336a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.613+0000 7f42d9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb22336a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49314/0 (socket says 172.21.15.46:49314) 2024-05-08T23:21:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.613+0000 7f42d9400700 1 -- 172.21.15.46:0/537412831 learned_addr learned my addr 172.21.15.46:0/537412831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:44.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.613+0000 7f42d9e00700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb22339750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.613+0000 7f42f03c6b80 10 client.0 _ll_register_callbacks cb 0x55fb22222b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 -- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 msgr2=0x55fb22337180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb22337180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 -- 172.21.15.46:0/537412831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 msgr2=0x55fb22336a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb22336a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 -- 172.21.15.46:0/537412831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d41aa4e0 con 0x55fb2232d890 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42d9e00700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb22339750 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42c809aa70 tx=0x7f42c8097480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947f005269e2b336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.614+0000 7f42b7400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80d1370 con 0x55fb2232d890 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f42c8005660 con 0x55fb2232d890 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb221d6270 con 0x55fb2232d890 2024-05-08T23:21:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42c80d2be0 con 0x55fb2232d890 2024-05-08T23:21:44.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42f03c6b80 10 client.20052 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:44.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.615+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb220aab60 con 0x55fb2232d890 2024-05-08T23:21:44.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.616+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb221d6270 con 0x55fb2232d890 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42c80a2a40 con 0x55fb2232d890 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 10 client.20052 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 10 client.20052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 20 client.20052 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb22346970 0x55fb22348d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 4 client.20052 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb222fc810 con 0x55fb22346970 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.617+0000 7f42d9400700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb22346970 0x55fb22348d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.618+0000 7f42d9400700 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb22346970 0x55fb22348d90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=160f162598b1ef9e server_cookie=bf096d6f0bbc81e1 in_seq=0 out_seq=0 2024-05-08T23:21:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.618+0000 7f42b7400700 10 client.20052 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.631+0000 7f42b7400700 1 -- 172.21.15.46:0/537412831 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb222fc810 con 0x55fb22346970 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.631+0000 7f42b7400700 10 client.20052 handle_command_reply: tid=0 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.631+0000 7f42f03c6b80 1 client.20052 shutdown 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.632+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 msgr2=0x55fb22339750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.632+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb22339750 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42c809aa70 tx=0x7f42c8097480 comp rx=0 tx=0).stop 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 shutdown_connections 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb22337180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb22346970 0x55fb22348d90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2232d890 0x55fb22339750 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/537412831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb22336a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 >> 172.21.15.46:0/537412831 conn(0x55fb2228b210 msgr2=0x55fb223275a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42b7400700 0 client.20052 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 shutdown_connections 2024-05-08T23:21:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.633+0000 7f42f03c6b80 1 -- 172.21.15.46:0/537412831 wait complete. 2024-05-08T23:21:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.634+0000 7f42f03c6b80 20 client.20052 trim_cache size 0 max 16384 2024-05-08T23:21:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.659+0000 7f42f03c6b80 1 Processor -- start 2024-05-08T23:21:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.659+0000 7f42f03c6b80 1 -- start start 2024-05-08T23:21:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.660+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.660+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 0x55fb22327d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.660+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22333720 0x55fb22336380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.660+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.660+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45906/0 (socket says 172.21.15.46:45906) 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 -- 172.21.15.46:0/4000722646 learned_addr learned my addr 172.21.15.46:0/4000722646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22324dc0 con 0x55fb22332e00 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb22329e50 con 0x55fb22333720 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2234df60 con 0x55fb2232d890 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9400700 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 0x55fb22327d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 -- 172.21.15.46:0/4000722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22333720 msgr2=0x55fb22336380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22333720 0x55fb22336380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 -- 172.21.15.46:0/4000722646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 msgr2=0x55fb22327d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 0x55fb22327d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 -- 172.21.15.46:0/4000722646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb22243560 con 0x55fb2232d890 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9400700 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 0x55fb22327d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.661+0000 7f42d9e00700 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f42c8004780 tx=0x7f42c8096fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4093affc3dfda75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.662+0000 7f42c3a00700 1 -- 172.21.15.46:0/4000722646 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80d9020 con 0x55fb2232d890 2024-05-08T23:21:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.662+0000 7f42c3a00700 1 -- 172.21.15.46:0/4000722646 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c80d2e90 con 0x55fb2232d890 2024-05-08T23:21:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.662+0000 7f42c3a00700 1 -- 172.21.15.46:0/4000722646 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c809b1e0 con 0x55fb2232d890 2024-05-08T23:21:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 msgr2=0x55fb2232d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f42c8004780 tx=0x7f42c8096fc0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 shutdown_connections 2024-05-08T23:21:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22333720 0x55fb22336380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2232d890 0x55fb2232d350 secure :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f42c8004780 tx=0x7f42c8096fc0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/4000722646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb22332e00 0x55fb22327d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.663+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 >> 172.21.15.46:0/4000722646 conn(0x55fb2228b210 msgr2=0x55fb22330c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.664+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 shutdown_connections 2024-05-08T23:21:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.664+0000 7f42f03c6b80 1 -- 172.21.15.46:0/4000722646 wait complete. 2024-05-08T23:21:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.664+0000 7f42f03c6b80 1 -- start start 2024-05-08T23:21:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.665+0000 7f42f03c6b80 1 Processor -- start 2024-05-08T23:21:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2233c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2233caa0 0x55fb2239be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2239c4d0 con 0x55fb2233caa0 2024-05-08T23:21:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2239c640 con 0x55fb22332e00 2024-05-08T23:21:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb223ad810 con 0x55fb22333720 2024-05-08T23:21:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.667+0000 7f42f03c6b80 10 client.0 _ll_register_callbacks cb 0x55fb22222b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49328/0 (socket says 172.21.15.46:49328) 2024-05-08T23:21:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2233c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2233c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45922/0 (socket says 172.21.15.46:45922) 2024-05-08T23:21:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42d9400700 1 -- 172.21.15.46:0/3981672432 learned_addr learned my addr 172.21.15.46:0/3981672432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.668+0000 7f42da800700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2233caa0 0x55fb2239be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.669+0000 7f42d9e00700 1 -- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 msgr2=0x55fb2233c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.669+0000 7f42d9e00700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2233c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.669+0000 7f42d9e00700 1 -- 172.21.15.46:0/3981672432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2233caa0 msgr2=0x55fb2239be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.669+0000 7f42d9e00700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2233caa0 0x55fb2239be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.669+0000 7f42d9e00700 1 -- 172.21.15.46:0/3981672432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb220aab60 con 0x55fb22332e00 2024-05-08T23:21:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42d9e00700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55fb22333b80 tx=0x7f42c8097480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3508e1bfe0173b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42b7400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80d9050 con 0x55fb22332e00 2024-05-08T23:21:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f42c8005660 con 0x55fb22332e00 2024-05-08T23:21:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42c80d4d90 con 0x55fb22332e00 2024-05-08T23:21:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42c809a440 con 0x55fb22332e00 2024-05-08T23:21:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42f03c6b80 10 client.30224 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.670+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb22102cd0 con 0x55fb22332e00 2024-05-08T23:21:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.671+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c80d4d90 con 0x55fb22332e00 2024-05-08T23:21:44.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42c80b30a0 con 0x55fb22332e00 2024-05-08T23:21:44.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 10 client.30224 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 10 client.30224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 20 client.30224 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb223aeb00 0x55fb223b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 4 client.30224 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb22275ec0 con 0x55fb223aeb00 2024-05-08T23:21:44.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.672+0000 7f42d9400700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb223aeb00 0x55fb223b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:44.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.673+0000 7f42d9400700 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb223aeb00 0x55fb223b0ef0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=865e24ed041e48e1 server_cookie=227114715226624a in_seq=0 out_seq=0 2024-05-08T23:21:44.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.673+0000 7f42b7400700 10 client.30224 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.676+0000 7f42b7400700 1 -- 172.21.15.46:0/3981672432 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb22275ec0 con 0x55fb223aeb00 2024-05-08T23:21:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.676+0000 7f42b7400700 10 client.30224 handle_command_reply: tid=0 2024-05-08T23:21:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.676+0000 7f42f03c6b80 1 client.30224 shutdown 2024-05-08T23:21:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 msgr2=0x55fb2233bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55fb22333b80 tx=0x7f42c8097480 comp rx=0 tx=0).stop 2024-05-08T23:21:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 shutdown_connections 2024-05-08T23:21:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x55fb223aeb00 0x55fb223b0ef0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb2233caa0 0x55fb2239be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb22332e00 0x55fb2233bec0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 --2- 172.21.15.46:0/3981672432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb22333720 0x55fb2233c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 >> 172.21.15.46:0/3981672432 conn(0x55fb2228b210 msgr2=0x55fb22331bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42b7400700 0 client.30224 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 shutdown_connections 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.677+0000 7f42f03c6b80 1 -- 172.21.15.46:0/3981672432 wait complete. 2024-05-08T23:21:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.678+0000 7f42f03c6b80 20 client.30224 trim_cache size 0 max 16384 2024-05-08T23:21:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 msgr2=0x7f42bc0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 0x7f42bc0a36e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42d4109640 tx=0x7f42c400b360 comp rx=0 tx=0).stop 2024-05-08T23:21:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 msgr2=0x7f42d4107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42c80095c0 tx=0x7f42c800f690 comp rx=0 tx=0).stop 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 shutdown_connections 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f42bc0a1220 0x7f42bc0a36e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d410b3c0 0x7f42d4108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d4101fa0 0x7f42d4107f10 secure :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42c80095c0 tx=0x7f42c800f690 comp rx=0 tx=0).stop 2024-05-08T23:21:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 --2- 172.21.15.46:0/2195669238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d41028c0 0x7f42d4108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.680+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 >> 172.21.15.46:0/2195669238 conn(0x7f42d40fb760 msgr2=0x7f42d40fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.681+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 shutdown_connections 2024-05-08T23:21:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:44.681+0000 7f42dc800700 1 -- 172.21.15.46:0/2195669238 wait complete. 2024-05-08T23:21:44.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:21:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.049+0000 7f308a400700 1 -- 172.21.15.46:0/1898579384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f308410d800 msgr2=0x7f308410e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.049+0000 7f308a400700 1 --2- 172.21.15.46:0/1898579384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f308410d800 0x7f308410e150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3074009990 tx=0x7f3074009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.050+0000 7f308a400700 1 -- 172.21.15.46:0/1898579384 shutdown_connections 2024-05-08T23:21:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.050+0000 7f308a400700 1 --2- 172.21.15.46:0/1898579384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f308410d800 0x7f308410e150 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.051+0000 7f308a400700 1 --2- 172.21.15.46:0/1898579384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3084108210 0x7f30841085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.051+0000 7f308a400700 1 --2- 172.21.15.46:0/1898579384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108bc0 0x7f308410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.051+0000 7f308a400700 1 -- 172.21.15.46:0/1898579384 >> 172.21.15.46:0/1898579384 conn(0x7f308406bd20 msgr2=0x7f308406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.051+0000 7f308a400700 1 -- 172.21.15.46:0/1898579384 shutdown_connections 2024-05-08T23:21:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.051+0000 7f308a400700 1 -- 172.21.15.46:0/1898579384 wait complete. 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.053+0000 7f308a400700 1 Processor -- start 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.053+0000 7f308a400700 1 -- start start 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.053+0000 7f308a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108210 0x7f30841a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.053+0000 7f308a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308410d800 0x7f30841a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.053+0000 7f308a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f308a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841ab270 con 0x7f308410d800 2024-05-08T23:21:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f308a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841ab3e0 con 0x7f30841aa7f0 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f308a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841ab5a0 con 0x7f3084108210 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3083e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308410d800 0x7f30841a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49334/0 (socket says 172.21.15.46:49334) 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 -- 172.21.15.46:0/3541108349 learned_addr learned my addr 172.21.15.46:0/3541108349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089000700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108210 0x7f30841a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 -- 172.21.15.46:0/3541108349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108210 msgr2=0x7f30841a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108210 0x7f30841a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 -- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308410d800 msgr2=0x7f30841a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308410d800 0x7f30841a6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.054+0000 7f3089a00700 1 -- 172.21.15.46:0/3541108349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740095f0 con 0x7f30841aa7f0 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.055+0000 7f3089a00700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f30740098e0 tx=0x7f3074003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.055+0000 7f3081600700 1 -- 172.21.15.46:0/3541108349 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307401d070 con 0x7f30841aa7f0 2024-05-08T23:21:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.055+0000 7f3081600700 1 -- 172.21.15.46:0/3541108349 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3074003e20 con 0x7f30841aa7f0 2024-05-08T23:21:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.056+0000 7f3081600700 1 -- 172.21.15.46:0/3541108349 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400fdd0 con 0x7f30841aa7f0 2024-05-08T23:21:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.056+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841b2aa0 con 0x7f30841aa7f0 2024-05-08T23:21:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.056+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841b2ef0 con 0x7f30841aa7f0 2024-05-08T23:21:45.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.064+0000 7f3081600700 1 -- 172.21.15.46:0/3541108349 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f307400f460 con 0x7f30841aa7f0 2024-05-08T23:21:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.066+0000 7f3081600700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 0x7f30680a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.067+0000 7f3081600700 1 -- 172.21.15.46:0/3541108349 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f30740ce370 con 0x7f30841aa7f0 2024-05-08T23:21:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.068+0000 7f3089000700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 0x7f30680a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.069+0000 7f309f4d2b80 1 Processor -- start 2024-05-08T23:21:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.070+0000 7f309f4d2b80 1 -- start start 2024-05-08T23:21:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.070+0000 7f3089000700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 0x7f30680a38f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3070006fd0 tx=0x7f3070008040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.072+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.072+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37929350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.072+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.072+0000 7f3089000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f37880150 con 0x564f3792f050 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37929350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3089a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3089a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49348/0 (socket says 172.21.15.46:49348) 2024-05-08T23:21:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3089a00700 1 -- 172.21.15.46:0/4208975493 learned_addr learned my addr 172.21.15.46:0/4208975493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3787db10 con 0x564f37929890 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f37879400 con 0x564f3792f970 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 -- 172.21.15.46:0/4208975493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 msgr2=0x564f37929cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 -- 172.21.15.46:0/4208975493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 msgr2=0x564f3792f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.073+0000 7f3083e00700 1 -- 172.21.15.46:0/4208975493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f376fa3b0 con 0x564f3792f970 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.074+0000 7f3083e00700 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37929350 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f308406cb10 tx=0x7f307c00c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ec54e48dc32c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.074+0000 7f307a000700 1 -- 172.21.15.46:0/4208975493 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00cc60 con 0x564f3792f970 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.074+0000 7f307a000700 1 -- 172.21.15.46:0/4208975493 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f307c00cdc0 con 0x564f3792f970 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.074+0000 7f307a000700 1 -- 172.21.15.46:0/4208975493 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0107f0 con 0x564f3792f970 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.075+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 msgr2=0x564f37929350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.075+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37929350 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f308406cb10 tx=0x7f307c00c190 comp rx=0 tx=0).stop 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 shutdown_connections 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/4208975493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37929350 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 >> 172.21.15.46:0/4208975493 conn(0x564f37887210 msgr2=0x564f379242d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 shutdown_connections 2024-05-08T23:21:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.076+0000 7f309f4d2b80 1 -- 172.21.15.46:0/4208975493 wait complete. 2024-05-08T23:21:45.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.077+0000 7f309f4d2b80 1 -- start start 2024-05-08T23:21:45.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.077+0000 7f309f4d2b80 1 Processor -- start 2024-05-08T23:21:45.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f37929890 0x564f37935910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37933340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f37815560 con 0x564f37929890 2024-05-08T23:21:45.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f37815200 con 0x564f3792f050 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f376fc6a0 con 0x564f3792f970 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f309f4d2b80 10 client.0 _ll_register_callbacks cb 0x564f3781eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.079+0000 7f3083e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3089a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37933340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49352/0 (socket says 172.21.15.46:49352) 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 -- 172.21.15.46:0/354729912 learned_addr learned my addr 172.21.15.46:0/354729912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 -- 172.21.15.46:0/354729912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 msgr2=0x564f37933340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37933340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 -- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f37929890 msgr2=0x564f37935910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f37929890 0x564f37935910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 -- 172.21.15.46:0/354729912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c007960 con 0x564f3792f050 2024-05-08T23:21:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f3083e00700 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f307c011f40 tx=0x7f307c011660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c03ae4ebb44974a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.080+0000 7f306ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.082+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c01a650 con 0x564f3792f050 2024-05-08T23:21:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.082+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f307c00d040 con 0x564f3792f050 2024-05-08T23:21:45.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.082+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f377d2270 con 0x564f3792f050 2024-05-08T23:21:45.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.083+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307c011d60 con 0x564f3792f050 2024-05-08T23:21:45.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.083+0000 7f309f4d2b80 10 client.30236 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:45.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.083+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x564f376a6b60 con 0x564f3792f050 2024-05-08T23:21:45.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f377d2270 con 0x564f3792f050 2024-05-08T23:21:45.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f307c0197e0 con 0x564f3792f050 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 10 client.30236 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 10 client.30236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 20 client.30236 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37943290 0x564f379456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 4 client.30236 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.084+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f378f8810 con 0x564f37943290 2024-05-08T23:21:45.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.085+0000 7f3089000700 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37943290 0x564f379456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.086+0000 7f3089000700 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37943290 0x564f379456b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7811b55cc4eec74a server_cookie=dff725cc35cc08d6 in_seq=0 out_seq=0 2024-05-08T23:21:45.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.086+0000 7f306ea00700 10 client.30236 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.087+0000 7f306ea00700 1 -- 172.21.15.46:0/354729912 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f378f8810 con 0x564f37943290 2024-05-08T23:21:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.087+0000 7f306ea00700 10 client.30236 handle_command_reply: tid=0 2024-05-08T23:21:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.087+0000 7f309f4d2b80 1 client.30236 shutdown 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 msgr2=0x564f37932c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f307c011f40 tx=0x7f307c011660 comp rx=0 tx=0).stop 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 shutdown_connections 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f37933340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37943290 0x564f379456b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f37929890 0x564f37935910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/354729912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f3792f050 0x564f37932c00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 >> 172.21.15.46:0/354729912 conn(0x564f37887210 msgr2=0x564f379240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.088+0000 7f306ea00700 0 client.30236 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.089+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 shutdown_connections 2024-05-08T23:21:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.089+0000 7f309f4d2b80 1 -- 172.21.15.46:0/354729912 wait complete. 2024-05-08T23:21:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.089+0000 7f309f4d2b80 20 client.30236 trim_cache size 0 max 16384 2024-05-08T23:21:45.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.111+0000 7f309f4d2b80 1 Processor -- start 2024-05-08T23:21:45.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.111+0000 7f309f4d2b80 1 -- start start 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f379240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3792a240 con 0x564f3792f050 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f379332e0 con 0x564f37929890 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f379334a0 con 0x564f3792f970 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45946/0 (socket says 172.21.15.46:45946) 2024-05-08T23:21:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 -- 172.21.15.46:0/2530095284 learned_addr learned my addr 172.21.15.46:0/2530095284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3083e00700 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f379240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 -- 172.21.15.46:0/2530095284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 msgr2=0x564f37929c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 -- 172.21.15.46:0/2530095284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 msgr2=0x564f379240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f379240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.112+0000 7f3089a00700 1 -- 172.21.15.46:0/2530095284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f3783f560 con 0x564f3792f970 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.113+0000 7f3089a00700 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f30740975a0 tx=0x7f307400f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd19ccb1dd37109d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.113+0000 7f307aa00700 1 -- 172.21.15.46:0/2530095284 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740ce150 con 0x564f3792f970 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.113+0000 7f307aa00700 1 -- 172.21.15.46:0/2530095284 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f307409b0a0 con 0x564f3792f970 2024-05-08T23:21:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.113+0000 7f307aa00700 1 -- 172.21.15.46:0/2530095284 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740cb8d0 con 0x564f3792f970 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 -- 172.21.15.46:0/2530095284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 msgr2=0x564f379452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f30740975a0 tx=0x7f307400f870 comp rx=0 tx=0).stop 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 -- 172.21.15.46:0/2530095284 shutdown_connections 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f379452a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f37929c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/2530095284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f379240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 -- 172.21.15.46:0/2530095284 >> 172.21.15.46:0/2530095284 conn(0x564f37887210 msgr2=0x564f37925820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 -- 172.21.15.46:0/2530095284 shutdown_connections 2024-05-08T23:21:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.114+0000 7f309f4d2b80 1 -- 172.21.15.46:0/2530095284 wait complete. 2024-05-08T23:21:45.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.115+0000 7f309f4d2b80 1 -- start start 2024-05-08T23:21:45.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.115+0000 7f309f4d2b80 1 Processor -- start 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f3793a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f3792aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3792b700 con 0x564f3792f050 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3792b870 con 0x564f37929890 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3792ba30 con 0x564f3792f970 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f309f4d2b80 10 client.0 _ll_register_callbacks cb 0x564f3781eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f3089000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f3793a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f3083e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f3089000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f3793a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49358/0 (socket says 172.21.15.46:49358) 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f3089000700 1 -- 172.21.15.46:0/1968972502 learned_addr learned my addr 172.21.15.46:0/1968972502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.116+0000 7f3089a00700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f3792aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 -- 172.21.15.46:0/1968972502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 msgr2=0x564f3793a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f3793a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 -- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 msgr2=0x564f3792aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 -- 172.21.15.46:0/1968972502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841b2ef0 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f3089a00700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f3792aff0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x564f379313e0 tx=0x7f3074009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf93704333453d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f306ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307409eed0 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.117+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f307c00cdc0 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3074020df0 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30740b9270 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f309f4d2b80 10 client.30301 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x564f376fecd0 con 0x564f3792f970 2024-05-08T23:21:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f307c00cdc0 con 0x564f3792f970 2024-05-08T23:21:45.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.118+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f307409a590 con 0x564f3792f970 2024-05-08T23:21:45.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 10 client.30301 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:45.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 10 client.30301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 20 client.30301 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37999c00 0x564f3799c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 4 client.30301 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f37871ec0 con 0x564f37999c00 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.119+0000 7f3089000700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37999c00 0x564f3799c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f3089000700 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37999c00 0x564f3799c020 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5e28f24e2c0cbfd server_cookie=e7e50eb4bec05dad in_seq=0 out_seq=0 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f306ea00700 10 client.30301 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f306ea00700 1 -- 172.21.15.46:0/1968972502 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f37871ec0 con 0x564f37999c00 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f306ea00700 10 client.30301 handle_command_reply: tid=0 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f309f4d2b80 1 client.30301 shutdown 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 msgr2=0x564f3792aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.120+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f3792aff0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x564f379313e0 tx=0x7f3074009310 comp rx=0 tx=0).stop 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 shutdown_connections 2024-05-08T23:21:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3792f970 0x564f3792aff0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f37929890 0x564f3793a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x564f37999c00 0x564f3799c020 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 --2- 172.21.15.46:0/1968972502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f3792f050 0x564f3792aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f306ea00700 0 client.30301 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 >> 172.21.15.46:0/1968972502 conn(0x564f37887210 msgr2=0x564f379452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 shutdown_connections 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 1 -- 172.21.15.46:0/1968972502 wait complete. 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.121+0000 7f309f4d2b80 20 client.30301 trim_cache size 0 max 16384 2024-05-08T23:21:45.124 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 msgr2=0x7f30680a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 0x7f30680a38f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3070006fd0 tx=0x7f3070008040 comp rx=0 tx=0).stop 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 msgr2=0x7f30841aac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f30740098e0 tx=0x7f3074003c80 comp rx=0 tx=0).stop 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 shutdown_connections 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30841aa7f0 0x7f30841aac70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3084108210 0x7f30841a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f30680a1430 0x7f30680a38f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 --2- 172.21.15.46:0/3541108349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308410d800 0x7f30841a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.122+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 >> 172.21.15.46:0/3541108349 conn(0x7f308406bd20 msgr2=0x7f308410b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.123+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 shutdown_connections 2024-05-08T23:21:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.123+0000 7f308a400700 1 -- 172.21.15.46:0/3541108349 wait complete. 2024-05-08T23:21:45.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:21:45.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95343 19.77 MB/sec execute 126 sec latency 450.675 ms 2024-05-08T23:21:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 -- 172.21.15.46:0/38096045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102020 msgr2=0x7fa828102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 --2- 172.21.15.46:0/38096045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102020 0x7fa828102400 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009990 tx=0x7fa81c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 -- 172.21.15.46:0/38096045 shutdown_connections 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 --2- 172.21.15.46:0/38096045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa82810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 --2- 172.21.15.46:0/38096045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102020 0x7fa828102400 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 --2- 172.21.15.46:0/38096045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102940 0x7fa82810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 -- 172.21.15.46:0/38096045 >> 172.21.15.46:0/38096045 conn(0x7fa8280fb760 msgr2=0x7fa8280fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:45 smithi086 ceph-mon[78610]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 394 op/s 2024-05-08T23:21:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 -- 172.21.15.46:0/38096045 shutdown_connections 2024-05-08T23:21:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.463+0000 7fa82fe00700 1 -- 172.21.15.46:0/38096045 wait complete. 2024-05-08T23:21:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.465+0000 7fa82fe00700 1 Processor -- start 2024-05-08T23:21:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.465+0000 7fa82fe00700 1 -- start start 2024-05-08T23:21:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.466+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 0x7fa8281a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa828109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810a580 con 0x7fa828102020 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82fe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810a6f0 con 0x7fa82810b3f0 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810a8b0 con 0x7fa828102940 2024-05-08T23:21:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.467+0000 7fa82d400700 1 -- 172.21.15.46:0/3962903717 learned_addr learned my addr 172.21.15.46:0/3962903717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.468+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa828109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82ca00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 0x7fa8281a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82de00700 1 -- 172.21.15.46:0/3962903717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 msgr2=0x7fa8281a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 0x7fa8281a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82de00700 1 -- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 msgr2=0x7fa8281a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82de00700 1 -- 172.21.15.46:0/3962903717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814009520 con 0x7fa82810b3f0 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.469+0000 7fa82d400700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.470+0000 7fa82ca00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 0x7fa8281a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.470+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa828109ed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa82000bb00 tx=0x7fa82000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.471+0000 7fa826000700 1 -- 172.21.15.46:0/3962903717 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000ca30 con 0x7fa82810b3f0 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.471+0000 7fa826000700 1 -- 172.21.15.46:0/3962903717 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82000cb90 con 0x7fa82810b3f0 2024-05-08T23:21:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.471+0000 7fa826000700 1 -- 172.21.15.46:0/3962903717 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200125a0 con 0x7fa82810b3f0 2024-05-08T23:21:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.472+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c0095f0 con 0x7fa82810b3f0 2024-05-08T23:21:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.472+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8281aa480 con 0x7fa82810b3f0 2024-05-08T23:21:45.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.477+0000 7fa826000700 1 -- 172.21.15.46:0/3962903717 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa82000cd00 con 0x7fa82810b3f0 2024-05-08T23:21:45.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.478+0000 7fa84397ab80 1 Processor -- start 2024-05-08T23:21:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.479+0000 7fa84397ab80 1 -- start start 2024-05-08T23:21:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be87d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be877890 0x5648be877cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be7ce150 con 0x5648be87ce00 2024-05-08T23:21:45.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be7cbb10 con 0x5648be877890 2024-05-08T23:21:45.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.480+0000 7fa84397ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be7c7400 con 0x5648be87d720 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.481+0000 7fa826000700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 0x7fa8100a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.481+0000 7fa82ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be87d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.481+0000 7fa82d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:45978/0 (socket says 172.21.15.46:45978) 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82d400700 1 -- 172.21.15.46:0/2223761336 learned_addr learned my addr 172.21.15.46:0/2223761336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 0x7fa8100a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82d400700 1 -- 172.21.15.46:0/2223761336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be877890 msgr2=0x5648be877cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82d400700 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be877890 0x5648be877cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa82d400700 1 -- 172.21.15.46:0/2223761336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 msgr2=0x5648be87d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.482+0000 7fa826000700 1 -- 172.21.15.46:0/3962903717 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa8200cc590 con 0x7fa82810b3f0 2024-05-08T23:21:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.483+0000 7fa82d400700 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be87d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.483+0000 7fa82d400700 1 -- 172.21.15.46:0/2223761336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648be6483b0 con 0x5648be87d720 2024-05-08T23:21:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.484+0000 7fa82de00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 0x7fa8100a3850 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa820016000 tx=0x7fa8200c3880 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.485+0000 7fa82d400700 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b6c0 tx=0x7fa81c00bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf008eb71f4def9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.486+0000 7fa81a600700 1 -- 172.21.15.46:0/2223761336 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01d030 con 0x5648be87d720 2024-05-08T23:21:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.486+0000 7fa81a600700 1 -- 172.21.15.46:0/2223761336 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c01e040 con 0x5648be87d720 2024-05-08T23:21:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.486+0000 7fa81a600700 1 -- 172.21.15.46:0/2223761336 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00f460 con 0x5648be87d720 2024-05-08T23:21:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.487+0000 7fa84397ab80 1 -- 172.21.15.46:0/2223761336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 msgr2=0x5648be877350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.487+0000 7fa84397ab80 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b6c0 tx=0x7fa81c00bb00 comp rx=0 tx=0).stop 2024-05-08T23:21:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 -- 172.21.15.46:0/2223761336 shutdown_connections 2024-05-08T23:21:45.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be877890 0x5648be877cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be87d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 --2- 172.21.15.46:0/2223761336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be877350 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 -- 172.21.15.46:0/2223761336 >> 172.21.15.46:0/2223761336 conn(0x5648be7d5210 msgr2=0x5648be871df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 -- 172.21.15.46:0/2223761336 shutdown_connections 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.488+0000 7fa84397ab80 1 -- 172.21.15.46:0/2223761336 wait complete. 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.489+0000 7fa84397ab80 1 -- start start 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.489+0000 7fa84397ab80 1 Processor -- start 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.491+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be877890 0x5648be880b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.491+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49390/0 (socket says 172.21.15.46:49390) 2024-05-08T23:21:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be881610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 -- 172.21.15.46:0/562483736 learned_addr learned my addr 172.21.15.46:0/562483736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be763560 con 0x5648be877890 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be763200 con 0x5648be87ce00 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be64a6a0 con 0x5648be87d720 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa84397ab80 10 client.0 _ll_register_callbacks cb 0x5648be76cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82ca00700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be877890 0x5648be880b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 -- 172.21.15.46:0/562483736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 msgr2=0x5648be881610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be881610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 -- 172.21.15.46:0/562483736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be877890 msgr2=0x5648be880b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be877890 0x5648be880b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82810e530 con 0x5648be87ce00 2024-05-08T23:21:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.492+0000 7fa82d400700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b6f0 tx=0x7fa81c0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c300f51ddbd1aec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa80ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c031070 con 0x5648be87ce00 2024-05-08T23:21:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa81c005310 con 0x5648be87ce00 2024-05-08T23:21:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648be720270 con 0x5648be87ce00 2024-05-08T23:21:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x5648be87ce00 2024-05-08T23:21:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.493+0000 7fa84397ab80 10 client.30245 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.494+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5648be5f4b60 con 0x5648be87ce00 2024-05-08T23:21:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.494+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648be720270 con 0x5648be87ce00 2024-05-08T23:21:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.496+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa81c034020 con 0x5648be87ce00 2024-05-08T23:21:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 10 client.30245 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 10 client.30245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 20 client.30245 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be890dc0 0x5648be8931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 4 client.30245 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.497+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648be846810 con 0x5648be890dc0 2024-05-08T23:21:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.499+0000 7fa82ca00700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be890dc0 0x5648be8931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.501+0000 7fa82ca00700 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be890dc0 0x5648be8931e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c4a83b4cf126cf server_cookie=d18f0e9547d41f29 in_seq=0 out_seq=0 2024-05-08T23:21:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.501+0000 7fa80ea00700 10 client.30245 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:45.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.502+0000 7fa80ea00700 1 -- 172.21.15.46:0/562483736 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648be846810 con 0x5648be890dc0 2024-05-08T23:21:45.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.502+0000 7fa80ea00700 10 client.30245 handle_command_reply: tid=0 2024-05-08T23:21:45.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.502+0000 7fa84397ab80 1 client.30245 shutdown 2024-05-08T23:21:45.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 msgr2=0x5648be8810d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b6f0 tx=0x7fa81c0047e0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 shutdown_connections 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be87d720 0x5648be881610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be890dc0 0x5648be8931e0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be877890 0x5648be880b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 --2- 172.21.15.46:0/562483736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be8810d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa80ea00700 0 client.30245 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 >> 172.21.15.46:0/562483736 conn(0x5648be7d5210 msgr2=0x5648be8715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.503+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 shutdown_connections 2024-05-08T23:21:45.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.504+0000 7fa84397ab80 1 -- 172.21.15.46:0/562483736 wait complete. 2024-05-08T23:21:45.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.504+0000 7fa84397ab80 20 client.30245 trim_cache size 0 max 16384 2024-05-08T23:21:45.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:45 smithi053 ceph-mon[77587]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 394 op/s 2024-05-08T23:21:45.529 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:45 smithi046 ceph-mon[36898]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 394 op/s 2024-05-08T23:21:45.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.532+0000 7fa84397ab80 1 Processor -- start 2024-05-08T23:21:45.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.532+0000 7fa84397ab80 1 -- start start 2024-05-08T23:21:45.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be877350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87d720 0x5648be881430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be87d430 con 0x5648be87d720 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be872b30 con 0x5648be87ce00 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa84397ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be882190 con 0x5648be877890 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa82d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa82d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49406/0 (socket says 172.21.15.46:49406) 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.533+0000 7fa82d400700 1 -- 172.21.15.46:0/602387969 learned_addr learned my addr 172.21.15.46:0/602387969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 -- 172.21.15.46:0/602387969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 msgr2=0x5648be877350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be877350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 -- 172.21.15.46:0/602387969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87d720 msgr2=0x5648be881430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87d720 0x5648be881430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 -- 172.21.15.46:0/602387969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648be78d560 con 0x5648be87ce00 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.534+0000 7fa82d400700 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0098e0 tx=0x7fa81c005920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4453d856a818d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.535+0000 7fa81b000700 1 -- 172.21.15.46:0/602387969 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c031070 con 0x5648be87ce00 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.535+0000 7fa81b000700 1 -- 172.21.15.46:0/602387969 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c005390 con 0x5648be87ce00 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.535+0000 7fa81b000700 1 -- 172.21.15.46:0/602387969 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x5648be87ce00 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 -- 172.21.15.46:0/602387969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 msgr2=0x5648be871d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0098e0 tx=0x7fa81c005920 comp rx=0 tx=0).stop 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 -- 172.21.15.46:0/602387969 shutdown_connections 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87d720 0x5648be881430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be877350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 --2- 172.21.15.46:0/602387969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87ce00 0x5648be871d20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 -- 172.21.15.46:0/602387969 >> 172.21.15.46:0/602387969 conn(0x5648be7d5210 msgr2=0x5648be87f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 -- 172.21.15.46:0/602387969 shutdown_connections 2024-05-08T23:21:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.536+0000 7fa84397ab80 1 -- 172.21.15.46:0/602387969 wait complete. 2024-05-08T23:21:45.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.537+0000 7fa84397ab80 1 -- start start 2024-05-08T23:21:45.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.537+0000 7fa84397ab80 1 Processor -- start 2024-05-08T23:21:45.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.539+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be886120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.539+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be886660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.539+0000 7fa84397ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa84397ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be8e6370 con 0x5648be87ce00 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa84397ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be8e64e0 con 0x5648be87d720 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa84397ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be8e66a0 con 0x5648be877890 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa84397ab80 10 client.0 _ll_register_callbacks cb 0x5648be76cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa82ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa82ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49416/0 (socket says 172.21.15.46:49416) 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.540+0000 7fa82ca00700 1 -- 172.21.15.46:0/3488866533 learned_addr learned my addr 172.21.15.46:0/3488866533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82d400700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be886660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 -- 172.21.15.46:0/3488866533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 msgr2=0x5648be886120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be886120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 -- 172.21.15.46:0/3488866533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 msgr2=0x5648be886660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be886660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 -- 172.21.15.46:0/3488866533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648be5f4b60 con 0x5648be87d720 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82d400700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be886660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa82ca00700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa814004790 tx=0x7fa81401d3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da9d87afce0eb3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.541+0000 7fa80ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814017040 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa814012860 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa81c01e040 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa814012a20 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa84397ab80 10 client.30251 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.542+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5648be64ccd0 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.543+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c01e040 con 0x5648be87d720 2024-05-08T23:21:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.543+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa81401a070 con 0x5648be87d720 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 10 client.30251 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 10 client.30251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 20 client.30251 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be8e7e80 0x5648be8ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 4 client.30251 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5648be7bfec0 con 0x5648be8e7e80 2024-05-08T23:21:45.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.544+0000 7fa82d400700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be8e7e80 0x5648be8ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.545+0000 7fa82d400700 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be8e7e80 0x5648be8ea270 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd17dc131e870b6a server_cookie=fedcaf66198ffeb1 in_seq=0 out_seq=0 2024-05-08T23:21:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.545+0000 7fa80ea00700 10 client.30251 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.546+0000 7fa80ea00700 1 -- 172.21.15.46:0/3488866533 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5648be7bfec0 con 0x5648be8e7e80 2024-05-08T23:21:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.546+0000 7fa80ea00700 10 client.30251 handle_command_reply: tid=0 2024-05-08T23:21:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.546+0000 7fa84397ab80 1 client.30251 shutdown 2024-05-08T23:21:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 msgr2=0x5648be886ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa814004790 tx=0x7fa81401d3d0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 shutdown_connections 2024-05-08T23:21:45.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648be87d720 0x5648be886ba0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648be877890 0x5648be886120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5648be8e7e80 0x5648be8ea270 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 --2- 172.21.15.46:0/3488866533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648be87ce00 0x5648be886660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 >> 172.21.15.46:0/3488866533 conn(0x5648be7d5210 msgr2=0x5648be87bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.547+0000 7fa80ea00700 0 client.30251 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.548+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 shutdown_connections 2024-05-08T23:21:45.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.549+0000 7fa84397ab80 1 -- 172.21.15.46:0/3488866533 wait complete. 2024-05-08T23:21:45.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.550+0000 7fa84397ab80 20 client.30251 trim_cache size 0 max 16384 2024-05-08T23:21:45.551 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 msgr2=0x7fa8100a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 0x7fa8100a3850 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa820016000 tx=0x7fa8200c3880 comp rx=0 tx=0).stop 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 msgr2=0x7fa828109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa828109ed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa82000bb00 tx=0x7fa82000be10 comp rx=0 tx=0).stop 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 shutdown_connections 2024-05-08T23:21:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82810b3f0 0x7fa828109ed0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa8100a1390 0x7fa8100a3850 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828102020 0x7fa8281a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 --2- 172.21.15.46:0/3962903717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa828102940 0x7fa8281a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.552+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 >> 172.21.15.46:0/3962903717 conn(0x7fa8280fb760 msgr2=0x7fa8280fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.553+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 shutdown_connections 2024-05-08T23:21:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:45.553+0000 7fa82fe00700 1 -- 172.21.15.46:0/3962903717 wait complete. 2024-05-08T23:21:46.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95699 19.73 MB/sec execute 127 sec latency 201.016 ms 2024-05-08T23:21:46.592 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:21:46.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 -- 172.21.15.46:0/934429441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988101a00 msgr2=0x7fe988105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 --2- 172.21.15.46:0/934429441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988101a00 0x7fe988105a40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe974009990 tx=0x7fe974009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 -- 172.21.15.46:0/934429441 shutdown_connections 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 --2- 172.21.15.46:0/934429441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988106170 0x7fe98810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 --2- 172.21.15.46:0/934429441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe988101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 --2- 172.21.15.46:0/934429441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988101a00 0x7fe988105a40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.937+0000 7fe98ce00700 1 -- 172.21.15.46:0/934429441 >> 172.21.15.46:0/934429441 conn(0x7fe9880fc8b0 msgr2=0x7fe9880fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.938+0000 7fe98ce00700 1 -- 172.21.15.46:0/934429441 shutdown_connections 2024-05-08T23:21:46.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.938+0000 7fe98ce00700 1 -- 172.21.15.46:0/934429441 wait complete. 2024-05-08T23:21:46.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.940+0000 7fe98ce00700 1 Processor -- start 2024-05-08T23:21:46.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.940+0000 7fe98ce00700 1 -- start start 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe98810daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988106170 0x7fe98810abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 0x7fe98810b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881a9de0 con 0x7fe988101050 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881a9f50 con 0x7fe988106170 2024-05-08T23:21:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.941+0000 7fe98ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881aa110 con 0x7fe98810b100 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 0x7fe98810b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe98810daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 0x7fe98810b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46004/0 (socket says 172.21.15.46:46004) 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe986800700 1 -- 172.21.15.46:0/3806700590 learned_addr learned my addr 172.21.15.46:0/3806700590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985400700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988106170 0x7fe98810abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 -- 172.21.15.46:0/3806700590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 msgr2=0x7fe98810b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 0x7fe98810b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 -- 172.21.15.46:0/3806700590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988106170 msgr2=0x7fe98810abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988106170 0x7fe98810abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.942+0000 7fe985e00700 1 -- 172.21.15.46:0/3806700590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740095f0 con 0x7fe988101050 2024-05-08T23:21:46.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.943+0000 7fe985e00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe98810daf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c970 tx=0x7fe97800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.945+0000 7fe97ea00700 1 -- 172.21.15.46:0/3806700590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978004440 con 0x7fe988101050 2024-05-08T23:21:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.945+0000 7fe97ea00700 1 -- 172.21.15.46:0/3806700590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe978003b40 con 0x7fe988101050 2024-05-08T23:21:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.945+0000 7fe97ea00700 1 -- 172.21.15.46:0/3806700590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978007bc0 con 0x7fe988101050 2024-05-08T23:21:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.945+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9881aa3f0 con 0x7fe988101050 2024-05-08T23:21:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.945+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9881aa8b0 con 0x7fe988101050 2024-05-08T23:21:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.955+0000 7fe97ea00700 1 -- 172.21.15.46:0/3806700590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe978003cb0 con 0x7fe988101050 2024-05-08T23:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.957+0000 7fe97ea00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 0x7fe96c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.957+0000 7fe9a08d8b80 1 Processor -- start 2024-05-08T23:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.957+0000 7fe97ea00700 1 -- 172.21.15.46:0/3806700590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe9780ccf50 con 0x7fe988101050 2024-05-08T23:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.958+0000 7fe985400700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 0x7fe96c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.958+0000 7fe9a08d8b80 1 -- start start 2024-05-08T23:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.958+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d24f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d1a01d0 con 0x55cf9d24f7a0 2024-05-08T23:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d19db90 con 0x55cf9d249910 2024-05-08T23:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d199480 con 0x55cf9d24ee80 2024-05-08T23:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe986800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe986800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49442/0 (socket says 172.21.15.46:49442) 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.959+0000 7fe986800700 1 -- 172.21.15.46:0/1955780671 learned_addr learned my addr 172.21.15.46:0/1955780671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe985400700 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe985e00700 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d24f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe986800700 1 -- 172.21.15.46:0/1955780671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 msgr2=0x55cf9d24f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe986800700 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d24f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe986800700 1 -- 172.21.15.46:0/1955780671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 msgr2=0x55cf9d2493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:46.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe986800700 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.960+0000 7fe986800700 1 -- 172.21.15.46:0/1955780671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf9d01a430 con 0x55cf9d249910 2024-05-08T23:21:46.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.961+0000 7fe986800700 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe98000a730 tx=0x7fe980007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a7e5fbf53a2377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.961+0000 7fe973400700 1 -- 172.21.15.46:0/1955780671 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980018070 con 0x55cf9d249910 2024-05-08T23:21:46.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.962+0000 7fe985400700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 0x7fe96c0a37d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe98810c2f0 tx=0x7fe97400b970 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.962+0000 7fe973400700 1 -- 172.21.15.46:0/1955780671 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe980010040 con 0x55cf9d249910 2024-05-08T23:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.962+0000 7fe973400700 1 -- 172.21.15.46:0/1955780671 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f050 con 0x55cf9d249910 2024-05-08T23:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.963+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/1955780671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 msgr2=0x55cf9d249d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.963+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe98000a730 tx=0x7fe980007990 comp rx=0 tx=0).stop 2024-05-08T23:21:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.964+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/1955780671 shutdown_connections 2024-05-08T23:21:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.964+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d249d50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.964+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d24f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.964+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/1955780671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.964+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/1955780671 >> 172.21.15.46:0/1955780671 conn(0x55cf9d1a7290 msgr2=0x55cf9d243e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.965+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/1955780671 shutdown_connections 2024-05-08T23:21:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.965+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/1955780671 wait complete. 2024-05-08T23:21:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.966+0000 7fe9a08d8b80 1 -- start start 2024-05-08T23:21:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.969+0000 7fe9a08d8b80 1 Processor -- start 2024-05-08T23:21:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d249910 0x55cf9d252ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24ee80 0x55cf9d253010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d1355e0 con 0x55cf9d24ee80 2024-05-08T23:21:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d135280 con 0x55cf9d24f7a0 2024-05-08T23:21:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.971+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d01c720 con 0x55cf9d249910 2024-05-08T23:21:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49444/0 (socket says 172.21.15.46:49444) 2024-05-08T23:21:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 -- 172.21.15.46:0/2322031250 learned_addr learned my addr 172.21.15.46:0/2322031250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 -- 172.21.15.46:0/2322031250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d249910 msgr2=0x55cf9d252ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d249910 0x55cf9d252ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 -- 172.21.15.46:0/2322031250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24ee80 msgr2=0x55cf9d253010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24ee80 0x55cf9d253010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 -- 172.21.15.46:0/2322031250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98000a410 con 0x55cf9d24f7a0 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.972+0000 7fe986800700 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe980007cf0 tx=0x7fe980003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0e8229a6cde21f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe9a08d8b80 10 client.0 _ll_register_callbacks cb 0x55cf9d13ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe967e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf9d0f22f0 con 0x55cf9d24f7a0 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe967e00700 1 -- 172.21.15.46:0/2322031250 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980018070 con 0x55cf9d24f7a0 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe967e00700 1 -- 172.21.15.46:0/2322031250 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe98001e450 con 0x55cf9d24f7a0 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.973+0000 7fe967e00700 1 -- 172.21.15.46:0/2322031250 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe980006000 con 0x55cf9d24f7a0 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.974+0000 7fe9a08d8b80 10 client.30263 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.974+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=36}) v3 -- 0x55cf9cfc6be0 con 0x55cf9d24f7a0 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe967e00700 1 -- 172.21.15.46:0/2322031250 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe98001e610 con 0x55cf9d24f7a0 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 10 client.30263 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 10 client.30263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 20 client.30263 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d273e90 0x55cf9d276310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 4 client.30263 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.975+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf9d218890 con 0x55cf9d273e90 2024-05-08T23:21:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.977+0000 7fe985e00700 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d273e90 0x55cf9d276310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.978+0000 7fe985e00700 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d273e90 0x55cf9d276310 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e9b097c2ee3bbf server_cookie=b8eb83e27893d1d8 in_seq=0 out_seq=0 2024-05-08T23:21:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.978+0000 7fe967e00700 10 client.30263 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:46.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.979+0000 7fe967e00700 1 -- 172.21.15.46:0/2322031250 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf9d218890 con 0x55cf9d273e90 2024-05-08T23:21:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.979+0000 7fe967e00700 10 client.30263 handle_command_reply: tid=0 2024-05-08T23:21:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.979+0000 7fe9a08d8b80 1 client.30263 shutdown 2024-05-08T23:21:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.982+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 msgr2=0x55cf9d253550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.982+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe980007cf0 tx=0x7fe980003ce0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 shutdown_connections 2024-05-08T23:21:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24f7a0 0x55cf9d253550 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d249910 0x55cf9d252ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d273e90 0x55cf9d276310 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/2322031250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24ee80 0x55cf9d253010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:46.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe967e00700 0 client.30263 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:46.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.983+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 >> 172.21.15.46:0/2322031250 conn(0x55cf9d1a7290 msgr2=0x55cf9d243620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.985+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 shutdown_connections 2024-05-08T23:21:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.987+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/2322031250 wait complete. 2024-05-08T23:21:46.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:46.988+0000 7fe9a08d8b80 20 client.30263 trim_cache size 0 max 16384 2024-05-08T23:21:47.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.028+0000 7fe9a08d8b80 1 Processor -- start 2024-05-08T23:21:47.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.029+0000 7fe9a08d8b80 1 -- start start 2024-05-08T23:21:47.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.030+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d249910 0x55cf9d2493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24ee80 0x55cf9d24e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d24f5e0 con 0x55cf9d249910 2024-05-08T23:21:47.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d27b560 con 0x55cf9d24ee80 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d23e880 con 0x55cf9d24f7a0 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe985e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d249910 0x55cf9d2493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:46028/0 (socket says 172.21.15.46:46028) 2024-05-08T23:21:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.031+0000 7fe986800700 1 -- 172.21.15.46:0/627011763 learned_addr learned my addr 172.21.15.46:0/627011763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe985400700 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24ee80 0x55cf9d24e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 -- 172.21.15.46:0/627011763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24ee80 msgr2=0x55cf9d24e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24ee80 0x55cf9d24e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 -- 172.21.15.46:0/627011763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d249910 msgr2=0x55cf9d2493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d249910 0x55cf9d2493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 -- 172.21.15.46:0/627011763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf9d249fc0 con 0x55cf9d24f7a0 2024-05-08T23:21:47.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.032+0000 7fe986800700 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe98001bf30 tx=0x7fe980015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfedc816c84c093 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.033+0000 7fe973e00700 1 -- 172.21.15.46:0/627011763 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f030 con 0x55cf9d24f7a0 2024-05-08T23:21:47.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.033+0000 7fe973e00700 1 -- 172.21.15.46:0/627011763 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98002b070 con 0x55cf9d24f7a0 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.033+0000 7fe973e00700 1 -- 172.21.15.46:0/627011763 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003e00 con 0x55cf9d24f7a0 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.034+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/627011763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 msgr2=0x55cf9d243da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.034+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe98001bf30 tx=0x7fe980015000 comp rx=0 tx=0).stop 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/627011763 shutdown_connections 2024-05-08T23:21:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24f7a0 0x55cf9d243da0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d249910 0x55cf9d2493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/627011763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d24ee80 0x55cf9d24e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/627011763 >> 172.21.15.46:0/627011763 conn(0x55cf9d1a7290 msgr2=0x55cf9d24dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.035+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/627011763 shutdown_connections 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.036+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/627011763 wait complete. 2024-05-08T23:21:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.037+0000 7fe9a08d8b80 1 -- start start 2024-05-08T23:21:47.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.039+0000 7fe9a08d8b80 1 Processor -- start 2024-05-08T23:21:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d26a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d26ac30 con 0x55cf9d24f7a0 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d26ada0 con 0x55cf9d249910 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9d2ca980 con 0x55cf9d24ee80 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.041+0000 7fe9a08d8b80 10 client.0 _ll_register_callbacks cb 0x55cf9d13ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49464/0 (socket says 172.21.15.46:49464) 2024-05-08T23:21:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 -- 172.21.15.46:0/3721394403 learned_addr learned my addr 172.21.15.46:0/3721394403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985400700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d26a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 -- 172.21.15.46:0/3721394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 msgr2=0x55cf9d26a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d26a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.042+0000 7fe985e00700 1 -- 172.21.15.46:0/3721394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 msgr2=0x55cf9d2ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.043+0000 7fe986800700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.043+0000 7fe985e00700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.043+0000 7fe985e00700 1 -- 172.21.15.46:0/3721394403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf9cfc6be0 con 0x55cf9d249910 2024-05-08T23:21:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.043+0000 7fe986800700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.045+0000 7fe985e00700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe9780996c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934dde836d62828 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.046+0000 7fe967e00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.046+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780a20d0 con 0x55cf9d249910 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.046+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe978099eb0 con 0x55cf9d249910 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.047+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe980010040 con 0x55cf9d249910 2024-05-08T23:21:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.048+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe97809a070 con 0x55cf9d249910 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.048+0000 7fe9a08d8b80 10 client.30269 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.048+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cf9d15f5e0 con 0x55cf9d249910 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.048+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe980010040 con 0x55cf9d249910 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.048+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe978095a40 con 0x55cf9d249910 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 10 client.30269 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 10 client.30269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 20 client.30269 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d2cc0e0 0x55cf9d2ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 4 client.30269 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:21:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cf9d191f40 con 0x55cf9d2cc0e0 2024-05-08T23:21:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.049+0000 7fe985400700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d2cc0e0 0x55cf9d2ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.050+0000 7fe985400700 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d2cc0e0 0x55cf9d2ce4d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7317711743a77598 server_cookie=de8631de7e52ba8f in_seq=0 out_seq=0 2024-05-08T23:21:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.050+0000 7fe967e00700 10 client.30269 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.052+0000 7fe967e00700 1 -- 172.21.15.46:0/3721394403 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cf9d191f40 con 0x55cf9d2cc0e0 2024-05-08T23:21:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.052+0000 7fe967e00700 10 client.30269 handle_command_reply: tid=0 2024-05-08T23:21:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.052+0000 7fe9a08d8b80 1 client.30269 shutdown 2024-05-08T23:21:47.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.053+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 msgr2=0x55cf9d269f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.053+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe9780996c0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 shutdown_connections 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55cf9d2cc0e0 0x55cf9d2ce4d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cf9d24f7a0 0x55cf9d2ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cf9d249910 0x55cf9d269f80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 --2- 172.21.15.46:0/3721394403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9d24ee80 0x55cf9d26a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 >> 172.21.15.46:0/3721394403 conn(0x55cf9d1a7290 msgr2=0x55cf9d2504c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe967e00700 0 client.30269 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.054+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 shutdown_connections 2024-05-08T23:21:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.055+0000 7fe9a08d8b80 1 -- 172.21.15.46:0/3721394403 wait complete. 2024-05-08T23:21:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.056+0000 7fe9a08d8b80 20 client.30269 trim_cache size 0 max 16384 2024-05-08T23:21:47.056 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:21:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "156cf27c-ef34-4692-bebb-1f39817ffa6a", 2024-05-08T23:21:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:21:47.057 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 msgr2=0x7fe96c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 0x7fe96c0a37d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe98810c2f0 tx=0x7fe97400b970 comp rx=0 tx=0).stop 2024-05-08T23:21:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 msgr2=0x7fe98810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe98810daf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c970 tx=0x7fe97800cc80 comp rx=0 tx=0).stop 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 shutdown_connections 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe98810b100 0x7fe98810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe96c0a1310 0x7fe96c0a37d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe988101050 0x7fe98810daf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 --2- 172.21.15.46:0/3806700590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe988106170 0x7fe98810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.058+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 >> 172.21.15.46:0/3806700590 conn(0x7fe9880fc8b0 msgr2=0x7fe9880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.060+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 shutdown_connections 2024-05-08T23:21:47.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.060+0000 7fe98ce00700 1 -- 172.21.15.46:0/3806700590 wait complete. 2024-05-08T23:21:47.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:47.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95965 19.73 MB/sec execute 128 sec latency 201.782 ms 2024-05-08T23:21:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 -- 172.21.15.46:0/1789259603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 msgr2=0x7f0b24101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 --2- 172.21.15.46:0/1789259603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24101460 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009990 tx=0x7f0b14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 -- 172.21.15.46:0/1789259603 shutdown_connections 2024-05-08T23:21:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 --2- 172.21.15.46:0/1789259603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b241061a0 0x7f0b2410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 --2- 172.21.15.46:0/1789259603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24101460 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 --2- 172.21.15.46:0/1789259603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b24101a30 0x7f0b24105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 -- 172.21.15.46:0/1789259603 >> 172.21.15.46:0/1789259603 conn(0x7f0b240fc8e0 msgr2=0x7f0b240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 -- 172.21.15.46:0/1789259603 shutdown_connections 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.365+0000 7f0b28a00700 1 -- 172.21.15.46:0/1789259603 wait complete. 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.366+0000 7f0b28a00700 1 Processor -- start 2024-05-08T23:21:47.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.366+0000 7f0b28a00700 1 -- start start 2024-05-08T23:21:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b241061a0 0x7f0b24199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 0x7f0b2419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2419e8f0 con 0x7f0b241061a0 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2419ea60 con 0x7f0b24101080 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b28a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2419ec20 con 0x7f0b2419de70 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b241061a0 0x7f0b24199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49472/0 (socket says 172.21.15.46:49472) 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 -- 172.21.15.46:0/3277349936 learned_addr learned my addr 172.21.15.46:0/3277349936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b22800700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 0x7f0b2419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 -- 172.21.15.46:0/3277349936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 msgr2=0x7f0b2419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 0x7f0b2419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 -- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b241061a0 msgr2=0x7f0b24199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b241061a0 0x7f0b24199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b21e00700 1 -- 172.21.15.46:0/3277349936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009520 con 0x7f0b24101080 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.367+0000 7f0b22800700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 0x7f0b2419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b21e00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005910 tx=0x7f0b14004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b1aa00700 1 -- 172.21.15.46:0/3277349936 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401d070 con 0x7f0b24101080 2024-05-08T23:21:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b140095f0 con 0x7f0b24101080 2024-05-08T23:21:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b1aa00700 1 -- 172.21.15.46:0/3277349936 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b14022470 con 0x7f0b24101080 2024-05-08T23:21:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b1aa00700 1 -- 172.21.15.46:0/3277349936 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14017650 con 0x7f0b24101080 2024-05-08T23:21:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.368+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b241a6460 con 0x7f0b24101080 2024-05-08T23:21:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.371+0000 7f0b1aa00700 1 -- 172.21.15.46:0/3277349936 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0b140225e0 con 0x7f0b24101080 2024-05-08T23:21:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.372+0000 7f0b3c49cb80 1 Processor -- start 2024-05-08T23:21:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 -- start start 2024-05-08T23:21:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b1aa00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 0x7f0b080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b1aa00700 1 -- 172.21.15.46:0/3277349936 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0b140ce430 con 0x7f0b24101080 2024-05-08T23:21:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22212890 0x561f22212cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f22169150 con 0x561f22217e00 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f22166b10 con 0x561f22218720 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f22162400 con 0x561f22212890 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b22800700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 0x7f0b080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b21400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b21400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49478/0 (socket says 172.21.15.46:49478) 2024-05-08T23:21:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b21400700 1 -- 172.21.15.46:0/1443668723 learned_addr learned my addr 172.21.15.46:0/1443668723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.373+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38836/0 (socket says 172.21.15.46:38836) 2024-05-08T23:21:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 -- 172.21.15.46:0/1443668723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22212890 msgr2=0x561f22212cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22212890 0x561f22212cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 -- 172.21.15.46:0/1443668723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 msgr2=0x561f222181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 -- 172.21.15.46:0/1443668723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f21fe33b0 con 0x561f22218720 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21e00700 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b21400700 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009e80 tx=0x7f0b10011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d0a209473a0a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b22800700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 0x7f0b080a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009410 tx=0x7f0b1c009920 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b0f000700 1 -- 172.21.15.46:0/1443668723 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10004250 con 0x561f22218720 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b0f000700 1 -- 172.21.15.46:0/1443668723 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b100043b0 con 0x561f22218720 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.374+0000 7f0b0f000700 1 -- 172.21.15.46:0/1443668723 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10019920 con 0x561f22218720 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1443668723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 msgr2=0x561f22212350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009e80 tx=0x7f0b10011720 comp rx=0 tx=0).stop 2024-05-08T23:21:47.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1443668723 shutdown_connections 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22212890 0x561f22212cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22217e00 0x561f222181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1443668723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22212350 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1443668723 >> 172.21.15.46:0/1443668723 conn(0x561f22170210 msgr2=0x561f2220cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1443668723 shutdown_connections 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.375+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1443668723 wait complete. 2024-05-08T23:21:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 -- start start 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 Processor -- start 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f2221c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 0x561f2221c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f220fe560 con 0x561f22218720 2024-05-08T23:21:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f220fe200 con 0x561f22212890 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.376+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f21fe56a0 con 0x561f22217e00 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b3c49cb80 10 client.0 _ll_register_callbacks cb 0x561f22107b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:49480/0 (socket says 172.21.15.46:49480) 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 -- 172.21.15.46:0/635805323 learned_addr learned my addr 172.21.15.46:0/635805323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b22800700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 0x561f2221c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21400700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f2221c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 -- 172.21.15.46:0/635805323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 msgr2=0x561f2221c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f2221c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 -- 172.21.15.46:0/635805323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 msgr2=0x561f2221c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 0x561f2221c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 -- 172.21.15.46:0/635805323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b241a6460 con 0x561f22212890 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b22800700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 0x561f2221c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b21e00700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0b14097bd0 tx=0x7f0b14009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3644664c301f7294 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b03400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1409b160 con 0x561f22212890 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.377+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0b1409ef90 con 0x561f22212890 2024-05-08T23:21:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f220bb270 con 0x561f22212890 2024-05-08T23:21:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b140a3550 con 0x561f22212890 2024-05-08T23:21:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 10 client.30281 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f0b10009ab0 con 0x561f22212890 2024-05-08T23:21:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f220bb270 con 0x561f22212890 2024-05-08T23:21:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b1409a650 con 0x561f22212890 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 10 client.30281 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 10 client.30281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 20 client.30281 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f2222bee0 0x561f2222e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 4 client.30281 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:47.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.378+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f221e1810 con 0x561f2222bee0 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.379+0000 7f0b21400700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f2222bee0 0x561f2222e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.379+0000 7f0b21400700 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f2222bee0 0x561f2222e300 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c68ba60cddbfdb server_cookie=eb099994da61674c in_seq=0 out_seq=0 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.379+0000 7f0b03400700 10 client.30281 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b03400700 1 -- 172.21.15.46:0/635805323 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f221e1810 con 0x561f2222bee0 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b03400700 10 client.30281 handle_command_reply: tid=0 2024-05-08T23:21:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 client.30281 shutdown 2024-05-08T23:21:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 msgr2=0x561f2221baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0b14097bd0 tx=0x7f0b14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 shutdown_connections 2024-05-08T23:21:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f2222bee0 0x561f2222e300 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22218720 0x561f2221c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22212890 0x561f2221baf0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b03400700 0 client.30281 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/635805323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f2221c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 >> 172.21.15.46:0/635805323 conn(0x561f22170210 msgr2=0x561f2220c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 shutdown_connections 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/635805323 wait complete. 2024-05-08T23:21:47.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.380+0000 7f0b3c49cb80 20 client.30281 trim_cache size 0 max 16384 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.398+0000 7f0b3c49cb80 1 Processor -- start 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.398+0000 7f0b3c49cb80 1 -- start start 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22217e00 0x561f2220cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22218720 0x561f222231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f220bb270 con 0x561f22212890 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f2220db30 con 0x561f22217e00 2024-05-08T23:21:47.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f2221e660 con 0x561f22218720 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b22800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22218720 0x561f222231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38866/0 (socket says 172.21.15.46:38866) 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 -- 172.21.15.46:0/2969280433 learned_addr learned my addr 172.21.15.46:0/2969280433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21400700 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22217e00 0x561f2220cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 -- 172.21.15.46:0/2969280433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22218720 msgr2=0x561f222231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22218720 0x561f222231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 -- 172.21.15.46:0/2969280433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22217e00 msgr2=0x561f2220cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22217e00 0x561f2220cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 -- 172.21.15.46:0/2969280433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f21f8fb60 con 0x561f22212890 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.399+0000 7f0b21e00700 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b1409b0f0 tx=0x7f0b1400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a0c9b955b5f7e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b0fa00700 1 -- 172.21.15.46:0/2969280433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14062080 con 0x561f22212890 2024-05-08T23:21:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b0fa00700 1 -- 172.21.15.46:0/2969280433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b1409aac0 con 0x561f22212890 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b0fa00700 1 -- 172.21.15.46:0/2969280433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b140b9360 con 0x561f22212890 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/2969280433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 msgr2=0x561f22212350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b1409b0f0 tx=0x7f0b1400b490 comp rx=0 tx=0).stop 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/2969280433 shutdown_connections 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22218720 0x561f222231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22212350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/2969280433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22217e00 0x561f2220cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/2969280433 >> 172.21.15.46:0/2969280433 conn(0x561f22170210 msgr2=0x561f2221c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/2969280433 shutdown_connections 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.400+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/2969280433 wait complete. 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.401+0000 7f0b3c49cb80 1 -- start start 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.401+0000 7f0b3c49cb80 1 Processor -- start 2024-05-08T23:21:47.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f22276870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22276db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f22281530 con 0x561f22212890 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f222816a0 con 0x561f22218720 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f22281860 con 0x561f22217e00 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b3c49cb80 10 client.0 _ll_register_callbacks cb 0x561f22107b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38878/0 (socket says 172.21.15.46:38878) 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 -- 172.21.15.46:0/1703207757 learned_addr learned my addr 172.21.15.46:0/1703207757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21400700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f22276870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b22800700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22276db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 -- 172.21.15.46:0/1703207757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 msgr2=0x561f22276870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f22276870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 -- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 msgr2=0x561f22276db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22276db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.402+0000 7f0b21e00700 1 -- 172.21.15.46:0/1703207757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b10009ab0 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b21e00700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b140a3020 tx=0x7f0b14022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc78e705f92e3bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14062030 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0b140a3550 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b100043b0 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b140b9360 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 10 client.20103 fetch_fsmap learned FSMap version 36 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561f22128560 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b100043b0 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b1400fac0 con 0x561f22212890 2024-05-08T23:21:47.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 10 client.20103 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 10 client.20103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 20 client.20103 populate_metadata read hostname 'smithi046' 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.403+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f22282b90 0x561f22284f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b3c49cb80 4 client.20103 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f2215aec0 con 0x561f22282b90 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b21400700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f22282b90 0x561f22284f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b21400700 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f22282b90 0x561f22284f80 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbdd3a9d0319c8a server_cookie=9d60fa439e1b90a2 in_seq=0 out_seq=0 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b03400700 10 client.20103 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.404+0000 7f0b03400700 1 -- 172.21.15.46:0/1703207757 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561f2215aec0 con 0x561f22282b90 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b03400700 10 client.20103 handle_command_reply: tid=0 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 client.20103 shutdown 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 msgr2=0x561f22276330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b140a3020 tx=0x7f0b14022c90 comp rx=0 tx=0).stop 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 shutdown_connections 2024-05-08T23:21:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f22218720 0x561f22276db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561f22282b90 0x561f22284f80 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f22212890 0x561f22276330 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 --2- 172.21.15.46:0/1703207757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f22217e00 0x561f22276870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b03400700 0 client.20103 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 >> 172.21.15.46:0/1703207757 conn(0x561f22170210 msgr2=0x561f22216bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 shutdown_connections 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 1 -- 172.21.15.46:0/1703207757 wait complete. 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.405+0000 7f0b3c49cb80 20 client.20103 trim_cache size 0 max 16384 2024-05-08T23:21:47.414 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "156cf27c-ef34-4692-bebb-1f39817ffa6a": { 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "156cf27c-ef34-4692-bebb-1f39817ffa6a", 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:21:47.415 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 msgr2=0x7f0b080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 0x7f0b080a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009410 tx=0x7f0b1c009920 comp rx=0 tx=0).stop 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 msgr2=0x7f0b24199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005910 tx=0x7f0b14004730 comp rx=0 tx=0).stop 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 shutdown_connections 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b2419de70 0x7f0b2419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b24101080 0x7f0b24199590 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0b080a1290 0x7f0b080a3750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 --2- 172.21.15.46:0/3277349936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b241061a0 0x7f0b24199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:47.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 >> 172.21.15.46:0/3277349936 conn(0x7f0b240fc8e0 msgr2=0x7f0b240fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:47.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 shutdown_connections 2024-05-08T23:21:47.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:21:47.406+0000 7f0b28a00700 1 -- 172.21.15.46:0/3277349936 wait complete. 2024-05-08T23:21:47.433 INFO:tasks.cephfs.filesystem:scrub status for tag:156cf27c-ef34-4692-bebb-1f39817ffa6a - {'path': '/', 'tag': '156cf27c-ef34-4692-bebb-1f39817ffa6a', 'options': 'recursive,force'} 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 451 op/s 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:47.715 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:47 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 451 op/s 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:47 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: pgmap v564: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 451 op/s 2024-05-08T23:21:47.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:47.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:47 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 96132 19.66 MB/sec execute 129 sec latency 362.811 ms 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:21:48.714 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:48 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:21:48.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:48 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:21:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:48 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:49.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 96446 19.63 MB/sec execute 130 sec latency 194.512 ms 2024-05-08T23:21:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:49 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:49 smithi053 ceph-mon[77587]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 374 op/s 2024-05-08T23:21:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:49 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:49 smithi046 ceph-mon[36898]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 374 op/s 2024-05-08T23:21:49.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:49 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:49.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:49 smithi086 ceph-mon[78610]: pgmap v565: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 374 op/s 2024-05-08T23:21:50.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 96817 19.63 MB/sec execute 131 sec latency 189.008 ms 2024-05-08T23:21:51.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 97231 19.58 MB/sec execute 132 sec latency 165.220 ms 2024-05-08T23:21:51.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:51 smithi053 ceph-mon[77587]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-05-08T23:21:51.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:51 smithi046 ceph-mon[36898]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-05-08T23:21:51.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:51 smithi086 ceph-mon[78610]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-05-08T23:21:52.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 97703 19.64 MB/sec execute 133 sec latency 170.621 ms 2024-05-08T23:21:53.269 INFO:tasks.workunit.client.0.smithi046.stdout: 10 98182 19.66 MB/sec execute 134 sec latency 201.289 ms 2024-05-08T23:21:53.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:53 smithi053 ceph-mon[77587]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 498 op/s 2024-05-08T23:21:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:53 smithi046 ceph-mon[36898]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 498 op/s 2024-05-08T23:21:53.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:53 smithi086 ceph-mon[78610]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 498 op/s 2024-05-08T23:21:54.268 INFO:tasks.workunit.client.0.smithi046.stdout: 10 98690 19.78 MB/sec execute 135 sec latency 142.160 ms 2024-05-08T23:21:55.269 INFO:tasks.workunit.client.0.smithi046.stdout: 10 99156 19.81 MB/sec execute 136 sec latency 197.824 ms 2024-05-08T23:21:55.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:55 smithi053 ceph-mon[77587]: pgmap v568: 129 pgs: 129 active+clean; 1021 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 533 op/s 2024-05-08T23:21:55.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:55 smithi046 ceph-mon[36898]: pgmap v568: 129 pgs: 129 active+clean; 1021 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 533 op/s 2024-05-08T23:21:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:55 smithi086 ceph-mon[78610]: pgmap v568: 129 pgs: 129 active+clean; 1021 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 533 op/s 2024-05-08T23:21:56.269 INFO:tasks.workunit.client.0.smithi046.stdout: 10 99543 19.85 MB/sec execute 137 sec latency 170.027 ms 2024-05-08T23:21:57.269 INFO:tasks.workunit.client.0.smithi046.stdout: 10 99982 19.82 MB/sec execute 138 sec latency 189.227 ms 2024-05-08T23:21:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:57 smithi053 ceph-mon[77587]: pgmap v569: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 553 op/s 2024-05-08T23:21:57.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:57 smithi046 ceph-mon[36898]: pgmap v569: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 553 op/s 2024-05-08T23:21:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:57 smithi086 ceph-mon[78610]: pgmap v569: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 553 op/s 2024-05-08T23:21:58.270 INFO:tasks.workunit.client.0.smithi046.stdout: 10 100322 19.83 MB/sec execute 139 sec latency 268.926 ms 2024-05-08T23:21:59.271 INFO:tasks.workunit.client.0.smithi046.stdout: 10 100794 19.88 MB/sec execute 140 sec latency 184.853 ms 2024-05-08T23:21:59.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:21:59 smithi053 ceph-mon[77587]: pgmap v570: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 393 op/s 2024-05-08T23:21:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:21:59 smithi046 ceph-mon[36898]: pgmap v570: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 393 op/s 2024-05-08T23:21:59.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:21:59 smithi086 ceph-mon[78610]: pgmap v570: 129 pgs: 129 active+clean; 1016 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 393 op/s 2024-05-08T23:22:00.271 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101168 19.85 MB/sec execute 141 sec latency 230.057 ms 2024-05-08T23:22:00.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:00.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:00 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:00.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:00.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:00 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:00.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:00.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:00 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:01.271 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101261 19.72 MB/sec execute 142 sec latency 726.417 ms 2024-05-08T23:22:01.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:01 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:01.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:01 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:01.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:01 smithi053 ceph-mon[77587]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-05-08T23:22:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:01 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:01 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:01.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:01 smithi046 ceph-mon[36898]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-05-08T23:22:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:01 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:01 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:01 smithi086 ceph-mon[78610]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-05-08T23:22:02.271 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101283 19.59 MB/sec execute 143 sec latency 1717.115 ms 2024-05-08T23:22:03.272 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101547 19.53 MB/sec execute 144 sec latency 1777.135 ms 2024-05-08T23:22:03.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:03 smithi086 ceph-mon[78610]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.5 MiB/s wr, 276 op/s 2024-05-08T23:22:04.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:03 smithi053 ceph-mon[77587]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.5 MiB/s wr, 276 op/s 2024-05-08T23:22:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:03 smithi046 ceph-mon[36898]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.5 MiB/s wr, 276 op/s 2024-05-08T23:22:04.272 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101574 19.42 MB/sec execute 145 sec latency 771.943 ms 2024-05-08T23:22:05.272 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101881 19.42 MB/sec execute 146 sec latency 941.611 ms 2024-05-08T23:22:05.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:05 smithi086 ceph-mon[78610]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 374 op/s 2024-05-08T23:22:06.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:05 smithi053 ceph-mon[77587]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 374 op/s 2024-05-08T23:22:06.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:05 smithi046 ceph-mon[36898]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 374 op/s 2024-05-08T23:22:06.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 102324 19.48 MB/sec execute 147 sec latency 130.710 ms 2024-05-08T23:22:06.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:06 smithi086 ceph-mon[78610]: osdmap e110: 12 total, 12 up, 12 in 2024-05-08T23:22:07.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:06 smithi053 ceph-mon[77587]: osdmap e110: 12 total, 12 up, 12 in 2024-05-08T23:22:07.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:06 smithi046 ceph-mon[36898]: osdmap e110: 12 total, 12 up, 12 in 2024-05-08T23:22:07.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 102705 19.47 MB/sec execute 148 sec latency 85.333 ms 2024-05-08T23:22:07.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:07 smithi086 ceph-mon[78610]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:08.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:07 smithi053 ceph-mon[77587]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:07 smithi046 ceph-mon[36898]: pgmap v575: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:08.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 103172 19.54 MB/sec execute 149 sec latency 170.970 ms 2024-05-08T23:22:09.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 103495 19.55 MB/sec execute 150 sec latency 231.821 ms 2024-05-08T23:22:09.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:09 smithi086 ceph-mon[78610]: pgmap v576: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:10.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:09 smithi053 ceph-mon[77587]: pgmap v576: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:09 smithi046 ceph-mon[36898]: pgmap v576: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 419 op/s 2024-05-08T23:22:10.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 103909 19.57 MB/sec execute 151 sec latency 171.361 ms 2024-05-08T23:22:11.273 INFO:tasks.workunit.client.0.smithi046.stdout: 10 104287 19.56 MB/sec execute 152 sec latency 135.346 ms 2024-05-08T23:22:12.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:11 smithi086 ceph-mon[78610]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:12.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:11 smithi053 ceph-mon[77587]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:12.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 104698 19.53 MB/sec execute 153 sec latency 106.907 ms 2024-05-08T23:22:12.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:11 smithi046 ceph-mon[36898]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:13.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 105163 19.58 MB/sec execute 154 sec latency 161.599 ms 2024-05-08T23:22:14.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:13 smithi086 ceph-mon[78610]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:14.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:13 smithi053 ceph-mon[77587]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:14.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 105651 19.66 MB/sec execute 155 sec latency 162.917 ms 2024-05-08T23:22:14.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:13 smithi046 ceph-mon[36898]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 468 op/s 2024-05-08T23:22:15.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 106144 19.67 MB/sec execute 156 sec latency 132.060 ms 2024-05-08T23:22:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:15 smithi086 ceph-mon[78610]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-05-08T23:22:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:15 smithi086 ceph-mon[78610]: osdmap e111: 12 total, 12 up, 12 in 2024-05-08T23:22:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:15 smithi053 ceph-mon[77587]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-05-08T23:22:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:15 smithi053 ceph-mon[77587]: osdmap e111: 12 total, 12 up, 12 in 2024-05-08T23:22:16.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 106503 19.65 MB/sec execute 157 sec latency 157.773 ms 2024-05-08T23:22:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:15 smithi046 ceph-mon[36898]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2024-05-08T23:22:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:15 smithi046 ceph-mon[36898]: osdmap e111: 12 total, 12 up, 12 in 2024-05-08T23:22:17.274 INFO:tasks.workunit.client.0.smithi046.stdout: 10 106965 19.69 MB/sec execute 158 sec latency 212.051 ms 2024-05-08T23:22:17.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:22:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.776+0000 7faffb400700 1 -- 172.21.15.46:0/2327568783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 msgr2=0x7faff4100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.776+0000 7faffb400700 1 --2- 172.21.15.46:0/2327568783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff4100320 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fafe400bb20 tx=0x7fafe400be30 comp rx=0 tx=0).stop 2024-05-08T23:22:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 -- 172.21.15.46:0/2327568783 shutdown_connections 2024-05-08T23:22:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 --2- 172.21.15.46:0/2327568783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 --2- 172.21.15.46:0/2327568783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff4100320 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 --2- 172.21.15.46:0/2327568783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff41008f0 0x7faff4104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 -- 172.21.15.46:0/2327568783 >> 172.21.15.46:0/2327568783 conn(0x7faff40fb760 msgr2=0x7faff40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 -- 172.21.15.46:0/2327568783 shutdown_connections 2024-05-08T23:22:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.778+0000 7faffb400700 1 -- 172.21.15.46:0/2327568783 wait complete. 2024-05-08T23:22:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.779+0000 7faffb400700 1 Processor -- start 2024-05-08T23:22:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 -- start start 2024-05-08T23:22:17.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff41a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 0x7faff419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419e1e0 con 0x7faff40fff40 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419e320 con 0x7faff4105060 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faffb400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419e4e0 con 0x7faff419d760 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faff8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff41a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7fafebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7fafebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32808/0 (socket says 172.21.15.46:32808) 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7faff9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 0x7faff419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.780+0000 7fafebe00700 1 -- 172.21.15.46:0/3013744842 learned_addr learned my addr 172.21.15.46:0/3013744842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 -- 172.21.15.46:0/3013744842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 msgr2=0x7faff419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 0x7faff419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 -- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 msgr2=0x7faff41a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff41a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 -- 172.21.15.46:0/3013744842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe400b800 con 0x7faff4105060 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7faff9400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 0x7faff419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.781+0000 7fafebe00700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e970 tx=0x7fafdc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.782+0000 7faff2000700 1 -- 172.21.15.46:0/3013744842 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc009790 con 0x7faff4105060 2024-05-08T23:22:17.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.782+0000 7faff2000700 1 -- 172.21.15.46:0/3013744842 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafdc00c7e0 con 0x7faff4105060 2024-05-08T23:22:17.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.782+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff41aa4a0 con 0x7faff4105060 2024-05-08T23:22:17.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.782+0000 7faff2000700 1 -- 172.21.15.46:0/3013744842 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0054e0 con 0x7faff4105060 2024-05-08T23:22:17.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.782+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41aa9b0 con 0x7faff4105060 2024-05-08T23:22:17.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.785+0000 7faff2000700 1 -- 172.21.15.46:0/3013744842 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fafdc00c950 con 0x7faff4105060 2024-05-08T23:22:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.787+0000 7faff2000700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 0x7fafd80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.787+0000 7faff2000700 1 -- 172.21.15.46:0/3013744842 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fafdc014070 con 0x7faff4105060 2024-05-08T23:22:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.787+0000 7faff8a00700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 0x7fafd80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.788+0000 7faff8a00700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 0x7fafd80a3750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe400d330 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.833+0000 7fb00eeeab80 1 Processor -- start 2024-05-08T23:22:17.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 -- start start 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85237720 0x556a85231350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85188150 con 0x556a85236e00 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85185b10 con 0x556a85231890 2024-05-08T23:22:17.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85181400 con 0x556a85237720 2024-05-08T23:22:17.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.834+0000 7faff8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59360/0 (socket says 172.21.15.46:59360) 2024-05-08T23:22:17.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 -- 172.21.15.46:0/2598150917 learned_addr learned my addr 172.21.15.46:0/2598150917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:17.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7fafebe00700 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85237720 0x556a85231350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 -- 172.21.15.46:0/2598150917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85237720 msgr2=0x556a85231350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85237720 0x556a85231350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 -- 172.21.15.46:0/2598150917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 msgr2=0x556a85231cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.835+0000 7faff8a00700 1 -- 172.21.15.46:0/2598150917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a850023b0 con 0x556a85236e00 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.836+0000 7faff8a00700 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fafe400dbb0 tx=0x7fafe401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21f17850609330e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.836+0000 7fafe9c00700 1 -- 172.21.15.46:0/2598150917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4010040 con 0x556a85236e00 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.836+0000 7fafe9c00700 1 -- 172.21.15.46:0/2598150917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe400dc90 con 0x556a85236e00 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.836+0000 7fafe9c00700 1 -- 172.21.15.46:0/2598150917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe402b6e0 con 0x556a85236e00 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.837+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2598150917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 msgr2=0x556a852371e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.837+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fafe400dbb0 tx=0x7fafe401f000 comp rx=0 tx=0).stop 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.838+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2598150917 shutdown_connections 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.838+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.838+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85236e00 0x556a852371e0 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fafe400dbb0 tx=0x7fafe401f000 comp rx=0 tx=0).stop 2024-05-08T23:22:17.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.838+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2598150917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85237720 0x556a85231350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.838+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2598150917 >> 172.21.15.46:0/2598150917 conn(0x556a8518f210 msgr2=0x556a8522bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.840+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2598150917 shutdown_connections 2024-05-08T23:22:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.840+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2598150917 wait complete. 2024-05-08T23:22:17.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.842+0000 7fb00eeeab80 1 -- start start 2024-05-08T23:22:17.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.842+0000 7fb00eeeab80 1 Processor -- start 2024-05-08T23:22:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 0x556a8523ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 0x556a8523b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a8523b790 0x556a852471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7faff8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 0x556a8523ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8511d560 con 0x556a8523b790 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8511d200 con 0x556a85237720 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a850046a0 con 0x556a85231890 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7faff8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 0x556a8523ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51106/0 (socket says 172.21.15.46:51106) 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7faff8a00700 1 -- 172.21.15.46:0/2566356381 learned_addr learned my addr 172.21.15.46:0/2566356381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.845+0000 7fb00eeeab80 10 client.0 _ll_register_callbacks cb 0x556a85126b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 0x556a8523b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 -- 172.21.15.46:0/2566356381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 msgr2=0x556a8523ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 0x556a8523ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 -- 172.21.15.46:0/2566356381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a8523b790 msgr2=0x556a852471c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a8523b790 0x556a852471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.846+0000 7fafebe00700 1 -- 172.21.15.46:0/2566356381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff410d450 con 0x556a85237720 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.847+0000 7fafebe00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 0x556a8523b250 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ca6f0 tx=0x7fafdc09a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da435cd790212bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.847+0000 7fafe2000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.847+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc09d070 con 0x556a85237720 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.847+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fafdc0cae40 con 0x556a85237720 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.847+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a850da270 con 0x556a85237720 2024-05-08T23:22:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.848+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafdc0a2530 con 0x556a85237720 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.848+0000 7fb00eeeab80 10 client.30296 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.848+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a84faeb60 con 0x556a85237720 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.848+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a850da270 con 0x556a85237720 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fafdc09a310 con 0x556a85237720 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 10 client.30296 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 10 client.30296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 20 client.30296 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a8524b790 0x556a8524dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 4 client.30296 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.849+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a85200810 con 0x556a8524b790 2024-05-08T23:22:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.850+0000 7faff8a00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a8524b790 0x556a8524dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.851+0000 7faff8a00700 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a8524b790 0x556a8524dbb0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67f8c32459fca9c server_cookie=bf6341771d33e48d in_seq=0 out_seq=0 2024-05-08T23:22:17.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.851+0000 7fafe2000700 10 client.30296 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:17.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.852+0000 7fafe2000700 1 -- 172.21.15.46:0/2566356381 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a85200810 con 0x556a8524b790 2024-05-08T23:22:17.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.852+0000 7fafe2000700 10 client.30296 handle_command_reply: tid=0 2024-05-08T23:22:17.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.852+0000 7fb00eeeab80 1 client.30296 shutdown 2024-05-08T23:22:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.853+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 msgr2=0x556a8523b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.853+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 0x556a8523b250 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ca6f0 tx=0x7fafdc09a160 comp rx=0 tx=0).stop 2024-05-08T23:22:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 shutdown_connections 2024-05-08T23:22:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a8524b790 0x556a8524dbb0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a8523b790 0x556a852471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a85231890 0x556a8523ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2566356381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85237720 0x556a8523b250 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 >> 172.21.15.46:0/2566356381 conn(0x556a8518f210 msgr2=0x556a8522b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fafe2000700 0 client.30296 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 shutdown_connections 2024-05-08T23:22:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2566356381 wait complete. 2024-05-08T23:22:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.854+0000 7fb00eeeab80 20 client.30296 trim_cache size 0 max 16384 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 Processor -- start 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 -- start start 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a85242110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8523da90 con 0x556a85237720 2024-05-08T23:22:17.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85228dc0 con 0x556a85231890 2024-05-08T23:22:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.892+0000 7fb00eeeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85252dc0 con 0x556a8523b790 2024-05-08T23:22:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.893+0000 7faff8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.893+0000 7fafebe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.893+0000 7fafebe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59372/0 (socket says 172.21.15.46:59372) 2024-05-08T23:22:17.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.893+0000 7fafebe00700 1 -- 172.21.15.46:0/1190371542 learned_addr learned my addr 172.21.15.46:0/1190371542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.894+0000 7fafebe00700 1 -- 172.21.15.46:0/1190371542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 msgr2=0x556a85242110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.894+0000 7fafebe00700 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a85242110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.894+0000 7fafebe00700 1 -- 172.21.15.46:0/1190371542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 msgr2=0x556a85231350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.894+0000 7fafebe00700 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.894+0000 7fafebe00700 1 -- 172.21.15.46:0/1190371542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a85147560 con 0x556a85237720 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.895+0000 7fafebe00700 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fafdc096b20 tx=0x7fafdc098660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0786d1b42e435c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.895+0000 7fafea600700 1 -- 172.21.15.46:0/1190371542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc09d070 con 0x556a85237720 2024-05-08T23:22:17.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.895+0000 7fafea600700 1 -- 172.21.15.46:0/1190371542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafdc016520 con 0x556a85237720 2024-05-08T23:22:17.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.895+0000 7fafea600700 1 -- 172.21.15.46:0/1190371542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0b84b0 con 0x556a85237720 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 -- 172.21.15.46:0/1190371542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 msgr2=0x556a8522bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fafdc096b20 tx=0x7fafdc098660 comp rx=0 tx=0).stop 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 -- 172.21.15.46:0/1190371542 shutdown_connections 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a85242110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85231350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/1190371542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a8522bd20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 -- 172.21.15.46:0/1190371542 >> 172.21.15.46:0/1190371542 conn(0x556a8518f210 msgr2=0x556a85246bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.897+0000 7fb00eeeab80 1 -- 172.21.15.46:0/1190371542 shutdown_connections 2024-05-08T23:22:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.898+0000 7fb00eeeab80 1 -- 172.21.15.46:0/1190371542 wait complete. 2024-05-08T23:22:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.898+0000 7fb00eeeab80 1 -- start start 2024-05-08T23:22:17.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.899+0000 7fb00eeeab80 1 Processor -- start 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.900+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85240400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.900+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a85240940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7fb00eeeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a8529acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7faff8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85240400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7faff8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85240400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32850/0 (socket says 172.21.15.46:32850) 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7faff8a00700 1 -- 172.21.15.46:0/2032022080 learned_addr learned my addr 172.21.15.46:0/2032022080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8529b210 con 0x556a85237720 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.901+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8529b380 con 0x556a85231890 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.902+0000 7fafebe00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a85240940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.902+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8529b540 con 0x556a8523b790 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.902+0000 7faff9400700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a8529acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.902+0000 7fb00eeeab80 10 client.0 _ll_register_callbacks cb 0x556a85126b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.903+0000 7fafebe00700 1 -- 172.21.15.46:0/2032022080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 msgr2=0x556a8529acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.903+0000 7fafebe00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a8529acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.903+0000 7fafebe00700 1 -- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 msgr2=0x556a85240400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.903+0000 7fafebe00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85240400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.903+0000 7fafebe00700 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a84faeb60 con 0x556a85237720 2024-05-08T23:22:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.904+0000 7fafebe00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a85240940 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x556a85239550 tx=0x7fafdc09e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f0fff618a83582 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.904+0000 7fafe2000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.904+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc09d050 con 0x556a85237720 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.904+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafe400dc90 con 0x556a85237720 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.905+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fafdc07a070 con 0x556a85237720 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.905+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafdc016330 con 0x556a85237720 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.905+0000 7fb00eeeab80 10 client.20118 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.905+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a85006cd0 con 0x556a85237720 2024-05-08T23:22:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.906+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe400dc90 con 0x556a85237720 2024-05-08T23:22:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fafdc099010 con 0x556a85237720 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 10 client.20118 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 10 client.20118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 20 client.20118 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a852a2810 0x556a852a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 4 client.20118 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.908+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556a85179ec0 con 0x556a852a2810 2024-05-08T23:22:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.909+0000 7faff8a00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a852a2810 0x556a852a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:17.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.914+0000 7faff8a00700 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a852a2810 0x556a852a4c00 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb51319db1deff01 server_cookie=e0d11a5b7cf229b1 in_seq=0 out_seq=0 2024-05-08T23:22:17.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.914+0000 7fafe2000700 10 client.20118 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.914+0000 7fafe2000700 1 -- 172.21.15.46:0/2032022080 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556a85179ec0 con 0x556a852a2810 2024-05-08T23:22:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.914+0000 7fafe2000700 10 client.20118 handle_command_reply: tid=0 2024-05-08T23:22:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.914+0000 7fb00eeeab80 1 client.20118 shutdown 2024-05-08T23:22:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 msgr2=0x556a85240940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a85240940 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x556a85239550 tx=0x7fafdc09e060 comp rx=0 tx=0).stop 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 shutdown_connections 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8523b790 0x556a8529acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a85231890 0x556a85240400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x556a852a2810 0x556a852a4c00 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 --2- 172.21.15.46:0/2032022080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a85237720 0x556a85240940 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x556a85239550 tx=0x7fafdc09e060 comp rx=0 tx=0).stop 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 >> 172.21.15.46:0/2032022080 conn(0x556a8518f210 msgr2=0x556a852395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fafe2000700 0 client.20118 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.915+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 shutdown_connections 2024-05-08T23:22:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.916+0000 7fb00eeeab80 1 -- 172.21.15.46:0/2032022080 wait complete. 2024-05-08T23:22:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.916+0000 7fb00eeeab80 20 client.20118 trim_cache size 0 max 16384 2024-05-08T23:22:17.917 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:22:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:22:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:22:17.917 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 msgr2=0x7fafd80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 0x7fafd80a3750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe400d330 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 msgr2=0x7faff41a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e970 tx=0x7fafdc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 shutdown_connections 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff419d760 0x7faff419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafd80a1290 0x7fafd80a3750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff40fff40 0x7faff41a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 --2- 172.21.15.46:0/3013744842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff4105060 0x7faff41a3b70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 >> 172.21.15.46:0/3013744842 conn(0x7faff40fb760 msgr2=0x7faff41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 shutdown_connections 2024-05-08T23:22:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:17.918+0000 7faffb400700 1 -- 172.21.15.46:0/3013744842 wait complete. 2024-05-08T23:22:17.952 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:22:17.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:18.219 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:17 smithi086 ceph-mon[78610]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:18.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:17 smithi053 ceph-mon[77587]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:18.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 107243 19.68 MB/sec execute 159 sec latency 249.068 ms 2024-05-08T23:22:18.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:17 smithi046 ceph-mon[36898]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 -- 172.21.15.46:0/1107210765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704101070 msgr2=0x7f5704101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 --2- 172.21.15.46:0/1107210765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704101070 0x7f5704101450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f56f40098f0 tx=0x7f56f4009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 -- 172.21.15.46:0/1107210765 shutdown_connections 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 --2- 172.21.15.46:0/1107210765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704106190 0x7f570410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 --2- 172.21.15.46:0/1107210765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704101070 0x7f5704101450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 --2- 172.21.15.46:0/1107210765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704101a20 0x7f5704105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.330+0000 7f5709400700 1 -- 172.21.15.46:0/1107210765 >> 172.21.15.46:0/1107210765 conn(0x7f57040fc930 msgr2=0x7f57040fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.331+0000 7f5709400700 1 -- 172.21.15.46:0/1107210765 shutdown_connections 2024-05-08T23:22:18.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.331+0000 7f5709400700 1 -- 172.21.15.46:0/1107210765 wait complete. 2024-05-08T23:22:18.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.332+0000 7f5709400700 1 Processor -- start 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.332+0000 7f5709400700 1 -- start start 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 0x7f5704108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704108760 0x7f57041a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570410be80 con 0x7f5704106190 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041992e0 con 0x7f5704108760 2024-05-08T23:22:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.333+0000 7f5709400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704199470 con 0x7f5704101070 2024-05-08T23:22:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.334+0000 7f5702800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.334+0000 7f5702800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51136/0 (socket says 172.21.15.46:51136) 2024-05-08T23:22:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.334+0000 7f5702800700 1 -- 172.21.15.46:0/2201570861 learned_addr learned my addr 172.21.15.46:0/2201570861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5701e00700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 0x7f5704108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 -- 172.21.15.46:0/2201570861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704108760 msgr2=0x7f57041a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704108760 0x7f57041a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 -- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 msgr2=0x7f5704108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 0x7f5704108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 -- 172.21.15.46:0/2201570861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0095f0 con 0x7f5704101070 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5701e00700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 0x7f5704108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.335+0000 7f5702800700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005950 tx=0x7f56f4010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.339+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f401c070 con 0x7f5704101070 2024-05-08T23:22:18.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.339+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56f400bba0 con 0x7f5704101070 2024-05-08T23:22:18.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.339+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4003a00 con 0x7f5704101070 2024-05-08T23:22:18.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.340+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f4009520 con 0x7f5704101070 2024-05-08T23:22:18.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.341+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5704199990 con 0x7f5704101070 2024-05-08T23:22:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.345+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5704102540 con 0x7f5704101070 2024-05-08T23:22:18.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.360+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f56f4020030 con 0x7f5704101070 2024-05-08T23:22:18.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.363+0000 7f56fb400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 0x7f56e80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:18.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.364+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f56f40ce3f0 con 0x7f5704101070 2024-05-08T23:22:18.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.364+0000 7f5701e00700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 0x7f56e80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:18.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.365+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56f40ce870 con 0x7f5704101070 2024-05-08T23:22:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.366+0000 7f5701e00700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 0x7f56e80a3850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57041093c0 tx=0x7f56ec009310 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:18.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.929+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5704109170 con 0x7f5704101070 2024-05-08T23:22:18.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.930+0000 7f56fb400700 1 -- 172.21.15.46:0/2201570861 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f56f4097460 con 0x7f5704101070 2024-05-08T23:22:18.933 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:22:18.934 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:22:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 msgr2=0x7f56e80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 0x7f56e80a3850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57041093c0 tx=0x7f56ec009310 comp rx=0 tx=0).stop 2024-05-08T23:22:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 msgr2=0x7f5704107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:18.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005950 tx=0x7f56f4010ff0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 shutdown_connections 2024-05-08T23:22:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5704108760 0x7f57041a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704101070 0x7f5704107ce0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f56e80a1390 0x7f56e80a3850 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 --2- 172.21.15.46:0/2201570861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5704106190 0x7f5704108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.937+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 >> 172.21.15.46:0/2201570861 conn(0x7f57040fc930 msgr2=0x7f57040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.938+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 shutdown_connections 2024-05-08T23:22:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:18.938+0000 7f5709400700 1 -- 172.21.15.46:0/2201570861 wait complete. 2024-05-08T23:22:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:22:18.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:22:19.276 INFO:tasks.workunit.client.0.smithi046.stdout: 10 107328 19.60 MB/sec execute 160 sec latency 1006.447 ms 2024-05-08T23:22:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 -- 172.21.15.46:0/417489230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281083c0 msgr2=0x7f002810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 --2- 172.21.15.46:0/417489230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281083c0 0x7f002810ff40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f00140098f0 tx=0x7f0014009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 -- 172.21.15.46:0/417489230 shutdown_connections 2024-05-08T23:22:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 --2- 172.21.15.46:0/417489230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281083c0 0x7f002810ff40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 --2- 172.21.15.46:0/417489230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281032a0 0x7f0028103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 --2- 172.21.15.46:0/417489230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0028103c50 0x7f0028107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.381+0000 7f002da00700 1 -- 172.21.15.46:0/417489230 >> 172.21.15.46:0/417489230 conn(0x7f00280feae0 msgr2=0x7f0028100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.382+0000 7f002da00700 1 -- 172.21.15.46:0/417489230 shutdown_connections 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.382+0000 7f002da00700 1 -- 172.21.15.46:0/417489230 wait complete. 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.383+0000 7f002da00700 1 Processor -- start 2024-05-08T23:22:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.383+0000 7f002da00700 1 -- start start 2024-05-08T23:22:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 0x7f002810ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 0x7f00281a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00281a9f60 con 0x7f00281083c0 2024-05-08T23:22:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00281aa0a0 con 0x7f00281032a0 2024-05-08T23:22:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f002da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00281aa260 con 0x7f00281a75a0 2024-05-08T23:22:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f0026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f0026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32894/0 (socket says 172.21.15.46:32894) 2024-05-08T23:22:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.384+0000 7f0026a00700 1 -- 172.21.15.46:0/3513059240 learned_addr learned my addr 172.21.15.46:0/3513059240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.385+0000 7f0026a00700 1 -- 172.21.15.46:0/3513059240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 msgr2=0x7f00281a9990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:22:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.385+0000 7f0026000700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 0x7f002810ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.385+0000 7f0027400700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 0x7f00281a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0026a00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 0x7f00281a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0026a00700 1 -- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 msgr2=0x7f002810ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0026a00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 0x7f002810ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0026a00700 1 -- 172.21.15.46:0/3513059240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014009520 con 0x7f00281032a0 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0026000700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 0x7f002810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.386+0000 7f0027400700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 0x7f00281a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.387+0000 7f0026a00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f002000c0c0 tx=0x7f002000c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.387+0000 7f001f400700 1 -- 172.21.15.46:0/3513059240 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020018070 con 0x7f00281032a0 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.387+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00281aa4e0 con 0x7f00281032a0 2024-05-08T23:22:19.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.388+0000 7f001f400700 1 -- 172.21.15.46:0/3513059240 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f002000f040 con 0x7f00281032a0 2024-05-08T23:22:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.388+0000 7f001f400700 1 -- 172.21.15.46:0/3513059240 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020018070 con 0x7f00281032a0 2024-05-08T23:22:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.388+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00281aaa40 con 0x7f00281032a0 2024-05-08T23:22:19.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.392+0000 7f001f400700 1 -- 172.21.15.46:0/3513059240 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0020007cc0 con 0x7f00281032a0 2024-05-08T23:22:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.396+0000 7f001f400700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 0x7f000c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.396+0000 7f001f400700 1 -- 172.21.15.46:0/3513059240 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f00200ccd10 con 0x7f00281032a0 2024-05-08T23:22:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.398+0000 7f0026000700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 0x7f000c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.398+0000 7f0041512b80 1 Processor -- start 2024-05-08T23:22:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.399+0000 7f0041512b80 1 -- start start 2024-05-08T23:22:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.399+0000 7f0041512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.399+0000 7f0026000700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 0x7f000c0a38b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f002810fe30 tx=0x7f0018009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.399+0000 7f0041512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c85350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0041512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0041512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6bdc150 con 0x5577d6c8b720 2024-05-08T23:22:19.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0041512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6bd9b10 con 0x5577d6c8ae00 2024-05-08T23:22:19.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0041512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6bd5400 con 0x5577d6c85890 2024-05-08T23:22:19.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51164/0 (socket says 172.21.15.46:51164) 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 -- 172.21.15.46:0/1188466039 learned_addr learned my addr 172.21.15.46:0/1188466039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0026a00700 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0026000700 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c85350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 -- 172.21.15.46:0/1188466039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 msgr2=0x5577d6c8b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 -- 172.21.15.46:0/1188466039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 msgr2=0x5577d6c85350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c85350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.400+0000 7f0027400700 1 -- 172.21.15.46:0/1188466039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d6a563b0 con 0x5577d6c85890 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.401+0000 7f0027400700 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0014009fd0 tx=0x7f001400be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f65cdeb93cd51cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.401+0000 7f0013e00700 1 -- 172.21.15.46:0/1188466039 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001401c070 con 0x5577d6c85890 2024-05-08T23:22:19.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.401+0000 7f0013e00700 1 -- 172.21.15.46:0/1188466039 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001400f460 con 0x5577d6c85890 2024-05-08T23:22:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.401+0000 7f0013e00700 1 -- 172.21.15.46:0/1188466039 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001400bb50 con 0x5577d6c85890 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 -- 172.21.15.46:0/1188466039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 msgr2=0x5577d6c85cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0014009fd0 tx=0x7f001400be90 comp rx=0 tx=0).stop 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 -- 172.21.15.46:0/1188466039 shutdown_connections 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85cd0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 --2- 172.21.15.46:0/1188466039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c85350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 -- 172.21.15.46:0/1188466039 >> 172.21.15.46:0/1188466039 conn(0x5577d6be3210 msgr2=0x5577d6c7fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.402+0000 7f0041512b80 1 -- 172.21.15.46:0/1188466039 shutdown_connections 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.403+0000 7f0041512b80 1 -- 172.21.15.46:0/1188466039 wait complete. 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.403+0000 7f0041512b80 1 -- start start 2024-05-08T23:22:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.403+0000 7f0041512b80 1 Processor -- start 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c85890 0x5577d6c8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6b71560 con 0x5577d6c85890 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6b71200 con 0x5577d6c8ae00 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6a586a0 con 0x5577d6c8b720 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0041512b80 10 client.0 _ll_register_callbacks cb 0x5577d6b7ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0027400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0027400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51180/0 (socket says 172.21.15.46:51180) 2024-05-08T23:22:19.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.404+0000 7f0027400700 1 -- 172.21.15.46:0/1681088851 learned_addr learned my addr 172.21.15.46:0/1681088851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0026000700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 -- 172.21.15.46:0/1681088851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 msgr2=0x5577d6c8f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 -- 172.21.15.46:0/1681088851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c85890 msgr2=0x5577d6c8eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c85890 0x5577d6c8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 -- 172.21.15.46:0/1681088851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00281107b0 con 0x5577d6c8b720 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0027400700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0014005e90 tx=0x7f001400fd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0c01cdbe656175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.405+0000 7f0010c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001401c040 con 0x5577d6c8b720 2024-05-08T23:22:19.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0014005040 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577d6b2e270 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001400bb80 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0041512b80 10 client.30361 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5577d6a02b60 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.406+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577d6b2e270 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0014027050 con 0x5577d6c8b720 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 10 client.30361 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 10 client.30361 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 20 client.30361 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6c9ed90 0x5577d6ca11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 4 client.30361 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577d6c54810 con 0x5577d6c9ed90 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.407+0000 7f0026a00700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6c9ed90 0x5577d6ca11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.408+0000 7f0026a00700 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6c9ed90 0x5577d6ca11b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6da8b358868bd9cf server_cookie=fc60cb7e7c206314 in_seq=0 out_seq=0 2024-05-08T23:22:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.408+0000 7f0010c00700 10 client.30361 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.409+0000 7f0010c00700 1 -- 172.21.15.46:0/1681088851 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577d6c54810 con 0x5577d6c9ed90 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.409+0000 7f0010c00700 10 client.30361 handle_command_reply: tid=0 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.409+0000 7f0041512b80 1 client.30361 shutdown 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 msgr2=0x5577d6c8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0014005e90 tx=0x7f001400fd70 comp rx=0 tx=0).stop 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 shutdown_connections 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6c9ed90 0x5577d6ca11b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c8b720 0x5577d6c8f570 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c85890 0x5577d6c8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 --2- 172.21.15.46:0/1681088851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c8f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 >> 172.21.15.46:0/1681088851 conn(0x5577d6be3210 msgr2=0x5577d6c7f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.410+0000 7f0010c00700 0 client.30361 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.411+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 shutdown_connections 2024-05-08T23:22:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.411+0000 7f0041512b80 1 -- 172.21.15.46:0/1681088851 wait complete. 2024-05-08T23:22:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.411+0000 7f0041512b80 20 client.30361 trim_cache size 0 max 16384 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.431+0000 7f0041512b80 1 Processor -- start 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.431+0000 7f0041512b80 1 -- start start 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6c78110 con 0x5577d6c8b720 2024-05-08T23:22:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6c9bcf0 con 0x5577d6c8ae00 2024-05-08T23:22:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0041512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6c8ff80 con 0x5577d6c85890 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0027400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51194/0 (socket says 172.21.15.46:51194) 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c7fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32918/0 (socket says 172.21.15.46:32918) 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 -- 172.21.15.46:0/1211724901 learned_addr learned my addr 172.21.15.46:0/1211724901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0027400700 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59430/0 (socket says 172.21.15.46:59430) 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 -- 172.21.15.46:0/1211724901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 msgr2=0x5577d6c7fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c7fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.432+0000 7f0026a00700 1 -- 172.21.15.46:0/1211724901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 msgr2=0x5577d6c96110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f0026a00700 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f0026a00700 1 -- 172.21.15.46:0/1211724901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d6b9b560 con 0x5577d6c85890 2024-05-08T23:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f0027400700 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f0026a00700 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0020096420 tx=0x7f002000efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ac63aa30d565a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f001cc00700 1 -- 172.21.15.46:0/1211724901 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000e020 con 0x5577d6c85890 2024-05-08T23:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f001cc00700 1 -- 172.21.15.46:0/1211724901 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00200112c0 con 0x5577d6c85890 2024-05-08T23:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.433+0000 7f001cc00700 1 -- 172.21.15.46:0/1211724901 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200b2270 con 0x5577d6c85890 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 -- 172.21.15.46:0/1211724901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 msgr2=0x5577d6c85350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0020096420 tx=0x7f002000efd0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 -- 172.21.15.46:0/1211724901 shutdown_connections 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6c96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6c85350 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 --2- 172.21.15.46:0/1211724901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6c7fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 -- 172.21.15.46:0/1211724901 >> 172.21.15.46:0/1211724901 conn(0x5577d6be3210 msgr2=0x5577d6c88c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 -- 172.21.15.46:0/1211724901 shutdown_connections 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.434+0000 7f0041512b80 1 -- 172.21.15.46:0/1211724901 wait complete. 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.435+0000 7f0041512b80 1 -- start start 2024-05-08T23:22:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.435+0000 7f0041512b80 1 Processor -- start 2024-05-08T23:22:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6ce9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6ce9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6ce9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6cf4420 con 0x5577d6c8b720 2024-05-08T23:22:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6cf4590 con 0x5577d6c8ae00 2024-05-08T23:22:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d6cf4750 con 0x5577d6c85890 2024-05-08T23:22:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0041512b80 10 client.0 _ll_register_callbacks cb 0x5577d6b7ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0026000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6ce9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0026a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6ce9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0026000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6ce9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32934/0 (socket says 172.21.15.46:32934) 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.437+0000 7f0026000700 1 -- 172.21.15.46:0/1451261985 learned_addr learned my addr 172.21.15.46:0/1451261985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0027400700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6ce9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0026a00700 1 -- 172.21.15.46:0/1451261985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 msgr2=0x5577d6ce9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0026a00700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6ce9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0026a00700 1 -- 172.21.15.46:0/1451261985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 msgr2=0x5577d6ce9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0026a00700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6ce9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0026a00700 1 -- 172.21.15.46:0/1451261985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577d6a02b60 con 0x5577d6c85890 2024-05-08T23:22:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.438+0000 7f0027400700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6ce9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0026a00700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6ce9220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f00200a1900 tx=0x7f00200096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2164ccf83c0a646 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0010c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002009a0b0 con 0x5577d6c85890 2024-05-08T23:22:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f001400f460 con 0x5577d6c85890 2024-05-08T23:22:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f00200a1e30 con 0x5577d6c85890 2024-05-08T23:22:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0020099c10 con 0x5577d6c85890 2024-05-08T23:22:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0041512b80 10 client.30367 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.439+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f00281aaa40 con 0x5577d6c85890 2024-05-08T23:22:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.440+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001400f460 con 0x5577d6c85890 2024-05-08T23:22:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.441+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00200a19a0 con 0x5577d6c85890 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 10 client.30367 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 10 client.30367 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 20 client.30367 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6cf5b20 0x5577d6cf7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 4 client.30367 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577d6bcdec0 con 0x5577d6cf5b20 2024-05-08T23:22:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.442+0000 7f0026000700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6cf5b20 0x5577d6cf7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.443+0000 7f0026000700 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6cf5b20 0x5577d6cf7f40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=641363e359b9b783 server_cookie=30d7f2b81fa10696 in_seq=0 out_seq=0 2024-05-08T23:22:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.444+0000 7f0010c00700 10 client.30367 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.444+0000 7f0010c00700 1 -- 172.21.15.46:0/1451261985 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577d6bcdec0 con 0x5577d6cf5b20 2024-05-08T23:22:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.444+0000 7f0010c00700 10 client.30367 handle_command_reply: tid=0 2024-05-08T23:22:19.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.444+0000 7f0041512b80 1 client.30367 shutdown 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 msgr2=0x5577d6ce9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6ce9220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f00200a1900 tx=0x7f00200096b0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 shutdown_connections 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x5577d6cf5b20 0x5577d6cf7f40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577d6c8b720 0x5577d6ce9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0010c00700 0 client.30367 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d6c85890 0x5577d6ce9220 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 --2- 172.21.15.46:0/1451261985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d6c8ae00 0x5577d6ce9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 >> 172.21.15.46:0/1451261985 conn(0x5577d6be3210 msgr2=0x5577d6c90320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.445+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 shutdown_connections 2024-05-08T23:22:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.446+0000 7f0041512b80 1 -- 172.21.15.46:0/1451261985 wait complete. 2024-05-08T23:22:19.446 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.446+0000 7f0041512b80 20 client.30367 trim_cache size 0 max 16384 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 msgr2=0x7f000c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 0x7f000c0a38b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f002810fe30 tx=0x7f0018009260 comp rx=0 tx=0).stop 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 msgr2=0x7f002810e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f002000c0c0 tx=0x7f002000c480 comp rx=0 tx=0).stop 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 shutdown_connections 2024-05-08T23:22:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00281a75a0 0x7f00281a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00281032a0 0x7f002810e8e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f000c0a13f0 0x7f000c0a38b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 --2- 172.21.15.46:0/3513059240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00281083c0 0x7f002810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.447+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 >> 172.21.15.46:0/3513059240 conn(0x7f00280feae0 msgr2=0x7f00280ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.448+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 shutdown_connections 2024-05-08T23:22:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.448+0000 7f002da00700 1 -- 172.21.15.46:0/3513059240 wait complete. 2024-05-08T23:22:19.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:22:19.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.824+0000 7f94c8c00700 1 -- 172.21.15.46:0/1655138501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4105190 msgr2=0x7f94c4105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.824+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1655138501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4105190 0x7f94c4105640 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009f90 tx=0x7f94bc00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 -- 172.21.15.46:0/1655138501 shutdown_connections 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1655138501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4105190 0x7f94c4105640 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1655138501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c40ffd90 0x7f94c4100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1655138501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 0x7f94c4104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 -- 172.21.15.46:0/1655138501 >> 172.21.15.46:0/1655138501 conn(0x7f94c40fb760 msgr2=0x7f94c40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 -- 172.21.15.46:0/1655138501 shutdown_connections 2024-05-08T23:22:19.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.825+0000 7f94c8c00700 1 -- 172.21.15.46:0/1655138501 wait complete. 2024-05-08T23:22:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.827+0000 7f94c8c00700 1 Processor -- start 2024-05-08T23:22:19.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.829+0000 7f94c8c00700 1 -- start start 2024-05-08T23:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 0x7f94c41a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 0x7f94c41a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419dec0 con 0x7f94c41006b0 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419e030 con 0x7f94c40ffd90 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c8c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419e1f0 con 0x7f94c4105190 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51224/0 (socket says 172.21.15.46:51224) 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c2800700 1 -- 172.21.15.46:0/1652394406 learned_addr learned my addr 172.21.15.46:0/1652394406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c1e00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 0x7f94c41a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.830+0000 7f94c1400700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 0x7f94c41a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c2800700 1 -- 172.21.15.46:0/1652394406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 msgr2=0x7f94c41a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c2800700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 0x7f94c41a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c2800700 1 -- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 msgr2=0x7f94c41a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c2800700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 0x7f94c41a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c2800700 1 -- 172.21.15.46:0/1652394406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc009c40 con 0x7f94c4105190 2024-05-08T23:22:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.831+0000 7f94c1e00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 0x7f94c41a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.832+0000 7f94c2800700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f94bc004c80 tx=0x7f94bc004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.832+0000 7f94baa00700 1 -- 172.21.15.46:0/1652394406 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00e040 con 0x7f94c4105190 2024-05-08T23:22:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.832+0000 7f94baa00700 1 -- 172.21.15.46:0/1652394406 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94bc008ed0 con 0x7f94c4105190 2024-05-08T23:22:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.832+0000 7f94baa00700 1 -- 172.21.15.46:0/1652394406 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0052a0 con 0x7f94c4105190 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.832+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c419e470 con 0x7f94c4105190 2024-05-08T23:22:19.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.833+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41aa6c0 con 0x7f94c4105190 2024-05-08T23:22:19.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.841+0000 7f94dc817b80 1 Processor -- start 2024-05-08T23:22:19.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.842+0000 7f94baa00700 1 -- 172.21.15.46:0/1652394406 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f94bc01d470 con 0x7f94c4105190 2024-05-08T23:22:19.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.842+0000 7f94dc817b80 1 -- start start 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a8f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a84e150 con 0x556a5a8f7890 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a84bb10 con 0x556a5a8fce00 2024-05-08T23:22:19.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.843+0000 7f94dc817b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a847400 con 0x556a5a8fd720 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.844+0000 7f94c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.844+0000 7f94c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59454/0 (socket says 172.21.15.46:59454) 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.844+0000 7f94c1400700 1 -- 172.21.15.46:0/2869402801 learned_addr learned my addr 172.21.15.46:0/2869402801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.844+0000 7f94c1400700 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.844+0000 7f94baa00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 0x7f94a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1400700 1 -- 172.21.15.46:0/2869402801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 msgr2=0x556a5a8f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1e00700 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a8f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1400700 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a8f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1400700 1 -- 172.21.15.46:0/2869402801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 msgr2=0x556a5a8fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1400700 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c1400700 1 -- 172.21.15.46:0/2869402801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5a6c83b0 con 0x556a5a8f7890 2024-05-08T23:22:19.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94c2800700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 0x7f94a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.845+0000 7f94baa00700 1 -- 172.21.15.46:0/1652394406 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f94bc0ce7a0 con 0x7f94c4105190 2024-05-08T23:22:19.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.846+0000 7f94c1400700 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f94b0003ef0 tx=0x7f94b0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2095ae17035abc5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.847+0000 7f94af000700 1 -- 172.21.15.46:0/2869402801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00140f0 con 0x556a5a8f7890 2024-05-08T23:22:19.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.847+0000 7f94af000700 1 -- 172.21.15.46:0/2869402801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b001a020 con 0x556a5a8f7890 2024-05-08T23:22:19.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.847+0000 7f94af000700 1 -- 172.21.15.46:0/2869402801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011400 con 0x556a5a8f7890 2024-05-08T23:22:19.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.847+0000 7f94c2800700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 0x7f94a80a37d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94bc097a30 tx=0x7f94bc086040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 -- 172.21.15.46:0/2869402801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 msgr2=0x556a5a8f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f94b0003ef0 tx=0x7f94b0004250 comp rx=0 tx=0).stop 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 -- 172.21.15.46:0/2869402801 shutdown_connections 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f7890 0x556a5a8f7cd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 --2- 172.21.15.46:0/2869402801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a8f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.848+0000 7f94dc817b80 1 -- 172.21.15.46:0/2869402801 >> 172.21.15.46:0/2869402801 conn(0x556a5a855210 msgr2=0x556a5a8f1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.849+0000 7f94dc817b80 1 -- 172.21.15.46:0/2869402801 shutdown_connections 2024-05-08T23:22:19.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.849+0000 7f94dc817b80 1 -- 172.21.15.46:0/2869402801 wait complete. 2024-05-08T23:22:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.850+0000 7f94dc817b80 1 -- start start 2024-05-08T23:22:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.850+0000 7f94dc817b80 1 Processor -- start 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a9010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8fd720 0x556a5a901600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a7e3560 con 0x556a5a8fd720 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a7e3200 con 0x556a5a8f7890 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a6ca6a0 con 0x556a5a8fce00 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94dc817b80 10 client.0 _ll_register_callbacks cb 0x556a5a7ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94c1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94c1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32956/0 (socket says 172.21.15.46:32956) 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.851+0000 7f94c1400700 1 -- 172.21.15.46:0/4106188961 learned_addr learned my addr 172.21.15.46:0/4106188961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1e00700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a9010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8fd720 0x556a5a901600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 -- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 msgr2=0x556a5a9010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a9010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 -- 172.21.15.46:0/4106188961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8fd720 msgr2=0x556a5a901600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8fd720 0x556a5a901600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 -- 172.21.15.46:0/4106188961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b0014410 con 0x556a5a8f7890 2024-05-08T23:22:19.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1e00700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a9010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:19.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94c1400700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004620 tx=0x7f94b0010ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e2ef885e2d8717 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.852+0000 7f94a3400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:19.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.856+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0010450 con 0x556a5a8f7890 2024-05-08T23:22:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.856+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f94b0000e40 con 0x556a5a8f7890 2024-05-08T23:22:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.856+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a5a7a0270 con 0x556a5a8f7890 2024-05-08T23:22:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.858+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b00105b0 con 0x556a5a8f7890 2024-05-08T23:22:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.858+0000 7f94dc817b80 10 client.30335 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a5a674b60 con 0x556a5a8f7890 2024-05-08T23:22:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a5a7a0270 con 0x556a5a8f7890 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94b00049f0 con 0x556a5a8f7890 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94dc817b80 10 client.30335 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94dc817b80 10 client.30335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.859+0000 7f94dc817b80 20 client.30335 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.860+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a911020 0x556a5a913440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.860+0000 7f94dc817b80 4 client.30335 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.860+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a5a8c6810 con 0x556a5a911020 2024-05-08T23:22:19.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.862+0000 7f94c1e00700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a911020 0x556a5a913440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.864+0000 7f94c1e00700 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a911020 0x556a5a913440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6df4ad4bfd9bb68 server_cookie=a197208e0b5ac5fd in_seq=0 out_seq=0 2024-05-08T23:22:19.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.864+0000 7f94a3400700 10 client.30335 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.865+0000 7f94a3400700 1 -- 172.21.15.46:0/4106188961 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a5a8c6810 con 0x556a5a911020 2024-05-08T23:22:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.865+0000 7f94a3400700 10 client.30335 handle_command_reply: tid=0 2024-05-08T23:22:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.866+0000 7f94dc817b80 1 client.30335 shutdown 2024-05-08T23:22:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.866+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 msgr2=0x556a5a900b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.866+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004620 tx=0x7f94b0010ac0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.866+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 shutdown_connections 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.866+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a911020 0x556a5a913440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.867+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8fd720 0x556a5a901600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.867+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8f7890 0x556a5a900b80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.867+0000 7f94dc817b80 1 --2- 172.21.15.46:0/4106188961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a9010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.867+0000 7f94a3400700 0 client.30335 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.867+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 >> 172.21.15.46:0/4106188961 conn(0x556a5a855210 msgr2=0x556a5a8f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.868+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 shutdown_connections 2024-05-08T23:22:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.868+0000 7f94dc817b80 1 -- 172.21.15.46:0/4106188961 wait complete. 2024-05-08T23:22:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.869+0000 7f94dc817b80 20 client.30335 trim_cache size 0 max 16384 2024-05-08T23:22:19.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.900+0000 7f94dc817b80 1 Processor -- start 2024-05-08T23:22:19.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.900+0000 7f94dc817b80 1 -- start start 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a901450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a7a0270 con 0x556a5a8f1d20 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a903e50 con 0x556a5a8fce00 2024-05-08T23:22:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.901+0000 7f94dc817b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a8fd430 con 0x556a5a8fd720 2024-05-08T23:22:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.902+0000 7f94c1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.902+0000 7f94c1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51246/0 (socket says 172.21.15.46:51246) 2024-05-08T23:22:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.902+0000 7f94c1e00700 1 -- 172.21.15.46:0/3871547646 learned_addr learned my addr 172.21.15.46:0/3871547646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.902+0000 7f94c1400700 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a901450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.902+0000 7f94c1e00700 1 -- 172.21.15.46:0/3871547646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 msgr2=0x556a5a8f7350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1e00700 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8f7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1e00700 1 -- 172.21.15.46:0/3871547646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 msgr2=0x556a5a901450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1e00700 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a901450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1e00700 1 -- 172.21.15.46:0/3871547646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5a80d560 con 0x556a5a8fd720 2024-05-08T23:22:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1400700 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a901450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.903+0000 7f94c1e00700 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f94b40124a0 tx=0x7f94b4014c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad42fa895bc0eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.904+0000 7f94afa00700 1 -- 172.21.15.46:0/3871547646 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4017040 con 0x556a5a8fd720 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.904+0000 7f94afa00700 1 -- 172.21.15.46:0/3871547646 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b4014390 con 0x556a5a8fd720 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.904+0000 7f94afa00700 1 -- 172.21.15.46:0/3871547646 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4012ea0 con 0x556a5a8fd720 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 -- 172.21.15.46:0/3871547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 msgr2=0x556a5a90cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f94b40124a0 tx=0x7f94b4014c20 comp rx=0 tx=0).stop 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 -- 172.21.15.46:0/3871547646 shutdown_connections 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a901450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fce00 0x556a5a8f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 --2- 172.21.15.46:0/3871547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fd720 0x556a5a90cc20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.905+0000 7f94dc817b80 1 -- 172.21.15.46:0/3871547646 >> 172.21.15.46:0/3871547646 conn(0x556a5a855210 msgr2=0x556a5a8f7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.906+0000 7f94dc817b80 1 -- 172.21.15.46:0/3871547646 shutdown_connections 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.906+0000 7f94dc817b80 1 -- 172.21.15.46:0/3871547646 wait complete. 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.906+0000 7f94dc817b80 1 -- start start 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.907+0000 7f94dc817b80 1 Processor -- start 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a906920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fd720 0x556a5a966520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a966a60 con 0x556a5a8f1d20 2024-05-08T23:22:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a966ba0 con 0x556a5a8fd720 2024-05-08T23:22:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5a966ce0 con 0x556a5a8fce00 2024-05-08T23:22:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.908+0000 7f94dc817b80 10 client.0 _ll_register_callbacks cb 0x556a5a7ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.909+0000 7f94c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.910+0000 7f94c1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59488/0 (socket says 172.21.15.46:59488) 2024-05-08T23:22:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.910+0000 7f94c1400700 1 -- 172.21.15.46:0/391779356 learned_addr learned my addr 172.21.15.46:0/391779356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.910+0000 7f94c1e00700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a906920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.910+0000 7f94c1400700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fd720 0x556a5a966520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1400700 1 -- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 msgr2=0x556a5a906920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1400700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a906920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1400700 1 -- 172.21.15.46:0/391779356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fd720 msgr2=0x556a5a966520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1400700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fd720 0x556a5a966520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1400700 1 -- 172.21.15.46:0/391779356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b4014670 con 0x556a5a8f1d20 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.911+0000 7f94c1e00700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a906920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.915+0000 7f94c1400700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f94b00045f0 tx=0x7f94b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc1fae6b3772b56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.915+0000 7f94a3400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:22:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.915+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003680 con 0x556a5a8f1d20 2024-05-08T23:22:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.915+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94b001a020 con 0x556a5a8f1d20 2024-05-08T23:22:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.916+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f94b001b030 con 0x556a5a8f1d20 2024-05-08T23:22:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.917+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b0003680 con 0x556a5a8f1d20 2024-05-08T23:22:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.917+0000 7f94dc817b80 10 client.20145 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.917+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a5a6cccd0 con 0x556a5a8f1d20 2024-05-08T23:22:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b001a020 con 0x556a5a8f1d20 2024-05-08T23:22:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94b0004780 con 0x556a5a8f1d20 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 10 client.20145 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 10 client.20145 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 20 client.20145 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a968150 0x556a5a96a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 4 client.20145 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.920+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a5a83fec0 con 0x556a5a968150 2024-05-08T23:22:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.921+0000 7f94c1e00700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a968150 0x556a5a96a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.928+0000 7f94c1e00700 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a968150 0x556a5a96a570 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=457b2b3f9cbd4cb9 server_cookie=573ec75e582c1b5 in_seq=0 out_seq=0 2024-05-08T23:22:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.929+0000 7f94a3400700 10 client.20145 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.929+0000 7f94a3400700 1 -- 172.21.15.46:0/391779356 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a5a83fec0 con 0x556a5a968150 2024-05-08T23:22:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.929+0000 7f94a3400700 10 client.20145 handle_command_reply: tid=0 2024-05-08T23:22:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.929+0000 7f94dc817b80 1 client.20145 shutdown 2024-05-08T23:22:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 msgr2=0x556a5a9063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f94b00045f0 tx=0x7f94b0000bc0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 shutdown_connections 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x556a5a968150 0x556a5a96a570 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556a5a8fd720 0x556a5a966520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a5a8f1d20 0x556a5a9063e0 secure :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f94b00045f0 tx=0x7f94b0000bc0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 --2- 172.21.15.46:0/391779356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a5a8fce00 0x556a5a906920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 >> 172.21.15.46:0/391779356 conn(0x556a5a855210 msgr2=0x556a5a8ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.930+0000 7f94a3400700 0 client.20145 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.931+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 shutdown_connections 2024-05-08T23:22:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.931+0000 7f94dc817b80 1 -- 172.21.15.46:0/391779356 wait complete. 2024-05-08T23:22:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.931+0000 7f94dc817b80 20 client.20145 trim_cache size 0 max 16384 2024-05-08T23:22:19.932 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.932+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 msgr2=0x7f94a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.932+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 0x7f94a80a37d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94bc097a30 tx=0x7f94bc086040 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.932+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 msgr2=0x7f94c419d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.932+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f94bc004c80 tx=0x7f94bc004cb0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 shutdown_connections 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c4105190 0x7f94c419d850 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c40ffd90 0x7f94c41a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f94a80a1310 0x7f94a80a37d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 --2- 172.21.15.46:0/1652394406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c41006b0 0x7f94c41a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 >> 172.21.15.46:0/1652394406 conn(0x7f94c40fb760 msgr2=0x7f94c4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 shutdown_connections 2024-05-08T23:22:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:19.933+0000 7f94c8c00700 1 -- 172.21.15.46:0/1652394406 wait complete. 2024-05-08T23:22:19.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:22:20.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:19 smithi086 ceph-mon[78610]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:20.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:19 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2201570861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.231+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3111090583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe0100590 msgr2=0x7f5fe010d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.231+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3111090583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe0100590 0x7f5fe010d250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009990 tx=0x7f5fcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3111090583 shutdown_connections 2024-05-08T23:22:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3111090583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fe010d790 0x7f5fe010fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3111090583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe00ffbe0 0x7f5fe00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3111090583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe0100590 0x7f5fe010d250 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3111090583 >> 172.21.15.46:0/3111090583 conn(0x7f5fe00fb800 msgr2=0x7f5fe00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3111090583 shutdown_connections 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.232+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3111090583 wait complete. 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.233+0000 7f5fe4a00700 1 Processor -- start 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 -- start start 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fe00ffbe0 0x7f5fe01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 0x7f5fe019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019e3c0 con 0x7f5fe0100590 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019e530 con 0x7f5fe010d790 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.234+0000 7f5fe4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe019e6f0 con 0x7f5fe00ffbe0 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.235+0000 7f5fdd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.235+0000 7f5fdd400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59502/0 (socket says 172.21.15.46:59502) 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.235+0000 7f5fdd400700 1 -- 172.21.15.46:0/3794977817 learned_addr learned my addr 172.21.15.46:0/3794977817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fde800700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 0x7f5fe019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fdd400700 1 -- 172.21.15.46:0/3794977817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fe00ffbe0 msgr2=0x7f5fe01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fdd400700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fe00ffbe0 0x7f5fe01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fdd400700 1 -- 172.21.15.46:0/3794977817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 msgr2=0x7f5fe019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fdd400700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 0x7f5fe019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fdd400700 1 -- 172.21.15.46:0/3794977817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0095f0 con 0x7f5fe0100590 2024-05-08T23:22:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.236+0000 7f5fde800700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 0x7f5fe019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.237+0000 7f5fdd400700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0095c0 tx=0x7f5fcc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.237+0000 7f5fd6a00700 1 -- 172.21.15.46:0/3794977817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01d070 con 0x7f5fe0100590 2024-05-08T23:22:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.237+0000 7f5fd6a00700 1 -- 172.21.15.46:0/3794977817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fcc0054f0 con 0x7f5fe0100590 2024-05-08T23:22:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.237+0000 7f5fd6a00700 1 -- 172.21.15.46:0/3794977817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc004bd0 con 0x7f5fe0100590 2024-05-08T23:22:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.237+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe019e970 con 0x7f5fe0100590 2024-05-08T23:22:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.238+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe019ebf0 con 0x7f5fe0100590 2024-05-08T23:22:20.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.246+0000 7f5ff85adb80 1 Processor -- start 2024-05-08T23:22:20.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.246+0000 7f5ff85adb80 1 -- start start 2024-05-08T23:22:20.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.246+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.246+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b3831150 con 0x5638b38da890 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b382eb10 con 0x5638b38dfe00 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b382a400 con 0x5638b38e0720 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51276/0 (socket says 172.21.15.46:51276) 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5fdde00700 1 -- 172.21.15.46:0/1633725928 learned_addr learned my addr 172.21.15.46:0/1633725928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.247+0000 7f5fdde00700 1 -- 172.21.15.46:0/1633725928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 msgr2=0x5638b38e01e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.248+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e01e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.248+0000 7f5fdde00700 1 -- 172.21.15.46:0/1633725928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 msgr2=0x5638b38dacd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.248+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38dacd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.248+0000 7f5fdde00700 1 -- 172.21.15.46:0/1633725928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b36ab3b0 con 0x5638b38e0720 2024-05-08T23:22:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.248+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ab20 tx=0x7f5fd000ae30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5f0fdf789aa5eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.249+0000 7f5fd6a00700 1 -- 172.21.15.46:0/3794977817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5fcc00fb80 con 0x7f5fe0100590 2024-05-08T23:22:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.250+0000 7f5fd6a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 0x7f5fc40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.251+0000 7f5fd6a00700 1 -- 172.21.15.46:0/3794977817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5fcc0cd7b0 con 0x7f5fe0100590 2024-05-08T23:22:20.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.251+0000 7f5fde800700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 0x7f5fc40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.252+0000 7f5fde800700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 0x7f5fc40a38a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5fe006a5a0 tx=0x7f5fd8009150 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.254+0000 7f5fcb000700 1 -- 172.21.15.46:0/1633725928 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000ccd0 con 0x5638b38e0720 2024-05-08T23:22:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.254+0000 7f5fcb000700 1 -- 172.21.15.46:0/1633725928 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd000ce30 con 0x5638b38e0720 2024-05-08T23:22:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.254+0000 7f5fcb000700 1 -- 172.21.15.46:0/1633725928 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00188a0 con 0x5638b38e0720 2024-05-08T23:22:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.255+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1633725928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 msgr2=0x5638b38da350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.255+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ab20 tx=0x7f5fd000ae30 comp rx=0 tx=0).stop 2024-05-08T23:22:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.256+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1633725928 shutdown_connections 2024-05-08T23:22:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.256+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.256+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.256+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1633725928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38da350 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.256+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1633725928 >> 172.21.15.46:0/1633725928 conn(0x5638b3838210 msgr2=0x5638b38d4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.257+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1633725928 shutdown_connections 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.257+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1633725928 wait complete. 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.258+0000 7f5ff85adb80 1 -- start start 2024-05-08T23:22:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.258+0000 7f5ff85adb80 1 Processor -- start 2024-05-08T23:22:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 0x5638b38e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 0x5638b38e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b37c6560 con 0x5638b38da890 2024-05-08T23:22:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b37c6200 con 0x5638b38e0720 2024-05-08T23:22:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b36ad6a0 con 0x5638b38dfe00 2024-05-08T23:22:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.260+0000 7f5ff85adb80 10 client.0 _ll_register_callbacks cb 0x5638b37cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:20.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:19 smithi053 ceph-mon[77587]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:20.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:19 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2201570861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.261+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.261+0000 7f5fdd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 0x5638b38e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.261+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38e3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59514/0 (socket says 172.21.15.46:59514) 2024-05-08T23:22:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.261+0000 7f5fdde00700 1 -- 172.21.15.46:0/3636997664 learned_addr learned my addr 172.21.15.46:0/3636997664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.261+0000 7f5fde800700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 0x5638b38e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fde800700 1 -- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 msgr2=0x5638b38e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fde800700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 0x5638b38e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fde800700 1 -- 172.21.15.46:0/3636997664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 msgr2=0x5638b38e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fde800700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fde800700 1 -- 172.21.15.46:0/3636997664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd000a750 con 0x5638b38e0720 2024-05-08T23:22:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.263+0000 7f5fdd400700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 0x5638b38e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.264+0000 7f5fde800700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 0x5638b38e4790 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007ce0 tx=0x7f5fd8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377c4e72a6144275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.264+0000 7f5fbf400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.265+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8016d60 con 0x5638b38e0720 2024-05-08T23:22:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.265+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638b3783270 con 0x5638b38e0720 2024-05-08T23:22:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.265+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5fd8021440 con 0x5638b38e0720 2024-05-08T23:22:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.266+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd8016d60 con 0x5638b38e0720 2024-05-08T23:22:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.266+0000 7f5ff85adb80 10 client.30347 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.266+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5638b3657b60 con 0x5638b38e0720 2024-05-08T23:22:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.266+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638b3783270 con 0x5638b38e0720 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.267+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5fd8020680 con 0x5638b38e0720 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.267+0000 7f5ff85adb80 10 client.30347 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.267+0000 7f5ff85adb80 10 client.30347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.267+0000 7f5ff85adb80 20 client.30347 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.267+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.268+0000 7f5ff85adb80 4 client.30347 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.268+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638b38a9810 con 0x5638b38fcb10 2024-05-08T23:22:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.272+0000 7f5fdde00700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.273+0000 7f5fdde00700 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38fef30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14c3c028abea9c28 server_cookie=231b28240b64adb7 in_seq=0 out_seq=0 2024-05-08T23:22:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.273+0000 7f5fbf400700 10 client.30347 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:20.276 INFO:tasks.workunit.client.0.smithi046.stdout: 10 107602 19.57 MB/sec execute 161 sec latency 1139.344 ms 2024-05-08T23:22:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:19 smithi046 ceph-mon[36898]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-08T23:22:20.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:19 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2201570861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.281+0000 7f5fbf400700 1 -- 172.21.15.46:0/3636997664 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638b38a9810 con 0x5638b38fcb10 2024-05-08T23:22:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.281+0000 7f5fbf400700 10 client.30347 handle_command_reply: tid=0 2024-05-08T23:22:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.281+0000 7f5ff85adb80 1 client.30347 shutdown 2024-05-08T23:22:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 msgr2=0x5638b38e4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 0x5638b38e4790 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007ce0 tx=0x7f5fd8004b40 comp rx=0 tx=0).stop 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 shutdown_connections 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38e0720 0x5638b38e4790 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38fef30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.282+0000 7f5fbf400700 0 client.30347 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.283+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/3636997664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38dfe00 0x5638b38e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.283+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 >> 172.21.15.46:0/3636997664 conn(0x5638b3838210 msgr2=0x5638b38d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.283+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 shutdown_connections 2024-05-08T23:22:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.283+0000 7f5ff85adb80 1 -- 172.21.15.46:0/3636997664 wait complete. 2024-05-08T23:22:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.283+0000 7f5ff85adb80 20 client.30347 trim_cache size 0 max 16384 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.313+0000 7f5ff85adb80 1 Processor -- start 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.314+0000 7f5ff85adb80 1 -- start start 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38e0720 0x5638b38efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b38d1dc0 con 0x5638b38e0720 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b3904110 con 0x5638b38dfe00 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b38d67e0 con 0x5638b38da890 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5fdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51300/0 (socket says 172.21.15.46:51300) 2024-05-08T23:22:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.315+0000 7f5fdde00700 1 -- 172.21.15.46:0/945034425 learned_addr learned my addr 172.21.15.46:0/945034425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 -- 172.21.15.46:0/945034425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 msgr2=0x5638b38da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 -- 172.21.15.46:0/945034425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38e0720 msgr2=0x5638b38efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38e0720 0x5638b38efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 -- 172.21.15.46:0/945034425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b37f0560 con 0x5638b38dfe00 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdde00700 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.316+0000 7f5fdd400700 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38f8ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0caaf0 tx=0x7f5fcc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fbb492d02cdb94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.317+0000 7f5fcba00700 1 -- 172.21.15.46:0/945034425 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc099ea0 con 0x5638b38dfe00 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.317+0000 7f5fcba00700 1 -- 172.21.15.46:0/945034425 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fcc005660 con 0x5638b38dfe00 2024-05-08T23:22:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.317+0000 7f5fcba00700 1 -- 172.21.15.46:0/945034425 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc099a30 con 0x5638b38dfe00 2024-05-08T23:22:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 -- 172.21.15.46:0/945034425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 msgr2=0x5638b38f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38f8ba0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0caaf0 tx=0x7f5fcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 -- 172.21.15.46:0/945034425 shutdown_connections 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38e0720 0x5638b38efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38da890 0x5638b38da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/945034425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38f8ba0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.318+0000 7f5ff85adb80 1 -- 172.21.15.46:0/945034425 >> 172.21.15.46:0/945034425 conn(0x5638b3838210 msgr2=0x5638b38e25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.319+0000 7f5ff85adb80 1 -- 172.21.15.46:0/945034425 shutdown_connections 2024-05-08T23:22:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.319+0000 7f5ff85adb80 1 -- 172.21.15.46:0/945034425 wait complete. 2024-05-08T23:22:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.319+0000 7f5ff85adb80 1 -- start start 2024-05-08T23:22:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.320+0000 7f5ff85adb80 1 Processor -- start 2024-05-08T23:22:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b38e9bd0 con 0x5638b38da890 2024-05-08T23:22:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b38e9d10 con 0x5638b38dfe00 2024-05-08T23:22:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b38e9ed0 con 0x5638b38e0720 2024-05-08T23:22:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.321+0000 7f5ff85adb80 10 client.0 _ll_register_callbacks cb 0x5638b37cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:20.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fde800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fdd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fde800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51308/0 (socket says 172.21.15.46:51308) 2024-05-08T23:22:20.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fde800700 1 -- 172.21.15.46:0/1438909409 learned_addr learned my addr 172.21.15.46:0/1438909409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.322+0000 7f5fdd400700 1 -- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 msgr2=0x5638b38e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fdd400700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fdd400700 1 -- 172.21.15.46:0/1438909409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 msgr2=0x5638b38eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fdd400700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fdd400700 1 -- 172.21.15.46:0/1438909409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638b36ab3b0 con 0x5638b38dfe00 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fde800700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fbf400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.323+0000 7f5fdd400700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e9150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc098890 tx=0x7f5fcc009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb942a218a6dd14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0a26a0 con 0x5638b38dfe00 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5fcc09ac40 con 0x5638b38dfe00 2024-05-08T23:22:20.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fd000ce30 con 0x5638b38dfe00 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fcc09a540 con 0x5638b38dfe00 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5ff85adb80 10 client.30353 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.324+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f5fe019e970 con 0x5638b38dfe00 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd000ce30 con 0x5638b38dfe00 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5fcc0cd070 con 0x5638b38dfe00 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 10 client.30353 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 10 client.30353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:22:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 20 client.30353 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 4 client.30353 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638b3822ec0 con 0x5638b38fcb10 2024-05-08T23:22:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.325+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.328+0000 7f5fbf400700 10 client.30353 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.328+0000 7f5fdde00700 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38f33a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32cf0a4ca21ed707 server_cookie=6df7a1b101540935 in_seq=0 out_seq=0 2024-05-08T23:22:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5fbf400700 1 -- 172.21.15.46:0/1438909409 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638b3822ec0 con 0x5638b38fcb10 2024-05-08T23:22:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5fbf400700 10 client.30353 handle_command_reply: tid=0 2024-05-08T23:22:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 client.30353 shutdown 2024-05-08T23:22:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 msgr2=0x5638b38e9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e9150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc098890 tx=0x7f5fcc009ee0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 shutdown_connections 2024-05-08T23:22:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b38e0720 0x5638b38e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5638b38fcb10 0x5638b38f33a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b38da890 0x5638b38eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 --2- 172.21.15.46:0/1438909409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638b38dfe00 0x5638b38e9150 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 >> 172.21.15.46:0/1438909409 conn(0x5638b3838210 msgr2=0x5638b38dedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5fbf400700 0 client.30353 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.331+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 shutdown_connections 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.332+0000 7f5ff85adb80 1 -- 172.21.15.46:0/1438909409 wait complete. 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.332+0000 7f5ff85adb80 20 client.30353 trim_cache size 0 max 16384 2024-05-08T23:22:20.332 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 msgr2=0x7f5fc40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 0x7f5fc40a38a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5fe006a5a0 tx=0x7f5fd8009150 comp rx=0 tx=0).stop 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 msgr2=0x7f5fe01999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0095c0 tx=0x7f5fcc00f690 comp rx=0 tx=0).stop 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 shutdown_connections 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe010d790 0x7f5fe019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fe00ffbe0 0x7f5fe01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5fc40a13e0 0x7f5fc40a38a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 --2- 172.21.15.46:0/3794977817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe0100590 0x7f5fe01999e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.333+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 >> 172.21.15.46:0/3794977817 conn(0x7f5fe00fb800 msgr2=0x7f5fe010eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.335+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 shutdown_connections 2024-05-08T23:22:20.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.335+0000 7f5fe4a00700 1 -- 172.21.15.46:0/3794977817 wait complete. 2024-05-08T23:22:20.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:22:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 -- 172.21.15.46:0/324692473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 msgr2=0x7f8c741089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 --2- 172.21.15.46:0/324692473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741089b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009990 tx=0x7f8c64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 -- 172.21.15.46:0/324692473 shutdown_connections 2024-05-08T23:22:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 --2- 172.21.15.46:0/324692473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c7410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 --2- 172.21.15.46:0/324692473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741089b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 --2- 172.21.15.46:0/324692473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c7410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 -- 172.21.15.46:0/324692473 >> 172.21.15.46:0/324692473 conn(0x7f8c7406d060 msgr2=0x7f8c74072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 -- 172.21.15.46:0/324692473 shutdown_connections 2024-05-08T23:22:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.726+0000 7f8c78a00700 1 -- 172.21.15.46:0/324692473 wait complete. 2024-05-08T23:22:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.728+0000 7f8c78a00700 1 Processor -- start 2024-05-08T23:22:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.730+0000 7f8c78a00700 1 -- start start 2024-05-08T23:22:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c741a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741ab000 con 0x7f8c74108ef0 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741ab170 con 0x7f8c741085d0 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.731+0000 7f8c78a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741ab330 con 0x7f8c7410d9d0 2024-05-08T23:22:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.732+0000 7f8c71e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.732+0000 7f8c72800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.732+0000 7f8c71400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c741a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.732+0000 7f8c72800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51340/0 (socket says 172.21.15.46:51340) 2024-05-08T23:22:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.732+0000 7f8c72800700 1 -- 172.21.15.46:0/352738112 learned_addr learned my addr 172.21.15.46:0/352738112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 -- 172.21.15.46:0/352738112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 msgr2=0x7f8c741a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 -- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 msgr2=0x7f8c741a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c741a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 -- 172.21.15.46:0/352738112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640095f0 con 0x7f8c7410d9d0 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c71400700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c741a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c72800700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00ef30 tx=0x7f8c6c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c71e00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c6aa00700 1 -- 172.21.15.46:0/352738112 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c018070 con 0x7f8c7410d9d0 2024-05-08T23:22:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c741ab5b0 con 0x7f8c7410d9d0 2024-05-08T23:22:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.733+0000 7f8c6aa00700 1 -- 172.21.15.46:0/352738112 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c00f040 con 0x7f8c7410d9d0 2024-05-08T23:22:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.734+0000 7f8c6aa00700 1 -- 172.21.15.46:0/352738112 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004280 con 0x7f8c7410d9d0 2024-05-08T23:22:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.735+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c741b2e50 con 0x7f8c7410d9d0 2024-05-08T23:22:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.739+0000 7f8c6aa00700 1 -- 172.21.15.46:0/352738112 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8c6c003a00 con 0x7f8c7410d9d0 2024-05-08T23:22:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.741+0000 7f8c6aa00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 0x7f8c580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.741+0000 7f8c6aa00700 1 -- 172.21.15.46:0/352738112 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8c6c0cc680 con 0x7f8c7410d9d0 2024-05-08T23:22:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.743+0000 7f8c71e00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 0x7f8c580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.745+0000 7f8c71e00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 0x7f8c580a37d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c64006010 tx=0x7f8c6400b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.745+0000 7f8c8c52ab80 1 Processor -- start 2024-05-08T23:22:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 -- start start 2024-05-08T23:22:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b0e00 0x55718a5b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a502150 con 0x55718a5b0e00 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a4ffb10 con 0x55718a5ab890 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.746+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a4fb400 con 0x55718a5b1720 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33052/0 (socket says 172.21.15.46:33052) 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 -- 172.21.15.46:0/2143318830 learned_addr learned my addr 172.21.15.46:0/2143318830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c71e00700 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b0e00 0x55718a5b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 -- 172.21.15.46:0/2143318830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 msgr2=0x55718a5ab350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5ab350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 -- 172.21.15.46:0/2143318830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b0e00 msgr2=0x55718a5b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b0e00 0x55718a5b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 -- 172.21.15.46:0/2143318830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718a37c3b0 con 0x55718a5ab890 2024-05-08T23:22:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.747+0000 7f8c72800700 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095df0 tx=0x7f8c6c099150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd882d531891f64e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.748+0000 7f8c5f000700 1 -- 172.21.15.46:0/2143318830 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c099a80 con 0x55718a5ab890 2024-05-08T23:22:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.748+0000 7f8c5f000700 1 -- 172.21.15.46:0/2143318830 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c099380 con 0x55718a5ab890 2024-05-08T23:22:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.748+0000 7f8c5f000700 1 -- 172.21.15.46:0/2143318830 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0977d0 con 0x55718a5ab890 2024-05-08T23:22:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.751+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/2143318830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 msgr2=0x55718a5abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.751+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095df0 tx=0x7f8c6c099150 comp rx=0 tx=0).stop 2024-05-08T23:22:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/2143318830 shutdown_connections 2024-05-08T23:22:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5abcd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b0e00 0x55718a5b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/2143318830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/2143318830 >> 172.21.15.46:0/2143318830 conn(0x55718a509210 msgr2=0x55718a5a5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/2143318830 shutdown_connections 2024-05-08T23:22:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.752+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/2143318830 wait complete. 2024-05-08T23:22:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.753+0000 7f8c8c52ab80 1 -- start start 2024-05-08T23:22:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.754+0000 7f8c8c52ab80 1 Processor -- start 2024-05-08T23:22:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5ab890 0x55718a5b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5b0e00 0x55718a5b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a497560 con 0x55718a5ab890 2024-05-08T23:22:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a497200 con 0x55718a5b0e00 2024-05-08T23:22:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a37e6a0 con 0x55718a5b1720 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c8c52ab80 10 client.0 _ll_register_callbacks cb 0x55718a4a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c72800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c72800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51352/0 (socket says 172.21.15.46:51352) 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.756+0000 7f8c72800700 1 -- 172.21.15.46:0/3955698558 learned_addr learned my addr 172.21.15.46:0/3955698558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c71400700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5b0e00 0x55718a5b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 -- 172.21.15.46:0/3955698558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5b0e00 msgr2=0x55718a5b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5b0e00 0x55718a5b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 -- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5ab890 msgr2=0x55718a5b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5ab890 0x55718a5b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 -- 172.21.15.46:0/3955698558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c741b2e50 con 0x55718a5b1720 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c72800700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095830 tx=0x7f8c6c097930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6112853c89c20c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c53400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0a1800 con 0x55718a5b1720 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.757+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8c6c0974c0 con 0x55718a5b1720 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.758+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55718a454270 con 0x55718a5b1720 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.758+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c0997f0 con 0x55718a5b1720 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.758+0000 7f8c8c52ab80 10 client.30406 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.758+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55718a328b60 con 0x55718a5b1720 2024-05-08T23:22:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.758+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55718a454270 con 0x55718a5b1720 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c6c0cc340 con 0x55718a5b1720 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 10 client.30406 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 10 client.30406 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 20 client.30406 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a5c4e80 0x55718a5c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 4 client.30406 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.759+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55718a57a810 con 0x55718a5c4e80 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.760+0000 7f8c71e00700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a5c4e80 0x55718a5c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.761+0000 7f8c71e00700 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a5c4e80 0x55718a5c7270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0bce0f37db9077d server_cookie=2d3d01554b51d4eb in_seq=0 out_seq=0 2024-05-08T23:22:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.761+0000 7f8c53400700 10 client.30406 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c53400700 1 -- 172.21.15.46:0/3955698558 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55718a57a810 con 0x55718a5c4e80 2024-05-08T23:22:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c53400700 10 client.30406 handle_command_reply: tid=0 2024-05-08T23:22:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 client.30406 shutdown 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 msgr2=0x55718a5b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c095830 tx=0x7f8c6c097930 comp rx=0 tx=0).stop 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 shutdown_connections 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b1720 0x55718a5b5570 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a5c4e80 0x55718a5c7270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5ab890 0x55718a5b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c53400700 0 client.30406 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3955698558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5b0e00 0x55718a5b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 >> 172.21.15.46:0/3955698558 conn(0x55718a509210 msgr2=0x55718a5a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 shutdown_connections 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.763+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3955698558 wait complete. 2024-05-08T23:22:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.764+0000 7f8c8c52ab80 20 client.30406 trim_cache size 0 max 16384 2024-05-08T23:22:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.790+0000 7f8c8c52ab80 1 Processor -- start 2024-05-08T23:22:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.791+0000 7f8c8c52ab80 1 -- start start 2024-05-08T23:22:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.791+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.791+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.791+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.792+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5cc440 con 0x55718a5b1720 2024-05-08T23:22:20.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.792+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5a7a00 con 0x55718a5ab890 2024-05-08T23:22:20.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.792+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5a0800 con 0x55718a5b0e00 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.792+0000 7f8c72800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.792+0000 7f8c71e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.793+0000 7f8c71e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33066/0 (socket says 172.21.15.46:33066) 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.793+0000 7f8c71e00700 1 -- 172.21.15.46:0/862423084 learned_addr learned my addr 172.21.15.46:0/862423084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.793+0000 7f8c71400700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71e00700 1 -- 172.21.15.46:0/862423084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 msgr2=0x55718a5b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71e00700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71e00700 1 -- 172.21.15.46:0/862423084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 msgr2=0x55718a5a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71e00700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71e00700 1 -- 172.21.15.46:0/862423084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718a4c1560 con 0x55718a5ab890 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c72800700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:20.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.794+0000 7f8c71400700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.795+0000 7f8c71e00700 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009990 tx=0x7f8c6401a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864edc68ad978504 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.796+0000 7f8c5fa00700 1 -- 172.21.15.46:0/862423084 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640048e0 con 0x55718a5ab890 2024-05-08T23:22:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.796+0000 7f8c5fa00700 1 -- 172.21.15.46:0/862423084 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6402c040 con 0x55718a5ab890 2024-05-08T23:22:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.797+0000 7f8c5fa00700 1 -- 172.21.15.46:0/862423084 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400f6c0 con 0x55718a5ab890 2024-05-08T23:22:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.797+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/862423084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 msgr2=0x55718a5ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.797+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009990 tx=0x7f8c6401a050 comp rx=0 tx=0).stop 2024-05-08T23:22:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.798+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/862423084 shutdown_connections 2024-05-08T23:22:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.798+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.798+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5ab350 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.798+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/862423084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.798+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/862423084 >> 172.21.15.46:0/862423084 conn(0x55718a509210 msgr2=0x55718a5b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.800+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/862423084 shutdown_connections 2024-05-08T23:22:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.801+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/862423084 wait complete. 2024-05-08T23:22:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.801+0000 7f8c8c52ab80 1 -- start start 2024-05-08T23:22:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.801+0000 7f8c8c52ab80 1 Processor -- start 2024-05-08T23:22:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.802+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.802+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c8c52ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5bb3d0 con 0x55718a5b1720 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5bb510 con 0x55718a5ab890 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c8c52ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718a5bb6d0 con 0x55718a5b0e00 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c8c52ab80 10 client.0 _ll_register_callbacks cb 0x55718a4a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51368/0 (socket says 172.21.15.46:51368) 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.803+0000 7f8c71400700 1 -- 172.21.15.46:0/3743157554 learned_addr learned my addr 172.21.15.46:0/3743157554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.804+0000 7f8c72800700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.804+0000 7f8c71e00700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.805+0000 7f8c71400700 1 -- 172.21.15.46:0/3743157554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 msgr2=0x55718a5bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.805+0000 7f8c71400700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5bc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.805+0000 7f8c71400700 1 -- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 msgr2=0x55718a5bae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.805+0000 7f8c71400700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5bae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.805+0000 7f8c71400700 1 -- 172.21.15.46:0/3743157554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55718a328b60 con 0x55718a5b0e00 2024-05-08T23:22:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.806+0000 7f8c71400700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8c741a7630 tx=0x7f8c600119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4cab909df53c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.806+0000 7f8c53400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.806+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60016070 con 0x55718a5b0e00 2024-05-08T23:22:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.807+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c6c099380 con 0x55718a5b0e00 2024-05-08T23:22:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.807+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8c6000f5d0 con 0x55718a5b0e00 2024-05-08T23:22:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.808+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6000f790 con 0x55718a5b0e00 2024-05-08T23:22:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.809+0000 7f8c8c52ab80 10 client.30409 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.810+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f8c7411d330 con 0x55718a5b0e00 2024-05-08T23:22:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.811+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c099380 con 0x55718a5b0e00 2024-05-08T23:22:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.811+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c60023c60 con 0x55718a5b0e00 2024-05-08T23:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 10 client.30409 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 10 client.30409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 20 client.30409 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a616850 0x55718a616c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 4 client.30409 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.812+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55718a4f3ec0 con 0x55718a616850 2024-05-08T23:22:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.816+0000 7f8c71e00700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a616850 0x55718a616c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.818+0000 7f8c71e00700 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a616850 0x55718a616c60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7c7cecd1e981c15 server_cookie=e897fe518316f8c7 in_seq=0 out_seq=0 2024-05-08T23:22:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.818+0000 7f8c53400700 10 client.30409 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.823+0000 7f8c53400700 1 -- 172.21.15.46:0/3743157554 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55718a4f3ec0 con 0x55718a616850 2024-05-08T23:22:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.823+0000 7f8c53400700 10 client.30409 handle_command_reply: tid=0 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.823+0000 7f8c8c52ab80 1 client.30409 shutdown 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 msgr2=0x55718a5ba8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8c741a7630 tx=0x7f8c600119a0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 shutdown_connections 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55718a616850 0x55718a616c60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55718a5b1720 0x55718a5bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55718a5ab890 0x55718a5bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c53400700 0 client.30409 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 --2- 172.21.15.46:0/3743157554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55718a5b0e00 0x55718a5ba8c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 >> 172.21.15.46:0/3743157554 conn(0x55718a509210 msgr2=0x55718a5b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 shutdown_connections 2024-05-08T23:22:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.825+0000 7f8c8c52ab80 1 -- 172.21.15.46:0/3743157554 wait complete. 2024-05-08T23:22:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.829+0000 7f8c8c52ab80 20 client.30409 trim_cache size 0 max 16384 2024-05-08T23:22:20.829 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.831+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 msgr2=0x7f8c580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 0x7f8c580a37d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c64006010 tx=0x7f8c6400b260 comp rx=0 tx=0).stop 2024-05-08T23:22:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 msgr2=0x7f8c741aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00ef30 tx=0x7f8c6c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 shutdown_connections 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c7410d9d0 0x7f8c741aa9c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c741085d0 0x7f8c741a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8c580a1310 0x7f8c580a37d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 --2- 172.21.15.46:0/352738112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c74108ef0 0x7f8c741a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 >> 172.21.15.46:0/352738112 conn(0x7f8c7406d060 msgr2=0x7f8c740716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.832+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 shutdown_connections 2024-05-08T23:22:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:20.834+0000 7f8c78a00700 1 -- 172.21.15.46:0/352738112 wait complete. 2024-05-08T23:22:20.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:22:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.234+0000 7f8986e00700 1 -- 172.21.15.46:0/1013486539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102060 msgr2=0x7f8980102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.234+0000 7f8986e00700 1 --2- 172.21.15.46:0/1013486539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102060 0x7f8980102440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f896c009990 tx=0x7f896c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 -- 172.21.15.46:0/1013486539 shutdown_connections 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 --2- 172.21.15.46:0/1013486539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f898010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 --2- 172.21.15.46:0/1013486539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102060 0x7f8980102440 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 --2- 172.21.15.46:0/1013486539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980102980 0x7f898010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 -- 172.21.15.46:0/1013486539 >> 172.21.15.46:0/1013486539 conn(0x7f89800fb760 msgr2=0x7f89800fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.235+0000 7f8986e00700 1 -- 172.21.15.46:0/1013486539 shutdown_connections 2024-05-08T23:22:21.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.236+0000 7f8986e00700 1 -- 172.21.15.46:0/1013486539 wait complete. 2024-05-08T23:22:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.237+0000 7f8986e00700 1 Processor -- start 2024-05-08T23:22:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.238+0000 7f8986e00700 1 -- start start 2024-05-08T23:22:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f8980197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 0x7f898019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019ca50 con 0x7f898019bfd0 2024-05-08T23:22:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019cbc0 con 0x7f8980102980 2024-05-08T23:22:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.240+0000 7f8986e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019cd80 con 0x7f898010b430 2024-05-08T23:22:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.241+0000 7f897fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.241+0000 7f897fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33092/0 (socket says 172.21.15.46:33092) 2024-05-08T23:22:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.241+0000 7f897fe00700 1 -- 172.21.15.46:0/660978666 learned_addr learned my addr 172.21.15.46:0/660978666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.241+0000 7f8984e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 0x7f898019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897fe00700 1 -- 172.21.15.46:0/660978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 msgr2=0x7f8980197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897f400700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f8980197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897fe00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f8980197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897fe00700 1 -- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 msgr2=0x7f898019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897fe00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 0x7f898019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.242+0000 7f897fe00700 1 -- 172.21.15.46:0/660978666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c0095f0 con 0x7f8980102980 2024-05-08T23:22:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.243+0000 7f897f400700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f8980197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.243+0000 7f8984e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 0x7f898019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.243+0000 7f897fe00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098e0 tx=0x7f896c003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.243+0000 7f897cc00700 1 -- 172.21.15.46:0/660978666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c01d070 con 0x7f8980102980 2024-05-08T23:22:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.244+0000 7f897cc00700 1 -- 172.21.15.46:0/660978666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c005580 con 0x7f8980102980 2024-05-08T23:22:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.244+0000 7f897cc00700 1 -- 172.21.15.46:0/660978666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00fcf0 con 0x7f8980102980 2024-05-08T23:22:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.244+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89801a4280 con 0x7f8980102980 2024-05-08T23:22:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.245+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a46b0 con 0x7f8980102980 2024-05-08T23:22:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.250+0000 7f897cc00700 1 -- 172.21.15.46:0/660978666 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f896c0050a0 con 0x7f8980102980 2024-05-08T23:22:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.254+0000 7f897cc00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 0x7f89640a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.254+0000 7f897cc00700 1 -- 172.21.15.46:0/660978666 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f896c0cdd80 con 0x7f8980102980 2024-05-08T23:22:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.254+0000 7f899a95ab80 1 Processor -- start 2024-05-08T23:22:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.255+0000 7f899a95ab80 1 -- start start 2024-05-08T23:22:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0cf4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0c4b150 con 0x55e9a0cfa720 2024-05-08T23:22:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0c48b10 con 0x55e9a0cf4890 2024-05-08T23:22:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f899a95ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0c44400 con 0x55e9a0cf9e00 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f897fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f8984e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f897fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51390/0 (socket says 172.21.15.46:51390) 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.256+0000 7f897fe00700 1 -- 172.21.15.46:0/1929136408 learned_addr learned my addr 172.21.15.46:0/1929136408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897fe00700 1 -- 172.21.15.46:0/1929136408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 msgr2=0x55e9a0cf4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897fe00700 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897fe00700 1 -- 172.21.15.46:0/1929136408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 msgr2=0x55e9a0cf4350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897f400700 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0cf4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897fe00700 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0cf4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897f400700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 0x7f89640a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.257+0000 7f897fe00700 1 -- 172.21.15.46:0/1929136408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a0ac53b0 con 0x55e9a0cf9e00 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.259+0000 7f897f400700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 0x7f89640a3810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e9a0cf0bd0 tx=0x7f8970008040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.259+0000 7f897fe00700 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f896c097010 tx=0x7f896c028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd345fd3fe310524 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.259+0000 7f8975600700 1 -- 172.21.15.46:0/1929136408 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00f460 con 0x55e9a0cf9e00 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.259+0000 7f8975600700 1 -- 172.21.15.46:0/1929136408 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c020e50 con 0x55e9a0cf9e00 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.259+0000 7f8975600700 1 -- 172.21.15.46:0/1929136408 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00f460 con 0x55e9a0cf9e00 2024-05-08T23:22:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 -- 172.21.15.46:0/1929136408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 msgr2=0x55e9a0cfa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f896c097010 tx=0x7f896c028000 comp rx=0 tx=0).stop 2024-05-08T23:22:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 -- 172.21.15.46:0/1929136408 shutdown_connections 2024-05-08T23:22:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfa1e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 --2- 172.21.15.46:0/1929136408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0cf4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.260+0000 7f899a95ab80 1 -- 172.21.15.46:0/1929136408 >> 172.21.15.46:0/1929136408 conn(0x55e9a0c52210 msgr2=0x55e9a0ceedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.261+0000 7f899a95ab80 1 -- 172.21.15.46:0/1929136408 shutdown_connections 2024-05-08T23:22:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.261+0000 7f899a95ab80 1 -- 172.21.15.46:0/1929136408 wait complete. 2024-05-08T23:22:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.262+0000 7f899a95ab80 1 -- start start 2024-05-08T23:22:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.262+0000 7f899a95ab80 1 Processor -- start 2024-05-08T23:22:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.263+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 0x55e9a0d00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.263+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.263+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f897f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f897f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33114/0 (socket says 172.21.15.46:33114) 2024-05-08T23:22:21.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f899a95ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0be0560 con 0x55e9a0cf4890 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f899a95ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0be0200 con 0x55e9a0cf9e00 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f899a95ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0ac76a0 con 0x55e9a0cfa720 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f899a95ab80 10 client.0 _ll_register_callbacks cb 0x55e9a0be9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f8984e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f8984e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51404/0 (socket says 172.21.15.46:51404) 2024-05-08T23:22:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f8984e00700 1 -- 172.21.15.46:0/3369307045 learned_addr learned my addr 172.21.15.46:0/3369307045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.264+0000 7f897fe00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 0x55e9a0d00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f8984e00700 1 -- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 msgr2=0x55e9a0cfda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f8984e00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f8984e00700 1 -- 172.21.15.46:0/3369307045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 msgr2=0x55e9a0d00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f8984e00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 0x55e9a0d00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f8984e00700 1 -- 172.21.15.46:0/3369307045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a46b0 con 0x55e9a0cfa720 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f897fe00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 0x55e9a0d00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.265+0000 7f897f400700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f8984e00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f897800b840 tx=0x7f897800bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67566a3f975ae719 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f896a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897801c070 con 0x55e9a0cfa720 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f897800a850 con 0x55e9a0cfa720 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9a0b9d270 con 0x55e9a0cfa720 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.266+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f897800aa10 con 0x55e9a0cfa720 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f899a95ab80 10 client.30415 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e9a0a71b60 con 0x55e9a0cfa720 2024-05-08T23:22:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9a0b9d270 con 0x55e9a0cfa720 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8978021420 con 0x55e9a0cfa720 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f899a95ab80 10 client.30415 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.267+0000 7f899a95ab80 10 client.30415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.268+0000 7f899a95ab80 20 client.30415 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.268+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d1e2e0 0x55e9a0d1e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.268+0000 7f899a95ab80 4 client.30415 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.268+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9a0cc3810 con 0x55e9a0d1e2e0 2024-05-08T23:22:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.268+0000 7f897fe00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d1e2e0 0x55e9a0d1e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.270+0000 7f896a000700 10 client.30415 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.271+0000 7f897fe00700 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d1e2e0 0x55e9a0d1e6f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb141d1a3eb87ac8 server_cookie=8188a01edb7034ba in_seq=0 out_seq=0 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.272+0000 7f896a000700 1 -- 172.21.15.46:0/3369307045 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9a0cc3810 con 0x55e9a0d1e2e0 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.272+0000 7f896a000700 10 client.30415 handle_command_reply: tid=0 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.272+0000 7f899a95ab80 1 client.30415 shutdown 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 msgr2=0x55e9a0cfe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f897800b840 tx=0x7f897800bb50 comp rx=0 tx=0).stop 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 shutdown_connections 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0cfe150 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d1e2e0 0x55e9a0d1e6f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf4890 0x55e9a0d00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f896a000700 0 client.30415 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3369307045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cfda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 >> 172.21.15.46:0/3369307045 conn(0x55e9a0c52210 msgr2=0x55e9a0cee5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.273+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 shutdown_connections 2024-05-08T23:22:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.274+0000 7f899a95ab80 1 -- 172.21.15.46:0/3369307045 wait complete. 2024-05-08T23:22:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.275+0000 7f899a95ab80 20 client.30415 trim_cache size 0 max 16384 2024-05-08T23:22:21.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 107927 19.54 MB/sec execute 162 sec latency 230.581 ms 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.306+0000 7f899a95ab80 1 Processor -- start 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.306+0000 7f899a95ab80 1 -- start start 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.306+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.306+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf9e00 0x55e9a0d1af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.306+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f899a95ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0cfedd0 con 0x55e9a0cf9e00 2024-05-08T23:22:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f899a95ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0cebdf0 con 0x55e9a0cf4890 2024-05-08T23:22:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f899a95ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0d1b9d0 con 0x55e9a0cfa720 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33120/0 (socket says 172.21.15.46:33120) 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 -- 172.21.15.46:0/47208274 learned_addr learned my addr 172.21.15.46:0/47208274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897f400700 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf9e00 0x55e9a0d1af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f8984e00700 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 -- 172.21.15.46:0/47208274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 msgr2=0x55e9a0ceed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 -- 172.21.15.46:0/47208274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf9e00 msgr2=0x55e9a0d1af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf9e00 0x55e9a0d1af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.307+0000 7f897fe00700 1 -- 172.21.15.46:0/47208274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a0ceb4e0 con 0x55e9a0cf4890 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.308+0000 7f897fe00700 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb990b342b32149 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.308+0000 7f8976000700 1 -- 172.21.15.46:0/47208274 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00f460 con 0x55e9a0cf4890 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.308+0000 7f8976000700 1 -- 172.21.15.46:0/47208274 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c09b210 con 0x55e9a0cf4890 2024-05-08T23:22:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.308+0000 7f8976000700 1 -- 172.21.15.46:0/47208274 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c09afd0 con 0x55e9a0cf4890 2024-05-08T23:22:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 -- 172.21.15.46:0/47208274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 msgr2=0x55e9a0cf4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 -- 172.21.15.46:0/47208274 shutdown_connections 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf4890 0x55e9a0cf4350 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 --2- 172.21.15.46:0/47208274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cf9e00 0x55e9a0d1af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.311+0000 7f899a95ab80 1 -- 172.21.15.46:0/47208274 >> 172.21.15.46:0/47208274 conn(0x55e9a0c52210 msgr2=0x55e9a0cf8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.312+0000 7f899a95ab80 1 -- 172.21.15.46:0/47208274 shutdown_connections 2024-05-08T23:22:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.312+0000 7f899a95ab80 1 -- 172.21.15.46:0/47208274 wait complete. 2024-05-08T23:22:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.313+0000 7f899a95ab80 1 -- start start 2024-05-08T23:22:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.313+0000 7f899a95ab80 1 Processor -- start 2024-05-08T23:22:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.315+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.315+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.315+0000 7f899a95ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f897f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f897f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33136/0 (socket says 172.21.15.46:33136) 2024-05-08T23:22:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f8984e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f8984e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59616/0 (socket says 172.21.15.46:59616) 2024-05-08T23:22:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f899a95ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0d74290 con 0x55e9a0cfa720 2024-05-08T23:22:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f899a95ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0d74400 con 0x55e9a0cf9e00 2024-05-08T23:22:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f899a95ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a0d745c0 con 0x55e9a0cf4890 2024-05-08T23:22:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.316+0000 7f899a95ab80 10 client.0 _ll_register_callbacks cb 0x55e9a0be9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51428/0 (socket says 172.21.15.46:51428) 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 -- 172.21.15.46:0/3285624628 learned_addr learned my addr 172.21.15.46:0/3285624628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 -- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 msgr2=0x55e9a0cee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 -- 172.21.15.46:0/3285624628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 msgr2=0x55e9a0ceed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.317+0000 7f897fe00700 1 -- 172.21.15.46:0/3285624628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9a0ac53b0 con 0x55e9a0cf4890 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.318+0000 7f8984e00700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T23:22:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.319+0000 7f897f400700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T23:22:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.319+0000 7f897fe00700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f896c0cb6c0 tx=0x7f896c028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1635ff5c0fb598 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.319+0000 7f896a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.319+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c09a000 con 0x55e9a0cf4890 2024-05-08T23:22:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.320+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f896c09a160 con 0x55e9a0cf4890 2024-05-08T23:22:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.320+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f896c020e50 con 0x55e9a0cf4890 2024-05-08T23:22:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.320+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896c09ad70 con 0x55e9a0cf4890 2024-05-08T23:22:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.320+0000 7f899a95ab80 10 client.30421 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.320+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e9a0c0a560 con 0x55e9a0cf4890 2024-05-08T23:22:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.321+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c020e50 con 0x55e9a0cf4890 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.321+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f896c0b3130 con 0x55e9a0cf4890 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 10 client.30421 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 10 client.30421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 20 client.30421 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d75850 0x55e9a0d77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 4 client.30421 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9a0c3cec0 con 0x55e9a0d75850 2024-05-08T23:22:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.322+0000 7f897f400700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d75850 0x55e9a0d77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.323+0000 7f896a000700 10 client.30421 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.323+0000 7f897f400700 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d75850 0x55e9a0d77c40 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba810461eec4a18 server_cookie=655e8455ab0bb170 in_seq=0 out_seq=0 2024-05-08T23:22:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.324+0000 7f896a000700 1 -- 172.21.15.46:0/3285624628 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9a0c3cec0 con 0x55e9a0d75850 2024-05-08T23:22:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.324+0000 7f896a000700 10 client.30421 handle_command_reply: tid=0 2024-05-08T23:22:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.324+0000 7f899a95ab80 1 client.30421 shutdown 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.324+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 msgr2=0x55e9a0cee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.324+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f896c0cb6c0 tx=0x7f896c028000 comp rx=0 tx=0).stop 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 shutdown_connections 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55e9a0d75850 0x55e9a0d77c40 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9a0cfa720 0x55e9a0ceed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9a0cf4890 0x55e9a0cee310 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 --2- 172.21.15.46:0/3285624628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9a0cf9e00 0x55e9a0cee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 >> 172.21.15.46:0/3285624628 conn(0x55e9a0c52210 msgr2=0x55e9a0d1f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f896a000700 0 client.30421 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 shutdown_connections 2024-05-08T23:22:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.325+0000 7f899a95ab80 1 -- 172.21.15.46:0/3285624628 wait complete. 2024-05-08T23:22:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.326+0000 7f899a95ab80 20 client.30421 trim_cache size 0 max 16384 2024-05-08T23:22:21.326 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 msgr2=0x7f89640a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 0x7f89640a3810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e9a0cf0bd0 tx=0x7f8970008040 comp rx=0 tx=0).stop 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 msgr2=0x7f89801976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f896c0098e0 tx=0x7f896c003b60 comp rx=0 tx=0).stop 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 shutdown_connections 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f89640a1350 0x7f89640a3810 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898019bfd0 0x7f898019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8980102980 0x7f89801976f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 --2- 172.21.15.46:0/660978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898010b430 0x7f8980197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 >> 172.21.15.46:0/660978666 conn(0x7f89800fb760 msgr2=0x7f89800fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.327+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 shutdown_connections 2024-05-08T23:22:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:21.328+0000 7f8986e00700 1 -- 172.21.15.46:0/660978666 wait complete. 2024-05-08T23:22:22.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:21 smithi086 ceph-mon[78610]: pgmap v583: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:21 smithi053 ceph-mon[77587]: pgmap v583: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:22.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 108309 19.51 MB/sec execute 163 sec latency 204.679 ms 2024-05-08T23:22:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:21 smithi046 ceph-mon[36898]: pgmap v583: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:22.363 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:22:22.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:22:23.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 108664 19.51 MB/sec execute 164 sec latency 257.836 ms 2024-05-08T23:22:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 -- 172.21.15.46:0/2157819019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c103c60 msgr2=0x7f634c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 --2- 172.21.15.46:0/2157819019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c103c60 0x7f634c107ca0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6338009990 tx=0x7f6338009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 -- 172.21.15.46:0/2157819019 shutdown_connections 2024-05-08T23:22:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 --2- 172.21.15.46:0/2157819019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1083d0 0x7f634c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 --2- 172.21.15.46:0/2157819019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c1032b0 0x7f634c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 --2- 172.21.15.46:0/2157819019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c103c60 0x7f634c107ca0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 -- 172.21.15.46:0/2157819019 >> 172.21.15.46:0/2157819019 conn(0x7f634c0feb10 msgr2=0x7f634c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 -- 172.21.15.46:0/2157819019 shutdown_connections 2024-05-08T23:22:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.336+0000 7f6353600700 1 -- 172.21.15.46:0/2157819019 wait complete. 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.337+0000 7f6353600700 1 Processor -- start 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 -- start start 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 0x7f634c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c19e910 con 0x7f634c1032b0 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c19ea80 con 0x7f634c19de90 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6353600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c19ec40 con 0x7f634c1083d0 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f634be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f6351600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.338+0000 7f634be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39246/0 (socket says 172.21.15.46:39246) 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54222/0 (socket says 172.21.15.46:54222) 2024-05-08T23:22:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 -- 172.21.15.46:0/2988145466 learned_addr learned my addr 172.21.15.46:0/2988145466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6350c00700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 0x7f634c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 -- 172.21.15.46:0/2988145466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 msgr2=0x7f634c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 -- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 msgr2=0x7f634c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 0x7f634c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 -- 172.21.15.46:0/2988145466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380095f0 con 0x7f634c19de90 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6350c00700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 0x7f634c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f634be00700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:23.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.339+0000 7f6351600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6344007ee0 tx=0x7f634400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.340+0000 7f6349600700 1 -- 172.21.15.46:0/2988145466 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634400dd20 con 0x7f634c19de90 2024-05-08T23:22:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.340+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c1a61a0 con 0x7f634c19de90 2024-05-08T23:22:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.340+0000 7f6349600700 1 -- 172.21.15.46:0/2988145466 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634400f040 con 0x7f634c19de90 2024-05-08T23:22:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.340+0000 7f6349600700 1 -- 172.21.15.46:0/2988145466 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440048a0 con 0x7f634c19de90 2024-05-08T23:22:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.340+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c1a66d0 con 0x7f634c19de90 2024-05-08T23:22:23.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.345+0000 7f6349600700 1 -- 172.21.15.46:0/2988145466 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6344003a00 con 0x7f634c19de90 2024-05-08T23:22:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.347+0000 7f636711cb80 1 Processor -- start 2024-05-08T23:22:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f6349600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 0x7f63340a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f636711cb80 1 -- start start 2024-05-08T23:22:23.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f6349600700 1 -- 172.21.15.46:0/2988145466 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f63440cc880 con 0x7f634c19de90 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f636711cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f636711cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e064fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.348+0000 7f636711cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e064fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f6350c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f6350c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54236/0 (socket says 172.21.15.46:54236) 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f636711cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0644a1d0 con 0x555e064fc120 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f636711cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e06447b90 con 0x555e06501670 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f636711cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e06443480 con 0x555e06501f90 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f634be00700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 0x7f63340a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.349+0000 7f6350c00700 1 -- 172.21.15.46:0/3081993121 learned_addr learned my addr 172.21.15.46:0/3081993121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e064fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f634be00700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e064fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 -- 172.21.15.46:0/3081993121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 msgr2=0x555e064fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e064fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 -- 172.21.15.46:0/3081993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 msgr2=0x555e06501a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.350+0000 7f6351600700 1 -- 172.21.15.46:0/3081993121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e062c4430 con 0x555e064fc120 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f634be00700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 0x7f63340a3850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f634c19ab00 tx=0x7f633800b5b0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f6351600700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e064fc580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f63440c9b60 tx=0x7f6344099370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94a1dd9fcb700dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f633dc00700 1 -- 172.21.15.46:0/3081993121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634409d170 con 0x555e064fc120 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f633dc00700 1 -- 172.21.15.46:0/3081993121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6344099c20 con 0x555e064fc120 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f633dc00700 1 -- 172.21.15.46:0/3081993121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440840f0 con 0x555e064fc120 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.351+0000 7f6350c00700 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 -- 172.21.15.46:0/3081993121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 msgr2=0x555e064fc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e064fc580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f63440c9b60 tx=0x7f6344099370 comp rx=0 tx=0).stop 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 -- 172.21.15.46:0/3081993121 shutdown_connections 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e064fc580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e06501a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 --2- 172.21.15.46:0/3081993121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e064fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.353+0000 7f636711cb80 1 -- 172.21.15.46:0/3081993121 >> 172.21.15.46:0/3081993121 conn(0x555e06451290 msgr2=0x555e064ede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.354+0000 7f636711cb80 1 -- 172.21.15.46:0/3081993121 shutdown_connections 2024-05-08T23:22:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.354+0000 7f636711cb80 1 -- 172.21.15.46:0/3081993121 wait complete. 2024-05-08T23:22:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.354+0000 7f636711cb80 1 -- start start 2024-05-08T23:22:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 Processor -- start 2024-05-08T23:22:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e064fc120 0x555e06505420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e06505ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e063df5e0 con 0x555e06501670 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e063df280 con 0x555e06501f90 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e062c6720 con 0x555e064fc120 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.355+0000 7f636711cb80 10 client.0 _ll_register_callbacks cb 0x555e063e8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56538/0 (socket says 172.21.15.46:56538) 2024-05-08T23:22:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 -- 172.21.15.46:0/2295893503 learned_addr learned my addr 172.21.15.46:0/2295893503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f6351600700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e06505ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 -- 172.21.15.46:0/2295893503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e064fc120 msgr2=0x555e06505420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e064fc120 0x555e06505420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 -- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 msgr2=0x555e06505ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e06505ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.356+0000 7f634be00700 1 -- 172.21.15.46:0/2295893503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6340009dd0 con 0x555e06501670 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.357+0000 7f634be00700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f634c068d20 tx=0x7f6338009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573970859b303728 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.357+0000 7f6332000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.359+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633801d030 con 0x555e06501670 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.359+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6338026750 con 0x555e06501670 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.359+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e0639c2f0 con 0x555e06501670 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.359+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6338026910 con 0x555e06501670 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.360+0000 7f636711cb80 10 client.20187 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.360+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f634c1a66d0 con 0x555e06501670 2024-05-08T23:22:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.360+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e0639c2f0 con 0x555e06501670 2024-05-08T23:22:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f633803bc50 con 0x555e06501670 2024-05-08T23:22:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 10 client.20187 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 10 client.20187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 20 client.20187 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e06515a70 0x555e06517e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 4 client.20187 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e064c2890 con 0x555e06515a70 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.361+0000 7f6350c00700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e06515a70 0x555e06517e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.362+0000 7f6350c00700 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e06515a70 0x555e06517e90 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1ffa0a3c33c33e server_cookie=1589a7ddaab177eb in_seq=0 out_seq=0 2024-05-08T23:22:23.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.362+0000 7f6332000700 10 client.20187 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.362+0000 7f6332000700 1 -- 172.21.15.46:0/2295893503 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e064c2890 con 0x555e06515a70 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.362+0000 7f6332000700 10 client.20187 handle_command_reply: tid=0 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.362+0000 7f636711cb80 1 client.20187 shutdown 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 msgr2=0x555e06505960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f634c068d20 tx=0x7f6338009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 shutdown_connections 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e06505ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e064fc120 0x555e06505420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e06515a70 0x555e06517e90 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 --2- 172.21.15.46:0/2295893503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e06501670 0x555e06505960 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 >> 172.21.15.46:0/2295893503 conn(0x555e06451290 msgr2=0x555e064ed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f6332000700 0 client.20187 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 shutdown_connections 2024-05-08T23:22:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.363+0000 7f636711cb80 1 -- 172.21.15.46:0/2295893503 wait complete. 2024-05-08T23:22:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.364+0000 7f636711cb80 20 client.20187 trim_cache size 0 max 16384 2024-05-08T23:22:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.387+0000 7f636711cb80 1 Processor -- start 2024-05-08T23:22:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.388+0000 7f636711cb80 1 -- start start 2024-05-08T23:22:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.388+0000 7f636711cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.388+0000 7f636711cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e064edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.388+0000 7f636711cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e06505970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f6350c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f6350c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56552/0 (socket says 172.21.15.46:56552) 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f636711cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e064e60c0 con 0x555e064fc120 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f636711cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e064fcae0 con 0x555e06501670 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f636711cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e064eae40 con 0x555e06501f90 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f6350c00700 1 -- 172.21.15.46:0/3705952588 learned_addr learned my addr 172.21.15.46:0/3705952588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f634be00700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e064edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.389+0000 7f6351600700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e06505970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f6350c00700 1 -- 172.21.15.46:0/3705952588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 msgr2=0x555e06505970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f6350c00700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e06505970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f6350c00700 1 -- 172.21.15.46:0/3705952588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 msgr2=0x555e064edda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f6350c00700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e064edda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f6350c00700 1 -- 172.21.15.46:0/3705952588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e06270be0 con 0x555e064fc120 2024-05-08T23:22:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.390+0000 7f634be00700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e064edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.391+0000 7f6350c00700 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63400046e0 tx=0x7f6340014970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1928260ed1154a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.391+0000 7f633e600700 1 -- 172.21.15.46:0/3705952588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340017040 con 0x555e064fc120 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.391+0000 7f633e600700 1 -- 172.21.15.46:0/3705952588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634001c3a0 con 0x555e064fc120 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.391+0000 7f633e600700 1 -- 172.21.15.46:0/3705952588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400054d0 con 0x555e064fc120 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.392+0000 7f636711cb80 1 -- 172.21.15.46:0/3705952588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 msgr2=0x555e06500c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.392+0000 7f636711cb80 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63400046e0 tx=0x7f6340014970 comp rx=0 tx=0).stop 2024-05-08T23:22:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 -- 172.21.15.46:0/3705952588 shutdown_connections 2024-05-08T23:22:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501f90 0x555e06505970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e06500c00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 --2- 172.21.15.46:0/3705952588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501670 0x555e064edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 -- 172.21.15.46:0/3705952588 >> 172.21.15.46:0/3705952588 conn(0x555e06451290 msgr2=0x555e065004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.393+0000 7f636711cb80 1 -- 172.21.15.46:0/3705952588 shutdown_connections 2024-05-08T23:22:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.394+0000 7f636711cb80 1 -- 172.21.15.46:0/3705952588 wait complete. 2024-05-08T23:22:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.395+0000 7f636711cb80 1 -- start start 2024-05-08T23:22:23.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.395+0000 7f636711cb80 1 Processor -- start 2024-05-08T23:22:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e0650b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 0x555e0650bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e0650bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0656c010 con 0x555e064fc120 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0656c180 con 0x555e06501f90 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0656c340 con 0x555e06501670 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.396+0000 7f636711cb80 10 client.0 _ll_register_callbacks cb 0x555e063e8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e0650bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6350c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e0650b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f634be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 0x555e0650bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f634be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 0x555e0650bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39276/0 (socket says 172.21.15.46:39276) 2024-05-08T23:22:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f634be00700 1 -- 172.21.15.46:0/2208560803 learned_addr learned my addr 172.21.15.46:0/2208560803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 -- 172.21.15.46:0/2208560803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 msgr2=0x555e0650bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 0x555e0650bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 -- 172.21.15.46:0/2208560803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 msgr2=0x555e0650b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e0650b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.397+0000 7f6351600700 1 -- 172.21.15.46:0/2208560803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c1a66d0 con 0x555e06501f90 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.398+0000 7f6351600700 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e0650bff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f634409d510 tx=0x7f6344098f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35fc4cadb668f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.398+0000 7f6332000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.398+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344095040 con 0x555e06501f90 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.398+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f634409d350 con 0x555e06501f90 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.398+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6344099c20 con 0x555e06501f90 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6344009410 con 0x555e06501f90 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 10 client.30404 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f63400120f0 con 0x555e06501f90 2024-05-08T23:22:23.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6344099c20 con 0x555e06501f90 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63440cb030 con 0x555e06501f90 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 10 client.30404 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 10 client.30404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 20 client.30404 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e0656d860 0x555e0656fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 4 client.30404 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.399+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555e0643bf40 con 0x555e0656d860 2024-05-08T23:22:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.400+0000 7f6350c00700 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e0656d860 0x555e0656fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.401+0000 7f6350c00700 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e0656d860 0x555e0656fc80 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc9050c99b4a128 server_cookie=9df74b3934fc98b3 in_seq=0 out_seq=0 2024-05-08T23:22:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.401+0000 7f6332000700 10 client.30404 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f6332000700 1 -- 172.21.15.46:0/2208560803 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555e0643bf40 con 0x555e0656d860 2024-05-08T23:22:23.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f6332000700 10 client.30404 handle_command_reply: tid=0 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 client.30404 shutdown 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 msgr2=0x555e0650bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e0650bff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f634409d510 tx=0x7f6344098f40 comp rx=0 tx=0).stop 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 shutdown_connections 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e06501f90 0x555e0650bff0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555e0656d860 0x555e0656fc80 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e064fc120 0x555e0650b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 --2- 172.21.15.46:0/2208560803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e06501670 0x555e0650bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 >> 172.21.15.46:0/2208560803 conn(0x555e06451290 msgr2=0x555e064f23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.403+0000 7f6332000700 0 client.30404 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.404+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 shutdown_connections 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.404+0000 7f636711cb80 1 -- 172.21.15.46:0/2208560803 wait complete. 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.404+0000 7f636711cb80 20 client.30404 trim_cache size 0 max 16384 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "b4dd985f-3e70-4253-b6f5-9890357b5e13", 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:22:23.405 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.405+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 msgr2=0x7f63340a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.405+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 0x7f63340a3850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f634c19ab00 tx=0x7f633800b5b0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 msgr2=0x7f634c19e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6344007ee0 tx=0x7f634400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 shutdown_connections 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f634c19de90 0x7f634c19e310 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f63340a1390 0x7f63340a3850 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f634c1032b0 0x7f634c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 --2- 172.21.15.46:0/2988145466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f634c1083d0 0x7f634c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 >> 172.21.15.46:0/2988145466 conn(0x7f634c0feb10 msgr2=0x7f634c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.406+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 shutdown_connections 2024-05-08T23:22:23.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.407+0000 7f6353600700 1 -- 172.21.15.46:0/2988145466 wait complete. 2024-05-08T23:22:23.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:22:23.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 -- 172.21.15.46:0/2483244688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df4103680 msgr2=0x7f2df4103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 --2- 172.21.15.46:0/2483244688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df4103680 0x7f2df4103b00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009f90 tx=0x7f2dec00b330 comp rx=0 tx=0).stop 2024-05-08T23:22:23.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 -- 172.21.15.46:0/2483244688 shutdown_connections 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 --2- 172.21.15.46:0/2483244688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df4103680 0x7f2df4103b00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 --2- 172.21.15.46:0/2483244688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4105bf0 0x7f2df4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 --2- 172.21.15.46:0/2483244688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df40686a0 0x7f2df4068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.790+0000 7f2dfa000700 1 -- 172.21.15.46:0/2483244688 >> 172.21.15.46:0/2483244688 conn(0x7f2df4075710 msgr2=0x7f2df4075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.791+0000 7f2dfa000700 1 -- 172.21.15.46:0/2483244688 shutdown_connections 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.791+0000 7f2dfa000700 1 -- 172.21.15.46:0/2483244688 wait complete. 2024-05-08T23:22:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.792+0000 7f2dfa000700 1 Processor -- start 2024-05-08T23:22:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.792+0000 7f2dfa000700 1 -- start start 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 0x7f2df419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 0x7f2df419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 0x7f2df4194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4195300 con 0x7f2df4103680 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4195470 con 0x7f2df4105bf0 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2dfa000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4195630 con 0x7f2df40686a0 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 0x7f2df419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 0x7f2df419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 0x7f2df4194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 0x7f2df419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39304/0 (socket says 172.21.15.46:39304) 2024-05-08T23:22:23.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df8c00700 1 -- 172.21.15.46:0/496637657 learned_addr learned my addr 172.21.15.46:0/496637657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.793+0000 7f2df3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 0x7f2df419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56578/0 (socket says 172.21.15.46:56578) 2024-05-08T23:22:23.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 -- 172.21.15.46:0/496637657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 msgr2=0x7f2df419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 0x7f2df419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 -- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 msgr2=0x7f2df419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 0x7f2df419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 -- 172.21.15.46:0/496637657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de001a600 con 0x7f2df4105bf0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df9600700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 0x7f2df4194b30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b6c0 tx=0x7f2dec00b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df1600700 1 -- 172.21.15.46:0/496637657 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00e040 con 0x7f2df4105bf0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2df1600700 1 -- 172.21.15.46:0/496637657 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dec01d470 con 0x7f2df4105bf0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.794+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de8009520 con 0x7f2df4105bf0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.795+0000 7f2df1600700 1 -- 172.21.15.46:0/496637657 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec01b6c0 con 0x7f2df4105bf0 2024-05-08T23:22:23.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.795+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec009c40 con 0x7f2df4105bf0 2024-05-08T23:22:23.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.799+0000 7f2df1600700 1 -- 172.21.15.46:0/496637657 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2dec01b820 con 0x7f2df4105bf0 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.800+0000 7f2e0f0bcb80 1 Processor -- start 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.800+0000 7f2e0f0bcb80 1 -- start start 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77cff4150 con 0x55c77d0a3050 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77cff1b10 con 0x55c77d09d890 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77cfed400 con 0x55c77d0a3970 2024-05-08T23:22:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39310/0 (socket says 172.21.15.46:39310) 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.801+0000 7f2df8c00700 1 -- 172.21.15.46:0/3568832064 learned_addr learned my addr 172.21.15.46:0/3568832064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 -- 172.21.15.46:0/3568832064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 msgr2=0x55c77d09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df3e00700 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 -- 172.21.15.46:0/3568832064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 msgr2=0x55c77d0a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 -- 172.21.15.46:0/3568832064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c77ce6e3b0 con 0x55c77d0a3970 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df3e00700 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2de80114c0 tx=0x7f2de80117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1470ab39903041bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2df1600700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 0x7f2ddc0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.802+0000 7f2de5c00700 1 -- 172.21.15.46:0/3568832064 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8019630 con 0x55c77d0a3970 2024-05-08T23:22:23.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.803+0000 7f2de5c00700 1 -- 172.21.15.46:0/3568832064 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de8019c50 con 0x55c77d0a3970 2024-05-08T23:22:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.803+0000 7f2de5c00700 1 -- 172.21.15.46:0/3568832064 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800fc90 con 0x55c77d0a3970 2024-05-08T23:22:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.803+0000 7f2df1600700 1 -- 172.21.15.46:0/496637657 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2dec0ce240 con 0x7f2df4105bf0 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2df3e00700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 0x7f2ddc0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3568832064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 msgr2=0x55c77d09d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2de80114c0 tx=0x7f2de80117d0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3568832064 shutdown_connections 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3568832064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3970 0x55c77d09d350 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3568832064 >> 172.21.15.46:0/3568832064 conn(0x55c77cffb210 msgr2=0x55c77d0982d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3568832064 shutdown_connections 2024-05-08T23:22:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.804+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3568832064 wait complete. 2024-05-08T23:22:23.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.805+0000 7f2df3e00700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 0x7f2ddc0a3840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2de001afd0 tx=0x7f2de001aa00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.805+0000 7f2e0f0bcb80 1 -- start start 2024-05-08T23:22:23.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.805+0000 7f2e0f0bcb80 1 Processor -- start 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3050 0x55c77d0a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d0a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77cf89560 con 0x55c77d0a3970 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77cf89200 con 0x55c77d0a3050 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ce706a0 con 0x55c77d09d890 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2e0f0bcb80 10 client.0 _ll_register_callbacks cb 0x55c77cf92b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39316/0 (socket says 172.21.15.46:39316) 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2df8c00700 1 -- 172.21.15.46:0/573435908 learned_addr learned my addr 172.21.15.46:0/573435908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.806+0000 7f2df3e00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3050 0x55c77d0a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df9600700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d0a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 -- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3050 msgr2=0x55c77d0a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3050 0x55c77d0a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 -- 172.21.15.46:0/573435908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 msgr2=0x55c77d0a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d0a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 -- 172.21.15.46:0/573435908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de8009c90 con 0x55c77d09d890 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2df8c00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2de801c040 tx=0x7f2de800fbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89aae6f83f9a0471 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.807+0000 7f2dda000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8019840 con 0x55c77d09d890 2024-05-08T23:22:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2de8019dc0 con 0x55c77d09d890 2024-05-08T23:22:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c77cf46270 con 0x55c77d09d890 2024-05-08T23:22:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de800fe90 con 0x55c77d09d890 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2e0f0bcb80 10 client.30448 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.808+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c77ce1ab60 con 0x55c77d09d890 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c77cf46270 con 0x55c77d09d890 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2de8028020 con 0x55c77d09d890 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 10 client.30448 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 10 client.30448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 20 client.30448 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d0b6fa0 0x55c77d0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 4 client.30448 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.809+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c77d06c810 con 0x55c77d0b6fa0 2024-05-08T23:22:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.810+0000 7f2df3e00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d0b6fa0 0x55c77d0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.810+0000 7f2df3e00700 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d0b6fa0 0x55c77d0b93c0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b635735285795570 server_cookie=2d22e7ffe8cfc199 in_seq=0 out_seq=0 2024-05-08T23:22:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.810+0000 7f2dda000700 10 client.30448 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.811+0000 7f2dda000700 1 -- 172.21.15.46:0/573435908 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c77d06c810 con 0x55c77d0b6fa0 2024-05-08T23:22:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.811+0000 7f2dda000700 10 client.30448 handle_command_reply: tid=0 2024-05-08T23:22:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.811+0000 7f2e0f0bcb80 1 client.30448 shutdown 2024-05-08T23:22:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.812+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 msgr2=0x55c77d0a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.812+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2de801c040 tx=0x7f2de800fbf0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 shutdown_connections 2024-05-08T23:22:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d0b6fa0 0x55c77d0b93c0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d0a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d0a9870 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/573435908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3050 0x55c77d0a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 >> 172.21.15.46:0/573435908 conn(0x55c77cffb210 msgr2=0x55c77d0980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2dda000700 0 client.30448 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 shutdown_connections 2024-05-08T23:22:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.813+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/573435908 wait complete. 2024-05-08T23:22:23.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.814+0000 7f2e0f0bcb80 20 client.30448 trim_cache size 0 max 16384 2024-05-08T23:22:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.850+0000 7f2e0f0bcb80 1 Processor -- start 2024-05-08T23:22:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.850+0000 7f2e0f0bcb80 1 -- start start 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d09d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 0x55c77d0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d0be5f0 con 0x55c77d0a3050 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d0987c0 con 0x55c77d0a3970 2024-05-08T23:22:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.851+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d099dd0 con 0x55c77d09d890 2024-05-08T23:22:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.852+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d09d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.852+0000 7f2df9600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 0x55c77d0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.852+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d09d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39322/0 (socket says 172.21.15.46:39322) 2024-05-08T23:22:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.852+0000 7f2df8c00700 1 -- 172.21.15.46:0/1001775805 learned_addr learned my addr 172.21.15.46:0/1001775805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.852+0000 7f2df3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 -- 172.21.15.46:0/1001775805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 msgr2=0x55c77d09d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d09d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 -- 172.21.15.46:0/1001775805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 msgr2=0x55c77d0a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 -- 172.21.15.46:0/1001775805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c77cfb3560 con 0x55c77d0a3970 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.853+0000 7f2df9600700 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 0x55c77d0a63f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0cb520 tx=0x7f2dec09adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6aaaa0c825c4543 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.854+0000 7f2de6600700 1 -- 172.21.15.46:0/1001775805 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec09af70 con 0x55c77d0a3970 2024-05-08T23:22:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.854+0000 7f2de6600700 1 -- 172.21.15.46:0/1001775805 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dec09eda0 con 0x55c77d0a3970 2024-05-08T23:22:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.854+0000 7f2de6600700 1 -- 172.21.15.46:0/1001775805 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec09af70 con 0x55c77d0a3970 2024-05-08T23:22:23.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.855+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/1001775805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 msgr2=0x55c77d0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 0x55c77d0a63f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0cb520 tx=0x7f2dec09adc0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/1001775805 shutdown_connections 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d0a3970 0x55c77d0a63f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d09d890 0x55c77d09d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/1001775805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3050 0x55c77d0a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.856+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/1001775805 >> 172.21.15.46:0/1001775805 conn(0x55c77cffb210 msgr2=0x55c77d0a5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.857+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/1001775805 shutdown_connections 2024-05-08T23:22:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.857+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/1001775805 wait complete. 2024-05-08T23:22:23.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.858+0000 7f2e0f0bcb80 1 -- start start 2024-05-08T23:22:23.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.858+0000 7f2e0f0bcb80 1 Processor -- start 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3050 0x55c77d0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d09f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d09f750 con 0x55c77d0a3970 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d09f8c0 con 0x55c77d09d890 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77d09fa80 con 0x55c77d0a3050 2024-05-08T23:22:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2e0f0bcb80 10 client.0 _ll_register_callbacks cb 0x55c77cf92b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2df8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54318/0 (socket says 172.21.15.46:54318) 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2df8c00700 1 -- 172.21.15.46:0/3516285034 learned_addr learned my addr 172.21.15.46:0/3516285034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2df3e00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3050 0x55c77d0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.859+0000 7f2df9600700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d09f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 -- 172.21.15.46:0/3516285034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3050 msgr2=0x55c77d0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3050 0x55c77d0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 -- 172.21.15.46:0/3516285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 msgr2=0x55c77d09f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d09f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 -- 172.21.15.46:0/3516285034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec009c40 con 0x55c77d09d890 2024-05-08T23:22:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2df8c00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2de800ea00 tx=0x7f2de80046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a040614f446a3f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2dda000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de80047a0 con 0x55c77d09d890 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de8019c50 con 0x55c77d09d890 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.860+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2de800f810 con 0x55c77d09d890 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.861+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de800f9d0 con 0x55c77d09d890 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.861+0000 7f2e0f0bcb80 10 client.30419 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.861+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c77ce72cd0 con 0x55c77d09d890 2024-05-08T23:22:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.861+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de8019c50 con 0x55c77d09d890 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.861+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2de803c030 con 0x55c77d09d890 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 10 client.30419 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 10 client.30419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 20 client.30419 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d10e4c0 0x55c77d1108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 4 client.30419 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c77cfe5ec0 con 0x55c77d10e4c0 2024-05-08T23:22:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.862+0000 7f2df3e00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d10e4c0 0x55c77d1108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.863+0000 7f2dda000700 10 client.30419 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.863+0000 7f2df3e00700 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d10e4c0 0x55c77d1108e0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe637b927138eb0 server_cookie=d14947ae7ee06b1b in_seq=0 out_seq=0 2024-05-08T23:22:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.864+0000 7f2dda000700 1 -- 172.21.15.46:0/3516285034 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c77cfe5ec0 con 0x55c77d10e4c0 2024-05-08T23:22:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.864+0000 7f2dda000700 10 client.30419 handle_command_reply: tid=0 2024-05-08T23:22:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.864+0000 7f2e0f0bcb80 1 client.30419 shutdown 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.864+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 msgr2=0x55c77d0ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.864+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2de800ea00 tx=0x7f2de80046e0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 shutdown_connections 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55c77d10e4c0 0x55c77d1108e0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c77d0a3970 0x55c77d09f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c77d09d890 0x55c77d0ae360 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 --2- 172.21.15.46:0/3516285034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c77d0a3050 0x55c77d0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 >> 172.21.15.46:0/3516285034 conn(0x55c77cffb210 msgr2=0x55c77d0a6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.865+0000 7f2dda000700 0 client.30419 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.866+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 shutdown_connections 2024-05-08T23:22:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.866+0000 7f2e0f0bcb80 1 -- 172.21.15.46:0/3516285034 wait complete. 2024-05-08T23:22:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.867+0000 7f2e0f0bcb80 20 client.30419 trim_cache size 0 max 16384 2024-05-08T23:22:23.866 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "b4dd985f-3e70-4253-b6f5-9890357b5e13": { 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "b4dd985f-3e70-4253-b6f5-9890357b5e13", 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:22:23.867 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 msgr2=0x7f2ddc0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 0x7f2ddc0a3840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2de001afd0 tx=0x7f2de001aa00 comp rx=0 tx=0).stop 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 msgr2=0x7f2df4194b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 0x7f2df4194b30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b6c0 tx=0x7f2dec00b9f0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 shutdown_connections 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2df4105bf0 0x7f2df4194b30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2df40686a0 0x7f2df419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f2ddc0a1380 0x7f2ddc0a3840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 --2- 172.21.15.46:0/496637657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df4103680 0x7f2df419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 >> 172.21.15.46:0/496637657 conn(0x7f2df4075710 msgr2=0x7f2df4100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.868+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 shutdown_connections 2024-05-08T23:22:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:23.869+0000 7f2dfa000700 1 -- 172.21.15.46:0/496637657 wait complete. 2024-05-08T23:22:23.893 INFO:tasks.cephfs.filesystem:scrub status for tag:b4dd985f-3e70-4253-b6f5-9890357b5e13 - {'path': '/', 'tag': 'b4dd985f-3e70-4253-b6f5-9890357b5e13', 'options': 'recursive,force'} 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: pgmap v584: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:23 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: pgmap v584: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:23 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: pgmap v584: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 402 op/s 2024-05-08T23:22:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:24.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:24.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:24.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:23 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:24.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 109002 19.50 MB/sec execute 165 sec latency 82.621 ms 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:25.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:25.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:24 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:25.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:24 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:22:25.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:22:25.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:25.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:22:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:24 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:22:25.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 109523 19.59 MB/sec execute 166 sec latency 81.150 ms 2024-05-08T23:22:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:25 smithi053 ceph-mon[77587]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 520 op/s 2024-05-08T23:22:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:25 smithi046 ceph-mon[36898]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 520 op/s 2024-05-08T23:22:26.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 110013 19.65 MB/sec execute 167 sec latency 156.488 ms 2024-05-08T23:22:26.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:25 smithi086 ceph-mon[78610]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 520 op/s 2024-05-08T23:22:27.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 110376 19.66 MB/sec execute 168 sec latency 194.991 ms 2024-05-08T23:22:28.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:27 smithi053 ceph-mon[77587]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-05-08T23:22:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:27 smithi046 ceph-mon[36898]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-05-08T23:22:28.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 110748 19.66 MB/sec execute 169 sec latency 218.512 ms 2024-05-08T23:22:28.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:27 smithi086 ceph-mon[78610]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-05-08T23:22:29.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 111172 19.63 MB/sec execute 170 sec latency 149.287 ms 2024-05-08T23:22:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:29 smithi053 ceph-mon[77587]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-08T23:22:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:29 smithi046 ceph-mon[36898]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-08T23:22:30.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 111599 19.65 MB/sec execute 171 sec latency 176.644 ms 2024-05-08T23:22:30.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:29 smithi086 ceph-mon[78610]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-08T23:22:31.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 112072 19.69 MB/sec execute 172 sec latency 87.011 ms 2024-05-08T23:22:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:31 smithi053 ceph-mon[77587]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-05-08T23:22:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:31 smithi046 ceph-mon[36898]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-05-08T23:22:32.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 112575 19.76 MB/sec execute 173 sec latency 228.695 ms 2024-05-08T23:22:32.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:31 smithi086 ceph-mon[78610]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-05-08T23:22:33.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 112997 19.78 MB/sec execute 174 sec latency 176.203 ms 2024-05-08T23:22:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:33 smithi053 ceph-mon[77587]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 401 op/s 2024-05-08T23:22:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:33 smithi046 ceph-mon[36898]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 401 op/s 2024-05-08T23:22:34.277 INFO:tasks.workunit.client.0.smithi046.stdout: 10 113386 19.78 MB/sec execute 175 sec latency 275.329 ms 2024-05-08T23:22:34.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:33 smithi086 ceph-mon[78610]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 401 op/s 2024-05-08T23:22:35.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 113804 19.79 MB/sec execute 176 sec latency 135.970 ms 2024-05-08T23:22:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:35 smithi053 ceph-mon[77587]: pgmap v590: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 500 op/s 2024-05-08T23:22:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:35 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:36 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:22:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:35 smithi046 ceph-mon[36898]: pgmap v590: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 500 op/s 2024-05-08T23:22:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:36.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:35 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:22:36.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 114214 19.81 MB/sec execute 177 sec latency 142.183 ms 2024-05-08T23:22:36.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:35 smithi086 ceph-mon[78610]: pgmap v590: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 500 op/s 2024-05-08T23:22:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:35 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:22:37.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 114623 19.82 MB/sec execute 178 sec latency 228.929 ms 2024-05-08T23:22:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:38 smithi053 ceph-mon[77587]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-08T23:22:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:38 smithi046 ceph-mon[36898]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-08T23:22:38.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 115009 19.81 MB/sec execute 179 sec latency 230.573 ms 2024-05-08T23:22:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:38 smithi086 ceph-mon[78610]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-05-08T23:22:39.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 115493 19.84 MB/sec execute 180 sec latency 173.129 ms 2024-05-08T23:22:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:40 smithi053 ceph-mon[77587]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-08T23:22:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:40 smithi046 ceph-mon[36898]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-08T23:22:40.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 116068 19.88 MB/sec execute 181 sec latency 180.636 ms 2024-05-08T23:22:40.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:40 smithi086 ceph-mon[78610]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-08T23:22:41.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 116583 19.94 MB/sec execute 182 sec latency 154.332 ms 2024-05-08T23:22:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:42 smithi046 ceph-mon[36898]: pgmap v593: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.3 MiB/s wr, 428 op/s 2024-05-08T23:22:42.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 117068 20.01 MB/sec execute 183 sec latency 68.560 ms 2024-05-08T23:22:42.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:42 smithi086 ceph-mon[78610]: pgmap v593: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.3 MiB/s wr, 428 op/s 2024-05-08T23:22:42.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:42 smithi053 ceph-mon[77587]: pgmap v593: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.3 MiB/s wr, 428 op/s 2024-05-08T23:22:43.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 117513 20.04 MB/sec execute 184 sec latency 89.584 ms 2024-05-08T23:22:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:44 smithi046 ceph-mon[36898]: pgmap v594: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-08T23:22:44.278 INFO:tasks.workunit.client.0.smithi046.stdout: 10 117868 20.03 MB/sec execute 185 sec latency 178.916 ms 2024-05-08T23:22:44.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:44 smithi086 ceph-mon[78610]: pgmap v594: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-08T23:22:44.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:44 smithi053 ceph-mon[77587]: pgmap v594: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-08T23:22:45.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 118222 20.00 MB/sec execute 186 sec latency 180.368 ms 2024-05-08T23:22:45.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:45 smithi086 ceph-mon[78610]: pgmap v595: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-05-08T23:22:45.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:45 smithi053 ceph-mon[77587]: pgmap v595: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-05-08T23:22:45.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:45 smithi046 ceph-mon[36898]: pgmap v595: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-05-08T23:22:46.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 118633 20.00 MB/sec execute 187 sec latency 235.311 ms 2024-05-08T23:22:47.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 119142 20.03 MB/sec execute 188 sec latency 180.430 ms 2024-05-08T23:22:47.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:47 smithi053 ceph-mon[77587]: pgmap v596: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 347 op/s 2024-05-08T23:22:47.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:47 smithi046 ceph-mon[36898]: pgmap v596: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 347 op/s 2024-05-08T23:22:47.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:47 smithi086 ceph-mon[78610]: pgmap v596: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 347 op/s 2024-05-08T23:22:48.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 119621 20.09 MB/sec execute 189 sec latency 165.304 ms 2024-05-08T23:22:49.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 120012 20.10 MB/sec execute 190 sec latency 161.438 ms 2024-05-08T23:22:49.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:49 smithi053 ceph-mon[77587]: pgmap v597: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-05-08T23:22:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:49 smithi046 ceph-mon[36898]: pgmap v597: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-05-08T23:22:49.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:49 smithi086 ceph-mon[78610]: pgmap v597: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-05-08T23:22:50.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 120396 20.10 MB/sec execute 191 sec latency 228.086 ms 2024-05-08T23:22:51.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 120813 20.10 MB/sec execute 192 sec latency 192.903 ms 2024-05-08T23:22:51.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:51 smithi053 ceph-mon[77587]: pgmap v598: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-05-08T23:22:51.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:51 smithi046 ceph-mon[36898]: pgmap v598: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-05-08T23:22:51.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:51 smithi086 ceph-mon[78610]: pgmap v598: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-05-08T23:22:52.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121190 20.09 MB/sec execute 193 sec latency 200.476 ms 2024-05-08T23:22:53.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121650 20.15 MB/sec execute 194 sec latency 198.854 ms 2024-05-08T23:22:53.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:53 smithi053 ceph-mon[77587]: pgmap v599: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:22:53.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:53 smithi046 ceph-mon[36898]: pgmap v599: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:22:53.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:22:53.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:53 smithi086 ceph-mon[78610]: pgmap v599: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-08T23:22:54.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121826 20.08 MB/sec execute 195 sec latency 677.950 ms 2024-05-08T23:22:55.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121853 19.99 MB/sec execute 196 sec latency 1577.725 ms 2024-05-08T23:22:55.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:55 smithi053 ceph-mon[77587]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 337 op/s 2024-05-08T23:22:55.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:55 smithi046 ceph-mon[36898]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 337 op/s 2024-05-08T23:22:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.790+0000 7fefab400700 1 -- 172.21.15.46:0/2306739008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 msgr2=0x7fefa41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.790+0000 7fefab400700 1 --2- 172.21.15.46:0/2306739008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa41140d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b0a0 tx=0x7fef9c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 -- 172.21.15.46:0/2306739008 shutdown_connections 2024-05-08T23:22:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 --2- 172.21.15.46:0/2306739008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa41140d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 --2- 172.21.15.46:0/2306739008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa40754d0 0x7fefa40758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 --2- 172.21.15.46:0/2306739008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa4075df0 0x7fefa41117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 -- 172.21.15.46:0/2306739008 >> 172.21.15.46:0/2306739008 conn(0x7fefa40feb10 msgr2=0x7fefa4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 -- 172.21.15.46:0/2306739008 shutdown_connections 2024-05-08T23:22:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.791+0000 7fefab400700 1 -- 172.21.15.46:0/2306739008 wait complete. 2024-05-08T23:22:55.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.793+0000 7fefab400700 1 Processor -- start 2024-05-08T23:22:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.793+0000 7fefab400700 1 -- start start 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 0x7fefa4111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 0x7fefa41115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410a9e0 con 0x7fefa40754d0 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410ab50 con 0x7fefa4111ce0 2024-05-08T23:22:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefab400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410ad10 con 0x7fefa4075df0 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.794+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 0x7fefa4111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 0x7fefa4111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37594/0 (socket says 172.21.15.46:37594) 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa8a00700 1 -- 172.21.15.46:0/1677650356 learned_addr learned my addr 172.21.15.46:0/1677650356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa9400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa410a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 0x7fefa41115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa9400700 1 -- 172.21.15.46:0/1677650356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 msgr2=0x7fefa41115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.795+0000 7fefa9400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 0x7fefa41115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.796+0000 7fefa9400700 1 -- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 msgr2=0x7fefa4111070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.796+0000 7fefa9400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 0x7fefa4111070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.796+0000 7fefa9400700 1 -- 172.21.15.46:0/1677650356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009d00 con 0x7fefa4111ce0 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.796+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 0x7fefa41115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:55.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.796+0000 7fefa9400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa410a210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fefa4068a90 tx=0x7fef9c007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.797+0000 7fefa1600700 1 -- 172.21.15.46:0/1677650356 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00e040 con 0x7fefa4111ce0 2024-05-08T23:22:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.797+0000 7fefa1600700 1 -- 172.21.15.46:0/1677650356 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9c01dd50 con 0x7fefa4111ce0 2024-05-08T23:22:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.797+0000 7fefa1600700 1 -- 172.21.15.46:0/1677650356 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c01b450 con 0x7fefa4111ce0 2024-05-08T23:22:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.797+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa410af90 con 0x7fefa4111ce0 2024-05-08T23:22:55.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.797+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa410ee40 con 0x7fefa4111ce0 2024-05-08T23:22:55.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.803+0000 7fefbefacb80 1 Processor -- start 2024-05-08T23:22:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.805+0000 7fefa1600700 1 -- 172.21.15.46:0/1677650356 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef9c01b5b0 con 0x7fefa4111ce0 2024-05-08T23:22:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.806+0000 7fefbefacb80 1 -- start start 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43555350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43555890 0x560d43555cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d434ac150 con 0x560d43555890 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d434a9b10 con 0x560d4355ae00 2024-05-08T23:22:55.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefbefacb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d434a5400 con 0x560d4355b720 2024-05-08T23:22:55.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefa3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefa3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55802/0 (socket says 172.21.15.46:55802) 2024-05-08T23:22:55.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefa3e00700 1 -- 172.21.15.46:0/1634540277 learned_addr learned my addr 172.21.15.46:0/1634540277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:55.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.808+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43555890 0x560d43555cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa3e00700 1 -- 172.21.15.46:0/1634540277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 msgr2=0x560d43555350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa8a00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43555350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43555350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa3e00700 1 -- 172.21.15.46:0/1634540277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43555890 msgr2=0x560d43555cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43555890 0x560d43555cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa3e00700 1 -- 172.21.15.46:0/1634540277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d433263b0 con 0x560d4355ae00 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.809+0000 7fefa8a00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43555350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.810+0000 7fefa3e00700 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef90013910 tx=0x7fef9000b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5713c0c4475ba638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.810+0000 7fefa1600700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 0x7fef8c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.810+0000 7fef95c00700 1 -- 172.21.15.46:0/1634540277 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004220 con 0x560d4355ae00 2024-05-08T23:22:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.810+0000 7fef95c00700 1 -- 172.21.15.46:0/1634540277 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef90019030 con 0x560d4355ae00 2024-05-08T23:22:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.811+0000 7fef95c00700 1 -- 172.21.15.46:0/1634540277 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004220 con 0x560d4355ae00 2024-05-08T23:22:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.811+0000 7fefa8a00700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 0x7fef8c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.811+0000 7fefa1600700 1 -- 172.21.15.46:0/1677650356 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fef9c0cee10 con 0x7fefa4111ce0 2024-05-08T23:22:55.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.813+0000 7fefbefacb80 1 -- 172.21.15.46:0/1634540277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 msgr2=0x560d4355b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.813+0000 7fefbefacb80 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef90013910 tx=0x7fef9000b4c0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefa8a00700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 0x7fef8c0a38e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x560d43551bd0 tx=0x7fef98008040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 -- 172.21.15.46:0/1634540277 shutdown_connections 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43555890 0x560d43555cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355ae00 0x560d4355b1e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 --2- 172.21.15.46:0/1634540277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43555350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 -- 172.21.15.46:0/1634540277 >> 172.21.15.46:0/1634540277 conn(0x560d434b3210 msgr2=0x560d4354fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 -- 172.21.15.46:0/1634540277 shutdown_connections 2024-05-08T23:22:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.814+0000 7fefbefacb80 1 -- 172.21.15.46:0/1634540277 wait complete. 2024-05-08T23:22:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.815+0000 7fefbefacb80 1 -- start start 2024-05-08T23:22:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.815+0000 7fefbefacb80 1 Processor -- start 2024-05-08T23:22:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.816+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 0x560d4355ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.816+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.817+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 0x560d4355f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.817+0000 7fefa3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefa3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37618/0 (socket says 172.21.15.46:37618) 2024-05-08T23:22:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 0x560d4355ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefbefacb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43441560 con 0x560d4355ae00 2024-05-08T23:22:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefbefacb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43441200 con 0x560d4355b720 2024-05-08T23:22:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefa3e00700 1 -- 172.21.15.46:0/712288797 learned_addr learned my addr 172.21.15.46:0/712288797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefa9400700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 0x560d4355f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d433286a0 con 0x560d43555890 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 0x560d4355ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41806/0 (socket says 172.21.15.46:41806) 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.818+0000 7fefbefacb80 10 client.0 _ll_register_callbacks cb 0x560d4344ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa3e00700 1 -- 172.21.15.46:0/712288797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 msgr2=0x560d4355ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa3e00700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 0x560d4355ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa3e00700 1 -- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 msgr2=0x560d4355f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa3e00700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 0x560d4355f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa3e00700 1 -- 172.21.15.46:0/712288797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900135f0 con 0x560d4355ae00 2024-05-08T23:22:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.819+0000 7fefa9400700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 0x560d4355f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.820+0000 7fefa3e00700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fef900038b0 tx=0x7fef90003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f9ffd1be56ef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.820+0000 7fef8a000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:22:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.821+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90024070 con 0x560d4355ae00 2024-05-08T23:22:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.821+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fef90003e60 con 0x560d4355ae00 2024-05-08T23:22:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.821+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d433fe270 con 0x560d4355ae00 2024-05-08T23:22:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.823+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef90004500 con 0x560d4355ae00 2024-05-08T23:22:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.825+0000 7fefbefacb80 10 client.20220 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.825+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560d432d2b60 con 0x560d4355ae00 2024-05-08T23:22:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.825+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d433fe270 con 0x560d4355ae00 2024-05-08T23:22:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.825+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef90004990 con 0x560d4355ae00 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 10 client.20220 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 10 client.20220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 20 client.20220 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d4356f1f0 0x560d43571610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 4 client.20220 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d43524810 con 0x560d4356f1f0 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.826+0000 7fefa8a00700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d4356f1f0 0x560d43571610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.827+0000 7fefa8a00700 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d4356f1f0 0x560d43571610 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c089279bd708cbd server_cookie=92be284c39992dae in_seq=0 out_seq=0 2024-05-08T23:22:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.827+0000 7fef8a000700 10 client.20220 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.829+0000 7fef8a000700 1 -- 172.21.15.46:0/712288797 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d43524810 con 0x560d4356f1f0 2024-05-08T23:22:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.829+0000 7fef8a000700 10 client.20220 handle_command_reply: tid=0 2024-05-08T23:22:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.829+0000 7fefbefacb80 1 client.20220 shutdown 2024-05-08T23:22:55.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.830+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 msgr2=0x560d4355f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.830+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fef900038b0 tx=0x7fef90003990 comp rx=0 tx=0).stop 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 shutdown_connections 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d4355b720 0x560d4355f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43555890 0x560d4355ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d4356f1f0 0x560d43571610 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 --2- 172.21.15.46:0/712288797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355ae00 0x560d4355f130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 >> 172.21.15.46:0/712288797 conn(0x560d434b3210 msgr2=0x560d4354f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fef8a000700 0 client.20220 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 shutdown_connections 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 1 -- 172.21.15.46:0/712288797 wait complete. 2024-05-08T23:22:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.831+0000 7fefbefacb80 20 client.20220 trim_cache size 0 max 16384 2024-05-08T23:22:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.863+0000 7fefbefacb80 1 Processor -- start 2024-05-08T23:22:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.863+0000 7fefbefacb80 1 -- start start 2024-05-08T23:22:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.863+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d4354c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 0x560d4354fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43561430 0x560d4355d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefbefacb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43561bc0 con 0x560d4355b720 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefbefacb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d4354cdc0 con 0x560d43555890 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefbefacb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d435767e0 con 0x560d43561430 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d4354c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d4354c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55824/0 (socket says 172.21.15.46:55824) 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefa8a00700 1 -- 172.21.15.46:0/3573112919 learned_addr learned my addr 172.21.15.46:0/3573112919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.864+0000 7fefa3e00700 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 0x560d4354fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.865+0000 7fefa3e00700 1 -- 172.21.15.46:0/3573112919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43561430 msgr2=0x560d4355d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.865+0000 7fefa3e00700 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43561430 0x560d4355d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.865+0000 7fefa3e00700 1 -- 172.21.15.46:0/3573112919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 msgr2=0x560d4354c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.865+0000 7fefa3e00700 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d4354c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.865+0000 7fefa3e00700 1 -- 172.21.15.46:0/3573112919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d4346b560 con 0x560d4355b720 2024-05-08T23:22:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.866+0000 7fefa3e00700 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 0x560d4354fd20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x560d4355fc80 tx=0x7fef90003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851e3b78732e2e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.866+0000 7fef96600700 1 -- 172.21.15.46:0/3573112919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90024040 con 0x560d4355b720 2024-05-08T23:22:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.867+0000 7fef96600700 1 -- 172.21.15.46:0/3573112919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9002f040 con 0x560d4355b720 2024-05-08T23:22:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.867+0000 7fef96600700 1 -- 172.21.15.46:0/3573112919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9001e4d0 con 0x560d4355b720 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 -- 172.21.15.46:0/3573112919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 msgr2=0x560d4354fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 0x560d4354fd20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x560d4355fc80 tx=0x7fef90003ab0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 -- 172.21.15.46:0/3573112919 shutdown_connections 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d43561430 0x560d4355d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d4354c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 --2- 172.21.15.46:0/3573112919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d4355b720 0x560d4354fd20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.868+0000 7fefbefacb80 1 -- 172.21.15.46:0/3573112919 >> 172.21.15.46:0/3573112919 conn(0x560d434b3210 msgr2=0x560d4355e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.870+0000 7fefbefacb80 1 -- 172.21.15.46:0/3573112919 shutdown_connections 2024-05-08T23:22:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.871+0000 7fefbefacb80 1 -- 172.21.15.46:0/3573112919 wait complete. 2024-05-08T23:22:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.872+0000 7fefbefacb80 1 -- start start 2024-05-08T23:22:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.872+0000 7fefbefacb80 1 Processor -- start 2024-05-08T23:22:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.873+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43566450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43561430 0x560d43564550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43564c40 con 0x560d43561430 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43564db0 con 0x560d43555890 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d43564f70 con 0x560d4355b720 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefa8a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55838/0 (socket says 172.21.15.46:55838) 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefa8a00700 1 -- 172.21.15.46:0/4056639603 learned_addr learned my addr 172.21.15.46:0/4056639603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefbefacb80 10 client.0 _ll_register_callbacks cb 0x560d4344ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.874+0000 7fefa3e00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43566450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa9400700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43561430 0x560d43564550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 -- 172.21.15.46:0/4056639603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 msgr2=0x560d43566450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43566450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 -- 172.21.15.46:0/4056639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43561430 msgr2=0x560d43564550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43561430 0x560d43564550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 -- 172.21.15.46:0/4056639603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d432d2b60 con 0x560d43555890 2024-05-08T23:22:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fefa8a00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef98010fd0 tx=0x7fef980047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee89f9fb8eeceac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.875+0000 7fef8a000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.876+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98015880 con 0x560d43555890 2024-05-08T23:22:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.876+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef90019030 con 0x560d43555890 2024-05-08T23:22:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.876+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fef980159e0 con 0x560d43555890 2024-05-08T23:22:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.876+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef98027900 con 0x560d43555890 2024-05-08T23:22:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.877+0000 7fefbefacb80 10 client.30437 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.877+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560d4332acd0 con 0x560d43555890 2024-05-08T23:22:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.877+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef90019030 con 0x560d43555890 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.877+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef98024ce0 con 0x560d43555890 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.877+0000 7fefbefacb80 10 client.30437 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefbefacb80 10 client.30437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefbefacb80 20 client.30437 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d435c6a40 0x560d435c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefbefacb80 4 client.30437 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d4349dec0 con 0x560d435c6a40 2024-05-08T23:22:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.878+0000 7fefa3e00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d435c6a40 0x560d435c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.879+0000 7fefa3e00700 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d435c6a40 0x560d435c8e60 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f879cf8e28929bf server_cookie=26571c11a88bbc2f in_seq=0 out_seq=0 2024-05-08T23:22:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.879+0000 7fef8a000700 10 client.30437 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fef8a000700 1 -- 172.21.15.46:0/4056639603 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d4349dec0 con 0x560d435c6a40 2024-05-08T23:22:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fef8a000700 10 client.30437 handle_command_reply: tid=0 2024-05-08T23:22:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 client.30437 shutdown 2024-05-08T23:22:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 msgr2=0x560d43565f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef98010fd0 tx=0x7fef980047e0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 shutdown_connections 2024-05-08T23:22:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x560d435c6a40 0x560d435c8e60 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560d43561430 0x560d43564550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fef8a000700 0 client.30437 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d43555890 0x560d43565f10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 --2- 172.21.15.46:0/4056639603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d4355b720 0x560d43566450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.880+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 >> 172.21.15.46:0/4056639603 conn(0x560d434b3210 msgr2=0x560d4355d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.882+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 shutdown_connections 2024-05-08T23:22:55.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.883+0000 7fefbefacb80 1 -- 172.21.15.46:0/4056639603 wait complete. 2024-05-08T23:22:55.885 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:22:55.885 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:22:55.885 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:22:55.885 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:22:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.884+0000 7fefbefacb80 20 client.30437 trim_cache size 0 max 16384 2024-05-08T23:22:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 msgr2=0x7fef8c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 0x7fef8c0a38e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x560d43551bd0 tx=0x7fef98008040 comp rx=0 tx=0).stop 2024-05-08T23:22:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 msgr2=0x7fefa410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa410a210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fefa4068a90 tx=0x7fef9c007b60 comp rx=0 tx=0).stop 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 shutdown_connections 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefa4111ce0 0x7fefa410a210 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fef8c0a1420 0x7fef8c0a38e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa40754d0 0x7fefa4111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 --2- 172.21.15.46:0/1677650356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefa4075df0 0x7fefa41115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.885+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 >> 172.21.15.46:0/1677650356 conn(0x7fefa40feb10 msgr2=0x7fefa4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.886+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 shutdown_connections 2024-05-08T23:22:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:55.886+0000 7fefab400700 1 -- 172.21.15.46:0/1677650356 wait complete. 2024-05-08T23:22:55.920 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:22:55.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:55.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:55 smithi086 ceph-mon[78610]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 337 op/s 2024-05-08T23:22:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.276+0000 7f0683200700 1 -- 172.21.15.46:0/3435852490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684105f90 msgr2=0x7f068410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.276+0000 7f0683200700 1 --2- 172.21.15.46:0/3435852490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684105f90 0x7f068410db30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b0a0 tx=0x7f067c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.278+0000 7f0683200700 1 -- 172.21.15.46:0/3435852490 shutdown_connections 2024-05-08T23:22:56.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121981 19.91 MB/sec execute 197 sec latency 1769.499 ms 2024-05-08T23:22:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.278+0000 7f0683200700 1 --2- 172.21.15.46:0/3435852490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684105f90 0x7f068410db30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.278+0000 7f0683200700 1 --2- 172.21.15.46:0/3435852490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840683d0 0x7f06840687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.278+0000 7f0683200700 1 --2- 172.21.15.46:0/3435852490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0684068d80 0x7f0684105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.278+0000 7f0683200700 1 -- 172.21.15.46:0/3435852490 >> 172.21.15.46:0/3435852490 conn(0x7f0684075690 msgr2=0x7f0684075aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.279+0000 7f0683200700 1 -- 172.21.15.46:0/3435852490 shutdown_connections 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.279+0000 7f0683200700 1 -- 172.21.15.46:0/3435852490 wait complete. 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.281+0000 7f0683200700 1 Processor -- start 2024-05-08T23:22:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.281+0000 7f0683200700 1 -- start start 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.281+0000 7f0683200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.281+0000 7f0683200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0684105f90 0x7f068410d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.282+0000 7f0683200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06841a1e50 0x7f06841a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.282+0000 7f0683200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a28a0 con 0x7f06841a1e50 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.282+0000 7f0683200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a2a10 con 0x7f06840683d0 2024-05-08T23:22:56.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.282+0000 7f0683200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a2bd0 con 0x7f0684105f90 2024-05-08T23:22:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.282+0000 7f0681e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55870/0 (socket says 172.21.15.46:55870) 2024-05-08T23:22:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 -- 172.21.15.46:0/831027529 learned_addr learned my addr 172.21.15.46:0/831027529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0682800700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06841a1e50 0x7f06841a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 -- 172.21.15.46:0/831027529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0684105f90 msgr2=0x7f068410d170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:22:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0684105f90 0x7f068410d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 -- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06841a1e50 msgr2=0x7f06841a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06841a1e50 0x7f06841a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.283+0000 7f0681e00700 1 -- 172.21.15.46:0/831027529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009d00 con 0x7f06840683d0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.284+0000 7f0681e00700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f067400c9b0 tx=0x7f067400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.284+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674007b00 con 0x7f06840683d0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.284+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0674007c60 con 0x7f06840683d0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.284+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674018660 con 0x7f06840683d0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.284+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06841aa100 con 0x7f06840683d0 2024-05-08T23:22:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.285+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06841aa5c0 con 0x7f06840683d0 2024-05-08T23:22:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.288+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0684101400 con 0x7f06840683d0 2024-05-08T23:22:56.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.290+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0674018880 con 0x7f06840683d0 2024-05-08T23:22:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.292+0000 7f067aa00700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 0x7f06680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:56.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.292+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f06740cd270 con 0x7f06840683d0 2024-05-08T23:22:56.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.295+0000 7f0681400700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 0x7f06680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:56.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.297+0000 7f0681400700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 0x7f06680a3750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f068410df50 tx=0x7f067000b370 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:56.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.298+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f06740962b0 con 0x7f06840683d0 2024-05-08T23:22:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.854+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0684102340 con 0x7f06840683d0 2024-05-08T23:22:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.856+0000 7f067aa00700 1 -- 172.21.15.46:0/831027529 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f06740998d0 con 0x7f06840683d0 2024-05-08T23:22:56.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:22:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 msgr2=0x7f06680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 0x7f06680a3750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f068410df50 tx=0x7f067000b370 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 msgr2=0x7f068410cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f067400c9b0 tx=0x7f067400cd70 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 shutdown_connections 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f06680a1290 0x7f06680a3750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06841a1e50 0x7f06841a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06840683d0 0x7f068410cc30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 --2- 172.21.15.46:0/831027529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0684105f90 0x7f068410d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 >> 172.21.15.46:0/831027529 conn(0x7f0684075690 msgr2=0x7f0684104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:56.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 shutdown_connections 2024-05-08T23:22:56.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:56.862+0000 7f0683200700 1 -- 172.21.15.46:0/831027529 wait complete. 2024-05-08T23:22:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:22:56.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:22:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 -- 172.21.15.46:0/1529178934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d54107de0 msgr2=0x7f5d54108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 --2- 172.21.15.46:0/1529178934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d54107de0 0x7f5d54108260 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009990 tx=0x7f5d44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 -- 172.21.15.46:0/1529178934 shutdown_connections 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 --2- 172.21.15.46:0/1529178934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d541087a0 0x7f5d54114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 --2- 172.21.15.46:0/1529178934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d5410d4f0 0x7f5d5410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 --2- 172.21.15.46:0/1529178934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d54107de0 0x7f5d54108260 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.253+0000 7f5d59800700 1 -- 172.21.15.46:0/1529178934 >> 172.21.15.46:0/1529178934 conn(0x7f5d540769f0 msgr2=0x7f5d54078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.254+0000 7f5d59800700 1 -- 172.21.15.46:0/1529178934 shutdown_connections 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.254+0000 7f5d59800700 1 -- 172.21.15.46:0/1529178934 wait complete. 2024-05-08T23:22:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.255+0000 7f5d59800700 1 Processor -- start 2024-05-08T23:22:57.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.257+0000 7f5d59800700 1 -- start start 2024-05-08T23:22:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.257+0000 7f5d59800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54107de0 0x7f5d541a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.257+0000 7f5d59800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d59800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d5410d4f0 0x7f5d5419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d59800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419dea0 con 0x7f5d5410d4f0 2024-05-08T23:22:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d59800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419e010 con 0x7f5d54107de0 2024-05-08T23:22:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d59800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419e1d0 con 0x7f5d541087a0 2024-05-08T23:22:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41860/0 (socket says 172.21.15.46:41860) 2024-05-08T23:22:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 -- 172.21.15.46:0/1305061918 learned_addr learned my addr 172.21.15.46:0/1305061918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 -- 172.21.15.46:0/1305061918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54107de0 msgr2=0x7f5d541a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54107de0 0x7f5d541a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 -- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d5410d4f0 msgr2=0x7f5d5419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d5410d4f0 0x7f5d5419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.258+0000 7f5d52000700 1 -- 172.21.15.46:0/1305061918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440095f0 con 0x7f5d541087a0 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d52000700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005740 tx=0x7f5d4400bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d4b400700 1 -- 172.21.15.46:0/1305061918 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4401d070 con 0x7f5d541087a0 2024-05-08T23:22:57.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d4b400700 1 -- 172.21.15.46:0/1305061918 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d44021410 con 0x7f5d541087a0 2024-05-08T23:22:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d4b400700 1 -- 172.21.15.46:0/1305061918 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44017650 con 0x7f5d541087a0 2024-05-08T23:22:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5419e450 con 0x7f5d541087a0 2024-05-08T23:22:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.259+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541aa570 con 0x7f5d541087a0 2024-05-08T23:22:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.264+0000 7f5d4b400700 1 -- 172.21.15.46:0/1305061918 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5d440177b0 con 0x7f5d541087a0 2024-05-08T23:22:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.266+0000 7f5d4b400700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 0x7f5d380a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.266+0000 7f5d6d3f5b80 1 Processor -- start 2024-05-08T23:22:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.266+0000 7f5d4b400700 1 -- 172.21.15.46:0/1305061918 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5d440cf400 con 0x7f5d541087a0 2024-05-08T23:22:57.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.268+0000 7f5d6d3f5b80 1 -- start start 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.268+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.268+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.268+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f4fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41864/0 (socket says 172.21.15.46:41864) 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d53400700 1 -- 172.21.15.46:0/3015126823 learned_addr learned my addr 172.21.15.46:0/3015126823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786ea6150 con 0x558786f54e00 2024-05-08T23:22:57.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786ea3b10 con 0x558786f55720 2024-05-08T23:22:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786e9f400 con 0x558786f4f890 2024-05-08T23:22:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.269+0000 7f5d52a00700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 0x7f5d380a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.270+0000 7f5d52a00700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 0x7f5d380a3960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0094f0 tx=0x7f5d3c009460 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.270+0000 7f5d52000700 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.271+0000 7f5d52000700 1 -- 172.21.15.46:0/3015126823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 msgr2=0x558786f4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.271+0000 7f5d52000700 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.271+0000 7f5d52000700 1 -- 172.21.15.46:0/3015126823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 msgr2=0x558786f551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.271+0000 7f5d52000700 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.271+0000 7f5d52000700 1 -- 172.21.15.46:0/3015126823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558786d203b0 con 0x558786f55720 2024-05-08T23:22:57.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.274+0000 7f5d52000700 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f4f350 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5d440cc6e0 tx=0x7f5d44005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501bbd3859d9f49a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.275+0000 7f5d43e00700 1 -- 172.21.15.46:0/3015126823 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4409ff60 con 0x558786f55720 2024-05-08T23:22:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.278+0000 7f5d43e00700 1 -- 172.21.15.46:0/3015126823 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4409e070 con 0x558786f55720 2024-05-08T23:22:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.278+0000 7f5d43e00700 1 -- 172.21.15.46:0/3015126823 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4409c390 con 0x558786f55720 2024-05-08T23:22:57.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122286 19.88 MB/sec execute 198 sec latency 182.398 ms 2024-05-08T23:22:57.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.279+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 msgr2=0x558786f4f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.279+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f4f350 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5d440cc6e0 tx=0x7f5d44005c70 comp rx=0 tx=0).stop 2024-05-08T23:22:57.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.280+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 shutdown_connections 2024-05-08T23:22:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.280+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.280+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.280+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3015126823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f4f350 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.280+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 >> 172.21.15.46:0/3015126823 conn(0x558786ead210 msgr2=0x558786f49df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.281+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 shutdown_connections 2024-05-08T23:22:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.281+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3015126823 wait complete. 2024-05-08T23:22:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.282+0000 7f5d6d3f5b80 1 -- start start 2024-05-08T23:22:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.283+0000 7f5d6d3f5b80 1 Processor -- start 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.285+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.285+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.285+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.285+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786e3b560 con 0x558786f55720 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.285+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786e3b200 con 0x558786f4f890 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.286+0000 7f5d53400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.286+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.287+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41866/0 (socket says 172.21.15.46:41866) 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.287+0000 7f5d52000700 1 -- 172.21.15.46:0/2186503401 learned_addr learned my addr 172.21.15.46:0/2186503401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.287+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786d226a0 con 0x558786f54e00 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.286+0000 7f5d52a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.287+0000 7f5d6d3f5b80 10 client.0 _ll_register_callbacks cb 0x558786e44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 -- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 msgr2=0x558786f58a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f58a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 -- 172.21.15.46:0/2186503401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 msgr2=0x558786f59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 -- 172.21.15.46:0/2186503401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541aa570 con 0x558786f4f890 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d52a00700 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5b790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009750 tx=0x7f5d3c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42675d7a96b7ff9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.288+0000 7f5d40c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004830 con 0x558786f4f890 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5d3c004990 con 0x558786f4f890 2024-05-08T23:22:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558786df8270 con 0x558786f4f890 2024-05-08T23:22:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3c004b50 con 0x558786f4f890 2024-05-08T23:22:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d6d3f5b80 10 client.30452 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.289+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558786cccb60 con 0x558786f4f890 2024-05-08T23:22:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558786df8270 con 0x558786f4f890 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d3c005280 con 0x558786f4f890 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 10 client.30452 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 10 client.30452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 20 client.30452 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786f79e30 0x558786f7c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 4 client.30452 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:57.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558786f1e810 con 0x558786f79e30 2024-05-08T23:22:57.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.290+0000 7f5d52000700 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786f79e30 0x558786f7c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.292+0000 7f5d52000700 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786f79e30 0x558786f7c250 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dbb9024b2faabe32 server_cookie=854817896ae47e81 in_seq=0 out_seq=0 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.292+0000 7f5d40c00700 10 client.30452 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d40c00700 1 -- 172.21.15.46:0/2186503401 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558786f1e810 con 0x558786f79e30 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d40c00700 10 client.30452 handle_command_reply: tid=0 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d6d3f5b80 1 client.30452 shutdown 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 msgr2=0x558786f5b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5b790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009750 tx=0x7f5d3c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 shutdown_connections 2024-05-08T23:22:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.293+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786f79e30 0x558786f7c250 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5b790 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/2186503401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 >> 172.21.15.46:0/2186503401 conn(0x558786ead210 msgr2=0x558786f495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d40c00700 0 client.30452 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 shutdown_connections 2024-05-08T23:22:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/2186503401 wait complete. 2024-05-08T23:22:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.294+0000 7f5d6d3f5b80 20 client.30452 trim_cache size 0 max 16384 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.329+0000 7f5d6d3f5b80 1 Processor -- start 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.329+0000 7f5d6d3f5b80 1 -- start start 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f49d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f5b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f5a080 con 0x558786f54e00 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f599e0 con 0x558786f55720 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f46df0 con 0x558786f4f890 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.330+0000 7f5d52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41878/0 (socket says 172.21.15.46:41878) 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 -- 172.21.15.46:0/3277757927 learned_addr learned my addr 172.21.15.46:0/3277757927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d53400700 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f5b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52000700 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f49d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 -- 172.21.15.46:0/3277757927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 msgr2=0x558786f5b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f5b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 -- 172.21.15.46:0/3277757927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 msgr2=0x558786f49d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f49d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.331+0000 7f5d52a00700 1 -- 172.21.15.46:0/3277757927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558786f4ff40 con 0x558786f4f890 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.332+0000 7f5d52a00700 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009410 tx=0x7f5d3c01cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54ae6ee1bd54189 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.332+0000 7f5d48c00700 1 -- 172.21.15.46:0/3277757927 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c012960 con 0x558786f4f890 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.332+0000 7f5d48c00700 1 -- 172.21.15.46:0/3277757927 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d3c005760 con 0x558786f4f890 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.332+0000 7f5d48c00700 1 -- 172.21.15.46:0/3277757927 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c051450 con 0x558786f4f890 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3277757927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 msgr2=0x558786f589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009410 tx=0x7f5d3c01cf80 comp rx=0 tx=0).stop 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3277757927 shutdown_connections 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f55720 0x558786f5b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f4f890 0x558786f589b0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/3277757927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f54e00 0x558786f49d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.333+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3277757927 >> 172.21.15.46:0/3277757927 conn(0x558786ead210 msgr2=0x558786f582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.335+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3277757927 shutdown_connections 2024-05-08T23:22:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.335+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/3277757927 wait complete. 2024-05-08T23:22:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.335+0000 7f5d6d3f5b80 1 -- start start 2024-05-08T23:22:57.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.335+0000 7f5d6d3f5b80 1 Processor -- start 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f50d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f51470 con 0x558786f55720 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f515e0 con 0x558786f4f890 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558786f517a0 con 0x558786f54e00 2024-05-08T23:22:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d6d3f5b80 10 client.0 _ll_register_callbacks cb 0x558786e44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.336+0000 7f5d52a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41882/0 (socket says 172.21.15.46:41882) 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52000700 1 -- 172.21.15.46:0/4143966319 learned_addr learned my addr 172.21.15.46:0/4143966319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 -- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 msgr2=0x558786f60420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 -- 172.21.15.46:0/4143966319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 msgr2=0x558786f50d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f50d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 -- 172.21.15.46:0/4143966319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558786cccb60 con 0x558786f4f890 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52000700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.337+0000 7f5d52a00700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5fee0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c015040 tx=0x7f5d3c01d7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=710bbd9aaf6c9a5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.338+0000 7f5d40c00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.338+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c012b70 con 0x558786f4f890 2024-05-08T23:22:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.338+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f5d3c012cd0 con 0x558786f4f890 2024-05-08T23:22:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.338+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558786e97ec0 con 0x558786f4f890 2024-05-08T23:22:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.340+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3c012e90 con 0x558786f4f890 2024-05-08T23:22:57.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.340+0000 7f5d6d3f5b80 10 client.30458 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:57.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.340+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558786e65560 con 0x558786f4f890 2024-05-08T23:22:57.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558786e97ec0 con 0x558786f4f890 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d3c05b440 con 0x558786f4f890 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 10 client.30458 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 10 client.30458 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 20 client.30458 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786fd10c0 0x558786fd34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 4 client.30458 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558786cedcf0 con 0x558786fd10c0 2024-05-08T23:22:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.341+0000 7f5d52000700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786fd10c0 0x558786fd34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.343+0000 7f5d52000700 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786fd10c0 0x558786fd34b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8081989c76904073 server_cookie=846f053797c53bee in_seq=0 out_seq=0 2024-05-08T23:22:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.343+0000 7f5d40c00700 10 client.30458 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.343+0000 7f5d40c00700 1 -- 172.21.15.46:0/4143966319 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558786cedcf0 con 0x558786fd10c0 2024-05-08T23:22:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.343+0000 7f5d40c00700 10 client.30458 handle_command_reply: tid=0 2024-05-08T23:22:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.343+0000 7f5d6d3f5b80 1 client.30458 shutdown 2024-05-08T23:22:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.344+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 msgr2=0x558786f5fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.344+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5fee0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c015040 tx=0x7f5d3c01d7d0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 shutdown_connections 2024-05-08T23:22:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x558786fd10c0 0x558786fd34b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558786f55720 0x558786f50d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558786f4f890 0x558786f5fee0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 --2- 172.21.15.46:0/4143966319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558786f54e00 0x558786f60420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d40c00700 0 client.30458 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.345+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 >> 172.21.15.46:0/4143966319 conn(0x558786ead210 msgr2=0x558786f57300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.346+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 shutdown_connections 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.346+0000 7f5d6d3f5b80 1 -- 172.21.15.46:0/4143966319 wait complete. 2024-05-08T23:22:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.346+0000 7f5d6d3f5b80 20 client.30458 trim_cache size 0 max 16384 2024-05-08T23:22:57.347 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 msgr2=0x7f5d380a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 0x7f5d380a3960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0094f0 tx=0x7f5d3c009460 comp rx=0 tx=0).stop 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 msgr2=0x7f5d541a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005740 tx=0x7f5d4400bfc0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 shutdown_connections 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f5d380a14a0 0x7f5d380a3960 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d5410d4f0 0x7f5d5419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d54107de0 0x7f5d541a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 --2- 172.21.15.46:0/1305061918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541087a0 0x7f5d541a3bf0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.347+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 >> 172.21.15.46:0/1305061918 conn(0x7f5d540769f0 msgr2=0x7f5d540790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.348+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 shutdown_connections 2024-05-08T23:22:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.348+0000 7f5d59800700 1 -- 172.21.15.46:0/1305061918 wait complete. 2024-05-08T23:22:57.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 -- 172.21.15.46:0/2029656816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c011a780 msgr2=0x7f72c011cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 --2- 172.21.15.46:0/2029656816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c011a780 0x7f72c011cb70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f72b800b0a0 tx=0x7f72b800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 -- 172.21.15.46:0/2029656816 shutdown_connections 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 --2- 172.21.15.46:0/2029656816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c011a780 0x7f72c011cb70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 --2- 172.21.15.46:0/2029656816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c010f5e0 0x7f72c010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 --2- 172.21.15.46:0/2029656816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 0x7f72c01084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.714+0000 7f72c6400700 1 -- 172.21.15.46:0/2029656816 >> 172.21.15.46:0/2029656816 conn(0x7f72c006bf30 msgr2=0x7f72c006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.715+0000 7f72c6400700 1 -- 172.21.15.46:0/2029656816 shutdown_connections 2024-05-08T23:22:57.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.715+0000 7f72c6400700 1 -- 172.21.15.46:0/2029656816 wait complete. 2024-05-08T23:22:57.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.716+0000 7f72c6400700 1 Processor -- start 2024-05-08T23:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.717+0000 7f72c6400700 1 -- start start 2024-05-08T23:22:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.717+0000 7f72c6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 0x7f72c01a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.717+0000 7f72c6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 0x7f72c01a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.717+0000 7f72c6400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 0x7f72c01aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72c6400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01ab180 con 0x7f72c010f5e0 2024-05-08T23:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72bea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 0x7f72c01a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72bea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 0x7f72c01a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37738/0 (socket says 172.21.15.46:37738) 2024-05-08T23:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72bea00700 1 -- 172.21.15.46:0/1858418044 learned_addr learned my addr 172.21.15.46:0/1858418044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 0x7f72c01aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01ab2f0 con 0x7f72c011a780 2024-05-08T23:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01ab4b0 con 0x7f72c0108030 2024-05-08T23:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.718+0000 7f72bf400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 0x7f72c01a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 -- 172.21.15.46:0/1858418044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 msgr2=0x7f72c01a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 0x7f72c01a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 -- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 msgr2=0x7f72c01a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 0x7f72c01a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 -- 172.21.15.46:0/1858418044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009d00 con 0x7f72c011a780 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 0x7f72c01aab40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f72b800b0a0 tx=0x7f72b8007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72b7e00700 1 -- 172.21.15.46:0/1858418044 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800e040 con 0x7f72c011a780 2024-05-08T23:22:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72b7e00700 1 -- 172.21.15.46:0/1858418044 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b800cec0 con 0x7f72c011a780 2024-05-08T23:22:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01ab730 con 0x7f72c011a780 2024-05-08T23:22:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.720+0000 7f72b7e00700 1 -- 172.21.15.46:0/1858418044 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b801b450 con 0x7f72c011a780 2024-05-08T23:22:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.721+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01b2fd0 con 0x7f72c011a780 2024-05-08T23:22:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.727+0000 7f72b7e00700 1 -- 172.21.15.46:0/1858418044 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f72b801d480 con 0x7f72c011a780 2024-05-08T23:22:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.728+0000 7f72b7e00700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 0x7f72a40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.728+0000 7f72b7e00700 1 -- 172.21.15.46:0/1858418044 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f72b80ce910 con 0x7f72c011a780 2024-05-08T23:22:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.730+0000 7f72bf400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 0x7f72a40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.731+0000 7f72d9ffdb80 1 Processor -- start 2024-05-08T23:22:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.731+0000 7f72d9ffdb80 1 -- start start 2024-05-08T23:22:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663c52150 con 0x55f663d11e10 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663c4fb10 con 0x55f663d0c8a0 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663c4b400 con 0x55f663d12730 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55972/0 (socket says 172.21.15.46:55972) 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.732+0000 7f72bfe00700 1 -- 172.21.15.46:0/4054130419 learned_addr learned my addr 172.21.15.46:0/4054130419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 -- 172.21.15.46:0/4054130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 msgr2=0x55f663d0c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 -- 172.21.15.46:0/4054130419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 msgr2=0x55f663d121f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 -- 172.21.15.46:0/4054130419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f663acc3b0 con 0x55f663d0c8a0 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.733+0000 7f72bfe00700 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b8097b40 tx=0x7f72b801d9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6c7ea9a62da4cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.734+0000 7f72b4c00700 1 -- 172.21.15.46:0/4054130419 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b809bd40 con 0x55f663d0c8a0 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.734+0000 7f72b4c00700 1 -- 172.21.15.46:0/4054130419 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b809b640 con 0x55f663d0c8a0 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.734+0000 7f72b4c00700 1 -- 172.21.15.46:0/4054130419 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8004d30 con 0x55f663d0c8a0 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/4054130419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 msgr2=0x55f663d0cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b8097b40 tx=0x7f72b801d9d0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/4054130419 shutdown_connections 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d0cce0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d121f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/4054130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.735+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/4054130419 >> 172.21.15.46:0/4054130419 conn(0x55f663c59210 msgr2=0x55f663cf5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.741+0000 7f72bf400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 0x7f72a40a3850 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005950 tx=0x7f72b000b410 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.741+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/4054130419 shutdown_connections 2024-05-08T23:22:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.741+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/4054130419 wait complete. 2024-05-08T23:22:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.742+0000 7f72d9ffdb80 1 -- start start 2024-05-08T23:22:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 Processor -- start 2024-05-08T23:22:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 0x55f663d16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 0x55f663d16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663be7560 con 0x55f663d12730 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663be7200 con 0x55f663d0c8a0 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663ace6a0 con 0x55f663d11e10 2024-05-08T23:22:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.743+0000 7f72d9ffdb80 10 client.0 _ll_register_callbacks cb 0x55f663bf0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.745+0000 7f72bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.745+0000 7f72bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 0x55f663d16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.745+0000 7f72bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55982/0 (socket says 172.21.15.46:55982) 2024-05-08T23:22:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.745+0000 7f72bf400700 1 -- 172.21.15.46:0/863108450 learned_addr learned my addr 172.21.15.46:0/863108450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.745+0000 7f72bfe00700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 0x55f663d16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 -- 172.21.15.46:0/863108450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 msgr2=0x55f663d15b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 -- 172.21.15.46:0/863108450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 msgr2=0x55f663d16580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 0x55f663d16580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 -- 172.21.15.46:0/863108450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01b2fd0 con 0x55f663d11e10 2024-05-08T23:22:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bf400700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bfe00700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 0x55f663d16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72bea00700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 0x55f663d16040 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72c01a77b0 tx=0x7f72ac00e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec4e833e7c1927f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.746+0000 7f72a9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.747+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00eed0 con 0x55f663d11e10 2024-05-08T23:22:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.747+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f72ac013470 con 0x55f663d11e10 2024-05-08T23:22:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.748+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f663ba4270 con 0x55f663d11e10 2024-05-08T23:22:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.749+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ac013630 con 0x55f663d11e10 2024-05-08T23:22:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.749+0000 7f72d9ffdb80 10 client.30484 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.749+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f663a78b60 con 0x55f663d11e10 2024-05-08T23:22:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.749+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f663ba4270 con 0x55f663d11e10 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72ac011070 con 0x55f663d11e10 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 10 client.30484 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 10 client.30484 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 20 client.30484 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d26110 0x55f663d28530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 4 client.30484 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.750+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f663cca810 con 0x55f663d26110 2024-05-08T23:22:57.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.751+0000 7f72bf400700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d26110 0x55f663d28530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.753+0000 7f72bf400700 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d26110 0x55f663d28530 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea0d5fdf7ff0d752 server_cookie=ce9b7c70c9a33015 in_seq=0 out_seq=0 2024-05-08T23:22:57.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.753+0000 7f72a9600700 10 client.30484 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:57.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.754+0000 7f72a9600700 1 -- 172.21.15.46:0/863108450 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f663cca810 con 0x55f663d26110 2024-05-08T23:22:57.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.754+0000 7f72a9600700 10 client.30484 handle_command_reply: tid=0 2024-05-08T23:22:57.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.754+0000 7f72d9ffdb80 1 client.30484 shutdown 2024-05-08T23:22:57.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.754+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 msgr2=0x55f663d16040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.754+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 0x55f663d16040 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72c01a77b0 tx=0x7f72ac00e730 comp rx=0 tx=0).stop 2024-05-08T23:22:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 shutdown_connections 2024-05-08T23:22:57.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d26110 0x55f663d28530 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d12730 0x55f663d16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d15b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/863108450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d11e10 0x55f663d16040 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 >> 172.21.15.46:0/863108450 conn(0x55f663c59210 msgr2=0x55f663cf55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72a9600700 0 client.30484 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 shutdown_connections 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.757+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/863108450 wait complete. 2024-05-08T23:22:57.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.758+0000 7f72d9ffdb80 20 client.30484 trim_cache size 0 max 16384 2024-05-08T23:22:57.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:57 smithi053 ceph-mon[77587]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 432 op/s 2024-05-08T23:22:57.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:57 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/831027529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:57.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:57 smithi046 ceph-mon[36898]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 432 op/s 2024-05-08T23:22:57.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:57 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/831027529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.781+0000 7f72d9ffdb80 1 Processor -- start 2024-05-08T23:22:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 -- start start 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d0c8a0 0x55f663cf5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663d186d0 con 0x55f663d0c8a0 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663cf2df0 con 0x55f663d11e10 2024-05-08T23:22:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.782+0000 7f72d9ffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663cf69d0 con 0x55f663d12730 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55990/0 (socket says 172.21.15.46:55990) 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d0c8a0 0x55f663cf5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 -- 172.21.15.46:0/396618240 learned_addr learned my addr 172.21.15.46:0/396618240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bfe00700 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 -- 172.21.15.46:0/396618240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 msgr2=0x55f663d1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d1d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 -- 172.21.15.46:0/396618240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d0c8a0 msgr2=0x55f663cf5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d0c8a0 0x55f663cf5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.783+0000 7f72bea00700 1 -- 172.21.15.46:0/396618240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f663c11560 con 0x55f663d11e10 2024-05-08T23:22:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.784+0000 7f72bea00700 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00a010 tx=0x7f72ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab81b640f3ed336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.784+0000 7f72b5600700 1 -- 172.21.15.46:0/396618240 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac005c50 con 0x55f663d11e10 2024-05-08T23:22:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.784+0000 7f72b5600700 1 -- 172.21.15.46:0/396618240 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72ac005db0 con 0x55f663d11e10 2024-05-08T23:22:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.785+0000 7f72b5600700 1 -- 172.21.15.46:0/396618240 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac013ce0 con 0x55f663d11e10 2024-05-08T23:22:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/396618240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 msgr2=0x55f663d1cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00a010 tx=0x7f72ac004750 comp rx=0 tx=0).stop 2024-05-08T23:22:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/396618240 shutdown_connections 2024-05-08T23:22:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d0c8a0 0x55f663cf5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/396618240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d11e10 0x55f663d1cf60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.787+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/396618240 >> 172.21.15.46:0/396618240 conn(0x55f663c59210 msgr2=0x55f663d2ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.790+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/396618240 shutdown_connections 2024-05-08T23:22:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.790+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/396618240 wait complete. 2024-05-08T23:22:57.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.790+0000 7f72d9ffdb80 1 -- start start 2024-05-08T23:22:57.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.791+0000 7f72d9ffdb80 1 Processor -- start 2024-05-08T23:22:57.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.791+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d1d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.791+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.791+0000 7f72d9ffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d1d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72bf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d1d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56002/0 (socket says 172.21.15.46:56002) 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72bf400700 1 -- 172.21.15.46:0/1072258939 learned_addr learned my addr 172.21.15.46:0/1072258939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72bea00700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663d0e4d0 con 0x55f663d11e10 2024-05-08T23:22:57.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663d0e640 con 0x55f663d0c8a0 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f663d0e800 con 0x55f663d12730 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.792+0000 7f72d9ffdb80 10 client.0 _ll_register_callbacks cb 0x55f663bf0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 -- 172.21.15.46:0/1072258939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 msgr2=0x55f663d1d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d1d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 -- 172.21.15.46:0/1072258939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 msgr2=0x55f663d0d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:57.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d0d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f663a78b60 con 0x55f663d12730 2024-05-08T23:22:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72a9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.793+0000 7f72bfe00700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0def0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72b8097b40 tx=0x7f72b809b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b290b1cfac041c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:57.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b809f120 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72b809b640 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f72b8084e40 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b809f120 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72d9ffdb80 10 client.30490 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.794+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f663ad0cd0 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.795+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b809b640 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.795+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72b809b8a0 con 0x55f663d12730 2024-05-08T23:22:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.796+0000 7f72d9ffdb80 10 client.30490 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.796+0000 7f72d9ffdb80 10 client.30490 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.796+0000 7f72d9ffdb80 20 client.30490 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.797+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d7c7e0 0x55f663d7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.797+0000 7f72d9ffdb80 4 client.30490 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.797+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f663c43ec0 con 0x55f663d7c7e0 2024-05-08T23:22:57.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.797+0000 7f72bf400700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d7c7e0 0x55f663d7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.801+0000 7f72a9600700 10 client.30490 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:57.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.801+0000 7f72bf400700 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d7c7e0 0x55f663d7ebd0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=555157fdc98bc789 server_cookie=2de2dea95b85cbdd in_seq=0 out_seq=0 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.801+0000 7f72a9600700 1 -- 172.21.15.46:0/1072258939 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f663c43ec0 con 0x55f663d7c7e0 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.801+0000 7f72a9600700 10 client.30490 handle_command_reply: tid=0 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.801+0000 7f72d9ffdb80 1 client.30490 shutdown 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 msgr2=0x55f663d0def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0def0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72b8097b40 tx=0x7f72b809b4c0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 shutdown_connections 2024-05-08T23:22:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55f663d7c7e0 0x55f663d7ebd0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f663d12730 0x55f663d0def0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72a9600700 0 client.30490 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f663d0c8a0 0x55f663d1d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 --2- 172.21.15.46:0/1072258939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f663d11e10 0x55f663d0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 >> 172.21.15.46:0/1072258939 conn(0x55f663c59210 msgr2=0x55f663d14360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 shutdown_connections 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.802+0000 7f72d9ffdb80 1 -- 172.21.15.46:0/1072258939 wait complete. 2024-05-08T23:22:57.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.803+0000 7f72d9ffdb80 20 client.30490 trim_cache size 0 max 16384 2024-05-08T23:22:57.806 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.803+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 msgr2=0x7f72a40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 0x7f72a40a3850 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005950 tx=0x7f72b000b410 comp rx=0 tx=0).stop 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 msgr2=0x7f72c01aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 0x7f72c01aab40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f72b800b0a0 tx=0x7f72b8007870 comp rx=0 tx=0).stop 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 shutdown_connections 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c011a780 0x7f72c01aab40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c0108030 0x7f72c01a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f72a40a1390 0x7f72a40a3850 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 --2- 172.21.15.46:0/1858418044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72c010f5e0 0x7f72c01a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 >> 172.21.15.46:0/1858418044 conn(0x7f72c006bf30 msgr2=0x7f72c0118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 shutdown_connections 2024-05-08T23:22:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:57.804+0000 7f72c6400700 1 -- 172.21.15.46:0/1858418044 wait complete. 2024-05-08T23:22:57.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:22:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:57 smithi086 ceph-mon[78610]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 432 op/s 2024-05-08T23:22:57.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:57 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/831027529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 -- 172.21.15.46:0/3917921008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 msgr2=0x7f80181084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 --2- 172.21.15.46:0/3917921008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181084b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8000009990 tx=0x7f8000009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 -- 172.21.15.46:0/3917921008 shutdown_connections 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 --2- 172.21.15.46:0/3917921008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801811a780 0x7f801811cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 --2- 172.21.15.46:0/3917921008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f801810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 --2- 172.21.15.46:0/3917921008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181084b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 -- 172.21.15.46:0/3917921008 >> 172.21.15.46:0/3917921008 conn(0x7f801806bf30 msgr2=0x7f801806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 -- 172.21.15.46:0/3917921008 shutdown_connections 2024-05-08T23:22:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.151+0000 7f801f200700 1 -- 172.21.15.46:0/3917921008 wait complete. 2024-05-08T23:22:58.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.153+0000 7f801f200700 1 Processor -- start 2024-05-08T23:22:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.154+0000 7f801f200700 1 -- start start 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f80181a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801811a780 0x7f80181aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181ab160 con 0x7f801811a780 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181ab2d0 con 0x7f8018108030 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f8017400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f80181a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f8017e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f801f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181ab490 con 0x7f801810f5e0 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f8017e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56016/0 (socket says 172.21.15.46:56016) 2024-05-08T23:22:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.156+0000 7f8017e00700 1 -- 172.21.15.46:0/585985616 learned_addr learned my addr 172.21.15.46:0/585985616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.157+0000 7f8017400700 1 -- 172.21.15.46:0/585985616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 msgr2=0x7f80181a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.157+0000 7f8017400700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.157+0000 7f8017400700 1 -- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801811a780 msgr2=0x7f80181aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.157+0000 7f8017400700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801811a780 0x7f80181aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.157+0000 7f8017400700 1 -- 172.21.15.46:0/585985616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000095f0 con 0x7f801810f5e0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f8017400700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f80181a6780 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f80000098e0 tx=0x7f800000f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f8014c00700 1 -- 172.21.15.46:0/585985616 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800001d070 con 0x7f801810f5e0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f8014c00700 1 -- 172.21.15.46:0/585985616 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f800000fb60 con 0x7f801810f5e0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80181ab710 con 0x7f801810f5e0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f8014c00700 1 -- 172.21.15.46:0/585985616 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000017650 con 0x7f801810f5e0 2024-05-08T23:22:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.158+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80181b2fb0 con 0x7f801810f5e0 2024-05-08T23:22:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.163+0000 7f8014c00700 1 -- 172.21.15.46:0/585985616 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f800000fcd0 con 0x7f801810f5e0 2024-05-08T23:22:58.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.165+0000 7f8014c00700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 0x7f7ffc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.165+0000 7f8014c00700 1 -- 172.21.15.46:0/585985616 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f80000ce620 con 0x7f801810f5e0 2024-05-08T23:22:58.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.166+0000 7f8017e00700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 0x7f7ffc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.167+0000 7f8017e00700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 0x7f7ffc0a37d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8010009ad0 tx=0x7f8010009290 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.168+0000 7f8032cd7b80 1 Processor -- start 2024-05-08T23:22:58.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 -- start start 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2ccfe00 0x5646a2cd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2c21150 con 0x5646a2ccfe00 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.169+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2c1eb10 con 0x5646a2cca890 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.170+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2c1a400 con 0x5646a2cd0720 2024-05-08T23:22:58.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.170+0000 7f801d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.170+0000 7f801d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56028/0 (socket says 172.21.15.46:56028) 2024-05-08T23:22:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.170+0000 7f801d200700 1 -- 172.21.15.46:0/1142406627 learned_addr learned my addr 172.21.15.46:0/1142406627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.172+0000 7f801d200700 1 -- 172.21.15.46:0/1142406627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 msgr2=0x5646a2cca350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:22:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.172+0000 7f801d200700 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cca350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.172+0000 7f801d200700 1 -- 172.21.15.46:0/1142406627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2ccfe00 msgr2=0x5646a2cd01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.172+0000 7f801d200700 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2ccfe00 0x5646a2cd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.172+0000 7f801d200700 1 -- 172.21.15.46:0/1142406627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a2a9b3b0 con 0x5646a2cca890 2024-05-08T23:22:58.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.173+0000 7f801d200700 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8008009d00 tx=0x7f800800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3432f6532ac865d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.174+0000 7f800d600700 1 -- 172.21.15.46:0/1142406627 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800e070 con 0x5646a2cca890 2024-05-08T23:22:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.174+0000 7f800d600700 1 -- 172.21.15.46:0/1142406627 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8008009830 con 0x5646a2cca890 2024-05-08T23:22:58.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.174+0000 7f800d600700 1 -- 172.21.15.46:0/1142406627 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800e070 con 0x5646a2cca890 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 -- 172.21.15.46:0/1142406627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 msgr2=0x5646a2ccacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8008009d00 tx=0x7f800800b330 comp rx=0 tx=0).stop 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 -- 172.21.15.46:0/1142406627 shutdown_connections 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2ccacd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2ccfe00 0x5646a2cd01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.175+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/1142406627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.176+0000 7f8032cd7b80 1 -- 172.21.15.46:0/1142406627 >> 172.21.15.46:0/1142406627 conn(0x5646a2c28210 msgr2=0x5646a2cc4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.176+0000 7f8032cd7b80 1 -- 172.21.15.46:0/1142406627 shutdown_connections 2024-05-08T23:22:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.176+0000 7f8032cd7b80 1 -- 172.21.15.46:0/1142406627 wait complete. 2024-05-08T23:22:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.177+0000 7f8032cd7b80 1 -- start start 2024-05-08T23:22:58.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.177+0000 7f8032cd7b80 1 Processor -- start 2024-05-08T23:22:58.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.179+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cd3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.179+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cd0720 0x5646a2cd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2bb6560 con 0x5646a2cca890 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2bb6200 con 0x5646a2cd0720 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2a9d6a0 con 0x5646a2ccfe00 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8032cd7b80 10 client.0 _ll_register_callbacks cb 0x5646a2bbfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:58.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8017400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8017400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41952/0 (socket says 172.21.15.46:41952) 2024-05-08T23:22:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8017400700 1 -- 172.21.15.46:0/392643532 learned_addr learned my addr 172.21.15.46:0/392643532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.180+0000 7f8017e00700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cd3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.181+0000 7f801d200700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cd0720 0x5646a2cd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.182+0000 7f8017e00700 1 -- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 msgr2=0x5646a2cd40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.182+0000 7f8017e00700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.182+0000 7f8017e00700 1 -- 172.21.15.46:0/392643532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cd0720 msgr2=0x5646a2cd4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.182+0000 7f8017e00700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cd0720 0x5646a2cd4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.182+0000 7f8017e00700 1 -- 172.21.15.46:0/392643532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080099b0 con 0x5646a2cca890 2024-05-08T23:22:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.184+0000 7f8017e00700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cd3b80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8010015700 tx=0x7f8010015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f6fd635f129226 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.184+0000 7f8006000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:22:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.184+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010015d40 con 0x5646a2cca890 2024-05-08T23:22:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.185+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646a2c99810 con 0x5646a2cca890 2024-05-08T23:22:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.185+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8010013450 con 0x5646a2cca890 2024-05-08T23:22:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.185+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8010015ea0 con 0x5646a2cca890 2024-05-08T23:22:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.185+0000 7f8032cd7b80 10 client.20250 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.185+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5646a2a47b60 con 0x5646a2cca890 2024-05-08T23:22:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646a2c99810 con 0x5646a2cca890 2024-05-08T23:22:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f801001e070 con 0x5646a2cca890 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 10 client.20250 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 10 client.20250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 20 client.20250 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2ce66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 4 client.20250 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.186+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646a2c12ec0 con 0x5646a2ce42d0 2024-05-08T23:22:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.188+0000 7f8017400700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2ce66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.188+0000 7f8017400700 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2ce66f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60407e9cd370dd1 server_cookie=51f4b0f467499ed7 in_seq=0 out_seq=0 2024-05-08T23:22:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.188+0000 7f8006000700 10 client.20250 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:58.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8006000700 1 -- 172.21.15.46:0/392643532 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646a2c12ec0 con 0x5646a2ce42d0 2024-05-08T23:22:58.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8006000700 10 client.20250 handle_command_reply: tid=0 2024-05-08T23:22:58.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 client.20250 shutdown 2024-05-08T23:22:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 msgr2=0x5646a2cd3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cd3b80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8010015700 tx=0x7f8010015730 comp rx=0 tx=0).stop 2024-05-08T23:22:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 shutdown_connections 2024-05-08T23:22:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cd0720 0x5646a2cd4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2ce66f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cd3b80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.196+0000 7f8006000700 0 client.20250 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.197+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/392643532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.197+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 >> 172.21.15.46:0/392643532 conn(0x5646a2c28210 msgr2=0x5646a2cc45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.197+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 shutdown_connections 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.197+0000 7f8032cd7b80 1 -- 172.21.15.46:0/392643532 wait complete. 2024-05-08T23:22:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.197+0000 7f8032cd7b80 20 client.20250 trim_cache size 0 max 16384 2024-05-08T23:22:58.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.220+0000 7f8032cd7b80 1 Processor -- start 2024-05-08T23:22:58.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.220+0000 7f8032cd7b80 1 -- start start 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.221+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.221+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.221+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cc4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2cc1dc0 con 0x5646a2cca890 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2ceb8c0 con 0x5646a2ccfe00 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2cd6ba0 con 0x5646a2cd0720 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56042/0 (socket says 172.21.15.46:56042) 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 -- 172.21.15.46:0/4232741609 learned_addr learned my addr 172.21.15.46:0/4232741609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017e00700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f801d200700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cc4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 -- 172.21.15.46:0/4232741609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 msgr2=0x5646a2cc4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cc4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 -- 172.21.15.46:0/4232741609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 msgr2=0x5646a2cca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017400700 1 -- 172.21.15.46:0/4232741609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a2be0560 con 0x5646a2ccfe00 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.222+0000 7f8017e00700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.223+0000 7f8017400700 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8000000c00 tx=0x7f800001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab0a6febc873d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.223+0000 7f800e000700 1 -- 172.21.15.46:0/4232741609 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a37a0 con 0x5646a2ccfe00 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.223+0000 7f800e000700 1 -- 172.21.15.46:0/4232741609 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8000099750 con 0x5646a2ccfe00 2024-05-08T23:22:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.223+0000 7f800e000700 1 -- 172.21.15.46:0/4232741609 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a37a0 con 0x5646a2ccfe00 2024-05-08T23:22:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.225+0000 7f8032cd7b80 1 -- 172.21.15.46:0/4232741609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 msgr2=0x5646a2ccf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.225+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8000000c00 tx=0x7f800001a040 comp rx=0 tx=0).stop 2024-05-08T23:22:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.226+0000 7f8032cd7b80 1 -- 172.21.15.46:0/4232741609 shutdown_connections 2024-05-08T23:22:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.226+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2cd0720 0x5646a2cc4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.226+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cca890 0x5646a2cca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.226+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/4232741609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2ccfe00 0x5646a2ccf350 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.226+0000 7f8032cd7b80 1 -- 172.21.15.46:0/4232741609 >> 172.21.15.46:0/4232741609 conn(0x5646a2c28210 msgr2=0x5646a2ccec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.227+0000 7f8032cd7b80 1 -- 172.21.15.46:0/4232741609 shutdown_connections 2024-05-08T23:22:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.227+0000 7f8032cd7b80 1 -- 172.21.15.46:0/4232741609 wait complete. 2024-05-08T23:22:58.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.228+0000 7f8032cd7b80 1 -- start start 2024-05-08T23:22:58.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 Processor -- start 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 0x5646a2cd96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2d39790 con 0x5646a2cd0720 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2d398d0 con 0x5646a2cca890 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.229+0000 7f8032cd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2d39a60 con 0x5646a2ccfe00 2024-05-08T23:22:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.230+0000 7f8032cd7b80 10 client.0 _ll_register_callbacks cb 0x5646a2bbfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.230+0000 7f8017e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.230+0000 7f8017e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56048/0 (socket says 172.21.15.46:56048) 2024-05-08T23:22:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.230+0000 7f8017e00700 1 -- 172.21.15.46:0/232289949 learned_addr learned my addr 172.21.15.46:0/232289949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.231+0000 7f8017400700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.231+0000 7f801d200700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 0x5646a2cd96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f8017e00700 1 -- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 msgr2=0x5646a2cd9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f8017e00700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f8017e00700 1 -- 172.21.15.46:0/232289949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 msgr2=0x5646a2cd96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f8017e00700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 0x5646a2cd96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f8017e00700 1 -- 172.21.15.46:0/232289949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646a2a47b60 con 0x5646a2cca890 2024-05-08T23:22:58.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.232+0000 7f801d200700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 0x5646a2cd96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:22:58.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.233+0000 7f8017e00700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f80100048a0 tx=0x7f801001e4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43aac2e4f1188f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.233+0000 7f8006000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:58.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.233+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801001ed90 con 0x5646a2cca890 2024-05-08T23:22:58.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.233+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f801001e7c0 con 0x5646a2cca890 2024-05-08T23:22:58.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.234+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646a2b73270 con 0x5646a2cca890 2024-05-08T23:22:58.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.235+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f801001eef0 con 0x5646a2cca890 2024-05-08T23:22:58.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.235+0000 7f8032cd7b80 10 client.30494 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:58.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.235+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f80181b2fb0 con 0x5646a2cca890 2024-05-08T23:22:58.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646a2b73270 con 0x5646a2cca890 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8010006150 con 0x5646a2cca890 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 10 client.30494 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 10 client.30494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 20 client.30494 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2d3d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 4 client.30494 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.237+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5646a2a68cf0 con 0x5646a2ce42d0 2024-05-08T23:22:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.238+0000 7f8017400700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2d3d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.239+0000 7f8017400700 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2d3d290 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73b62b6baae24e82 server_cookie=d9a65da70c09044 in_seq=0 out_seq=0 2024-05-08T23:22:58.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.239+0000 7f8006000700 10 client.30494 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.245+0000 7f8006000700 1 -- 172.21.15.46:0/232289949 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5646a2a68cf0 con 0x5646a2ce42d0 2024-05-08T23:22:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.245+0000 7f8006000700 10 client.30494 handle_command_reply: tid=0 2024-05-08T23:22:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.245+0000 7f8032cd7b80 1 client.30494 shutdown 2024-05-08T23:22:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.245+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 msgr2=0x5646a2cd8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.245+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f80100048a0 tx=0x7f801001e4f0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 shutdown_connections 2024-05-08T23:22:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5646a2ce42d0 0x5646a2d3d290 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646a2cd0720 0x5646a2cd96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646a2cca890 0x5646a2cd8c20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 --2- 172.21.15.46:0/232289949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646a2ccfe00 0x5646a2cd9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 >> 172.21.15.46:0/232289949 conn(0x5646a2c28210 msgr2=0x5646a2cd4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8006000700 0 client.30494 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 shutdown_connections 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.246+0000 7f8032cd7b80 1 -- 172.21.15.46:0/232289949 wait complete. 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.247+0000 7f8032cd7b80 20 client.30494 trim_cache size 0 max 16384 2024-05-08T23:22:58.248 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 msgr2=0x7f7ffc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 0x7f7ffc0a37d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8010009ad0 tx=0x7f8010009290 comp rx=0 tx=0).stop 2024-05-08T23:22:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 msgr2=0x7f80181a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f80181a6780 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f80000098e0 tx=0x7f800000f670 comp rx=0 tx=0).stop 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 shutdown_connections 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f7ffc0a1310 0x7f7ffc0a37d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801811a780 0x7f80181aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018108030 0x7f80181a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 --2- 172.21.15.46:0/585985616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801810f5e0 0x7f80181a6780 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.249+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 >> 172.21.15.46:0/585985616 conn(0x7f801806bf30 msgr2=0x7f8018118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.250+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 shutdown_connections 2024-05-08T23:22:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.250+0000 7f801f200700 1 -- 172.21.15.46:0/585985616 wait complete. 2024-05-08T23:22:58.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122580 19.87 MB/sec execute 199 sec latency 228.812 ms 2024-05-08T23:22:58.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:22:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.628+0000 7f36df400700 1 -- 172.21.15.46:0/3078812197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 msgr2=0x7f36d810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.628+0000 7f36df400700 1 --2- 172.21.15.46:0/3078812197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d810ae40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098f0 tx=0x7f36cc009c00 comp rx=0 tx=0).stop 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 -- 172.21.15.46:0/3078812197 shutdown_connections 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 --2- 172.21.15.46:0/3078812197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d810b410 0x7f36d810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 --2- 172.21.15.46:0/3078812197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d8102040 0x7f36d8102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 --2- 172.21.15.46:0/3078812197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d810ae40 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 -- 172.21.15.46:0/3078812197 >> 172.21.15.46:0/3078812197 conn(0x7f36d80fb760 msgr2=0x7f36d80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 -- 172.21.15.46:0/3078812197 shutdown_connections 2024-05-08T23:22:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.629+0000 7f36df400700 1 -- 172.21.15.46:0/3078812197 wait complete. 2024-05-08T23:22:58.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.632+0000 7f36df400700 1 Processor -- start 2024-05-08T23:22:58.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.632+0000 7f36df400700 1 -- start start 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d8102040 0x7f36d819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d810b410 0x7f36d8109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d810a780 con 0x7f36d8102040 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d810a8f0 con 0x7f36d810b410 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36df400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d810aab0 con 0x7f36d8102960 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:41978/0 (socket says 172.21.15.46:41978) 2024-05-08T23:22:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 -- 172.21.15.46:0/1199479538 learned_addr learned my addr 172.21.15.46:0/1199479538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 -- 172.21.15.46:0/1199479538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d810b410 msgr2=0x7f36d8109fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d810b410 0x7f36d8109fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 -- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d8102040 msgr2=0x7f36d819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d8102040 0x7f36d819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.633+0000 7f36d7e00700 1 -- 172.21.15.46:0/1199479538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009520 con 0x7f36d8102960 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.634+0000 7f36d7e00700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005950 tx=0x7f36cc00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.634+0000 7f36d5600700 1 -- 172.21.15.46:0/1199479538 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc01c070 con 0x7f36d8102960 2024-05-08T23:22:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.634+0000 7f36d5600700 1 -- 172.21.15.46:0/1199479538 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc005040 con 0x7f36d8102960 2024-05-08T23:22:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.634+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d810ad30 con 0x7f36d8102960 2024-05-08T23:22:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.634+0000 7f36d5600700 1 -- 172.21.15.46:0/1199479538 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc003a80 con 0x7f36d8102960 2024-05-08T23:22:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.635+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d8199d40 con 0x7f36d8102960 2024-05-08T23:22:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.641+0000 7f36d5600700 1 -- 172.21.15.46:0/1199479538 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f36cc020030 con 0x7f36d8102960 2024-05-08T23:22:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.642+0000 7f36f2febb80 1 Processor -- start 2024-05-08T23:22:58.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.643+0000 7f36d5600700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 0x7f36c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.644+0000 7f36d5600700 1 -- 172.21.15.46:0/1199479538 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f36cc0ce080 con 0x7f36d8102960 2024-05-08T23:22:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.644+0000 7f36f2febb80 1 -- start start 2024-05-08T23:22:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.644+0000 7f36dd400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 0x7f36c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fdd890 0x5602e6fddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6f34150 con 0x5602e6fdd890 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6f31b10 con 0x5602e6fe2e00 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36f2febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6f2d400 con 0x5602e6fe3720 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56082/0 (socket says 172.21.15.46:56082) 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.645+0000 7f36dca00700 1 -- 172.21.15.46:0/615382375 learned_addr learned my addr 172.21.15.46:0/615382375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dd400700 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fdd890 0x5602e6fddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 -- 172.21.15.46:0/615382375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 msgr2=0x5602e6fdd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fdd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 -- 172.21.15.46:0/615382375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fdd890 msgr2=0x5602e6fddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fdd890 0x5602e6fddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 -- 172.21.15.46:0/615382375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e6dae3b0 con 0x5602e6fe2e00 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.646+0000 7f36dca00700 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009bc0 tx=0x7f36c8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d570ed8c7c36fb8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.647+0000 7f36dd400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 0x7f36c00a3750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f36d819ec20 tx=0x7f36d000a610 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.647+0000 7f36c5c00700 1 -- 172.21.15.46:0/615382375 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800eb60 con 0x5602e6fe2e00 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.647+0000 7f36c5c00700 1 -- 172.21.15.46:0/615382375 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c8004cf0 con 0x5602e6fe2e00 2024-05-08T23:22:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.647+0000 7f36c5c00700 1 -- 172.21.15.46:0/615382375 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8005700 con 0x5602e6fe2e00 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 -- 172.21.15.46:0/615382375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 msgr2=0x5602e6fe31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009bc0 tx=0x7f36c8009ed0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 -- 172.21.15.46:0/615382375 shutdown_connections 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fdd890 0x5602e6fddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe2e00 0x5602e6fe31e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 --2- 172.21.15.46:0/615382375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.648+0000 7f36f2febb80 1 -- 172.21.15.46:0/615382375 >> 172.21.15.46:0/615382375 conn(0x5602e6f3b210 msgr2=0x5602e6fd7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.649+0000 7f36f2febb80 1 -- 172.21.15.46:0/615382375 shutdown_connections 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.649+0000 7f36f2febb80 1 -- 172.21.15.46:0/615382375 wait complete. 2024-05-08T23:22:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.650+0000 7f36f2febb80 1 -- start start 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.650+0000 7f36f2febb80 1 Processor -- start 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fe7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe77d0 0x5602e6ff3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6ec9560 con 0x5602e6fe77d0 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6ec9200 con 0x5602e6fdd890 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6db06a0 con 0x5602e6fe3720 2024-05-08T23:22:58.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.651+0000 7f36f2febb80 10 client.0 _ll_register_callbacks cb 0x5602e6ed2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.652+0000 7f36dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.652+0000 7f36dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56094/0 (socket says 172.21.15.46:56094) 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.652+0000 7f36dca00700 1 -- 172.21.15.46:0/2055127142 learned_addr learned my addr 172.21.15.46:0/2055127142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36d7e00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fe7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36dca00700 1 -- 172.21.15.46:0/2055127142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 msgr2=0x5602e6fe7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36dca00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fe7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36dca00700 1 -- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe77d0 msgr2=0x5602e6ff3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36dca00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe77d0 0x5602e6ff3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.653+0000 7f36dca00700 1 -- 172.21.15.46:0/2055127142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c80098a0 con 0x5602e6fdd890 2024-05-08T23:22:58.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.654+0000 7f36dca00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f36c8005320 tx=0x7f36c8005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4aca2ba512fe40d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.655+0000 7f36ba000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:58.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.655+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800eb60 con 0x5602e6fdd890 2024-05-08T23:22:58.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.656+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f36c8014070 con 0x5602e6fdd890 2024-05-08T23:22:58.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.656+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602e6e86270 con 0x5602e6fdd890 2024-05-08T23:22:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.660+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c8005da0 con 0x5602e6fdd890 2024-05-08T23:22:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.661+0000 7f36f2febb80 10 client.30503 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.661+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5602e6d5ab60 con 0x5602e6fdd890 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.662+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602e6e86270 con 0x5602e6fdd890 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.662+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36c8010c10 con 0x5602e6fdd890 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.662+0000 7f36f2febb80 10 client.30503 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.662+0000 7f36f2febb80 10 client.30503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.662+0000 7f36f2febb80 20 client.30503 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.663+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e6ff7790 0x5602e6ff9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.663+0000 7f36f2febb80 4 client.30503 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.663+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602e6fac810 con 0x5602e6ff7790 2024-05-08T23:22:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.663+0000 7f36d7e00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e6ff7790 0x5602e6ff9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.664+0000 7f36ba000700 10 client.30503 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.665+0000 7f36d7e00700 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e6ff7790 0x5602e6ff9b80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3581329d943ff435 server_cookie=67a9cde2db721d8c in_seq=0 out_seq=0 2024-05-08T23:22:58.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.668+0000 7f36ba000700 1 -- 172.21.15.46:0/2055127142 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602e6fac810 con 0x5602e6ff7790 2024-05-08T23:22:58.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.668+0000 7f36ba000700 10 client.30503 handle_command_reply: tid=0 2024-05-08T23:22:58.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.668+0000 7f36f2febb80 1 client.30503 shutdown 2024-05-08T23:22:58.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.669+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 msgr2=0x5602e6fe6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.669+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f36c8005320 tx=0x7f36c8005660 comp rx=0 tx=0).stop 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 shutdown_connections 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e6ff7790 0x5602e6ff9b80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe77d0 0x5602e6ff3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fdd890 0x5602e6fe6d50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2055127142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fe3720 0x5602e6fe7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 >> 172.21.15.46:0/2055127142 conn(0x5602e6f3b210 msgr2=0x5602e6fd75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36ba000700 0 client.30503 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 shutdown_connections 2024-05-08T23:22:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.670+0000 7f36f2febb80 1 -- 172.21.15.46:0/2055127142 wait complete. 2024-05-08T23:22:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.671+0000 7f36f2febb80 20 client.30503 trim_cache size 0 max 16384 2024-05-08T23:22:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.698+0000 7f36f2febb80 1 Processor -- start 2024-05-08T23:22:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.698+0000 7f36f2febb80 1 -- start start 2024-05-08T23:22:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e6fd7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6fee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6fd4dc0 con 0x5602e6fe3720 2024-05-08T23:22:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6ffedb0 con 0x5602e6fe77d0 2024-05-08T23:22:58.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36f2febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6fe52c0 con 0x5602e6fdd890 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.699+0000 7f36dca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42002/0 (socket says 172.21.15.46:42002) 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 -- 172.21.15.46:0/2954518944 learned_addr learned my addr 172.21.15.46:0/2954518944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36d7e00700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e6fd7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dd400700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6fee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 -- 172.21.15.46:0/2954518944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 msgr2=0x5602e6fee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6fee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 -- 172.21.15.46:0/2954518944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 msgr2=0x5602e6fd7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e6fd7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 -- 172.21.15.46:0/2954518944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e6ef3560 con 0x5602e6fdd890 2024-05-08T23:22:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dd400700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6fee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.700+0000 7f36dca00700 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009530 tx=0x7f36c8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb04049b9db9745 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.703+0000 7f36c6600700 1 -- 172.21.15.46:0/2954518944 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800e6e0 con 0x5602e6fdd890 2024-05-08T23:22:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.703+0000 7f36c6600700 1 -- 172.21.15.46:0/2954518944 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c8021830 con 0x5602e6fdd890 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.703+0000 7f36c6600700 1 -- 172.21.15.46:0/2954518944 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8021ef0 con 0x5602e6fdd890 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.704+0000 7f36f2febb80 1 -- 172.21.15.46:0/2954518944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 msgr2=0x5602e6fdd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.704+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009530 tx=0x7f36c8009560 comp rx=0 tx=0).stop 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 -- 172.21.15.46:0/2954518944 shutdown_connections 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6fee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e6fdd350 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2954518944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e6fd7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 -- 172.21.15.46:0/2954518944 >> 172.21.15.46:0/2954518944 conn(0x5602e6f3b210 msgr2=0x5602e6fe1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.705+0000 7f36f2febb80 1 -- 172.21.15.46:0/2954518944 shutdown_connections 2024-05-08T23:22:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.706+0000 7f36f2febb80 1 -- 172.21.15.46:0/2954518944 wait complete. 2024-05-08T23:22:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.706+0000 7f36f2febb80 1 -- start start 2024-05-08T23:22:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.707+0000 7f36f2febb80 1 Processor -- start 2024-05-08T23:22:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e7046610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e7046b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6fee040 con 0x5602e6fe3720 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6fee1b0 con 0x5602e6fe77d0 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e6fee370 con 0x5602e6fdd890 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36f2febb80 10 client.0 _ll_register_callbacks cb 0x5602e6ed2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.709+0000 7f36dd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.710+0000 7f36dd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56126/0 (socket says 172.21.15.46:56126) 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.710+0000 7f36d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e7046b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.710+0000 7f36d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e7046b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37854/0 (socket says 172.21.15.46:37854) 2024-05-08T23:22:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.710+0000 7f36d7e00700 1 -- 172.21.15.46:0/2889725664 learned_addr learned my addr 172.21.15.46:0/2889725664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 -- 172.21.15.46:0/2889725664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 msgr2=0x5602e7046610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e7046610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 -- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 msgr2=0x5602e7046b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e7046b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 -- 172.21.15.46:0/2889725664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602e6d5ab60 con 0x5602e6fe77d0 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36dd400700 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5602e6fe2bc0 tx=0x7f36d0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfd3f19ae7ee9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.711+0000 7f36ba000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d000fc70 con 0x5602e6fe77d0 2024-05-08T23:22:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36c8004cf0 con 0x5602e6fe77d0 2024-05-08T23:22:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f36d0006e40 con 0x5602e6fe77d0 2024-05-08T23:22:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d000fdd0 con 0x5602e6fe77d0 2024-05-08T23:22:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36f2febb80 10 client.30506 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.712+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5602e6db2cd0 con 0x5602e6fe77d0 2024-05-08T23:22:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.714+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c8004cf0 con 0x5602e6fe77d0 2024-05-08T23:22:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36d002a4f0 con 0x5602e6fe77d0 2024-05-08T23:22:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 10 client.30506 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 10 client.30506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:22:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 20 client.30506 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e704e520 0x5602e7050940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 4 client.30506 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.715+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5602e6f25ec0 con 0x5602e704e520 2024-05-08T23:22:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.717+0000 7f36dca00700 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e704e520 0x5602e7050940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.720+0000 7f36dca00700 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e704e520 0x5602e7050940 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2af94e35ff3e6ae server_cookie=124c7e48eebc3086 in_seq=0 out_seq=0 2024-05-08T23:22:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.720+0000 7f36ba000700 10 client.30506 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.720+0000 7f36ba000700 1 -- 172.21.15.46:0/2889725664 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5602e6f25ec0 con 0x5602e704e520 2024-05-08T23:22:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.720+0000 7f36ba000700 10 client.30506 handle_command_reply: tid=0 2024-05-08T23:22:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.720+0000 7f36f2febb80 1 client.30506 shutdown 2024-05-08T23:22:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.721+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 msgr2=0x5602e6feda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.721+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5602e6fe2bc0 tx=0x7f36d0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 shutdown_connections 2024-05-08T23:22:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e6fe77d0 0x5602e6feda60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e6fdd890 0x5602e7046610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x5602e704e520 0x5602e7050940 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 --2- 172.21.15.46:0/2889725664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5602e6fe3720 0x5602e7046b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36ba000700 0 client.30506 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 >> 172.21.15.46:0/2889725664 conn(0x5602e6f3b210 msgr2=0x5602e6fe1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 shutdown_connections 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.722+0000 7f36f2febb80 1 -- 172.21.15.46:0/2889725664 wait complete. 2024-05-08T23:22:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.723+0000 7f36f2febb80 20 client.30506 trim_cache size 0 max 16384 2024-05-08T23:22:58.724 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.725+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 msgr2=0x7f36c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.725+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 0x7f36c00a3750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f36d819ec20 tx=0x7f36d000a610 comp rx=0 tx=0).stop 2024-05-08T23:22:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.725+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 msgr2=0x7f36d819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.725+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005950 tx=0x7f36cc00bfc0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 shutdown_connections 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d810b410 0x7f36d8109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f36c00a1290 0x7f36c00a3750 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d8102040 0x7f36d819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 --2- 172.21.15.46:0/1199479538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d8102960 0x7f36d819e550 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 >> 172.21.15.46:0/1199479538 conn(0x7f36d80fb760 msgr2=0x7f36d8100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 shutdown_connections 2024-05-08T23:22:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:58.727+0000 7f36df400700 1 -- 172.21.15.46:0/1199479538 wait complete. 2024-05-08T23:22:58.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:22:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.138+0000 7f83bf600700 1 -- 172.21.15.46:0/2645710528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 msgr2=0x7f83b811cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.138+0000 7f83bf600700 1 --2- 172.21.15.46:0/2645710528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b811cb70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83b000b0a0 tx=0x7f83b000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 -- 172.21.15.46:0/2645710528 shutdown_connections 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 --2- 172.21.15.46:0/2645710528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b811cb70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 --2- 172.21.15.46:0/2645710528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b810f5e0 0x7f83b810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 --2- 172.21.15.46:0/2645710528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b8108030 0x7f83b81084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 -- 172.21.15.46:0/2645710528 >> 172.21.15.46:0/2645710528 conn(0x7f83b806bf30 msgr2=0x7f83b806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 -- 172.21.15.46:0/2645710528 shutdown_connections 2024-05-08T23:22:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.139+0000 7f83bf600700 1 -- 172.21.15.46:0/2645710528 wait complete. 2024-05-08T23:22:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.141+0000 7f83bf600700 1 Processor -- start 2024-05-08T23:22:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.141+0000 7f83bf600700 1 -- start start 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b8108030 0x7f83b81a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b81aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81ab150 con 0x7f83b8108030 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81ab2c0 con 0x7f83b811a780 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bf600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81ab480 con 0x7f83b810f5e0 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b81aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b81aab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56130/0 (socket says 172.21.15.46:56130) 2024-05-08T23:22:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42026/0 (socket says 172.21.15.46:42026) 2024-05-08T23:22:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.142+0000 7f83bd600700 1 -- 172.21.15.46:0/881755815 learned_addr learned my addr 172.21.15.46:0/881755815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83bcc00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b8108030 0x7f83b81a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83b7e00700 1 -- 172.21.15.46:0/881755815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 msgr2=0x7f83b81aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83b7e00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b81aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83b7e00700 1 -- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b8108030 msgr2=0x7f83b81a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83b7e00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b8108030 0x7f83b81a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.143+0000 7f83b7e00700 1 -- 172.21.15.46:0/881755815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009520 con 0x7f83b810f5e0 2024-05-08T23:22:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.144+0000 7f83b7e00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.145+0000 7f83b5600700 1 -- 172.21.15.46:0/881755815 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400cb70 con 0x7f83b810f5e0 2024-05-08T23:22:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.145+0000 7f83b5600700 1 -- 172.21.15.46:0/881755815 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a400ee90 con 0x7f83b810f5e0 2024-05-08T23:22:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.145+0000 7f83b5600700 1 -- 172.21.15.46:0/881755815 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4003c80 con 0x7f83b810f5e0 2024-05-08T23:22:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.145+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac009520 con 0x7f83b810f5e0 2024-05-08T23:22:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.145+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b0009d00 con 0x7f83b810f5e0 2024-05-08T23:22:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.150+0000 7f83b5600700 1 -- 172.21.15.46:0/881755815 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f83a4003e00 con 0x7f83b810f5e0 2024-05-08T23:22:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.152+0000 7f83b5600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 0x7f83a00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.153+0000 7f83bcc00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 0x7f83a00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.154+0000 7f83b5600700 1 -- 172.21.15.46:0/881755815 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83a4014070 con 0x7f83b810f5e0 2024-05-08T23:22:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.155+0000 7f83bcc00700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 0x7f83a00a3850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009d70 tx=0x7f83ac009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.157+0000 7f83d3088b80 1 Processor -- start 2024-05-08T23:22:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.158+0000 7f83d3088b80 1 -- start start 2024-05-08T23:22:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.159+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daafe00 0x562a2dab01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.159+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2dab0720 0x562a2daaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.159+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83d3088b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2da01150 con 0x562a2dab0720 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83d3088b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2d9feb10 con 0x562a2daafe00 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83d3088b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2d9fa400 con 0x562a2daaa890 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42038/0 (socket says 172.21.15.46:42038) 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 -- 172.21.15.46:0/3668705335 learned_addr learned my addr 172.21.15.46:0/3668705335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 -- 172.21.15.46:0/3668705335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daafe00 msgr2=0x562a2dab01e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daafe00 0x562a2dab01e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 -- 172.21.15.46:0/3668705335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2dab0720 msgr2=0x562a2daaa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2dab0720 0x562a2daaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.160+0000 7f83bd600700 1 -- 172.21.15.46:0/3668705335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a2d87b3b0 con 0x562a2daaa890 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.161+0000 7f83bd600700 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009c90 tx=0x7f83b00093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990343c2e6a0c7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.161+0000 7f83a9c00700 1 -- 172.21.15.46:0/3668705335 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000e040 con 0x562a2daaa890 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.161+0000 7f83a9c00700 1 -- 172.21.15.46:0/3668705335 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b0007db0 con 0x562a2daaa890 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.162+0000 7f83a9c00700 1 -- 172.21.15.46:0/3668705335 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b00077f0 con 0x562a2daaa890 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.162+0000 7f83d3088b80 1 -- 172.21.15.46:0/3668705335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 msgr2=0x562a2daaacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.162+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009c90 tx=0x7f83b00093a0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.164+0000 7f83d3088b80 1 -- 172.21.15.46:0/3668705335 shutdown_connections 2024-05-08T23:22:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.164+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaacd0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.164+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daafe00 0x562a2dab01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.164+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3668705335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2dab0720 0x562a2daaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.164+0000 7f83d3088b80 1 -- 172.21.15.46:0/3668705335 >> 172.21.15.46:0/3668705335 conn(0x562a2da08210 msgr2=0x562a2daa4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.165+0000 7f83d3088b80 1 -- 172.21.15.46:0/3668705335 shutdown_connections 2024-05-08T23:22:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.166+0000 7f83d3088b80 1 -- 172.21.15.46:0/3668705335 wait complete. 2024-05-08T23:22:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.167+0000 7f83d3088b80 1 -- start start 2024-05-08T23:22:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.167+0000 7f83d3088b80 1 Processor -- start 2024-05-08T23:22:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daaa890 0x562a2dab3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2d996560 con 0x562a2daaa890 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2d996200 con 0x562a2dab0720 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2d87d6a0 con 0x562a2daafe00 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83d3088b80 10 client.0 _ll_register_callbacks cb 0x562a2d99fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42040/0 (socket says 172.21.15.46:42040) 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.168+0000 7f83b7e00700 1 -- 172.21.15.46:0/3059082609 learned_addr learned my addr 172.21.15.46:0/3059082609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83bd600700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83b7e00700 1 -- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 msgr2=0x562a2dab45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83b7e00700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83b7e00700 1 -- 172.21.15.46:0/3059082609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daaa890 msgr2=0x562a2dab3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83b7e00700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daaa890 0x562a2dab3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.169+0000 7f83b7e00700 1 -- 172.21.15.46:0/3059082609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b811d3e0 con 0x562a2daafe00 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.170+0000 7f83b7e00700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c5b0 tx=0x7f83a40999f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a87ae9ad9938fbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.170+0000 7f839e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:22:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.170+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40cd1b0 con 0x562a2daafe00 2024-05-08T23:22:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.171+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83a409de40 con 0x562a2daafe00 2024-05-08T23:22:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.171+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a2d953270 con 0x562a2daafe00 2024-05-08T23:22:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.171+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a40a2490 con 0x562a2daafe00 2024-05-08T23:22:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.172+0000 7f83d3088b80 10 client.30529 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.172+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562a2d827b60 con 0x562a2daafe00 2024-05-08T23:22:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.172+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a2d953270 con 0x562a2daafe00 2024-05-08T23:22:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.172+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f83a40a2000 con 0x562a2daafe00 2024-05-08T23:22:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 10 client.30529 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 10 client.30529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 20 client.30529 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2dac4150 0x562a2dac6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 4 client.30529 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.173+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a2da79810 con 0x562a2dac4150 2024-05-08T23:22:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.174+0000 7f83bcc00700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2dac4150 0x562a2dac6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.177+0000 7f83bcc00700 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2dac4150 0x562a2dac6570 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eec12b07dcf82f4 server_cookie=683c3219ef0b7cae in_seq=0 out_seq=0 2024-05-08T23:22:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.177+0000 7f839e000700 10 client.30529 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.179+0000 7f839e000700 1 -- 172.21.15.46:0/3059082609 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a2da79810 con 0x562a2dac4150 2024-05-08T23:22:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.179+0000 7f839e000700 10 client.30529 handle_command_reply: tid=0 2024-05-08T23:22:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.179+0000 7f83d3088b80 1 client.30529 shutdown 2024-05-08T23:22:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 msgr2=0x562a2dab4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c5b0 tx=0x7f83a40999f0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 shutdown_connections 2024-05-08T23:22:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2dac4150 0x562a2dac6570 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daaa890 0x562a2dab3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.180+0000 7f83d3088b80 1 --2- 172.21.15.46:0/3059082609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daafe00 0x562a2dab4090 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.181+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 >> 172.21.15.46:0/3059082609 conn(0x562a2da08210 msgr2=0x562a2daa45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.181+0000 7f839e000700 0 client.30529 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:59.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.182+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 shutdown_connections 2024-05-08T23:22:59.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.183+0000 7f83d3088b80 1 -- 172.21.15.46:0/3059082609 wait complete. 2024-05-08T23:22:59.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.184+0000 7f83d3088b80 20 client.30529 trim_cache size 0 max 16384 2024-05-08T23:22:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.213+0000 7f83d3088b80 1 Processor -- start 2024-05-08T23:22:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.213+0000 7f83d3088b80 1 -- start start 2024-05-08T23:22:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2daa4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2dac3f80 con 0x562a2daafe00 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2dac1990 con 0x562a2dab0720 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83d3088b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2daa67e0 con 0x562a2daaa890 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83bcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83bcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42048/0 (socket says 172.21.15.46:42048) 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83bd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83bcc00700 1 -- 172.21.15.46:0/2946877064 learned_addr learned my addr 172.21.15.46:0/2946877064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:59.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.214+0000 7f83b7e00700 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2daa4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 -- 172.21.15.46:0/2946877064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 msgr2=0x562a2daaa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 -- 172.21.15.46:0/2946877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 msgr2=0x562a2daa4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2daa4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 -- 172.21.15.46:0/2946877064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a2daaaf40 con 0x562a2dab0720 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.215+0000 7f83bd600700 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab43f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009c60 tx=0x7f83b0006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a894aaded0db62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.216+0000 7f83aa600700 1 -- 172.21.15.46:0/2946877064 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000e040 con 0x562a2dab0720 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.216+0000 7f83aa600700 1 -- 172.21.15.46:0/2946877064 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b0003a00 con 0x562a2dab0720 2024-05-08T23:22:59.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.216+0000 7f83aa600700 1 -- 172.21.15.46:0/2946877064 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b001c6a0 con 0x562a2dab0720 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 -- 172.21.15.46:0/2946877064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 msgr2=0x562a2dab43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab43f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009c60 tx=0x7f83b0006210 comp rx=0 tx=0).stop 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 -- 172.21.15.46:0/2946877064 shutdown_connections 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2dab0720 0x562a2dab43f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2daaa890 0x562a2daaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2946877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2daa4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.217+0000 7f83d3088b80 1 -- 172.21.15.46:0/2946877064 >> 172.21.15.46:0/2946877064 conn(0x562a2da08210 msgr2=0x562a2dab2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.218+0000 7f83d3088b80 1 -- 172.21.15.46:0/2946877064 shutdown_connections 2024-05-08T23:22:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.218+0000 7f83d3088b80 1 -- 172.21.15.46:0/2946877064 wait complete. 2024-05-08T23:22:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.218+0000 7f83d3088b80 1 -- start start 2024-05-08T23:22:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.219+0000 7f83d3088b80 1 Processor -- start 2024-05-08T23:22:59.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2db15b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2dab0720 0x562a2db160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2dab91e0 con 0x562a2daafe00 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2dab9350 con 0x562a2daaa890 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2dab9510 con 0x562a2dab0720 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.221+0000 7f83d3088b80 10 client.0 _ll_register_callbacks cb 0x562a2d99fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56160/0 (socket says 172.21.15.46:56160) 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 -- 172.21.15.46:0/2364841236 learned_addr learned my addr 172.21.15.46:0/2364841236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83b7e00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2db15b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bd600700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2dab0720 0x562a2db160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 -- 172.21.15.46:0/2364841236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2dab0720 msgr2=0x562a2db160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2dab0720 0x562a2db160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 -- 172.21.15.46:0/2364841236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 msgr2=0x562a2db15b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2db15b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.222+0000 7f83bcc00700 1 -- 172.21.15.46:0/2364841236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a2d87b3b0 con 0x562a2daaa890 2024-05-08T23:22:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.223+0000 7f83b7e00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2db15b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.223+0000 7f83bcc00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83ac004200 tx=0x7f83ac0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c758816be9d30a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.223+0000 7f839e000700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:22:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.223+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac023070 con 0x562a2daaa890 2024-05-08T23:22:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.224+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83b0007db0 con 0x562a2daaa890 2024-05-08T23:22:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.224+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f83ac003800 con 0x562a2daaa890 2024-05-08T23:22:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.226+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ac0039c0 con 0x562a2daaa890 2024-05-08T23:22:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.226+0000 7f83d3088b80 10 client.30521 fetch_fsmap learned FSMap version 36 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.226+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562a2d9c0560 con 0x562a2daaa890 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b0007db0 con 0x562a2daaa890 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f83ac038540 con 0x562a2daaa890 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 10 client.30521 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 10 client.30521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 20 client.30521 populate_metadata read hostname 'smithi046' 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2db2c080 0x562a2db2e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 4 client.30521 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a2d9f2ec0 con 0x562a2db2c080 2024-05-08T23:22:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.227+0000 7f83b7e00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2db2c080 0x562a2db2e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.228+0000 7f83b7e00700 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2db2c080 0x562a2db2e4a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a266ed6f9d5d4b0 server_cookie=410bf00fe94c9c4c in_seq=0 out_seq=0 2024-05-08T23:22:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.228+0000 7f839e000700 10 client.30521 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.229+0000 7f839e000700 1 -- 172.21.15.46:0/2364841236 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a2d9f2ec0 con 0x562a2db2c080 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.229+0000 7f839e000700 10 client.30521 handle_command_reply: tid=0 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.229+0000 7f83d3088b80 1 client.30521 shutdown 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 msgr2=0x562a2db15630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83ac004200 tx=0x7f83ac0047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 shutdown_connections 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a2dab0720 0x562a2db160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a2daaa890 0x562a2db15630 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x562a2db2c080 0x562a2db2e4a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 --2- 172.21.15.46:0/2364841236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a2daafe00 0x562a2db15b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 >> 172.21.15.46:0/2364841236 conn(0x562a2da08210 msgr2=0x562a2daaec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f839e000700 0 client.30521 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 shutdown_connections 2024-05-08T23:22:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.230+0000 7f83d3088b80 1 -- 172.21.15.46:0/2364841236 wait complete. 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.231+0000 7f83d3088b80 20 client.30521 trim_cache size 0 max 16384 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.231+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 msgr2=0x7f83a00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.231+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 0x7f83a00a3850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009d70 tx=0x7f83ac009260 comp rx=0 tx=0).stop 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.231+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 msgr2=0x7f83b81a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:59.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.231+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400ed00 comp rx=0 tx=0).stop 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 shutdown_connections 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b811a780 0x7f83b81aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f83a00a1390 0x7f83a00a3850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b8108030 0x7f83b81a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 --2- 172.21.15.46:0/881755815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b810f5e0 0x7f83b81a6770 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:59.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.233+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 >> 172.21.15.46:0/881755815 conn(0x7f83b806bf30 msgr2=0x7f83b8118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:59.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.234+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 shutdown_connections 2024-05-08T23:22:59.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:22:59.234+0000 7f83bf600700 1 -- 172.21.15.46:0/881755815 wait complete. 2024-05-08T23:22:59.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122949 19.86 MB/sec execute 200 sec latency 195.883 ms 2024-05-08T23:22:59.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:22:59 smithi046 ceph-mon[36898]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 366 op/s 2024-05-08T23:22:59.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:22:59 smithi053 ceph-mon[77587]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 366 op/s 2024-05-08T23:22:59.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:22:59 smithi086 ceph-mon[78610]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 366 op/s 2024-05-08T23:23:00.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:23:00.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:23:00.279 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123273 19.84 MB/sec execute 201 sec latency 228.898 ms 2024-05-08T23:23:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.622+0000 7fc1a8400700 1 -- 172.21.15.46:0/167014230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a0105080 msgr2=0x7fc1a00694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.622+0000 7fc1a8400700 1 --2- 172.21.15.46:0/167014230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a0105080 0x7fc1a00694d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc190009990 tx=0x7fc190009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 -- 172.21.15.46:0/167014230 shutdown_connections 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 --2- 172.21.15.46:0/167014230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a0105080 0x7fc1a00694d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 --2- 172.21.15.46:0/167014230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a00fff60 0x7fc1a0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 --2- 172.21.15.46:0/167014230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a0100910 0x7fc1a0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 -- 172.21.15.46:0/167014230 >> 172.21.15.46:0/167014230 conn(0x7fc1a00fb7a0 msgr2=0x7fc1a00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 -- 172.21.15.46:0/167014230 shutdown_connections 2024-05-08T23:23:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.624+0000 7fc1a8400700 1 -- 172.21.15.46:0/167014230 wait complete. 2024-05-08T23:23:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.627+0000 7fc1a8400700 1 Processor -- start 2024-05-08T23:23:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.628+0000 7fc1a8400700 1 -- start start 2024-05-08T23:23:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.629+0000 7fc1a8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.629+0000 7fc1a8400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0105080 0x7fc1a0197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a8400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a019bbd0 0x7fc1a019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a8400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a019c650 con 0x7fc1a0105080 2024-05-08T23:23:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a8400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a019c7c0 con 0x7fc1a019bbd0 2024-05-08T23:23:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a019c980 con 0x7fc1a00fff60 2024-05-08T23:23:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42080/0 (socket says 172.21.15.46:42080) 2024-05-08T23:23:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a5a00700 1 -- 172.21.15.46:0/531365190 learned_addr learned my addr 172.21.15.46:0/531365190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a5000700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0105080 0x7fc1a0197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.630+0000 7fc1a6400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a019bbd0 0x7fc1a019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 -- 172.21.15.46:0/531365190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a019bbd0 msgr2=0x7fc1a019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a019bbd0 0x7fc1a019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 -- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0105080 msgr2=0x7fc1a0197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0105080 0x7fc1a0197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 -- 172.21.15.46:0/531365190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009c40 con 0x7fc1a00fff60 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.631+0000 7fc1a5a00700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c370 tx=0x7fc19800c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.632+0000 7fc19e000700 1 -- 172.21.15.46:0/531365190 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198017070 con 0x7fc1a00fff60 2024-05-08T23:23:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.632+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1900095f0 con 0x7fc1a00fff60 2024-05-08T23:23:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.632+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a0106fa0 con 0x7fc1a00fff60 2024-05-08T23:23:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.632+0000 7fc19e000700 1 -- 172.21.15.46:0/531365190 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19800f040 con 0x7fc1a00fff60 2024-05-08T23:23:00.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.632+0000 7fc19e000700 1 -- 172.21.15.46:0/531365190 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e050 con 0x7fc1a00fff60 2024-05-08T23:23:00.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.638+0000 7fc19e000700 1 -- 172.21.15.46:0/531365190 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc198004440 con 0x7fc1a00fff60 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.640+0000 7fc1bbffdb80 1 Processor -- start 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.640+0000 7fc1bbffdb80 1 -- start start 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc19e000700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 0x7fc1840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558092fe1d0 con 0x5558093ace80 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558092fbb90 con 0x5558093a7910 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.641+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558092f7480 con 0x5558093ad7a0 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.642+0000 7fc19e000700 1 -- 172.21.15.46:0/531365190 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc1980cc820 con 0x7fc1a00fff60 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.642+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.642+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56182/0 (socket says 172.21.15.46:56182) 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.642+0000 7fc1a6400700 1 -- 172.21.15.46:0/1640741002 learned_addr learned my addr 172.21.15.46:0/1640741002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:00.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.642+0000 7fc1a5000700 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a6400700 1 -- 172.21.15.46:0/1640741002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 msgr2=0x5558093a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a6400700 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a6400700 1 -- 172.21.15.46:0/1640741002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 msgr2=0x5558093ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a6400700 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a5a00700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 0x7fc1840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.643+0000 7fc1a6400700 1 -- 172.21.15.46:0/1640741002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555809178430 con 0x5558093a7910 2024-05-08T23:23:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.646+0000 7fc1a6400700 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b6f0 tx=0x7fc190005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc16fefec32552e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.646+0000 7fc1a5a00700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 0x7fc1840a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1980c9c10 tx=0x7fc19800e1b0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.649+0000 7fc196a00700 1 -- 172.21.15.46:0/1640741002 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000b750 con 0x5558093a7910 2024-05-08T23:23:00.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.649+0000 7fc196a00700 1 -- 172.21.15.46:0/1640741002 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19000bd70 con 0x5558093a7910 2024-05-08T23:23:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.649+0000 7fc196a00700 1 -- 172.21.15.46:0/1640741002 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000fcd0 con 0x5558093a7910 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/1640741002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 msgr2=0x5558093a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b6f0 tx=0x7fc190005f00 comp rx=0 tx=0).stop 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/1640741002 shutdown_connections 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x5558093a7d50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/1640741002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/1640741002 >> 172.21.15.46:0/1640741002 conn(0x555809305290 msgr2=0x5558093a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/1640741002 shutdown_connections 2024-05-08T23:23:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.650+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/1640741002 wait complete. 2024-05-08T23:23:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.651+0000 7fc1bbffdb80 1 -- start start 2024-05-08T23:23:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.652+0000 7fc1bbffdb80 1 Processor -- start 2024-05-08T23:23:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ace80 0x5558093b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558092935e0 con 0x5558093a7910 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555809293280 con 0x5558093ad7a0 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55580917a720 con 0x5558093ace80 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1bbffdb80 10 client.0 _ll_register_callbacks cb 0x55580929cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.653+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37932/0 (socket says 172.21.15.46:37932) 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 -- 172.21.15.46:0/2026367745 learned_addr learned my addr 172.21.15.46:0/2026367745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 -- 172.21.15.46:0/2026367745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ace80 msgr2=0x5558093b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ace80 0x5558093b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 -- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 msgr2=0x5558093b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.654+0000 7fc1a6400700 1 -- 172.21.15.46:0/2026367745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc190020040 con 0x5558093a7910 2024-05-08T23:23:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.655+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc190009990 tx=0x7fc19000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6376496047bf0404 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.655+0000 7fc18b400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.658+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190032070 con 0x5558093a7910 2024-05-08T23:23:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.658+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc190000e40 con 0x5558093a7910 2024-05-08T23:23:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.658+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558092502f0 con 0x5558093a7910 2024-05-08T23:23:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.659+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19000bb10 con 0x5558093a7910 2024-05-08T23:23:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.659+0000 7fc1bbffdb80 10 client.20286 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.659+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555809124be0 con 0x5558093a7910 2024-05-08T23:23:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558092502f0 con 0x5558093a7910 2024-05-08T23:23:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc190035020 con 0x5558093a7910 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 10 client.20286 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 10 client.20286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 20 client.20286 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5558093c95f0 0x5558093cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 4 client.20286 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.660+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555809376890 con 0x5558093c95f0 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.661+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5558093c95f0 0x5558093cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.662+0000 7fc18b400700 10 client.20286 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.662+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5558093c95f0 0x5558093cb9e0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c01db2c4bcedb8 server_cookie=cad86bef61cfede in_seq=0 out_seq=0 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.663+0000 7fc18b400700 1 -- 172.21.15.46:0/2026367745 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555809376890 con 0x5558093c95f0 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.663+0000 7fc18b400700 10 client.20286 handle_command_reply: tid=0 2024-05-08T23:23:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.663+0000 7fc1bbffdb80 1 client.20286 shutdown 2024-05-08T23:23:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.664+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 msgr2=0x5558093b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.664+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc190009990 tx=0x7fc19000f740 comp rx=0 tx=0).stop 2024-05-08T23:23:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 shutdown_connections 2024-05-08T23:23:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5558093c95f0 0x5558093cb9e0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093a7910 0x5558093b0c10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2026367745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ace80 0x5558093b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 >> 172.21.15.46:0/2026367745 conn(0x555809305290 msgr2=0x5558093a1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc18b400700 0 client.20286 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 shutdown_connections 2024-05-08T23:23:00.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.665+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2026367745 wait complete. 2024-05-08T23:23:00.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.666+0000 7fc1bbffdb80 20 client.20286 trim_cache size 0 max 16384 2024-05-08T23:23:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.704+0000 7fc1bbffdb80 1 Processor -- start 2024-05-08T23:23:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.705+0000 7fc1bbffdb80 1 -- start start 2024-05-08T23:23:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093c6620 con 0x5558093ace80 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093b3710 con 0x5558093ad7a0 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093b3e40 con 0x5558093a7910 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42116/0 (socket says 172.21.15.46:42116) 2024-05-08T23:23:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.706+0000 7fc1a6400700 1 -- 172.21.15.46:0/719701855 learned_addr learned my addr 172.21.15.46:0/719701855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.707+0000 7fc1a5000700 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.707+0000 7fc1a6400700 1 -- 172.21.15.46:0/719701855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 msgr2=0x5558093ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a6400700 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a6400700 1 -- 172.21.15.46:0/719701855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 msgr2=0x5558093cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a6400700 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a6400700 1 -- 172.21.15.46:0/719701855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558092bd5e0 con 0x5558093a7910 2024-05-08T23:23:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a5000700 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.708+0000 7fc1a6400700 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc190000c00 tx=0x7fc190005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d4b07befca7abf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.709+0000 7fc197400700 1 -- 172.21.15.46:0/719701855 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190032070 con 0x5558093a7910 2024-05-08T23:23:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.710+0000 7fc197400700 1 -- 172.21.15.46:0/719701855 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19002de30 con 0x5558093a7910 2024-05-08T23:23:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.710+0000 7fc197400700 1 -- 172.21.15.46:0/719701855 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000fac0 con 0x5558093a7910 2024-05-08T23:23:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/719701855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 msgr2=0x5558093a73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc190000c00 tx=0x7fc190005e80 comp rx=0 tx=0).stop 2024-05-08T23:23:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/719701855 shutdown_connections 2024-05-08T23:23:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093ad7a0 0x5558093ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093a7910 0x5558093a73d0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/719701855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558093cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.712+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/719701855 >> 172.21.15.46:0/719701855 conn(0x555809305290 msgr2=0x5558093abce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.713+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/719701855 shutdown_connections 2024-05-08T23:23:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.713+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/719701855 wait complete. 2024-05-08T23:23:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.713+0000 7fc1bbffdb80 1 -- start start 2024-05-08T23:23:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.714+0000 7fc1bbffdb80 1 Processor -- start 2024-05-08T23:23:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x555809418d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558094192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093c0b00 con 0x5558093ace80 2024-05-08T23:23:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093c0c40 con 0x5558093a7910 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558093c0e00 con 0x5558093ad7a0 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.716+0000 7fc1bbffdb80 10 client.0 _ll_register_callbacks cb 0x55580929cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42132/0 (socket says 172.21.15.46:42132) 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 -- 172.21.15.46:0/2595811447 learned_addr learned my addr 172.21.15.46:0/2595811447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x555809418d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558094192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 -- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 msgr2=0x555809418d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x555809418d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.717+0000 7fc1a6400700 1 -- 172.21.15.46:0/2595811447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 msgr2=0x5558094192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.718+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558094192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.718+0000 7fc1a6400700 1 -- 172.21.15.46:0/2595811447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555809124be0 con 0x5558093ad7a0 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.718+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558094192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.718+0000 7fc1a6400700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc190005f50 tx=0x7fc1900058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad061f9725f22551 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.718+0000 7fc18b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.722+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190032070 con 0x5558093ad7a0 2024-05-08T23:23:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.722+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc190000e40 con 0x5558093ad7a0 2024-05-08T23:23:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.723+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc19000bd70 con 0x5558093ad7a0 2024-05-08T23:23:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.724+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19000bb10 con 0x5558093ad7a0 2024-05-08T23:23:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.725+0000 7fc1bbffdb80 10 client.30547 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.725+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55580917cd50 con 0x5558093ad7a0 2024-05-08T23:23:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19000bd70 con 0x5558093ad7a0 2024-05-08T23:23:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc190035070 con 0x5558093ad7a0 2024-05-08T23:23:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 10 client.30547 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 10 client.30547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 20 client.30547 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555809420760 0x555809422b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 4 client.30547 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.726+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5558092eff40 con 0x555809420760 2024-05-08T23:23:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.728+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555809420760 0x555809422b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.729+0000 7fc18b400700 10 client.30547 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.729+0000 7fc1a5000700 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555809420760 0x555809422b50 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403459ef5928948a server_cookie=8218b894419b67c5 in_seq=0 out_seq=0 2024-05-08T23:23:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.731+0000 7fc18b400700 1 -- 172.21.15.46:0/2595811447 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5558092eff40 con 0x555809420760 2024-05-08T23:23:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.731+0000 7fc18b400700 10 client.30547 handle_command_reply: tid=0 2024-05-08T23:23:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.731+0000 7fc1bbffdb80 1 client.30547 shutdown 2024-05-08T23:23:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 msgr2=0x5558093c0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc190005f50 tx=0x7fc1900058e0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 shutdown_connections 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558093ad7a0 0x5558093c0420 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558093a7910 0x555809418d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x555809420760 0x555809422b50 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 --2- 172.21.15.46:0/2595811447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558093ace80 0x5558094192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc18b400700 0 client.30547 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.732+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 >> 172.21.15.46:0/2595811447 conn(0x555809305290 msgr2=0x5558093b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.733+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 shutdown_connections 2024-05-08T23:23:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.733+0000 7fc1bbffdb80 1 -- 172.21.15.46:0/2595811447 wait complete. 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.733+0000 7fc1bbffdb80 20 client.30547 trim_cache size 0 max 16384 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a838da62-1593-4fd9-8414-03e8ada53b12", 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:23:00.734 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:23:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.735+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 msgr2=0x7fc1840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.736+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 0x7fc1840a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1980c9c10 tx=0x7fc19800e1b0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.736+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 msgr2=0x7fc1a01972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.736+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c370 tx=0x7fc19800c680 comp rx=0 tx=0).stop 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 shutdown_connections 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1a019bbd0 0x7fc1a019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1a00fff60 0x7fc1a01972f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc1840a1290 0x7fc1840a3750 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 --2- 172.21.15.46:0/531365190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0105080 0x7fc1a0197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.737+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 >> 172.21.15.46:0/531365190 conn(0x7fc1a00fb7a0 msgr2=0x7fc1a0103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.739+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 shutdown_connections 2024-05-08T23:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:00.740+0000 7fc1a8400700 1 -- 172.21.15.46:0/531365190 wait complete. 2024-05-08T23:23:00.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 -- 172.21.15.46:0/2452785466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc100590 msgr2=0x7f55dc10d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 --2- 172.21.15.46:0/2452785466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc100590 0x7f55dc10d250 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009f90 tx=0x7f55cc00b330 comp rx=0 tx=0).stop 2024-05-08T23:23:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 -- 172.21.15.46:0/2452785466 shutdown_connections 2024-05-08T23:23:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 --2- 172.21.15.46:0/2452785466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 0x7f55dc10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 --2- 172.21.15.46:0/2452785466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 --2- 172.21.15.46:0/2452785466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc100590 0x7f55dc10d250 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.174+0000 7f55e3800700 1 -- 172.21.15.46:0/2452785466 >> 172.21.15.46:0/2452785466 conn(0x7f55dc0fb800 msgr2=0x7f55dc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.175+0000 7f55e3800700 1 -- 172.21.15.46:0/2452785466 shutdown_connections 2024-05-08T23:23:01.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.175+0000 7f55e3800700 1 -- 172.21.15.46:0/2452785466 wait complete. 2024-05-08T23:23:01.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.177+0000 7f55e3800700 1 Processor -- start 2024-05-08T23:23:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.177+0000 7f55e3800700 1 -- start start 2024-05-08T23:23:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc100590 0x7f55dc102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 0x7f55dc102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1a9d80 con 0x7f55dc10d790 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc199490 con 0x7f55dc100590 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc199650 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.178+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:42150/0 (socket says 172.21.15.46:42150) 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 -- 172.21.15.46:0/3867648273 learned_addr learned my addr 172.21.15.46:0/3867648273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e1800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 0x7f55dc102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 -- 172.21.15.46:0/3867648273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc100590 msgr2=0x7f55dc102400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:23:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc100590 0x7f55dc102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 -- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 msgr2=0x7f55dc102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 0x7f55dc102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.179+0000 7f55e0e00700 1 -- 172.21.15.46:0/3867648273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc009c40 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.181+0000 7f55e0e00700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9c0 tx=0x7f55d000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.181+0000 7f55d9600700 1 -- 172.21.15.46:0/3867648273 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000cbd0 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.181+0000 7f55d9600700 1 -- 172.21.15.46:0/3867648273 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d000cd30 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.181+0000 7f55d9600700 1 -- 172.21.15.46:0/3867648273 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0005580 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.182+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc199930 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.182+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc199df0 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.188+0000 7f55d9600700 1 -- 172.21.15.46:0/3867648273 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f55d0004750 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.191+0000 7f55d9600700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 0x7f55c40a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.191+0000 7f55d9600700 1 -- 172.21.15.46:0/3867648273 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f55d0014070 con 0x7f55dc0ffbe0 2024-05-08T23:23:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.192+0000 7f55dbe00700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 0x7f55c40a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.193+0000 7f55f72f9b80 1 Processor -- start 2024-05-08T23:23:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.193+0000 7f55dbe00700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 0x7f55c40a3920 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f55dc1035f0 tx=0x7f55cc000f40 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 -- start start 2024-05-08T23:23:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e482e00 0x55b40e4831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e3d4150 con 0x55b40e483720 2024-05-08T23:23:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e3d1b10 con 0x55b40e47d890 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.194+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e3cd400 con 0x55b40e482e00 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.195+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.195+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37982/0 (socket says 172.21.15.46:37982) 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.195+0000 7f55dbe00700 1 -- 172.21.15.46:0/195390011 learned_addr learned my addr 172.21.15.46:0/195390011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.195+0000 7f55e0e00700 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e482e00 0x55b40e4831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.195+0000 7f55e1800700 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 -- 172.21.15.46:0/195390011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e482e00 msgr2=0x55b40e4831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e482e00 0x55b40e4831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 -- 172.21.15.46:0/195390011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 msgr2=0x55b40e47d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 -- 172.21.15.46:0/195390011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40e24e3b0 con 0x55b40e47d890 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.196+0000 7f55e1800700 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47dcd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009b30 tx=0x7f55d4009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e047bc778a54bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.197+0000 7f55ca000700 1 -- 172.21.15.46:0/195390011 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4010ac0 con 0x55b40e47d890 2024-05-08T23:23:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.197+0000 7f55ca000700 1 -- 172.21.15.46:0/195390011 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d4015070 con 0x55b40e47d890 2024-05-08T23:23:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.197+0000 7f55ca000700 1 -- 172.21.15.46:0/195390011 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d40116a0 con 0x55b40e47d890 2024-05-08T23:23:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.198+0000 7f55f72f9b80 1 -- 172.21.15.46:0/195390011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 msgr2=0x55b40e47dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.198+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47dcd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009b30 tx=0x7f55d4009e40 comp rx=0 tx=0).stop 2024-05-08T23:23:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.199+0000 7f55f72f9b80 1 -- 172.21.15.46:0/195390011 shutdown_connections 2024-05-08T23:23:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.200+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47dcd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.200+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e482e00 0x55b40e4831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.200+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/195390011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.200+0000 7f55f72f9b80 1 -- 172.21.15.46:0/195390011 >> 172.21.15.46:0/195390011 conn(0x55b40e3db210 msgr2=0x55b40e477df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.201+0000 7f55f72f9b80 1 -- 172.21.15.46:0/195390011 shutdown_connections 2024-05-08T23:23:01.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.202+0000 7f55f72f9b80 1 -- 172.21.15.46:0/195390011 wait complete. 2024-05-08T23:23:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.203+0000 7f55f72f9b80 1 -- start start 2024-05-08T23:23:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.203+0000 7f55f72f9b80 1 Processor -- start 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.206+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e486b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.206+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e487060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.206+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e4875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e369560 con 0x55b40e482e00 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e369200 con 0x55b40e47d890 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e2506a0 con 0x55b40e483720 2024-05-08T23:23:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55f72f9b80 10 client.0 _ll_register_callbacks cb 0x55b40e372b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e486b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55e1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e4875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e486b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56224/0 (socket says 172.21.15.46:56224) 2024-05-08T23:23:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55e0e00700 1 -- 172.21.15.46:0/1463970005 learned_addr learned my addr 172.21.15.46:0/1463970005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.207+0000 7f55dbe00700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e487060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.208+0000 7f55e1800700 1 -- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 msgr2=0x55b40e486b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.208+0000 7f55e1800700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e486b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.208+0000 7f55e1800700 1 -- 172.21.15.46:0/1463970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 msgr2=0x55b40e487060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.208+0000 7f55e1800700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e487060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.208+0000 7f55e1800700 1 -- 172.21.15.46:0/1463970005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d4009810 con 0x55b40e483720 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.209+0000 7f55e1800700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e4875a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f55d40097b0 tx=0x7f55d4004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824705f48096275a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.209+0000 7f55bea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4020040 con 0x55b40e483720 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f55d4017070 con 0x55b40e483720 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b40e326270 con 0x55b40e483720 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d401c890 con 0x55b40e483720 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55f72f9b80 10 client.30559 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.210+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b40e1fab60 con 0x55b40e483720 2024-05-08T23:23:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.211+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b40e326270 con 0x55b40e483720 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.211+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55d402d070 con 0x55b40e483720 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 10 client.30559 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 10 client.30559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 20 client.30559 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4a7fb0 0x55b40e4aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 4 client.30559 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.212+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b40e44c810 con 0x55b40e4a7fb0 2024-05-08T23:23:01.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.215+0000 7f55e0e00700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4a7fb0 0x55b40e4aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.217+0000 7f55e0e00700 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4a7fb0 0x55b40e4aa3d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ae52bcf0bb05d5 server_cookie=c993fbdcfeac4424 in_seq=0 out_seq=0 2024-05-08T23:23:01.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.217+0000 7f55bea00700 10 client.30559 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.218+0000 7f55bea00700 1 -- 172.21.15.46:0/1463970005 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b40e44c810 con 0x55b40e4a7fb0 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.218+0000 7f55bea00700 10 client.30559 handle_command_reply: tid=0 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.218+0000 7f55f72f9b80 1 client.30559 shutdown 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 msgr2=0x55b40e4875a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e4875a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f55d40097b0 tx=0x7f55d4004440 comp rx=0 tx=0).stop 2024-05-08T23:23:01.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 shutdown_connections 2024-05-08T23:23:01.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e4875a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e486b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4a7fb0 0x55b40e4aa3d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1463970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e487060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 >> 172.21.15.46:0/1463970005 conn(0x55b40e3db210 msgr2=0x55b40e4775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55bea00700 0 client.30559 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.219+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 shutdown_connections 2024-05-08T23:23:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.220+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1463970005 wait complete. 2024-05-08T23:23:01.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.221+0000 7f55f72f9b80 20 client.30559 trim_cache size 0 max 16384 2024-05-08T23:23:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.254+0000 7f55f72f9b80 1 Processor -- start 2024-05-08T23:23:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.255+0000 7f55f72f9b80 1 -- start start 2024-05-08T23:23:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.255+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.255+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e477d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.255+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e49f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.256+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56236/0 (socket says 172.21.15.46:56236) 2024-05-08T23:23:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e0e00700 1 -- 172.21.15.46:0/1098312275 learned_addr learned my addr 172.21.15.46:0/1098312275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.256+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e326270 con 0x55b40e482e00 2024-05-08T23:23:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.256+0000 7f55e1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e49f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e488120 con 0x55b40e47d890 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e479e50 con 0x55b40e483720 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.256+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e477d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e1800700 1 -- 172.21.15.46:0/1098312275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 msgr2=0x55b40e47d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e1800700 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e1800700 1 -- 172.21.15.46:0/1098312275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 msgr2=0x55b40e477d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e1800700 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e477d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.257+0000 7f55e1800700 1 -- 172.21.15.46:0/1098312275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40e393560 con 0x55b40e483720 2024-05-08T23:23:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.258+0000 7f55e1800700 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e49f130 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e660 tx=0x7f55d4009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6794f6752c079 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.259+0000 7f55caa00700 1 -- 172.21.15.46:0/1098312275 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4020040 con 0x55b40e483720 2024-05-08T23:23:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.259+0000 7f55caa00700 1 -- 172.21.15.46:0/1098312275 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d4005d50 con 0x55b40e483720 2024-05-08T23:23:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.259+0000 7f55caa00700 1 -- 172.21.15.46:0/1098312275 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d401c890 con 0x55b40e483720 2024-05-08T23:23:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 msgr2=0x55b40e49f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e49f130 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e660 tx=0x7f55d4009e40 comp rx=0 tx=0).stop 2024-05-08T23:23:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 shutdown_connections 2024-05-08T23:23:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e483720 0x55b40e49f130 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e47d890 0x55b40e47d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/1098312275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e482e00 0x55b40e477d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.260+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 >> 172.21.15.46:0/1098312275 conn(0x55b40e3db210 msgr2=0x55b40e480c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.261+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 shutdown_connections 2024-05-08T23:23:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.261+0000 7f55f72f9b80 1 -- 172.21.15.46:0/1098312275 wait complete. 2024-05-08T23:23:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.261+0000 7f55f72f9b80 1 -- start start 2024-05-08T23:23:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.262+0000 7f55f72f9b80 1 Processor -- start 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e47d890 0x55b40e49f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e47f460 con 0x55b40e483720 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e47f5d0 con 0x55b40e482e00 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40e47f790 con 0x55b40e47d890 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55f72f9b80 10 client.0 _ll_register_callbacks cb 0x55b40e372b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.263+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56244/0 (socket says 172.21.15.46:56244) 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 -- 172.21.15.46:0/285395601 learned_addr learned my addr 172.21.15.46:0/285395601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55e0e00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e47d890 0x55b40e49f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 -- 172.21.15.46:0/285395601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e47d890 msgr2=0x55b40e49f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e47d890 0x55b40e49f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 -- 172.21.15.46:0/285395601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 msgr2=0x55b40e47eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:01.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 -- 172.21.15.46:0/285395601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b40e1fab60 con 0x55b40e482e00 2024-05-08T23:23:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.264+0000 7f55dbe00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b40e479bd0 tx=0x7f55cc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b102fd5bb5ac53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.265+0000 7f55bea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:23:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.265+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00e040 con 0x55b40e482e00 2024-05-08T23:23:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.265+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f55cc007de0 con 0x55b40e482e00 2024-05-08T23:23:01.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.265+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55d4015070 con 0x55b40e482e00 2024-05-08T23:23:01.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.266+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55cc02b480 con 0x55b40e482e00 2024-05-08T23:23:01.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.266+0000 7f55f72f9b80 10 client.30548 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:01.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.266+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b40e252cd0 con 0x55b40e482e00 2024-05-08T23:23:01.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.269+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d4015070 con 0x55b40e482e00 2024-05-08T23:23:01.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55cc01b8a0 con 0x55b40e482e00 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 10 client.30548 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 10 client.30548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 20 client.30548 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4fe9f0 0x55b40e500e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 4 client.30548 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b40e3c5ec0 con 0x55b40e4fe9f0 2024-05-08T23:23:01.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.270+0000 7f55e0e00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4fe9f0 0x55b40e500e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:01.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.272+0000 7f55e0e00700 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4fe9f0 0x55b40e500e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3fa596fa2587b9 server_cookie=728a9940087dca3c in_seq=0 out_seq=0 2024-05-08T23:23:01.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.272+0000 7f55bea00700 10 client.30548 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:01.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.272+0000 7f55bea00700 1 -- 172.21.15.46:0/285395601 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b40e3c5ec0 con 0x55b40e4fe9f0 2024-05-08T23:23:01.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.272+0000 7f55bea00700 10 client.30548 handle_command_reply: tid=0 2024-05-08T23:23:01.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.273+0000 7f55f72f9b80 1 client.30548 shutdown 2024-05-08T23:23:01.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 msgr2=0x55b40e47e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b40e479bd0 tx=0x7f55cc009920 comp rx=0 tx=0).stop 2024-05-08T23:23:01.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 shutdown_connections 2024-05-08T23:23:01.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b40e4fe9f0 0x55b40e500e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b40e483720 0x55b40e47eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40e47d890 0x55b40e49f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 --2- 172.21.15.46:0/285395601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b40e482e00 0x55b40e47e9a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 >> 172.21.15.46:0/285395601 conn(0x55b40e3db210 msgr2=0x55b40e4853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55bea00700 0 client.30548 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.274+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 shutdown_connections 2024-05-08T23:23:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.276+0000 7f55f72f9b80 1 -- 172.21.15.46:0/285395601 wait complete. 2024-05-08T23:23:01.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.276+0000 7f55f72f9b80 20 client.30548 trim_cache size 0 max 16384 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "a838da62-1593-4fd9-8414-03e8ada53b12": { 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a838da62-1593-4fd9-8414-03e8ada53b12", 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:23:01.278 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:23:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 msgr2=0x7f55c40a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 0x7f55c40a3920 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f55dc1035f0 tx=0x7f55cc000f40 comp rx=0 tx=0).stop 2024-05-08T23:23:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 msgr2=0x7f55dc101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:01.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123520 19.82 MB/sec execute 202 sec latency 316.708 ms 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9c0 tx=0x7f55d000ed80 comp rx=0 tx=0).stop 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 shutdown_connections 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f55c40a1460 0x7f55c40a3920 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55dc10d790 0x7f55dc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55dc0ffbe0 0x7f55dc101ec0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 --2- 172.21.15.46:0/3867648273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55dc100590 0x7f55dc102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.279+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 >> 172.21.15.46:0/3867648273 conn(0x7f55dc0fb800 msgr2=0x7f55dc0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.280+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 shutdown_connections 2024-05-08T23:23:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:01.280+0000 7f55e3800700 1 -- 172.21.15.46:0/3867648273 wait complete. 2024-05-08T23:23:01.317 INFO:tasks.cephfs.filesystem:scrub status for tag:a838da62-1593-4fd9-8414-03e8ada53b12 - {'path': '/', 'tag': 'a838da62-1593-4fd9-8414-03e8ada53b12', 'options': 'recursive,force'} 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:01.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:01 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:01.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:01.965 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:01 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:02.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:01 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123786 19.79 MB/sec execute 203 sec latency 342.594 ms 2024-05-08T23:23:02.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:02 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:02.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:02 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:02 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:02.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:02.966 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:02 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:03.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:02 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:03.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:03.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:02 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:03.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123900 19.72 MB/sec execute 204 sec latency 506.733 ms 2024-05-08T23:23:03.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:03 smithi086 ceph-mon[78610]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 474 op/s 2024-05-08T23:23:04.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:03 smithi053 ceph-mon[77587]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 474 op/s 2024-05-08T23:23:04.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:03 smithi046 ceph-mon[36898]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 474 op/s 2024-05-08T23:23:04.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123923 19.64 MB/sec execute 205 sec latency 468.177 ms 2024-05-08T23:23:05.280 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123930 19.56 MB/sec execute 206 sec latency 347.423 ms 2024-05-08T23:23:05.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:05 smithi086 ceph-mon[78610]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 508 op/s 2024-05-08T23:23:06.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:05 smithi053 ceph-mon[77587]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 508 op/s 2024-05-08T23:23:06.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:05 smithi046 ceph-mon[36898]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 508 op/s 2024-05-08T23:23:06.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 124034 19.49 MB/sec execute 207 sec latency 295.937 ms 2024-05-08T23:23:07.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 124417 19.52 MB/sec execute 208 sec latency 248.785 ms 2024-05-08T23:23:07.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:07 smithi086 ceph-mon[78610]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 732 op/s 2024-05-08T23:23:08.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:07 smithi053 ceph-mon[77587]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 732 op/s 2024-05-08T23:23:08.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:07 smithi046 ceph-mon[36898]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 732 op/s 2024-05-08T23:23:08.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 124749 19.51 MB/sec execute 209 sec latency 192.936 ms 2024-05-08T23:23:09.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 125126 19.48 MB/sec execute 210 sec latency 176.825 ms 2024-05-08T23:23:09.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:09 smithi086 ceph-mon[78610]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 538 op/s 2024-05-08T23:23:10.011 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:09 smithi053 ceph-mon[77587]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 538 op/s 2024-05-08T23:23:10.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:09 smithi046 ceph-mon[36898]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 538 op/s 2024-05-08T23:23:10.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 125558 19.51 MB/sec execute 211 sec latency 232.153 ms 2024-05-08T23:23:11.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 126012 19.52 MB/sec execute 212 sec latency 179.862 ms 2024-05-08T23:23:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:11 smithi086 ceph-mon[78610]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 710 op/s 2024-05-08T23:23:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:11 smithi086 ceph-mon[78610]: osdmap e112: 12 total, 12 up, 12 in 2024-05-08T23:23:12.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:11 smithi053 ceph-mon[77587]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 710 op/s 2024-05-08T23:23:12.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:11 smithi053 ceph-mon[77587]: osdmap e112: 12 total, 12 up, 12 in 2024-05-08T23:23:12.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:11 smithi046 ceph-mon[36898]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 710 op/s 2024-05-08T23:23:12.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:11 smithi046 ceph-mon[36898]: osdmap e112: 12 total, 12 up, 12 in 2024-05-08T23:23:12.281 INFO:tasks.workunit.client.0.smithi046.stdout: 10 126480 19.56 MB/sec execute 213 sec latency 163.930 ms 2024-05-08T23:23:13.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 126791 19.55 MB/sec execute 214 sec latency 79.646 ms 2024-05-08T23:23:13.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:13 smithi086 ceph-mon[78610]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-05-08T23:23:14.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:13 smithi053 ceph-mon[77587]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-05-08T23:23:14.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:13 smithi046 ceph-mon[36898]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-05-08T23:23:14.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 127202 19.57 MB/sec execute 215 sec latency 96.877 ms 2024-05-08T23:23:15.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 127635 19.59 MB/sec execute 216 sec latency 346.210 ms 2024-05-08T23:23:15.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:15 smithi086 ceph-mon[78610]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 576 op/s 2024-05-08T23:23:15.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:15.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:15 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:16.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:15 smithi053 ceph-mon[77587]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 576 op/s 2024-05-08T23:23:16.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:16.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:15 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:16.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:15 smithi046 ceph-mon[36898]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.9 MiB/s wr, 576 op/s 2024-05-08T23:23:16.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:16.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:15 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:16.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 128042 19.58 MB/sec execute 217 sec latency 1346.260 ms 2024-05-08T23:23:16.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:16.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:16 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:17.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:17.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:16 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:17.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:17.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:16 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:17.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 128422 19.58 MB/sec execute 218 sec latency 2346.336 ms 2024-05-08T23:23:17.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:17 smithi086 ceph-mon[78610]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:18.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:17 smithi053 ceph-mon[77587]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:18.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:17 smithi046 ceph-mon[36898]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:18.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 128747 19.60 MB/sec execute 219 sec latency 3282.584 ms 2024-05-08T23:23:19.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 129174 19.60 MB/sec execute 220 sec latency 168.794 ms 2024-05-08T23:23:19.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:19 smithi086 ceph-mon[78610]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:20.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:19 smithi053 ceph-mon[77587]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:20.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:19 smithi046 ceph-mon[36898]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-05-08T23:23:20.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 129594 19.58 MB/sec execute 221 sec latency 183.637 ms 2024-05-08T23:23:21.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:20 smithi086 ceph-mon[78610]: osdmap e113: 12 total, 12 up, 12 in 2024-05-08T23:23:21.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:20 smithi053 ceph-mon[77587]: osdmap e113: 12 total, 12 up, 12 in 2024-05-08T23:23:21.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:20 smithi046 ceph-mon[36898]: osdmap e113: 12 total, 12 up, 12 in 2024-05-08T23:23:21.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 130040 19.61 MB/sec execute 222 sec latency 212.935 ms 2024-05-08T23:23:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:21 smithi086 ceph-mon[78610]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 306 op/s 2024-05-08T23:23:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:21 smithi053 ceph-mon[77587]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 306 op/s 2024-05-08T23:23:22.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:21 smithi046 ceph-mon[36898]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 306 op/s 2024-05-08T23:23:22.282 INFO:tasks.workunit.client.0.smithi046.stdout: 10 130565 19.64 MB/sec execute 223 sec latency 177.492 ms 2024-05-08T23:23:23.283 INFO:tasks.workunit.client.0.smithi046.stdout: 10 130990 19.69 MB/sec execute 224 sec latency 245.401 ms 2024-05-08T23:23:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:23 smithi086 ceph-mon[78610]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 324 op/s 2024-05-08T23:23:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:23 smithi053 ceph-mon[77587]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 324 op/s 2024-05-08T23:23:24.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:23 smithi046 ceph-mon[36898]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 324 op/s 2024-05-08T23:23:24.283 INFO:tasks.workunit.client.0.smithi046.stdout: 10 131373 19.69 MB/sec execute 225 sec latency 228.956 ms 2024-05-08T23:23:25.283 INFO:tasks.workunit.client.0.smithi046.stdout: 10 131653 19.67 MB/sec execute 226 sec latency 392.466 ms 2024-05-08T23:23:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:25 smithi086 ceph-mon[78610]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.1 MiB/s wr, 298 op/s 2024-05-08T23:23:26.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:25 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:25 smithi053 ceph-mon[77587]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.1 MiB/s wr, 298 op/s 2024-05-08T23:23:26.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:25 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:25 smithi046 ceph-mon[36898]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.1 MiB/s wr, 298 op/s 2024-05-08T23:23:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:25 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:26.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 131706 19.59 MB/sec execute 227 sec latency 864.019 ms 2024-05-08T23:23:27.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 131929 19.57 MB/sec execute 228 sec latency 293.905 ms 2024-05-08T23:23:28.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:27 smithi086 ceph-mon[78610]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:28.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:27 smithi053 ceph-mon[77587]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:28.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:27 smithi046 ceph-mon[36898]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:28.285 INFO:tasks.workunit.client.0.smithi046.stdout: 10 132211 19.56 MB/sec execute 229 sec latency 182.475 ms 2024-05-08T23:23:29.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 132460 19.54 MB/sec execute 230 sec latency 364.577 ms 2024-05-08T23:23:30.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:29 smithi086 ceph-mon[78610]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:30.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:29 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:30.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:29 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:30.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:29 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:23:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:29 smithi053 ceph-mon[77587]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:29 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:29 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:29 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:23:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:29 smithi046 ceph-mon[36898]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 344 op/s 2024-05-08T23:23:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:29 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:29 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:29 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:23:30.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 132807 19.49 MB/sec execute 231 sec latency 241.571 ms 2024-05-08T23:23:31.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 133016 19.46 MB/sec execute 232 sec latency 441.275 ms 2024-05-08T23:23:31.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 -- 172.21.15.46:0/2413125731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c1140f0 msgr2=0x7fe18c1164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 --2- 172.21.15.46:0/2413125731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c1140f0 0x7fe18c1164e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe174009990 tx=0x7fe174009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 -- 172.21.15.46:0/2413125731 shutdown_connections 2024-05-08T23:23:31.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 --2- 172.21.15.46:0/2413125731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c1140f0 0x7fe18c1164e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 --2- 172.21.15.46:0/2413125731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c107fa0 0x7fe18c108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 --2- 172.21.15.46:0/2413125731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1088c0 0x7fe18c113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 -- 172.21.15.46:0/2413125731 >> 172.21.15.46:0/2413125731 conn(0x7fe18c06bf30 msgr2=0x7fe18c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 -- 172.21.15.46:0/2413125731 shutdown_connections 2024-05-08T23:23:31.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.766+0000 7fe192c00700 1 -- 172.21.15.46:0/2413125731 wait complete. 2024-05-08T23:23:31.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.768+0000 7fe192c00700 1 Processor -- start 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 -- start start 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c1088c0 0x7fe18c110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 0x7fe18c110e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c111480 con 0x7fe18c1140f0 2024-05-08T23:23:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c1115c0 con 0x7fe18c107fa0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe192c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c111780 con 0x7fe18c1088c0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe18b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c1088c0 0x7fe18c110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe18be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe18be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60644/0 (socket says 172.21.15.46:60644) 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.769+0000 7fe18be00700 1 -- 172.21.15.46:0/2486188074 learned_addr learned my addr 172.21.15.46:0/2486188074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe190c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 0x7fe18c110e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 -- 172.21.15.46:0/2486188074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c1088c0 msgr2=0x7fe18c110930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c1088c0 0x7fe18c110930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 -- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 msgr2=0x7fe18c110e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 0x7fe18c110e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 -- 172.21.15.46:0/2486188074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740095f0 con 0x7fe18c107fa0 2024-05-08T23:23:31.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe190c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 0x7fe18c110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe18be00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe184007f10 tx=0x7fe18400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.770+0000 7fe188c00700 1 -- 172.21.15.46:0/2486188074 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184017070 con 0x7fe18c107fa0 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.771+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c1b27a0 con 0x7fe18c107fa0 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.771+0000 7fe188c00700 1 -- 172.21.15.46:0/2486188074 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18400f040 con 0x7fe18c107fa0 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.771+0000 7fe188c00700 1 -- 172.21.15.46:0/2486188074 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184005740 con 0x7fe18c107fa0 2024-05-08T23:23:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.771+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c1b2cb0 con 0x7fe18c107fa0 2024-05-08T23:23:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.777+0000 7fe188c00700 1 -- 172.21.15.46:0/2486188074 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe1840058a0 con 0x7fe18c107fa0 2024-05-08T23:23:31.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.778+0000 7fe1a674ab80 1 Processor -- start 2024-05-08T23:23:31.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.778+0000 7fe1a674ab80 1 -- start start 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853e720 0x55f368538350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368538890 0x55f368538cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36848f150 con 0x55f368538890 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36848cb10 con 0x55f36853de00 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368488400 con 0x55f36853e720 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe190c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe190c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60648/0 (socket says 172.21.15.46:60648) 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe190c00700 1 -- 172.21.15.46:0/408951145 learned_addr learned my addr 172.21.15.46:0/408951145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe18b400700 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368538890 0x55f368538cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.779+0000 7fe190c00700 1 -- 172.21.15.46:0/408951145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853e720 msgr2=0x55f368538350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853e720 0x55f368538350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 -- 172.21.15.46:0/408951145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368538890 msgr2=0x55f368538cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368538890 0x55f368538cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe188c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 0x7fe1700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 -- 172.21.15.46:0/408951145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3683093b0 con 0x55f36853de00 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe188c00700 1 -- 172.21.15.46:0/2486188074 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe1840cc890 con 0x7fe18c107fa0 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe1740095b0 tx=0x7fe17400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82f61da5cc790b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.780+0000 7fe190c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 0x7fe1700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.781+0000 7fe190c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 0x7fe1700a37d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe17400bc60 tx=0x7fe174020000 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.782+0000 7fe181200700 1 -- 172.21.15.46:0/408951145 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174003990 con 0x55f36853de00 2024-05-08T23:23:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.782+0000 7fe181200700 1 -- 172.21.15.46:0/408951145 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe17401e040 con 0x55f36853de00 2024-05-08T23:23:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.782+0000 7fe181200700 1 -- 172.21.15.46:0/408951145 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400fe70 con 0x55f36853de00 2024-05-08T23:23:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.783+0000 7fe1a674ab80 1 -- 172.21.15.46:0/408951145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 msgr2=0x55f36853e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.783+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe1740095b0 tx=0x7fe17400bb80 comp rx=0 tx=0).stop 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 -- 172.21.15.46:0/408951145 shutdown_connections 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368538890 0x55f368538cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853de00 0x55f36853e1e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/408951145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853e720 0x55f368538350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 -- 172.21.15.46:0/408951145 >> 172.21.15.46:0/408951145 conn(0x55f368496210 msgr2=0x55f368532df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 -- 172.21.15.46:0/408951145 shutdown_connections 2024-05-08T23:23:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 -- 172.21.15.46:0/408951145 wait complete. 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.784+0000 7fe1a674ab80 1 -- start start 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 Processor -- start 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f368538890 0x55f368541c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 0x55f368542700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368424560 con 0x55f36853e720 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368424200 con 0x55f368538890 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36830b6a0 con 0x55f36853de00 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.785+0000 7fe1a674ab80 10 client.0 _ll_register_callbacks cb 0x55f36842db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:51764/0 (socket says 172.21.15.46:51764) 2024-05-08T23:23:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 -- 172.21.15.46:0/4132075146 learned_addr learned my addr 172.21.15.46:0/4132075146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18be00700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f368538890 0x55f368541c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe190c00700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 0x55f368542700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 -- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f368538890 msgr2=0x55f368541c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f368538890 0x55f368541c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 -- 172.21.15.46:0/4132075146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 msgr2=0x55f368542700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 0x55f368542700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe18b400700 1 -- 172.21.15.46:0/4132075146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c109c70 con 0x55f36853de00 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.786+0000 7fe190c00700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 0x55f368542700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.787+0000 7fe18b400700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009f00 tx=0x7fe17c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598195bd2f16a3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.787+0000 7fe179600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.792+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c012650 con 0x55f36853de00 2024-05-08T23:23:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.792+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe17c0127b0 con 0x55f36853de00 2024-05-08T23:23:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.792+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3683e1270 con 0x55f36853de00 2024-05-08T23:23:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.793+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17c012970 con 0x55f36853de00 2024-05-08T23:23:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.793+0000 7fe1a674ab80 10 client.30571 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:31.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.793+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f3682b5b60 con 0x55f36853de00 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3683e1270 con 0x55f36853de00 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe17c00c070 con 0x55f36853de00 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 10 client.30571 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 10 client.30571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 20 client.30571 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685523a0 0x55f3685547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 4 client.30571 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f368507810 con 0x55f3685523a0 2024-05-08T23:23:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.794+0000 7fe18be00700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685523a0 0x55f3685547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.795+0000 7fe18be00700 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685523a0 0x55f3685547c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c57d7b384dd9db server_cookie=4b8a672228c989e9 in_seq=0 out_seq=0 2024-05-08T23:23:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.795+0000 7fe179600700 10 client.30571 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.796+0000 7fe179600700 1 -- 172.21.15.46:0/4132075146 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f368507810 con 0x55f3685523a0 2024-05-08T23:23:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.796+0000 7fe179600700 10 client.30571 handle_command_reply: tid=0 2024-05-08T23:23:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.796+0000 7fe1a674ab80 1 client.30571 shutdown 2024-05-08T23:23:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 msgr2=0x55f3685421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009f00 tx=0x7fe17c0116c0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 shutdown_connections 2024-05-08T23:23:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685523a0 0x55f3685547c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f36853e720 0x55f368542700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe179600700 0 client.30571 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f368538890 0x55f368541c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/4132075146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f36853de00 0x55f3685421c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.797+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 >> 172.21.15.46:0/4132075146 conn(0x55f368496210 msgr2=0x55f3685325a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.798+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 shutdown_connections 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.798+0000 7fe1a674ab80 1 -- 172.21.15.46:0/4132075146 wait complete. 2024-05-08T23:23:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.798+0000 7fe1a674ab80 20 client.30571 trim_cache size 0 max 16384 2024-05-08T23:23:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.841+0000 7fe1a674ab80 1 Processor -- start 2024-05-08T23:23:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.842+0000 7fe1a674ab80 1 -- start start 2024-05-08T23:23:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.842+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368538350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368541580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368532d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368534070 con 0x55f368496210 2024-05-08T23:23:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368559990 con 0x55f36853e720 2024-05-08T23:23:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368534c40 con 0x55f368538890 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe18b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368541580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe18b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368541580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47028/0 (socket says 172.21.15.46:47028) 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe18b400700 1 -- 172.21.15.46:0/3732716802 learned_addr learned my addr 172.21.15.46:0/3732716802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.843+0000 7fe190c00700 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368532d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 -- 172.21.15.46:0/3732716802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 msgr2=0x55f368541580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368541580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 -- 172.21.15.46:0/3732716802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 msgr2=0x55f368538350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368538350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 -- 172.21.15.46:0/3732716802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f368538f40 con 0x55f36853e720 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.844+0000 7fe190c00700 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368532d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe174006010 tx=0x7fe174005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7662e085517af8c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.845+0000 7fe181c00700 1 -- 172.21.15.46:0/3732716802 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174003ad0 con 0x55f36853e720 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.845+0000 7fe181c00700 1 -- 172.21.15.46:0/3732716802 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe174003c30 con 0x55f36853e720 2024-05-08T23:23:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.845+0000 7fe181c00700 1 -- 172.21.15.46:0/3732716802 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400f6c0 con 0x55f36853e720 2024-05-08T23:23:31.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 -- 172.21.15.46:0/3732716802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 msgr2=0x55f368532d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368532d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe174006010 tx=0x7fe174005a00 comp rx=0 tx=0).stop 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 -- 172.21.15.46:0/3732716802 shutdown_connections 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368532d20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368538350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/3732716802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368541580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.848+0000 7fe1a674ab80 1 -- 172.21.15.46:0/3732716802 >> 172.21.15.46:0/3732716802 conn(0x55f36854de90 msgr2=0x55f368541c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.849+0000 7fe1a674ab80 1 -- 172.21.15.46:0/3732716802 shutdown_connections 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.849+0000 7fe1a674ab80 1 -- 172.21.15.46:0/3732716802 wait complete. 2024-05-08T23:23:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.849+0000 7fe1a674ab80 1 -- start start 2024-05-08T23:23:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.850+0000 7fe1a674ab80 1 Processor -- start 2024-05-08T23:23:31.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.851+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368549150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368547650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368549a30 con 0x55f368496210 2024-05-08T23:23:31.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3685480d0 con 0x55f36853e720 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f368548290 con 0x55f368538890 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe1a674ab80 10 client.0 _ll_register_callbacks cb 0x55f36842db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe190c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe190c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43270/0 (socket says 172.21.15.46:43270) 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe190c00700 1 -- 172.21.15.46:0/605527743 learned_addr learned my addr 172.21.15.46:0/605527743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe190c00700 1 -- 172.21.15.46:0/605527743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 msgr2=0x55f368547650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.853+0000 7fe18b400700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368547650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe190c00700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368547650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe190c00700 1 -- 172.21.15.46:0/605527743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 msgr2=0x55f368549150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe190c00700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368549150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe190c00700 1 -- 172.21.15.46:0/605527743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3683093b0 con 0x55f36853e720 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe18b400700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368547650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe190c00700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe174005850 tx=0x7fe174009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad8f325472288925 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.854+0000 7fe179600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.855+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17402eb10 con 0x55f36853e720 2024-05-08T23:23:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.855+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe17401e040 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.857+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fe17400f460 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.857+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17400f620 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.857+0000 7fe1a674ab80 10 client.30566 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f36844e560 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe17401e040 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe17404a8b0 con 0x55f36853e720 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe1a674ab80 10 client.30566 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe1a674ab80 10 client.30566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.858+0000 7fe1a674ab80 20 client.30566 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.859+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685a9740 0x55f3685abb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.859+0000 7fe1a674ab80 4 client.30566 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.859+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f368480ec0 con 0x55f3685a9740 2024-05-08T23:23:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.859+0000 7fe18be00700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685a9740 0x55f3685abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.861+0000 7fe18be00700 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685a9740 0x55f3685abb30 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b781b1b3e96f4374 server_cookie=4225fa590e17332d in_seq=0 out_seq=0 2024-05-08T23:23:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.861+0000 7fe179600700 10 client.30566 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:31.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.865+0000 7fe179600700 1 -- 172.21.15.46:0/605527743 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f368480ec0 con 0x55f3685a9740 2024-05-08T23:23:31.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.865+0000 7fe179600700 10 client.30566 handle_command_reply: tid=0 2024-05-08T23:23:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.865+0000 7fe1a674ab80 1 client.30566 shutdown 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 msgr2=0x55f368547b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe174005850 tx=0x7fe174009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 shutdown_connections 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36853e720 0x55f368547b90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55f3685a9740 0x55f3685abb30 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f368496210 0x55f368549150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 --2- 172.21.15.46:0/605527743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f368538890 0x55f368547650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 >> 172.21.15.46:0/605527743 conn(0x55f36854de90 msgr2=0x55f368542460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.866+0000 7fe179600700 0 client.30566 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.867+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 shutdown_connections 2024-05-08T23:23:31.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.867+0000 7fe1a674ab80 1 -- 172.21.15.46:0/605527743 wait complete. 2024-05-08T23:23:31.880 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:23:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:23:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:23:31.881 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.868+0000 7fe1a674ab80 20 client.30566 trim_cache size 0 max 16384 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.869+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 msgr2=0x7fe1700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.869+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 0x7fe1700a37d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe17400bc60 tx=0x7fe174020000 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.869+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 msgr2=0x7fe18c113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.869+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe184007f10 tx=0x7fe18400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 shutdown_connections 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe1700a1310 0x7fe1700a37d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c1140f0 0x7fe18c110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c107fa0 0x7fe18c113810 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 --2- 172.21.15.46:0/2486188074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe18c1088c0 0x7fe18c110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.870+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 >> 172.21.15.46:0/2486188074 conn(0x7fe18c06bf30 msgr2=0x7fe18c06fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.871+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 shutdown_connections 2024-05-08T23:23:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:31.872+0000 7fe192c00700 1 -- 172.21.15.46:0/2486188074 wait complete. 2024-05-08T23:23:31.914 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:23:31.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 -- 172.21.15.46:0/2859552785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341088c0 msgr2=0x7f2034113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 --2- 172.21.15.46:0/2859552785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341088c0 0x7f2034113bb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f20200098f0 tx=0x7f2020009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 -- 172.21.15.46:0/2859552785 shutdown_connections 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 --2- 172.21.15.46:0/2859552785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341140f0 0x7f20341164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 --2- 172.21.15.46:0/2859552785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 --2- 172.21.15.46:0/2859552785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341088c0 0x7f2034113bb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.256+0000 7f2039800700 1 -- 172.21.15.46:0/2859552785 >> 172.21.15.46:0/2859552785 conn(0x7f203406c3a0 msgr2=0x7f203406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.257+0000 7f2039800700 1 -- 172.21.15.46:0/2859552785 shutdown_connections 2024-05-08T23:23:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.257+0000 7f2039800700 1 -- 172.21.15.46:0/2859552785 wait complete. 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.259+0000 7f2039800700 1 Processor -- start 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 -- start start 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341088c0 0x7f2034113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 0x7f20341108c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034111090 con 0x7f20341140f0 2024-05-08T23:23:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034111200 con 0x7f2034107fa0 2024-05-08T23:23:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.260+0000 7f2039800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341113c0 con 0x7f20341088c0 2024-05-08T23:23:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:31 smithi053 ceph-mon[77587]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-05-08T23:23:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.261+0000 7f2032a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.261+0000 7f2032a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43280/0 (socket says 172.21.15.46:43280) 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.261+0000 7f2032a00700 1 -- 172.21.15.46:0/2527187988 learned_addr learned my addr 172.21.15.46:0/2527187988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.261+0000 7f2032000700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341088c0 0x7f2034113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2033400700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 0x7f20341108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2032a00700 1 -- 172.21.15.46:0/2527187988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341088c0 msgr2=0x7f2034113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2032a00700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341088c0 0x7f2034113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2032a00700 1 -- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 msgr2=0x7f20341108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2032a00700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 0x7f20341108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2032a00700 1 -- 172.21.15.46:0/2527187988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009520 con 0x7f2034107fa0 2024-05-08T23:23:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.262+0000 7f2033400700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 0x7f20341108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.263+0000 7f2032a00700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f202400e9c0 tx=0x7f202400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.263+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202400cbd0 con 0x7f2034107fa0 2024-05-08T23:23:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.263+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202400cd30 con 0x7f2034107fa0 2024-05-08T23:23:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.263+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024005590 con 0x7f2034107fa0 2024-05-08T23:23:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.264+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20341b2ab0 con 0x7f2034107fa0 2024-05-08T23:23:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.265+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20341b2e80 con 0x7f2034107fa0 2024-05-08T23:23:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.268+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2034072d30 con 0x7f2034107fa0 2024-05-08T23:23:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.269+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2024004750 con 0x7f2034107fa0 2024-05-08T23:23:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.272+0000 7f202b400700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 0x7f20180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.273+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f2024014070 con 0x7f2034107fa0 2024-05-08T23:23:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.273+0000 7f2032000700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 0x7f20180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.274+0000 7f2032000700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 0x7f20180a37d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2034111f00 tx=0x7f2020019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:31 smithi046 ceph-mon[36898]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-05-08T23:23:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.277+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2024095d10 con 0x7f2034107fa0 2024-05-08T23:23:32.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 133342 19.48 MB/sec execute 233 sec latency 202.982 ms 2024-05-08T23:23:32.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:31 smithi086 ceph-mon[78610]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 452 op/s 2024-05-08T23:23:32.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.747+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f203410aec0 con 0x7f2034107fa0 2024-05-08T23:23:32.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.747+0000 7f202b400700 1 -- 172.21.15.46:0/2527187988 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f2024099330 con 0x7f2034107fa0 2024-05-08T23:23:32.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:23:32.748 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:23:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 msgr2=0x7f20180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 0x7f20180a37d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2034111f00 tx=0x7f2020019040 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 msgr2=0x7f2034113610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f202400e9c0 tx=0x7f202400ed80 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 shutdown_connections 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f20180a1310 0x7f20180a37d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20341140f0 0x7f20341108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2034107fa0 0x7f2034113610 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 --2- 172.21.15.46:0/2527187988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20341088c0 0x7f2034113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 >> 172.21.15.46:0/2527187988 conn(0x7f203406c3a0 msgr2=0x7f203406f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 shutdown_connections 2024-05-08T23:23:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:32.751+0000 7f2039800700 1 -- 172.21.15.46:0/2527187988 wait complete. 2024-05-08T23:23:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:23:32.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:23:33.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:32 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2527187988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 -- 172.21.15.46:0/1927549091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 msgr2=0x7ff7b410ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 --2- 172.21.15.46:0/1927549091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b410ae00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009990 tx=0x7ff7a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 -- 172.21.15.46:0/1927549091 shutdown_connections 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 --2- 172.21.15.46:0/1927549091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b410b3d0 0x7ff7b410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 --2- 172.21.15.46:0/1927549091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b4102000 0x7ff7b41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 --2- 172.21.15.46:0/1927549091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b410ae00 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 -- 172.21.15.46:0/1927549091 >> 172.21.15.46:0/1927549091 conn(0x7ff7b40fb760 msgr2=0x7ff7b40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 -- 172.21.15.46:0/1927549091 shutdown_connections 2024-05-08T23:23:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.050+0000 7ff7bc600700 1 -- 172.21.15.46:0/1927549091 wait complete. 2024-05-08T23:23:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 Processor -- start 2024-05-08T23:23:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 -- start start 2024-05-08T23:23:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b4102000 0x7ff7b4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b410b3d0 0x7ff7b419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b419e440 con 0x7ff7b4102000 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b419e5b0 con 0x7ff7b410b3d0 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.052+0000 7ff7bc600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b419e770 con 0x7ff7b4102920 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7ba600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b410b3d0 0x7ff7b419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47094/0 (socket says 172.21.15.46:47094) 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 -- 172.21.15.46:0/2997339995 learned_addr learned my addr 172.21.15.46:0/2997339995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b4102000 0x7ff7b4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 -- 172.21.15.46:0/2997339995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b410b3d0 msgr2=0x7ff7b419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b410b3d0 0x7ff7b419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 -- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b4102000 msgr2=0x7ff7b4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b4102000 0x7ff7b4199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.053+0000 7ff7b9200700 1 -- 172.21.15.46:0/2997339995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00095f0 con 0x7ff7b4102920 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7b2000700 1 -- 172.21.15.46:0/2997339995 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a001d070 con 0x7ff7b4102920 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7b2000700 1 -- 172.21.15.46:0/2997339995 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a0022470 con 0x7ff7b4102920 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b419e9f0 con 0x7ff7b4102920 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7b2000700 1 -- 172.21.15.46:0/2997339995 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0017650 con 0x7ff7b4102920 2024-05-08T23:23:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.054+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b41a6290 con 0x7ff7b4102920 2024-05-08T23:23:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.058+0000 7ff7b2000700 1 -- 172.21.15.46:0/2997339995 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff7a00225e0 con 0x7ff7b4102920 2024-05-08T23:23:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.059+0000 7ff7d0060b80 1 Processor -- start 2024-05-08T23:23:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.059+0000 7ff7b2000700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 0x7ff79c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.059+0000 7ff7b2000700 1 -- 172.21.15.46:0/2997339995 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7a00ce400 con 0x7ff7b4102920 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.059+0000 7ff7d0060b80 1 -- start start 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.059+0000 7ff7ba600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 0x7ff79c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 0x55be6b6cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6c70a0 0x55be6b6c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b615150 con 0x55be6b6cc610 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b612b10 con 0x55be6b6ccf30 2024-05-08T23:23:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b60e400 con 0x55be6b6c70a0 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43312/0 (socket says 172.21.15.46:43312) 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7b9200700 1 -- 172.21.15.46:0/2102446125 learned_addr learned my addr 172.21.15.46:0/2102446125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7ba600700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6c70a0 0x55be6b6c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.060+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 0x55be6b6cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7ba600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 0x7ff79c0a37d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009550 tx=0x7ff7ac009c20 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 -- 172.21.15.46:0/2102446125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6c70a0 msgr2=0x55be6b6c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6c70a0 0x55be6b6c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 -- 172.21.15.46:0/2102446125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 msgr2=0x55be6b6cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 0x55be6b6cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 -- 172.21.15.46:0/2102446125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6b48f3b0 con 0x55be6b6ccf30 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 0x55be6b6cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0097cc0 tx=0x7ff7a009b020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365d3d416d4cf47f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7a6a00700 1 -- 172.21.15.46:0/2102446125 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00b1020 con 0x55be6b6ccf30 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7a6a00700 1 -- 172.21.15.46:0/2102446125 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a009b950 con 0x55be6b6ccf30 2024-05-08T23:23:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.061+0000 7ff7a6a00700 1 -- 172.21.15.46:0/2102446125 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00ce080 con 0x55be6b6ccf30 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2102446125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 msgr2=0x55be6b6c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0097cc0 tx=0x7ff7a009b020 comp rx=0 tx=0).stop 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2102446125 shutdown_connections 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6c70a0 0x55be6b6c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6cc610 0x55be6b6cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2102446125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6ccf30 0x55be6b6c6b60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2102446125 >> 172.21.15.46:0/2102446125 conn(0x55be6b61c210 msgr2=0x55be6b6b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.062+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2102446125 shutdown_connections 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.063+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2102446125 wait complete. 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.063+0000 7ff7d0060b80 1 -- start start 2024-05-08T23:23:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.063+0000 7ff7d0060b80 1 Processor -- start 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b6d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b6d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b5aa560 con 0x55be6b6c70a0 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b5aa200 con 0x55be6b6cc610 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b4916a0 con 0x55be6b6ccf30 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7d0060b80 10 client.0 _ll_register_callbacks cb 0x55be6b5b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7ba600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b6d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b6d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7ba600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b6d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47122/0 (socket says 172.21.15.46:47122) 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7ba600700 1 -- 172.21.15.46:0/2983041291 learned_addr learned my addr 172.21.15.46:0/2983041291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.064+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 -- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 msgr2=0x55be6b6d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b6d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 -- 172.21.15.46:0/2983041291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 msgr2=0x55be6b6d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 -- 172.21.15.46:0/2983041291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b41a6290 con 0x55be6b6cc610 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b6d0870 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0097cc0 tx=0x7ff7a0022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2078cba433206e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff79b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:23:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00b1070 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7a009b250 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be6b567270 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a009b6c0 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7d0060b80 10 client.30584 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.065+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55be6b43bb60 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be6b567270 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7a00a1080 con 0x55be6b6cc610 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 10 client.30584 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 10 client.30584 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 20 client.30584 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6e0210 0x55be6b6e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 4 client.30584 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be6b68d810 con 0x55be6b6e0210 2024-05-08T23:23:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.066+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6e0210 0x55be6b6e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.067+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6e0210 0x55be6b6e2600 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97d52cfaf9a5d8d7 server_cookie=d4f51820fa354778 in_seq=0 out_seq=0 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.067+0000 7ff79b400700 10 client.30584 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff79b400700 1 -- 172.21.15.46:0/2983041291 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be6b68d810 con 0x55be6b6e0210 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff79b400700 10 client.30584 handle_command_reply: tid=0 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 client.30584 shutdown 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 msgr2=0x55be6b6d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b6d0870 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0097cc0 tx=0x7ff7a0022af0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 shutdown_connections 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6e0210 0x55be6b6e2600 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b6d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2983041291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b6d0870 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff79b400700 0 client.30584 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 >> 172.21.15.46:0/2983041291 conn(0x55be6b61c210 msgr2=0x55be6b6b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 shutdown_connections 2024-05-08T23:23:33.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.068+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2983041291 wait complete. 2024-05-08T23:23:33.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.069+0000 7ff7d0060b80 20 client.30584 trim_cache size 0 max 16384 2024-05-08T23:23:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 Processor -- start 2024-05-08T23:23:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 -- start start 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 0x55be6b6b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6ccf30 0x55be6b6cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b6c7ab0 con 0x55be6b6ccf30 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b6b5dc0 con 0x55be6b6c70a0 2024-05-08T23:23:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.086+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b6e77a0 con 0x55be6b6cc610 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 0x55be6b6b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43320/0 (socket says 172.21.15.46:43320) 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 -- 172.21.15.46:0/635509628 learned_addr learned my addr 172.21.15.46:0/635509628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7ba600700 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6ccf30 0x55be6b6cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 0x55be6b6b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47132/0 (socket says 172.21.15.46:47132) 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 -- 172.21.15.46:0/635509628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 msgr2=0x55be6b6b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 0x55be6b6b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 -- 172.21.15.46:0/635509628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6ccf30 msgr2=0x55be6b6cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6ccf30 0x55be6b6cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 -- 172.21.15.46:0/635509628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6b5d4560 con 0x55be6b6c70a0 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.087+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55be6b6e2e70 tx=0x7ff7a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c11a32d4399e282 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.088+0000 7ff7a7400700 1 -- 172.21.15.46:0/635509628 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8011d40 con 0x55be6b6c70a0 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.088+0000 7ff7a7400700 1 -- 172.21.15.46:0/635509628 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a8018040 con 0x55be6b6c70a0 2024-05-08T23:23:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.088+0000 7ff7a7400700 1 -- 172.21.15.46:0/635509628 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80056c0 con 0x55be6b6c70a0 2024-05-08T23:23:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 -- 172.21.15.46:0/635509628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 msgr2=0x55be6b6dc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55be6b6e2e70 tx=0x7ff7a8009260 comp rx=0 tx=0).stop 2024-05-08T23:23:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 -- 172.21.15.46:0/635509628 shutdown_connections 2024-05-08T23:23:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6ccf30 0x55be6b6cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6c70a0 0x55be6b6dc7c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/635509628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6cc610 0x55be6b6b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.089+0000 7ff7d0060b80 1 -- 172.21.15.46:0/635509628 >> 172.21.15.46:0/635509628 conn(0x55be6b61c210 msgr2=0x55be6b6b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.090+0000 7ff7d0060b80 1 -- 172.21.15.46:0/635509628 shutdown_connections 2024-05-08T23:23:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.090+0000 7ff7d0060b80 1 -- 172.21.15.46:0/635509628 wait complete. 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.090+0000 7ff7d0060b80 1 -- start start 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 Processor -- start 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b731650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b731d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b7324a0 con 0x55be6b6c70a0 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b6d5670 con 0x55be6b6cc610 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6b6d5800 con 0x55be6b6ccf30 2024-05-08T23:23:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.091+0000 7ff7d0060b80 10 client.0 _ll_register_callbacks cb 0x55be6b5b3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58546/0 (socket says 172.21.15.46:58546) 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 -- 172.21.15.46:0/2283827840 learned_addr learned my addr 172.21.15.46:0/2283827840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b731650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7ba600700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b731d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 -- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 msgr2=0x55be6b731d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b731d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 -- 172.21.15.46:0/2283827840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 msgr2=0x55be6b731650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b731650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9c00700 1 -- 172.21.15.46:0/2283827840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a8009950 con 0x55be6b6c70a0 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.092+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b731650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.093+0000 7ff7b9c00700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000c00 tx=0x7ff7a8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1776dfc81ce06188 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.093+0000 7ff79b400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8011ea0 con 0x55be6b6c70a0 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7ff7a80047c0 con 0x55be6b6c70a0 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7a009b950 con 0x55be6b6c70a0 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a8004980 con 0x55be6b6c70a0 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff7d0060b80 10 client.20340 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.094+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55be6b493cd0 con 0x55be6b6c70a0 2024-05-08T23:23:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a009b950 con 0x55be6b6c70a0 2024-05-08T23:23:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7a802d070 con 0x55be6b6c70a0 2024-05-08T23:23:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 10 client.20340 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 10 client.20340 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:23:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 20 client.20340 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6bfb90 0x55be6b6c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 4 client.20340 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.095+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be6b606ec0 con 0x55be6b6bfb90 2024-05-08T23:23:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.096+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6bfb90 0x55be6b6c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.096+0000 7ff7b9200700 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6bfb90 0x55be6b6c1fb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e244db5f242cc0b1 server_cookie=d40e68482b8a66d1 in_seq=0 out_seq=0 2024-05-08T23:23:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.096+0000 7ff79b400700 10 client.20340 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:23:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff79b400700 1 -- 172.21.15.46:0/2283827840 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be6b606ec0 con 0x55be6b6bfb90 2024-05-08T23:23:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff79b400700 10 client.20340 handle_command_reply: tid=0 2024-05-08T23:23:33.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 client.20340 shutdown 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 msgr2=0x55be6b6d7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8000c00 tx=0x7ff7a8011b40 comp rx=0 tx=0).stop 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 shutdown_connections 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55be6b6bfb90 0x55be6b6c1fb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6b6ccf30 0x55be6b731d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be6b6c70a0 0x55be6b6d7ba0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 --2- 172.21.15.46:0/2283827840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6b6cc610 0x55be6b731650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 >> 172.21.15.46:0/2283827840 conn(0x55be6b61c210 msgr2=0x55be6b6cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff79b400700 0 client.20340 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 shutdown_connections 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.097+0000 7ff7d0060b80 1 -- 172.21.15.46:0/2283827840 wait complete. 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.098+0000 7ff7d0060b80 20 client.20340 trim_cache size 0 max 16384 2024-05-08T23:23:33.099 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:23:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 msgr2=0x7ff79c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 0x7ff79c0a37d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009550 tx=0x7ff7ac009c20 comp rx=0 tx=0).stop 2024-05-08T23:23:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 msgr2=0x7ff7b4199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0005740 comp rx=0 tx=0).stop 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 shutdown_connections 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b410b3d0 0x7ff7b419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff79c0a1310 0x7ff79c0a37d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7b4102000 0x7ff7b4199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 --2- 172.21.15.46:0/2997339995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b4102920 0x7ff7b4199ab0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 >> 172.21.15.46:0/2997339995 conn(0x7ff7b40fb760 msgr2=0x7ff7b40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 shutdown_connections 2024-05-08T23:23:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.100+0000 7ff7bc600700 1 -- 172.21.15.46:0/2997339995 wait complete. 2024-05-08T23:23:33.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:23:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:32 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2527187988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:33.284 INFO:tasks.workunit.client.0.smithi046.stdout: 10 133717 19.49 MB/sec execute 234 sec latency 761.967 ms 2024-05-08T23:23:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.408+0000 7fd84f400700 1 -- 172.21.15.46:0/1941079349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 msgr2=0x7fd84810bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.408+0000 7fd84f400700 1 --2- 172.21.15.46:0/1941079349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 0x7fd84810bf20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c180 tx=0x7fd84000c490 comp rx=0 tx=0).stop 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 -- 172.21.15.46:0/1941079349 shutdown_connections 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 --2- 172.21.15.46:0/1941079349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 0x7fd84810bf20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 --2- 172.21.15.46:0/1941079349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8480ff280 0x7fd8480ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 --2- 172.21.15.46:0/1941079349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8480ffc30 0x7fd848103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 -- 172.21.15.46:0/1941079349 >> 172.21.15.46:0/1941079349 conn(0x7fd848075eb0 msgr2=0x7fd8480fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 -- 172.21.15.46:0/1941079349 shutdown_connections 2024-05-08T23:23:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.409+0000 7fd84f400700 1 -- 172.21.15.46:0/1941079349 wait complete. 2024-05-08T23:23:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 Processor -- start 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 -- start start 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8480ff280 0x7fd848195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 0x7fd8481955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819a370 con 0x7fd8480ff280 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819a4e0 con 0x7fd8481998f0 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.411+0000 7fd84f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819a6a0 con 0x7fd8481043a0 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.412+0000 7fd84d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.412+0000 7fd84d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43354/0 (socket says 172.21.15.46:43354) 2024-05-08T23:23:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.412+0000 7fd84d400700 1 -- 172.21.15.46:0/802382394 learned_addr learned my addr 172.21.15.46:0/802382394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 -- 172.21.15.46:0/802382394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 msgr2=0x7fd8481955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 0x7fd8481955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 -- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8480ff280 msgr2=0x7fd848195060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8480ff280 0x7fd848195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 -- 172.21.15.46:0/802382394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84000ad50 con 0x7fd8481998f0 2024-05-08T23:23:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84d400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd840003c30 tx=0x7fd840003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd845600700 1 -- 172.21.15.46:0/802382394 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000f050 con 0x7fd8481998f0 2024-05-08T23:23:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd845600700 1 -- 172.21.15.46:0/802382394 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84000a4b0 con 0x7fd8481998f0 2024-05-08T23:23:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd845600700 1 -- 172.21.15.46:0/802382394 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84001c6b0 con 0x7fd8481998f0 2024-05-08T23:23:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.414+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8481a1ba0 con 0x7fd8481998f0 2024-05-08T23:23:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.415+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8481a1f10 con 0x7fd8481998f0 2024-05-08T23:23:33.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.419+0000 7fd845600700 1 -- 172.21.15.46:0/802382394 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd840005040 con 0x7fd8481998f0 2024-05-08T23:23:33.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.423+0000 7fd845600700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 0x7fd8300a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.423+0000 7fd845600700 1 -- 172.21.15.46:0/802382394 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd8400ceaa0 con 0x7fd8481998f0 2024-05-08T23:23:33.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.424+0000 7fd84ca00700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 0x7fd8300a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.425+0000 7fd862f55b80 1 Processor -- start 2024-05-08T23:23:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.426+0000 7fd84ca00700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 0x7fd8300a7ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c005d00 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.426+0000 7fd862f55b80 1 -- start start 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c45b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c4564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c3a4150 con 0x55fb5c4560a0 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c3a1b10 con 0x55fb5c45bf30 2024-05-08T23:23:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd862f55b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c39d400 con 0x55fb5c45b610 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c45b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c45b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47188/0 (socket says 172.21.15.46:47188) 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.427+0000 7fd84ca00700 1 -- 172.21.15.46:0/2675328893 learned_addr learned my addr 172.21.15.46:0/2675328893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd84d400700 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c4564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 -- 172.21.15.46:0/2675328893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 msgr2=0x55fb5c45b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c45b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 -- 172.21.15.46:0/2675328893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 msgr2=0x55fb5c4564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c4564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.428+0000 7fd847e00700 1 -- 172.21.15.46:0/2675328893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5c21e3b0 con 0x55fb5c45bf30 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.429+0000 7fd847e00700 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455b60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd848196560 tx=0x7fd834009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4619f3db8b4b1e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.429+0000 7fd839c00700 1 -- 172.21.15.46:0/2675328893 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834004230 con 0x55fb5c45bf30 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.429+0000 7fd839c00700 1 -- 172.21.15.46:0/2675328893 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd834004cf0 con 0x55fb5c45bf30 2024-05-08T23:23:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.429+0000 7fd839c00700 1 -- 172.21.15.46:0/2675328893 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400eeb0 con 0x55fb5c45bf30 2024-05-08T23:23:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.433+0000 7fd862f55b80 1 -- 172.21.15.46:0/2675328893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 msgr2=0x55fb5c455b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.433+0000 7fd862f55b80 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455b60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd848196560 tx=0x7fd834009d90 comp rx=0 tx=0).stop 2024-05-08T23:23:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.434+0000 7fd862f55b80 1 -- 172.21.15.46:0/2675328893 shutdown_connections 2024-05-08T23:23:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.434+0000 7fd862f55b80 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c4564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.434+0000 7fd862f55b80 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c45b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.434+0000 7fd862f55b80 1 --2- 172.21.15.46:0/2675328893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455b60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.434+0000 7fd862f55b80 1 -- 172.21.15.46:0/2675328893 >> 172.21.15.46:0/2675328893 conn(0x55fb5c3ab210 msgr2=0x55fb5c447df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.435+0000 7fd862f55b80 1 -- 172.21.15.46:0/2675328893 shutdown_connections 2024-05-08T23:23:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.435+0000 7fd862f55b80 1 -- 172.21.15.46:0/2675328893 wait complete. 2024-05-08T23:23:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.435+0000 7fd862f55b80 1 -- start start 2024-05-08T23:23:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.436+0000 7fd862f55b80 1 Processor -- start 2024-05-08T23:23:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c44f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd862f55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c339560 con 0x55fb5c4560a0 2024-05-08T23:23:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd847e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd847e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47198/0 (socket says 172.21.15.46:47198) 2024-05-08T23:23:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.437+0000 7fd847e00700 1 -- 172.21.15.46:0/354108225 learned_addr learned my addr 172.21.15.46:0/354108225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.438+0000 7fd84d400700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.438+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c339200 con 0x55fb5c45bf30 2024-05-08T23:23:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.438+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c2206a0 con 0x55fb5c45b610 2024-05-08T23:23:33.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.438+0000 7fd862f55b80 10 client.0 _ll_register_callbacks cb 0x55fb5c342b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.439+0000 7fd847e00700 1 -- 172.21.15.46:0/354108225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 msgr2=0x55fb5c455590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.439+0000 7fd847e00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.439+0000 7fd847e00700 1 -- 172.21.15.46:0/354108225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 msgr2=0x55fb5c44f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:23:33.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd84ca00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c44f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd847e00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c44f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd847e00700 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd834009790 con 0x55fb5c45b610 2024-05-08T23:23:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd84d400700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd84ca00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c44f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd847e00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd834005950 tx=0x7fd83400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7b926a8ba67b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.440+0000 7fd82e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834005430 con 0x55fb5c45b610 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd834014070 con 0x55fb5c45b610 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb5c2f6270 con 0x55fb5c45b610 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd83400e810 con 0x55fb5c45b610 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd862f55b80 10 client.30601 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.441+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb5c1cab60 con 0x55fb5c45b610 2024-05-08T23:23:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb5c2f6270 con 0x55fb5c45b610 2024-05-08T23:23:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd834018c10 con 0x55fb5c45b610 2024-05-08T23:23:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 10 client.30601 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 10 client.30601 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:23:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 20 client.30601 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c478080 0x55fb5c47a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 4 client.30601 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.443+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb5c41c810 con 0x55fb5c478080 2024-05-08T23:23:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.444+0000 7fd84ca00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c478080 0x55fb5c47a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.445+0000 7fd84ca00700 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c478080 0x55fb5c47a4a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f891d8447bdb7cd server_cookie=d176cbd0d6a644c3 in_seq=0 out_seq=0 2024-05-08T23:23:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.445+0000 7fd82e000700 10 client.30601 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:23:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.447+0000 7fd82e000700 1 -- 172.21.15.46:0/354108225 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb5c41c810 con 0x55fb5c478080 2024-05-08T23:23:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.447+0000 7fd82e000700 10 client.30601 handle_command_reply: tid=0 2024-05-08T23:23:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.447+0000 7fd862f55b80 1 client.30601 shutdown 2024-05-08T23:23:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 msgr2=0x55fb5c44fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd834005950 tx=0x7fd83400ed30 comp rx=0 tx=0).stop 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 shutdown_connections 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c478080 0x55fb5c47a4a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c455590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c44f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 --2- 172.21.15.46:0/354108225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c44fd90 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 >> 172.21.15.46:0/354108225 conn(0x55fb5c3ab210 msgr2=0x55fb5c4475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd82e000700 0 client.30601 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:23:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.448+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 shutdown_connections 2024-05-08T23:23:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.450+0000 7fd862f55b80 1 -- 172.21.15.46:0/354108225 wait complete. 2024-05-08T23:23:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.451+0000 7fd862f55b80 20 client.30601 trim_cache size 0 max 16384 2024-05-08T23:23:33.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:32 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2527187988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.484+0000 7fd862f55b80 1 Processor -- start 2024-05-08T23:23:33.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.485+0000 7fd862f55b80 1 -- start start 2024-05-08T23:23:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.485+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c4554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c4611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd862f55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c477eb0 con 0x55fb5c4560a0 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd862f55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c44c1b0 con 0x55fb5c45bf30 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd862f55b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c444dc0 con 0x55fb5c45b610 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58578/0 (socket says 172.21.15.46:58578) 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd84ca00700 1 -- 172.21.15.46:0/3623730477 learned_addr learned my addr 172.21.15.46:0/3623730477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.486+0000 7fd847e00700 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c4554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 -- 172.21.15.46:0/3623730477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 msgr2=0x55fb5c4554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c4554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 -- 172.21.15.46:0/3623730477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 msgr2=0x55fb5c4611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84d400700 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c4611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c4611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 -- 172.21.15.46:0/3623730477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5c363560 con 0x55fb5c4560a0 2024-05-08T23:23:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.487+0000 7fd84ca00700 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00afd0 tx=0x7fd83c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4800ae112af18f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.488+0000 7fd83a600700 1 -- 172.21.15.46:0/3623730477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ca30 con 0x55fb5c4560a0 2024-05-08T23:23:33.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.488+0000 7fd83a600700 1 -- 172.21.15.46:0/3623730477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd83c00fda0 con 0x55fb5c4560a0 2024-05-08T23:23:33.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.488+0000 7fd83a600700 1 -- 172.21.15.46:0/3623730477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c01e650 con 0x55fb5c4560a0 2024-05-08T23:23:33.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.489+0000 7fd862f55b80 1 -- 172.21.15.46:0/3623730477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 msgr2=0x55fb5c45d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.489+0000 7fd862f55b80 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00afd0 tx=0x7fd83c013000 comp rx=0 tx=0).stop 2024-05-08T23:23:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.490+0000 7fd862f55b80 1 -- 172.21.15.46:0/3623730477 shutdown_connections 2024-05-08T23:23:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.490+0000 7fd862f55b80 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c4611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.490+0000 7fd862f55b80 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c4560a0 0x55fb5c45d320 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.490+0000 7fd862f55b80 1 --2- 172.21.15.46:0/3623730477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c45b610 0x55fb5c4554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.490+0000 7fd862f55b80 1 -- 172.21.15.46:0/3623730477 >> 172.21.15.46:0/3623730477 conn(0x55fb5c3ab210 msgr2=0x55fb5c45dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.492+0000 7fd862f55b80 1 -- 172.21.15.46:0/3623730477 shutdown_connections 2024-05-08T23:23:33.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.493+0000 7fd862f55b80 1 -- 172.21.15.46:0/3623730477 wait complete. 2024-05-08T23:23:33.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.493+0000 7fd862f55b80 1 -- start start 2024-05-08T23:23:33.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.494+0000 7fd862f55b80 1 Processor -- start 2024-05-08T23:23:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.496+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 0x55fb5c450af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.496+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.496+0000 7fd862f55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c451570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd847e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 0x55fb5c450af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd847e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58592/0 (socket says 172.21.15.46:58592) 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd84ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 0x55fb5c450af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47230/0 (socket says 172.21.15.46:47230) 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd847e00700 1 -- 172.21.15.46:0/1286611474 learned_addr learned my addr 172.21.15.46:0/1286611474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd84d400700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c451570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c4cd570 con 0x55fb5c45b610 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c4cd6b0 con 0x55fb5c45bf30 2024-05-08T23:23:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c4cd870 con 0x55fb5c4560a0 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.497+0000 7fd862f55b80 10 client.0 _ll_register_callbacks cb 0x55fb5c342b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 -- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 msgr2=0x55fb5c450af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 0x55fb5c450af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 -- 172.21.15.46:0/1286611474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 msgr2=0x55fb5c451570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c451570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c00ccb0 con 0x55fb5c45b610 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd847e00700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a80 tx=0x7fd834004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee9a49f5eca9956 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.498+0000 7fd82e000700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.499+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400e830 con 0x55fb5c45b610 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.499+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fd834000e40 con 0x55fb5c45b610 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.499+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd834004cf0 con 0x55fb5c45b610 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.500+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd834005da0 con 0x55fb5c45b610 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.500+0000 7fd862f55b80 10 client.20349 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.500+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb5c222cd0 con 0x55fb5c45b610 2024-05-08T23:23:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.501+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd834004cf0 con 0x55fb5c45b610 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.501+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd83400e350 con 0x55fb5c45b610 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 10 client.20349 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 10 client.20349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 20 client.20349 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c4cece0 0x55fb5c4d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 4 client.20349 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb5c395ec0 con 0x55fb5c4cece0 2024-05-08T23:23:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.502+0000 7fd84ca00700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c4cece0 0x55fb5c4d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.503+0000 7fd84ca00700 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c4cece0 0x55fb5c4d1100 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35f92e961cdef405 server_cookie=d597f4b680298bd6 in_seq=0 out_seq=0 2024-05-08T23:23:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.503+0000 7fd82e000700 10 client.20349 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:23:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.504+0000 7fd82e000700 1 -- 172.21.15.46:0/1286611474 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb5c395ec0 con 0x55fb5c4cece0 2024-05-08T23:23:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.504+0000 7fd82e000700 10 client.20349 handle_command_reply: tid=0 2024-05-08T23:23:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.504+0000 7fd862f55b80 1 client.20349 shutdown 2024-05-08T23:23:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.505+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 msgr2=0x55fb5c451030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.505+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a80 tx=0x7fd834004c10 comp rx=0 tx=0).stop 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 shutdown_connections 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x55fb5c4cece0 0x55fb5c4d1100 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c45bf30 0x55fb5c451570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5c4560a0 0x55fb5c450af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 --2- 172.21.15.46:0/1286611474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb5c45b610 0x55fb5c451030 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 >> 172.21.15.46:0/1286611474 conn(0x55fb5c3ab210 msgr2=0x55fb5c461120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.506+0000 7fd82e000700 0 client.20349 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:23:33.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.508+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 shutdown_connections 2024-05-08T23:23:33.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.508+0000 7fd862f55b80 1 -- 172.21.15.46:0/1286611474 wait complete. 2024-05-08T23:23:33.509 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:23:33.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.508+0000 7fd862f55b80 20 client.20349 trim_cache size 0 max 16384 2024-05-08T23:23:33.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 msgr2=0x7fd8300a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 0x7fd8300a7ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c005d00 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 msgr2=0x7fd848199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd840003c30 tx=0x7fd840003d10 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 shutdown_connections 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8481998f0 0x7fd848199d70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fd8300a56e0 0x7fd8300a7ba0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8480ff280 0x7fd848195060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 --2- 172.21.15.46:0/802382394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8481043a0 0x7fd8481955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.509+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 >> 172.21.15.46:0/802382394 conn(0x7fd848075eb0 msgr2=0x7fd848102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.510+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 shutdown_connections 2024-05-08T23:23:33.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.510+0000 7fd84f400700 1 -- 172.21.15.46:0/802382394 wait complete. 2024-05-08T23:23:33.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:23:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 -- 172.21.15.46:0/3454400104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 msgr2=0x7f1d0810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/3454400104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d0810ada0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 -- 172.21.15.46:0/3454400104 shutdown_connections 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/3454400104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 0x7f1d0810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/3454400104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d08102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/3454400104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d0810ada0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.883+0000 7f1d0ec00700 1 -- 172.21.15.46:0/3454400104 >> 172.21.15.46:0/3454400104 conn(0x7f1d080fb760 msgr2=0x7f1d080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.884+0000 7f1d0ec00700 1 -- 172.21.15.46:0/3454400104 shutdown_connections 2024-05-08T23:23:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.884+0000 7f1d0ec00700 1 -- 172.21.15.46:0/3454400104 wait complete. 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.885+0000 7f1d0ec00700 1 Processor -- start 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.885+0000 7f1d0ec00700 1 -- start start 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d08107c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 0x7f1d08108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08108920 con 0x7f1d08101fa0 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08108a90 con 0x7f1d081028c0 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081a9e00 con 0x7f1d0810b3c0 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d07400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d08107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58610/0 (socket says 172.21.15.46:58610) 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d07e00700 1 -- 172.21.15.46:0/1038933548 learned_addr learned my addr 172.21.15.46:0/1038933548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.886+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 0x7f1d08108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 -- 172.21.15.46:0/1038933548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 msgr2=0x7f1d08108190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 0x7f1d08108190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 -- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 msgr2=0x7f1d0810ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 -- 172.21.15.46:0/1038933548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00009c40 con 0x7f1d081028c0 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07e00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d07400700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d08107c50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098e0 tx=0x7f1cf400bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.887+0000 7f1d04c00700 1 -- 172.21.15.46:0/1038933548 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf401d070 con 0x7f1d081028c0 2024-05-08T23:23:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.888+0000 7f1d04c00700 1 -- 172.21.15.46:0/1038933548 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf400fb50 con 0x7f1d081028c0 2024-05-08T23:23:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.888+0000 7f1d04c00700 1 -- 172.21.15.46:0/1038933548 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40173f0 con 0x7f1d081028c0 2024-05-08T23:23:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.888+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf40095f0 con 0x7f1d081028c0 2024-05-08T23:23:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.888+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d081aa2d0 con 0x7f1d081028c0 2024-05-08T23:23:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.891+0000 7f1d04c00700 1 -- 172.21.15.46:0/1038933548 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1cf400fcc0 con 0x7f1d081028c0 2024-05-08T23:23:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.893+0000 7f1d22715b80 1 Processor -- start 2024-05-08T23:23:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.893+0000 7f1d04c00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 0x7f1cec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.893+0000 7f1d04c00700 1 -- 172.21.15.46:0/1038933548 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1cf40ce480 con 0x7f1d081028c0 2024-05-08T23:23:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.893+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 0x7f1cec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.893+0000 7f1d22715b80 1 -- start start 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 0x5557cd2e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd240150 con 0x5557cd2eee00 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd23db10 con 0x5557cd2ef720 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d22715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd239400 con 0x5557cd2e9890 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d07400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 0x7f1cec0a37d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009fd0 tx=0x7f1d0000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d07400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43414/0 (socket says 172.21.15.46:43414) 2024-05-08T23:23:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.894+0000 7f1d07400700 1 -- 172.21.15.46:0/2346836482 learned_addr learned my addr 172.21.15.46:0/2346836482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 0x5557cd2e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 -- 172.21.15.46:0/2346836482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 msgr2=0x5557cd2e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 0x5557cd2e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 -- 172.21.15.46:0/2346836482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 msgr2=0x5557cd2e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 -- 172.21.15.46:0/2346836482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd0ba3b0 con 0x5557cd2eee00 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 0x5557cd2e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.895+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2ef1e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8007cf0 tx=0x7f1cf800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df117c6670d82b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.896+0000 7f1cfd200700 1 -- 172.21.15.46:0/2346836482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800fbe0 con 0x5557cd2eee00 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.896+0000 7f1cfd200700 1 -- 172.21.15.46:0/2346836482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf800fd40 con 0x5557cd2eee00 2024-05-08T23:23:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.896+0000 7f1cfd200700 1 -- 172.21.15.46:0/2346836482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80056e0 con 0x5557cd2eee00 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 -- 172.21.15.46:0/2346836482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 msgr2=0x5557cd2ef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2ef1e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8007cf0 tx=0x7f1cf800f360 comp rx=0 tx=0).stop 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 -- 172.21.15.46:0/2346836482 shutdown_connections 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2e9890 0x5557cd2e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2ef1e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2346836482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 -- 172.21.15.46:0/2346836482 >> 172.21.15.46:0/2346836482 conn(0x5557cd247210 msgr2=0x5557cd2e3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 -- 172.21.15.46:0/2346836482 shutdown_connections 2024-05-08T23:23:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.897+0000 7f1d22715b80 1 -- 172.21.15.46:0/2346836482 wait complete. 2024-05-08T23:23:33.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.898+0000 7f1d22715b80 1 -- start start 2024-05-08T23:23:33.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.898+0000 7f1d22715b80 1 Processor -- start 2024-05-08T23:23:33.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd1d5560 con 0x5557cd2eee00 2024-05-08T23:23:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd1d5200 con 0x5557cd2e9890 2024-05-08T23:23:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd0bc6a0 con 0x5557cd2ef720 2024-05-08T23:23:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d22715b80 10 client.0 _ll_register_callbacks cb 0x5557cd1deb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d0cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d07400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d0cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47246/0 (socket says 172.21.15.46:47246) 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.899+0000 7f1d0cc00700 1 -- 172.21.15.46:0/3932968227 learned_addr learned my addr 172.21.15.46:0/3932968227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 -- 172.21.15.46:0/3932968227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 msgr2=0x5557cd2f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 -- 172.21.15.46:0/3932968227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 msgr2=0x5557cd2f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 -- 172.21.15.46:0/3932968227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf8007920 con 0x5557cd2ef720 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d07400700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004670 tx=0x7f1d00004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e8dc1473eed4ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1cf1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000e040 con 0x5557cd2ef720 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557cd192270 con 0x5557cd2ef720 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.900+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1d00009230 con 0x5557cd2ef720 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.901+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d000093f0 con 0x5557cd2ef720 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.901+0000 7f1d22715b80 10 client.30616 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.901+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5557cd066b60 con 0x5557cd2ef720 2024-05-08T23:23:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.901+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557cd192270 con 0x5557cd2ef720 2024-05-08T23:23:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d00029060 con 0x5557cd2ef720 2024-05-08T23:23:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 10 client.30616 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 10 client.30616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:23:33.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 20 client.30616 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd3031d0 0x5557cd3055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 4 client.30616 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557cd2b8810 con 0x5557cd3031d0 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.902+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd3031d0 0x5557cd3055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.903+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd3031d0 0x5557cd3055f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d26ea6b3f34db45f server_cookie=be23cc1dc7cf2ae6 in_seq=0 out_seq=0 2024-05-08T23:23:33.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.903+0000 7f1cf1600700 10 client.30616 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:23:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1cf1600700 1 -- 172.21.15.46:0/3932968227 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557cd2b8810 con 0x5557cd3031d0 2024-05-08T23:23:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1cf1600700 10 client.30616 handle_command_reply: tid=0 2024-05-08T23:23:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 client.30616 shutdown 2024-05-08T23:23:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 msgr2=0x5557cd2f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004670 tx=0x7f1d00004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 shutdown_connections 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2f3510 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd3031d0 0x5557cd3055f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3932968227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1cf1600700 0 client.30616 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 >> 172.21.15.46:0/3932968227 conn(0x5557cd247210 msgr2=0x5557cd2e35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 shutdown_connections 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.907+0000 7f1d22715b80 1 -- 172.21.15.46:0/3932968227 wait complete. 2024-05-08T23:23:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.908+0000 7f1d22715b80 20 client.30616 trim_cache size 0 max 16384 2024-05-08T23:23:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.927+0000 7f1d22715b80 1 Processor -- start 2024-05-08T23:23:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.927+0000 7f1d22715b80 1 -- start start 2024-05-08T23:23:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd30a800 con 0x5557cd2eee00 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd2f40d0 con 0x5557cd2e9890 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d22715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd3008e0 con 0x5557cd2ef720 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d0cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43428/0 (socket says 172.21.15.46:43428) 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 -- 172.21.15.46:0/2171285815 learned_addr learned my addr 172.21.15.46:0/2171285815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 -- 172.21.15.46:0/2171285815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 msgr2=0x5557cd2e3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2e3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 -- 172.21.15.46:0/2171285815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 msgr2=0x5557cd2e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.928+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.929+0000 7f1d07e00700 1 -- 172.21.15.46:0/2171285815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd1ff560 con 0x5557cd2e9890 2024-05-08T23:23:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.929+0000 7f1d07e00700 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80052b0 tx=0x7f1cf80052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fdcef49c463e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.929+0000 7f1cfdc00700 1 -- 172.21.15.46:0/2171285815 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8020040 con 0x5557cd2e9890 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.929+0000 7f1cfdc00700 1 -- 172.21.15.46:0/2171285815 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf8021040 con 0x5557cd2e9890 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.929+0000 7f1cfdc00700 1 -- 172.21.15.46:0/2171285815 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8028500 con 0x5557cd2e9890 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 -- 172.21.15.46:0/2171285815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 msgr2=0x5557cd2fec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80052b0 tx=0x7f1cf80052e0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 -- 172.21.15.46:0/2171285815 shutdown_connections 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2ef720 0x5557cd2e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2e9890 0x5557cd2fec20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 --2- 172.21.15.46:0/2171285815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2eee00 0x5557cd2e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 -- 172.21.15.46:0/2171285815 >> 172.21.15.46:0/2171285815 conn(0x5557cd247210 msgr2=0x5557cd2e0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 -- 172.21.15.46:0/2171285815 shutdown_connections 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.930+0000 7f1d22715b80 1 -- 172.21.15.46:0/2171285815 wait complete. 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 -- start start 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 Processor -- start 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 0x5557cd2fa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 0x5557cd2ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd2eb420 con 0x5557cd2e9890 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd2eb590 con 0x5557cd2ef720 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.931+0000 7f1d22715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd2eb750 con 0x5557cd2eee00 2024-05-08T23:23:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d22715b80 10 client.0 _ll_register_callbacks cb 0x5557cd1deb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 0x5557cd2ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 0x5557cd2fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 0x5557cd2ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47258/0 (socket says 172.21.15.46:47258) 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07400700 1 -- 172.21.15.46:0/3632422697 learned_addr learned my addr 172.21.15.46:0/3632422697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d0cc00700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 -- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 msgr2=0x5557cd2ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 0x5557cd2ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 -- 172.21.15.46:0/3632422697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 msgr2=0x5557cd2eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.932+0000 7f1d07e00700 1 -- 172.21.15.46:0/3632422697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557cd0ba3b0 con 0x5557cd2e9890 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1d07e00700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 0x5557cd2fa360 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80052b0 tx=0x7f1cf8007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e47a43d6dbae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1cf1600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8005620 con 0x5557cd2e9890 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f1cf8004230 con 0x5557cd2e9890 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cf800fd40 con 0x5557cd2e9890 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf80043f0 con 0x5557cd2e9890 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1d22715b80 10 client.20367 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.933+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5557cd0becd0 con 0x5557cd2e9890 2024-05-08T23:23:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf803d020 con 0x5557cd2e9890 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1cf8028480 con 0x5557cd2e9890 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 10 client.20367 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 10 client.20367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 20 client.20367 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd359920 0x5557cd35bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 4 client.20367 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557cd231ec0 con 0x5557cd359920 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.934+0000 7f1d07400700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd359920 0x5557cd35bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.935+0000 7f1d07400700 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd359920 0x5557cd35bd10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5c880a3a5cca802 server_cookie=db0b07d17633e83b in_seq=0 out_seq=0 2024-05-08T23:23:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.935+0000 7f1cf1600700 10 client.20367 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1cf1600700 1 -- 172.21.15.46:0/3632422697 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557cd231ec0 con 0x5557cd359920 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1cf1600700 10 client.20367 handle_command_reply: tid=0 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 client.20367 shutdown 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 msgr2=0x5557cd2fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 0x5557cd2fa360 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80052b0 tx=0x7f1cf8007c40 comp rx=0 tx=0).stop 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 shutdown_connections 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557cd2ef720 0x5557cd2eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5557cd359920 0x5557cd35bd10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5557cd2e9890 0x5557cd2fa360 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 --2- 172.21.15.46:0/3632422697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557cd2eee00 0x5557cd2ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 >> 172.21.15.46:0/3632422697 conn(0x5557cd247210 msgr2=0x5557cd2e9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1cf1600700 0 client.20367 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:23:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 shutdown_connections 2024-05-08T23:23:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.939+0000 7f1d22715b80 1 -- 172.21.15.46:0/3632422697 wait complete. 2024-05-08T23:23:33.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d22715b80 20 client.20367 trim_cache size 0 max 16384 2024-05-08T23:23:33.945 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 msgr2=0x7f1cec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 0x7f1cec0a37d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009fd0 tx=0x7f1d0000b040 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 msgr2=0x7f1d08107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d08107c50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098e0 tx=0x7f1cf400bfb0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 shutdown_connections 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d0810b3c0 0x7f1d08108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f1cec0a1310 0x7f1cec0a37d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d08101fa0 0x7f1d0810ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 --2- 172.21.15.46:0/1038933548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d081028c0 0x7f1d08107c50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 >> 172.21.15.46:0/1038933548 conn(0x7f1d080fb760 msgr2=0x7f1d080fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 shutdown_connections 2024-05-08T23:23:33.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:33.940+0000 7f1d0ec00700 1 -- 172.21.15.46:0/1038933548 wait complete. 2024-05-08T23:23:33.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 -- 172.21.15.46:0/805759007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 msgr2=0x7f788c10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 --2- 172.21.15.46:0/805759007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c10ada0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 -- 172.21.15.46:0/805759007 shutdown_connections 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 --2- 172.21.15.46:0/805759007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 --2- 172.21.15.46:0/805759007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 --2- 172.21.15.46:0/805759007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c10ada0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 -- 172.21.15.46:0/805759007 >> 172.21.15.46:0/805759007 conn(0x7f788c0fb760 msgr2=0x7f788c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 -- 172.21.15.46:0/805759007 shutdown_connections 2024-05-08T23:23:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.213+0000 7f7890a00700 1 -- 172.21.15.46:0/805759007 wait complete. 2024-05-08T23:23:34.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.214+0000 7f7890a00700 1 Processor -- start 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.214+0000 7f7890a00700 1 -- start start 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19c1e0 con 0x7f788c101fa0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19c350 con 0x7f788c10b3c0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7890a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19c510 con 0x7f788c1028c0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f788a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58672/0 (socket says 172.21.15.46:58672) 2024-05-08T23:23:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889e00700 1 -- 172.21.15.46:0/3258399539 learned_addr learned my addr 172.21.15.46:0/3258399539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889e00700 1 -- 172.21.15.46:0/3258399539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 msgr2=0x7f788c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.215+0000 7f7889e00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7889e00700 1 -- 172.21.15.46:0/3258399539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 msgr2=0x7f788c19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7889e00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7889e00700 1 -- 172.21.15.46:0/3258399539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780095f0 con 0x7f788c101fa0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7889400700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f788a800700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7889e00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9a0 tx=0x7f787c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7882a00700 1 -- 172.21.15.46:0/3258399539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c009790 con 0x7f788c101fa0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c19c7f0 con 0x7f788c101fa0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7882a00700 1 -- 172.21.15.46:0/3258399539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787c010430 con 0x7f788c101fa0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7882a00700 1 -- 172.21.15.46:0/3258399539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c018680 con 0x7f788c101fa0 2024-05-08T23:23:34.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.216+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0ffd00 con 0x7f788c101fa0 2024-05-08T23:23:34.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.219+0000 7f7882a00700 1 -- 172.21.15.46:0/3258399539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f787c0105a0 con 0x7f788c101fa0 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.221+0000 7f7882a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 0x7f78700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.221+0000 7f7882a00700 1 -- 172.21.15.46:0/3258399539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f787c014070 con 0x7f788c101fa0 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f7889400700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 0x7f78700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 Processor -- start 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 -- start start 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f7889400700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 0x7f78700a3750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f788c198810 tx=0x7f787800b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba47145150 con 0x55ba471f3e00 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba47142b10 con 0x55ba471ee890 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.222+0000 7f78a4469b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4713e400 con 0x55ba471f4720 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47292/0 (socket says 172.21.15.46:47292) 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 -- 172.21.15.46:0/4137963439 learned_addr learned my addr 172.21.15.46:0/4137963439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889e00700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f788a800700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 -- 172.21.15.46:0/4137963439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 msgr2=0x55ba471eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 -- 172.21.15.46:0/4137963439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 msgr2=0x55ba471f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889400700 1 -- 172.21.15.46:0/4137963439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba46fbf3b0 con 0x55ba471f4720 2024-05-08T23:23:34.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.223+0000 7f7889e00700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.224+0000 7f7889400700 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea4ada2e0787174 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.224+0000 7f7877000700 1 -- 172.21.15.46:0/4137963439 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787802b070 con 0x55ba471f4720 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.224+0000 7f7877000700 1 -- 172.21.15.46:0/4137963439 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787801e040 con 0x55ba471f4720 2024-05-08T23:23:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.224+0000 7f7877000700 1 -- 172.21.15.46:0/4137963439 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800f940 con 0x55ba471f4720 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 -- 172.21.15.46:0/4137963439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 msgr2=0x55ba471ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878020000 comp rx=0 tx=0).stop 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 -- 172.21.15.46:0/4137963439 shutdown_connections 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 --2- 172.21.15.46:0/4137963439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471ee350 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 -- 172.21.15.46:0/4137963439 >> 172.21.15.46:0/4137963439 conn(0x55ba4714c210 msgr2=0x55ba471e8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 -- 172.21.15.46:0/4137963439 shutdown_connections 2024-05-08T23:23:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.225+0000 7f78a4469b80 1 -- 172.21.15.46:0/4137963439 wait complete. 2024-05-08T23:23:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.226+0000 7f78a4469b80 1 -- start start 2024-05-08T23:23:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.226+0000 7f78a4469b80 1 Processor -- start 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 0x55ba471f8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba470da560 con 0x55ba471f3e00 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba470da200 con 0x55ba471f4720 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba46fc16a0 con 0x55ba471ee890 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f78a4469b80 10 client.0 _ll_register_callbacks cb 0x55ba470e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.227+0000 7f7889400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f7889400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58690/0 (socket says 172.21.15.46:58690) 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 0x55ba471f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f7889400700 1 -- 172.21.15.46:0/1106775192 learned_addr learned my addr 172.21.15.46:0/1106775192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f7889e00700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 -- 172.21.15.46:0/1106775192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 msgr2=0x55ba471f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 -- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 msgr2=0x55ba471f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 -- 172.21.15.46:0/1106775192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0ffd00 con 0x55ba471f4720 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f7889400700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f788a800700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 0x55ba471f8650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7884004660 tx=0x7f7884004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84100f17645f74a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.228+0000 7f786b400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884014070 con 0x55ba471f4720 2024-05-08T23:23:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba47097270 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f788400ec50 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f788400ee10 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f78a4469b80 10 client.30632 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f788c10e540 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.229+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba47097270 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7884022060 con 0x55ba471f4720 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 10 client.30632 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 10 client.30632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 20 client.30632 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47208350 0x55ba4720a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 4 client.30632 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba471bd810 con 0x55ba47208350 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.230+0000 7f7889e00700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47208350 0x55ba4720a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f7889e00700 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47208350 0x55ba4720a770 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dec188646cc04f88 server_cookie=74efea6b1250441f in_seq=0 out_seq=0 2024-05-08T23:23:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f786b400700 10 client.30632 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f786b400700 1 -- 172.21.15.46:0/1106775192 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba471bd810 con 0x55ba47208350 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f786b400700 10 client.30632 handle_command_reply: tid=0 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f78a4469b80 1 client.30632 shutdown 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 msgr2=0x55ba471f8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.231+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 0x55ba471f8650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7884004660 tx=0x7f7884004970 comp rx=0 tx=0).stop 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 shutdown_connections 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f4720 0x55ba471f8650 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47208350 0x55ba4720a770 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1106775192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 >> 172.21.15.46:0/1106775192 conn(0x55ba4714c210 msgr2=0x55ba471e85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f786b400700 0 client.30632 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:23:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 shutdown_connections 2024-05-08T23:23:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 1 -- 172.21.15.46:0/1106775192 wait complete. 2024-05-08T23:23:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.232+0000 7f78a4469b80 20 client.30632 trim_cache size 0 max 16384 2024-05-08T23:23:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.254+0000 7f78a4469b80 1 Processor -- start 2024-05-08T23:23:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.254+0000 7f78a4469b80 1 -- start start 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba471e3800 con 0x55ba471f3e00 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba471e1040 con 0x55ba471ee890 2024-05-08T23:23:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.255+0000 7f78a4469b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba471f4430 con 0x55ba471f4720 2024-05-08T23:23:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43476/0 (socket says 172.21.15.46:43476) 2024-05-08T23:23:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 -- 172.21.15.46:0/605190523 learned_addr learned my addr 172.21.15.46:0/605190523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889400700 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 -- 172.21.15.46:0/605190523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 msgr2=0x55ba471f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 -- 172.21.15.46:0/605190523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 msgr2=0x55ba471e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.256+0000 7f7889e00700 1 -- 172.21.15.46:0/605190523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba46f6bb60 con 0x55ba471ee890 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.257+0000 7f7889e00700 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f787c0964c0 tx=0x7f787c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429bad116a9104d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.257+0000 7f7877a00700 1 -- 172.21.15.46:0/605190523 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c0a2440 con 0x55ba471ee890 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.257+0000 7f7877a00700 1 -- 172.21.15.46:0/605190523 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787c0ade90 con 0x55ba471ee890 2024-05-08T23:23:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.257+0000 7f7877a00700 1 -- 172.21.15.46:0/605190523 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c0994b0 con 0x55ba471ee890 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.258+0000 7f78a4469b80 1 -- 172.21.15.46:0/605190523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 msgr2=0x55ba471ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.258+0000 7f78a4469b80 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f787c0964c0 tx=0x7f787c0cc040 comp rx=0 tx=0).stop 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 -- 172.21.15.46:0/605190523 shutdown_connections 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471f4720 0x55ba471f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471ee890 0x55ba471ee350 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 --2- 172.21.15.46:0/605190523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f3e00 0x55ba471e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 -- 172.21.15.46:0/605190523 >> 172.21.15.46:0/605190523 conn(0x55ba4714c210 msgr2=0x55ba471f64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.259+0000 7f78a4469b80 1 -- 172.21.15.46:0/605190523 shutdown_connections 2024-05-08T23:23:34.261 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:33 smithi053 ceph-mon[77587]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-08T23:23:34.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.261+0000 7f78a4469b80 1 -- 172.21.15.46:0/605190523 wait complete. 2024-05-08T23:23:34.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.262+0000 7f78a4469b80 1 -- start start 2024-05-08T23:23:34.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.262+0000 7f78a4469b80 1 Processor -- start 2024-05-08T23:23:34.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f3e00 0x55ba471fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba471fdee0 con 0x55ba471f4720 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba472665a0 con 0x55ba471f3e00 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba472666e0 con 0x55ba471ee890 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.263+0000 7f78a4469b80 10 client.0 _ll_register_callbacks cb 0x55ba470e3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.264+0000 7f788a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.264+0000 7f788a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58708/0 (socket says 172.21.15.46:58708) 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.264+0000 7f788a800700 1 -- 172.21.15.46:0/1785758099 learned_addr learned my addr 172.21.15.46:0/1785758099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.264+0000 7f7889400700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f3e00 0x55ba471fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 -- 172.21.15.46:0/1785758099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 msgr2=0x55ba471fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 -- 172.21.15.46:0/1785758099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f3e00 msgr2=0x55ba471fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f3e00 0x55ba471fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 -- 172.21.15.46:0/1785758099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c05fe70 con 0x55ba471f4720 2024-05-08T23:23:34.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.265+0000 7f788a800700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7884004380 tx=0x7f7884024e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd283fac463f5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.266+0000 7f786b400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:34.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.266+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884014070 con 0x55ba471f4720 2024-05-08T23:23:34.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.266+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f7884024920 con 0x55ba471f4720 2024-05-08T23:23:34.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.267+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f787801e040 con 0x55ba471f4720 2024-05-08T23:23:34.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.268+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78840243f0 con 0x55ba471f4720 2024-05-08T23:23:34.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.268+0000 7f78a4469b80 10 client.20385 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:34.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.268+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ba47104560 con 0x55ba471f4720 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.268+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787801e040 con 0x55ba471f4720 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f788401b650 con 0x55ba471f4720 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 10 client.20385 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 10 client.20385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 20 client.20385 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:34.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47267da0 0x55ba4726a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 4 client.20385 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ba47136ec0 con 0x55ba47267da0 2024-05-08T23:23:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.269+0000 7f7889e00700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47267da0 0x55ba4726a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.270+0000 7f786b400700 10 client.20385 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:23:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.270+0000 7f7889e00700 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47267da0 0x55ba4726a1c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9927cf6824a6d65a server_cookie=4d81858baec852ec in_seq=0 out_seq=0 2024-05-08T23:23:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.271+0000 7f786b400700 1 -- 172.21.15.46:0/1785758099 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ba47136ec0 con 0x55ba47267da0 2024-05-08T23:23:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.271+0000 7f786b400700 10 client.20385 handle_command_reply: tid=0 2024-05-08T23:23:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.271+0000 7f78a4469b80 1 client.20385 shutdown 2024-05-08T23:23:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.271+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 msgr2=0x55ba47266060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.271+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7884004380 tx=0x7f7884024e10 comp rx=0 tx=0).stop 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 shutdown_connections 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55ba47267da0 0x55ba4726a1c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba471f4720 0x55ba47266060 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba471ee890 0x55ba471fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 --2- 172.21.15.46:0/1785758099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba471f3e00 0x55ba471fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 >> 172.21.15.46:0/1785758099 conn(0x55ba4714c210 msgr2=0x55ba471f2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.272+0000 7f786b400700 0 client.20385 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:23:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.274+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 shutdown_connections 2024-05-08T23:23:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.274+0000 7f78a4469b80 1 -- 172.21.15.46:0/1785758099 wait complete. 2024-05-08T23:23:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:23:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.275+0000 7f78a4469b80 20 client.20385 trim_cache size 0 max 16384 2024-05-08T23:23:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:33 smithi046 ceph-mon[36898]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-08T23:23:34.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 msgr2=0x7f78700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 0x7f78700a3750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f788c198810 tx=0x7f787800b260 comp rx=0 tx=0).stop 2024-05-08T23:23:34.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 msgr2=0x7f788c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9a0 tx=0x7f787c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 shutdown_connections 2024-05-08T23:23:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f788c10b3c0 0x7f788c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f78700a1290 0x7f78700a3750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f788c101fa0 0x7f788c1972c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 --2- 172.21.15.46:0/3258399539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788c1028c0 0x7f788c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.277+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 >> 172.21.15.46:0/3258399539 conn(0x7f788c0fb760 msgr2=0x7f788c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.278+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 shutdown_connections 2024-05-08T23:23:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.278+0000 7f7890a00700 1 -- 172.21.15.46:0/3258399539 wait complete. 2024-05-08T23:23:34.285 INFO:tasks.workunit.client.0.smithi046.stdout: 10 134088 19.47 MB/sec execute 235 sec latency 1762.962 ms 2024-05-08T23:23:34.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:23:34.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:33 smithi086 ceph-mon[78610]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 398 op/s 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.592+0000 7f3e28e00700 1 -- 172.21.15.46:0/696357858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e24105fe0 msgr2=0x7f3e2410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.592+0000 7f3e28e00700 1 --2- 172.21.15.46:0/696357858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e24105fe0 0x7f3e2410db60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b0a0 tx=0x7f3e1c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 -- 172.21.15.46:0/696357858 shutdown_connections 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 --2- 172.21.15.46:0/696357858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e24105fe0 0x7f3e2410db60 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 --2- 172.21.15.46:0/696357858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e241012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 --2- 172.21.15.46:0/696357858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24101870 0x7f3e241058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 -- 172.21.15.46:0/696357858 >> 172.21.15.46:0/696357858 conn(0x7f3e240fc740 msgr2=0x7f3e240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 -- 172.21.15.46:0/696357858 shutdown_connections 2024-05-08T23:23:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.593+0000 7f3e28e00700 1 -- 172.21.15.46:0/696357858 wait complete. 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.594+0000 7f3e28e00700 1 Processor -- start 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.594+0000 7f3e28e00700 1 -- start start 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 0x7f3e24199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 0x7f3e2419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419e9a0 con 0x7f3e24100ec0 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419eb10 con 0x7f3e24105fe0 2024-05-08T23:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e28e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419ecd0 con 0x7f3e2419df80 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e23400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e23400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58716/0 (socket says 172.21.15.46:58716) 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e23400700 1 -- 172.21.15.46:0/3362898441 learned_addr learned my addr 172.21.15.46:0/3362898441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.595+0000 7f3e23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 0x7f3e2419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 -- 172.21.15.46:0/3362898441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 msgr2=0x7f3e2419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 0x7f3e24199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 0x7f3e2419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 -- 172.21.15.46:0/3362898441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 msgr2=0x7f3e24199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 0x7f3e24199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 -- 172.21.15.46:0/3362898441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c009d00 con 0x7f3e24100ec0 2024-05-08T23:23:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 0x7f3e2419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 0x7f3e24199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e23400700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9a0 tx=0x7f3e1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e1be00700 1 -- 172.21.15.46:0/3362898441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14009790 con 0x7f3e24100ec0 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e1be00700 1 -- 172.21.15.46:0/3362898441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e1400cbd0 con 0x7f3e24100ec0 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e241a62b0 con 0x7f3e24100ec0 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e1be00700 1 -- 172.21.15.46:0/3362898441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14018680 con 0x7f3e24100ec0 2024-05-08T23:23:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.596+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e241a67e0 con 0x7f3e24100ec0 2024-05-08T23:23:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.600+0000 7f3e1be00700 1 -- 172.21.15.46:0/3362898441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3e140187e0 con 0x7f3e24100ec0 2024-05-08T23:23:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.602+0000 7f3e1be00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 0x7f3e080a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.602+0000 7f3e3de32b80 1 Processor -- start 2024-05-08T23:23:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.602+0000 7f3e1be00700 1 -- 172.21.15.46:0/3362898441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3e14014070 con 0x7f3e24100ec0 2024-05-08T23:23:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.602+0000 7f3e3de32b80 1 -- start start 2024-05-08T23:23:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 0x7f3e080a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e277d0 0x55b943e27bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e220a0 0x55b943e224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943d70150 con 0x55b943e280f0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943d6db10 con 0x55b943e277d0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943d69400 con 0x55b943e220a0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58730/0 (socket says 172.21.15.46:58730) 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e22a00700 1 -- 172.21.15.46:0/3197455832 learned_addr learned my addr 172.21.15.46:0/3197455832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.603+0000 7f3e23400700 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e277d0 0x55b943e27bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e23e00700 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e220a0 0x55b943e224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 -- 172.21.15.46:0/3197455832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e220a0 msgr2=0x55b943e224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e220a0 0x55b943e224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 -- 172.21.15.46:0/3197455832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e277d0 msgr2=0x55b943e27bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e277d0 0x55b943e27bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 -- 172.21.15.46:0/3197455832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b943bea3b0 con 0x55b943e280f0 2024-05-08T23:23:34.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 0x7f3e080a3870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e10013510 tx=0x7f3e10009f90 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.604+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e10013c60 tx=0x7f3e1001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d78689ae25dc00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.605+0000 7f3e18c00700 1 -- 172.21.15.46:0/3197455832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10011db0 con 0x55b943e280f0 2024-05-08T23:23:34.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.605+0000 7f3e18c00700 1 -- 172.21.15.46:0/3197455832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e10019070 con 0x55b943e280f0 2024-05-08T23:23:34.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.605+0000 7f3e18c00700 1 -- 172.21.15.46:0/3197455832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1001f440 con 0x55b943e280f0 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3197455832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 msgr2=0x55b943e21b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e10013c60 tx=0x7f3e1001d000 comp rx=0 tx=0).stop 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3197455832 shutdown_connections 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e220a0 0x55b943e224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e277d0 0x55b943e27bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3197455832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e21b60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3197455832 >> 172.21.15.46:0/3197455832 conn(0x55b943d77210 msgr2=0x55b943e142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3197455832 shutdown_connections 2024-05-08T23:23:34.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.606+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3197455832 wait complete. 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.607+0000 7f3e3de32b80 1 -- start start 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.607+0000 7f3e3de32b80 1 Processor -- start 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e2b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943d05560 con 0x55b943e280f0 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943d05200 con 0x55b943e220a0 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943bec6a0 con 0x55b943e277d0 2024-05-08T23:23:34.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e3de32b80 10 client.0 _ll_register_callbacks cb 0x55b943d0eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47360/0 (socket says 172.21.15.46:47360) 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.608+0000 7f3e22a00700 1 -- 172.21.15.46:0/3455550515 learned_addr learned my addr 172.21.15.46:0/3455550515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e23400700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e23e00700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 -- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 msgr2=0x55b943e2b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e2b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 -- 172.21.15.46:0/3455550515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 msgr2=0x55b943e2bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e2bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 -- 172.21.15.46:0/3455550515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e10011380 con 0x55b943e277d0 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e23e00700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e2bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e22a00700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005e10 tx=0x7f3e100057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6a797b28f21072 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.609+0000 7f3e0d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003bd0 con 0x55b943e277d0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b943cc2270 con 0x55b943e277d0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3e10003d30 con 0x55b943e277d0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e10003f10 con 0x55b943e277d0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e3de32b80 10 client.30646 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b943b96b60 con 0x55b943e277d0 2024-05-08T23:23:34.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.610+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b943cc2270 con 0x55b943e277d0 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e10026070 con 0x55b943e277d0 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 10 client.30646 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 10 client.30646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 20 client.30646 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e3bb30 0x55b943e3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 4 client.30646 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b943de8810 con 0x55b943e3bb30 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.611+0000 7f3e23400700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e3bb30 0x55b943e3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e0d600700 10 client.30646 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e23400700 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e3bb30 0x55b943e3df50 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f2ac58145939a9 server_cookie=9f6ec281c38f2ea9 in_seq=0 out_seq=0 2024-05-08T23:23:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e0d600700 1 -- 172.21.15.46:0/3455550515 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b943de8810 con 0x55b943e3bb30 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e0d600700 10 client.30646 handle_command_reply: tid=0 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e3de32b80 1 client.30646 shutdown 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 msgr2=0x55b943e2ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.612+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005e10 tx=0x7f3e100057a0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 shutdown_connections 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e3bb30 0x55b943e3df50 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e2bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e2b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e0d600700 0 client.30646 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/3455550515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e2ba30 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 >> 172.21.15.46:0/3455550515 conn(0x55b943d77210 msgr2=0x55b943e137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 shutdown_connections 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 1 -- 172.21.15.46:0/3455550515 wait complete. 2024-05-08T23:23:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.613+0000 7f3e3de32b80 20 client.30646 trim_cache size 0 max 16384 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.631+0000 7f3e3de32b80 1 Processor -- start 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.631+0000 7f3e3de32b80 1 -- start start 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e220a0 0x55b943e3df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e280f0 0x55b943e32af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e10dc0 con 0x55b943e220a0 2024-05-08T23:23:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e14830 con 0x55b943e280f0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e15820 con 0x55b943e277d0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47374/0 (socket says 172.21.15.46:47374) 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e22a00700 1 -- 172.21.15.46:0/2182803122 learned_addr learned my addr 172.21.15.46:0/2182803122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e23e00700 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e280f0 0x55b943e32af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e23400700 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e220a0 0x55b943e3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e22a00700 1 -- 172.21.15.46:0/2182803122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e280f0 msgr2=0x55b943e32af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.632+0000 7f3e22a00700 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e280f0 0x55b943e32af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e22a00700 1 -- 172.21.15.46:0/2182803122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e220a0 msgr2=0x55b943e3df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e22a00700 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e220a0 0x55b943e3df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e22a00700 1 -- 172.21.15.46:0/2182803122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b943d2f560 con 0x55b943e277d0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e22a00700 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3e100097a0 tx=0x7f3e10000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b79bc9e7ce2bed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e19600700 1 -- 172.21.15.46:0/2182803122 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1001fde0 con 0x55b943e277d0 2024-05-08T23:23:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e19600700 1 -- 172.21.15.46:0/2182803122 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e100050d0 con 0x55b943e277d0 2024-05-08T23:23:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.633+0000 7f3e19600700 1 -- 172.21.15.46:0/2182803122 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1001f660 con 0x55b943e277d0 2024-05-08T23:23:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 -- 172.21.15.46:0/2182803122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 msgr2=0x55b943e22480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3e100097a0 tx=0x7f3e10000bc0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 -- 172.21.15.46:0/2182803122 shutdown_connections 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e280f0 0x55b943e32af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e220a0 0x55b943e3df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/2182803122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e22480 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 -- 172.21.15.46:0/2182803122 >> 172.21.15.46:0/2182803122 conn(0x55b943d77210 msgr2=0x55b943e10210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 -- 172.21.15.46:0/2182803122 shutdown_connections 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.634+0000 7f3e3de32b80 1 -- 172.21.15.46:0/2182803122 wait complete. 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.635+0000 7f3e3de32b80 1 -- start start 2024-05-08T23:23:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.635+0000 7f3e3de32b80 1 Processor -- start 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e33040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e311c0 con 0x55b943e280f0 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e31330 con 0x55b943e220a0 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b943e314f0 con 0x55b943e277d0 2024-05-08T23:23:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e3de32b80 10 client.0 _ll_register_callbacks cb 0x55b943d0eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e23e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e33040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e23e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58750/0 (socket says 172.21.15.46:58750) 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.638+0000 7f3e23e00700 1 -- 172.21.15.46:0/4194152802 learned_addr learned my addr 172.21.15.46:0/4194152802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 -- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 msgr2=0x55b943e32b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 -- 172.21.15.46:0/4194152802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 msgr2=0x55b943e30ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 -- 172.21.15.46:0/4194152802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b943b96b60 con 0x55b943e277d0 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e23e00700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e22a00700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e33040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55b943e2c590 tx=0x7f3e1001fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669c111705dd6725 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e0d600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003cd0 con 0x55b943e277d0 2024-05-08T23:23:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.639+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f3e10003e30 con 0x55b943e277d0 2024-05-08T23:23:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.640+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e10019070 con 0x55b943e277d0 2024-05-08T23:23:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.641+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e10005540 con 0x55b943e277d0 2024-05-08T23:23:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.641+0000 7f3e3de32b80 10 client.30652 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.641+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b943beecd0 con 0x55b943e277d0 2024-05-08T23:23:34.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e10019070 con 0x55b943e277d0 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e10026070 con 0x55b943e277d0 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 10 client.30652 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 10 client.30652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 20 client.30652 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e91c00 0x55b943e93ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 4 client.30652 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:34.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.642+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b943d61ec0 con 0x55b943e91c00 2024-05-08T23:23:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.644+0000 7f3e23400700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e91c00 0x55b943e93ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.645+0000 7f3e23400700 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e91c00 0x55b943e93ff0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03e6b49a2f505d7 server_cookie=e9ca7a943144ecba in_seq=0 out_seq=0 2024-05-08T23:23:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.645+0000 7f3e0d600700 10 client.30652 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.645+0000 7f3e0d600700 1 -- 172.21.15.46:0/4194152802 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b943d61ec0 con 0x55b943e91c00 2024-05-08T23:23:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.645+0000 7f3e0d600700 10 client.30652 handle_command_reply: tid=0 2024-05-08T23:23:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.645+0000 7f3e3de32b80 1 client.30652 shutdown 2024-05-08T23:23:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 msgr2=0x55b943e33040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e33040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55b943e2c590 tx=0x7f3e1001fed0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 shutdown_connections 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x55b943e91c00 0x55b943e93ff0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b943e280f0 0x55b943e30ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b943e220a0 0x55b943e32b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e0d600700 0 client.30652 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 --2- 172.21.15.46:0/4194152802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b943e277d0 0x55b943e33040 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 >> 172.21.15.46:0/4194152802 conn(0x55b943d77210 msgr2=0x55b943e22480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 shutdown_connections 2024-05-08T23:23:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.646+0000 7f3e3de32b80 1 -- 172.21.15.46:0/4194152802 wait complete. 2024-05-08T23:23:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e3de32b80 20 client.30652 trim_cache size 0 max 16384 2024-05-08T23:23:34.648 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:23:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 msgr2=0x7f3e080a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 0x7f3e080a3870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e10013510 tx=0x7f3e10009f90 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 msgr2=0x7f3e24199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9a0 tx=0x7f3e1400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 shutdown_connections 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2419df80 0x7f3e2419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f3e080a13b0 0x7f3e080a3870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100ec0 0x7f3e24199760 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 --2- 172.21.15.46:0/3362898441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e24105fe0 0x7f3e24199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.647+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 >> 172.21.15.46:0/3362898441 conn(0x7f3e240fc740 msgr2=0x7f3e240fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.648+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 shutdown_connections 2024-05-08T23:23:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:34.648+0000 7f3e28e00700 1 -- 172.21.15.46:0/3362898441 wait complete. 2024-05-08T23:23:35.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 134484 19.48 MB/sec execute 236 sec latency 2763.024 ms 2024-05-08T23:23:35.669 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:23:35.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:23:35.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 -- 172.21.15.46:0/4293838928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 msgr2=0x7f97781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 --2- 172.21.15.46:0/4293838928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97781012b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f976c009990 tx=0x7f976c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 -- 172.21.15.46:0/4293838928 shutdown_connections 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 --2- 172.21.15.46:0/4293838928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778105ff0 0x7f977810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 --2- 172.21.15.46:0/4293838928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97781012b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 --2- 172.21.15.46:0/4293838928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778101880 0x7f97781058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 -- 172.21.15.46:0/4293838928 >> 172.21.15.46:0/4293838928 conn(0x7f97780fc750 msgr2=0x7f97780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.927+0000 7f977e600700 1 -- 172.21.15.46:0/4293838928 shutdown_connections 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.928+0000 7f977e600700 1 -- 172.21.15.46:0/4293838928 wait complete. 2024-05-08T23:23:35.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.929+0000 7f977e600700 1 Processor -- start 2024-05-08T23:23:35.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.929+0000 7f977e600700 1 -- start start 2024-05-08T23:23:35.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 0x7f97781a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97781a5860 con 0x7f9778073d20 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97781a59d0 con 0x7f9778100ed0 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97781a5b90 con 0x7f9778105ff0 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f9777e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f9777e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47384/0 (socket says 172.21.15.46:47384) 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97780732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43564/0 (socket says 172.21.15.46:43564) 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f9777e00700 1 -- 172.21.15.46:0/2863363516 learned_addr learned my addr 172.21.15.46:0/2863363516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:35.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.930+0000 7f977dc00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 0x7f97781a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 -- 172.21.15.46:0/2863363516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 msgr2=0x7f97780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97780732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 -- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 msgr2=0x7f97781a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 0x7f97781a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 -- 172.21.15.46:0/2863363516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0095f0 con 0x7f9778105ff0 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f977dc00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 0x7f97781a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9777e00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f976400e9f0 tx=0x7f976400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9775600700 1 -- 172.21.15.46:0/2863363516 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764009750 con 0x7f9778105ff0 2024-05-08T23:23:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9775600700 1 -- 172.21.15.46:0/2863363516 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9764004cf0 con 0x7f9778105ff0 2024-05-08T23:23:35.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97781a5e70 con 0x7f9778105ff0 2024-05-08T23:23:35.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f9775600700 1 -- 172.21.15.46:0/2863363516 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400eee0 con 0x7f9778105ff0 2024-05-08T23:23:35.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.931+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97781a62f0 con 0x7f9778105ff0 2024-05-08T23:23:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.938+0000 7f9775600700 1 -- 172.21.15.46:0/2863363516 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9764004750 con 0x7f9778105ff0 2024-05-08T23:23:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.938+0000 7f9793655b80 1 Processor -- start 2024-05-08T23:23:35.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.939+0000 7f9793655b80 1 -- start start 2024-05-08T23:23:35.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.941+0000 7f9775600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 0x7f97600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.942+0000 7f9775600700 1 -- 172.21.15.46:0/2863363516 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9764014070 con 0x7f9778105ff0 2024-05-08T23:23:35.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f977dc00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 0x7f97600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03011d0 con 0x559da03b88f0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da02feb90 con 0x559da03b2be0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.943+0000 7f9793655b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da02fa480 con 0x559da03b35a0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43570/0 (socket says 172.21.15.46:43570) 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 -- 172.21.15.46:0/1583023769 learned_addr learned my addr 172.21.15.46:0/1583023769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f977d200700 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 -- 172.21.15.46:0/1583023769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 msgr2=0x559da03bf6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:23:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bf6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 -- 172.21.15.46:0/1583023769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 msgr2=0x559da03b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.944+0000 7f9777e00700 1 -- 172.21.15.46:0/1583023769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da017b430 con 0x559da03b2be0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.945+0000 7f9777e00700 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9764096550 tx=0x7f9764099c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c85c58932ec2bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.945+0000 7f977dc00700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 0x7f97600a3870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97700093a0 tx=0x7f977000b040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.945+0000 7f976a000700 1 -- 172.21.15.46:0/1583023769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976409a050 con 0x559da03b2be0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.945+0000 7f976a000700 1 -- 172.21.15.46:0/1583023769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f976409de80 con 0x559da03b2be0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.945+0000 7f976a000700 1 -- 172.21.15.46:0/1583023769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640ca910 con 0x559da03b2be0 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 -- 172.21.15.46:0/1583023769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 msgr2=0x559da03b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9764096550 tx=0x7f9764099c10 comp rx=0 tx=0).stop 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 -- 172.21.15.46:0/1583023769 shutdown_connections 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.946+0000 7f9793655b80 1 --2- 172.21.15.46:0/1583023769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03b3060 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.947+0000 7f9793655b80 1 -- 172.21.15.46:0/1583023769 >> 172.21.15.46:0/1583023769 conn(0x559da0308290 msgr2=0x559da03a5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.947+0000 7f9793655b80 1 -- 172.21.15.46:0/1583023769 shutdown_connections 2024-05-08T23:23:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.948+0000 7f9793655b80 1 -- 172.21.15.46:0/1583023769 wait complete. 2024-05-08T23:23:35.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.948+0000 7f9793655b80 1 -- start start 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 Processor -- start 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da02965e0 con 0x559da03b88f0 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da0296280 con 0x559da03b2be0 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.949+0000 7f9793655b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da017d720 con 0x559da03b35a0 2024-05-08T23:23:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f9793655b80 10 client.0 _ll_register_callbacks cb 0x559da029fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43578/0 (socket says 172.21.15.46:43578) 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 -- 172.21.15.46:0/2710200670 learned_addr learned my addr 172.21.15.46:0/2710200670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f9777e00700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977dc00700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 -- 172.21.15.46:0/2710200670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 msgr2=0x559da03bcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 -- 172.21.15.46:0/2710200670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 msgr2=0x559da03bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.950+0000 7f977d200700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03bd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f977d200700 1 -- 172.21.15.46:0/2710200670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97781a62f0 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f977d200700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f976c005800 tx=0x7f976c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e230f76d70c806a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f975ea00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c01d070 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f976c004500 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.951+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da02532f0 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.952+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f976c0046c0 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.952+0000 7f9793655b80 10 client.30662 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.952+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559da0127be0 con 0x559da03b2be0 2024-05-08T23:23:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.952+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559da02532f0 con 0x559da03b2be0 2024-05-08T23:23:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f976c00f870 con 0x559da03b2be0 2024-05-08T23:23:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 10 client.30662 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 10 client.30662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 20 client.30662 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03ddcb0 0x559da03e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 4 client.30662 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.953+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da0379890 con 0x559da03ddcb0 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.954+0000 7f9777e00700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03ddcb0 0x559da03e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.954+0000 7f9777e00700 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03ddcb0 0x559da03e00d0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3277d8b3eac15e8 server_cookie=db62e53b05e6e594 in_seq=0 out_seq=0 2024-05-08T23:23:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.954+0000 7f975ea00700 10 client.30662 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.955+0000 7f975ea00700 1 -- 172.21.15.46:0/2710200670 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559da0379890 con 0x559da03ddcb0 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.955+0000 7f975ea00700 10 client.30662 handle_command_reply: tid=0 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.955+0000 7f9793655b80 1 client.30662 shutdown 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 msgr2=0x559da03bca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f976c005800 tx=0x7f976c00b610 comp rx=0 tx=0).stop 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 shutdown_connections 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03ddcb0 0x559da03e00d0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b88f0 0x559da03bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da03bca50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 --2- 172.21.15.46:0/2710200670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b35a0 0x559da03bcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 >> 172.21.15.46:0/2710200670 conn(0x559da0308290 msgr2=0x559da03a50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f975ea00700 0 client.30662 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 shutdown_connections 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.956+0000 7f9793655b80 1 -- 172.21.15.46:0/2710200670 wait complete. 2024-05-08T23:23:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.957+0000 7f9793655b80 20 client.30662 trim_cache size 0 max 16384 2024-05-08T23:23:35.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.981+0000 7f9793655b80 1 Processor -- start 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 -- start start 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b2be0 0x559da03ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03be050 con 0x559da03b2be0 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03a68c0 con 0x559da03b35a0 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.982+0000 7f9793655b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03a6f10 con 0x559da03b88f0 2024-05-08T23:23:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43582/0 (socket says 172.21.15.46:43582) 2024-05-08T23:23:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 -- 172.21.15.46:0/385060294 learned_addr learned my addr 172.21.15.46:0/385060294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f977dc00700 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f977d200700 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b2be0 0x559da03ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 -- 172.21.15.46:0/385060294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 msgr2=0x559da03c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 -- 172.21.15.46:0/385060294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b2be0 msgr2=0x559da03ba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b2be0 0x559da03ba650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.983+0000 7f9777e00700 1 -- 172.21.15.46:0/385060294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da02c05e0 con 0x559da03b35a0 2024-05-08T23:23:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.984+0000 7f9777e00700 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9764096ac0 tx=0x7f9764099f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4566181ccddadb85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.985+0000 7f976aa00700 1 -- 172.21.15.46:0/385060294 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640b7070 con 0x559da03b35a0 2024-05-08T23:23:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.985+0000 7f976aa00700 1 -- 172.21.15.46:0/385060294 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97640a24d0 con 0x559da03b35a0 2024-05-08T23:23:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.985+0000 7f976aa00700 1 -- 172.21.15.46:0/385060294 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976407ea90 con 0x559da03b35a0 2024-05-08T23:23:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 -- 172.21.15.46:0/385060294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 msgr2=0x559da03c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9764096ac0 tx=0x7f9764099f20 comp rx=0 tx=0).stop 2024-05-08T23:23:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 -- 172.21.15.46:0/385060294 shutdown_connections 2024-05-08T23:23:35.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b2be0 0x559da03ba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 --2- 172.21.15.46:0/385060294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b35a0 0x559da03c3bb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 -- 172.21.15.46:0/385060294 >> 172.21.15.46:0/385060294 conn(0x559da0308290 msgr2=0x559da03e49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 -- 172.21.15.46:0/385060294 shutdown_connections 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.986+0000 7f9793655b80 1 -- 172.21.15.46:0/385060294 wait complete. 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.987+0000 7f9793655b80 1 -- start start 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.987+0000 7f9793655b80 1 Processor -- start 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da042f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b35a0 0x559da042f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da042ffa0 con 0x559da03b35a0 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03bc900 con 0x559da03b2be0 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da03bcac0 con 0x559da03b88f0 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f9793655b80 10 client.0 _ll_register_callbacks cb 0x559da029fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47412/0 (socket says 172.21.15.46:47412) 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 -- 172.21.15.46:0/2053705194 learned_addr learned my addr 172.21.15.46:0/2053705194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 -- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 msgr2=0x559da042f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da042f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 -- 172.21.15.46:0/2053705194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b35a0 msgr2=0x559da042f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b35a0 0x559da042f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.988+0000 7f977dc00700 1 -- 172.21.15.46:0/2053705194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559da017b430 con 0x559da03b88f0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.989+0000 7f977dc00700 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9770015e70 tx=0x7f9770004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286f5733a9cf3c29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.989+0000 7f975ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.989+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000e050 con 0x559da03b88f0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.989+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f97700223f0 con 0x559da03b88f0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.989+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f976409de80 con 0x559da03b88f0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.990+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97700225f0 con 0x559da03b88f0 2024-05-08T23:23:35.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.990+0000 7f9793655b80 10 client.30667 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.990+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559da017fd50 con 0x559da03b88f0 2024-05-08T23:23:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.991+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f976409de80 con 0x559da03b88f0 2024-05-08T23:23:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.993+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f977002a7f0 con 0x559da03b88f0 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 10 client.30667 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 10 client.30667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 20 client.30667 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03c3b00 0x559da03c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 4 client.30667 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559da02f2f40 con 0x559da03c3b00 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.994+0000 7f977d200700 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03c3b00 0x559da03c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.995+0000 7f977d200700 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03c3b00 0x559da03c3f10 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df857b11ea933ef8 server_cookie=ef604ece7a2e3ade in_seq=0 out_seq=0 2024-05-08T23:23:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.995+0000 7f975ea00700 10 client.30667 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f975ea00700 1 -- 172.21.15.46:0/2053705194 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559da02f2f40 con 0x559da03c3b00 2024-05-08T23:23:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f975ea00700 10 client.30667 handle_command_reply: tid=0 2024-05-08T23:23:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f9793655b80 1 client.30667 shutdown 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 msgr2=0x559da03bc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9770015e70 tx=0x7f9770004ad0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.997+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 shutdown_connections 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559da03b88f0 0x559da03bc3c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559da03b2be0 0x559da042f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x559da03c3b00 0x559da03c3f10 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f9793655b80 1 --2- 172.21.15.46:0/2053705194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559da03b35a0 0x559da042f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 >> 172.21.15.46:0/2053705194 conn(0x559da0308290 msgr2=0x559da03b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.998+0000 7f975ea00700 0 client.30667 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.999+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 shutdown_connections 2024-05-08T23:23:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.999+0000 7f9793655b80 1 -- 172.21.15.46:0/2053705194 wait complete. 2024-05-08T23:23:36.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:35.999+0000 7f9793655b80 20 client.30667 trim_cache size 0 max 16384 2024-05-08T23:23:36.000 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:23:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-08T23:23:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "94e2c03f-7d7a-4b31-aa72-03b5cfe92566", 2024-05-08T23:23:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-08T23:23:36.001 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:23:36.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.000+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 msgr2=0x7f97600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.000+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 0x7f97600a3870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97700093a0 tx=0x7f977000b040 comp rx=0 tx=0).stop 2024-05-08T23:23:36.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 msgr2=0x7f97780737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f976400e9f0 tx=0x7f976400ed00 comp rx=0 tx=0).stop 2024-05-08T23:23:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 shutdown_connections 2024-05-08T23:23:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f97600a13b0 0x7f97600a3870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9778073d20 0x7f97781a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9778100ed0 0x7f97780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 --2- 172.21.15.46:0/2863363516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9778105ff0 0x7f97780737e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.001+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 >> 172.21.15.46:0/2863363516 conn(0x7f97780fc750 msgr2=0x7f97780fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.003+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 shutdown_connections 2024-05-08T23:23:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.003+0000 7f977e600700 1 -- 172.21.15.46:0/2863363516 wait complete. 2024-05-08T23:23:36.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:35 smithi046 ceph-mon[36898]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 402 op/s 2024-05-08T23:23:36.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:36.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:35 smithi053 ceph-mon[77587]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 402 op/s 2024-05-08T23:23:36.285 INFO:tasks.workunit.client.0.smithi046.stdout: 10 134828 19.47 MB/sec execute 237 sec latency 3763.081 ms 2024-05-08T23:23:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 -- 172.21.15.46:0/2151393961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788112ef0 msgr2=0x7f67881152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 --2- 172.21.15.46:0/2151393961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788112ef0 0x7f67881152e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f67700098f0 tx=0x7f6770009c00 comp rx=0 tx=0).stop 2024-05-08T23:23:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 -- 172.21.15.46:0/2151393961 shutdown_connections 2024-05-08T23:23:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 --2- 172.21.15.46:0/2151393961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788112ef0 0x7f67881152e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 --2- 172.21.15.46:0/2151393961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788109940 0x7f6788109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 --2- 172.21.15.46:0/2151393961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 0x7f67881129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.309+0000 7f678e800700 1 -- 172.21.15.46:0/2151393961 >> 172.21.15.46:0/2151393961 conn(0x7f678806bf30 msgr2=0x7f678806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.310+0000 7f678e800700 1 -- 172.21.15.46:0/2151393961 shutdown_connections 2024-05-08T23:23:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.310+0000 7f678e800700 1 -- 172.21.15.46:0/2151393961 wait complete. 2024-05-08T23:23:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.312+0000 7f678e800700 1 Processor -- start 2024-05-08T23:23:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.312+0000 7f678e800700 1 -- start start 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788109940 0x7f67881a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 0x7f67881a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67881a5b30 con 0x7f6788109940 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67881a5ca0 con 0x7f6788112ef0 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f678e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67881a5e60 con 0x7f678810a260 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f6787e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f6787e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43604/0 (socket says 172.21.15.46:43604) 2024-05-08T23:23:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.313+0000 7f6787e00700 1 -- 172.21.15.46:0/1199676945 learned_addr learned my addr 172.21.15.46:0/1199676945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787400700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788109940 0x7f67881a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6786a00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 0x7f67881a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787e00700 1 -- 172.21.15.46:0/1199676945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 msgr2=0x7f67881a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787e00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 0x7f67881a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787e00700 1 -- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788109940 msgr2=0x7f67881a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787e00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788109940 0x7f67881a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.314+0000 7f6787e00700 1 -- 172.21.15.46:0/1199676945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800095f0 con 0x7f6788112ef0 2024-05-08T23:23:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.315+0000 7f6787e00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6770009230 tx=0x7f677000f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.316+0000 7f677fe00700 1 -- 172.21.15.46:0/1199676945 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677001c070 con 0x7f6788112ef0 2024-05-08T23:23:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.316+0000 7f677fe00700 1 -- 172.21.15.46:0/1199676945 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f677000bc10 con 0x7f6788112ef0 2024-05-08T23:23:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.316+0000 7f677fe00700 1 -- 172.21.15.46:0/1199676945 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67700038e0 con 0x7f6788112ef0 2024-05-08T23:23:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.316+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6770009520 con 0x7f6788112ef0 2024-05-08T23:23:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.316+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67881ae8c0 con 0x7f6788112ef0 2024-05-08T23:23:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.321+0000 7f677fe00700 1 -- 172.21.15.46:0/1199676945 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6770020030 con 0x7f6788112ef0 2024-05-08T23:23:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.323+0000 7f67a2413b80 1 Processor -- start 2024-05-08T23:23:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.323+0000 7f67a2413b80 1 -- start start 2024-05-08T23:23:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.325+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.325+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x560808755b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x5608087564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f67a2413b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608086a4150 con 0x5608087560a0 2024-05-08T23:23:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f67a2413b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608086a1b10 con 0x56080875b610 2024-05-08T23:23:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f67a2413b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080869d400 con 0x56080875bf30 2024-05-08T23:23:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f6786a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f6786a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43608/0 (socket says 172.21.15.46:43608) 2024-05-08T23:23:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f6786a00700 1 -- 172.21.15.46:0/1795642583 learned_addr learned my addr 172.21.15.46:0/1795642583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f677fe00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 0x7f676c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.326+0000 7f6787400700 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x560808755b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f677fe00700 1 -- 172.21.15.46:0/1199676945 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f67700cdfd0 con 0x7f6788112ef0 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f6786a00700 1 -- 172.21.15.46:0/1795642583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 msgr2=0x560808755b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f6787e00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 0x7f676c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f6786a00700 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x560808755b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f6786a00700 1 -- 172.21.15.46:0/1795642583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 msgr2=0x5608087564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.327+0000 7f6786a00700 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x5608087564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.328+0000 7f6787e00700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 0x7f676c0a37c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f67700a2bc0 tx=0x7f6770005040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.328+0000 7f6786a00700 1 -- 172.21.15.46:0/1795642583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56080851e3b0 con 0x56080875b610 2024-05-08T23:23:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.329+0000 7f6786a00700 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f677800a480 tx=0x7f677800a790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6709470f8bc826 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.329+0000 7f677cc00700 1 -- 172.21.15.46:0/1795642583 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778009bd0 con 0x56080875b610 2024-05-08T23:23:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.329+0000 7f677cc00700 1 -- 172.21.15.46:0/1795642583 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6778010b10 con 0x56080875b610 2024-05-08T23:23:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.329+0000 7f677cc00700 1 -- 172.21.15.46:0/1795642583 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778016c60 con 0x56080875b610 2024-05-08T23:23:36.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.330+0000 7f67a2413b80 1 -- 172.21.15.46:0/1795642583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 msgr2=0x56080875b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.330+0000 7f67a2413b80 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f677800a480 tx=0x7f677800a790 comp rx=0 tx=0).stop 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 -- 172.21.15.46:0/1795642583 shutdown_connections 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x5608087564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x56080875b9f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 --2- 172.21.15.46:0/1795642583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x560808755b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 -- 172.21.15.46:0/1795642583 >> 172.21.15.46:0/1795642583 conn(0x5608086ab210 msgr2=0x560808747df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 -- 172.21.15.46:0/1795642583 shutdown_connections 2024-05-08T23:23:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.331+0000 7f67a2413b80 1 -- 172.21.15.46:0/1795642583 wait complete. 2024-05-08T23:23:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.332+0000 7f67a2413b80 1 -- start start 2024-05-08T23:23:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.332+0000 7f67a2413b80 1 Processor -- start 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080875f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875bf30 0x56080875fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560808639560 con 0x5608087560a0 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560808639200 con 0x56080875bf30 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608085206a0 con 0x56080875b610 2024-05-08T23:23:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.333+0000 7f67a2413b80 10 client.0 _ll_register_callbacks cb 0x560808642b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.334+0000 7f6787400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.334+0000 7f6787400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:47432/0 (socket says 172.21.15.46:47432) 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.334+0000 7f6787400700 1 -- 172.21.15.46:0/2546055417 learned_addr learned my addr 172.21.15.46:0/2546055417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.334+0000 7f6787400700 1 -- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875bf30 msgr2=0x56080875fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.334+0000 7f6787400700 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875bf30 0x56080875fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6787400700 1 -- 172.21.15.46:0/2546055417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 msgr2=0x56080875f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6787400700 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080875f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6787400700 1 -- 172.21.15.46:0/2546055417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6778009d30 con 0x56080875b610 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6787400700 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6780013810 tx=0x7f6780013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd78d7603b5a8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6775600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780011af0 con 0x56080875b610 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6780011c50 con 0x56080875b610 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.335+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608085f6270 con 0x56080875b610 2024-05-08T23:23:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.336+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6780011e10 con 0x56080875b610 2024-05-08T23:23:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.336+0000 7f67a2413b80 10 client.30676 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.336+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5608084cab60 con 0x56080875b610 2024-05-08T23:23:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608085f6270 con 0x56080875b610 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f678001d630 con 0x56080875b610 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 10 client.30676 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 10 client.30676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 20 client.30676 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56080876fc30 0x560808772050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 4 client.30676 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.337+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56080871c810 con 0x56080876fc30 2024-05-08T23:23:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.341+0000 7f6786a00700 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56080876fc30 0x560808772050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.342+0000 7f6786a00700 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56080876fc30 0x560808772050 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e6495acc1c346d server_cookie=ec04fbbd46463fdd in_seq=0 out_seq=0 2024-05-08T23:23:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.342+0000 7f6775600700 10 client.30676 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.343+0000 7f6775600700 1 -- 172.21.15.46:0/2546055417 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56080871c810 con 0x56080876fc30 2024-05-08T23:23:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.343+0000 7f6775600700 10 client.30676 handle_command_reply: tid=0 2024-05-08T23:23:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.343+0000 7f67a2413b80 1 client.30676 shutdown 2024-05-08T23:23:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.343+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 msgr2=0x56080875f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.343+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6780013810 tx=0x7f6780013bd0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 shutdown_connections 2024-05-08T23:23:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875bf30 0x56080875fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x56080876fc30 0x560808772050 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080875f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2546055417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875b610 0x56080875f940 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 >> 172.21.15.46:0/2546055417 conn(0x5608086ab210 msgr2=0x5608087475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f6775600700 0 client.30676 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 shutdown_connections 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 1 -- 172.21.15.46:0/2546055417 wait complete. 2024-05-08T23:23:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.344+0000 7f67a2413b80 20 client.30676 trim_cache size 0 max 16384 2024-05-08T23:23:36.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.373+0000 7f67a2413b80 1 Processor -- start 2024-05-08T23:23:36.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.374+0000 7f67a2413b80 1 -- start start 2024-05-08T23:23:36.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080876bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x56080875fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080875d320 con 0x5608087560a0 2024-05-08T23:23:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560808759de0 con 0x56080875b610 2024-05-08T23:23:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f67a2413b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560808748260 con 0x56080875bf30 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f6787400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f6787400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:43632/0 (socket says 172.21.15.46:43632) 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f6787400700 1 -- 172.21.15.46:0/2957162594 learned_addr learned my addr 172.21.15.46:0/2957162594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.375+0000 7f6786a00700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080876bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6786a00700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x56080875fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 -- 172.21.15.46:0/2957162594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 msgr2=0x56080875fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x56080875fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 -- 172.21.15.46:0/2957162594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 msgr2=0x56080876bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080876bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 -- 172.21.15.46:0/2957162594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560808663560 con 0x56080875b610 2024-05-08T23:23:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6786a00700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x56080875fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f6787400700 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6780013fd0 tx=0x7f6780001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4e5a9593020288 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.376+0000 7f677d600700 1 -- 172.21.15.46:0/2957162594 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678001d3f0 con 0x56080875b610 2024-05-08T23:23:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.377+0000 7f677d600700 1 -- 172.21.15.46:0/2957162594 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678001d550 con 0x56080875b610 2024-05-08T23:23:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.377+0000 7f677d600700 1 -- 172.21.15.46:0/2957162594 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800118f0 con 0x56080875b610 2024-05-08T23:23:36.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 -- 172.21.15.46:0/2957162594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 msgr2=0x560808747d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6780013fd0 tx=0x7f6780001e70 comp rx=0 tx=0).stop 2024-05-08T23:23:36.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 -- 172.21.15.46:0/2957162594 shutdown_connections 2024-05-08T23:23:36.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x56080875fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x56080876bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2957162594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808747d20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.378+0000 7f67a2413b80 1 -- 172.21.15.46:0/2957162594 >> 172.21.15.46:0/2957162594 conn(0x5608086ab210 msgr2=0x560808744210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.379+0000 7f67a2413b80 1 -- 172.21.15.46:0/2957162594 shutdown_connections 2024-05-08T23:23:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.379+0000 7f67a2413b80 1 -- 172.21.15.46:0/2957162594 wait complete. 2024-05-08T23:23:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.379+0000 7f67a2413b80 1 -- start start 2024-05-08T23:23:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.381+0000 7f67a2413b80 1 Processor -- start 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808766480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x5608087c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560808766d60 con 0x5608087560a0 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608087c5670 con 0x56080875b610 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.382+0000 7f67a2413b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608087c57b0 con 0x56080875bf30 2024-05-08T23:23:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.383+0000 7f67a2413b80 10 client.0 _ll_register_callbacks cb 0x560808642b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.383+0000 7f6786a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.383+0000 7f6786a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58856/0 (socket says 172.21.15.46:58856) 2024-05-08T23:23:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.383+0000 7f6786a00700 1 -- 172.21.15.46:0/2241963753 learned_addr learned my addr 172.21.15.46:0/2241963753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:23:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.384+0000 7f6786a00700 1 -- 172.21.15.46:0/2241963753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 msgr2=0x5608087c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.384+0000 7f6786a00700 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x5608087c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.384+0000 7f6786a00700 1 -- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 msgr2=0x560808766480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.384+0000 7f6786a00700 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808766480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.384+0000 7f6786a00700 1 -- 172.21.15.46:0/2241963753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608084cab60 con 0x5608087560a0 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.385+0000 7f6786a00700 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f677800a7d0 tx=0x7f67780096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e057a7edb60a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.385+0000 7f6775600700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:23:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.385+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778010c80 con 0x5608087560a0 2024-05-08T23:23:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.385+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f6778006e40 con 0x5608087560a0 2024-05-08T23:23:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.386+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6778010b10 con 0x5608087560a0 2024-05-08T23:23:36.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.387+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6778010de0 con 0x5608087560a0 2024-05-08T23:23:36.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.387+0000 7f67a2413b80 10 client.20424 fetch_fsmap learned FSMap version 36 2024-05-08T23:23:36.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.387+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560808522cd0 con 0x5608087560a0 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.388+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6778010b10 con 0x5608087560a0 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6778008070 con 0x5608087560a0 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 10 client.20424 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 10 client.20424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 20 client.20424 populate_metadata read hostname 'smithi046' 2024-05-08T23:23:36.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5608087c6e70 0x5608087c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 4 client.20424 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:36.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560808695ec0 con 0x5608087c6e70 2024-05-08T23:23:36.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.389+0000 7f6787400700 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5608087c6e70 0x5608087c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.391+0000 7f6787400700 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5608087c6e70 0x5608087c9290 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebba78b30fae40ad server_cookie=43c0008655c604d3 in_seq=0 out_seq=0 2024-05-08T23:23:36.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.391+0000 7f6775600700 10 client.20424 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:36.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.391+0000 7f6775600700 1 -- 172.21.15.46:0/2241963753 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560808695ec0 con 0x5608087c6e70 2024-05-08T23:23:36.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.391+0000 7f6775600700 10 client.20424 handle_command_reply: tid=0 2024-05-08T23:23:36.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.391+0000 7f67a2413b80 1 client.20424 shutdown 2024-05-08T23:23:36.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.393+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 msgr2=0x560808765f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.393+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f677800a7d0 tx=0x7f67780096d0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 shutdown_connections 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56080875bf30 0x5608087c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5608087c6e70 0x5608087c9290 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608087560a0 0x560808765f40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 --2- 172.21.15.46:0/2241963753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56080875b610 0x560808766480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f6775600700 0 client.20424 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:23:36.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.394+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 >> 172.21.15.46:0/2241963753 conn(0x5608086ab210 msgr2=0x560808744210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.395+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 shutdown_connections 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.395+0000 7f67a2413b80 1 -- 172.21.15.46:0/2241963753 wait complete. 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.395+0000 7f67a2413b80 20 client.20424 trim_cache size 0 max 16384 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:23:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "94e2c03f-7d7a-4b31-aa72-03b5cfe92566": { 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "94e2c03f-7d7a-4b31-aa72-03b5cfe92566", 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:23:36.397 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.396+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 msgr2=0x7f676c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.396+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 0x7f676c0a37c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f67700a2bc0 tx=0x7f6770005040 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.396+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 msgr2=0x7f67881a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.396+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6770009230 tx=0x7f677000f9e0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 shutdown_connections 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6788112ef0 0x7f67881a5530 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f676c0a1300 0x7f676c0a37c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788109940 0x7f67881a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 --2- 172.21.15.46:0/1199676945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678810a260 0x7f67881a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.397+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 >> 172.21.15.46:0/1199676945 conn(0x7f678806bf30 msgr2=0x7f678806fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.398+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 shutdown_connections 2024-05-08T23:23:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:23:36.398+0000 7f678e800700 1 -- 172.21.15.46:0/1199676945 wait complete. 2024-05-08T23:23:36.433 INFO:tasks.cephfs.filesystem:scrub status for tag:94e2c03f-7d7a-4b31-aa72-03b5cfe92566 - {'path': '/', 'tag': '94e2c03f-7d7a-4b31-aa72-03b5cfe92566', 'options': 'recursive,force'} 2024-05-08T23:23:36.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:35 smithi086 ceph-mon[78610]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 402 op/s 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub queued for path: / 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: active paths [/] 2024-05-08T23:23:37.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:36 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub queued for path: / 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: active paths [/] 2024-05-08T23:23:37.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:36 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.285 INFO:tasks.workunit.client.0.smithi046.stdout: 10 135215 19.49 MB/sec execute 238 sec latency 4763.148 ms 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub queued for path: / 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: active paths [/] 2024-05-08T23:23:37.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:36 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:37 smithi053 ceph-mon[77587]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 549 op/s 2024-05-08T23:23:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:37 smithi046 ceph-mon[36898]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 549 op/s 2024-05-08T23:23:38.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 135599 19.50 MB/sec execute 239 sec latency 5763.209 ms 2024-05-08T23:23:38.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:37 smithi086 ceph-mon[78610]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 549 op/s 2024-05-08T23:23:39.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:38 smithi053 ceph-mon[77587]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:38 smithi046 ceph-mon[36898]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 135951 19.48 MB/sec execute 240 sec latency 5866.214 ms 2024-05-08T23:23:39.472 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:38 smithi086 ceph-mon[78610]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:39 smithi053 ceph-mon[77587]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 418 op/s 2024-05-08T23:23:40.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:39 smithi046 ceph-mon[36898]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 418 op/s 2024-05-08T23:23:40.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 136285 19.48 MB/sec execute 241 sec latency 217.375 ms 2024-05-08T23:23:40.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:39 smithi086 ceph-mon[78610]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 418 op/s 2024-05-08T23:23:41.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 136736 19.49 MB/sec execute 242 sec latency 232.806 ms 2024-05-08T23:23:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:42 smithi053 ceph-mon[77587]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-08T23:23:42.837 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:42 smithi086 ceph-mon[78610]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-08T23:23:42.838 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:42 smithi046 ceph-mon[36898]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-08T23:23:42.838 INFO:tasks.workunit.client.0.smithi046.stdout: 10 137136 19.50 MB/sec execute 243 sec latency 183.470 ms 2024-05-08T23:23:43.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 137563 19.50 MB/sec execute 244 sec latency 156.734 ms 2024-05-08T23:23:44.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:44 smithi053 ceph-mon[77587]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 411 op/s 2024-05-08T23:23:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:44 smithi046 ceph-mon[36898]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 411 op/s 2024-05-08T23:23:44.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 137885 19.50 MB/sec execute 245 sec latency 214.651 ms 2024-05-08T23:23:44.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:44 smithi086 ceph-mon[78610]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 411 op/s 2024-05-08T23:23:45.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 138441 19.55 MB/sec execute 246 sec latency 129.232 ms 2024-05-08T23:23:46.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:46 smithi053 ceph-mon[77587]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 383 op/s 2024-05-08T23:23:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:46 smithi046 ceph-mon[36898]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 383 op/s 2024-05-08T23:23:46.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 138868 19.58 MB/sec execute 247 sec latency 200.300 ms 2024-05-08T23:23:46.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:46 smithi086 ceph-mon[78610]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 383 op/s 2024-05-08T23:23:47.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 139236 19.57 MB/sec execute 248 sec latency 234.614 ms 2024-05-08T23:23:48.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:48 smithi053 ceph-mon[77587]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 481 op/s 2024-05-08T23:23:48.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:48 smithi046 ceph-mon[36898]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 481 op/s 2024-05-08T23:23:48.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 139656 19.61 MB/sec execute 249 sec latency 220.888 ms 2024-05-08T23:23:48.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:48 smithi086 ceph-mon[78610]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 481 op/s 2024-05-08T23:23:49.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 140015 19.60 MB/sec execute 250 sec latency 233.917 ms 2024-05-08T23:23:50.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:50 smithi053 ceph-mon[77587]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 321 op/s 2024-05-08T23:23:50.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:50 smithi046 ceph-mon[36898]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 321 op/s 2024-05-08T23:23:50.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 140354 19.58 MB/sec execute 251 sec latency 224.502 ms 2024-05-08T23:23:50.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:50 smithi086 ceph-mon[78610]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 321 op/s 2024-05-08T23:23:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:51 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:51 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:51 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:51.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:51 smithi046 ceph-mon[36898]: scrub summary: idle 2024-05-08T23:23:51.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 140730 19.58 MB/sec execute 252 sec latency 203.677 ms 2024-05-08T23:23:51.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:51 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:51.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:51 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:51.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:51 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:51.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:51 smithi086 ceph-mon[78610]: scrub summary: idle 2024-05-08T23:23:51.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:51 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:51.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:51 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:51.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:51 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:51.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:51 smithi053 ceph-mon[77587]: scrub summary: idle 2024-05-08T23:23:52.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:52 smithi046 ceph-mon[36898]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-05-08T23:23:52.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141140 19.60 MB/sec execute 253 sec latency 190.087 ms 2024-05-08T23:23:52.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:52 smithi086 ceph-mon[78610]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-05-08T23:23:52.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:52 smithi053 ceph-mon[77587]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-05-08T23:23:53.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141564 19.60 MB/sec execute 254 sec latency 180.609 ms 2024-05-08T23:23:54.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:54 smithi046 ceph-mon[36898]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-08T23:23:54.286 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141957 19.59 MB/sec execute 255 sec latency 182.889 ms 2024-05-08T23:23:54.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:54 smithi086 ceph-mon[78610]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-08T23:23:54.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:54 smithi053 ceph-mon[77587]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-08T23:23:55.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 142360 19.60 MB/sec execute 256 sec latency 214.996 ms 2024-05-08T23:23:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:56 smithi046 ceph-mon[36898]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-05-08T23:23:56.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 142841 19.64 MB/sec execute 257 sec latency 166.779 ms 2024-05-08T23:23:56.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:56 smithi086 ceph-mon[78610]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-05-08T23:23:56.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:56 smithi053 ceph-mon[77587]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-05-08T23:23:57.287 INFO:tasks.workunit.client.0.smithi046.stdout: 10 143307 19.68 MB/sec execute 258 sec latency 166.485 ms 2024-05-08T23:23:58.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 143692 19.69 MB/sec execute 259 sec latency 196.510 ms 2024-05-08T23:23:58.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:23:58 smithi086 ceph-mon[78610]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.8 MiB/s wr, 369 op/s 2024-05-08T23:23:58.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:23:58 smithi053 ceph-mon[77587]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.8 MiB/s wr, 369 op/s 2024-05-08T23:23:58.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:23:58 smithi046 ceph-mon[36898]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.8 MiB/s wr, 369 op/s 2024-05-08T23:23:59.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 144002 19.67 MB/sec execute 260 sec latency 220.154 ms 2024-05-08T23:24:00.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 144272 19.63 MB/sec execute 261 sec latency 408.310 ms 2024-05-08T23:24:00.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:00 smithi086 ceph-mon[78610]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-05-08T23:24:00.511 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:00 smithi053 ceph-mon[77587]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-05-08T23:24:00.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:00 smithi046 ceph-mon[36898]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.7 MiB/s wr, 249 op/s 2024-05-08T23:24:01.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 144381 19.56 MB/sec execute 262 sec latency 638.345 ms 2024-05-08T23:24:02.288 INFO:tasks.workunit.client.0.smithi046.stdout: 10 144394 19.49 MB/sec execute 263 sec latency 1638.500 ms 2024-05-08T23:24:02.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:02 smithi086 ceph-mon[78610]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 329 op/s 2024-05-08T23:24:02.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:02 smithi053 ceph-mon[77587]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 329 op/s 2024-05-08T23:24:02.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:02 smithi046 ceph-mon[36898]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 329 op/s 2024-05-08T23:24:02.758 INFO:tasks.workunit.client.0.smithi046.stdout:[144015] write failed on handle 11151 (Resource temporarily unavailable) 2024-05-08T23:24:02.758 INFO:tasks.workunit.client.0.smithi046.stdout:Child failed with status 1 2024-05-08T23:24:02.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:24:02.848 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-08T23:24:02.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T23:24:03.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:03 smithi046 ceph-mon[36898]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 289 op/s 2024-05-08T23:24:03.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:03 smithi086 ceph-mon[78610]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 289 op/s 2024-05-08T23:24:03.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:03 smithi053 ceph-mon[77587]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 289 op/s 2024-05-08T23:24:04.538 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-08T23:24:06.046 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f9e5fb679ae44846a59dbb32b7353210 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-08T23:24:06.049 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-08T23:24:06.060 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-08T23:24:06.061 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:24:06.061 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:24:06.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-08T23:24:06.067 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:05 smithi086 ceph-mon[78610]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 303 op/s 2024-05-08T23:24:06.067 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:05 smithi053 ceph-mon[77587]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 303 op/s 2024-05-08T23:24:06.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:05 smithi046 ceph-mon[36898]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 303 op/s 2024-05-08T23:24:06.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-08T23:24:06.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T23:24:06.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:24:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 -- 172.21.15.46:0/3146461935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe274101090 msgr2=0x7fe274101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 --2- 172.21.15.46:0/3146461935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe274101090 0x7fe274101470 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe264009990 tx=0x7fe264009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 -- 172.21.15.46:0/3146461935 shutdown_connections 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 --2- 172.21.15.46:0/3146461935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2741061b0 0x7fe27410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 --2- 172.21.15.46:0/3146461935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe274101090 0x7fe274101470 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 --2- 172.21.15.46:0/3146461935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101a40 0x7fe274105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 -- 172.21.15.46:0/3146461935 >> 172.21.15.46:0/3146461935 conn(0x7fe2740fc970 msgr2=0x7fe2740fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 -- 172.21.15.46:0/3146461935 shutdown_connections 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.437+0000 7fe279800700 1 -- 172.21.15.46:0/3146461935 wait complete. 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.438+0000 7fe279800700 1 Processor -- start 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.438+0000 7fe279800700 1 -- start start 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2741061b0 0x7fe274199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 0x7fe27419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27419e990 con 0x7fe2741061b0 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27419eb00 con 0x7fe274101090 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe279800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27419ecc0 con 0x7fe27419df10 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe272000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2741061b0 0x7fe274199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe272a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe272a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57964/0 (socket says 172.21.15.46:57964) 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe272a00700 1 -- 172.21.15.46:0/3439043482 learned_addr learned my addr 172.21.15.46:0/3439043482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.439+0000 7fe273400700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 0x7fe27419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 -- 172.21.15.46:0/3439043482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 msgr2=0x7fe27419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 0x7fe27419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 -- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2741061b0 msgr2=0x7fe274199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2741061b0 0x7fe274199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 -- 172.21.15.46:0/3439043482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640095f0 con 0x7fe274101090 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe273400700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 0x7fe27419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe272a00700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe264000c00 tx=0x7fe264005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26401d070 con 0x7fe274101090 2024-05-08T23:24:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2741a61c0 con 0x7fe274101090 2024-05-08T23:24:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe264022470 con 0x7fe274101090 2024-05-08T23:24:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264017650 con 0x7fe274101090 2024-05-08T23:24:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.440+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2741a6640 con 0x7fe274101090 2024-05-08T23:24:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.441+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe274102560 con 0x7fe274101090 2024-05-08T23:24:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.444+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe2640225e0 con 0x7fe274101090 2024-05-08T23:24:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.445+0000 7fe26b400700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 0x7fe2580a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.445+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fe2640cea50 con 0x7fe274101090 2024-05-08T23:24:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.445+0000 7fe272000700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 0x7fe2580a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.446+0000 7fe272000700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 0x7fe2580a3740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe27419ab80 tx=0x7fe260009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.448+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe264097a70 con 0x7fe274101090 2024-05-08T23:24:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 -- 172.21.15.46:0/1929727715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a8101080 msgr2=0x7f19a8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 --2- 172.21.15.46:0/1929727715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a8101080 0x7f19a8101460 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1998009990 tx=0x7f1998009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 -- 172.21.15.46:0/1929727715 shutdown_connections 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 --2- 172.21.15.46:0/1929727715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 0x7f19a810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 --2- 172.21.15.46:0/1929727715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a8101080 0x7f19a8101460 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 --2- 172.21.15.46:0/1929727715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101a30 0x7f19a8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 -- 172.21.15.46:0/1929727715 >> 172.21.15.46:0/1929727715 conn(0x7f19a80fc8e0 msgr2=0x7f19a80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 -- 172.21.15.46:0/1929727715 shutdown_connections 2024-05-08T23:24:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.642+0000 7f19aca00700 1 -- 172.21.15.46:0/1929727715 wait complete. 2024-05-08T23:24:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.643+0000 7f19aca00700 1 Processor -- start 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.643+0000 7f19aca00700 1 -- start start 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 0x7f19a8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 0x7f19a8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a819e940 con 0x7f19a819dec0 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a819eab0 con 0x7f19a8101080 2024-05-08T23:24:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19aca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a819ec70 con 0x7f19a81061a0 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48636/0 (socket says 172.21.15.46:48636) 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 0x7f19a8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 -- 172.21.15.46:0/3381093367 learned_addr learned my addr 172.21.15.46:0/3381093367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a5400700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 0x7f19a8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 -- 172.21.15.46:0/3381093367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 msgr2=0x7f19a8199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 0x7f19a8199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 -- 172.21.15.46:0/3381093367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 msgr2=0x7f19a8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 0x7f19a8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.644+0000 7f19a6800700 1 -- 172.21.15.46:0/3381093367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980095f0 con 0x7f19a819dec0 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f19a5e00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 0x7f19a8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f19a6800700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ef30 tx=0x7f19a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f199ea00700 1 -- 172.21.15.46:0/3381093367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0018070 con 0x7f19a819dec0 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a81a6170 con 0x7f19a819dec0 2024-05-08T23:24:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f199ea00700 1 -- 172.21.15.46:0/3381093367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a000f040 con 0x7f19a819dec0 2024-05-08T23:24:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f199ea00700 1 -- 172.21.15.46:0/3381093367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000cca0 con 0x7f19a819dec0 2024-05-08T23:24:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.645+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81a6650 con 0x7f19a819dec0 2024-05-08T23:24:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.648+0000 7f199ea00700 1 -- 172.21.15.46:0/3381093367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f19a0003a00 con 0x7f19a819dec0 2024-05-08T23:24:06.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.650+0000 7f19c05feb80 1 Processor -- start 2024-05-08T23:24:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 -- start start 2024-05-08T23:24:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f199ea00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 0x7f198c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f199ea00700 1 -- 172.21.15.46:0/3381093367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f19a00ccc40 con 0x7f19a819dec0 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8d69150 con 0x561ed8e12890 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8d66b10 con 0x561ed8e18720 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19c05feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8d62400 con 0x561ed8e17e00 2024-05-08T23:24:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.651+0000 7f19a5400700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 0x7f198c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35484/0 (socket says 172.21.15.46:35484) 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5e00700 1 -- 172.21.15.46:0/3913761882 learned_addr learned my addr 172.21.15.46:0/3913761882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5400700 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 -- 172.21.15.46:0/3913761882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 msgr2=0x561ed8e181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 -- 172.21.15.46:0/3913761882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 msgr2=0x561ed8e12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 -- 172.21.15.46:0/3913761882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed8be33b0 con 0x561ed8e12890 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5400700 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a5400700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 0x7f198c0a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19a819ab30 tx=0x7f19940118e0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.652+0000 7f19a6800700 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19a00c9ed0 tx=0x7f19a00994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95671f8774f3d876 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f1993000700 1 -- 172.21.15.46:0/3913761882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a001b320 con 0x561ed8e12890 2024-05-08T23:24:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f1993000700 1 -- 172.21.15.46:0/3913761882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a009a020 con 0x561ed8e12890 2024-05-08T23:24:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f1993000700 1 -- 172.21.15.46:0/3913761882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0097d50 con 0x561ed8e12890 2024-05-08T23:24:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 -- 172.21.15.46:0/3913761882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 msgr2=0x561ed8e12cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19a00c9ed0 tx=0x7f19a00994e0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 -- 172.21.15.46:0/3913761882 shutdown_connections 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12cd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3913761882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.653+0000 7f19c05feb80 1 -- 172.21.15.46:0/3913761882 >> 172.21.15.46:0/3913761882 conn(0x561ed8d70210 msgr2=0x561ed8e0cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.654+0000 7f19c05feb80 1 -- 172.21.15.46:0/3913761882 shutdown_connections 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.654+0000 7f19c05feb80 1 -- 172.21.15.46:0/3913761882 wait complete. 2024-05-08T23:24:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.654+0000 7f19c05feb80 1 -- start start 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.654+0000 7f19c05feb80 1 Processor -- start 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e1bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e17e00 0x561ed8e1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 0x561ed8e1c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8cfe560 con 0x561ed8e12890 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8cfe200 con 0x561ed8e17e00 2024-05-08T23:24:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8be56a0 con 0x561ed8e18720 2024-05-08T23:24:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19c05feb80 10 client.0 _ll_register_callbacks cb 0x561ed8d07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e1bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19a6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 0x561ed8e1c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e1bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48656/0 (socket says 172.21.15.46:48656) 2024-05-08T23:24:06.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19a5e00700 1 -- 172.21.15.46:0/3289066784 learned_addr learned my addr 172.21.15.46:0/3289066784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.655+0000 7f19a5400700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e17e00 0x561ed8e1c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 -- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e17e00 msgr2=0x561ed8e1c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e17e00 0x561ed8e1c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 -- 172.21.15.46:0/3289066784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 msgr2=0x561ed8e1bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e1bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 -- 172.21.15.46:0/3289066784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81a6650 con 0x561ed8e18720 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19a6800700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 0x561ed8e1c5a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f19a00c9f00 tx=0x7f19a001b320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6591ac64e9cc4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f1987400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0099d80 con 0x561ed8e18720 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ed8cbb270 con 0x561ed8e18720 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f19a009d750 con 0x561ed8e18720 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a00b2260 con 0x561ed8e18720 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19c05feb80 10 client.30694 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.656+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561ed8b8fb60 con 0x561ed8e18720 2024-05-08T23:24:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a0065020 con 0x561ed8e18720 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19a00b90b0 con 0x561ed8e18720 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 10 client.30694 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 10 client.30694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 20 client.30694 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e2bcf0 0x561ed8e2e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 4 client.30694 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ed8de1810 con 0x561ed8e2bcf0 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.657+0000 7f19a5e00700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e2bcf0 0x561ed8e2e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.658+0000 7f19a5e00700 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e2bcf0 0x561ed8e2e110 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a326daba0b376b7 server_cookie=2850babcf6e664f in_seq=0 out_seq=0 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.658+0000 7f1987400700 10 client.30694 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f1987400700 1 -- 172.21.15.46:0/3289066784 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ed8de1810 con 0x561ed8e2bcf0 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f1987400700 10 client.30694 handle_command_reply: tid=0 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 client.30694 shutdown 2024-05-08T23:24:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 msgr2=0x561ed8e1c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 0x561ed8e1c5a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f19a00c9f00 tx=0x7f19a001b320 comp rx=0 tx=0).stop 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 shutdown_connections 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e18720 0x561ed8e1c5a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e2bcf0 0x561ed8e2e110 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e1bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 --2- 172.21.15.46:0/3289066784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e17e00 0x561ed8e1c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f1987400700 0 client.30694 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 >> 172.21.15.46:0/3289066784 conn(0x561ed8d70210 msgr2=0x561ed8e0c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 shutdown_connections 2024-05-08T23:24:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 1 -- 172.21.15.46:0/3289066784 wait complete. 2024-05-08T23:24:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.659+0000 7f19c05feb80 20 client.30694 trim_cache size 0 max 16384 2024-05-08T23:24:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.677+0000 7f19c05feb80 1 Processor -- start 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.677+0000 7f19c05feb80 1 -- start start 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e1d190 con 0x561ed8e12890 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e19bc0 con 0x561ed8e18720 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19c05feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e0e640 con 0x561ed8e17e00 2024-05-08T23:24:06.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48670/0 (socket says 172.21.15.46:48670) 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 -- 172.21.15.46:0/2636641394 learned_addr learned my addr 172.21.15.46:0/2636641394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5400700 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a6800700 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 -- 172.21.15.46:0/2636641394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 msgr2=0x561ed8e17320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e17320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 -- 172.21.15.46:0/2636641394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 msgr2=0x561ed8e0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.678+0000 7f19a5e00700 1 -- 172.21.15.46:0/2636641394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed8d28560 con 0x561ed8e12890 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.679+0000 7f19a5e00700 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1998006010 tx=0x7f199800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2526eb18a78e27fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.679+0000 7f1993a00700 1 -- 172.21.15.46:0/2636641394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199801d040 con 0x561ed8e12890 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.679+0000 7f1993a00700 1 -- 172.21.15.46:0/2636641394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f199801e040 con 0x561ed8e12890 2024-05-08T23:24:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.679+0000 7f1993a00700 1 -- 172.21.15.46:0/2636641394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19980297c0 con 0x561ed8e12890 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- 172.21.15.46:0/2636641394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 msgr2=0x561ed8e12350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1998006010 tx=0x7f199800b260 comp rx=0 tx=0).stop 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- 172.21.15.46:0/2636641394 shutdown_connections 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e18720 0x561ed8e0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e12890 0x561ed8e12350 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 --2- 172.21.15.46:0/2636641394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- 172.21.15.46:0/2636641394 >> 172.21.15.46:0/2636641394 conn(0x561ed8d70210 msgr2=0x561ed8e16c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- 172.21.15.46:0/2636641394 shutdown_connections 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- 172.21.15.46:0/2636641394 wait complete. 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 -- start start 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.680+0000 7f19c05feb80 1 Processor -- start 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e12890 0x561ed8e23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e14e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e15770 con 0x561ed8e18720 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e13ee0 con 0x561ed8e12890 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed8e140a0 con 0x561ed8e17e00 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19c05feb80 10 client.0 _ll_register_callbacks cb 0x561ed8d07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19a6800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48680/0 (socket says 172.21.15.46:48680) 2024-05-08T23:24:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19a6800700 1 -- 172.21.15.46:0/1582468450 learned_addr learned my addr 172.21.15.46:0/1582468450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19a5400700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.681+0000 7f19a5e00700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e12890 0x561ed8e23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 -- 172.21.15.46:0/1582468450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 msgr2=0x561ed8e14e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e14e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 -- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e12890 msgr2=0x561ed8e23650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e12890 0x561ed8e23650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 -- 172.21.15.46:0/1582468450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a810e590 con 0x561ed8e18720 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19a6800700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19a00995d0 tx=0x7f19a00994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb272f666a6859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f1987400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0099280 con 0x561ed8e18720 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f19a009d750 con 0x561ed8e18720 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a009a020 con 0x561ed8e18720 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a00b2260 con 0x561ed8e18720 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19c05feb80 10 client.20448 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.682+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561ed8be7cd0 con 0x561ed8e18720 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a009a020 con 0x561ed8e18720 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19a0099b80 con 0x561ed8e18720 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 10 client.20448 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 10 client.20448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 20 client.20448 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e82a40 0x561ed8e84e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 4 client.20448 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ed8d5aec0 con 0x561ed8e82a40 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.683+0000 7f19a5e00700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e82a40 0x561ed8e84e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f1987400700 10 client.20448 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19a5e00700 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e82a40 0x561ed8e84e30 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f56e3ab459f5658 server_cookie=2079db29be603b1b in_seq=0 out_seq=0 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f1987400700 1 -- 172.21.15.46:0/1582468450 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561ed8d5aec0 con 0x561ed8e82a40 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f1987400700 10 client.20448 handle_command_reply: tid=0 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 client.20448 shutdown 2024-05-08T23:24:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 msgr2=0x561ed8e139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19a00995d0 tx=0x7f19a00994e0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 shutdown_connections 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x561ed8e82a40 0x561ed8e84e30 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561ed8e18720 0x561ed8e139a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ed8e12890 0x561ed8e23650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 --2- 172.21.15.46:0/1582468450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ed8e17e00 0x561ed8e14e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 >> 172.21.15.46:0/1582468450 conn(0x561ed8d70210 msgr2=0x561ed8e174a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.684+0000 7f1987400700 0 client.20448 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 shutdown_connections 2024-05-08T23:24:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19c05feb80 1 -- 172.21.15.46:0/1582468450 wait complete. 2024-05-08T23:24:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19c05feb80 20 client.20448 trim_cache size 0 max 16384 2024-05-08T23:24:06.694 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:06.695 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-08T23:24:06.695 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-08T23:24:06.695 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 msgr2=0x7f198c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 0x7f198c0a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19a819ab30 tx=0x7f19940118e0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 msgr2=0x7f19a819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ef30 tx=0x7f19a000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 shutdown_connections 2024-05-08T23:24:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f198c0a1290 0x7f198c0a3750 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19a819dec0 0x7f19a819e340 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a8101080 0x7f19a8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 --2- 172.21.15.46:0/3381093367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a81061a0 0x7f19a8199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.685+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 >> 172.21.15.46:0/3381093367 conn(0x7f19a80fc8e0 msgr2=0x7f19a80fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.686+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 shutdown_connections 2024-05-08T23:24:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.686+0000 7f19aca00700 1 -- 172.21.15.46:0/3381093367 wait complete. 2024-05-08T23:24:06.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:06.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.715+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe27419a930 con 0x7fe2580a1280 2024-05-08T23:24:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.729+0000 7fe26b400700 1 -- 172.21.15.46:0/3439043482 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fe27419a930 con 0x7fe2580a1280 2024-05-08T23:24:06.728 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:04:01", "first": "2024-05-08T23:05:00", "last": "2024-05-08T23:24:00", "last_pruned": "2024-05-08T23:24:00", "created_count": 20, "pruned_count": 14, "active": true} 2024-05-08T23:24:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 msgr2=0x7fe2580a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 0x7fe2580a3740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe27419ab80 tx=0x7fe260009260 comp rx=0 tx=0).stop 2024-05-08T23:24:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 msgr2=0x7fe274199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe264000c00 tx=0x7fe264005740 comp rx=0 tx=0).stop 2024-05-08T23:24:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 shutdown_connections 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27419df10 0x7fe27419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe274101090 0x7fe274199630 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fe2580a1280 0x7fe2580a3740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 --2- 172.21.15.46:0/3439043482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2741061b0 0x7fe274199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 >> 172.21.15.46:0/3439043482 conn(0x7fe2740fc970 msgr2=0x7fe2740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 shutdown_connections 2024-05-08T23:24:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:06.731+0000 7fe279800700 1 -- 172.21.15.46:0/3439043482 wait complete. 2024-05-08T23:24:06.747 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:06 smithi046 ceph-mon[36898]: osdmap e114: 12 total, 12 up, 12 in 2024-05-08T23:24:06.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T23:24:06.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:06 smithi053 ceph-mon[77587]: osdmap e114: 12 total, 12 up, 12 in 2024-05-08T23:24:06.969 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:06 smithi086 ceph-mon[78610]: osdmap e114: 12 total, 12 up, 12 in 2024-05-08T23:24:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.025+0000 7fa656c00700 1 -- 172.21.15.46:0/2871461262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650101850 msgr2=0x7fa650105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.025+0000 7fa656c00700 1 --2- 172.21.15.46:0/2871461262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650101850 0x7fa650105890 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009990 tx=0x7fa63c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.026+0000 7fa656c00700 1 -- 172.21.15.46:0/2871461262 shutdown_connections 2024-05-08T23:24:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.026+0000 7fa656c00700 1 --2- 172.21.15.46:0/2871461262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650105fc0 0x7fa65010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.026+0000 7fa656c00700 1 --2- 172.21.15.46:0/2871461262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650100ea0 0x7fa650101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.026+0000 7fa656c00700 1 --2- 172.21.15.46:0/2871461262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650101850 0x7fa650105890 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.026+0000 7fa656c00700 1 -- 172.21.15.46:0/2871461262 >> 172.21.15.46:0/2871461262 conn(0x7fa6500fc700 msgr2=0x7fa6500feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.028+0000 7fa656c00700 1 -- 172.21.15.46:0/2871461262 shutdown_connections 2024-05-08T23:24:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.029+0000 7fa656c00700 1 -- 172.21.15.46:0/2871461262 wait complete. 2024-05-08T23:24:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.030+0000 7fa656c00700 1 Processor -- start 2024-05-08T23:24:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.033+0000 7fa656c00700 1 -- start start 2024-05-08T23:24:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.033+0000 7fa656c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 0x7fa6500732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 0x7fa6500737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650073d20 0x7fa6501a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa655800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 0x7fa6500732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa655800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 0x7fa6500732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35526/0 (socket says 172.21.15.46:35526) 2024-05-08T23:24:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa655800700 1 -- 172.21.15.46:0/2049518819 learned_addr learned my addr 172.21.15.46:0/2049518819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501a58a0 con 0x7fa650105fc0 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501a59e0 con 0x7fa650073d20 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501a5ba0 con 0x7fa650100ea0 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa656200700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650073d20 0x7fa6501a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.034+0000 7fa654e00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 0x7fa6500737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.036+0000 7fa654e00700 1 -- 172.21.15.46:0/2049518819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 msgr2=0x7fa6500732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.036+0000 7fa654e00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 0x7fa6500732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.036+0000 7fa654e00700 1 -- 172.21.15.46:0/2049518819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650073d20 msgr2=0x7fa6501a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.036+0000 7fa654e00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650073d20 0x7fa6501a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.036+0000 7fa654e00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0095f0 con 0x7fa650105fc0 2024-05-08T23:24:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.040+0000 7fa654e00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 0x7fa6500737e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa65006a310 tx=0x7fa63c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.040+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c01d070 con 0x7fa650105fc0 2024-05-08T23:24:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.040+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6501a5e20 con 0x7fa650105fc0 2024-05-08T23:24:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.041+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63c00fb80 con 0x7fa650105fc0 2024-05-08T23:24:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.041+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501a61e0 con 0x7fa650105fc0 2024-05-08T23:24:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.041+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c022cb0 con 0x7fa650105fc0 2024-05-08T23:24:07.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.046+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650109290 con 0x7fa650105fc0 2024-05-08T23:24:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.053+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa63c00fcf0 con 0x7fa650105fc0 2024-05-08T23:24:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.056+0000 7fa64e000700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 0x7fa6380a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.056+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fa63c0ce540 con 0x7fa650105fc0 2024-05-08T23:24:07.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.058+0000 7fa655800700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 0x7fa6380a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.060+0000 7fa655800700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 0x7fa6380a3850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa644009ad0 tx=0x7fa644009290 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.073+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa63c097560 con 0x7fa650105fc0 2024-05-08T23:24:07.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 -- 172.21.15.46:0/1890530295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 msgr2=0x7f4b5c10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 --2- 172.21.15.46:0/1890530295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c10f9a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0098f0 tx=0x7f4b4c009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 -- 172.21.15.46:0/1890530295 shutdown_connections 2024-05-08T23:24:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 --2- 172.21.15.46:0/1890530295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c11a780 0x7f4b5c11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 --2- 172.21.15.46:0/1890530295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c10f9a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 --2- 172.21.15.46:0/1890530295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108030 0x7f4b5c1084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.206+0000 7f4b61600700 1 -- 172.21.15.46:0/1890530295 >> 172.21.15.46:0/1890530295 conn(0x7f4b5c06bf80 msgr2=0x7f4b5c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.208+0000 7f4b61600700 1 -- 172.21.15.46:0/1890530295 shutdown_connections 2024-05-08T23:24:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.210+0000 7f4b61600700 1 -- 172.21.15.46:0/1890530295 wait complete. 2024-05-08T23:24:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.214+0000 7f4b61600700 1 Processor -- start 2024-05-08T23:24:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.215+0000 7f4b61600700 1 -- start start 2024-05-08T23:24:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b61600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 0x7f4b5c118b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b61600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c1190d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b61600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c11a780 0x7f4b5c111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b61600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c112510 con 0x7f4b5c11a780 2024-05-08T23:24:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b61600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c112680 con 0x7f4b5c10f5c0 2024-05-08T23:24:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b52000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c1190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b52000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c1190d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58054/0 (socket says 172.21.15.46:58054) 2024-05-08T23:24:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.219+0000 7f4b52000700 1 -- 172.21.15.46:0/112575367 learned_addr learned my addr 172.21.15.46:0/112575367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.220+0000 7f4b5aa00700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 0x7f4b5c118b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.220+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c112840 con 0x7f4b5c108030 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5b400700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c11a780 0x7f4b5c111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 -- 172.21.15.46:0/112575367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 msgr2=0x7f4b5c1190d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c1190d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 -- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c11a780 msgr2=0x7f4b5c111cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c11a780 0x7f4b5c111cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009520 con 0x7f4b5c108030 2024-05-08T23:24:07.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.221+0000 7f4b5aa00700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 0x7f4b5c118b50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c000c00 tx=0x7f4b4c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.223+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c01c070 con 0x7f4b5c108030 2024-05-08T23:24:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.223+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4c00baa0 con 0x7f4b5c108030 2024-05-08T23:24:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.223+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c01c070 con 0x7f4b5c108030 2024-05-08T23:24:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.223+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c112a60 con 0x7f4b5c108030 2024-05-08T23:24:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.224+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c116920 con 0x7f4b5c108030 2024-05-08T23:24:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.230+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c06d320 con 0x7f4b5c108030 2024-05-08T23:24:07.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.237+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4b4c020030 con 0x7f4b5c108030 2024-05-08T23:24:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.239+0000 7f4b53e00700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 0x7f4b400a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.239+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f4b4c0cdd00 con 0x7f4b5c108030 2024-05-08T23:24:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.241+0000 7f4b52000700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 0x7f4b400a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.243+0000 7f4b52000700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 0x7f4b400a3870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c113350 tx=0x7f4b4400a400 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.245+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b4c096d20 con 0x7f4b5c108030 2024-05-08T23:24:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.678+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa650074730 con 0x7fa650105fc0 2024-05-08T23:24:07.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.680+0000 7fa64e000700 1 -- 172.21.15.46:0/2049518819 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fa63c09ab80 con 0x7fa650105fc0 2024-05-08T23:24:07.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:07.681 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 msgr2=0x7fa6380a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 0x7fa6380a3850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa644009ad0 tx=0x7fa644009290 comp rx=0 tx=0).stop 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 msgr2=0x7fa6500737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 0x7fa6500737e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa65006a310 tx=0x7fa63c00f690 comp rx=0 tx=0).stop 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 shutdown_connections 2024-05-08T23:24:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa650073d20 0x7fa6501a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa650100ea0 0x7fa6500732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa6380a1390 0x7fa6380a3850 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 --2- 172.21.15.46:0/2049518819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa650105fc0 0x7fa6500737e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 >> 172.21.15.46:0/2049518819 conn(0x7fa6500fc700 msgr2=0x7fa6500fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 shutdown_connections 2024-05-08T23:24:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.686+0000 7fa656c00700 1 -- 172.21.15.46:0/2049518819 wait complete. 2024-05-08T23:24:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:24:07.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T23:24:07.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.756+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 --> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5c113170 con 0x7f4b400a13b0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.774+0000 7f4b53e00700 1 -- 172.21.15.46:0/112575367 <== mgr.15501 v2:172.21.15.46:6800/652597198 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4b5c113170 con 0x7f4b400a13b0 2024-05-08T23:24:07.773 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-05-08T23:24:07.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:07 smithi046 ceph-mon[36898]: pgmap v639: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:07.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:07 smithi046 ceph-mon[36898]: from='client.30683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 msgr2=0x7f4b400a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 0x7f4b400a3870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c113350 tx=0x7f4b4400a400 comp rx=0 tx=0).stop 2024-05-08T23:24:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 msgr2=0x7f4b5c118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 0x7f4b5c118b50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c000c00 tx=0x7f4b4c00fa00 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 shutdown_connections 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f4b400a13b0 0x7f4b400a3870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c11a780 0x7f4b5c111cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c108030 0x7f4b5c118b50 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 --2- 172.21.15.46:0/112575367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c10f5c0 0x7f4b5c1190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 >> 172.21.15.46:0/112575367 conn(0x7f4b5c06bf80 msgr2=0x7f4b5c06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.778+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 shutdown_connections 2024-05-08T23:24:07.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.779+0000 7f4b61600700 1 -- 172.21.15.46:0/112575367 wait complete. 2024-05-08T23:24:07.808 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T23:24:07.818 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T23:24:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 -- 172.21.15.46:0/77857332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 msgr2=0x7f0c28073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/77857332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c28073750 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009990 tx=0x7f0c18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 -- 172.21.15.46:0/77857332 shutdown_connections 2024-05-08T23:24:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/77857332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c2810d6d0 0x7f0c2810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/77857332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c28073750 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/77857332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c28073c90 0x7f0c2810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 -- 172.21.15.46:0/77857332 >> 172.21.15.46:0/77857332 conn(0x7f0c280fc960 msgr2=0x7f0c280fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 -- 172.21.15.46:0/77857332 shutdown_connections 2024-05-08T23:24:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.961+0000 7f0c2ec00700 1 -- 172.21.15.46:0/77857332 wait complete. 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.962+0000 7f0c2ec00700 1 Processor -- start 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.962+0000 7f0c2ec00700 1 -- start start 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c28073c90 0x7f0c28199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2810d6d0 0x7f0c2819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2819e480 con 0x7f0c28073c90 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2819e5f0 con 0x7f0c2810d6d0 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2819e7b0 con 0x7f0c28073370 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c1fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c28073c90 0x7f0c28199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35568/0 (socket says 172.21.15.46:35568) 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 -- 172.21.15.46:0/4074793826 learned_addr learned my addr 172.21.15.46:0/4074793826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2810d6d0 0x7f0c2819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 -- 172.21.15.46:0/4074793826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2810d6d0 msgr2=0x7f0c2819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2810d6d0 0x7f0c2819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 -- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c28073c90 msgr2=0x7f0c28199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c28073c90 0x7f0c28199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.963+0000 7f0c27e00700 1 -- 172.21.15.46:0/4074793826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180095f0 con 0x7f0c28073370 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.964+0000 7f0c27e00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c18005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.964+0000 7f0c25600700 1 -- 172.21.15.46:0/4074793826 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801d070 con 0x7f0c28073370 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.964+0000 7f0c25600700 1 -- 172.21.15.46:0/4074793826 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c18022470 con 0x7f0c28073370 2024-05-08T23:24:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.964+0000 7f0c25600700 1 -- 172.21.15.46:0/4074793826 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18017650 con 0x7f0c28073370 2024-05-08T23:24:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.965+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2819ea30 con 0x7f0c28073370 2024-05-08T23:24:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.965+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2819ecb0 con 0x7f0c28073370 2024-05-08T23:24:07.967 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:07 smithi086 ceph-mon[78610]: pgmap v639: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:07.967 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:07 smithi086 ceph-mon[78610]: from='client.30683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.969+0000 7f0c25600700 1 -- 172.21.15.46:0/4074793826 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0c180225e0 con 0x7f0c28073370 2024-05-08T23:24:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.970+0000 7f0c426bdb80 1 Processor -- start 2024-05-08T23:24:07.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 -- start start 2024-05-08T23:24:07.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596baca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bacacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596ba21150 con 0x55596baca890 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596ba1eb10 con 0x55596bacfe00 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596ba1a400 con 0x55596bad0720 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c2cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.971+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596baca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c2cc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58086/0 (socket says 172.21.15.46:58086) 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c2cc00700 1 -- 172.21.15.46:0/1630805748 learned_addr learned my addr 172.21.15.46:0/1630805748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bacacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c25600700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 0x7f0c0c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 -- 172.21.15.46:0/1630805748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 msgr2=0x55596baca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596baca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 -- 172.21.15.46:0/1630805748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 msgr2=0x55596bad01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c1fe00700 1 -- 172.21.15.46:0/1630805748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596b89b3b0 con 0x55596baca890 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c25600700 1 -- 172.21.15.46:0/4074793826 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f0c180ce1c0 con 0x7f0c28073370 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.972+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 0x7f0c0c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.973+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bacacd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009c60 tx=0x7f0c10009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe3c9d6804ecd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.973+0000 7f0c1d200700 1 -- 172.21.15.46:0/1630805748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011d10 con 0x55596baca890 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.973+0000 7f0c1d200700 1 -- 172.21.15.46:0/1630805748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c10011e70 con 0x55596baca890 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.973+0000 7f0c1d200700 1 -- 172.21.15.46:0/1630805748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10005280 con 0x55596baca890 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1630805748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 msgr2=0x55596bacacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 0x7f0c0c0a37d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0c20007620 tx=0x7f0c20009040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bacacd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009c60 tx=0x7f0c10009f70 comp rx=0 tx=0).stop 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1630805748 shutdown_connections 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bacacd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1630805748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596baca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1630805748 >> 172.21.15.46:0/1630805748 conn(0x55596ba28210 msgr2=0x55596bac4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1630805748 shutdown_connections 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1630805748 wait complete. 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.974+0000 7f0c426bdb80 1 -- start start 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 Processor -- start 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bad3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b9b6560 con 0x55596baca890 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b9b6200 con 0x55596bacfe00 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596b89d6a0 con 0x55596bad0720 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c426bdb80 10 client.0 _ll_register_callbacks cb 0x55596b9bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c2cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bad3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c2cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35592/0 (socket says 172.21.15.46:35592) 2024-05-08T23:24:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.975+0000 7f0c2cc00700 1 -- 172.21.15.46:0/1931547390 learned_addr learned my addr 172.21.15.46:0/1931547390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 -- 172.21.15.46:0/1931547390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 msgr2=0x55596bad4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 -- 172.21.15.46:0/1931547390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 msgr2=0x55596bad3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bad3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 -- 172.21.15.46:0/1931547390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20011300 con 0x55596bad0720 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c27e00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bad3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:07.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0c20011fa0 tx=0x7f0c20004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b796dd8bc4fd77fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.976+0000 7f0c15600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000e040 con 0x55596bad0720 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55596b973270 con 0x55596bad0720 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f0c20017e40 con 0x55596bad0720 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c20022440 con 0x55596bad0720 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 10 client.30724 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f0c100098c0 con 0x55596bad0720 2024-05-08T23:24:07.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55596b973270 con 0x55596bad0720 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c2002b6e0 con 0x55596bad0720 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 10 client.30724 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 10 client.30724 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 20 client.30724 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bae4300 0x55596bae6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 4 client.30724 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.977+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55596ba99810 con 0x55596bae4300 2024-05-08T23:24:07.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.978+0000 7f0c27e00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bae4300 0x55596bae6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.978+0000 7f0c27e00700 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bae4300 0x55596bae6720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c939cad03d806f05 server_cookie=3d903e721ba7670c in_seq=0 out_seq=0 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.978+0000 7f0c15600700 10 client.30724 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c15600700 1 -- 172.21.15.46:0/1931547390 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55596ba99810 con 0x55596bae4300 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c15600700 10 client.30724 handle_command_reply: tid=0 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 client.30724 shutdown 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 msgr2=0x55596bad4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0c20011fa0 tx=0x7f0c20004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 shutdown_connections 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bae4300 0x55596bae6720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bad4540 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596bad3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/1931547390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bad4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c15600700 0 client.30724 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 >> 172.21.15.46:0/1931547390 conn(0x55596ba28210 msgr2=0x55596bac45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 shutdown_connections 2024-05-08T23:24:07.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.979+0000 7f0c426bdb80 1 -- 172.21.15.46:0/1931547390 wait complete. 2024-05-08T23:24:07.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.980+0000 7f0c426bdb80 20 client.30724 trim_cache size 0 max 16384 2024-05-08T23:24:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.993+0000 7f0c426bdb80 1 Processor -- start 2024-05-08T23:24:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 -- start start 2024-05-08T23:24:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bac4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bacf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596bad0560 con 0x55596baca890 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596baeb930 con 0x55596bacfe00 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596bae18e0 con 0x55596bad0720 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48762/0 (socket says 172.21.15.46:48762) 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 -- 172.21.15.46:0/3054352189 learned_addr learned my addr 172.21.15.46:0/3054352189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:07.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bacf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bac4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 -- 172.21.15.46:0/3054352189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 msgr2=0x55596bacf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bacf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 -- 172.21.15.46:0/3054352189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 msgr2=0x55596bac4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bac4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.994+0000 7f0c27e00700 1 -- 172.21.15.46:0/3054352189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596b847b60 con 0x55596baca890 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809ad20 tx=0x7f0c180d0690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d16e96e2215721f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c1dc00700 1 -- 172.21.15.46:0/3054352189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180ce480 con 0x55596baca890 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c1dc00700 1 -- 172.21.15.46:0/3054352189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1809adf0 con 0x55596baca890 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c1dc00700 1 -- 172.21.15.46:0/3054352189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180d19c0 con 0x55596baca890 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3054352189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 msgr2=0x55596baca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.995+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809ad20 tx=0x7f0c180d0690 comp rx=0 tx=0).stop 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3054352189 shutdown_connections 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596bad0720 0x55596bacf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596baca890 0x55596baca350 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3054352189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bacfe00 0x55596bac4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3054352189 >> 172.21.15.46:0/3054352189 conn(0x55596ba28210 msgr2=0x55596baceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3054352189 shutdown_connections 2024-05-08T23:24:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3054352189 wait complete. 2024-05-08T23:24:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 -- start start 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.996+0000 7f0c426bdb80 1 Processor -- start 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 0x55596bad8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bad0720 0x55596bad9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596bad9920 con 0x55596bacfe00 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596bb39800 con 0x55596bad0720 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596bb39940 con 0x55596baca890 2024-05-08T23:24:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c426bdb80 10 client.0 _ll_register_callbacks cb 0x55596b9bfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c1fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 0x55596bad8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35610/0 (socket says 172.21.15.46:35610) 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 -- 172.21.15.46:0/3338889286 learned_addr learned my addr 172.21.15.46:0/3338889286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 0x55596bad8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48778/0 (socket says 172.21.15.46:48778) 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c2cc00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bad0720 0x55596bad9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 -- 172.21.15.46:0/3338889286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bad0720 msgr2=0x55596bad9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bad0720 0x55596bad9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 -- 172.21.15.46:0/3338889286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 msgr2=0x55596bad8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 0x55596bad8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.997+0000 7f0c27e00700 1 -- 172.21.15.46:0/3338889286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2819ea30 con 0x55596baca890 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c27e00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809afb0 tx=0x7f0c1809ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc2e1adc7d4abe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c15600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180a3410 con 0x55596baca890 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f0c1809ee50 con 0x55596baca890 2024-05-08T23:24:08.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c10011e70 con 0x55596baca890 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c1809b020 con 0x55596baca890 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c426bdb80 10 client.30730 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55596b9e0560 con 0x55596baca890 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.998+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c10011e70 con 0x55596baca890 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c1809a5a0 con 0x55596baca890 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 10 client.30730 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 10 client.30730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 20 client.30730 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bb3ad60 0x55596bb3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 4 client.30730 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 --> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55596ba12ec0 con 0x55596bb3ad60 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:07.999+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bb3ad60 0x55596bb3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c1fe00700 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bb3ad60 0x55596bb3d180 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6cb708cb966a6515 server_cookie=7c4f901625bcf1c3 in_seq=0 out_seq=0 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c15600700 10 client.30730 ms_handle_connect on v2:172.21.15.86:6834/2919086413 2024-05-08T23:24:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c15600700 1 -- 172.21.15.46:0/3338889286 <== mds.4 v2:172.21.15.86:6834/2919086413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55596ba12ec0 con 0x55596bb3ad60 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c15600700 10 client.30730 handle_command_reply: tid=0 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 client.30730 shutdown 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 msgr2=0x55596badb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809afb0 tx=0x7f0c1809ac70 comp rx=0 tx=0).stop 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 shutdown_connections 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:6834/2919086413,v1:172.21.15.86:6835/2919086413] conn(0x55596bb3ad60 0x55596bb3d180 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55596bad0720 0x55596bad9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55596baca890 0x55596badb660 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 --2- 172.21.15.46:0/3338889286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55596bacfe00 0x55596bad8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c15600700 0 client.30730 ms_handle_reset on v2:172.21.15.86:6834/2919086413 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 >> 172.21.15.46:0/3338889286 conn(0x55596ba28210 msgr2=0x55596bad1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 shutdown_connections 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.000+0000 7f0c426bdb80 1 -- 172.21.15.46:0/3338889286 wait complete. 2024-05-08T23:24:08.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c426bdb80 20 client.30730 trim_cache size 0 max 16384 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:24:08.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 msgr2=0x7f0c0c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 0x7f0c0c0a37d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0c20007620 tx=0x7f0c20009040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 msgr2=0x7f0c281995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c18005740 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 shutdown_connections 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c2810d6d0 0x7f0c2819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28073370 0x7f0c281995b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f0c0c0a1310 0x7f0c0c0a37d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 --2- 172.21.15.46:0/4074793826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c28073c90 0x7f0c28199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 >> 172.21.15.46:0/4074793826 conn(0x7f0c280fc960 msgr2=0x7f0c280fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 shutdown_connections 2024-05-08T23:24:08.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.001+0000 7f0c2ec00700 1 -- 172.21.15.46:0/4074793826 wait complete. 2024-05-08T23:24:08.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:07 smithi053 ceph-mon[77587]: pgmap v639: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:08.013 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:07 smithi053 ceph-mon[77587]: from='client.30683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:08.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T23:24:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 -- 172.21.15.46:0/3942006536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4100ed0 msgr2=0x7fc5f41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 --2- 172.21.15.46:0/3942006536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4100ed0 0x7fc5f41012b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009990 tx=0x7fc5e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 -- 172.21.15.46:0/3942006536 shutdown_connections 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 --2- 172.21.15.46:0/3942006536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f4105ff0 0x7fc5f410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 --2- 172.21.15.46:0/3942006536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4100ed0 0x7fc5f41012b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 --2- 172.21.15.46:0/3942006536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4101880 0x7fc5f41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 -- 172.21.15.46:0/3942006536 >> 172.21.15.46:0/3942006536 conn(0x7fc5f40fc770 msgr2=0x7fc5f40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 -- 172.21.15.46:0/3942006536 shutdown_connections 2024-05-08T23:24:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.223+0000 7fc5fb200700 1 -- 172.21.15.46:0/3942006536 wait complete. 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.224+0000 7fc5fb200700 1 Processor -- start 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 -- start start 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4105ff0 0x7fc5f41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f419db70 0x7fc5f419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f419e5f0 con 0x7fc5f4100ed0 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f419e760 con 0x7fc5f4105ff0 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f419e920 con 0x7fc5f419db70 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48806/0 (socket says 172.21.15.46:48806) 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5f9e00700 1 -- 172.21.15.46:0/2996012467 learned_addr learned my addr 172.21.15.46:0/2996012467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5fa800700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f419db70 0x7fc5f419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.225+0000 7fc5f9400700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4105ff0 0x7fc5f41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 -- 172.21.15.46:0/2996012467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f419db70 msgr2=0x7fc5f419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f419db70 0x7fc5f419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 -- 172.21.15.46:0/2996012467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4105ff0 msgr2=0x7fc5f41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4105ff0 0x7fc5f41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 -- 172.21.15.46:0/2996012467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80095f0 con 0x7fc5f4100ed0 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8005910 tx=0x7fc5e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5eea00700 1 -- 172.21.15.46:0/2996012467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e801d070 con 0x7fc5f4100ed0 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f41a5e20 con 0x7fc5f4100ed0 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5eea00700 1 -- 172.21.15.46:0/2996012467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e8022470 con 0x7fc5f4100ed0 2024-05-08T23:24:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f41a6250 con 0x7fc5f4100ed0 2024-05-08T23:24:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.226+0000 7fc5eea00700 1 -- 172.21.15.46:0/2996012467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80175f0 con 0x7fc5f4100ed0 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.229+0000 7fc5eea00700 1 -- 172.21.15.46:0/2996012467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc5e80225e0 con 0x7fc5f4100ed0 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc5eea00700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 0x7fc5dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 Processor -- start 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc5eea00700 1 -- 172.21.15.46:0/2996012467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc5e80ce370 con 0x7fc5f4100ed0 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc5f9400700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 0x7fc5dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 -- start start 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 0x560f145f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1454c150 con 0x560f145fb050 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f14549b10 con 0x560f145fb970 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.231+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f14545400 con 0x560f145f5890 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9400700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 0x7fc5dc0a37f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5f419a7e0 tx=0x7fc5e4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48812/0 (socket says 172.21.15.46:48812) 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3576221343 learned_addr learned my addr 172.21.15.46:0/3576221343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5fa800700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 0x560f145f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3576221343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 msgr2=0x560f145f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3576221343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 msgr2=0x560f145f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 0x560f145f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3576221343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f143c63b0 con 0x560f145fb050 2024-05-08T23:24:08.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5fa800700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 0x560f145f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.232+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80a2f10 tx=0x7fc5e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569b3e4079e732e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc5e3000700 1 -- 172.21.15.46:0/3576221343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80ce170 con 0x560f145fb050 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc5e3000700 1 -- 172.21.15.46:0/3576221343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e80aee90 con 0x560f145fb050 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc5e3000700 1 -- 172.21.15.46:0/3576221343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80a1070 con 0x560f145fb050 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3576221343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 msgr2=0x560f145fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80a2f10 tx=0x7fc5e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3576221343 shutdown_connections 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145fb430 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3576221343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb970 0x560f145f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.233+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3576221343 >> 172.21.15.46:0/3576221343 conn(0x560f14553210 msgr2=0x560f145f02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.234+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3576221343 shutdown_connections 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.234+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3576221343 wait complete. 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.234+0000 7fc6102e5b80 1 -- start start 2024-05-08T23:24:08.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.234+0000 7fc6102e5b80 1 Processor -- start 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f146018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f145ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f144e1560 con 0x560f145fb970 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f144e1200 con 0x560f145fb050 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f143c86a0 con 0x560f145f5890 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc6102e5b80 10 client.0 _ll_register_callbacks cb 0x560f144eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc5f9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc5f9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58144/0 (socket says 172.21.15.46:58144) 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc5f9400700 1 -- 172.21.15.46:0/3975966055 learned_addr learned my addr 172.21.15.46:0/3975966055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc5fa800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f145ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.235+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f146018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 -- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 msgr2=0x560f146018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f146018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 -- 172.21.15.46:0/3975966055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 msgr2=0x560f145ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f145ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 -- 172.21.15.46:0/3975966055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0011230 con 0x560f145fb050 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401e750 tx=0x7fc5e40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86645bf8346725 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5d7400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4004020 con 0x560f145fb050 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f1449e270 con 0x560f145fb050 2024-05-08T23:24:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.236+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc5e4004180 con 0x560f145fb050 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e4004340 con 0x560f145fb050 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 10 client.30740 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fc5f41a6250 con 0x560f145fb050 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f1449e270 con 0x560f145fb050 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e401c810 con 0x560f145fb050 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 10 client.30740 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 10 client.30740 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 20 client.30740 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f1460f100 0x560f14611520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 4 client.30740 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f145c4810 con 0x560f1460f100 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.237+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f1460f100 0x560f14611520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.238+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f1460f100 0x560f14611520 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfe34a4f66bd8f36 server_cookie=c9a96424f939410a in_seq=0 out_seq=0 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.238+0000 7fc5d7400700 10 client.30740 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc5d7400700 1 -- 172.21.15.46:0/3975966055 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f145c4810 con 0x560f1460f100 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc5d7400700 10 client.30740 handle_command_reply: tid=0 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 client.30740 shutdown 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 msgr2=0x560f145febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401e750 tx=0x7fc5e40047c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 shutdown_connections 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f1460f100 0x560f14611520 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f145ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f146018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3975966055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145febd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc5d7400700 0 client.30740 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 >> 172.21.15.46:0/3975966055 conn(0x560f14553210 msgr2=0x560f145f00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 shutdown_connections 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3975966055 wait complete. 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.239+0000 7fc6102e5b80 20 client.30740 trim_cache size 0 max 16384 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 Processor -- start 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 -- start start 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f1460ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145ff8f0 con 0x560f145fb970 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145ecdc0 con 0x560f145fb050 2024-05-08T23:24:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.253+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145f1230 con 0x560f145f5890 2024-05-08T23:24:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5fa800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f1460ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35644/0 (socket says 172.21.15.46:35644) 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3727071584 learned_addr learned my addr 172.21.15.46:0/3727071584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3727071584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 msgr2=0x560f145f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3727071584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 msgr2=0x560f1460ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f1460ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 -- 172.21.15.46:0/3727071584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f14372b60 con 0x560f145f5890 2024-05-08T23:24:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8005cd0 tx=0x7fc5e802e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc785eb413448c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.254+0000 7fc5e3a00700 1 -- 172.21.15.46:0/3727071584 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e809b750 con 0x560f145f5890 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc5e3a00700 1 -- 172.21.15.46:0/3727071584 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e809ab90 con 0x560f145f5890 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc5e3a00700 1 -- 172.21.15.46:0/3727071584 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80a1130 con 0x560f145f5890 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3727071584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 msgr2=0x560f145f8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8005cd0 tx=0x7fc5e802e040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3727071584 shutdown_connections 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb970 0x560f1460ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145f5890 0x560f145f8cc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3727071584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145fb050 0x560f145f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3727071584 >> 172.21.15.46:0/3727071584 conn(0x560f14553210 msgr2=0x560f14615e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3727071584 shutdown_connections 2024-05-08T23:24:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.255+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3727071584 wait complete. 2024-05-08T23:24:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 -- start start 2024-05-08T23:24:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 Processor -- start 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145f5890 0x560f14606530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 0x560f145f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145f6a70 con 0x560f145fb050 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145f6be0 con 0x560f145f5890 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f145f6da0 con 0x560f145fb970 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.256+0000 7fc6102e5b80 10 client.0 _ll_register_callbacks cb 0x560f144eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145f5890 0x560f14606530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48852/0 (socket says 172.21.15.46:48852) 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 -- 172.21.15.46:0/3732741089 learned_addr learned my addr 172.21.15.46:0/3732741089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5fa800700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 0x560f145f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 -- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 msgr2=0x560f145f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 0x560f145f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 -- 172.21.15.46:0/3732741089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145f5890 msgr2=0x560f14606530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145f5890 0x560f14606530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 -- 172.21.15.46:0/3732741089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f143c63b0 con 0x560f145fb050 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5fa800700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 0x560f145f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5f9400700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401ccc0 tx=0x7fc5e401ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a44a3a23793d112 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.257+0000 7fc5d7400700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4004350 con 0x560f145fb050 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5e80aee90 con 0x560f145fb050 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fc5e40044b0 con 0x560f145fb050 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e4003680 con 0x560f145fb050 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 10 client.20493 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560f1450b560 con 0x560f145fb050 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e80aee90 con 0x560f145fb050 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e4034820 con 0x560f145fb050 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 10 client.20493 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 10 client.20493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 20 client.20493 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f14665fc0 0x560f146683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 4 client.20493 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.258+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 --> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f1453dec0 con 0x560f14665fc0 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f14665fc0 0x560f146683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc5f9e00700 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f14665fc0 0x560f146683e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4a89a445fb05b7 server_cookie=f2488b3228369d1c in_seq=0 out_seq=0 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc5d7400700 10 client.20493 ms_handle_connect on v2:172.21.15.86:6836/2553915750 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc5d7400700 1 -- 172.21.15.46:0/3732741089 <== mds.3 v2:172.21.15.86:6836/2553915750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f1453dec0 con 0x560f14665fc0 2024-05-08T23:24:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc5d7400700 10 client.20493 handle_command_reply: tid=0 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.259+0000 7fc6102e5b80 1 client.20493 shutdown 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 msgr2=0x560f145f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc5e401ccc0 tx=0x7fc5e401ed40 comp rx=0 tx=0).stop 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 shutdown_connections 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:6836/2553915750,v1:172.21.15.86:6837/2553915750] conn(0x560f14665fc0 0x560f146683e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f145fb970 0x560f145f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560f145f5890 0x560f14606530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 --2- 172.21.15.46:0/3732741089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f145fb050 0x560f145f7bd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc5d7400700 0 client.20493 ms_handle_reset on v2:172.21.15.86:6836/2553915750 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 >> 172.21.15.46:0/3732741089 conn(0x560f14553210 msgr2=0x560f145f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 shutdown_connections 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 1 -- 172.21.15.46:0/3732741089 wait complete. 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc6102e5b80 20 client.20493 trim_cache size 0 max 16384 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.260+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 msgr2=0x7fc5dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 0x7fc5dc0a37f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5f419a7e0 tx=0x7fc5e4009260 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 msgr2=0x7fc5f4199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8005910 tx=0x7fc5e8004730 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 shutdown_connections 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f419db70 0x7fc5f419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc5dc0a1330 0x7fc5dc0a37f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5f4100ed0 0x7fc5f4199290 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 --2- 172.21.15.46:0/2996012467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f4105ff0 0x7fc5f41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 >> 172.21.15.46:0/2996012467 conn(0x7fc5f40fc770 msgr2=0x7fc5f40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 shutdown_connections 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.261+0000 7fc5fb200700 1 -- 172.21.15.46:0/2996012467 wait complete. 2024-05-08T23:24:08.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:24:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 -- 172.21.15.46:0/2170625730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 msgr2=0x7fdf84114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 --2- 172.21.15.46:0/2170625730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf84114210 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00b0a0 tx=0x7fdf7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 -- 172.21.15.46:0/2170625730 shutdown_connections 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 --2- 172.21.15.46:0/2170625730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf84114210 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 --2- 172.21.15.46:0/2170625730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073340 0x7fdf84073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 --2- 172.21.15.46:0/2170625730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073c60 0x7fdf841118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 -- 172.21.15.46:0/2170625730 >> 172.21.15.46:0/2170625730 conn(0x7fdf840fc8e0 msgr2=0x7fdf840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.546+0000 7fdf8c000700 1 -- 172.21.15.46:0/2170625730 shutdown_connections 2024-05-08T23:24:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.547+0000 7fdf8c000700 1 -- 172.21.15.46:0/2170625730 wait complete. 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 Processor -- start 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 -- start start 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073c60 0x7fdf8419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf841a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf841a2910 con 0x7fdf84073c60 2024-05-08T23:24:08.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf841a2a80 con 0x7fdf84111e20 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.548+0000 7fdf8c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf841a2c40 con 0x7fdf84073340 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.549+0000 7fdf89600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.549+0000 7fdf8a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf841a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.549+0000 7fdf89600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35666/0 (socket says 172.21.15.46:35666) 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.549+0000 7fdf89600700 1 -- 172.21.15.46:0/3323976703 learned_addr learned my addr 172.21.15.46:0/3323976703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.549+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073c60 0x7fdf8419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 -- 172.21.15.46:0/3323976703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 msgr2=0x7fdf841a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf841a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 -- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073c60 msgr2=0x7fdf8419df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073c60 0x7fdf8419df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 -- 172.21.15.46:0/3323976703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c009d00 con 0x7fdf84073340 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf89600700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9a0 tx=0x7fdf7800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf82000700 1 -- 172.21.15.46:0/3323976703 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78009790 con 0x7fdf84073340 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf841a2f20 con 0x7fdf84073340 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.550+0000 7fdf82000700 1 -- 172.21.15.46:0/3323976703 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7800cbd0 con 0x7fdf84073340 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.551+0000 7fdf82000700 1 -- 172.21.15.46:0/3323976703 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78018680 con 0x7fdf84073340 2024-05-08T23:24:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.551+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf841aa7d0 con 0x7fdf84073340 2024-05-08T23:24:08.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.555+0000 7fdf82000700 1 -- 172.21.15.46:0/3323976703 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdf7800cd40 con 0x7fdf84073340 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.557+0000 7fdf9fa83b80 1 Processor -- start 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.557+0000 7fdf9fa83b80 1 -- start start 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb7e00 0x5558d1bb81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bb2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1b09150 con 0x5558d1bb7e00 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1b06b10 con 0x5558d1bb2890 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1b02400 con 0x5558d1bb8720 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf89600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf88c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bb2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf8a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb7e00 0x5558d1bb81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf89600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35678/0 (socket says 172.21.15.46:35678) 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.558+0000 7fdf89600700 1 -- 172.21.15.46:0/659222772 learned_addr learned my addr 172.21.15.46:0/659222772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf82000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 0x7fdf6c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf82000700 1 -- 172.21.15.46:0/3323976703 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fdf78014070 con 0x7fdf84073340 2024-05-08T23:24:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf88c00700 1 -- 172.21.15.46:0/659222772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 msgr2=0x5558d1bb2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf88c00700 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf88c00700 1 -- 172.21.15.46:0/659222772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb7e00 msgr2=0x5558d1bb81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf88c00700 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb7e00 0x5558d1bb81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf88c00700 1 -- 172.21.15.46:0/659222772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558d19833b0 con 0x5558d1bb2890 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.559+0000 7fdf8a000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 0x7fdf6c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.560+0000 7fdf8a000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 0x7fdf6c0a37d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00b780 tx=0x7fdf7c00b3b0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.562+0000 7fdf88c00700 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bb2cd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdf84114350 tx=0x7fdf70008dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ac3b25a60e03b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.563+0000 7fdf76600700 1 -- 172.21.15.46:0/659222772 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000ec50 con 0x5558d1bb2890 2024-05-08T23:24:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.565+0000 7fdf76600700 1 -- 172.21.15.46:0/659222772 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7000edb0 con 0x5558d1bb2890 2024-05-08T23:24:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.565+0000 7fdf76600700 1 -- 172.21.15.46:0/659222772 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70010430 con 0x5558d1bb2890 2024-05-08T23:24:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.566+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/659222772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 msgr2=0x5558d1bb2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.566+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bb2cd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdf84114350 tx=0x7fdf70008dc0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/659222772 shutdown_connections 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bb2cd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb7e00 0x5558d1bb81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/659222772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/659222772 >> 172.21.15.46:0/659222772 conn(0x5558d1b10210 msgr2=0x5558d1bacdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/659222772 shutdown_connections 2024-05-08T23:24:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.567+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/659222772 wait complete. 2024-05-08T23:24:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.568+0000 7fdf9fa83b80 1 -- start start 2024-05-08T23:24:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.568+0000 7fdf9fa83b80 1 Processor -- start 2024-05-08T23:24:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bbbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb8720 0x5558d1bbc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1a9e560 con 0x5558d1bb8720 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1a9e200 con 0x5558d1bb2890 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d19856a0 con 0x5558d1bb7e00 2024-05-08T23:24:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.569+0000 7fdf9fa83b80 10 client.0 _ll_register_callbacks cb 0x5558d1aa7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35686/0 (socket says 172.21.15.46:35686) 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 -- 172.21.15.46:0/3176793419 learned_addr learned my addr 172.21.15.46:0/3176793419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf8a000700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb8720 0x5558d1bbc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 -- 172.21.15.46:0/3176793419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 msgr2=0x5558d1bbbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bbbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 -- 172.21.15.46:0/3176793419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb8720 msgr2=0x5558d1bbc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb8720 0x5558d1bbc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.570+0000 7fdf88c00700 1 -- 172.21.15.46:0/3176793419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf70008740 con 0x5558d1bb7e00 2024-05-08T23:24:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.571+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005640 tx=0x7fdf70005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d948bd2d07e2675 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.571+0000 7fdf6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:24:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001f070 con 0x5558d1bb7e00 2024-05-08T23:24:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fdf70004960 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558d1a5b270 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf70004b20 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf9fa83b80 10 client.30757 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5558d192fb60 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.572+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558d1a5b270 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf70010d00 con 0x5558d1bb7e00 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 10 client.30757 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 10 client.30757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 20 client.30757 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1bdd030 0x5558d1bdf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 4 client.30757 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558d1b81810 con 0x5558d1bdd030 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.573+0000 7fdf89600700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1bdd030 0x5558d1bdf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.574+0000 7fdf89600700 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1bdd030 0x5558d1bdf450 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c4944b22b56985c server_cookie=575da43e2c1e943 in_seq=0 out_seq=0 2024-05-08T23:24:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.574+0000 7fdf6aa00700 10 client.30757 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:24:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.574+0000 7fdf6aa00700 1 -- 172.21.15.46:0/3176793419 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558d1b81810 con 0x5558d1bdd030 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.574+0000 7fdf6aa00700 10 client.30757 handle_command_reply: tid=0 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.574+0000 7fdf9fa83b80 1 client.30757 shutdown 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 msgr2=0x5558d1bbc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005640 tx=0x7fdf70005670 comp rx=0 tx=0).stop 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 shutdown_connections 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1bdd030 0x5558d1bdf450 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb8720 0x5558d1bbc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb2890 0x5558d1bbbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3176793419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bbc0c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 >> 172.21.15.46:0/3176793419 conn(0x5558d1b10210 msgr2=0x5558d1bac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf6aa00700 0 client.30757 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 shutdown_connections 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3176793419 wait complete. 2024-05-08T23:24:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.575+0000 7fdf9fa83b80 20 client.30757 trim_cache size 0 max 16384 2024-05-08T23:24:08.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.595+0000 7fdf9fa83b80 1 Processor -- start 2024-05-08T23:24:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.595+0000 7fdf9fa83b80 1 -- start start 2024-05-08T23:24:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bb2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bb8460 con 0x5558d1bb2890 2024-05-08T23:24:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bcfe90 con 0x5558d1bb7e00 2024-05-08T23:24:08.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bd9cf0 con 0x5558d1bb8720 2024-05-08T23:24:08.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf88c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf88c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58192/0 (socket says 172.21.15.46:58192) 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf88c00700 1 -- 172.21.15.46:0/3604081467 learned_addr learned my addr 172.21.15.46:0/3604081467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.596+0000 7fdf89600700 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bb2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 -- 172.21.15.46:0/3604081467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 msgr2=0x5558d1bb5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 -- 172.21.15.46:0/3604081467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 msgr2=0x5558d1bb2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bb2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 -- 172.21.15.46:0/3604081467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558d1ac8560 con 0x5558d1bb7e00 2024-05-08T23:24:08.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf88c00700 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdf70008a30 tx=0x7fdf70005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df93906723902a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf77000700 1 -- 172.21.15.46:0/3604081467 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001f070 con 0x5558d1bb7e00 2024-05-08T23:24:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf77000700 1 -- 172.21.15.46:0/3604081467 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf70018730 con 0x5558d1bb7e00 2024-05-08T23:24:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.597+0000 7fdf77000700 1 -- 172.21.15.46:0/3604081467 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70018c60 con 0x5558d1bb7e00 2024-05-08T23:24:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3604081467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 msgr2=0x5558d1bacd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdf70008a30 tx=0x7fdf70005960 comp rx=0 tx=0).stop 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3604081467 shutdown_connections 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb8720 0x5558d1bb5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bb2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/3604081467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb7e00 0x5558d1bacd20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3604081467 >> 172.21.15.46:0/3604081467 conn(0x5558d1b10210 msgr2=0x5558d1be3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.598+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3604081467 shutdown_connections 2024-05-08T23:24:08.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.599+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/3604081467 wait complete. 2024-05-08T23:24:08.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.599+0000 7fdf9fa83b80 1 -- start start 2024-05-08T23:24:08.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.599+0000 7fdf9fa83b80 1 Processor -- start 2024-05-08T23:24:08.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bd4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bd4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bd2900 con 0x5558d1bb2890 2024-05-08T23:24:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bd2a70 con 0x5558d1bb8720 2024-05-08T23:24:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d1bd2c30 con 0x5558d1bb7e00 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf9fa83b80 10 client.0 _ll_register_callbacks cb 0x5558d1aa7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf8a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf8a000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58200/0 (socket says 172.21.15.46:58200) 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf8a000700 1 -- 172.21.15.46:0/1622645489 learned_addr learned my addr 172.21.15.46:0/1622645489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf88c00700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bd4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf89600700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bd4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.600+0000 7fdf8a000700 1 -- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 msgr2=0x5558d1bd4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf8a000700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bd4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf8a000700 1 -- 172.21.15.46:0/1622645489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 msgr2=0x5558d1bd4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf8a000700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bd4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf8a000700 1 -- 172.21.15.46:0/1622645489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558d192fb60 con 0x5558d1bb8720 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf8a000700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009c50 tx=0x7fdf7c0093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ac7c103f5a5d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00e040 con 0x5558d1bb8720 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf7000edb0 con 0x5558d1bb8720 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fdf7c028040 con 0x5558d1bb8720 2024-05-08T23:24:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7c0074e0 con 0x5558d1bb8720 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf9fa83b80 10 client.30761 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.601+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5558d1987cd0 con 0x5558d1bb8720 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7000edb0 con 0x5558d1bb8720 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf7c005040 con 0x5558d1bb8720 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 10 client.30761 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 10 client.30761 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 20 client.30761 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1c33840 0x5558d1c35c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 4 client.30761 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 --> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5558d1afaec0 con 0x5558d1c33840 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.602+0000 7fdf89600700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1c33840 0x5558d1c35c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf6aa00700 10 client.30761 ms_handle_connect on v2:172.21.15.86:6838/4115816206 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf89600700 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1c33840 0x5558d1c35c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae5724d0fd0be16 server_cookie=5a47ed31e72c89a8 in_seq=0 out_seq=0 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf6aa00700 1 -- 172.21.15.46:0/1622645489 <== mds.2 v2:172.21.15.86:6838/4115816206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5558d1afaec0 con 0x5558d1c33840 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf6aa00700 10 client.30761 handle_command_reply: tid=0 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 client.30761 shutdown 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 msgr2=0x5558d1bd21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009c50 tx=0x7fdf7c0093c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 shutdown_connections 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558d1bb8720 0x5558d1bd21f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:6838/4115816206,v1:172.21.15.86:6839/4115816206] conn(0x5558d1c33840 0x5558d1c35c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5558d1bb2890 0x5558d1bd4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 --2- 172.21.15.46:0/1622645489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d1bb7e00 0x5558d1bd4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf6aa00700 0 client.30761 ms_handle_reset on v2:172.21.15.86:6838/4115816206 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.603+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 >> 172.21.15.46:0/1622645489 conn(0x5558d1b10210 msgr2=0x5558d1bba3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.604+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 shutdown_connections 2024-05-08T23:24:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.604+0000 7fdf9fa83b80 1 -- 172.21.15.46:0/1622645489 wait complete. 2024-05-08T23:24:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.604+0000 7fdf9fa83b80 20 client.30761 trim_cache size 0 max 16384 2024-05-08T23:24:08.613 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:24:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 msgr2=0x7fdf6c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 0x7fdf6c0a37d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00b780 tx=0x7fdf7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 msgr2=0x7fdf8419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9a0 tx=0x7fdf7800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 shutdown_connections 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf84111e20 0x7fdf841a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf84073340 0x7fdf8419da30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fdf6c0a1310 0x7fdf6c0a37d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 --2- 172.21.15.46:0/3323976703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84073c60 0x7fdf8419df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 >> 172.21.15.46:0/3323976703 conn(0x7fdf840fc8e0 msgr2=0x7fdf840fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 shutdown_connections 2024-05-08T23:24:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.605+0000 7fdf8c000700 1 -- 172.21.15.46:0/3323976703 wait complete. 2024-05-08T23:24:08.623 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:08 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2049518819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:08.623 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:08 smithi046 ceph-mon[36898]: from='client.30712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:08.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:24:08.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.928+0000 7f14e2600700 1 -- 172.21.15.46:0/963698292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc06c3f0 msgr2=0x7f14dc06a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.928+0000 7f14e2600700 1 --2- 172.21.15.46:0/963698292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc06c3f0 0x7f14dc06a620 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b0a0 tx=0x7f14cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 -- 172.21.15.46:0/963698292 shutdown_connections 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 --2- 172.21.15.46:0/963698292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14dc11a300 0x7f14dc11c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 --2- 172.21.15.46:0/963698292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc06c3f0 0x7f14dc06a620 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 --2- 172.21.15.46:0/963698292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 -- 172.21.15.46:0/963698292 >> 172.21.15.46:0/963698292 conn(0x7f14dc06df70 msgr2=0x7f14dc107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 -- 172.21.15.46:0/963698292 shutdown_connections 2024-05-08T23:24:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.929+0000 7f14e2600700 1 -- 172.21.15.46:0/963698292 wait complete. 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.930+0000 7f14e2600700 1 Processor -- start 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 -- start start 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14dc06c3f0 0x7f14dc1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc11a300 0x7f14dc1aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc1ab1b0 con 0x7f14dc06c3f0 2024-05-08T23:24:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc1ab320 con 0x7f14dc06ab60 2024-05-08T23:24:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14e2600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc1ab4e0 con 0x7f14dc11a300 2024-05-08T23:24:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14db400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14db400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58210/0 (socket says 172.21.15.46:58210) 2024-05-08T23:24:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.931+0000 7f14db400700 1 -- 172.21.15.46:0/2488176023 learned_addr learned my addr 172.21.15.46:0/2488176023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 -- 172.21.15.46:0/2488176023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc11a300 msgr2=0x7f14dc1aab70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:24:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc11a300 0x7f14dc1aab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 -- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14dc06c3f0 msgr2=0x7f14dc1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14dc06c3f0 0x7f14dc1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 -- 172.21.15.46:0/2488176023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc009d00 con 0x7f14dc06ab60 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.932+0000 7f14db400700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0062a0 tx=0x7f14cc005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.933+0000 7f14d3e00700 1 -- 172.21.15.46:0/2488176023 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00e040 con 0x7f14dc06ab60 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.933+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc1ab760 con 0x7f14dc06ab60 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.933+0000 7f14d3e00700 1 -- 172.21.15.46:0/2488176023 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14cc01d480 con 0x7f14dc06ab60 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.933+0000 7f14d3e00700 1 -- 172.21.15.46:0/2488176023 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc01b690 con 0x7f14dc06ab60 2024-05-08T23:24:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.933+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1b3000 con 0x7f14dc06ab60 2024-05-08T23:24:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.937+0000 7f14d3e00700 1 -- 172.21.15.46:0/2488176023 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f14cc01b7f0 con 0x7f14dc06ab60 2024-05-08T23:24:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.940+0000 7f14d3e00700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 0x7f14c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.940+0000 7f14d3e00700 1 -- 172.21.15.46:0/2488176023 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f14cc0cffb0 con 0x7f14dc06ab60 2024-05-08T23:24:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.941+0000 7f14daa00700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 0x7f14c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.941+0000 7f14f6096b80 1 Processor -- start 2024-05-08T23:24:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.941+0000 7f14f6096b80 1 -- start start 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14daa00700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 0x7f14c00a37d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14dc1a77e0 tx=0x7f14c8006cd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d570350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 0x55dd7d570cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d4c7150 con 0x55dd7d576720 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d4c4b10 con 0x55dd7d570890 2024-05-08T23:24:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14f6096b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d4c0400 con 0x55dd7d575e00 2024-05-08T23:24:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35738/0 (socket says 172.21.15.46:35738) 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14db400700 1 -- 172.21.15.46:0/2184141860 learned_addr learned my addr 172.21.15.46:0/2184141860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 0x55dd7d570cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.942+0000 7f14daa00700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d570350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 -- 172.21.15.46:0/2184141860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 msgr2=0x55dd7d5761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 -- 172.21.15.46:0/2184141860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 msgr2=0x55dd7d570350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d570350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 -- 172.21.15.46:0/2184141860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7d3413b0 con 0x55dd7d570890 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14db400700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.943+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 0x55dd7d570cd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06c800 tx=0x7f14d40099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e99c1ac468e04e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.944+0000 7f14d0c00700 1 -- 172.21.15.46:0/2184141860 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40107b0 con 0x55dd7d570890 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.944+0000 7f14d0c00700 1 -- 172.21.15.46:0/2184141860 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d4010dd0 con 0x55dd7d570890 2024-05-08T23:24:08.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.944+0000 7f14d0c00700 1 -- 172.21.15.46:0/2184141860 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4011ea0 con 0x55dd7d570890 2024-05-08T23:24:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.946+0000 7f14f6096b80 1 -- 172.21.15.46:0/2184141860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 msgr2=0x55dd7d570cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.946+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 0x55dd7d570cd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06c800 tx=0x7f14d40099b0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.947+0000 7f14f6096b80 1 -- 172.21.15.46:0/2184141860 shutdown_connections 2024-05-08T23:24:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.947+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d570890 0x55dd7d570cd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.947+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d575e00 0x55dd7d5761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.947+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2184141860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d570350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.947+0000 7f14f6096b80 1 -- 172.21.15.46:0/2184141860 >> 172.21.15.46:0/2184141860 conn(0x55dd7d4ce210 msgr2=0x55dd7d56adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.948+0000 7f14f6096b80 1 -- 172.21.15.46:0/2184141860 shutdown_connections 2024-05-08T23:24:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.948+0000 7f14f6096b80 1 -- 172.21.15.46:0/2184141860 wait complete. 2024-05-08T23:24:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.948+0000 7f14f6096b80 1 -- start start 2024-05-08T23:24:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.949+0000 7f14f6096b80 1 Processor -- start 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.949+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d579b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.949+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.949+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14f6096b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d45c560 con 0x55dd7d570890 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14daa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14daa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58228/0 (socket says 172.21.15.46:58228) 2024-05-08T23:24:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14daa00700 1 -- 172.21.15.46:0/2767904427 learned_addr learned my addr 172.21.15.46:0/2767904427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14dbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d45c200 con 0x55dd7d575e00 2024-05-08T23:24:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d3436a0 con 0x55dd7d576720 2024-05-08T23:24:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.950+0000 7f14f6096b80 10 client.0 _ll_register_callbacks cb 0x55dd7d465b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14db400700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d579b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14daa00700 1 -- 172.21.15.46:0/2767904427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 msgr2=0x55dd7d57a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14daa00700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14daa00700 1 -- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 msgr2=0x55dd7d579b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14daa00700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d579b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.951+0000 7f14daa00700 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1b3000 con 0x55dd7d575e00 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.952+0000 7f14daa00700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f14c8012660 tx=0x7f14c8004850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adef123a1d71b25e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.952+0000 7f14c5600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.952+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c801a050 con 0x55dd7d575e00 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.952+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f14c8012e30 con 0x55dd7d575e00 2024-05-08T23:24:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.952+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd7d419270 con 0x55dd7d575e00 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.953+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c8029450 con 0x55dd7d575e00 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.953+0000 7f14f6096b80 10 client.30770 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.953+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f14d4009330 con 0x55dd7d575e00 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.953+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd7d419270 con 0x55dd7d575e00 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14c801f490 con 0x55dd7d575e00 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 10 client.30770 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 10 client.30770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 20 client.30770 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d58a2e0 0x55dd7d58c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 4 client.30770 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.954+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd7d53f810 con 0x55dd7d58a2e0 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.955+0000 7f14db400700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d58a2e0 0x55dd7d58c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.955+0000 7f14c5600700 10 client.30770 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:24:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.955+0000 7f14db400700 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d58a2e0 0x55dd7d58c700 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b33a4c0523c34d7a server_cookie=7b3c22eee5442cbd in_seq=0 out_seq=0 2024-05-08T23:24:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.956+0000 7f14c5600700 1 -- 172.21.15.46:0/2767904427 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd7d53f810 con 0x55dd7d58a2e0 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.956+0000 7f14c5600700 10 client.30770 handle_command_reply: tid=0 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.956+0000 7f14f6096b80 1 client.30770 shutdown 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 msgr2=0x55dd7d57a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f14c8012660 tx=0x7f14c8004850 comp rx=0 tx=0).stop 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 shutdown_connections 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d58a2e0 0x55dd7d58c700 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d579b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2767904427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d57a080 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14c5600700 0 client.30770 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 >> 172.21.15.46:0/2767904427 conn(0x55dd7d4ce210 msgr2=0x55dd7d56a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 shutdown_connections 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.957+0000 7f14f6096b80 1 -- 172.21.15.46:0/2767904427 wait complete. 2024-05-08T23:24:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.958+0000 7f14f6096b80 20 client.30770 trim_cache size 0 max 16384 2024-05-08T23:24:08.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:08 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2049518819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:08.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:08 smithi086 ceph-mon[78610]: from='client.30712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:08.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.976+0000 7f14f6096b80 1 Processor -- start 2024-05-08T23:24:08.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 -- start start 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d575340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d56ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d565800 con 0x55dd7d576720 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d577b10 con 0x55dd7d575e00 2024-05-08T23:24:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.977+0000 7f14f6096b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d5780b0 con 0x55dd7d570890 2024-05-08T23:24:08.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35762/0 (socket says 172.21.15.46:35762) 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 -- 172.21.15.46:0/2794155316 learned_addr learned my addr 172.21.15.46:0/2794155316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14daa00700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d575340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 -- 172.21.15.46:0/2794155316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 msgr2=0x55dd7d575340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d575340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14db400700 1 -- 172.21.15.46:0/2794155316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 msgr2=0x55dd7d56ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.978+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d56ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14db400700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d56ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14db400700 1 -- 172.21.15.46:0/2794155316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7d2edb60 con 0x55dd7d570890 2024-05-08T23:24:08.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14dbe00700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d56ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14db400700 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0cd350 tx=0x7f14cc009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15bf6abc3cf3a7f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14d1600700 1 -- 172.21.15.46:0/2794155316 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc09b090 con 0x55dd7d570890 2024-05-08T23:24:08.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14d1600700 1 -- 172.21.15.46:0/2794155316 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14cc0b4070 con 0x55dd7d570890 2024-05-08T23:24:08.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.979+0000 7f14d1600700 1 -- 172.21.15.46:0/2794155316 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc09cf50 con 0x55dd7d570890 2024-05-08T23:24:08.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.980+0000 7f14f6096b80 1 -- 172.21.15.46:0/2794155316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 msgr2=0x55dd7d570350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.980+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0cd350 tx=0x7f14cc009040 comp rx=0 tx=0).stop 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 -- 172.21.15.46:0/2794155316 shutdown_connections 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d576720 0x55dd7d56ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d570890 0x55dd7d570350 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 --2- 172.21.15.46:0/2794155316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d575340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 -- 172.21.15.46:0/2794155316 >> 172.21.15.46:0/2794155316 conn(0x55dd7d4ce210 msgr2=0x55dd7d574c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 -- 172.21.15.46:0/2794155316 shutdown_connections 2024-05-08T23:24:08.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 -- 172.21.15.46:0/2794155316 wait complete. 2024-05-08T23:24:08.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.983+0000 7f14f6096b80 1 -- start start 2024-05-08T23:24:08.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.984+0000 7f14f6096b80 1 Processor -- start 2024-05-08T23:24:08.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.984+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d5db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d5dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d5dc4e0 con 0x55dd7d570890 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d5dc650 con 0x55dd7d575e00 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7d57f620 con 0x55dd7d576720 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.985+0000 7f14f6096b80 10 client.0 _ll_register_callbacks cb 0x55dd7d465b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14db400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d5db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14dbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14db400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d5db7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48966/0 (socket says 172.21.15.46:48966) 2024-05-08T23:24:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14db400700 1 -- 172.21.15.46:0/3312126128 learned_addr learned my addr 172.21.15.46:0/3312126128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d5dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 -- 172.21.15.46:0/3312126128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 msgr2=0x55dd7d57f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 -- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 msgr2=0x55dd7d5db7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d5db7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.986+0000 7f14daa00700 1 -- 172.21.15.46:0/3312126128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1ab760 con 0x55dd7d575e00 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.987+0000 7f14daa00700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d5dbd00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009e90 tx=0x7f14c8015740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ec3c8fcf483d46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.987+0000 7f14c5600700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.987+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c801a040 con 0x55dd7d575e00 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.987+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d4010dd0 con 0x55dd7d575e00 2024-05-08T23:24:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.987+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f14c8032480 con 0x55dd7d575e00 2024-05-08T23:24:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.988+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c801fbb0 con 0x55dd7d575e00 2024-05-08T23:24:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.988+0000 7f14f6096b80 10 client.30776 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.988+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55dd7d486560 con 0x55dd7d575e00 2024-05-08T23:24:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d4010dd0 con 0x55dd7d575e00 2024-05-08T23:24:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14c8029450 con 0x55dd7d575e00 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 10 client.30776 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 10 client.30776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 20 client.30776 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d5f24c0 0x55dd7d5f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 4 client.30776 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.989+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 --> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd7d4b8ec0 con 0x55dd7d5f24c0 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.990+0000 7f14db400700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d5f24c0 0x55dd7d5f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.990+0000 7f14db400700 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d5f24c0 0x55dd7d5f48e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fa15f22d85623de server_cookie=534514ec6baf32e in_seq=0 out_seq=0 2024-05-08T23:24:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.990+0000 7f14c5600700 10 client.30776 ms_handle_connect on v2:172.21.15.46:6838/1484730309 2024-05-08T23:24:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.991+0000 7f14c5600700 1 -- 172.21.15.46:0/3312126128 <== mds.1 v2:172.21.15.46:6838/1484730309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd7d4b8ec0 con 0x55dd7d5f24c0 2024-05-08T23:24:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.992+0000 7f14c5600700 10 client.30776 handle_command_reply: tid=0 2024-05-08T23:24:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 client.30776 shutdown 2024-05-08T23:24:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 msgr2=0x55dd7d5dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d5dbd00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009e90 tx=0x7f14c8015740 comp rx=0 tx=0).stop 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 shutdown_connections 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd7d576720 0x55dd7d57f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:6838/1484730309,v1:172.21.15.46:6839/1484730309] conn(0x55dd7d5f24c0 0x55dd7d5f48e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd7d570890 0x55dd7d5db7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 --2- 172.21.15.46:0/3312126128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd7d575e00 0x55dd7d5dbd00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 >> 172.21.15.46:0/3312126128 conn(0x55dd7d4ce210 msgr2=0x55dd7d57a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.993+0000 7f14c5600700 0 client.30776 ms_handle_reset on v2:172.21.15.46:6838/1484730309 2024-05-08T23:24:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.994+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 shutdown_connections 2024-05-08T23:24:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.994+0000 7f14f6096b80 1 -- 172.21.15.46:0/3312126128 wait complete. 2024-05-08T23:24:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.994+0000 7f14f6096b80 20 client.30776 trim_cache size 0 max 16384 2024-05-08T23:24:08.996 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:24:08.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.998+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 msgr2=0x7f14c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.998+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 0x7f14c00a37d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14dc1a77e0 tx=0x7f14c8006cd0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.998+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 msgr2=0x7f14dc1a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.998+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0062a0 tx=0x7f14cc005b80 comp rx=0 tx=0).stop 2024-05-08T23:24:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 shutdown_connections 2024-05-08T23:24:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14dc11a300 0x7f14dc1aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14dc06ab60 0x7f14dc1a6290 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f14c00a1310 0x7f14c00a37d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 --2- 172.21.15.46:0/2488176023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14dc06c3f0 0x7f14dc1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:08.999+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 >> 172.21.15.46:0/2488176023 conn(0x7f14dc06df70 msgr2=0x7f14dc11ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.000+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 shutdown_connections 2024-05-08T23:24:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.000+0000 7f14e2600700 1 -- 172.21.15.46:0/2488176023 wait complete. 2024-05-08T23:24:09.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:08 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2049518819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:09.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:08 smithi053 ceph-mon[77587]: from='client.30712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:24:09.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:24:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 -- 172.21.15.46:0/2445909724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c10f5e0 msgr2=0x7f314c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 --2- 172.21.15.46:0/2445909724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c10f5e0 0x7f314c10f9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 -- 172.21.15.46:0/2445909724 shutdown_connections 2024-05-08T23:24:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 --2- 172.21.15.46:0/2445909724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c11a780 0x7f314c11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 --2- 172.21.15.46:0/2445909724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c10f5e0 0x7f314c10f9c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 --2- 172.21.15.46:0/2445909724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c108030 0x7f314c1084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 -- 172.21.15.46:0/2445909724 >> 172.21.15.46:0/2445909724 conn(0x7f314c06bf30 msgr2=0x7f314c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 -- 172.21.15.46:0/2445909724 shutdown_connections 2024-05-08T23:24:09.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.348+0000 7f3154200700 1 -- 172.21.15.46:0/2445909724 wait complete. 2024-05-08T23:24:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.349+0000 7f3154200700 1 Processor -- start 2024-05-08T23:24:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3154200700 1 -- start start 2024-05-08T23:24:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3154200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3154200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 0x7f314c1a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3154200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 0x7f314c1aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3151800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3152200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 0x7f314c1aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3150e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 0x7f314c1a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3154200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1ab1a0 con 0x7f314c11a780 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.350+0000 7f3151800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35780/0 (socket says 172.21.15.46:35780) 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3151800700 1 -- 172.21.15.46:0/3857243188 learned_addr learned my addr 172.21.15.46:0/3857243188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1ab310 con 0x7f314c10f5e0 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.351+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1ab4d0 con 0x7f314c108030 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 -- 172.21.15.46:0/3857243188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 msgr2=0x7f314c1a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 -- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 msgr2=0x7f314c1aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 0x7f314c1aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 -- 172.21.15.46:0/3857243188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0095f0 con 0x7f314c10f5e0 2024-05-08T23:24:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f3150e00700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 0x7f314c1a67c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f314000d750 tx=0x7f314000db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f314a000700 1 -- 172.21.15.46:0/3857243188 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140004220 con 0x7f314c10f5e0 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f314a000700 1 -- 172.21.15.46:0/3857243188 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3140004cf0 con 0x7f314c10f5e0 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.352+0000 7f314a000700 1 -- 172.21.15.46:0/3857243188 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000b6f0 con 0x7f314c10f5e0 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.353+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c1ab7b0 con 0x7f314c10f5e0 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.353+0000 7f3151800700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.353+0000 7f3152200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 0x7f314c1aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.353+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c1b2ff0 con 0x7f314c10f5e0 2024-05-08T23:24:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.358+0000 7f314a000700 1 -- 172.21.15.46:0/3857243188 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3140004750 con 0x7f314c10f5e0 2024-05-08T23:24:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 Processor -- start 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 -- start start 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8223e00 0x5559e82241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e821ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e8175150 con 0x5559e821e890 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e8172b10 con 0x5559e8224720 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.360+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e816e400 con 0x5559e8223e00 2024-05-08T23:24:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58270/0 (socket says 172.21.15.46:58270) 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 -- 172.21.15.46:0/916292563 learned_addr learned my addr 172.21.15.46:0/916292563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3152200700 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8223e00 0x5559e82241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 -- 172.21.15.46:0/916292563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8223e00 msgr2=0x5559e82241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8223e00 0x5559e82241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 -- 172.21.15.46:0/916292563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 msgr2=0x5559e821ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e821ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.361+0000 7f3151800700 1 -- 172.21.15.46:0/916292563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e7fef3b0 con 0x5559e8224720 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.362+0000 7f3151800700 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f313c009fa0 tx=0x7f313c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6508b4bb29019eef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.362+0000 7f314a000700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 0x7f31340a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.362+0000 7f314a000700 1 -- 172.21.15.46:0/3857243188 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f31400ccf40 con 0x7f314c10f5e0 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.362+0000 7f3152200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 0x7f31340a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.362+0000 7f313aa00700 1 -- 172.21.15.46:0/916292563 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00b9d0 con 0x5559e8224720 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.363+0000 7f313aa00700 1 -- 172.21.15.46:0/916292563 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c01e040 con 0x5559e8224720 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.363+0000 7f313aa00700 1 -- 172.21.15.46:0/916292563 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c005720 con 0x5559e8224720 2024-05-08T23:24:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.363+0000 7f3167cfbb80 1 -- 172.21.15.46:0/916292563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 msgr2=0x5559e821e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.363+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f313c009fa0 tx=0x7f313c003680 comp rx=0 tx=0).stop 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.363+0000 7f3152200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 0x7f31340a3850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f314400c610 tx=0x7f314400e040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 -- 172.21.15.46:0/916292563 shutdown_connections 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e821ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8223e00 0x5559e82241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/916292563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8224720 0x5559e821e350 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 -- 172.21.15.46:0/916292563 >> 172.21.15.46:0/916292563 conn(0x5559e817c210 msgr2=0x5559e8218df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.365+0000 7f3167cfbb80 1 -- 172.21.15.46:0/916292563 shutdown_connections 2024-05-08T23:24:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.366+0000 7f3167cfbb80 1 -- 172.21.15.46:0/916292563 wait complete. 2024-05-08T23:24:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.366+0000 7f3167cfbb80 1 -- start start 2024-05-08T23:24:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 Processor -- start 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e8227ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e810a560 con 0x5559e821e890 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e810a200 con 0x5559e8223e00 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e7ff16a0 con 0x5559e8224720 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.367+0000 7f3167cfbb80 10 client.0 _ll_register_callbacks cb 0x5559e8113b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58284/0 (socket says 172.21.15.46:58284) 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 -- 172.21.15.46:0/3676606413 learned_addr learned my addr 172.21.15.46:0/3676606413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 -- 172.21.15.46:0/3676606413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 msgr2=0x5559e8228720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 -- 172.21.15.46:0/3676606413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 msgr2=0x5559e8227ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e8227ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.368+0000 7f3150e00700 1 -- 172.21.15.46:0/3676606413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c11d3e0 con 0x5559e8223e00 2024-05-08T23:24:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f3150e00700 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3140099b20 tx=0x7f3140099780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e012881242408f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f312f400700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314009d9c0 con 0x5559e8223e00 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f31400a2010 con 0x5559e8223e00 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559e80c7270 con 0x5559e8223e00 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f314000f3c0 con 0x5559e8223e00 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.369+0000 7f3167cfbb80 10 client.30788 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5559e7f9bb60 con 0x5559e8223e00 2024-05-08T23:24:09.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559e80c7270 con 0x5559e8223e00 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3140099080 con 0x5559e8223e00 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 10 client.30788 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 10 client.30788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 20 client.30788 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e82380f0 0x5559e823a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 4 client.30788 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.370+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559e81ed810 con 0x5559e82380f0 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.371+0000 7f3151800700 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e82380f0 0x5559e823a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.371+0000 7f3151800700 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e82380f0 0x5559e823a510 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9e890f0ff7a18d server_cookie=5a76d375f48fe580 in_seq=0 out_seq=0 2024-05-08T23:24:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.371+0000 7f312f400700 10 client.30788 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.373+0000 7f312f400700 1 -- 172.21.15.46:0/3676606413 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559e81ed810 con 0x5559e82380f0 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.373+0000 7f312f400700 10 client.30788 handle_command_reply: tid=0 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.373+0000 7f3167cfbb80 1 client.30788 shutdown 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.373+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 msgr2=0x5559e82281e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.373+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3140099b20 tx=0x7f3140099780 comp rx=0 tx=0).stop 2024-05-08T23:24:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 shutdown_connections 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e82380f0 0x5559e823a510 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f312f400700 0 client.30788 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e821e890 0x5559e8227ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/3676606413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e8223e00 0x5559e82281e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 >> 172.21.15.46:0/3676606413 conn(0x5559e817c210 msgr2=0x5559e82185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 shutdown_connections 2024-05-08T23:24:09.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.374+0000 7f3167cfbb80 1 -- 172.21.15.46:0/3676606413 wait complete. 2024-05-08T23:24:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.375+0000 7f3167cfbb80 20 client.30788 trim_cache size 0 max 16384 2024-05-08T23:24:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.400+0000 7f3167cfbb80 1 Processor -- start 2024-05-08T23:24:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.401+0000 7f3167cfbb80 1 -- start start 2024-05-08T23:24:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.401+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e821ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e8218d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e821ae50 con 0x5559e8223e00 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e823f720 con 0x5559e821e890 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e822ac20 con 0x5559e8224720 2024-05-08T23:24:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3152200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3152200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:35802/0 (socket says 172.21.15.46:35802) 2024-05-08T23:24:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3152200700 1 -- 172.21.15.46:0/2077758840 learned_addr learned my addr 172.21.15.46:0/2077758840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.402+0000 7f3150e00700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e8218d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3151800700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e821ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 -- 172.21.15.46:0/2077758840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 msgr2=0x5559e821ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e821ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 -- 172.21.15.46:0/2077758840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 msgr2=0x5559e8218d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e8218d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 -- 172.21.15.46:0/2077758840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e8134560 con 0x5559e8224720 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3151800700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e821ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.403+0000 7f3152200700 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3144018520 tx=0x7f3144014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63c680cd6b82700 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.404+0000 7f313b400700 1 -- 172.21.15.46:0/2077758840 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144022420 con 0x5559e8224720 2024-05-08T23:24:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.404+0000 7f313b400700 1 -- 172.21.15.46:0/2077758840 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3144012040 con 0x5559e8224720 2024-05-08T23:24:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.404+0000 7f313b400700 1 -- 172.21.15.46:0/2077758840 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31440168c0 con 0x5559e8224720 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.405+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2077758840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 msgr2=0x5559e8228540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.405+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3144018520 tx=0x7f3144014000 comp rx=0 tx=0).stop 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2077758840 shutdown_connections 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e8228540 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e821ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2077758840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e8218d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2077758840 >> 172.21.15.46:0/2077758840 conn(0x5559e817c210 msgr2=0x5559e8215210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2077758840 shutdown_connections 2024-05-08T23:24:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.406+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2077758840 wait complete. 2024-05-08T23:24:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.407+0000 7f3167cfbb80 1 -- start start 2024-05-08T23:24:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.407+0000 7f3167cfbb80 1 Processor -- start 2024-05-08T23:24:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.408+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e822d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.408+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e822d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3167cfbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e822db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3151800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e822d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e828d750 con 0x5559e8223e00 2024-05-08T23:24:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e828d8c0 con 0x5559e821e890 2024-05-08T23:24:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3167cfbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e828da80 con 0x5559e8224720 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3167cfbb80 10 client.0 _ll_register_callbacks cb 0x5559e8113b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3152200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e822db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3151800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e822d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58306/0 (socket says 172.21.15.46:58306) 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3151800700 1 -- 172.21.15.46:0/2642624341 learned_addr learned my addr 172.21.15.46:0/2642624341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.409+0000 7f3150e00700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e822d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 -- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 msgr2=0x5559e822d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e822d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 -- 172.21.15.46:0/2642624341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 msgr2=0x5559e822d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e822d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 -- 172.21.15.46:0/2642624341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314400cd40 con 0x5559e8224720 2024-05-08T23:24:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3150e00700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e822d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f312f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-08T23:24:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f3152200700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e822db70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f31440184f0 tx=0x7f3144009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72551d169aa93b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.410+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144022830 con 0x5559e8224720 2024-05-08T23:24:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.411+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f313c01e040 con 0x5559e8224720 2024-05-08T23:24:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.411+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f3144022990 con 0x5559e8224720 2024-05-08T23:24:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.411+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3144022830 con 0x5559e8224720 2024-05-08T23:24:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.411+0000 7f3167cfbb80 10 client.30787 fetch_fsmap learned FSMap version 36 2024-05-08T23:24:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.411+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5559e7ff3cd0 con 0x5559e8224720 2024-05-08T23:24:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.412+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c01e040 con 0x5559e8224720 2024-05-08T23:24:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.412+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3144040420 con 0x5559e8224720 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 10 client.30787 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 10 client.30787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 20 client.30787 populate_metadata read hostname 'smithi046' 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e828f0b0 0x5559e82914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 4 client.30787 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.413+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 --> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559e8166ec0 con 0x5559e828f0b0 2024-05-08T23:24:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.414+0000 7f3151800700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e828f0b0 0x5559e82914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.415+0000 7f3151800700 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e828f0b0 0x5559e82914d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f86556e9ac2dc2 server_cookie=f3f0cf563db8fedd in_seq=0 out_seq=0 2024-05-08T23:24:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.415+0000 7f312f400700 10 client.30787 ms_handle_connect on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.416+0000 7f312f400700 1 -- 172.21.15.46:0/2642624341 <== mds.0 v2:172.21.15.53:6838/3374696949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559e8166ec0 con 0x5559e828f0b0 2024-05-08T23:24:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.416+0000 7f312f400700 10 client.30787 handle_command_reply: tid=0 2024-05-08T23:24:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.416+0000 7f3167cfbb80 1 client.30787 shutdown 2024-05-08T23:24:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 msgr2=0x5559e822db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e822db70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f31440184f0 tx=0x7f3144009890 comp rx=0 tx=0).stop 2024-05-08T23:24:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 shutdown_connections 2024-05-08T23:24:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559e8224720 0x5559e822db70 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559e821e890 0x5559e822d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.53:6838/3374696949,v1:172.21.15.53:6839/3374696949] conn(0x5559e828f0b0 0x5559e82914d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 --2- 172.21.15.46:0/2642624341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559e8223e00 0x5559e822d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 >> 172.21.15.46:0/2642624341 conn(0x5559e817c210 msgr2=0x5559e8226530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f312f400700 0 client.30787 ms_handle_reset on v2:172.21.15.53:6838/3374696949 2024-05-08T23:24:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.417+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 shutdown_connections 2024-05-08T23:24:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.418+0000 7f3167cfbb80 1 -- 172.21.15.46:0/2642624341 wait complete. 2024-05-08T23:24:09.419 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-08T23:24:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.419+0000 7f3167cfbb80 20 client.30787 trim_cache size 0 max 16384 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 msgr2=0x7f31340a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 0x7f31340a3850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f314400c610 tx=0x7f314400e040 comp rx=0 tx=0).stop 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 msgr2=0x7f314c1a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 0x7f314c1a67c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f314000d750 tx=0x7f314000db10 comp rx=0 tx=0).stop 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 shutdown_connections 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f31340a1390 0x7f31340a3850 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314c11a780 0x7f314c1aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c108030 0x7f314c1a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 --2- 172.21.15.46:0/3857243188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f314c10f5e0 0x7f314c1a67c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 >> 172.21.15.46:0/3857243188 conn(0x7f314c06bf30 msgr2=0x7f314c118770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 shutdown_connections 2024-05-08T23:24:09.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:09.420+0000 7f3154200700 1 -- 172.21.15.46:0/3857243188 wait complete. 2024-05-08T23:24:09.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:09 smithi046 ceph-mon[36898]: pgmap v640: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:09.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:09 smithi086 ceph-mon[78610]: pgmap v640: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:10.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:09 smithi053 ceph-mon[77587]: pgmap v640: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 445 op/s 2024-05-08T23:24:10.451 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-08T23:24:10.451 INFO:tasks.fwd_scrub:done joining 2024-05-08T23:24:10.451 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T23:24:10.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 -- 172.21.15.46:0/3408908157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4073cf0 msgr2=0x7ff2d410d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/3408908157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4073cf0 0x7ff2d410d100 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009990 tx=0x7ff2c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 -- 172.21.15.46:0/3408908157 shutdown_connections 2024-05-08T23:24:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/3408908157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d410d640 0x7ff2d410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/3408908157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d40733d0 0x7ff2d40737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/3408908157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4073cf0 0x7ff2d410d100 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 -- 172.21.15.46:0/3408908157 >> 172.21.15.46:0/3408908157 conn(0x7ff2d40fc930 msgr2=0x7ff2d40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 -- 172.21.15.46:0/3408908157 shutdown_connections 2024-05-08T23:24:10.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.668+0000 7ff2dbc00700 1 -- 172.21.15.46:0/3408908157 wait complete. 2024-05-08T23:24:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 Processor -- start 2024-05-08T23:24:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 -- start start 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 0x7ff2d4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d4073cf0 0x7ff2d4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.669+0000 7ff2dbc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419e470 con 0x7ff2d410d640 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2dbc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419e5e0 con 0x7ff2d40733d0 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2dbc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419e7a0 con 0x7ff2d4073cf0 2024-05-08T23:24:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 0x7ff2d4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49028/0 (socket says 172.21.15.46:49028) 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9c00700 1 -- 172.21.15.46:0/2177601372 learned_addr learned my addr 172.21.15.46:0/2177601372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d3e00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d4073cf0 0x7ff2d4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 -- 172.21.15.46:0/2177601372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d4073cf0 msgr2=0x7ff2d4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d4073cf0 0x7ff2d4199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 -- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 msgr2=0x7ff2d419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9200700 1 -- 172.21.15.46:0/2177601372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c00095f0 con 0x7ff2d40733d0 2024-05-08T23:24:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.670+0000 7ff2d9c00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2d9200700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 0x7ff2d4199550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d950 tx=0x7ff2c800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800f890 con 0x7ff2d40733d0 2024-05-08T23:24:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d419ea80 con 0x7ff2d40733d0 2024-05-08T23:24:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c8010460 con 0x7ff2d40733d0 2024-05-08T23:24:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800e610 con 0x7ff2d40733d0 2024-05-08T23:24:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.671+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41a62c0 con 0x7ff2d40733d0 2024-05-08T23:24:10.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.672+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4108850 con 0x7ff2d40733d0 2024-05-08T23:24:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.674+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff2c80105d0 con 0x7ff2d40733d0 2024-05-08T23:24:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.675+0000 7ff2d1600700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 0x7ff2bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.675+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff2c80ccde0 con 0x7ff2d40733d0 2024-05-08T23:24:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.675+0000 7ff2d3e00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 0x7ff2bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.676+0000 7ff2d3e00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 0x7ff2bc0a3750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2d419aaa0 tx=0x7ff2c000b540 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.680+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff2c8095d70 con 0x7ff2d40733d0 2024-05-08T23:24:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.905+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2d4074a80 con 0x7ff2d40733d0 2024-05-08T23:24:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.907+0000 7ff2d1600700 1 -- 172.21.15.46:0/2177601372 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25387 (secure 0 0 0) 0x7ff2c8016090 con 0x7ff2d40733d0 2024-05-08T23:24:10.907 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:10.908 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":114,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:24:05.500123+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 msgr2=0x7ff2bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 0x7ff2bc0a3750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff2d419aaa0 tx=0x7ff2c000b540 comp rx=0 tx=0).stop 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 msgr2=0x7ff2d4199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 0x7ff2d4199550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d950 tx=0x7ff2c800dc60 comp rx=0 tx=0).stop 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 shutdown_connections 2024-05-08T23:24:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff2bc0a1290 0x7ff2bc0a3750 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d410d640 0x7ff2d419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d40733d0 0x7ff2d4199550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 --2- 172.21.15.46:0/2177601372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2d4073cf0 0x7ff2d4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 >> 172.21.15.46:0/2177601372 conn(0x7ff2d40fc930 msgr2=0x7ff2d40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 shutdown_connections 2024-05-08T23:24:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:10.910+0000 7ff2dbc00700 1 -- 172.21.15.46:0/2177601372 wait complete. 2024-05-08T23:24:10.929 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:24:10.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.215+0000 7f64a3c00700 1 -- 172.21.15.46:0/852512605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c0ffd90 msgr2=0x7f649c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.215+0000 7f64a3c00700 1 --2- 172.21.15.46:0/852512605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c0ffd90 0x7f649c100170 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6490009990 tx=0x7f6490009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.217+0000 7f64a3c00700 1 -- 172.21.15.46:0/852512605 shutdown_connections 2024-05-08T23:24:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.217+0000 7f64a3c00700 1 --2- 172.21.15.46:0/852512605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649c105190 0x7f649c105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.217+0000 7f64a3c00700 1 --2- 172.21.15.46:0/852512605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c0ffd90 0x7f649c100170 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.217+0000 7f64a3c00700 1 --2- 172.21.15.46:0/852512605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c1006b0 0x7f649c104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.217+0000 7f64a3c00700 1 -- 172.21.15.46:0/852512605 >> 172.21.15.46:0/852512605 conn(0x7f649c0fb760 msgr2=0x7f649c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.218+0000 7f64a3c00700 1 -- 172.21.15.46:0/852512605 shutdown_connections 2024-05-08T23:24:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.219+0000 7f64a3c00700 1 -- 172.21.15.46:0/852512605 wait complete. 2024-05-08T23:24:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.220+0000 7f64a3c00700 1 Processor -- start 2024-05-08T23:24:11.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 -- start start 2024-05-08T23:24:11.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649c0ffd90 0x7f649c072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c1006b0 0x7f649c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06be80 con 0x7f649c1006b0 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06bff0 con 0x7f649c105190 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a3c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06c1b0 con 0x7f649c0ffd90 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a1c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58356/0 (socket says 172.21.15.46:58356) 2024-05-08T23:24:11.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.221+0000 7f64a1c00700 1 -- 172.21.15.46:0/1714390927 learned_addr learned my addr 172.21.15.46:0/1714390927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.222+0000 7f64a1c00700 1 -- 172.21.15.46:0/1714390927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649c0ffd90 msgr2=0x7f649c072560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:24:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.222+0000 7f64a1c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649c0ffd90 0x7f649c072560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.222+0000 7f64a1c00700 1 -- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c1006b0 msgr2=0x7f649c072ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:24:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.222+0000 7f64a1c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c1006b0 0x7f649c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.222+0000 7f64a1c00700 1 -- 172.21.15.46:0/1714390927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900095f0 con 0x7f649c105190 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.223+0000 7f64a1c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f649400ef30 tx=0x7f649400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.224+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494018070 con 0x7f649c105190 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.224+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c06c430 con 0x7f649c105190 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.224+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f649400f040 con 0x7f649c105190 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.224+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494003dc0 con 0x7f649c105190 2024-05-08T23:24:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.224+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c070410 con 0x7f649c105190 2024-05-08T23:24:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.226+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c068a40 con 0x7f649c105190 2024-05-08T23:24:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.230+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f649400caa0 con 0x7f649c105190 2024-05-08T23:24:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.232+0000 7f6499600700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 0x7f64840a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.232+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f64940ccd90 con 0x7f649c105190 2024-05-08T23:24:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.233+0000 7f64a1200700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 0x7f64840a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.235+0000 7f64a1200700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 0x7f64840a3850 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6490009350 tx=0x7f6490005fb0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.237+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6494095db0 con 0x7f649c105190 2024-05-08T23:24:11.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.477+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f649c06cb10 con 0x7f649c105190 2024-05-08T23:24:11.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.477+0000 7f6499600700 1 -- 172.21.15.46:0/1714390927 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64940993d0 con 0x7f649c105190 2024-05-08T23:24:11.477 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:24:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 msgr2=0x7f64840a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 0x7f64840a3850 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6490009350 tx=0x7f6490005fb0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 msgr2=0x7f649c06b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f649400ef30 tx=0x7f649400c450 comp rx=0 tx=0).stop 2024-05-08T23:24:11.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 shutdown_connections 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f649c105190 0x7f649c06b6b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649c0ffd90 0x7f649c072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f64840a1390 0x7f64840a3850 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 --2- 172.21.15.46:0/1714390927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c1006b0 0x7f649c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 >> 172.21.15.46:0/1714390927 conn(0x7f649c0fb760 msgr2=0x7f649c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 shutdown_connections 2024-05-08T23:24:11.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.480+0000 7f64a3c00700 1 -- 172.21.15.46:0/1714390927 wait complete. 2024-05-08T23:24:11.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:24:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 -- 172.21.15.46:0/838649309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 msgr2=0x7f22b4101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 --2- 172.21.15.46:0/838649309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b4101460 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009990 tx=0x7f22a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 -- 172.21.15.46:0/838649309 shutdown_connections 2024-05-08T23:24:11.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 --2- 172.21.15.46:0/838649309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b41061a0 0x7f22b410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 --2- 172.21.15.46:0/838649309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b4101460 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 --2- 172.21.15.46:0/838649309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b4101a30 0x7f22b4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 -- 172.21.15.46:0/838649309 >> 172.21.15.46:0/838649309 conn(0x7f22b40fc8e0 msgr2=0x7f22b40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 -- 172.21.15.46:0/838649309 shutdown_connections 2024-05-08T23:24:11.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.704+0000 7f22bb600700 1 -- 172.21.15.46:0/838649309 wait complete. 2024-05-08T23:24:11.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.705+0000 7f22bb600700 1 Processor -- start 2024-05-08T23:24:11.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.705+0000 7f22bb600700 1 -- start start 2024-05-08T23:24:11.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b419bbd0 0x7f22b419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419c650 con 0x7f22b41061a0 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419c7c0 con 0x7f22b4101080 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22bb600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b419c980 con 0x7f22b419bbd0 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49058/0 (socket says 172.21.15.46:49058) 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b3e00700 1 -- 172.21.15.46:0/1197351775 learned_addr learned my addr 172.21.15.46:0/1197351775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58376/0 (socket says 172.21.15.46:58376) 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b9600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b419bbd0 0x7f22b419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 -- 172.21.15.46:0/1197351775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b419bbd0 msgr2=0x7f22b419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b419bbd0 0x7f22b419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 -- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 msgr2=0x7f22b4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b8c00700 1 -- 172.21.15.46:0/1197351775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80095f0 con 0x7f22b4101080 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.706+0000 7f22b3e00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22b8c00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f22a80098e0 tx=0x7f22a8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a801d070 con 0x7f22b4101080 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b41a3e80 con 0x7f22b4101080 2024-05-08T23:24:11.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22a8022470 con 0x7f22b4101080 2024-05-08T23:24:11.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8017650 con 0x7f22b4101080 2024-05-08T23:24:11.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.707+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b41a42a0 con 0x7f22b4101080 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.708+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b41093c0 con 0x7f22b4101080 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.710+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f22a80225e0 con 0x7f22b4101080 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.711+0000 7f22b1600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 0x7f229c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.711+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7f22a80ce420 con 0x7f22b4101080 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.712+0000 7f22b3e00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 0x7f229c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:11.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.712+0000 7f22b3e00700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 0x7f229c0a3750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22b4198840 tx=0x7f22a0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:11.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.716+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22a80975c0 con 0x7f22b4101080 2024-05-08T23:24:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:11 smithi046 ceph-mon[36898]: pgmap v641: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 470 op/s 2024-05-08T23:24:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2177601372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:11.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:11 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1714390927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.932+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22b4101a30 con 0x7f22b4101080 2024-05-08T23:24:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.933+0000 7f22b1600700 1 -- 172.21.15.46:0/1197351775 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22a8027020 con 0x7f22b4101080 2024-05-08T23:24:11.932 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:24:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 msgr2=0x7f229c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 0x7f229c0a3750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22b4198840 tx=0x7f22a0009260 comp rx=0 tx=0).stop 2024-05-08T23:24:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 msgr2=0x7f22b41972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f22a80098e0 tx=0x7f22a8005740 comp rx=0 tx=0).stop 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 shutdown_connections 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b419bbd0 0x7f22b419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22b4101080 0x7f22b41972f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f229c0a1290 0x7f229c0a3750 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 --2- 172.21.15.46:0/1197351775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b41061a0 0x7f22b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 >> 172.21.15.46:0/1197351775 conn(0x7f22b40fc8e0 msgr2=0x7f22b40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 shutdown_connections 2024-05-08T23:24:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:11.935+0000 7f22bb600700 1 -- 172.21.15.46:0/1197351775 wait complete. 2024-05-08T23:24:11.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:24:11.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:11 smithi086 ceph-mon[78610]: pgmap v641: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 470 op/s 2024-05-08T23:24:11.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2177601372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:11.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:11 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1714390927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:12.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:11 smithi053 ceph-mon[77587]: pgmap v641: 129 pgs: 3 active+clean+snaptrim, 119 active+clean, 7 active+clean+snaptrim_wait; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 470 op/s 2024-05-08T23:24:12.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2177601372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:12.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:11 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1714390927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:12.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 -- 172.21.15.46:0/2909835232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc1061a0 msgr2=0x7fafdc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 --2- 172.21.15.46:0/2909835232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc1061a0 0x7fafdc10dd20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fafd400b0a0 tx=0x7fafd400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 -- 172.21.15.46:0/2909835232 shutdown_connections 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 --2- 172.21.15.46:0/2909835232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc1061a0 0x7fafdc10dd20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 --2- 172.21.15.46:0/2909835232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc101080 0x7fafdc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 --2- 172.21.15.46:0/2909835232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc101a30 0x7fafdc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 -- 172.21.15.46:0/2909835232 >> 172.21.15.46:0/2909835232 conn(0x7fafdc0fc8e0 msgr2=0x7fafdc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 -- 172.21.15.46:0/2909835232 shutdown_connections 2024-05-08T23:24:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.159+0000 7fafe3e00700 1 -- 172.21.15.46:0/2909835232 wait complete. 2024-05-08T23:24:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.160+0000 7fafe3e00700 1 Processor -- start 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 -- start start 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc1061a0 0x7fafdc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc19df30 0x7fafdc19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19e9b0 con 0x7fafdc101080 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19eb20 con 0x7fafdc1061a0 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19ece0 con 0x7fafdc19df30 2024-05-08T23:24:12.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe1400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60382/0 (socket says 172.21.15.46:60382) 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe1400700 1 -- 172.21.15.46:0/3121938796 learned_addr learned my addr 172.21.15.46:0/3121938796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe0a00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc1061a0 0x7fafdc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.161+0000 7fafe1e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc19df30 0x7fafdc19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 -- 172.21.15.46:0/3121938796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc19df30 msgr2=0x7fafdc19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc19df30 0x7fafdc19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 -- 172.21.15.46:0/3121938796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc1061a0 msgr2=0x7fafdc199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc1061a0 0x7fafdc199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 -- 172.21.15.46:0/3121938796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009d00 con 0x7fafdc101080 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe1400700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fafd000e9a0 tx=0x7fafd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0009790 con 0x7fafdc101080 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd0010430 con 0x7fafdc101080 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018680 con 0x7fafdc101080 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.162+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc1a6240 con 0x7fafdc101080 2024-05-08T23:24:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.163+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc1a66d0 con 0x7fafdc101080 2024-05-08T23:24:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.164+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc102550 con 0x7fafdc101080 2024-05-08T23:24:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.165+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fafd00105a0 con 0x7fafdc101080 2024-05-08T23:24:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.168+0000 7fafda000700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 0x7fafc40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.168+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fafd0014070 con 0x7fafdc101080 2024-05-08T23:24:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.168+0000 7fafe0a00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 0x7fafc40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.168+0000 7fafe0a00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 0x7fafc40a37d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fafdc19aba0 tx=0x7fafc8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.172+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fafd0052d60 con 0x7fafdc101080 2024-05-08T23:24:12.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.393+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafdc067d10 con 0x7fafdc101080 2024-05-08T23:24:12.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.394+0000 7fafda000700 1 -- 172.21.15.46:0/3121938796 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafdc067d10 con 0x7fafdc101080 2024-05-08T23:24:12.393 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 msgr2=0x7fafc40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 0x7fafc40a37d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fafdc19aba0 tx=0x7fafc8009260 comp rx=0 tx=0).stop 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 msgr2=0x7fafdc1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fafd000e9a0 tx=0x7fafd000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 shutdown_connections 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc19df30 0x7fafdc19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fafc40a1310 0x7fafc40a37d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafdc101080 0x7fafdc1996a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 --2- 172.21.15.46:0/3121938796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafdc1061a0 0x7fafdc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 >> 172.21.15.46:0/3121938796 conn(0x7fafdc0fc8e0 msgr2=0x7fafdc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 shutdown_connections 2024-05-08T23:24:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.396+0000 7fafe3e00700 1 -- 172.21.15.46:0/3121938796 wait complete. 2024-05-08T23:24:12.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:24:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.622+0000 7fb6fac00700 1 -- 172.21.15.46:0/585847745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f41061a0 msgr2=0x7fb6f410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.622+0000 7fb6fac00700 1 --2- 172.21.15.46:0/585847745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f41061a0 0x7fb6f410dd20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00b0a0 tx=0x7fb6ec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 -- 172.21.15.46:0/585847745 shutdown_connections 2024-05-08T23:24:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 --2- 172.21.15.46:0/585847745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f41061a0 0x7fb6f410dd20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 --2- 172.21.15.46:0/585847745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f4101080 0x7fb6f4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 --2- 172.21.15.46:0/585847745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f4101a30 0x7fb6f4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 -- 172.21.15.46:0/585847745 >> 172.21.15.46:0/585847745 conn(0x7fb6f40fc8e0 msgr2=0x7fb6f40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 -- 172.21.15.46:0/585847745 shutdown_connections 2024-05-08T23:24:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.623+0000 7fb6fac00700 1 -- 172.21.15.46:0/585847745 wait complete. 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 Processor -- start 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 -- start start 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 0x7fb6f4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 0x7fb6f419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419c770 con 0x7fb6f4101080 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419c8e0 con 0x7fb6f41061a0 2024-05-08T23:24:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.624+0000 7fb6fac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419caa0 con 0x7fb6f419bcf0 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60402/0 (socket says 172.21.15.46:60402) 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 -- 172.21.15.46:0/3450465105 learned_addr learned my addr 172.21.15.46:0/3450465105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3400700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 0x7fb6f4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f8c00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 0x7fb6f419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 -- 172.21.15.46:0/3450465105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 msgr2=0x7fb6f419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 0x7fb6f419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 -- 172.21.15.46:0/3450465105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 msgr2=0x7fb6f4197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 0x7fb6f4197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 -- 172.21.15.46:0/3450465105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec009d00 con 0x7fb6f4101080 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f8c00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 0x7fb6f419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3400700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 0x7fb6f4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.625+0000 7fb6f3e00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400e9c0 tx=0x7fb6e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.626+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40097b0 con 0x7fb6f4101080 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.626+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e400ee60 con 0x7fb6f4101080 2024-05-08T23:24:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.626+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40054f0 con 0x7fb6f4101080 2024-05-08T23:24:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.626+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f41a3fa0 con 0x7fb6f4101080 2024-05-08T23:24:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.626+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f41a4430 con 0x7fb6f4101080 2024-05-08T23:24:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.627+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f4109470 con 0x7fb6f4101080 2024-05-08T23:24:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.629+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb6e4004750 con 0x7fb6f4101080 2024-05-08T23:24:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.630+0000 7fb6f0c00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 0x7fb6d80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.630+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7fb6e4014070 con 0x7fb6f4101080 2024-05-08T23:24:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.630+0000 7fb6f3400700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 0x7fb6d80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.631+0000 7fb6f3400700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 0x7fb6d80a37d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4198960 tx=0x7fb6e0009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.634+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6e40962d0 con 0x7fb6f4101080 2024-05-08T23:24:12.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1197351775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:12.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:12 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3121938796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:12.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.855+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb6f41987b0 con 0x7fb6f4101080 2024-05-08T23:24:12.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.856+0000 7fb6f0c00700 1 -- 172.21.15.46:0/3450465105 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v114) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb6e40998f0 con 0x7fb6f4101080 2024-05-08T23:24:12.855 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:24:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 msgr2=0x7fb6d80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 0x7fb6d80a37d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4198960 tx=0x7fb6e0009260 comp rx=0 tx=0).stop 2024-05-08T23:24:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 msgr2=0x7fb6f4197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400e9c0 tx=0x7fb6e400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 shutdown_connections 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f419bcf0 0x7fb6f419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fb6d80a1310 0x7fb6d80a37d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f4101080 0x7fb6f4197410 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 --2- 172.21.15.46:0/3450465105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f41061a0 0x7fb6f4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.858+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 >> 172.21.15.46:0/3450465105 conn(0x7fb6f40fc8e0 msgr2=0x7fb6f40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.859+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 shutdown_connections 2024-05-08T23:24:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:12.859+0000 7fb6fac00700 1 -- 172.21.15.46:0/3450465105 wait complete. 2024-05-08T23:24:12.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T23:24:12.963 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1197351775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:12.964 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:12 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3121938796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:13.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1197351775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:13.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:12 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3121938796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 -- 172.21.15.46:0/1736341434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 msgr2=0x7ff83c0737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 --2- 172.21.15.46:0/1736341434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c0737b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009990 tx=0x7ff82c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 -- 172.21.15.46:0/1736341434 shutdown_connections 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 --2- 172.21.15.46:0/1736341434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c10d640 0x7ff83c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 --2- 172.21.15.46:0/1736341434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c0737b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 --2- 172.21.15.46:0/1736341434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c073cf0 0x7ff83c10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 -- 172.21.15.46:0/1736341434 >> 172.21.15.46:0/1736341434 conn(0x7ff83c0fc930 msgr2=0x7ff83c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 -- 172.21.15.46:0/1736341434 shutdown_connections 2024-05-08T23:24:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.094+0000 7ff841400700 1 -- 172.21.15.46:0/1736341434 wait complete. 2024-05-08T23:24:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.095+0000 7ff841400700 1 Processor -- start 2024-05-08T23:24:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.095+0000 7ff841400700 1 -- start start 2024-05-08T23:24:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c073cf0 0x7ff83c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c10d640 0x7ff83c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e450 con 0x7ff83c0733d0 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e5c0 con 0x7ff83c073cf0 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff841400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e780 con 0x7ff83c10d640 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60430/0 (socket says 172.21.15.46:60430) 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 -- 172.21.15.46:0/2886381649 learned_addr learned my addr 172.21.15.46:0/2886381649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff839e00700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c073cf0 0x7ff83c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83b200700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c10d640 0x7ff83c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 -- 172.21.15.46:0/2886381649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c10d640 msgr2=0x7ff83c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c10d640 0x7ff83c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 -- 172.21.15.46:0/2886381649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c073cf0 msgr2=0x7ff83c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c073cf0 0x7ff83c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.096+0000 7ff83a800700 1 -- 172.21.15.46:0/2886381649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0095f0 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff83a800700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c01d070 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82c022470 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c19ea00 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c017650 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.097+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c1a62a0 con 0x7ff83c0733d0 2024-05-08T23:24:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.098+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c108850 con 0x7ff83c0733d0 2024-05-08T23:24:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.100+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff82c0225e0 con 0x7ff83c0733d0 2024-05-08T23:24:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.101+0000 7ff833400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.101+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9642+0+0 (secure 0 0 0) 0x7ff82c0ce480 con 0x7ff83c0733d0 2024-05-08T23:24:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.102+0000 7ff839e00700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.102+0000 7ff839e00700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 0x7ff8200a37d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff83c19aa80 tx=0x7ff828009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.105+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff82c0975c0 con 0x7ff83c0733d0 2024-05-08T23:24:13.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.394+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7ff83c074a80 con 0x7ff83c0733d0 2024-05-08T23:24:13.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.406+0000 7ff833400700 1 -- 172.21.15.46:0/2886381649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733938 (secure 0 0 0) 0x7ff82c027090 con 0x7ff83c0733d0 2024-05-08T23:24:13.416 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 26, 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 15501, 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:13.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 652597198 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-05-08T23:24:13.418 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 652597198 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/652597198", 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-05-08T23:03:48.069923+0000", 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-05-08T23:24:13.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 25490, 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540138322906710015, 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:13.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:24:13.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.423 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.424 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.427 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.431 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-08T23:24:13.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-08T23:24:13.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-08T23:24:13.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.450 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-08T23:24:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:24:13.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:13.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.464 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.467 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-08T23:24:13.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-08T23:24:13.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-08T23:24:13.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:24:13.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.482 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:24:13.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-08T23:24:13.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:24:13.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-08T23:24:13.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:24:13.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:24:13.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-08T23:24:13.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.508 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-08T23:24:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.515 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:24:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:24:13.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-08T23:24:13.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.530 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-08T23:24:13.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:24:13.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:24:13.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:24:13.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.556 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.557 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-08T23:24:13.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-08T23:24:13.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-08T23:24:13.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.571 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:24:13.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.579 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.583 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.584 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.587 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.590 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.593 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.594 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.599 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:24:13.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:24:13.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:24:13.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:24:13.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:24:13.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:24:13.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.630 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:24:13.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.643 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:24:13.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.651 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:24:13.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.655 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:24:13.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:24:13.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.665 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-08T23:24:13.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.678 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.682 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:13.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-08T23:24:13.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.698 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.708 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:13.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.717 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:13.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.724 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.727 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:13.728 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:13.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.731 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:13.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:13.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:13.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-08T23:24:14.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.012 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.016 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-08T23:24:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-08T23:24:14.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.032 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.040 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.042 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.047 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.047 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.047 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:24:14.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:24:14.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.060 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:24:14.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.069 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:24:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-08T23:24:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-08T23:24:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-08T23:24:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-08T23:24:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-08T23:24:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:24:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-08T23:24:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-08T23:24:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-08T23:24:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-08T23:24:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-08T23:24:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-08T23:24:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-08T23:24:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-08T23:24:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-08T23:24:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-08T23:24:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-08T23:24:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:24:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-08T23:24:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-08T23:24:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-08T23:24:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 25498, 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540138322906710015, 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-08T23:24:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-08T23:24:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-08T23:24:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-08T23:24:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-08T23:24:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:24:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-08T23:24:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-08T23:24:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:24:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-08T23:24:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-08T23:24:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-08T23:24:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-08T23:24:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:24:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:24:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:24:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:24:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:24:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-08T23:24:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-08T23:24:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-08T23:24:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:24:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:24:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-08T23:24:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-08T23:24:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:24:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-08T23:24:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-08T23:24:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-08T23:24:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:24:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:24:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:24:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:24:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:24:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:24:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:24:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-08T23:24:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:24:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:24:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-08T23:24:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-08T23:24:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-08T23:24:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-08T23:24:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-08T23:24:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-08T23:24:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-08T23:24:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-08T23:24:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:24:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:24:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-08T23:24:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:24:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-08T23:24:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:24:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-08T23:24:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-08T23:24:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-08T23:24:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-08T23:24:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-08T23:24:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-08T23:24:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-08T23:24:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-08T23:24:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-08T23:24:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-08T23:24:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:24:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-08T23:24:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-08T23:24:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-08T23:24:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:24:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-08T23:24:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-08T23:24:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-05-08T23:24:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "restful", 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_schedule" 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-08T23:24:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:24:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-08T23:24:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:24:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-08T23:24:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:24:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-08T23:24:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-08T23:24:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:24:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-08T23:24:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-08T23:24:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:24:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-08T23:24:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:24:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:24:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-08T23:24:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-08T23:24:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:24:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-08T23:24:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-08T23:24:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-08T23:24:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-08T23:24:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-08T23:24:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-08T23:24:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:24:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:24:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:24:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:24:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:24:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:24:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:24:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:24:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:24:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:24:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:24:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-08T23:24:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-08T23:24:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-08T23:24:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-08T23:24:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-08T23:24:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-08T23:24:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:24:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:24:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-08T23:24:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-08T23:24:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-08T23:24:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:24:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-08T23:24:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-08T23:24:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:24:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-08T23:24:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:24:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-08T23:24:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:24:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.490 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.491 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.494 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.495 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.500 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.501 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.504 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.505 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.510 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.513 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.514 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.517 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.518 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.521 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.525 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.530 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.531 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.534 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.535 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.538 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.539 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.543 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.547 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-08T23:24:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:15.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.558 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.569 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.573 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-08T23:24:15.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-08T23:24:15.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-08T23:24:15.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.587 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-08T23:24:15.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.591 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.595 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.596 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-08T23:24:15.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.603 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-08T23:24:15.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.608 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.612 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.616 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-08T23:24:15.618 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.620 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.627 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.639 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-08T23:24:15.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-08T23:24:15.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.670 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.674 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.679 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.683 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-08T23:24:15.686 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.688 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-08T23:24:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:24:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-08T23:24:15.710 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-08T23:24:15.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:24:15.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-08T23:24:15.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-08T23:24:15.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-08T23:24:15.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:24:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-08T23:24:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-08T23:24:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-08T23:24:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-08T23:24:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-08T23:24:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-08T23:24:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-08T23:24:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-08T23:24:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-05-08T23:24:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-08T23:24:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-08T23:24:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-08T23:24:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 87, 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1020499179 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2225122269 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2078849806 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-08T23:24:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3820328378 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3227763681 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 445521884 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:24:15.799 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:13 smithi053 ceph-mon[77587]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 410 op/s 2024-05-08T23:24:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3450465105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:15.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:13 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2886381649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:24:15.801 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:13 smithi086 ceph-mon[78610]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 410 op/s 2024-05-08T23:24:15.801 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3450465105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:15.801 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:13 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2886381649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 msgr2=0x7ff8200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 0x7ff8200a37d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff83c19aa80 tx=0x7ff828009260 comp rx=0 tx=0).stop 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 msgr2=0x7ff83c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c005740 comp rx=0 tx=0).stop 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 shutdown_connections 2024-05-08T23:24:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff83c10d640 0x7ff83c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7ff8200a1310 0x7ff8200a37d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c0733d0 0x7ff83c199580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 --2- 172.21.15.46:0/2886381649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff83c073cf0 0x7ff83c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 >> 172.21.15.46:0/2886381649 conn(0x7ff83c0fc930 msgr2=0x7ff83c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 shutdown_connections 2024-05-08T23:24:15.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:13.414+0000 7ff841400700 1 -- 172.21.15.46:0/2886381649 wait complete. 2024-05-08T23:24:15.809 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T23:24:15.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T23:24:15.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:13 smithi046 ceph-mon[36898]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 410 op/s 2024-05-08T23:24:15.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3450465105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:15.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:13 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2886381649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:24:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:15 smithi086 ceph-mon[78610]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 389 op/s 2024-05-08T23:24:16.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:15 smithi086 ceph-mon[78610]: osdmap e115: 12 total, 12 up, 12 in 2024-05-08T23:24:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:24:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:15 smithi053 ceph-mon[77587]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 389 op/s 2024-05-08T23:24:16.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:15 smithi053 ceph-mon[77587]: osdmap e115: 12 total, 12 up, 12 in 2024-05-08T23:24:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:15 smithi046 ceph-mon[36898]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 389 op/s 2024-05-08T23:24:16.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:15 smithi046 ceph-mon[36898]: osdmap e115: 12 total, 12 up, 12 in 2024-05-08T23:24:17.886 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.004521871, 2024-05-08T23:24:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000873201, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 275, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 200832, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.031825275, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.024276813, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.003111596, 2024-05-08T23:24:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 272, 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.037842100, 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000139125 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 200832 2024-05-08T23:24:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 375, 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 362, 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1605104, 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 206656, 2024-05-08T23:24:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.079347374, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.032290763, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.045097557, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.007965887, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 362, 2024-05-08T23:24:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.032962621, 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000091056 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1605104, 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 206656 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-08T23:24:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 150008, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-05-08T23:24:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-08T23:24:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 373, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1549672, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 373, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 373, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1549672, 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:17.906 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:18.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:17 smithi046 ceph-mon[36898]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:18.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:17 smithi086 ceph-mon[78610]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:18.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:17 smithi053 ceph-mon[77587]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:18.315 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-08T23:24:18.315 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-08T23:24:18.315 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-08T23:24:18.315 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-08T23:24:18.315 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-08T23:24:18.316 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T23:24:18.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T23:24:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1551, 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1775, 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 572431598, 2024-05-08T23:24:19.877 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1081555, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 7, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.165991923, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.067892717, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 1.033565014, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.057848220, 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:19.878 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1775, 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.140929058, 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000079396 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1709648, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 731680 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1882, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1911, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 871691692, 2024-05-08T23:24:19.880 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 698692, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 9, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.622345538, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.061829404, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 1.464941562, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.076719897, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1908, 2024-05-08T23:24:19.881 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.116661754, 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000061143 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 113696 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3105, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 3107, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 873317372, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 965719, 2024-05-08T23:24:19.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.673776816, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.122062890, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 1.411829836, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.110746898, 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:19.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3105, 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.196151692, 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000063172 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.885 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:19.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:19.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-05-08T23:24:19.887 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 9588, 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.489219050, 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000051024 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-05-08T23:24:19.888 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 0, 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 0, 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-05-08T23:24:19.889 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:19.890 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 0, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 80004, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 45654, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 34350, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 14785, 2024-05-08T23:24:19.891 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 9, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:19.892 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-05-08T23:24:19.893 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:19.894 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:19.895 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 0 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:19.898 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:19.899 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 0, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 364825, 2024-05-08T23:24:19.900 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 128, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-05-08T23:24:19.901 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 1799378895, 2024-05-08T23:24:19.902 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 2318242217, 2024-05-08T23:24:19.902 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 2318220724, 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 364825 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 3857, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 79505, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 75648, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 233, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 542, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 309, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 80004, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 80099, 2024-05-08T23:24:19.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 95, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 0, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 0, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 0, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 134168, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 207156 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 0, 2024-05-08T23:24:19.905 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 0, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 0, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.907 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-05-08T23:24:19.908 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.909 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.910 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.911 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.912 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.912 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-05-08T23:24:19.912 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.914 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-05-08T23:24:19.915 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.916 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.917 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.918 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.919 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:19.920 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.921 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-05-08T23:24:19.922 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.923 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:19.924 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-05-08T23:24:19.925 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 5, 2024-05-08T23:24:19.926 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 4, 2024-05-08T23:24:19.926 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-05-08T23:24:19.926 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-05-08T23:24:19.926 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 0, 2024-05-08T23:24:19.926 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 0, 2024-05-08T23:24:19.927 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 2859, 2024-05-08T23:24:19.927 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:19.931 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:19.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:19.933 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 406757, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 4979, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:19.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 209, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 66617455, 2024-05-08T23:24:19.935 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 434403, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-05-08T23:24:19.936 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 6111, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 21, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 3, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 6111, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6111, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 21.660781055, 2024-05-08T23:24:19.937 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003544555 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6111, 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 6112 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 6111, 2024-05-08T23:24:19.938 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 6111, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 0, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 3273, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 0, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 2836, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 0, 2024-05-08T23:24:19.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 1, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-05-08T23:24:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-05-08T23:24:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-05-08T23:24:19.943 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 115, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 31, 2024-05-08T23:24:19.944 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 11, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 11, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 0, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 2, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 0 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.945 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 0, 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-05-08T23:24:19.946 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 0, 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:19.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 6534, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 2316828041, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 6534, 2024-05-08T23:24:19.948 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6534, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 2316828041, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:19.950 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 6111, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 2314067314, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6111, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 2314067314, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.951 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:19.952 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 6111, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 6111, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6111, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 6111, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.953 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:19.954 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:19.955 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x5620a8678140": { 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:19.956 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:19.957 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:19.958 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:19.958 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:19.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:19.958 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:20.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:19 smithi086 ceph-mon[78610]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:20.216 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:19 smithi046 ceph-mon[36898]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:20.244 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-08T23:24:20.245 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-08T23:24:20.245 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-05-08T23:24:20.245 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:20.245 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:20.245 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T23:24:20.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T23:24:20.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:19 smithi053 ceph-mon[77587]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-05-08T23:24:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:24:21.778 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 159545, 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 227332, 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 48876412, 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 259352581, 2024-05-08T23:24:21.779 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 20, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 30.348665630, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 22.335698972, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 7.901781512, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 1.176877054, 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:21.780 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 227330, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 20.837965156, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000091663 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 190129, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.067824707, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000356 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.781 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 3952 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 20397, 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 24795, 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 6447005, 2024-05-08T23:24:21.782 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 100133864, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 2.908434822, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 1.115109828, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.821010600, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.831572964, 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:21.783 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 24795, 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.982862374, 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000120300 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 15225, 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.003133117, 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000205 2024-05-08T23:24:21.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1741696, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 249520 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 314892, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 210360, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 157256552, 2024-05-08T23:24:21.785 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 125096007, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 78, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 23.805494809, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 10.052271857, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 16.679891597, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.927618723, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 210357, 2024-05-08T23:24:21.786 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 12.008762056, 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000057087 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 480974, 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.079639291, 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000165 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.787 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 636976 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-05-08T23:24:21.788 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 46557, 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 40.548671968, 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000870946 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-05-08T23:24:21.789 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 27807, 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.720695629, 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000133804 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 99329, 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 92110, 2024-05-08T23:24:21.790 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 92110, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 301.280140700, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003270873 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 7219, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:21.791 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 20, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 1, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 672, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 85, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 587, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 551, 2024-05-08T23:24:21.792 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 65, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 65, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 6, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 182524, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 100163, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 7217, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 14, 2024-05-08T23:24:21.793 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 3068, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-05-08T23:24:21.794 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 3, 2024-05-08T23:24:21.795 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 97, 2024-05-08T23:24:21.795 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:21.795 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 950, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 705, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 211650580, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 6, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 950, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 90, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 90, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 860, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 12, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 7731, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 48418, 2024-05-08T23:24:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 25266, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 9881, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 4555, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 76884 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.798 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 37, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 27, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 18, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 24, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 49, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:21.799 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 14614, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 14614, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-05-08T23:24:21.800 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 1, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 1, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.801 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 73687, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 73687, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 99, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-05-08T23:24:21.802 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 99, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 275852230, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-05-08T23:24:21.803 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 23224, 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 106.483249797, 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004585052 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 495, 2024-05-08T23:24:21.804 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 15109, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 14614, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 115, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 124, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 9, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 672, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 22785, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 22113, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 65, 2024-05-08T23:24:21.805 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 15764, 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 15699, 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 70296, 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 198964 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 154854, 2024-05-08T23:24:21.806 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 11, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 99329, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 2869, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 9006, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 14607, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 103.406539679, 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.007079245 2024-05-08T23:24:21.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5244, 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 36.277162967, 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006917841 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.808 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-05-08T23:24:21.809 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 24105, 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 28.818702253, 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001195548 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.811 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:21.812 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-05-08T23:24:21.814 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.002377987, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000475597 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 28277, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 56.190111975, 2024-05-08T23:24:21.816 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001987131 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3439, 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 22.788422763, 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.006626467 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:21.817 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.818 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-05-08T23:24:21.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 135, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.797634042, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.020723215 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.821 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1684, 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.516627102, 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000900609 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-05-08T23:24:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 14614, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 49.482561932, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003385969 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 2, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 1, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 14870, 2024-05-08T23:24:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 14870, 2024-05-08T23:24:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 2861, 2024-05-08T23:24:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:21.835 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 419708, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 445, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 264, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 3, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:21.836 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 209, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:21.837 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 2053565, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 16912, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.838 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 55568, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 274396030, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 27, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 55568, 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-05-08T23:24:21.839 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 55568, 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 395.590017155, 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.007119025 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 55568, 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 56034 2024-05-08T23:24:21.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 55568, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 1658, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 53910, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 10, 2024-05-08T23:24:21.841 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 708, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 23932, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 415, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-05-08T23:24:21.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 1, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 29304, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 950, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 20, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:21.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:21.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-05-08T23:24:21.845 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-05-08T23:24:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 115, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 33, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 12, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-05-08T23:24:21.847 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 480, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 214 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-05-08T23:24:21.848 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 2 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-05-08T23:24:21.849 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 5, 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 14614, 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:21.850 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 494830, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 170213683, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 494830, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 494830, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 170213683, 2024-05-08T23:24:21.851 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:21.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 55568, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 275990124, 2024-05-08T23:24:21.853 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 55568, 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 275990124, 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.854 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:21.855 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 55568, 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 55568, 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 55568, 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 55568, 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.856 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:21.857 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 14614, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1594094, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 14614, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 708, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1594094, 2024-05-08T23:24:21.858 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x559c272ca000": { 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:21.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 73687, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 271657926, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 73687, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 23224, 2024-05-08T23:24:21.860 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 271657926, 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:21.861 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:22.140 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:21 smithi046 ceph-mon[36898]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2024-05-08T23:24:22.170 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-08T23:24:22.171 INFO:tasks.check_counter:Daemon mds.g mds.imported=3 2024-05-08T23:24:22.171 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-05-08T23:24:22.171 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:22.171 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:22.171 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T23:24:22.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T23:24:22.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:21 smithi086 ceph-mon[78610]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2024-05-08T23:24:22.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:21 smithi053 ceph-mon[77587]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2024-05-08T23:24:22.546 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:24:23.723 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:24:23.723 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:23.723 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-08T23:24:23.723 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-08T23:24:23.723 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.003565596, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000898389, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:24:23.724 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:23.725 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 6, 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 281, 2024-05-08T23:24:23.726 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3312, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 201984, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.031426500, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.023137586, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.003588943, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:24:23.727 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 279, 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.028951037, 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000103767 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:23.728 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 201984 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 377, 2024-05-08T23:24:23.729 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 366, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1603744, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 209760, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.072898052, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.027826365, 2024-05-08T23:24:23.730 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.045055909, 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.008087147, 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 365, 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.025705484, 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000070425 2024-05-08T23:24:23.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1603744, 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 209760 2024-05-08T23:24:23.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:23.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:23.735 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:23.736 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:23.737 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 150008, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-08T23:24:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:23.739 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-08T23:24:23.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 379, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1550452, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 379, 2024-05-08T23:24:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 379, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1550452, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-08T23:24:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-08T23:24:23.743 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:24:24.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:23 smithi046 ceph-mon[36898]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 151 KiB/s wr, 32 op/s 2024-05-08T23:24:24.087 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T23:24:24.087 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T23:24:24.087 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-08T23:24:24.087 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-08T23:24:24.087 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-08T23:24:24.088 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T23:24:24.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T23:24:24.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:23 smithi086 ceph-mon[78610]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 151 KiB/s wr, 32 op/s 2024-05-08T23:24:24.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:23 smithi053 ceph-mon[77587]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 151 KiB/s wr, 32 op/s 2024-05-08T23:24:24.360 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:24:25.413 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 7960, 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 5257, 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4411702, 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 4318064, 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 24, 2024-05-08T23:24:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.664561978, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.220148039, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.480342403, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.050252266, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5255, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.418485929, 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000460225 2024-05-08T23:24:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10750, 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002414375, 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000224 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1752544, 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 799088 2024-05-08T23:24:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 356232, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 179136, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 170941329, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 32295938, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 271, 2024-05-08T23:24:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 21.230488628, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 7.216573596, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 19.746838142, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.079920226, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 179138, 2024-05-08T23:24:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5.113309583, 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000028543 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 533635, 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.095623050, 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000179 2024-05-08T23:24:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 84336, 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 126304 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 48389, 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 55161, 2024-05-08T23:24:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 14699769, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 14730212, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 34, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 9, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 4.061674963, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 1.715382623, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 2.896538818, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.133452485, 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 55155, 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.523248287, 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000045748 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 56224, 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.037105008, 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000659 2024-05-08T23:24:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 722096, 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 234864 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1706, 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.716712475, 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000420112 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 267, 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.050967495, 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000190889 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-08T23:24:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 37098, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 1048, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1048, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 31.255847233, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.029824281 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-08T23:24:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 36050, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 12, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 87, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 12, 2024-05-08T23:24:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 54, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 21, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 70, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 163, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 9, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 9, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 7, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 37410, 2024-05-08T23:24:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 1150, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 36049, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 15, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 10, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-08T23:24:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 9, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 150, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 1, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 3, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 461, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 656, 2024-05-08T23:24:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 216484495, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 461, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 287, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 36, 2024-05-08T23:24:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 251, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 174, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 1, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 7226, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 7639, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 7495, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 209, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 51, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 508 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 26, 2024-05-08T23:24:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 18, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 24, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 23, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 136, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 136, 2024-05-08T23:24:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 36, 2024-05-08T23:24:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 10, 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-08T23:24:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 1160, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 1160, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 2, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-08T23:24:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 10804132, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-08T23:24:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 655, 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 6.124530839, 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.009350428 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-08T23:24:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 88, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 337, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 249, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 121, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 132, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 11, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 87, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 445, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 358, 2024-05-08T23:24:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 9, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 231, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 222, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 62168, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 37430, 2024-05-08T23:24:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 11, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 37098, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 85, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 14, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 200, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.200577469, 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001002887 2024-05-08T23:24:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 235, 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.063021478, 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000268176 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 109, 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 18.853367581, 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.172966675 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000190447, 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000190447 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-08T23:24:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 20, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003959061, 2024-05-08T23:24:25.448 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000197953 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 20, 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.029539238, 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001476961 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-08T23:24:25.449 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 20, 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.927395452, 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.246369772 2024-05-08T23:24:25.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 245, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.688630123, 2024-05-08T23:24:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002810735 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 33, 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.040265370, 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001220162 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:25.452 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 14, 2024-05-08T23:24:25.454 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 6.205752787, 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.443268056 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.455 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002353284, 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001176642 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001035339, 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000258834 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.457 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9, 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.025317334, 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002813037 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.458 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 136, 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.214442270, 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001576781 2024-05-08T23:24:25.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 7, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 5, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 27, 2024-05-08T23:24:25.461 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 13, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 1200, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:25.462 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:25.463 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:25.464 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:25.465 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 358065, 2024-05-08T23:24:25.466 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 251, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 616, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 7, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:25.467 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 209, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 584837, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 4712, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-08T23:24:25.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2107, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 6661991, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 30, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2107, 2024-05-08T23:24:25.470 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2107, 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 27.502445001, 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.013052892 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-08T23:24:25.471 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2107, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2149 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2107, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 540, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 1567, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-08T23:24:25.472 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 11, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 43, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 662, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 80, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-08T23:24:25.473 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 785, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 497, 2024-05-08T23:24:25.474 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 22, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:25.475 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-08T23:24:25.476 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-08T23:24:25.477 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 115, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 38, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-05-08T23:24:25.478 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 42, 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 7 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-08T23:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.480 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 5, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 136, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.481 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 412577, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 153518090, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 412577, 2024-05-08T23:24:25.482 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 412577, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 153518090, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.483 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:25.484 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:25.485 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2107, 2024-05-08T23:24:25.485 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 6676815, 2024-05-08T23:24:25.485 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2107, 2024-05-08T23:24:25.485 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6676815, 2024-05-08T23:24:25.485 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:25.486 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:25.487 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2107, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2107, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2107, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2107, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.488 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:25.489 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 136, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 14824, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 136, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 7, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 14824, 2024-05-08T23:24:25.490 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:25.491 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55e7b5482000": { 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1160, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 6609828, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:25.492 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1160, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 655, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6609828, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:25.493 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:25.494 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:25.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:25.494 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-08T23:24:25.782 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-05-08T23:24:25.782 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-05-08T23:24:25.782 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-08T23:24:25.783 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=20 2024-05-08T23:24:25.783 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=14 2024-05-08T23:24:25.783 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T23:24:25.783 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T23:24:26.041 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:24:26.209 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:25 smithi053 ceph-mon[77587]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-05-08T23:24:26.213 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:25 smithi086 ceph-mon[78610]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-05-08T23:24:26.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:25 smithi046 ceph-mon[36898]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 87 KiB/s wr, 29 op/s 2024-05-08T23:24:27.079 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3211, 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 3213, 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 122560082, 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 998628, 2024-05-08T23:24:27.080 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.390566192, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.089357104, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.174968638, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.097719134, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3211, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.149842732, 2024-05-08T23:24:27.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000046665 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:27.082 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1155, 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1430, 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 87407491, 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 991225, 2024-05-08T23:24:27.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 9, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.174466797, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.041666214, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.085841642, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.033910213, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1428, 2024-05-08T23:24:27.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.089123524, 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000062411 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 633824 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1106, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1085, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 64803999, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 428445, 2024-05-08T23:24:27.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.173881492, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.039933738, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.104647966, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.029732491, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1084, 2024-05-08T23:24:27.087 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.069346648, 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000063972 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.088 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1707040, 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 216032 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-08T23:24:27.089 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8577, 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.215004921, 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000025067 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:27.090 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-08T23:24:27.091 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-08T23:24:27.092 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 15187, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 1, 2024-05-08T23:24:27.093 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 15186, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 15186, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 5, 2024-05-08T23:24:27.094 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-08T23:24:27.095 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:27.096 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:27.097 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:27.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-08T23:24:27.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-08T23:24:27.100 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-08T23:24:27.102 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 73689, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-08T23:24:27.103 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 99, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 275856933, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 275854583, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.104 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 73689 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 474, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 15088, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 14614, 2024-05-08T23:24:27.105 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 43, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 44, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 1, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 15189, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 15189, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-08T23:24:27.106 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 66560, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-08T23:24:27.107 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-08T23:24:27.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.111 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.113 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.114 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-08T23:24:27.115 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:27.118 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.119 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.120 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:27.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.122 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.122 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.122 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:27.122 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.123 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:27.125 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-08T23:24:27.126 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3305, 2024-05-08T23:24:27.127 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:27.128 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:27.129 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:27.130 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:27.131 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 259437, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 625, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:27.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 209, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 12038691, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 73838, 2024-05-08T23:24:27.134 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 5043, 2024-05-08T23:24:27.135 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 5043, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5043, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 8.747825613, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001734647 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-08T23:24:27.136 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5043, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5044 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 5043, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 5043, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-08T23:24:27.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 2637, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 2404, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-08T23:24:27.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:27.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-08T23:24:27.140 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-08T23:24:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 115, 2024-05-08T23:24:27.142 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 31, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 11, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 1, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-08T23:24:27.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:27.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 5468, 2024-05-08T23:24:27.146 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 274254787, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 5468, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5468, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 274254787, 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:27.147 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 5043, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 271673500, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5043, 2024-05-08T23:24:27.149 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 271673500, 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:27.150 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:27.151 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-08T23:24:27.151 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:27.151 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:27.151 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:27.151 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:27.152 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:27.152 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 5043, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 5043, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5043, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5043, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:27.154 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.155 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5589a53f6140": { 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:27.156 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:27.157 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:27.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:27.158 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-08T23:24:27.466 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-08T23:24:27.467 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-08T23:24:27.467 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-08T23:24:27.467 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:27.467 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:27.467 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T23:24:27.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T23:24:27.722 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:24:28.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:27 smithi086 ceph-mon[78610]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 79 KiB/s wr, 22 op/s 2024-05-08T23:24:28.244 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:27 smithi053 ceph-mon[77587]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 79 KiB/s wr, 22 op/s 2024-05-08T23:24:28.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:27 smithi046 ceph-mon[36898]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 79 KiB/s wr, 22 op/s 2024-05-08T23:24:28.770 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-08T23:24:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 428, 2024-05-08T23:24:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 405, 2024-05-08T23:24:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1710188, 2024-05-08T23:24:28.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 216359, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.072473988, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.023308635, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.045693498, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.009656572, 2024-05-08T23:24:28.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 405, 2024-05-08T23:24:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.026117978, 2024-05-08T23:24:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000064488 2024-05-08T23:24:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1709920, 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 216048 2024-05-08T23:24:28.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2796, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 3072, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 5017400, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1502195, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-05-08T23:24:28.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.285351174, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.100083830, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.070344890, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.086883811, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3069, 2024-05-08T23:24:28.776 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.171917446, 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000056017 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 633200 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1416, 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1418, 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2770488, 2024-05-08T23:24:28.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 440392, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.132019659, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.034576201, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.035662592, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.049027344, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1416, 2024-05-08T23:24:28.779 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.064876389, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000045816 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:28.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:28.781 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6917, 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.192578801, 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000027841 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:28.782 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-08T23:24:28.783 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-08T23:24:28.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 179, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 179, 2024-05-08T23:24:28.785 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 179, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 4, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-08T23:24:28.786 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-08T23:24:28.787 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 656, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 216484495, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2024-05-08T23:24:28.788 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:28.789 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:28.790 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-08T23:24:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-08T23:24:28.792 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-08T23:24:28.793 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-08T23:24:28.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 1160, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-08T23:24:28.795 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-08T23:24:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-08T23:24:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-08T23:24:28.796 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 10804132, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 10744703, 2024-05-08T23:24:28.797 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 1160 2024-05-08T23:24:28.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 44, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 253, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 209, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 15, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 28, 2024-05-08T23:24:28.799 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 13, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 179, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 328, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 149, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-08T23:24:28.800 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 48188, 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:28.801 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.803 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.804 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-08T23:24:28.806 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.807 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-08T23:24:28.809 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.810 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.811 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-08T23:24:28.813 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-08T23:24:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 7, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 5, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 5, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-08T23:24:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 6410, 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 200008, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 109, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-08T23:24:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 209, 2024-05-08T23:24:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 332205, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2138, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-08T23:24:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 4211, 2024-05-08T23:24:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 26, 2024-05-08T23:24:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 4, 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 4211, 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4211, 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.308913869, 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001023251 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4211, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4212 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 4211, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 4211, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-08T23:24:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 2650, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1558, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-08T23:24:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-08T23:24:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-08T23:24:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-08T23:24:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-08T23:24:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-08T23:24:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-08T23:24:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-08T23:24:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-08T23:24:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-08T23:24:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-08T23:24:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 115, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 31, 2024-05-08T23:24:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 6, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 9, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 3, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-08T23:24:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-08T23:24:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 4636, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 9056177, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 4636, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4636, 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 9056177, 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4211, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 6609838, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4211, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6609838, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4211, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 4211, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4211, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 4211, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55b61522a140": { 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:28.860 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-08T23:24:29.140 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T23:24:29.140 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T23:24:29.141 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-08T23:24:29.141 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:29.141 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:29.141 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T23:24:29.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T23:24:29.400 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:24:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:29 smithi086 ceph-mon[78610]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T23:24:30.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:29 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:29 smithi053 ceph-mon[77587]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T23:24:30.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:29 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:29 smithi046 ceph-mon[36898]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T23:24:30.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:29 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:30.611 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-08T23:24:30.611 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2001, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1980, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3573400, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 707352, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.237266338, 2024-05-08T23:24:30.612 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.072186417, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.088639826, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.066314978, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1979, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.123250143, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000062279 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.613 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1707584, 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 217216 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.614 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2540, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2819, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 13228092, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1424671, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.227848931, 2024-05-08T23:24:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.071150424, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.063362789, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.073849386, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2816, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.170193313, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000060437 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 635296 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 8, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 8, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4621, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1704, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.003179220, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000143390, 2024-05-08T23:24:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.002685642, 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000107501, 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.004153402, 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000593343 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-08T23:24:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6939, 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.196663088, 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000028341 2024-05-08T23:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-08T23:24:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 2111, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 2111, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 2111, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-08T23:24:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 4, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-08T23:24:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-08T23:24:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-08T23:24:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-08T23:24:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 9430, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 10, 2024-05-08T23:24:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 18135178, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 18134604, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 9430 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 77, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 2090, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 2013, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 25, 2024-05-08T23:24:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 25, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 0, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 2111, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 2111, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-08T23:24:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-08T23:24:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 51844, 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-08T23:24:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-08T23:24:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-08T23:24:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-08T23:24:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-08T23:24:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 5378, 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-08T23:24:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 208479, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 124, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-08T23:24:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 209, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 1719828, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 10199, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 4118, 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-05-08T23:24:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 4118, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4118, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 8.883822970, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002157314 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-08T23:24:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4118, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4119 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 4118, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 4118, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-08T23:24:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 2560, 2024-05-08T23:24:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-08T23:24:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1556, 2024-05-08T23:24:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-08T23:24:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-08T23:24:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-08T23:24:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-08T23:24:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-08T23:24:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-08T23:24:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-08T23:24:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-08T23:24:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-08T23:24:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-08T23:24:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-08T23:24:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 115, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 31, 2024-05-08T23:24:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 7, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 9, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 2, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-08T23:24:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-08T23:24:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-08T23:24:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-08T23:24:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 4545, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 16372294, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 4545, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4545, 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 16372294, 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4118, 2024-05-08T23:24:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 13940879, 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4118, 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 13940879, 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4118, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 4118, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4118, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 4118, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-08T23:24:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x555aef2800a0": { 2024-05-08T23:24:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-08T23:24:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-08T23:24:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-08T23:24:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-08T23:24:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-08T23:24:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-08T23:24:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-08T23:24:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-08T23:24:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-08T23:24:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-08T23:24:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-08T23:24:30.701 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-08T23:24:31.206 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-08T23:24:31.207 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-08T23:24:31.207 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-08T23:24:31.207 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:31.207 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:31.207 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T23:24:31.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T23:24:31.727 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:24:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:31 smithi053 ceph-mon[77587]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-05-08T23:24:32.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:31 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:31 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:31 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:31 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.263 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:31 smithi086 ceph-mon[78610]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-05-08T23:24:32.263 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:31 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.264 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:31 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.264 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:31 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.264 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:31 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:31 smithi046 ceph-mon[36898]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-05-08T23:24:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:31 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:31 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:31 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:32.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:31 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3522, 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2044, 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3093971, 2024-05-08T23:24:33.026 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 626820, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 20, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.221837361, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.069574340, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.185942268, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.007845477, 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:33.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2044, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.115925775, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000056715 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4623, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000868687, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000187 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.028 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1728656, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 234912 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 264891, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 132345, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 143951712, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 22972404, 2024-05-08T23:24:33.029 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 77, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 11.940757533, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 5.186047235, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 9.282758216, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000470495, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 132342, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.526798804, 2024-05-08T23:24:33.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000019092 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 396438, 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.054290977, 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000136 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-05-08T23:24:33.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 635920 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 8881, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 8947, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2537103, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1321467, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:33.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.463067434, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.183347714, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.355593519, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.006015266, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8945, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.219419455, 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000024529 2024-05-08T23:24:33.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9993, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.004992615, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000499 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 59, 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001505194, 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000025511 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2, 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000014892, 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000007446 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 7751, 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 1, 2024-05-08T23:24:33.037 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000146407, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000146407 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 7750, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:33.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 46, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 7, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 39, 2024-05-08T23:24:33.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 35, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 1, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 1, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 4, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 7792, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 27, 2024-05-08T23:24:33.040 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 7749, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 16, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-05-08T23:24:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-05-08T23:24:33.042 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-05-08T23:24:33.042 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-05-08T23:24:33.042 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:33.042 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 1, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 705, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 211650580, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:33.043 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:33.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-08T23:24:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 69, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 42, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 64, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-08T23:24:33.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 7, 2024-05-08T23:24:33.048 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 7, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 1, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-05-08T23:24:33.049 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 1, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4196615, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.015932485, 2024-05-08T23:24:33.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002276069 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 49, 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 49, 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-05-08T23:24:33.051 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 103, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 130, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 27, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 46, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 46, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 1, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 1, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-05-08T23:24:33.052 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 52936, 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 198964 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 7752, 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:33.053 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 7751, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 39, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.054 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.056 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-08T23:24:33.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000146407, 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000146407 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.059 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.060 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.061 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.062 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.064 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.065 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.066 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-08T23:24:33.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-08T23:24:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.072 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 4, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 1, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1197, 2024-05-08T23:24:33.073 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:33.074 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:33.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:33.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:33.078 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:33.079 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 255416, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 217, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:33.080 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 209, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 356835, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2930, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:33.082 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 44, 2024-05-08T23:24:33.083 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 12703, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 44, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 44, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.277993827, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006318041 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-08T23:24:33.084 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 44, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 66 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 44, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 0, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 44, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-08T23:24:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 0, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 7, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 8, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-08T23:24:33.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 6, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:33.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-08T23:24:33.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-08T23:24:33.089 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-08T23:24:33.090 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 115, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 33, 2024-05-08T23:24:33.091 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 8, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 11, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 3, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 14, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 1 2024-05-08T23:24:33.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.093 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-08T23:24:33.093 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-08T23:24:33.093 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:33.093 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-08T23:24:33.093 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:33.094 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 277290, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 124723715, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 277290, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 277290, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 124723715, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.097 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:33.098 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:33.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:33.098 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:33.098 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:33.098 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 44, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 12703, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 44, 2024-05-08T23:24:33.099 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 12703, 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:33.101 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 44, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 44, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 44, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 44, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.102 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:33.103 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:33.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.105 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55f1ebcd8000": { 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 7, 2024-05-08T23:24:33.106 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2311, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 7, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 7, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2311, 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:33.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:33.108 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:33.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:33.108 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.108 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:33.108 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-08T23:24:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:32 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:32 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.342 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.342 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.342 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.342 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.343 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.343 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.343 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.343 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:32 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:33.411 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T23:24:33.411 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T23:24:33.411 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-08T23:24:33.412 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:33.412 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:33.412 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T23:24:33.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T23:24:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:24:34.079 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:24:34.079 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:24:34.079 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:24:34.079 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:24:34.079 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:34.080 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:33 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:24:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:24:34.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.263 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:34.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:34.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:33 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-08T23:24:34.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi046 to 222.3M 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi046 to 233137049: error parsing value: Value '233137049' is below minimum 939524096 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:34.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:33 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4771, 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 5021, 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1409241, 2024-05-08T23:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 16190800, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 12, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.384642287, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.124881971, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.087667940, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.140499424, 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5019, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.535851411, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000106764 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 522944 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 282052, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 147639, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 149136736, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 25950793, 2024-05-08T23:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 46, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 13.179114624, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 5.738987124, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 10.137346010, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.037788961, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 147637, 2024-05-08T23:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.671574485, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000024868 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 426189, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.064879382, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000152 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-05-08T23:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 118256 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 17181, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 25868, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 7127412, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 18895864, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 54, 2024-05-08T23:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.110409028, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.482923186, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.636566583, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.098950586, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 25867, 2024-05-08T23:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.505435397, 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000096858 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 20995, 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006839788, 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000325 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1719376, 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 233312 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4969, 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.543370414, 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000713095 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4224, 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.266051435, 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000062985 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 11049, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 11048, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 11048, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 41.756835362, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003779583 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-08T23:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 1, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 147, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 23, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 124, 2024-05-08T23:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 114, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 4, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 4, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 6, 2024-05-08T23:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 22457, 2024-05-08T23:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 10426, 2024-05-08T23:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-05-08T23:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 18, 2024-05-08T23:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 190, 2024-05-08T23:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-08T23:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-08T23:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 2, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 6, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 86, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 705, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 211650580, 2024-05-08T23:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 86, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 86, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 111, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 6757, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 1163, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 394, 2024-05-08T23:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 243, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 10832 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 69, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 42, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 65, 2024-05-08T23:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 2013, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 2013, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 9432, 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 9432, 2024-05-08T23:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 10, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 10, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-05-08T23:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 18139881, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2400, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 14.068695807, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005861956 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 104, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 2117, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 2013, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 105, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 131, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 26, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 147, 2024-05-08T23:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 2958, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 2811, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 4, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 2108, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 2104, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 56988, 2024-05-08T23:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 198964 2024-05-08T23:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 19267, 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 11049, 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 208, 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 2347, 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-08T23:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2013, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 27.117860865, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.013471366 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 48, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.255819876, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.067829580 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2410, 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.853519869, 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000769095 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-08T23:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4222, 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.602218528, 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000616347 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 190, 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.412570669, 2024-05-08T23:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.012697740 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:34.706 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 152, 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.104798984, 2024-05-08T23:24:34.707 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000689467 2024-05-08T23:24:34.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.709 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-08T23:24:34.709 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2013, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4.410046571, 2024-05-08T23:24:34.711 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002190783 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.712 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-05-08T23:24:34.713 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 2493, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 2493, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1247, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:34.714 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:34.715 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:34.716 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:34.718 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 318053, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 252, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:34.719 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 209, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:34.720 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 570130, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 4475, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.721 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 7153, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 14334019, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 7153, 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-08T23:24:34.722 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7153, 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 103.850753838, 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.014518489 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7153, 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 7320 2024-05-08T23:24:34.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 7153, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 281, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 6872, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-05-08T23:24:34.724 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 195, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 2595, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 132, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 4041, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-08T23:24:34.725 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 86, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:34.726 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-08T23:24:34.727 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-08T23:24:34.728 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 115, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-08T23:24:34.729 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 32, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 19, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 7, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 169, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 72 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.730 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-05-08T23:24:34.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 5, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 2013, 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:34.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 304000, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 130533052, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 304000, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 304000, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 130533052, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:34.735 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 7153, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 14553564, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 7153, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 14553564, 2024-05-08T23:24:34.736 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 7153, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 7153, 2024-05-08T23:24:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 7153, 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 7153, 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:34.739 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2013, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 219545, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2013, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:34.740 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 195, 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 219545, 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.741 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55a8b426c000": { 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 9432, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 13945577, 2024-05-08T23:24:34.742 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 9432, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2400, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 13945577, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:34.743 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:34.744 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.744 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:34.744 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-08T23:24:35.026 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T23:24:35.027 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-05-08T23:24:35.027 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-08T23:24:35.027 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:35.027 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:35.027 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T23:24:35.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T23:24:35.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:34 smithi086 ceph-mon[78610]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:24:35.214 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:34 smithi086 ceph-mon[78610]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:24:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:24:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:34 smithi053 ceph-mon[77587]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:24:35.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:34 smithi053 ceph-mon[77587]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:24:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:34 smithi046 ceph-mon[36898]: Adjusting osd_memory_target on smithi086 to 182.5M 2024-05-08T23:24:35.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:34 smithi046 ceph-mon[36898]: Unable to set osd_memory_target on smithi086 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2024-05-08T23:24:36.038 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:35 smithi086 ceph-mon[78610]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1024775, 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1550204, 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 302618134, 2024-05-08T23:24:36.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 2037349661, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 13, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 52.222264082, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 25.378412741, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 31.407565385, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 2.900171125, 2024-05-08T23:24:36.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1550203, 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 107.648434599, 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000069441 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1380955, 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.319799172, 2024-05-08T23:24:36.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000231 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1742144, 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 254848 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 444112, 2024-05-08T23:24:36.232 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 681914, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 89815790, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1094295617, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 20, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 28.675344786, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 11.843473446, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 14.602233939, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 3.476256054, 2024-05-08T23:24:36.233 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 681911, 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 32.897775819, 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048243 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 895762, 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.157267798, 2024-05-08T23:24:36.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000175 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 119520 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 255923, 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 433143, 2024-05-08T23:24:36.235 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 57431360, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 803786371, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 76, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 10, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 17.776811934, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 7.041205358, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 8.287076060, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 2.423282450, 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:36.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 433142, 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 21.847718836, 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000050440 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 523944, 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.098727896, 2024-05-08T23:24:36.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000188 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 526128 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-08T23:24:36.238 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 211247, 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 269.091348002, 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001273823 2024-05-08T23:24:36.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.240 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:36.240 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:36.240 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:36.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 140107, 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 36.199933202, 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000258373 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 516994, 2024-05-08T23:24:36.241 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 507091, 2024-05-08T23:24:36.242 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-08T23:24:36.242 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 507091, 2024-05-08T23:24:36.242 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3195.756266008, 2024-05-08T23:24:36.242 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006302135 2024-05-08T23:24:36.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.243 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-08T23:24:36.243 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 9903, 2024-05-08T23:24:36.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:36.244 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:36.244 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 656, 2024-05-08T23:24:36.244 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 46, 2024-05-08T23:24:36.245 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 45, 2024-05-08T23:24:36.245 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 9450, 2024-05-08T23:24:36.245 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 4687, 2024-05-08T23:24:36.246 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 4757, 2024-05-08T23:24:36.246 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 6, 2024-05-08T23:24:36.246 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 2687, 2024-05-08T23:24:36.246 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 111, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 606, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 606, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 9, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 700212, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 457405, 2024-05-08T23:24:36.247 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 9896, 2024-05-08T23:24:36.248 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 12, 2024-05-08T23:24:36.248 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:36.248 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:36.248 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 4099, 2024-05-08T23:24:36.249 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-08T23:24:36.249 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-08T23:24:36.249 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 5, 2024-05-08T23:24:36.250 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 95, 2024-05-08T23:24:36.250 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 8, 2024-05-08T23:24:36.250 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 139, 2024-05-08T23:24:36.250 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:36.251 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 9062, 2024-05-08T23:24:36.251 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:36.251 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 705, 2024-05-08T23:24:36.251 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 211650580, 2024-05-08T23:24:36.252 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-05-08T23:24:36.252 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 9062, 2024-05-08T23:24:36.252 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:36.252 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:36.252 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:36.253 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 1316, 2024-05-08T23:24:36.253 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:36.253 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 1316, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 7746, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 7, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 130650, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 305195, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:36.254 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 241273, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 66664, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 27830, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 332366 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 34, 2024-05-08T23:24:36.255 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 13, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 118, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 75568, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 75568, 2024-05-08T23:24:36.256 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:36.257 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 5, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 92, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 364877, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 304384, 2024-05-08T23:24:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 304384, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 60493, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 1624, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 614, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 486, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 486, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 131, 2024-05-08T23:24:36.259 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 3, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 1799378895, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 2319137201, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 118906, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 859.127046133, 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007225262 2024-05-08T23:24:36.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3921, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 79536, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 75615, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 256, 2024-05-08T23:24:36.261 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 559, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 303, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 9450, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 157739, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 148289, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 606, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 79550, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 78944, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 99576, 2024-05-08T23:24:36.262 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 198964 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 751184, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 11320, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 516994, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 9457, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 22652, 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-08T23:24:36.263 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 76167, 2024-05-08T23:24:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 384.751745844, 2024-05-08T23:24:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005051423 2024-05-08T23:24:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-08T23:24:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 11408, 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 173.377024841, 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.015197845 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 131867, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 465.212965732, 2024-05-08T23:24:36.266 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003527895 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-08T23:24:36.267 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.269 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-08T23:24:36.270 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 99, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.650709669, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006572824 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.271 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9, 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.016160407, 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001795600 2024-05-08T23:24:36.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 194899, 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1617.637856820, 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.008299877 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15714, 2024-05-08T23:24:36.273 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 91.616434308, 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005830242 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.275 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002367880, 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002367880 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 227, 2024-05-08T23:24:36.277 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 8.034106561, 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.035392539 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1132, 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 10.537764863, 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.009308979 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-08T23:24:36.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 75568, 2024-05-08T23:24:36.281 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 443.919129083, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005874432 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 5, 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 4, 2024-05-08T23:24:36.283 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 46478, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 46478, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 3294, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-08T23:24:36.284 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:36.285 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:36.286 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:36.287 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:36.288 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 503845, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 2047, 2024-05-08T23:24:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:36.290 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 209, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 17400338, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 176982, 2024-05-08T23:24:36.291 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-08T23:24:36.292 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 286608, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 2335288405, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 27, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 286608, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 286608, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2094.925467987, 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007309375 2024-05-08T23:24:36.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 286608, 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 289750 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 286608, 2024-05-08T23:24:36.294 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 10712, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 275896, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 126, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 1112, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1650, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 120559, 2024-05-08T23:24:36.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 905, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 46, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 152383, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:36.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 9062, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 1730, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-08T23:24:36.297 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-08T23:24:36.298 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-08T23:24:36.299 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 115, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 33, 2024-05-08T23:24:36.300 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 1732, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 445 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-08T23:24:36.301 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 1 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.302 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 5, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 75568, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.303 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1724806, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 307322112, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:36.304 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1724806, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1724806, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 307322112, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.305 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:36.306 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 286608, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2343538549, 2024-05-08T23:24:36.307 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 286608, 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2343538549, 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.308 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:36.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 286608, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 286608, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 286608, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 286608, 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:36.310 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:36.311 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 75568, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 8250144, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 75568, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:36.312 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1653, 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 8250144, 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55f8c8b72000": { 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 189, 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 364877, 2024-05-08T23:24:36.314 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2314942897, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 364877, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 118906, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2314942708, 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:36.315 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:36.316 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:36.316 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:36.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:36.316 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-08T23:24:36.317 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:35 smithi046 ceph-mon[36898]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-05-08T23:24:36.317 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:35 smithi053 ceph-mon[77587]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-05-08T23:24:36.601 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-05-08T23:24:36.601 INFO:tasks.check_counter:Daemon mds.i mds.imported=8 2024-05-08T23:24:36.601 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-08T23:24:36.602 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:36.602 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:36.602 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T23:24:36.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T23:24:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:24:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-08T23:24:37.803 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:24:37.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1767, 2024-05-08T23:24:37.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1744, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2160016, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 634344, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.162149662, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.048878116, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.064774811, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.045142539, 2024-05-08T23:24:37.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1744, 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.126163796, 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000072341 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1711120, 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 218848 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1252, 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1533, 2024-05-08T23:24:37.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 308472, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1026034, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.104176744, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.037165952, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.021231675, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.035860405, 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1530, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.089097473, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000058233 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 637216 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1343, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1345, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 354122, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 417674, 2024-05-08T23:24:37.809 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.084360338, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.026069544, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.022197627, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.027236263, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1343, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.055423943, 2024-05-08T23:24:37.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000041268 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:24:37.811 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:37.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6550, 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.150603162, 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000022992 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.814 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-05-08T23:24:37.815 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-05-08T23:24:37.816 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-05-08T23:24:37.817 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:24:37.818 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:24:37.819 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:24:37.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-05-08T23:24:37.821 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-08T23:24:37.822 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-05-08T23:24:37.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-05-08T23:24:37.824 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 7, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 1, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-05-08T23:24:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4196615, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4196474, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 7 2024-05-08T23:24:37.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-05-08T23:24:37.827 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 48524, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 207156 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.828 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.829 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-08T23:24:37.830 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.832 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.833 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-08T23:24:37.834 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.836 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.837 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.838 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.840 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.841 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.842 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.844 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-08T23:24:37.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.846 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 4, 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-05-08T23:24:37.847 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 4, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 5348, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:24:37.848 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:24:37.849 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-08T23:24:37.850 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-08T23:24:37.851 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:24:37.852 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 190177, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 53, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-08T23:24:37.853 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 10024, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 209, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8320, 2024-05-08T23:24:37.854 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-08T23:24:37.855 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3929, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3929, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3929, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.504945060, 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000637552 2024-05-08T23:24:37.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3929, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3930 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3929, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3929, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-05-08T23:24:37.857 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2583, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1344, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-08T23:24:37.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-05-08T23:24:37.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-08T23:24:37.860 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-08T23:24:37.861 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-08T23:24:37.862 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 115, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 5, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 7, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 2, 2024-05-08T23:24:37.863 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-08T23:24:37.864 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-08T23:24:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:37.866 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4358, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2405517, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4358, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4358, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2405517, 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:37.867 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:37.868 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3929, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2316, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3929, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2316, 2024-05-08T23:24:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-08T23:24:37.870 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3929, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3929, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3929, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3929, 2024-05-08T23:24:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:37.872 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-05-08T23:24:37.873 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5624f9ff80a0": { 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-08T23:24:37.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-05-08T23:24:37.875 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-08T23:24:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-08T23:24:38.184 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-08T23:24:38.184 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-08T23:24:38.184 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-08T23:24:38.184 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:24:38.184 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:24:38.185 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T23:24:38.196 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T23:24:38.197 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:24:38.197 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:38.257 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:24:38.257 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:24:38.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:37 smithi053 ceph-mon[77587]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 95 KiB/s wr, 14 op/s 2024-05-08T23:24:38.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:37 smithi046 ceph-mon[36898]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 95 KiB/s wr, 14 op/s 2024-05-08T23:24:38.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:38.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:37 smithi086 ceph-mon[78610]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 95 KiB/s wr, 14 op/s 2024-05-08T23:24:38.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 -- 172.21.15.46:0/1395857908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8101070 msgr2=0x7fa3d8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1395857908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8101070 0x7fa3d8101450 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009990 tx=0x7fa3cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 -- 172.21.15.46:0/1395857908 shutdown_connections 2024-05-08T23:24:38.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1395857908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d8106190 0x7fa3d810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1395857908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8101070 0x7fa3d8101450 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1395857908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101a20 0x7fa3d8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 -- 172.21.15.46:0/1395857908 >> 172.21.15.46:0/1395857908 conn(0x7fa3d80fc930 msgr2=0x7fa3d80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:38.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 -- 172.21.15.46:0/1395857908 shutdown_connections 2024-05-08T23:24:38.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.567+0000 7fa3e0000700 1 -- 172.21.15.46:0/1395857908 wait complete. 2024-05-08T23:24:38.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 Processor -- start 2024-05-08T23:24:38.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 -- start start 2024-05-08T23:24:38.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101070 0x7fa3d8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8106190 0x7fa3d8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819e980 con 0x7fa3d819df00 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819eaf0 con 0x7fa3d8101070 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.568+0000 7fa3e0000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819ecb0 con 0x7fa3d8106190 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3dd600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101070 0x7fa3d8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39104/0 (socket says 172.21.15.46:39104) 2024-05-08T23:24:38.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 -- 172.21.15.46:0/1673304487 learned_addr learned my addr 172.21.15.46:0/1673304487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3dcc00700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8106190 0x7fa3d8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 -- 172.21.15.46:0/1673304487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8106190 msgr2=0x7fa3d8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8106190 0x7fa3d8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 -- 172.21.15.46:0/1673304487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101070 msgr2=0x7fa3d8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101070 0x7fa3d8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 -- 172.21.15.46:0/1673304487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0095f0 con 0x7fa3d819df00 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.569+0000 7fa3de000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000ef30 tx=0x7fa3d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:38.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.570+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0018070 con 0x7fa3d819df00 2024-05-08T23:24:38.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.570+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d81a61b0 con 0x7fa3d819df00 2024-05-08T23:24:38.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.570+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3d000f040 con 0x7fa3d819df00 2024-05-08T23:24:38.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.570+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000cd00 con 0x7fa3d819df00 2024-05-08T23:24:38.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.570+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d81a6690 con 0x7fa3d819df00 2024-05-08T23:24:38.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.571+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d8102540 con 0x7fa3d819df00 2024-05-08T23:24:38.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.573+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3d0003a00 con 0x7fa3d819df00 2024-05-08T23:24:38.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.574+0000 7fa3d6000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 0x7fa3c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:38.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.574+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fa3d00cd590 con 0x7fa3d819df00 2024-05-08T23:24:38.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.575+0000 7fa3dd600700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 0x7fa3c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:38.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.575+0000 7fa3dd600700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 0x7fa3c00a3750 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc006010 tx=0x7fa3cc00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:38.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.577+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3d0096600 con 0x7fa3d819df00 2024-05-08T23:24:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.798+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3d8101a20 con 0x7fa3d819df00 2024-05-08T23:24:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.800+0000 7fa3d6000700 1 -- 172.21.15.46:0/1673304487 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+25250 (secure 0 0 0) 0x7fa3d0025380 con 0x7fa3d819df00 2024-05-08T23:24:38.801 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:38.802 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:24:14.936136+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 msgr2=0x7fa3c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 0x7fa3c00a3750 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc006010 tx=0x7fa3cc00b260 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 msgr2=0x7fa3d819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000ef30 tx=0x7fa3d000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 shutdown_connections 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fa3c00a1290 0x7fa3c00a3750 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d819df00 0x7fa3d819e380 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8101070 0x7fa3d8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 --2- 172.21.15.46:0/1673304487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d8106190 0x7fa3d8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 >> 172.21.15.46:0/1673304487 conn(0x7fa3d80fc930 msgr2=0x7fa3d80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 shutdown_connections 2024-05-08T23:24:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:38.803+0000 7fa3e0000700 1 -- 172.21.15.46:0/1673304487 wait complete. 2024-05-08T23:24:38.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:58:17.469909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:01:49.619332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:01:50.925439+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:01:54.477718+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:24:38.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:24:39.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 -- 172.21.15.46:0/3695419007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c103c60 msgr2=0x7f001c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 --2- 172.21.15.46:0/3695419007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c103c60 0x7f001c107ca0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0008009990 tx=0x7f0008009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 -- 172.21.15.46:0/3695419007 shutdown_connections 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 --2- 172.21.15.46:0/3695419007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c1083d0 0x7f001c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 --2- 172.21.15.46:0/3695419007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1032b0 0x7f001c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 --2- 172.21.15.46:0/3695419007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c103c60 0x7f001c107ca0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 -- 172.21.15.46:0/3695419007 >> 172.21.15.46:0/3695419007 conn(0x7f001c0feb10 msgr2=0x7f001c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 -- 172.21.15.46:0/3695419007 shutdown_connections 2024-05-08T23:24:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.030+0000 7f0024600700 1 -- 172.21.15.46:0/3695419007 wait complete. 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.031+0000 7f0024600700 1 Processor -- start 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 -- start start 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1083d0 0x7f001c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c19de90 0x7f001c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19e910 con 0x7f001c19de90 2024-05-08T23:24:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19ea80 con 0x7f001c1083d0 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0024600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19ec40 con 0x7f001c1032b0 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0021c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0021c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:39226/0 (socket says 172.21.15.46:39226) 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0021c00700 1 -- 172.21.15.46:0/778722418 learned_addr learned my addr 172.21.15.46:0/778722418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0021200700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1083d0 0x7f001c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.032+0000 7f0022600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c19de90 0x7f001c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 -- 172.21.15.46:0/778722418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1083d0 msgr2=0x7f001c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1083d0 0x7f001c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 -- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c19de90 msgr2=0x7f001c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c19de90 0x7f001c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 -- 172.21.15.46:0/778722418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080095f0 con 0x7f001c1032b0 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0021c00700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f001000e9c0 tx=0x7f001000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100097b0 con 0x7f001c1032b0 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001000ee60 con 0x7f001c1032b0 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1a61a0 con 0x7f001c1032b0 2024-05-08T23:24:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100054f0 con 0x7f001c1032b0 2024-05-08T23:24:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.033+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a6680 con 0x7f001c1032b0 2024-05-08T23:24:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.034+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c10b6a0 con 0x7f001c1032b0 2024-05-08T23:24:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.036+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0010004750 con 0x7f001c1032b0 2024-05-08T23:24:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.038+0000 7f001a000700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.038+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f0010014070 con 0x7f001c1032b0 2024-05-08T23:24:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.038+0000 7f0021200700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.039+0000 7f0021200700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 0x7f00040a37d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f001c19ab00 tx=0x7f0008005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.040+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0010096270 con 0x7f001c1032b0 2024-05-08T23:24:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.259+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f001c19a8b0 con 0x7f001c1032b0 2024-05-08T23:24:39.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.259+0000 7f001a000700 1 -- 172.21.15.46:0/778722418 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0010099890 con 0x7f001c1032b0 2024-05-08T23:24:39.260 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-08T23:24:39.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:38 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1673304487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 msgr2=0x7f00040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 0x7f00040a37d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f001c19ab00 tx=0x7f0008005fd0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 msgr2=0x7f001c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f001000e9c0 tx=0x7f001000ecd0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 shutdown_connections 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f001c19de90 0x7f001c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c1032b0 0x7f001c1995b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 --2- 172.21.15.46:0/778722418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f001c1083d0 0x7f001c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 >> 172.21.15.46:0/778722418 conn(0x7f001c0feb10 msgr2=0x7f001c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 shutdown_connections 2024-05-08T23:24:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.262+0000 7f0024600700 1 -- 172.21.15.46:0/778722418 wait complete. 2024-05-08T23:24:39.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:38 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1673304487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:39.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:24:39.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:38 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1673304487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:39.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 -- 172.21.15.46:0/1832242703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc101080 msgr2=0x7f8cbc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 --2- 172.21.15.46:0/1832242703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc101080 0x7f8cbc101460 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009990 tx=0x7f8cac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 -- 172.21.15.46:0/1832242703 shutdown_connections 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 --2- 172.21.15.46:0/1832242703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 --2- 172.21.15.46:0/1832242703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc101080 0x7f8cbc101460 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 --2- 172.21.15.46:0/1832242703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc101a30 0x7f8cbc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.483+0000 7f8cc2600700 1 -- 172.21.15.46:0/1832242703 >> 172.21.15.46:0/1832242703 conn(0x7f8cbc0fc8e0 msgr2=0x7f8cbc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.484+0000 7f8cc2600700 1 -- 172.21.15.46:0/1832242703 shutdown_connections 2024-05-08T23:24:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.484+0000 7f8cc2600700 1 -- 172.21.15.46:0/1832242703 wait complete. 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 Processor -- start 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 -- start start 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc19dd80 0x7f8cbc19e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19e800 con 0x7f8cbc101080 2024-05-08T23:24:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19e970 con 0x7f8cbc1061a0 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cc2600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc19eb30 con 0x7f8cbc19dd80 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cbb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cbaa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cbbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc19dd80 0x7f8cbc19e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cbb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39148/0 (socket says 172.21.15.46:39148) 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.485+0000 7f8cbb400700 1 -- 172.21.15.46:0/3592735017 learned_addr learned my addr 172.21.15.46:0/3592735017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:39.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 -- 172.21.15.46:0/3592735017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc19dd80 msgr2=0x7f8cbc19e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc19dd80 0x7f8cbc19e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 -- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 msgr2=0x7f8cbc1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 -- 172.21.15.46:0/3592735017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0095f0 con 0x7f8cbc1061a0 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbb400700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cbaa00700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc1999e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e9c0 tx=0x7f8ca800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004220 con 0x7f8cbc1061a0 2024-05-08T23:24:39.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc1a6090 con 0x7f8cbc1061a0 2024-05-08T23:24:39.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca8004cf0 con 0x7f8cbc1061a0 2024-05-08T23:24:39.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800ce40 con 0x7f8cbc1061a0 2024-05-08T23:24:39.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.486+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1a6570 con 0x7f8cbc1061a0 2024-05-08T23:24:39.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.487+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc102550 con 0x7f8cbc1061a0 2024-05-08T23:24:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.490+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8ca8004750 con 0x7f8cbc1061a0 2024-05-08T23:24:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.491+0000 7f8cb3e00700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 0x7f8ca00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.491+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f8ca8014070 con 0x7f8cbc1061a0 2024-05-08T23:24:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.491+0000 7f8cbb400700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 0x7f8ca00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.492+0000 7f8cbb400700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 0x7f8ca00a3750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.494+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8ca8099bc0 con 0x7f8cbc1061a0 2024-05-08T23:24:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.718+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8cbc067d10 con 0x7f8cbc1061a0 2024-05-08T23:24:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.719+0000 7f8cb3e00700 1 -- 172.21.15.46:0/3592735017 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ca8004e60 con 0x7f8cbc1061a0 2024-05-08T23:24:39.719 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:24:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 msgr2=0x7f8ca00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 0x7f8ca00a3750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac00b260 comp rx=0 tx=0).stop 2024-05-08T23:24:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 msgr2=0x7f8cbc1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc1999e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e9c0 tx=0x7f8ca800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 shutdown_connections 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc19dd80 0x7f8cbc19e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ca00a1290 0x7f8ca00a3750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc101080 0x7f8cbc1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 --2- 172.21.15.46:0/3592735017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc1061a0 0x7f8cbc1999e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.721+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 >> 172.21.15.46:0/3592735017 conn(0x7f8cbc0fc8e0 msgr2=0x7f8cbc0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.722+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 shutdown_connections 2024-05-08T23:24:39.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.722+0000 7f8cc2600700 1 -- 172.21.15.46:0/3592735017 wait complete. 2024-05-08T23:24:39.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:24:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 -- 172.21.15.46:0/3616534982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c101080 msgr2=0x7f433c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 --2- 172.21.15.46:0/3616534982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c101080 0x7f433c101460 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f432c009990 tx=0x7f432c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 -- 172.21.15.46:0/3616534982 shutdown_connections 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 --2- 172.21.15.46:0/3616534982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c1061a0 0x7f433c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 --2- 172.21.15.46:0/3616534982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c101080 0x7f433c101460 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 --2- 172.21.15.46:0/3616534982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101a30 0x7f433c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 -- 172.21.15.46:0/3616534982 >> 172.21.15.46:0/3616534982 conn(0x7f433c0fc8e0 msgr2=0x7f433c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 -- 172.21.15.46:0/3616534982 shutdown_connections 2024-05-08T23:24:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.943+0000 7f4343000700 1 -- 172.21.15.46:0/3616534982 wait complete. 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.944+0000 7f4343000700 1 Processor -- start 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.944+0000 7f4343000700 1 -- start start 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 0x7f433c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 0x7f433c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c19e8d0 con 0x7f433c1061a0 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c19ea40 con 0x7f433c19de50 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4343000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c19ec00 con 0x7f433c101080 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f433b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f433b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39164/0 (socket says 172.21.15.46:39164) 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f433b400700 1 -- 172.21.15.46:0/4277916601 learned_addr learned my addr 172.21.15.46:0/4277916601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f433be00700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 0x7f433c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.945+0000 7f4341000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 0x7f433c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 -- 172.21.15.46:0/4277916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 msgr2=0x7f433c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 0x7f433c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 -- 172.21.15.46:0/4277916601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 msgr2=0x7f433c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 0x7f433c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 -- 172.21.15.46:0/4277916601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0095f0 con 0x7f433c1061a0 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433be00700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 0x7f433c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4341000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 0x7f433c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f433b400700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328009750 con 0x7f433c1061a0 2024-05-08T23:24:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4328004cf0 con 0x7f433c1061a0 2024-05-08T23:24:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c1a6160 con 0x7f433c1061a0 2024-05-08T23:24:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328005690 con 0x7f433c1061a0 2024-05-08T23:24:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.946+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c1a6640 con 0x7f433c1061a0 2024-05-08T23:24:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.947+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c102550 con 0x7f433c1061a0 2024-05-08T23:24:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.949+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f432800ccd0 con 0x7f433c1061a0 2024-05-08T23:24:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.951+0000 7f4338c00700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 0x7f43200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.952+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f4328014070 con 0x7f433c1061a0 2024-05-08T23:24:39.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.952+0000 7f433be00700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 0x7f43200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:39.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.952+0000 7f433be00700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 0x7f43200a37d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f432c006010 tx=0x7f432c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:39.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:39.956+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4328096400 con 0x7f433c1061a0 2024-05-08T23:24:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.178+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f433c101a30 con 0x7f433c1061a0 2024-05-08T23:24:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.178+0000 7f4338c00700 1 -- 172.21.15.46:0/4277916601 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4328099a20 con 0x7f433c1061a0 2024-05-08T23:24:40.180 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-08T23:24:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 msgr2=0x7f43200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 0x7f43200a37d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f432c006010 tx=0x7f432c00b260 comp rx=0 tx=0).stop 2024-05-08T23:24:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 msgr2=0x7f433c199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800edb0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 shutdown_connections 2024-05-08T23:24:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f433c19de50 0x7f433c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433c101080 0x7f433c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f43200a1310 0x7f43200a37d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 --2- 172.21.15.46:0/4277916601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c1061a0 0x7f433c199ab0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 >> 172.21.15.46:0/4277916601 conn(0x7f433c0fc8e0 msgr2=0x7f433c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 shutdown_connections 2024-05-08T23:24:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.181+0000 7f4343000700 1 -- 172.21.15.46:0/4277916601 wait complete. 2024-05-08T23:24:40.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:24:40.259 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:39 smithi046 ceph-mon[36898]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-05-08T23:24:40.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:39 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/778722418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.260 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:39 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3592735017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:39 smithi053 ceph-mon[77587]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-05-08T23:24:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:39 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/778722418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:39 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3592735017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 -- 172.21.15.46:0/3857584985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9018103280 msgr2=0x7f9018103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 --2- 172.21.15.46:0/3857584985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9018103280 0x7f9018103660 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f90080098f0 tx=0x7f9008009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 -- 172.21.15.46:0/3857584985 shutdown_connections 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 --2- 172.21.15.46:0/3857584985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90181083a0 0x7f901810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 --2- 172.21.15.46:0/3857584985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9018103280 0x7f9018103660 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 --2- 172.21.15.46:0/3857584985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103c30 0x7f9018107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 -- 172.21.15.46:0/3857584985 >> 172.21.15.46:0/3857584985 conn(0x7f90180fea80 msgr2=0x7f9018100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 -- 172.21.15.46:0/3857584985 shutdown_connections 2024-05-08T23:24:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.405+0000 7f901ce00700 1 -- 172.21.15.46:0/3857584985 wait complete. 2024-05-08T23:24:40.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.406+0000 7f901ce00700 1 Processor -- start 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 -- start start 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 0x7f9018199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901819de20 0x7f901819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901819e8a0 con 0x7f9018103280 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901819ea10 con 0x7f90181083a0 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f901ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901819ebd0 con 0x7f901819de20 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f9015400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f9015e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 0x7f9018199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f9016800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901819de20 0x7f901819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f9015400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48626/0 (socket says 172.21.15.46:48626) 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.407+0000 7f9015400700 1 -- 172.21.15.46:0/469936974 learned_addr learned my addr 172.21.15.46:0/469936974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 -- 172.21.15.46:0/469936974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901819de20 msgr2=0x7f901819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901819de20 0x7f901819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 -- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 msgr2=0x7f9018199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 0x7f9018199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 -- 172.21.15.46:0/469936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040095f0 con 0x7f90181083a0 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015e00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 0x7f9018199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:40.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f9015400700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f900400ec70 tx=0x7f900400ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040099d0 con 0x7f90181083a0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9008009520 con 0x7f90181083a0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.408+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90040044e0 con 0x7f90181083a0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.409+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004010430 con 0x7f90181083a0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.409+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181a6480 con 0x7f90181083a0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.410+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901810b5c0 con 0x7f90181083a0 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.412+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9004003f10 con 0x7f90181083a0 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.413+0000 7f900ea00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 0x7f8ffc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.413+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f9004014070 con 0x7f90181083a0 2024-05-08T23:24:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.414+0000 7f9015e00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 0x7f8ffc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.414+0000 7f9015e00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 0x7f8ffc0a3750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90080098c0 tx=0x7f9008019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.418+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f90040965e0 con 0x7f90181083a0 2024-05-08T23:24:40.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:39 smithi086 ceph-mon[78610]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 11 op/s 2024-05-08T23:24:40.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:39 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/778722418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:39 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3592735017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:24:40.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.637+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9018103c30 con 0x7f90181083a0 2024-05-08T23:24:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.637+0000 7f900ea00700 1 -- 172.21.15.46:0/469936974 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9004099c00 con 0x7f90181083a0 2024-05-08T23:24:40.638 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-08T23:24:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 msgr2=0x7f8ffc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 0x7f8ffc0a3750 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90080098c0 tx=0x7f9008019040 comp rx=0 tx=0).stop 2024-05-08T23:24:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 msgr2=0x7f9018199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f900400ec70 tx=0x7f900400ef80 comp rx=0 tx=0).stop 2024-05-08T23:24:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 shutdown_connections 2024-05-08T23:24:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901819de20 0x7f901819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f8ffc0a1290 0x7f8ffc0a3750 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9018103280 0x7f9018199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 --2- 172.21.15.46:0/469936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90181083a0 0x7f9018199ad0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 >> 172.21.15.46:0/469936974 conn(0x7f90180fea80 msgr2=0x7f90180ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 shutdown_connections 2024-05-08T23:24:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.640+0000 7f901ce00700 1 -- 172.21.15.46:0/469936974 wait complete. 2024-05-08T23:24:40.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 -- 172.21.15.46:0/3035473823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c101080 msgr2=0x7f815c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 --2- 172.21.15.46:0/3035473823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c101080 0x7f815c101460 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f814c009990 tx=0x7f814c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 -- 172.21.15.46:0/3035473823 shutdown_connections 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 --2- 172.21.15.46:0/3035473823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c1061a0 0x7f815c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 --2- 172.21.15.46:0/3035473823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c101080 0x7f815c101460 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 --2- 172.21.15.46:0/3035473823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c101a30 0x7f815c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 -- 172.21.15.46:0/3035473823 >> 172.21.15.46:0/3035473823 conn(0x7f815c0fc8e0 msgr2=0x7f815c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 -- 172.21.15.46:0/3035473823 shutdown_connections 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.865+0000 7f8161000700 1 -- 172.21.15.46:0/3035473823 wait complete. 2024-05-08T23:24:40.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 Processor -- start 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 -- start start 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c101080 0x7f815c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 0x7f815c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c19e8d0 con 0x7f815c19de50 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c19ea40 con 0x7f815c1061a0 2024-05-08T23:24:40.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.866+0000 7f8161000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c19ec00 con 0x7f815c101080 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 0x7f815c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f8159400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f8159e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c101080 0x7f815c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f8159400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:48636/0 (socket says 172.21.15.46:48636) 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f8159400700 1 -- 172.21.15.46:0/996496308 learned_addr learned my addr 172.21.15.46:0/996496308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 -- 172.21.15.46:0/996496308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c101080 msgr2=0x7f815c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c101080 0x7f815c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 -- 172.21.15.46:0/996496308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 msgr2=0x7f815c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 -- 172.21.15.46:0/996496308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0095f0 con 0x7f815c19de50 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f8159400700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.867+0000 7f815a800700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 0x7f815c19e2d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8154007f10 tx=0x7f815400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.868+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154004db0 con 0x7f815c19de50 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.868+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815400f040 con 0x7f815c19de50 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.868+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c1a6160 con 0x7f815c19de50 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.868+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154005990 con 0x7f815c19de50 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.868+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c1a65f0 con 0x7f815c19de50 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.869+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c102550 con 0x7f815c19de50 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.871+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f815400db20 con 0x7f815c19de50 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.872+0000 7f8152a00700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 0x7f81400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.872+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f81540cd1e0 con 0x7f815c19de50 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.872+0000 7f8159e00700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 0x7f81400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.873+0000 7f8159e00700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 0x7f81400a3750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b5c0 tx=0x7f814c005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:40.875+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f81540961c0 con 0x7f815c19de50 2024-05-08T23:24:41.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.112+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f815c101a30 con 0x7f815c19de50 2024-05-08T23:24:41.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.113+0000 7f8152a00700 1 -- 172.21.15.46:0/996496308 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v115) v1 ==== 81+0+999 (secure 0 0 0) 0x7f8154023020 con 0x7f815c19de50 2024-05-08T23:24:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:24:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4047035481 2024-05-09T22:56:16.844189+0000 2024-05-08T23:24:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3841409271 2024-05-09T22:55:52.116773+0000 2024-05-08T23:24:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/580235593 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1980881468 2024-05-09T22:55:52.116773+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2056989721 2024-05-09T22:56:16.844189+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/387189775 2024-05-09T22:55:52.116773+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/652393065 2024-05-09T22:56:16.844189+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/664114694 2024-05-09T22:55:52.116773+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2015473581 2024-05-09T22:56:16.844189+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3800995796 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2265080110 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2731582335 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.116 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1222248037 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.116 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2731582335 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.116 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1289022895 2024-05-09T23:03:48.069677+0000 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 msgr2=0x7f81400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 0x7f81400a3750 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b5c0 tx=0x7f814c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 msgr2=0x7f815c19e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 0x7f815c19e2d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8154007f10 tx=0x7f815400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 shutdown_connections 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f81400a1290 0x7f81400a3750 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f815c19de50 0x7f815c19e2d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815c101080 0x7f815c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 --2- 172.21.15.46:0/996496308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f815c1061a0 0x7f815c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.115+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 >> 172.21.15.46:0/996496308 conn(0x7f815c0fc8e0 msgr2=0x7f815c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.116+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 shutdown_connections 2024-05-08T23:24:41.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:41.116+0000 7f8161000700 1 -- 172.21.15.46:0/996496308 wait complete. 2024-05-08T23:24:41.119 INFO:teuthology.orchestra.run.smithi046.stderr:listed 17 entries 2024-05-08T23:24:41.137 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:24:41.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:41.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:24:41.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:40 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4277916601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:40 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/469936974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:40 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4277916601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:40 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/469936974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.472 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:40 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4277916601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.472 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:40 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/469936974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:24:41.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:24:41.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:41.624 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:24:41.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:41.680 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-08T23:24:41.732 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:24:41.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:41.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:24:41.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:24:41.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-08T23:24:41.901 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T23:24:41.914 INFO:tasks.cephadm:Teardown begin 2024-05-08T23:24:41.914 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:24:41.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.119+0000 7f48fe600700 1 -- 172.21.15.46:0/3277213290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 msgr2=0x7f48f81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.119+0000 7f48fe600700 1 --2- 172.21.15.46:0/3277213290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f81023a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f48e80098f0 tx=0x7f48e8009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 -- 172.21.15.46:0/3277213290 shutdown_connections 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 --2- 172.21.15.46:0/3277213290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f810b3e0 0x7f48f810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 --2- 172.21.15.46:0/3277213290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f81023a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 --2- 172.21.15.46:0/3277213290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f81028e0 0x7f48f810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 -- 172.21.15.46:0/3277213290 >> 172.21.15.46:0/3277213290 conn(0x7f48f80fb820 msgr2=0x7f48f80fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 -- 172.21.15.46:0/3277213290 shutdown_connections 2024-05-08T23:24:42.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.120+0000 7f48fe600700 1 -- 172.21.15.46:0/3277213290 wait complete. 2024-05-08T23:24:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.121+0000 7f48fe600700 1 Processor -- start 2024-05-08T23:24:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.121+0000 7f48fe600700 1 -- start start 2024-05-08T23:24:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 0x7f48f8199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f819e3e0 con 0x7f48f810b3e0 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f819e550 con 0x7f48f8101fc0 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48fe600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f819e710 con 0x7f48f81028e0 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58886/0 (socket says 172.21.15.46:58886) 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f7e00700 1 -- 172.21.15.46:0/3346241309 learned_addr learned my addr 172.21.15.46:0/3346241309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52682/0 (socket says 172.21.15.46:52682) 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.122+0000 7f48f6a00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 0x7f48f8199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 -- 172.21.15.46:0/3346241309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 msgr2=0x7f48f8199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 0x7f48f8199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 -- 172.21.15.46:0/3346241309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 msgr2=0x7f48f8199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 -- 172.21.15.46:0/3346241309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40095f0 con 0x7f48f810b3e0 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7400700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f6a00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 0x7f48f8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48f7e00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f48f000ef30 tx=0x7f48f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0018070 con 0x7f48f810b3e0 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e8009520 con 0x7f48f810b3e0 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f000f040 con 0x7f48f810b3e0 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f81a6050 con 0x7f48f810b3e0 2024-05-08T23:24:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.123+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000ccc0 con 0x7f48f810b3e0 2024-05-08T23:24:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.124+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8068a40 con 0x7f48f810b3e0 2024-05-08T23:24:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.126+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f48f0003a00 con 0x7f48f810b3e0 2024-05-08T23:24:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.128+0000 7f48efe00700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 0x7f48dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.128+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f48f00cd5e0 con 0x7f48f810b3e0 2024-05-08T23:24:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.128+0000 7f48f7400700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 0x7f48dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.128+0000 7f48f7400700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 0x7f48dc0a3750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009fd0 tx=0x7f48e8019040 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.130+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48f0096650 con 0x7f48f810b3e0 2024-05-08T23:24:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:42 smithi053 ceph-mon[77587]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-05-08T23:24:42.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:42 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/996496308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:24:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:42 smithi046 ceph-mon[36898]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-05-08T23:24:42.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:42 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/996496308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:24:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.400+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48f8067d10 con 0x7f48f810b3e0 2024-05-08T23:24:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.401+0000 7f48efe00700 1 -- 172.21.15.46:0/3346241309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7f48f0025380 con 0x7f48f810b3e0 2024-05-08T23:24:42.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:42.402 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:24:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.403+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 msgr2=0x7f48dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.403+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 0x7f48dc0a3750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009fd0 tx=0x7f48e8019040 comp rx=0 tx=0).stop 2024-05-08T23:24:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.403+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 msgr2=0x7f48f819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.403+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f48f000ef30 tx=0x7f48f000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 shutdown_connections 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48dc0a1290 0x7f48dc0a3750 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f810b3e0 0x7f48f819dda0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f8101fc0 0x7f48f8199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 --2- 172.21.15.46:0/3346241309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f81028e0 0x7f48f8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 >> 172.21.15.46:0/3346241309 conn(0x7f48f80fb820 msgr2=0x7f48f80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 shutdown_connections 2024-05-08T23:24:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.404+0000 7f48fe600700 1 -- 172.21.15.46:0/3346241309 wait complete. 2024-05-08T23:24:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:24:42.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:42.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:42 smithi086 ceph-mon[78610]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-05-08T23:24:42.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:42 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/996496308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:24:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.623+0000 7fc267200700 1 -- 172.21.15.46:0/1855793242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260101a20 msgr2=0x7fc260105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.623+0000 7fc267200700 1 --2- 172.21.15.46:0/1855793242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260101a20 0x7fc260105a60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009990 tx=0x7fc24c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 -- 172.21.15.46:0/1855793242 shutdown_connections 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 --2- 172.21.15.46:0/1855793242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc260106190 0x7fc26010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 --2- 172.21.15.46:0/1855793242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 0x7fc260101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 --2- 172.21.15.46:0/1855793242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260101a20 0x7fc260105a60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 -- 172.21.15.46:0/1855793242 >> 172.21.15.46:0/1855793242 conn(0x7fc2600fc930 msgr2=0x7fc2600fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 -- 172.21.15.46:0/1855793242 shutdown_connections 2024-05-08T23:24:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.624+0000 7fc267200700 1 -- 172.21.15.46:0/1855793242 wait complete. 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 Processor -- start 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 -- start start 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 0x7fc260199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26019de60 0x7fc26019e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019e8e0 con 0x7fc260106190 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019ea50 con 0x7fc26019de60 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc267200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019ec10 con 0x7fc260101070 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc25f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.625+0000 7fc25fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 0x7fc260199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58908/0 (socket says 172.21.15.46:58908) 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 -- 172.21.15.46:0/2608538588 learned_addr learned my addr 172.21.15.46:0/2608538588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc265200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26019de60 0x7fc26019e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 -- 172.21.15.46:0/2608538588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 msgr2=0x7fc260199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 0x7fc260199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 -- 172.21.15.46:0/2608538588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26019de60 msgr2=0x7fc26019e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26019de60 0x7fc26019e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 -- 172.21.15.46:0/2608538588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0095f0 con 0x7fc260106190 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25f400700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc24c000c00 tx=0x7fc24c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c01d070 con 0x7fc260106190 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc24c021410 con 0x7fc260106190 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c017650 con 0x7fc260106190 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.626+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2601a6110 con 0x7fc260106190 2024-05-08T23:24:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.627+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2601a64e0 con 0x7fc260106190 2024-05-08T23:24:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.628+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc260102540 con 0x7fc260106190 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.630+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc24c021580 con 0x7fc260106190 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.631+0000 7fc25cc00700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 0x7fc2440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.631+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7fc24c0ce960 con 0x7fc260106190 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.631+0000 7fc25fe00700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 0x7fc2440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.632+0000 7fc25fe00700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 0x7fc2440a37d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc250009c60 tx=0x7fc250009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.633+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc24c0979d0 con 0x7fc260106190 2024-05-08T23:24:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.904+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc260067d10 con 0x7fc260106190 2024-05-08T23:24:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.905+0000 7fc25cc00700 1 -- 172.21.15.46:0/2608538588 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10904 (secure 0 0 0) 0x7fc24c09aff0 con 0x7fc260106190 2024-05-08T23:24:42.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3787979221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3787979221},{"type":"v1","addr":"172.21.15.46:6835","nonce":3787979221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1402919380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1402919380},{"type":"v1","addr":"172.21.15.46:6841","nonce":1402919380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:01:51.460837+0000","modified":"2024-05-08T23:20:08.424396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24583,"mds_2":24571,"mds_3":24547,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/3863967478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3863967478},{"type":"v1","addr":"172.21.15.53:6833","nonce":3863967478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/272583724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":272583724},{"type":"v1","addr":"172.21.15.86:6833","nonce":272583724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6837/1812878749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1812878749},{"type":"v1","addr":"172.21.15.46:6837","nonce":1812878749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1845130080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1845130080},{"type":"v1","addr":"172.21.15.53:6835","nonce":1845130080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/170717769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":170717769},{"type":"v1","addr":"172.21.15.53:6837","nonce":170717769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6835/2919086413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2919086413},{"type":"v1","addr":"172.21.15.86:6835","nonce":2919086413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/2553915750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2553915750},{"type":"v1","addr":"172.21.15.86:6837","nonce":2553915750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.86:6839/4115816206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":4115816206},{"type":"v1","addr":"172.21.15.86:6839","nonce":4115816206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24583":{"gid":24583,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6839/1484730309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1484730309},{"type":"v1","addr":"172.21.15.46:6839","nonce":1484730309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3374696949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3374696949},{"type":"v1","addr":"172.21.15.53:6839","nonce":3374696949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:24:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 msgr2=0x7fc2440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 0x7fc2440a37d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc250009c60 tx=0x7fc250009260 comp rx=0 tx=0).stop 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 msgr2=0x7fc260199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc24c000c00 tx=0x7fc24c004730 comp rx=0 tx=0).stop 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 shutdown_connections 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26019de60 0x7fc26019e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260101070 0x7fc260199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fc2440a1310 0x7fc2440a37d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 --2- 172.21.15.46:0/2608538588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc260106190 0x7fc260199ac0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 >> 172.21.15.46:0/2608538588 conn(0x7fc2600fc930 msgr2=0x7fc2600fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 shutdown_connections 2024-05-08T23:24:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:42.908+0000 7fc267200700 1 -- 172.21.15.46:0/2608538588 wait complete. 2024-05-08T23:24:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-05-08T23:24:42.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:24:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 -- 172.21.15.46:0/2941071743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073340 msgr2=0x7faec4073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 --2- 172.21.15.46:0/2941071743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073340 0x7faec4073720 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009990 tx=0x7faeb4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 -- 172.21.15.46:0/2941071743 shutdown_connections 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 --2- 172.21.15.46:0/2941071743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 0x7faec4114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 --2- 172.21.15.46:0/2941071743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073340 0x7faec4073720 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 --2- 172.21.15.46:0/2941071743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073c60 0x7faec41116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 -- 172.21.15.46:0/2941071743 >> 172.21.15.46:0/2941071743 conn(0x7faec40fc730 msgr2=0x7faec40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 -- 172.21.15.46:0/2941071743 shutdown_connections 2024-05-08T23:24:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.128+0000 7faec3200700 1 -- 172.21.15.46:0/2941071743 wait complete. 2024-05-08T23:24:43.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.129+0000 7faec3200700 1 Processor -- start 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 -- start start 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073340 0x7faec419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 0x7faec41a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41a1590 con 0x7faec4073c60 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41a1700 con 0x7faec4111c30 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41a18c0 con 0x7faec4073340 2024-05-08T23:24:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faebbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faebbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58920/0 (socket says 172.21.15.46:58920) 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faebbe00700 1 -- 172.21.15.46:0/3907846168 learned_addr learned my addr 172.21.15.46:0/3907846168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec1e00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073340 0x7faec419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.130+0000 7faec2800700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 0x7faec41a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 -- 172.21.15.46:0/3907846168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073340 msgr2=0x7faec419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073340 0x7faec419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 -- 172.21.15.46:0/3907846168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 msgr2=0x7faec41a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 0x7faec41a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 -- 172.21.15.46:0/3907846168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb40095f0 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebbe00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7faeac00e9c0 tx=0x7faeac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0097b0 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeac00ee60 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec41a1ba0 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0054f0 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.131+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec41aa500 con 0x7faec4073c60 2024-05-08T23:24:43.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.132+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec4104130 con 0x7faec4073c60 2024-05-08T23:24:43.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.134+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faeac004750 con 0x7faec4073c60 2024-05-08T23:24:43.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.135+0000 7faebaa00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 0x7faea80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.136+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7faeac014070 con 0x7faec4073c60 2024-05-08T23:24:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.136+0000 7faec1e00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 0x7faea80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.136+0000 7faec1e00700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 0x7faea80a3750 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faeb400b2e0 tx=0x7faeb4005fd0 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.139+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faeac0963a0 con 0x7faec4073c60 2024-05-08T23:24:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:43 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3346241309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.262 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:43 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/2608538588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:43 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3346241309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:43 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/2608538588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.360+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faec419ea60 con 0x7faec4073c60 2024-05-08T23:24:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.362+0000 7faebaa00700 1 -- 172.21.15.46:0/3907846168 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+25250 (secure 0 0 0) 0x7faeac0999c0 con 0x7faec4073c60 2024-05-08T23:24:43.363 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:24:43.363 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":115,"fsid":"ebe57712-0d8d-11ef-bc97-c7b262605968","created":"2024-05-08T22:55:20.134668+0000","modified":"2024-05-08T23:24:14.936136+0000","last_up_change":"2024-05-08T23:01:12.605862+0000","last_in_change":"2024-05-08T23:00:55.862570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:58:17.469909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:01:49.619332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:01:50.925439+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:01:54.477718+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f77bd5d5-1794-4093-becb-18c682bf3da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6803","nonce":2436508778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6805","nonce":2436508778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6809","nonce":2436508778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2436508778},{"type":"v1","addr":"172.21.15.46:6807","nonce":2436508778}]},"public_addr":"172.21.15.46:6803/2436508778","cluster_addr":"172.21.15.46:6805/2436508778","heartbeat_back_addr":"172.21.15.46:6809/2436508778","heartbeat_front_addr":"172.21.15.46:6807/2436508778","state":["exists","up"]},{"osd":1,"uuid":"c389d713-72f0-4a78-854e-85a7a352ee10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6801","nonce":3799790044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6803","nonce":3799790044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6807","nonce":3799790044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3799790044},{"type":"v1","addr":"172.21.15.53:6805","nonce":3799790044}]},"public_addr":"172.21.15.53:6801/3799790044","cluster_addr":"172.21.15.53:6803/3799790044","heartbeat_back_addr":"172.21.15.53:6807/3799790044","heartbeat_front_addr":"172.21.15.53:6805/3799790044","state":["exists","up"]},{"osd":2,"uuid":"5089fe37-bb4d-41e9-92a2-5fdb7c7f0c68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6801","nonce":1780351335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6803","nonce":1780351335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6807","nonce":1780351335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1780351335},{"type":"v1","addr":"172.21.15.86:6805","nonce":1780351335}]},"public_addr":"172.21.15.86:6801/1780351335","cluster_addr":"172.21.15.86:6803/1780351335","heartbeat_back_addr":"172.21.15.86:6807/1780351335","heartbeat_front_addr":"172.21.15.86:6805/1780351335","state":["exists","up"]},{"osd":3,"uuid":"d63af76a-815c-4170-ab60-6ed7dbe596ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6811","nonce":1550587514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6813","nonce":1550587514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6817","nonce":1550587514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1550587514},{"type":"v1","addr":"172.21.15.46:6815","nonce":1550587514}]},"public_addr":"172.21.15.46:6811/1550587514","cluster_addr":"172.21.15.46:6813/1550587514","heartbeat_back_addr":"172.21.15.46:6817/1550587514","heartbeat_front_addr":"172.21.15.46:6815/1550587514","state":["exists","up"]},{"osd":4,"uuid":"3ca7a742-9c35-448a-9287-63b6fe77a632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6809","nonce":2784307263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6811","nonce":2784307263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6815","nonce":2784307263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2784307263},{"type":"v1","addr":"172.21.15.53:6813","nonce":2784307263}]},"public_addr":"172.21.15.53:6809/2784307263","cluster_addr":"172.21.15.53:6811/2784307263","heartbeat_back_addr":"172.21.15.53:6815/2784307263","heartbeat_front_addr":"172.21.15.53:6813/2784307263","state":["exists","up"]},{"osd":5,"uuid":"c0ef3604-5955-42ac-b4f7-5d211ef7d553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6809","nonce":2217036731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6811","nonce":2217036731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6815","nonce":2217036731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2217036731},{"type":"v1","addr":"172.21.15.86:6813","nonce":2217036731}]},"public_addr":"172.21.15.86:6809/2217036731","cluster_addr":"172.21.15.86:6811/2217036731","heartbeat_back_addr":"172.21.15.86:6815/2217036731","heartbeat_front_addr":"172.21.15.86:6813/2217036731","state":["exists","up"]},{"osd":6,"uuid":"9977b050-bdba-4adc-8d0a-8ee2935643bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6819","nonce":1222901662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6821","nonce":1222901662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6825","nonce":1222901662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1222901662},{"type":"v1","addr":"172.21.15.46:6823","nonce":1222901662}]},"public_addr":"172.21.15.46:6819/1222901662","cluster_addr":"172.21.15.46:6821/1222901662","heartbeat_back_addr":"172.21.15.46:6825/1222901662","heartbeat_front_addr":"172.21.15.46:6823/1222901662","state":["exists","up"]},{"osd":7,"uuid":"1305daf7-451a-42b5-bc27-04165d5de6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6817","nonce":521136929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6819","nonce":521136929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6823","nonce":521136929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":521136929},{"type":"v1","addr":"172.21.15.53:6821","nonce":521136929}]},"public_addr":"172.21.15.53:6817/521136929","cluster_addr":"172.21.15.53:6819/521136929","heartbeat_back_addr":"172.21.15.53:6823/521136929","heartbeat_front_addr":"172.21.15.53:6821/521136929","state":["exists","up"]},{"osd":8,"uuid":"a79a15f0-16af-47ba-af13-ceb2e1df5f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6817","nonce":4052399727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6819","nonce":4052399727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6823","nonce":4052399727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4052399727},{"type":"v1","addr":"172.21.15.86:6821","nonce":4052399727}]},"public_addr":"172.21.15.86:6817/4052399727","cluster_addr":"172.21.15.86:6819/4052399727","heartbeat_back_addr":"172.21.15.86:6823/4052399727","heartbeat_front_addr":"172.21.15.86:6821/4052399727","state":["exists","up"]},{"osd":9,"uuid":"885ac456-a429-4688-b3f7-c66c1d785c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6827","nonce":2347234971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6829","nonce":2347234971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6833","nonce":2347234971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2347234971},{"type":"v1","addr":"172.21.15.46:6831","nonce":2347234971}]},"public_addr":"172.21.15.46:6827/2347234971","cluster_addr":"172.21.15.46:6829/2347234971","heartbeat_back_addr":"172.21.15.46:6833/2347234971","heartbeat_front_addr":"172.21.15.46:6831/2347234971","state":["exists","up"]},{"osd":10,"uuid":"66e06a38-ad3e-4b9d-af1f-8f62447ec12a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6825","nonce":75118943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6827","nonce":75118943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6831","nonce":75118943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":75118943},{"type":"v1","addr":"172.21.15.53:6829","nonce":75118943}]},"public_addr":"172.21.15.53:6825/75118943","cluster_addr":"172.21.15.53:6827/75118943","heartbeat_back_addr":"172.21.15.53:6831/75118943","heartbeat_front_addr":"172.21.15.53:6829/75118943","state":["exists","up"]},{"osd":11,"uuid":"2b97781b-2ef0-485c-b42b-47099ce3ff54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6825","nonce":3801696519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6827","nonce":3801696519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6831","nonce":3801696519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3801696519},{"type":"v1","addr":"172.21.15.86:6829","nonce":3801696519}]},"public_addr":"172.21.15.86:6825/3801696519","cluster_addr":"172.21.15.86:6827/3801696519","heartbeat_back_addr":"172.21.15.86:6831/3801696519","heartbeat_front_addr":"172.21.15.86:6829/3801696519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:36.031562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:57:55.764861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:14.030595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:34.932674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:58:54.158676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:12.538763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:33.977660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:59:53.523075+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:11.840536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:33.902767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:00:52.705499+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:01:10.617591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/4047035481":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3841409271":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/580235593":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1980881468":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6801/2056989721":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/387189775":"2024-05-09T22:55:52.116773+0000","172.21.15.46:6800/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/652393065":"2024-05-09T22:56:16.844189+0000","172.21.15.46:6801/664114694":"2024-05-09T22:55:52.116773+0000","172.21.15.46:0/2015473581":"2024-05-09T22:56:16.844189+0000","172.21.15.46:0/3800995796":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/2265080110":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6801/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1222248037":"2024-05-09T23:03:48.069677+0000","172.21.15.46:6800/2731582335":"2024-05-09T23:03:48.069677+0000","172.21.15.46:0/1289022895":"2024-05-09T23:03:48.069677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:24:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 msgr2=0x7faea80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 0x7faea80a3750 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faeb400b2e0 tx=0x7faeb4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 msgr2=0x7faec419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7faeac00e9c0 tx=0x7faeac00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 shutdown_connections 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec4111c30 0x7faec41a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faec4073340 0x7faec419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7faea80a1290 0x7faea80a3750 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 --2- 172.21.15.46:0/3907846168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4073c60 0x7faec419dcf0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 >> 172.21.15.46:0/3907846168 conn(0x7faec40fc730 msgr2=0x7faec40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 shutdown_connections 2024-05-08T23:24:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.365+0000 7faec3200700 1 -- 172.21.15.46:0/3907846168 wait complete. 2024-05-08T23:24:43.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:24:43.463 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:43 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3346241309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:43 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/2608538588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 -- 172.21.15.46:0/1376847953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8101870 msgr2=0x7f48d81058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 --2- 172.21.15.46:0/1376847953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8101870 0x7f48d81058b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f48c40098f0 tx=0x7f48c4009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 -- 172.21.15.46:0/1376847953 shutdown_connections 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 --2- 172.21.15.46:0/1376847953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d8105fe0 0x7f48d810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 --2- 172.21.15.46:0/1376847953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 0x7f48d81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 --2- 172.21.15.46:0/1376847953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8101870 0x7f48d81058b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 -- 172.21.15.46:0/1376847953 >> 172.21.15.46:0/1376847953 conn(0x7f48d80fc740 msgr2=0x7f48d80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.584+0000 7f48dd200700 1 -- 172.21.15.46:0/1376847953 shutdown_connections 2024-05-08T23:24:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.585+0000 7f48dd200700 1 -- 172.21.15.46:0/1376847953 wait complete. 2024-05-08T23:24:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.586+0000 7f48dd200700 1 Processor -- start 2024-05-08T23:24:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.586+0000 7f48dd200700 1 -- start start 2024-05-08T23:24:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 0x7f48d8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8105fe0 0x7f48d8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819e950 con 0x7f48d819ded0 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819eac0 con 0x7f48d8100ec0 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48dd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d819ec80 con 0x7f48d8105fe0 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58928/0 (socket says 172.21.15.46:58928) 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 0x7f48d8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 -- 172.21.15.46:0/1350608997 learned_addr learned my addr 172.21.15.46:0/1350608997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d6a00700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8105fe0 0x7f48d8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 -- 172.21.15.46:0/1350608997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8105fe0 msgr2=0x7f48d8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8105fe0 0x7f48d8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 -- 172.21.15.46:0/1350608997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 msgr2=0x7f48d8199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 0x7f48d8199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.587+0000 7f48d7e00700 1 -- 172.21.15.46:0/1350608997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0009d00 con 0x7f48d819ded0 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48d7e00700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f48d000e3c0 tx=0x7f48d000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00090d0 con 0x7f48d819ded0 2024-05-08T23:24:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d000f040 con 0x7f48d819ded0 2024-05-08T23:24:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c4009520 con 0x7f48d819ded0 2024-05-08T23:24:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0007c20 con 0x7f48d819ded0 2024-05-08T23:24:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.588+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d81a6530 con 0x7f48d819ded0 2024-05-08T23:24:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.589+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d8102390 con 0x7f48d819ded0 2024-05-08T23:24:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.593+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f48d0003a00 con 0x7f48d819ded0 2024-05-08T23:24:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.594+0000 7f48cfe00700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 0x7f48bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.594+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9554+0+0 (secure 0 0 0) 0x7f48d00cd340 con 0x7f48d819ded0 2024-05-08T23:24:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.594+0000 7f48d7400700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 0x7f48bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.595+0000 7f48d7400700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 0x7f48bc0a37f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009c60 tx=0x7f48c8009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.596+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48d00963b0 con 0x7f48d819ded0 2024-05-08T23:24:43.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.862+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f48d806a680 con 0x7f48d819ded0 2024-05-08T23:24:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.875+0000 7f48cfe00700 1 -- 172.21.15.46:0/1350608997 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48d00999d0 con 0x7f48d819ded0 2024-05-08T23:24:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.877+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 msgr2=0x7f48bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.877+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 0x7f48bc0a37f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009c60 tx=0x7f48c8009260 comp rx=0 tx=0).stop 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.877+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 msgr2=0x7f48d819e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.877+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f48d000e3c0 tx=0x7f48d000e6d0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 shutdown_connections 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f48bc0a1330 0x7f48bc0a37f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d819ded0 0x7f48d819e350 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d8100ec0 0x7f48d8199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 --2- 172.21.15.46:0/1350608997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d8105fe0 0x7f48d8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 >> 172.21.15.46:0/1350608997 conn(0x7f48d80fc740 msgr2=0x7f48d80fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 shutdown_connections 2024-05-08T23:24:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:43.878+0000 7f48dd200700 1 -- 172.21.15.46:0/1350608997 wait complete. 2024-05-08T23:24:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:24:43.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:24:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.115+0000 7fbc1f800700 1 -- 172.21.15.46:0/731237122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 msgr2=0x7fbc18101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.115+0000 7fbc1f800700 1 --2- 172.21.15.46:0/731237122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc18101460 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009990 tx=0x7fbc0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 -- 172.21.15.46:0/731237122 shutdown_connections 2024-05-08T23:24:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 --2- 172.21.15.46:0/731237122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 0x7fbc1810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 --2- 172.21.15.46:0/731237122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc18101460 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 --2- 172.21.15.46:0/731237122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18101a30 0x7fbc18105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 -- 172.21.15.46:0/731237122 >> 172.21.15.46:0/731237122 conn(0x7fbc180fc8e0 msgr2=0x7fbc180fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 -- 172.21.15.46:0/731237122 shutdown_connections 2024-05-08T23:24:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.116+0000 7fbc1f800700 1 -- 172.21.15.46:0/731237122 wait complete. 2024-05-08T23:24:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 Processor -- start 2024-05-08T23:24:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 -- start start 2024-05-08T23:24:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 0x7fbc180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181a5b30 con 0x7fbc18073d20 2024-05-08T23:24:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181a5c70 con 0x7fbc181061a0 2024-05-08T23:24:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.117+0000 7fbc1f800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181a5e30 con 0x7fbc18101080 2024-05-08T23:24:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58988/0 (socket says 172.21.15.46:58988) 2024-05-08T23:24:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc180732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55930/0 (socket says 172.21.15.46:55930) 2024-05-08T23:24:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 -- 172.21.15.46:0/3513598690 learned_addr learned my addr 172.21.15.46:0/3513598690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc17e00700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 0x7fbc180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 -- 172.21.15.46:0/3513598690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 msgr2=0x7fbc180732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc180732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 -- 172.21.15.46:0/3513598690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 msgr2=0x7fbc180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 0x7fbc180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 -- 172.21.15.46:0/3513598690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0095f0 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc1d800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000ef30 tx=0x7fbc1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.118+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10018070 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.119+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc1000f040 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.119+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc181a60b0 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.119+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000cd00 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.119+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc181a65c0 con 0x7fbc18073d20 2024-05-08T23:24:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.120+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc181093c0 con 0x7fbc18073d20 2024-05-08T23:24:44.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.122+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc10003a00 con 0x7fbc18073d20 2024-05-08T23:24:44.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.123+0000 7fbc15600700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 0x7fbc000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:44.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.123+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9896+0+0 (secure 0 0 0) 0x7fbc100cd700 con 0x7fbc18073d20 2024-05-08T23:24:44.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.123+0000 7fbc1ce00700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 0x7fbc000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.124+0000 7fbc1ce00700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 0x7fbc000a37d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c00b260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.125+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc10096620 con 0x7fbc18073d20 2024-05-08T23:24:44.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.214 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: starting mds.c at 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:24:43 smithi086 ceph-mds[116114]: -- 172.21.15.86:0/3075332212 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc2475e1a0 con 0x55cc24734c00 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: starting mds.i at 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-05-08T23:24:44.215 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3907846168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: osdmap e116: 12 total, 12 up, 12 in 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:24:44.216 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:44 smithi086 ceph-mon[78610]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: starting mds.f at 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.217 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:24:43 smithi086 ceph-mds[115094]: -- 172.21.15.86:0/2775458973 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e5e3461a0 con 0x560e5e31d000 2024-05-08T23:24:44.218 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:24:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: starting mds.l at 2024-05-08T23:24:44.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.263 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: starting mds.b at 2024-05-08T23:24:44.263 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.263 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.263 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: starting mds.e at 2024-05-08T23:24:44.263 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: 2024-05-08T23:24:43.924+0000 7fd4aa0b9700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-08T23:24:44.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-05-08T23:24:44.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3907846168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:44.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:24:44.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:24:44.264 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:24:44.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: osdmap e116: 12 total, 12 up, 12 in 2024-05-08T23:24:44.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:24:44.265 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:44 smithi053 ceph-mon[77587]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T23:24:44.265 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:24:43 smithi053 ceph-mds[113992]: -- 172.21.15.53:0/3418714954 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56341e3a21a0 con 0x56341e378800 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: starting mds.h at 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.266 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:24:43 smithi053 ceph-mds[113306]: -- 172.21.15.53:0/3712681786 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e13cddc1a0 con 0x55e13cdb2800 2024-05-08T23:24:44.267 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:24:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: starting mds.k at 2024-05-08T23:24:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a[36875]: 2024-05-08T23:24:43.863+0000 7fe698200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:24:44.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3907846168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: osdmap e116: 12 total, 12 up, 12 in 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1350608997' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:24:44.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:44 smithi046 ceph-mon[36898]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T23:24:44.279 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.279 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.279 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:24:43 smithi046 ceph-mds[87912]: -- 172.21.15.46:0/387874101 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557ae489a1a0 con 0x557ae4872800 2024-05-08T23:24:44.279 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: starting mds.d at 2024-05-08T23:24:44.279 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: ignoring --setuser ceph since I am not root 2024-05-08T23:24:44.280 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: ignoring --setgroup ceph since I am not root 2024-05-08T23:24:44.280 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:24:43 smithi046 ceph-mds[88479]: -- 172.21.15.46:0/914136050 <== mon.1 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9642201a0 con 0x55e9641f8800 2024-05-08T23:24:44.280 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:24:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: starting mds.g at 2024-05-08T23:24:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.400+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc180747d0 con 0x7fbc18073d20 2024-05-08T23:24:44.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.879+0000 7fbc15600700 1 -- 172.21.15.46:0/3513598690 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc10099c40 con 0x7fbc18073d20 2024-05-08T23:24:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 msgr2=0x7fbc000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 0x7fbc000a37d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c00b260 comp rx=0 tx=0).stop 2024-05-08T23:24:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 msgr2=0x7fbc181a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000ef30 tx=0x7fbc1000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 shutdown_connections 2024-05-08T23:24:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fbc000a1310 0x7fbc000a37d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18073d20 0x7fbc181a5560 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc18101080 0x7fbc180732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 --2- 172.21.15.46:0/3513598690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc181061a0 0x7fbc180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 >> 172.21.15.46:0/3513598690 conn(0x7fbc180fc8e0 msgr2=0x7fbc180fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 shutdown_connections 2024-05-08T23:24:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:44.882+0000 7fbc1f800700 1 -- 172.21.15.46:0/3513598690 wait complete. 2024-05-08T23:24:44.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:24:45.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 -- 172.21.15.46:0/3478726894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28241028c0 msgr2=0x7f282410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 --2- 172.21.15.46:0/3478726894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28241028c0 0x7f282410ada0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2810009990 tx=0x7f2810009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 -- 172.21.15.46:0/3478726894 shutdown_connections 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 --2- 172.21.15.46:0/3478726894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f282410b3c0 0x7f282410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 --2- 172.21.15.46:0/3478726894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 0x7f2824102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 --2- 172.21.15.46:0/3478726894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28241028c0 0x7f282410ada0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 -- 172.21.15.46:0/3478726894 >> 172.21.15.46:0/3478726894 conn(0x7f28240fb760 msgr2=0x7f28240fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:45.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 -- 172.21.15.46:0/3478726894 shutdown_connections 2024-05-08T23:24:45.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.119+0000 7f2829200700 1 -- 172.21.15.46:0/3478726894 wait complete. 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 Processor -- start 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 -- start start 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 0x7f2824199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f282410b3c0 0x7f282419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e460 con 0x7f28241028c0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e5d0 con 0x7f2824101fa0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.120+0000 7f2829200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e790 con 0x7f282410b3c0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59014/0 (socket says 172.21.15.46:59014) 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 -- 172.21.15.46:0/1925706756 learned_addr learned my addr 172.21.15.46:0/1925706756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821e00700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 0x7f2824199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2822800700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f282410b3c0 0x7f282419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 -- 172.21.15.46:0/1925706756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f282410b3c0 msgr2=0x7f282419de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f282410b3c0 0x7f282419de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 -- 172.21.15.46:0/1925706756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 msgr2=0x7f2824199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 0x7f2824199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 -- 172.21.15.46:0/1925706756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009c40 con 0x7f28241028c0 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.121+0000 7f2821400700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2810000c00 tx=0x7f2810005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.122+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281001d070 con 0x7f28241028c0 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.122+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28100095f0 con 0x7f28241028c0 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.122+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2810022470 con 0x7f28241028c0 2024-05-08T23:24:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.122+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000bee0 con 0x7f28241028c0 2024-05-08T23:24:45.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.122+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28241a60d0 con 0x7f28241028c0 2024-05-08T23:24:45.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.123+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2824108890 con 0x7f28241028c0 2024-05-08T23:24:45.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.125+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f28100225e0 con 0x7f28241028c0 2024-05-08T23:24:45.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.126+0000 7f281aa00700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 0x7f28080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:45.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.126+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9896+0+0 (secure 0 0 0) 0x7f28100ce2b0 con 0x7f28241028c0 2024-05-08T23:24:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.126+0000 7f2821e00700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 0x7f28080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.127+0000 7f2821e00700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 0x7f28080a3750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2814009c60 tx=0x7f2814009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.131+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f281005dd60 con 0x7f28241028c0 2024-05-08T23:24:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:24:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:24:45.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:24:45.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: Cluster is now healthy 2024-05-08T23:24:45.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6832/2353083981,v1:172.21.15.53:6834/2353083981] up:boot 2024-05-08T23:24:45.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6833/3020294129,v1:172.21.15.53:6835/3020294129] up:boot 2024-05-08T23:24:45.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6836/601538692,v1:172.21.15.53:6837/601538692] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.53:6838/699032354,v1:172.21.15.53:6839/699032354] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6836/3031844923,v1:172.21.15.86:6837/3031844923] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6838/814256642,v1:172.21.15.86:6839/814256642] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6832/1370541812,v1:172.21.15.86:6834/1370541812] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.86:6833/1298192100,v1:172.21.15.86:6835/1298192100] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6836/598474628,v1:172.21.15.46:6837/598474628] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: mds.? [v2:172.21.15.46:6838/3175568581,v1:172.21.15.46:6839/3175568581] up:boot 2024-05-08T23:24:45.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: fsmap 12 up:standby 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:24:45.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:24:45.281 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:24:45.281 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:45 smithi046 ceph-mon[36898]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:24:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:45.348+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f282419a840 con 0x7f28241028c0 2024-05-08T23:24:45.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:24:45.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:24:45.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: Cluster is now healthy 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6832/2353083981,v1:172.21.15.53:6834/2353083981] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6833/3020294129,v1:172.21.15.53:6835/3020294129] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6836/601538692,v1:172.21.15.53:6837/601538692] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.53:6838/699032354,v1:172.21.15.53:6839/699032354] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6836/3031844923,v1:172.21.15.86:6837/3031844923] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6838/814256642,v1:172.21.15.86:6839/814256642] up:boot 2024-05-08T23:24:45.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6832/1370541812,v1:172.21.15.86:6834/1370541812] up:boot 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.86:6833/1298192100,v1:172.21.15.86:6835/1298192100] up:boot 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6836/598474628,v1:172.21.15.46:6837/598474628] up:boot 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: mds.? [v2:172.21.15.46:6838/3175568581,v1:172.21.15.46:6839/3175568581] up:boot 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: fsmap 12 up:standby 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:24:45.466 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:24:45.467 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:45 smithi086 ceph-mon[78610]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:24:45.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:24:45.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:24:45.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:24:45.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: Cluster is now healthy 2024-05-08T23:24:45.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6832/2353083981,v1:172.21.15.53:6834/2353083981] up:boot 2024-05-08T23:24:45.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6833/3020294129,v1:172.21.15.53:6835/3020294129] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6836/601538692,v1:172.21.15.53:6837/601538692] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.53:6838/699032354,v1:172.21.15.53:6839/699032354] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6836/3031844923,v1:172.21.15.86:6837/3031844923] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6838/814256642,v1:172.21.15.86:6839/814256642] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6832/1370541812,v1:172.21.15.86:6834/1370541812] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.86:6833/1298192100,v1:172.21.15.86:6835/1298192100] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6836/598474628,v1:172.21.15.46:6837/598474628] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: mds.? [v2:172.21.15.46:6838/3175568581,v1:172.21.15.46:6839/3175568581] up:boot 2024-05-08T23:24:45.525 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/3513598690' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: fsmap 12 up:standby 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:24:45.526 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:24:45.527 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:24:45.527 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:24:45.527 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:45 smithi053 ceph-mon[77587]: from='mgr.15501 172.21.15.46:0/2375118896' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:24:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.024+0000 7f281aa00700 1 -- 172.21.15.46:0/1925706756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7f281009acc0 con 0x7f28241028c0 2024-05-08T23:24:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.028+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 msgr2=0x7f28080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.028+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 0x7f28080a3750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2814009c60 tx=0x7f2814009260 comp rx=0 tx=0).stop 2024-05-08T23:24:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.028+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 msgr2=0x7f2824199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2810000c00 tx=0x7f2810005740 comp rx=0 tx=0).stop 2024-05-08T23:24:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 shutdown_connections 2024-05-08T23:24:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f282410b3c0 0x7f282419de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2824101fa0 0x7f2824199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f28080a1290 0x7f28080a3750 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 --2- 172.21.15.46:0/1925706756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28241028c0 0x7f2824199a80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 >> 172.21.15.46:0/1925706756 conn(0x7f28240fb760 msgr2=0x7f28240fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 shutdown_connections 2024-05-08T23:24:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.029+0000 7f2829200700 1 -- 172.21.15.46:0/1925706756 wait complete. 2024-05-08T23:24:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:24:46.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:24:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:46 smithi046 ceph-mon[36898]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 408 KiB/s wr, 15 op/s 2024-05-08T23:24:46.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:46 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.279+0000 7fcf3de00700 1 -- 172.21.15.46:0/3160908746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38101870 msgr2=0x7fcf381058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.279+0000 7fcf3de00700 1 --2- 172.21.15.46:0/3160908746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38101870 0x7fcf381058b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcf240098f0 tx=0x7fcf24009c00 comp rx=0 tx=0).stop 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 -- 172.21.15.46:0/3160908746 shutdown_connections 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 --2- 172.21.15.46:0/3160908746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38105fe0 0x7fcf3810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 --2- 172.21.15.46:0/3160908746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf38100ec0 0x7fcf381012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 --2- 172.21.15.46:0/3160908746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38101870 0x7fcf381058b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 -- 172.21.15.46:0/3160908746 >> 172.21.15.46:0/3160908746 conn(0x7fcf380fc740 msgr2=0x7fcf380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:46.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 -- 172.21.15.46:0/3160908746 shutdown_connections 2024-05-08T23:24:46.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.280+0000 7fcf3de00700 1 -- 172.21.15.46:0/3160908746 wait complete. 2024-05-08T23:24:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.281+0000 7fcf3de00700 1 Processor -- start 2024-05-08T23:24:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.281+0000 7fcf3de00700 1 -- start start 2024-05-08T23:24:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 0x7fcf38197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3819bcd0 0x7fcf3819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3819c750 con 0x7fcf38100ec0 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3819c8c0 con 0x7fcf38105fe0 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3819ca80 con 0x7fcf3819bcd0 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf37e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 0x7fcf38197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59026/0 (socket says 172.21.15.46:59026) 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3ca00700 1 -- 172.21.15.46:0/4019990466 learned_addr learned my addr 172.21.15.46:0/4019990466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.282+0000 7fcf3d400700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3819bcd0 0x7fcf3819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 -- 172.21.15.46:0/4019990466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3819bcd0 msgr2=0x7fcf3819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3819bcd0 0x7fcf3819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 -- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 msgr2=0x7fcf38197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 -- 172.21.15.46:0/4019990466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009d00 con 0x7fcf38105fe0 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf3ca00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf37e00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 0x7fcf38197980 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009230 tx=0x7fcf2400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:46.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401c070 con 0x7fcf38105fe0 2024-05-08T23:24:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2400ba20 con 0x7fcf38105fe0 2024-05-08T23:24:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240038e0 con 0x7fcf38105fe0 2024-05-08T23:24:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf24009520 con 0x7fcf38105fe0 2024-05-08T23:24:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.283+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf381a4210 con 0x7fcf38105fe0 2024-05-08T23:24:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.284+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf381092b0 con 0x7fcf38105fe0 2024-05-08T23:24:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.286+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf24020070 con 0x7fcf38105fe0 2024-05-08T23:24:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.287+0000 7fcf35600700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 0x7fcf200a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.287+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9313+0+0 (secure 0 0 0) 0x7fcf240cdba0 con 0x7fcf38105fe0 2024-05-08T23:24:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.288+0000 7fcf3ca00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 0x7fcf200a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.288+0000 7fcf3ca00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 0x7fcf200a3870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011550 tx=0x7fcf2c009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.293+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf24096d00 con 0x7fcf38105fe0 2024-05-08T23:24:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:46 smithi086 ceph-mon[78610]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 408 KiB/s wr, 15 op/s 2024-05-08T23:24:46.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:46 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:46.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:46 smithi053 ceph-mon[77587]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 408 KiB/s wr, 15 op/s 2024-05-08T23:24:46.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:46 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:46.639+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf38102390 con 0x7fcf38105fe0 2024-05-08T23:24:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.031+0000 7fcf35600700 1 -- 172.21.15.46:0/4019990466 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcf2409a320 con 0x7fcf38105fe0 2024-05-08T23:24:47.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 msgr2=0x7fcf200a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 0x7fcf200a3870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011550 tx=0x7fcf2c009260 comp rx=0 tx=0).stop 2024-05-08T23:24:47.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 msgr2=0x7fcf38197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 0x7fcf38197980 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009230 tx=0x7fcf2400fa00 comp rx=0 tx=0).stop 2024-05-08T23:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 shutdown_connections 2024-05-08T23:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3819bcd0 0x7fcf3819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7fcf200a13b0 0x7fcf200a3870 secure :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011550 tx=0x7fcf2c009260 comp rx=0 tx=0).stop 2024-05-08T23:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf38100ec0 0x7fcf38197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 --2- 172.21.15.46:0/4019990466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf38105fe0 0x7fcf38197980 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.039+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 >> 172.21.15.46:0/4019990466 conn(0x7fcf380fc740 msgr2=0x7fcf380feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.040+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 shutdown_connections 2024-05-08T23:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.041+0000 7fcf3de00700 1 -- 172.21.15.46:0/4019990466 wait complete. 2024-05-08T23:24:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-05-08T23:24:47.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T23:24:47.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:47 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:47.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:47 smithi046 ceph-mon[36898]: osdmap e117: 12 total, 12 up, 12 in 2024-05-08T23:24:47.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:47 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/4019990466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:47 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 -- 172.21.15.46:0/1723500917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d40733d0 msgr2=0x7f10d40737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 --2- 172.21.15.46:0/1723500917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d40733d0 0x7f10d40737b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009990 tx=0x7f10c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 -- 172.21.15.46:0/1723500917 shutdown_connections 2024-05-08T23:24:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 --2- 172.21.15.46:0/1723500917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d410d650 0x7f10d410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 --2- 172.21.15.46:0/1723500917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d40733d0 0x7f10d40737b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 --2- 172.21.15.46:0/1723500917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d4073cf0 0x7f10d410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 -- 172.21.15.46:0/1723500917 >> 172.21.15.46:0/1723500917 conn(0x7f10d40fc8e0 msgr2=0x7f10d40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 -- 172.21.15.46:0/1723500917 shutdown_connections 2024-05-08T23:24:47.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.296+0000 7f10dc800700 1 -- 172.21.15.46:0/1723500917 wait complete. 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 Processor -- start 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 -- start start 2024-05-08T23:24:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4073cf0 0x7f10d4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d410d650 0x7f10d419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e430 con 0x7f10d4073cf0 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e5a0 con 0x7f10d410d650 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.297+0000 7f10dc800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e760 con 0x7f10d40733d0 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.46:55994/0 (socket says 172.21.15.46:55994) 2024-05-08T23:24:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 -- 172.21.15.46:0/771248110 learned_addr learned my addr 172.21.15.46:0/771248110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9400700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4073cf0 0x7f10d4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10da800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d410d650 0x7f10d419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 -- 172.21.15.46:0/771248110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d410d650 msgr2=0x7f10d419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d410d650 0x7f10d419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 -- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4073cf0 msgr2=0x7f10d4199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4073cf0 0x7f10d4199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 -- 172.21.15.46:0/771248110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c80095f0 con 0x7f10d40733d0 2024-05-08T23:24:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.298+0000 7f10d9e00700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005740 tx=0x7f10c8005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:47.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.299+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c801d070 con 0x7f10d40733d0 2024-05-08T23:24:47.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.299+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c8022470 con 0x7f10d40733d0 2024-05-08T23:24:47.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.299+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d419e9e0 con 0x7f10d40733d0 2024-05-08T23:24:47.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.299+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8017650 con 0x7f10d40733d0 2024-05-08T23:24:47.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.299+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d41a6280 con 0x7f10d40733d0 2024-05-08T23:24:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.300+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d4108860 con 0x7f10d40733d0 2024-05-08T23:24:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.301+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f10c80225e0 con 0x7f10d40733d0 2024-05-08T23:24:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.303+0000 7f10cea00700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 0x7f10bc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.303+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8725+0+0 (secure 0 0 0) 0x7f10c80cdff0 con 0x7f10d40733d0 2024-05-08T23:24:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.303+0000 7f10d9400700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 0x7f10bc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:47.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.304+0000 7f10d9400700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 0x7f10bc0a37d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f10d419aa60 tx=0x7f10c4009260 comp rx=0 tx=0).ready entity=mgr.15501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:47.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.305+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f10c8097500 con 0x7f10d40733d0 2024-05-08T23:24:47.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:47 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:47.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:47 smithi086 ceph-mon[78610]: osdmap e117: 12 total, 12 up, 12 in 2024-05-08T23:24:47.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:47 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/4019990466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:47 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:47 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/1925706756' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:47.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:47 smithi053 ceph-mon[77587]: osdmap e117: 12 total, 12 up, 12 in 2024-05-08T23:24:47.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:47 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/4019990466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:47 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:47.525+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10d4074a80 con 0x7f10d40733d0 2024-05-08T23:24:48.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.036+0000 7f10cea00700 1 -- 172.21.15.46:0/771248110 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v119) v1 ==== 178+0+0 (secure 0 0 0) 0x7f10c809ab20 con 0x7f10d40733d0 2024-05-08T23:24:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.041+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 msgr2=0x7f10bc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.041+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 0x7f10bc0a37d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f10d419aa60 tx=0x7f10c4009260 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.041+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 msgr2=0x7f10d4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.041+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005740 tx=0x7f10c8005770 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 shutdown_connections 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d410d650 0x7f10d419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10d40733d0 0x7f10d4199510 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:6800/652597198,v1:172.21.15.46:6801/652597198] conn(0x7f10bc0a1310 0x7f10bc0a37d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 --2- 172.21.15.46:0/771248110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4073cf0 0x7f10d4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 >> 172.21.15.46:0/771248110 conn(0x7f10d40fc8e0 msgr2=0x7f10d40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 shutdown_connections 2024-05-08T23:24:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T23:24:48.042+0000 7f10dc800700 1 -- 172.21.15.46:0/771248110 wait complete. 2024-05-08T23:24:48.044 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-05-08T23:24:48.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:24:48.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:24:48.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:24:48.222 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T23:24:48.222 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:24:48.264 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:24:48.295 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:24:48.312 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T23:24:48.312 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T23:24:48.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a 2024-05-08T23:24:48.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-mon[36898]: pgmap v662: 97 pgs: 97 active+clean; 315 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:24:48.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-mon[36898]: osdmap e118: 12 total, 12 up, 12 in 2024-05-08T23:24:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-mon[36898]: from='client.? 172.21.15.46:0/771248110' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-mon[36898]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.464 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:48 smithi086 ceph-mon[78610]: pgmap v662: 97 pgs: 97 active+clean; 315 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:24:48.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:48 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:48.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:48 smithi086 ceph-mon[78610]: osdmap e118: 12 total, 12 up, 12 in 2024-05-08T23:24:48.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:48 smithi086 ceph-mon[78610]: from='client.? 172.21.15.46:0/771248110' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.465 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:48 smithi086 ceph-mon[78610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:48 smithi053 ceph-mon[77587]: pgmap v662: 97 pgs: 97 active+clean; 315 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:24:48.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:48 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:24:48.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:48 smithi053 ceph-mon[77587]: osdmap e118: 12 total, 12 up, 12 in 2024-05-08T23:24:48.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:48 smithi053 ceph-mon[77587]: from='client.? 172.21.15.46:0/771248110' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.512 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:48 smithi053 ceph-mon[77587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:24:48.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 systemd[1]: Stopping Ceph mon.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:49.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a[36875]: 2024-05-08T23:24:48.786+0000 7fe69f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:24:49.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a[36875]: 2024-05-08T23:24:48.786+0000 7fe69f000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T23:24:49.522 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 08 23:24:49 smithi046 bash[135857]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-a 2024-05-08T23:24:49.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T23:24:49.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:49.643 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T23:24:49.644 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T23:24:49.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.b 2024-05-08T23:24:50.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:49 smithi053 systemd[1]: Stopping Ceph mon.b for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:50.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-b[77564]: 2024-05-08T23:24:49.912+0000 7f46d74d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:24:50.012 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-b[77564]: 2024-05-08T23:24:49.912+0000 7f46d74d0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T23:24:50.381 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 08 23:24:50 smithi053 bash[123105]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-b 2024-05-08T23:24:50.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T23:24:50.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:50.446 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T23:24:50.446 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T23:24:50.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.c 2024-05-08T23:24:50.831 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:50 smithi086 systemd[1]: Stopping Ceph mon.c for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:51.092 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-c[78587]: 2024-05-08T23:24:50.830+0000 7fa379ffd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:24:51.092 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-c[78587]: 2024-05-08T23:24:50.830+0000 7fa379ffd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T23:24:51.422 INFO:journalctl@ceph.mon.c.smithi086.stdout:May 08 23:24:51 smithi086 bash[123375]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mon-c 2024-05-08T23:24:51.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T23:24:51.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:51.490 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T23:24:51.490 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T23:24:51.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x 2024-05-08T23:24:51.863 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:24:51 smithi046 systemd[1]: Stopping Ceph mgr.x for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:52.118 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:24:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x[37150]: 2024-05-08T23:24:51.860+0000 7fcfefa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:24:52.530 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 08 23:24:52 smithi046 bash[135995]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-x 2024-05-08T23:24:52.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T23:24:52.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:52.683 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T23:24:52.683 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T23:24:52.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.y 2024-05-08T23:24:53.012 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:24:52 smithi053 systemd[1]: Stopping Ceph mgr.y for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:53.412 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:24:53 smithi053 bash[123246]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-y 2024-05-08T23:24:53.412 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 08 23:24:53 smithi053 systemd[1]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T23:24:53.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T23:24:53.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:53.460 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T23:24:53.461 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T23:24:53.461 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.z 2024-05-08T23:24:53.844 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:24:53 smithi086 systemd[1]: Stopping Ceph mgr.z for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:54.213 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:24:53 smithi086 bash[123514]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mgr-z 2024-05-08T23:24:54.214 INFO:journalctl@ceph.mgr.z.smithi086.stdout:May 08 23:24:54 smithi086 systemd[1]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T23:24:54.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T23:24:54.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:24:54.261 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T23:24:54.261 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T23:24:54.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.0 2024-05-08T23:24:54.636 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 23:24:54 smithi046 systemd[1]: Stopping Ceph osd.0 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:24:55.027 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 23:24:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T23:24:54.634+0000 7f8c01800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:24:55.027 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 23:24:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T23:24:54.634+0000 7f8c01800700 -1 osd.0 119 *** Got signal Terminated *** 2024-05-08T23:24:55.028 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 23:24:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0[55092]: 2024-05-08T23:24:54.634+0000 7f8c01800700 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:00.528 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 08 23:25:00 smithi046 bash[136134]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-0 2024-05-08T23:25:01.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T23:25:02.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:02.015 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T23:25:02.015 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T23:25:02.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.1 2024-05-08T23:25:02.287 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 23:25:02 smithi053 systemd[1]: Stopping Ceph osd.1 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:02.762 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 23:25:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T23:25:02.285+0000 7fbfccd52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:02.763 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 23:25:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T23:25:02.285+0000 7fbfccd52700 -1 osd.1 119 *** Got signal Terminated *** 2024-05-08T23:25:02.763 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 23:25:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1[82539]: 2024-05-08T23:25:02.285+0000 7fbfccd52700 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:08.262 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 08 23:25:07 smithi053 bash[123399]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-1 2024-05-08T23:25:09.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T23:25:09.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:09.196 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T23:25:09.196 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T23:25:09.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.2 2024-05-08T23:25:09.714 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:09 smithi086 systemd[1]: Stopping Ceph osd.2 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:09.714 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T23:25:09.444+0000 7fdde4b6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:09.714 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T23:25:09.444+0000 7fdde4b6d700 -1 osd.2 119 *** Got signal Terminated *** 2024-05-08T23:25:09.714 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2[84364]: 2024-05-08T23:25:09.444+0000 7fdde4b6d700 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:15.714 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:15 smithi086 bash[123661]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-2 2024-05-08T23:25:16.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T23:25:17.483 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:16 smithi086 systemd[1]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.2.service: Succeeded. 2024-05-08T23:25:17.483 INFO:journalctl@ceph.osd.2.smithi086.stdout:May 08 23:25:16 smithi086 systemd[1]: Stopped Ceph osd.2 for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:25:17.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:17.532 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T23:25:17.532 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T23:25:17.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.3 2024-05-08T23:25:17.909 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:17 smithi046 systemd[1]: Stopping Ceph osd.3 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:18.277 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T23:25:17.907+0000 7fe603800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:18.278 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T23:25:17.907+0000 7fe603800700 -1 osd.3 119 *** Got signal Terminated *** 2024-05-08T23:25:18.278 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:17 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T23:25:17.907+0000 7fe603800700 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:21.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:21.042+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:21.777 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:21 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T23:25:21.474+0000 7fe5ff000700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.307145+0000 front 2024-05-08T23:24:59.307177+0000 (oldest deadline 2024-05-08T23:25:21.007397+0000) 2024-05-08T23:25:22.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:22.011+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:22.777 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:22 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3[63536]: 2024-05-08T23:25:22.452+0000 7fe5ff000700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.307145+0000 front 2024-05-08T23:24:59.307177+0000 (oldest deadline 2024-05-08T23:25:21.007397+0000) 2024-05-08T23:25:23.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:22.970+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:23.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:23 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:23.032+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:24.214 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:23.947+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:24.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:23.998+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:24.277 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:23 smithi046 bash[136431]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-3 2024-05-08T23:25:24.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:23 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:23.998+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:24.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:24 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:24.317+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:25.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:24 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:24.981+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:25.312 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:24 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:24.971+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:25.313 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:24 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:24.992+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:25.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T23:25:25.604 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:25 smithi046 systemd[1]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.3.service: Succeeded. 2024-05-08T23:25:25.604 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 08 23:25:25 smithi046 systemd[1]: Stopped Ceph osd.3 for ebe57712-0d8d-11ef-bc97-c7b262605968. 2024-05-08T23:25:25.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:25.632 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T23:25:25.632 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T23:25:25.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.4 2024-05-08T23:25:25.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:25 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:25.312+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:26.012 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:25 smithi053 systemd[1]: Stopping Ceph osd.4 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:26.013 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:25 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T23:25:25.882+0000 7fd92cc8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:26.013 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:25 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T23:25:25.882+0000 7fd92cc8b700 -1 osd.4 119 *** Got signal Terminated *** 2024-05-08T23:25:26.013 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:25 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T23:25:25.882+0000 7fd92cc8b700 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:26.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:25 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:25.960+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:26.214 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:26 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:26.008+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:26.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:26 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:26.007+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:26.713 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:26 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:26.329+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:27.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:26 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:26.991+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:27.343 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:26 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:26.988+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:27.343 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:27 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:27.026+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:27.713 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:27 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:27.343+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:28.381 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:27 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:27.965+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:28.381 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:28 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:28.020+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:28.401 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:28.029+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:28.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:28 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:28.381+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:28.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:28 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:28.401+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:29.214 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:29.036+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:29.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:28 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:28.931+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:29.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:28 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:28.931+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:29.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:29.009+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:29.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:29.009+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:29.512 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:29 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T23:25:29.180+0000 7fd925279700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.804079+0000 front 2024-05-08T23:25:06.804054+0000 (oldest deadline 2024-05-08T23:25:28.504189+0000) 2024-05-08T23:25:29.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:29.357+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:29.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:29.357+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:29.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:29 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:29.365+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:30.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:30.023+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:30.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:30.023+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:30.353 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:29.980+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:30.353 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:29.980+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:30.354 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:29 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:29.992+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:30.512 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:30 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4[91198]: 2024-05-08T23:25:30.144+0000 7fd925279700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.804079+0000 front 2024-05-08T23:25:06.804054+0000 (oldest deadline 2024-05-08T23:25:28.504189+0000) 2024-05-08T23:25:30.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:30 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:30.353+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:30.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:30 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:30.353+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:30.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:30.388+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:31.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:30 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:30.958+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:31.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:30 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:30.958+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:31.214 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:30 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:30.994+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:31.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:30.983+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:31.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:30.983+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:31.713 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:31 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:31.314+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:31.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:31 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:31.314+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:31.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:31 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:31.349+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:32.012 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 08 23:25:31 smithi053 bash[123654]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-4 2024-05-08T23:25:32.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:32.016+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:32.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:32.016+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:32.331 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:31 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:31.988+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:32.331 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:31 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:31.988+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:32.332 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:31 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:31.964+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:32.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:32 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:32.331+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:32.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:32 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:32.331+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:32.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:32 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:32.327+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:32.929 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:32 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:32.753+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:32.930 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:32 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:32.582+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:32.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T23:25:32.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:32.992 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T23:25:32.992 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T23:25:32.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.5 2024-05-08T23:25:33.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:33.007+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:33.278 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:33.007+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:33.365 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:33.026+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:33.365 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:33.026+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:33.366 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:33.007+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:33.366 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:33.007+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:33.366 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 systemd[1]: Stopping Ceph osd.5 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:33.366 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:33.249+0000 7f19e7cbc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:33.367 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:33.249+0000 7f19e7cbc700 -1 osd.5 119 *** Got signal Terminated *** 2024-05-08T23:25:33.367 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:33.249+0000 7f19e7cbc700 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:33.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:33.365+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:33.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:33.365+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:33.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:33.351+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:34.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:33 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:33.781+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:34.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:33 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:33.560+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:34.330 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:34 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:34.075+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:34.330 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:34 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:34.075+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:34.330 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:33.980+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:34.330 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:33 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:33.980+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:34.366 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:34 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:34.033+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:34.366 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:34 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:34.033+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:34.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:34 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:34.330+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:34.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:34 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:34.330+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:34.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:34 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:34.366+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:35.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:34 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:34.809+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:35.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:34 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:34.596+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:35.306 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:35.084+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:35.307 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:35.084+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:35.307 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:35.003+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:35.307 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:35.003+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:35.412 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:35.053+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:35.413 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:35.053+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:35.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:35.306+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:35.715 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:35.306+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:35.715 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:35 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:35.306+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:09.169700+0000 front 2024-05-08T23:25:09.169657+0000 (oldest deadline 2024-05-08T23:25:34.970271+0000) 2024-05-08T23:25:35.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:35 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:35.413+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:35.822 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:35 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:35.551+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:35.823 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:35 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:35.551+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:36.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:35 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:35.822+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:36.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:35 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:35.822+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:36.297 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:36.060+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:36.297 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:36.060+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:36.297 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:36.060+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:36.298 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:36.028+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:36.298 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:36.028+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:36.393 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:36.059+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:36.393 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:36.059+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:36.393 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:36.059+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:36.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:36.297+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:36.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:36.297+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:36.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:36.297+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:09.169700+0000 front 2024-05-08T23:25:09.169657+0000 (oldest deadline 2024-05-08T23:25:34.970271+0000) 2024-05-08T23:25:36.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:36 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:36.393+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:36.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:36 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:36.544+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:36.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:36 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:36.544+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:37.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:36 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:36.850+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:37.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:36 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:36.850+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:37.304 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:37.015+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:37.305 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:37.015+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:37.305 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:37.015+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:37.305 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:36.984+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:37.305 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:36.984+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:37.305 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:36 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:36.984+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:37.434 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:37.065+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:37.434 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:37.065+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:37.434 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:37.065+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:37.714 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:37.304+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.567932+0000 front 2024-05-08T23:24:57.567871+0000 (oldest deadline 2024-05-08T23:25:23.368176+0000) 2024-05-08T23:25:37.715 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:37.304+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:05.669236+0000 front 2024-05-08T23:25:05.669370+0000 (oldest deadline 2024-05-08T23:25:29.169242+0000) 2024-05-08T23:25:37.715 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5[93018]: 2024-05-08T23:25:37.304+0000 7f19e42b2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:09.169700+0000 front 2024-05-08T23:25:09.169657+0000 (oldest deadline 2024-05-08T23:25:34.970271+0000) 2024-05-08T23:25:37.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:37.433+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:37.778 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:37.433+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:38.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:37 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:37.813+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:38.013 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:37 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:37.813+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:38.014 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:37 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:37.587+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:38.014 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:37 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:37.587+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:38.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:38.029+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:38.215 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:38.029+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:38.215 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:38.029+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:38.216 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:37.960+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:38.216 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:37.960+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:38.216 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:37 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:37.960+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:38.450 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:38.109+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:38.450 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:38.109+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:38.451 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:38.109+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:38.778 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:38.450+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:38.779 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:38 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:38.450+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:39.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:38.807+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:39.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:38.807+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:39.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:38.567+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:39.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:38 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:38.567+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:39.214 INFO:journalctl@ceph.osd.5.smithi086.stdout:May 08 23:25:38 smithi086 bash[123924]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-5 2024-05-08T23:25:39.214 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:39.034+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:39.215 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:39.034+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:39.215 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:39 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:39.034+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:39.215 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:38.989+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:39.215 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:38.989+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:39.215 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:38 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:38.989+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:39.449 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:39.127+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:39.449 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:39.127+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:39.449 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:39.127+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:39.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:39.449+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:39.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:39 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:39.449+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:40.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:39.835+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:40.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:39.835+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:40.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:39.599+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:40.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:39.599+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:40.076 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.003+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:40.076 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.003+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:40.076 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.003+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:40.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T23:25:40.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:40.169 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T23:25:40.169 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T23:25:40.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.6 2024-05-08T23:25:40.455 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:40.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:40.455 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:40.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:40.455 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:40.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:40.455 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 systemd[1]: Stopping Ceph osd.6 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:40.463 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:40.075+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:40.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:40.075+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:40.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:40.075+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:40.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:40.454+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:40.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:40.454+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:40.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:40.546+0000 7f1779200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:40.778 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:40.546+0000 7f1779200700 -1 osd.6 119 *** Got signal Terminated *** 2024-05-08T23:25:40.778 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:40 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:40.546+0000 7f1779200700 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:41.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:40.792+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:41.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:40.792+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:41.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:40.616+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:41.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:40 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:40.616+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:41.438 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:41.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:41.439 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:41.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:41.439 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:41.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:41.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:41.074+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:41.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:41.074+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:41.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:41 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:41.074+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:41.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.991+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:41.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.991+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:41.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:40 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:40.991+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:41.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:41.438+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:41.777 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:41 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:41.438+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:42.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:41.822+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:42.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:41.822+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:42.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:41.604+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:42.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:41 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:41.604+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:42.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:42.037+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:42.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:42.037+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:42.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:42.037+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:42.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:42.040+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:42.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:42.040+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:42.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:42 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:42.040+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:42.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:42.430+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:42.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:42.430+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:42.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:42.190+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:42.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:42.190+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:42.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:42 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:42.190+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:43.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:42.782+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:43.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:42.782+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:43.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:42.567+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:43.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:42 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:42.567+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:43.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:43.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:43.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:43.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:43.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:43.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:43.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:43.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:43.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:43.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:43.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:43 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:43.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:43.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:43.415+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:43.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:43.415+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:43.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:43.168+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:43.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:43.168+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:43.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:43 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:43.168+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:44.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:43.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:44.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:43.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:44.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:43.741+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:44.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:43 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:43.741+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:44.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:44.043+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:44.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:44.043+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:44.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:44.043+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:44.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:44.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:44.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:44.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:44.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:44 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:44.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:44.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:44.398+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:44.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:44.398+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:44.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:44.184+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:44.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:44.184+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:44.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:44 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:44.184+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:45.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:44.749+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:45.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:44.749+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:45.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:44.520+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:45.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:44.520+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:45.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:44.520+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:45.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:45.066+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:45.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:45.066+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:45.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:45.066+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:45.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:45.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:45.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:45.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:45.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:45 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:45.014+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:45.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:45.382+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:04.234974+0000 front 2024-05-08T23:25:04.235010+0000 (oldest deadline 2024-05-08T23:25:28.335223+0000) 2024-05-08T23:25:45.527 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6[72000]: 2024-05-08T23:25:45.382+0000 7f176fa00700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:11.836922+0000 front 2024-05-08T23:25:11.837023+0000 (oldest deadline 2024-05-08T23:25:37.137071+0000) 2024-05-08T23:25:45.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:45.200+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:45.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:45.200+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:45.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:45.200+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:45.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:45 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:45.200+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:46.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:45.795+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:46.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:45.795+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:46.012 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:45.795+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:46.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:45.563+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:46.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:45.563+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:46.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:45 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:45.563+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:46.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:46.046+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:46.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:46.046+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:46.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:46.046+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:46.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:46.062+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:46.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:46.062+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:46.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:46 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:46.062+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:46.528 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 08 23:25:46 smithi046 bash[136688]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-6 2024-05-08T23:25:46.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:46.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:46.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:46.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:46.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:46.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:46.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:46 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:46.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:46.826 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:46.522+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:46.826 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:46.522+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:46.826 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:46.522+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:47.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:46.826+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:47.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:46.826+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:47.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:46 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:46.826+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:47.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:47.063+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:47.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:47.063+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:47.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:47.063+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:47.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:47.035+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:47.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:47.035+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:47.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:47 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:47.035+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:47.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:47.175+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:47.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:47.175+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:47.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:47.175+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:47.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:47 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:47.175+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:47.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:47.502+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:47.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:47.502+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:47.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:47.502+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:47.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T23:25:48.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:48.031 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T23:25:48.032 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T23:25:48.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.7 2024-05-08T23:25:48.064 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:47.811+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:48.064 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:47.811+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:48.064 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:47 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:47.811+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:48.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:48.143+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:48.277 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:48.143+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:48.278 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:48.143+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:48.278 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:48 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:48.143+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:48.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:48.065+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:48.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:48.065+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:48.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:48.065+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:48.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:48.057+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:48.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:48.057+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:48.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:48 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:48.057+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:48.510 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 systemd[1]: Stopping Ceph osd.7 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:48.510 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.287+0000 7f12d8c3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:48.511 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.287+0000 7f12d8c3f700 -1 osd.7 119 *** Got signal Terminated *** 2024-05-08T23:25:48.511 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.287+0000 7f12d8c3f700 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:48.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:48.510+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:48.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:48.510+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:48.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:48.510+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:49.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.858+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:49.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.858+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:49.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:48.858+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:49.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:49.061+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:49.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:49.061+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:49.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:49.061+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:49.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:49.007+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:49.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:49.007+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:49.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:49 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:49.007+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:49.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:49.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:49.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:49.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:49.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:49.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:49.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:49 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:49.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:49.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:49.511+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:49.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:49.511+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:49.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:49.511+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:50.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:49.831+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:50.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:49.831+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:50.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:49 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:49.831+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:50.463 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:50.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:50.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:50.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:50.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:50.082+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:50.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:50.023+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:50.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:50.023+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:50.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:50 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:50.023+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:50.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:50.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:50.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:50.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:50.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:50.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:50.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:50 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:50.179+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:50.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:50.489+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:50.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:50.489+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:50.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:50.489+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:51.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:50.873+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:51.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:50.873+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:51.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:50 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:50.873+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:51.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:51.079+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:51.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:51.079+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:51.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:51.079+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:51.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.011+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:51.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.011+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:51.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.011+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:51.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:51.210+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:51.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:51.210+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:51.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:51.210+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:51.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:51 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:51.210+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:51.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:51.443+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:51.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:51.443+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:51.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:51.443+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:52.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:51.869+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:52.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:51.869+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:52.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:51 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:51.869+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:52.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.974+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:52.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.974+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:52.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:51 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:51.974+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:52.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:52 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:52.127+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:52.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:52 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:52.127+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:52.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:52 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:52.127+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:52.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:52.221+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:52.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:52.221+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:52.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:52.221+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:52.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:52.221+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:52.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:52 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:52.221+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:52.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:52.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:52.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:52.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:52.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:52.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:53.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:52.875+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.549693+0000 front 2024-05-08T23:25:06.549717+0000 (oldest deadline 2024-05-08T23:25:31.750906+0000) 2024-05-08T23:25:53.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:52.875+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:13.452525+0000 front 2024-05-08T23:25:13.452294+0000 (oldest deadline 2024-05-08T23:25:35.052888+0000) 2024-05-08T23:25:53.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7[99142]: 2024-05-08T23:25:52.875+0000 7f12d122d700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.955361+0000 front 2024-05-08T23:25:21.955330+0000 (oldest deadline 2024-05-08T23:25:45.455316+0000) 2024-05-08T23:25:53.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:53.016+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:53.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:53.016+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:53.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:53.016+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:53.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:53.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:53.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:53.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:53.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:53.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:53.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:53 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:53.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:53.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:53 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:53.220+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:53.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:53 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:53.220+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:53.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:53 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:53.220+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:53.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:53 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:53.220+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:53.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:53 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:53.220+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:53.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:53.494+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:53.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:53.494+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:53.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:53.494+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:53.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:53 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:53.495+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:54.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:54.039+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:54.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:54.039+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:54.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:54.039+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:54.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:54.110+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:54.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:54.110+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:54.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:54.110+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:54.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:54 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:54.110+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:54.512 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 08 23:25:54 smithi053 bash[123913]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-7 2024-05-08T23:25:54.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:54.250+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:54.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:54.250+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:54.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:54.250+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:54.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:54.250+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:54.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:54 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:54.250+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:55.012 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:54.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:55.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:54.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:55.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:54.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:55.013 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:54 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:54.528+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:55.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.081+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.081+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.081+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.081+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:55.085+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:55.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:55.085+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:55.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:55.085+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:55.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:55.085+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:55.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:55 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:55.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:55.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:55 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:55.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:55.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:55 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:55.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:55.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:55 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:55.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:55.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:55 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:55.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:55.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:25:55.555 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T23:25:55.556 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T23:25:55.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.8 2024-05-08T23:25:55.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:55.480+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:55.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:55.480+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:55.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:55.480+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:55.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:55 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:55.480+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:55.966 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 systemd[1]: Stopping Ceph osd.8 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:25:55.966 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.805+0000 7fcbf53cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:25:55.967 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.805+0000 7fcbf53cd700 -1 osd.8 119 *** Got signal Terminated *** 2024-05-08T23:25:55.967 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:55 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:55.805+0000 7fcbf53cd700 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:25:56.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:56.135+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:56.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:56.135+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:56.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:56.135+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:56.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:56.135+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:56.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:56.098+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:56.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:56.098+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:56.466 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:56.099+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:56.466 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:56.099+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:25:56.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:56.248+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:56.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:56.248+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:56.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:56.248+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:56.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:56.248+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:56.529 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:56 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:56.248+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:56.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:56.465+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:56.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:56.465+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:56.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:56.465+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:56.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:56 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:56.465+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:57.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:57.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:57.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:57.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:57.116+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:57.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:57.116+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:57.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:57.116+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:57.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:57 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:57.116+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:25:57.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:57.249+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:57.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:57.249+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:57.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:57.249+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:57.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:57.249+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:57.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:57 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:57.249+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:57.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:57 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:57.440+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:57.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:57 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:57.440+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:57.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:57 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:57.440+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:57.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:57 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:57.440+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:58.105 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:58.103+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:58.105 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:58.103+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:58.105 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:58.067+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:58.105 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:58.067+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:58.105 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:58.067+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:58.106 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:58.067+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:58.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:58.103+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:58.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:58 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:58.103+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:25:58.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:58 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:58.231+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:58.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:58 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:58.231+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:58.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:58 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:58.231+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:58.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:58 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:58.231+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:58.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:58 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:58.231+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:58.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:58 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:58.449+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:58.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:58 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:58.449+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:58.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:58 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:58.449+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:58.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:58 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:58.449+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:25:59.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:59.080+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:25:59.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:59.080+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:25:59.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:59.080+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:25:59.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:25:59.080+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:25:59.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:59.116+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:25:59.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:59.116+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:25:59.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:59.116+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:25:59.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:25:59 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:25:59.116+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:25:59.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:59 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:59.189+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:25:59.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:59 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:59.189+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:25:59.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:59 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:59.189+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:25:59.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:59 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:59.189+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:25:59.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:25:59 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:25:59.189+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:25:59.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:59 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:59.418+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:25:59.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:59 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:59.418+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:25:59.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:59 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:59.418+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:25:59.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:25:59 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:25:59.418+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:00.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:00.101+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:00.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:00.101+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:00.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:00.101+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:00.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:00.101+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:00.464 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:26:00.129+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:58.634479+0000 front 2024-05-08T23:24:58.634499+0000 (oldest deadline 2024-05-08T23:25:23.934613+0000) 2024-05-08T23:26:00.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:26:00.129+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:03.935198+0000 front 2024-05-08T23:25:03.935266+0000 (oldest deadline 2024-05-08T23:25:28.635078+0000) 2024-05-08T23:26:00.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:26:00.129+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.736134+0000 front 2024-05-08T23:25:12.736110+0000 (oldest deadline 2024-05-08T23:25:35.636266+0000) 2024-05-08T23:26:00.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:26:00.129+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.540273+0000 front 2024-05-08T23:25:29.540360+0000 (oldest deadline 2024-05-08T23:25:54.840382+0000) 2024-05-08T23:26:00.465 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8[100856]: 2024-05-08T23:26:00.129+0000 7fcbf19c3700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:37.741482+0000 front 2024-05-08T23:25:37.741466+0000 (oldest deadline 2024-05-08T23:26:00.041605+0000) 2024-05-08T23:26:00.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:00.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:00.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:00.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:00.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:00.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:00 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:00.193+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:00.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:00 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:00.462+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:00.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:00 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:00.462+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:00.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:00 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:00.462+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:00.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:00 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:00.462+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:00.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:00 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:00.462+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:01.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:01 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:01.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:01.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:01 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:01.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:01.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:01 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:01.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:01.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:01 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:01.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:01.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:01.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:01.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:01.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:01.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:01.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:01 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:01.227+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:01.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:01 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:01.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:01.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:01 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:01.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:01.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:01 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:01.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:01.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:01 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:01.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:01.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:01 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:01.466+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:01.964 INFO:journalctl@ceph.osd.8.smithi086.stdout:May 08 23:26:01 smithi086 bash[124180]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-8 2024-05-08T23:26:02.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:02 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:02.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:02.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:02 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:02.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:02.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:02 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:02.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:02.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:02 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:02.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:02.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:02 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:02.146+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:02.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:02.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:02.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:02.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:02.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:02.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:02 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:02.207+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:02.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:02.426+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:02.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:02.426+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:02.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:02.426+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:02.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:02.426+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:02.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:02 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:02.426+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:02.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T23:26:03.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:03.035 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T23:26:03.035 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T23:26:03.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.9 2024-05-08T23:26:03.410 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 systemd[1]: Stopping Ceph osd.9 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:03.410 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.186+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.410+0000 7f9db0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:03.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.410+0000 7f9db0600700 -1 osd.9 119 *** Got signal Terminated *** 2024-05-08T23:26:03.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:03 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:03.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:03.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:03 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:03.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:03.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:03 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:03.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:03.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:03 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:03.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:03.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:03 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:03.100+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:03.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:03 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:03.388+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:03.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:03 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:03.388+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:03.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:03 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:03.388+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:03.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:03 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:03.388+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:03.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:03 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:03.388+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:03.777 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:03 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:03.410+0000 7f9db0600700 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:26:04.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:04.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:04.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:04.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:04.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:04.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:04.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:04.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:04.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:04.078+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:04.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:04.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:04.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:04.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:04.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:04.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:04 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:04.141+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:04.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:04 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:04.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:04.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:04 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:04.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:04.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:04 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:04.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:04.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:04 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:04.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:04.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:04 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:04.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:05.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:05 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:05.070+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:05.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:05 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:05.070+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:05.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:05 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:05.070+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:05.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:05 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:05.070+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:05.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:05 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:05.070+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:05.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:05.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:05.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:05.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:05.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:05.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:05 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:05.131+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:05.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:05 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:05.330+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:05.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:05 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:05.330+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:05.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:05 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:05.330+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:05.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:05 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:05.330+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:05.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:05 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:05.330+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:06.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:06 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:06.106+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:06.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:06 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:06.106+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:06.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:06 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:06.106+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:06.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:06 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:06.106+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:06.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:06 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:06.106+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:06.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:06.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:06.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:06.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:06.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:06.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:06 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:06.098+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:06.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:06 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:06.296+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:06.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:06 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:06.296+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:06.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:06 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:06.296+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:06.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:06 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:06.296+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:06.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:06 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:06.296+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:07.463 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:07 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:07.144+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:07.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:07 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:07.144+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:07.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:07 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:07.144+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:07.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:07 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:07.144+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:07.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:07 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:07.144+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:07.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:07.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:07.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:07.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:07.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:07.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:07 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:07.137+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:07.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:07 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:07.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:07.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:07 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:07.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:07.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:07 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:07.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:07.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:07 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:07.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:07.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:07 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:07.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:08.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:08.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:08.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:08.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:08.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:08.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:08.156+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:08.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:57.598068+0000 front 2024-05-08T23:24:57.598098+0000 (oldest deadline 2024-05-08T23:25:22.898361+0000) 2024-05-08T23:26:08.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:02.899339+0000 front 2024-05-08T23:25:02.899505+0000 (oldest deadline 2024-05-08T23:25:28.799333+0000) 2024-05-08T23:26:08.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:10.500528+0000 front 2024-05-08T23:25:10.500462+0000 (oldest deadline 2024-05-08T23:25:35.800497+0000) 2024-05-08T23:26:08.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.702186+0000 front 2024-05-08T23:25:21.702212+0000 (oldest deadline 2024-05-08T23:25:44.602337+0000) 2024-05-08T23:26:08.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:28.604545+0000 front 2024-05-08T23:25:28.604660+0000 (oldest deadline 2024-05-08T23:25:52.104750+0000) 2024-05-08T23:26:08.528 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:08 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9[80528]: 2024-05-08T23:26:08.124+0000 7f9dabe00700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:36.206439+0000 front 2024-05-08T23:25:36.206478+0000 (oldest deadline 2024-05-08T23:25:59.706637+0000) 2024-05-08T23:26:08.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:08 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:08.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:08.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:08 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:08.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:08.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:08 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:08.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:08.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:08 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:08.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:08.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:08 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:08.307+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:09.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:09 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:09.134+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:09.527 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 08 23:26:09 smithi046 bash[136955]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-9 2024-05-08T23:26:09.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:09 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:09.353+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:09.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:09 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:09.353+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:09.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:09 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:09.353+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:09.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:09 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:09.353+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:09.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:09 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:09.353+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:10.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:10.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:10.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:10.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:10.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:10.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:10 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:10.136+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:10.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:10 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:10.346+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:10.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:10 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:10.346+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:10.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:10 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:10.346+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:10.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:10 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:10.346+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:10.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:10 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:10.346+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:11.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T23:26:11.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:11.177 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T23:26:11.177 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T23:26:11.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.10 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:11.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:11 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:11.133+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:11.512 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 systemd[1]: Stopping Ceph osd.10 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.309+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:11.513 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.434+0000 7ff977737700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:11.514 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.434+0000 7ff977737700 -1 osd.10 119 *** Got signal Terminated *** 2024-05-08T23:26:11.514 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:11 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:11.434+0000 7ff977737700 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:12.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:12 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:12.142+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:12.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:12.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:12.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:12.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:12.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:12.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:12 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:12.337+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:13.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:13 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:13.188+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:13.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:13.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:13.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:13.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:13.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:13.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:13 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:13.322+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:14.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:14 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:14.230+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:14.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:14.764 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:14.764 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:14.764 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:14.764 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:14.764 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:14 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:14.340+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:15.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:15.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:15.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:15.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:15.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:15.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:15.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:15 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:15.211+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:15.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:15.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:15.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:15.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:15.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:15.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:15 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:15.354+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:16.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:16.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:16 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:16.167+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:16.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:07.135270+0000 front 2024-05-08T23:25:07.135426+0000 (oldest deadline 2024-05-08T23:25:32.435426+0000) 2024-05-08T23:26:16.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:14.136556+0000 front 2024-05-08T23:25:14.136604+0000 (oldest deadline 2024-05-08T23:25:35.236587+0000) 2024-05-08T23:26:16.762 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-08T23:25:21.539012+0000 front 2024-05-08T23:25:21.539035+0000 (oldest deadline 2024-05-08T23:25:43.839086+0000) 2024-05-08T23:26:16.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:27.340202+0000 front 2024-05-08T23:25:27.340226+0000 (oldest deadline 2024-05-08T23:25:53.240625+0000) 2024-05-08T23:26:16.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:34.342067+0000 front 2024-05-08T23:25:34.342080+0000 (oldest deadline 2024-05-08T23:26:00.241962+0000) 2024-05-08T23:26:16.763 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:16 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10[107161]: 2024-05-08T23:26:16.373+0000 7ff973d2d700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:45.543322+0000 front 2024-05-08T23:25:45.543381+0000 (oldest deadline 2024-05-08T23:26:10.743725+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:17.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:17.465 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:17 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:17.189+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:17.512 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 08 23:26:17 smithi053 bash[124180]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-10 2024-05-08T23:26:18.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T23:26:18.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:18.444 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T23:26:18.444 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T23:26:18.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.11 2024-05-08T23:26:18.463 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:18.464 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.209+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:18.963 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 systemd[1]: Stopping Ceph osd.11 for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:18.964 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.693+0000 7f66a5d35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:18.964 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.694+0000 7f66a5d35700 -1 osd.11 119 *** Got signal Terminated *** 2024-05-08T23:26:18.964 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:18 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:18.694+0000 7f66a5d35700 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:26:19.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:19.916 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:19.916 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:19.916 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:19.916 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:19.916 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:19.917 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:19 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:19.245+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:20.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:20.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:20 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:20.262+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:21.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:21.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:21.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:21.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:21.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:21.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:21.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:21 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:21.261+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:22.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:22 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:22.232+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:23.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-08T23:24:59.439874+0000 front 2024-05-08T23:24:59.439907+0000 (oldest deadline 2024-05-08T23:25:20.539874+0000) 2024-05-08T23:26:23.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-08T23:25:06.342096+0000 front 2024-05-08T23:25:06.342010+0000 (oldest deadline 2024-05-08T23:25:32.742559+0000) 2024-05-08T23:26:23.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.2 since back 2024-05-08T23:25:12.742964+0000 front 2024-05-08T23:25:12.742989+0000 (oldest deadline 2024-05-08T23:25:36.243212+0000) 2024-05-08T23:26:23.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-08T23:25:29.046290+0000 front 2024-05-08T23:25:29.046177+0000 (oldest deadline 2024-05-08T23:25:52.447157+0000) 2024-05-08T23:26:23.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.5 since back 2024-05-08T23:25:32.447509+0000 front 2024-05-08T23:25:32.447620+0000 (oldest deadline 2024-05-08T23:26:01.248417+0000) 2024-05-08T23:26:23.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-08T23:25:44.149748+0000 front 2024-05-08T23:25:44.149835+0000 (oldest deadline 2024-05-08T23:26:07.649629+0000) 2024-05-08T23:26:23.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-08T23:25:52.851314+0000 front 2024-05-08T23:25:52.851388+0000 (oldest deadline 2024-05-08T23:26:14.551251+0000) 2024-05-08T23:26:23.715 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:23 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11[109447]: 2024-05-08T23:26:23.220+0000 7f669e323700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.8 since back 2024-05-08T23:25:57.952466+0000 front 2024-05-08T23:25:57.952508+0000 (oldest deadline 2024-05-08T23:26:22.652487+0000) 2024-05-08T23:26:24.714 INFO:journalctl@ceph.osd.11.smithi086.stdout:May 08 23:26:24 smithi086 bash[124438]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-osd-11 2024-05-08T23:26:25.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T23:26:25.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:25.578 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T23:26:25.579 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T23:26:25.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.a 2024-05-08T23:26:25.950 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:26:25 smithi046 systemd[1]: Stopping Ceph mds.a for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:26.277 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:26:25 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-a[88843]: 2024-05-08T23:26:25.949+0000 7f9bea200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:26.277 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:26:25 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-a[88843]: 2024-05-08T23:26:25.949+0000 7f9bea200700 -1 mds.a *** got signal Terminated *** 2024-05-08T23:26:30.523 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 08 23:26:30 smithi046 bash[137218]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-a 2024-05-08T23:26:30.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T23:26:30.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:30.627 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T23:26:30.627 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T23:26:30.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.d 2024-05-08T23:26:30.973 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:26:30 smithi046 systemd[1]: Stopping Ceph mds.d for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:31.277 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:26:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: 2024-05-08T23:26:30.972+0000 7f6fa8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:31.277 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:26:30 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d[87888]: 2024-05-08T23:26:30.972+0000 7f6fa8200700 -1 mds.d *** got signal Terminated *** 2024-05-08T23:26:32.527 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 08 23:26:32 smithi046 bash[137358]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-d 2024-05-08T23:26:32.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T23:26:32.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:32.663 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T23:26:32.663 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T23:26:32.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.g 2024-05-08T23:26:33.028 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:26:32 smithi046 systemd[1]: Stopping Ceph mds.g for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:33.527 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:26:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: 2024-05-08T23:26:33.080+0000 7f9e2a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:33.527 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:26:33 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g[88456]: 2024-05-08T23:26:33.080+0000 7f9e2a200700 -1 mds.g *** got signal Terminated *** 2024-05-08T23:26:36.527 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 08 23:26:36 smithi046 bash[137496]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-g 2024-05-08T23:26:36.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T23:26:36.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:36.650 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T23:26:36.650 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T23:26:36.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.j 2024-05-08T23:26:37.027 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:26:36 smithi046 systemd[1]: Stopping Ceph mds.j for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:37.527 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:26:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-j[87498]: 2024-05-08T23:26:37.038+0000 7f43b2a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:37.527 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:26:37 smithi046 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-j[87498]: 2024-05-08T23:26:37.038+0000 7f43b2a00700 -1 mds.j *** got signal Terminated *** 2024-05-08T23:26:38.747 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 08 23:26:38 smithi046 bash[137634]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-j 2024-05-08T23:26:38.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T23:26:38.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:38.858 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T23:26:38.859 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T23:26:38.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.b 2024-05-08T23:26:39.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:26:38 smithi053 systemd[1]: Stopping Ceph mds.b for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:39.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:26:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:26:39.115+0000 7f2ad01bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:39.262 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:26:39 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b[114314]: 2024-05-08T23:26:39.115+0000 7f2ad01bc700 -1 mds.b *** got signal Terminated *** 2024-05-08T23:26:43.512 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 08 23:26:43 smithi053 bash[124435]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-b 2024-05-08T23:26:43.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T23:26:43.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:43.763 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T23:26:43.763 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T23:26:43.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.e 2024-05-08T23:26:44.012 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:26:43 smithi053 systemd[1]: Stopping Ceph mds.e for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:44.512 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:26:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: 2024-05-08T23:26:44.056+0000 7fd4a80b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:44.512 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:26:44 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e[113627]: 2024-05-08T23:26:44.056+0000 7fd4a80b5700 -1 mds.e *** got signal Terminated *** 2024-05-08T23:26:48.421 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 08 23:26:48 smithi053 bash[124573]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-e 2024-05-08T23:26:48.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T23:26:48.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:48.467 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T23:26:48.467 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T23:26:48.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.h 2024-05-08T23:26:48.756 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:26:48 smithi053 systemd[1]: Stopping Ceph mds.h for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:49.011 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:26:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: 2024-05-08T23:26:48.755+0000 7fe80e808700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:49.012 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:26:48 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h[113969]: 2024-05-08T23:26:48.755+0000 7fe80e808700 -1 mds.h *** got signal Terminated *** 2024-05-08T23:26:52.432 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 08 23:26:52 smithi053 bash[124714]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-h 2024-05-08T23:26:52.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T23:26:52.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:52.477 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T23:26:52.478 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T23:26:52.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.k 2024-05-08T23:26:52.754 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:26:52 smithi053 systemd[1]: Stopping Ceph mds.k for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:53.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:26:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: 2024-05-08T23:26:52.753+0000 7f0b227d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:53.012 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:26:52 smithi053 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k[113282]: 2024-05-08T23:26:52.753+0000 7f0b227d3700 -1 mds.k *** got signal Terminated *** 2024-05-08T23:26:56.512 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 08 23:26:56 smithi053 bash[124854]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-k 2024-05-08T23:26:56.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T23:26:56.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:26:56.573 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T23:26:56.573 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T23:26:56.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.c 2024-05-08T23:26:56.964 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:26:56 smithi086 systemd[1]: Stopping Ceph mds.c for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:26:56.964 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:26:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: 2024-05-08T23:26:56.814+0000 7f8e97cf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:26:56.964 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:26:56 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c[115751]: 2024-05-08T23:26:56.814+0000 7f8e97cf6700 -1 mds.c *** got signal Terminated *** 2024-05-08T23:27:00.380 INFO:journalctl@ceph.mds.c.smithi086.stdout:May 08 23:27:00 smithi086 bash[124703]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-c 2024-05-08T23:27:00.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T23:27:00.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:27:00.424 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T23:27:00.424 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T23:27:00.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.f 2024-05-08T23:27:00.689 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:27:00 smithi086 systemd[1]: Stopping Ceph mds.f for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:27:00.964 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:27:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: 2024-05-08T23:27:00.688+0000 7f54d5316700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:27:00.964 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:27:00 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f[115412]: 2024-05-08T23:27:00.688+0000 7f54d5316700 -1 mds.f *** got signal Terminated *** 2024-05-08T23:27:04.426 INFO:journalctl@ceph.mds.f.smithi086.stdout:May 08 23:27:04 smithi086 bash[124843]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-f 2024-05-08T23:27:04.432 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T23:27:04.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:27:04.467 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T23:27:04.468 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T23:27:04.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.i 2024-05-08T23:27:04.713 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:27:04 smithi086 systemd[1]: Stopping Ceph mds.i for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:27:05.214 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:27:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: 2024-05-08T23:27:04.733+0000 7f02c89dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:27:05.214 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:27:04 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i[116091]: 2024-05-08T23:27:04.733+0000 7f02c89dd700 -1 mds.i *** got signal Terminated *** 2024-05-08T23:27:08.439 INFO:journalctl@ceph.mds.i.smithi086.stdout:May 08 23:27:08 smithi086 bash[124983]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-i 2024-05-08T23:27:08.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T23:27:08.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:27:08.478 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T23:27:08.479 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T23:27:08.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.l 2024-05-08T23:27:08.714 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:27:08 smithi086 systemd[1]: Stopping Ceph mds.l for ebe57712-0d8d-11ef-bc97-c7b262605968... 2024-05-08T23:27:09.213 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:27:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: 2024-05-08T23:27:08.740+0000 7f9d7daa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:27:09.214 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:27:08 smithi086 ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l[115071]: 2024-05-08T23:27:08.740+0000 7f9d7daa5700 -1 mds.l *** got signal Terminated *** 2024-05-08T23:27:12.432 INFO:journalctl@ceph.mds.l.smithi086.stdout:May 08 23:27:12 smithi086 bash[125124]: ceph-ebe57712-0d8d-11ef-bc97-c7b262605968-mds-l 2024-05-08T23:27:12.438 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebe57712-0d8d-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T23:27:12.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:27:12.472 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T23:27:12.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:27:12.742 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:27:15.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:27:16.011 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:27:17.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:27:17.798 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:27:19.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:27:19.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:27:19.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:27:19.475 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T23:27:19.478 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi046/crash 2024-05-08T23:27:19.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:27:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:27:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:27:19.582 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi053/crash 2024-05-08T23:27:19.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:27:19.627 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:27:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:27:19.630 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi086/crash 2024-05-08T23:27:19.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:27:19.677 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:27:19.677 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:27:19.678 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T23:27:19.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T23:27:19.755 INFO:teuthology.orchestra.run.smithi046.stderr:grep: /var/log/ceph/ebe57712-0d8d-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T23:27:19.760 INFO:tasks.cephadm:Compressing logs... 2024-05-08T23:27:19.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:27:19.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:27:19.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:27:19.799 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:27:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:27:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:39:39.172 INFO:tasks.cephadm:Archiving logs... 2024-05-08T23:39:39.173 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi046/log 2024-05-08T23:39:39.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:40:20.733 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi053/log 2024-05-08T23:40:20.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:40:49.978 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi086/log 2024-05-08T23:40:49.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:42:24.395 INFO:tasks.cephadm:Removing cluster... 2024-05-08T23:42:24.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force 2024-05-08T23:42:24.681 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:42:26.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force 2024-05-08T23:42:26.258 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:42:26.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebe57712-0d8d-11ef-bc97-c7b262605968 --force 2024-05-08T23:42:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: ebe57712-0d8d-11ef-bc97-c7b262605968 2024-05-08T23:42:28.036 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T23:42:28.037 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:42:28.084 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:42:28.107 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:42:28.131 INFO:tasks.cephadm:Teardown complete 2024-05-08T23:42:28.132 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T23:42:28.149 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T23:42:28.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:42:28.154 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:42:28.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:42:28.309 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T23:42:28.319 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T23:42:28.320 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:42:28.324 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:42:28.327 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:42:28.343 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2024-05-08T23:42:28.345 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +1181us[+1183us] +/- 53ms 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1079us[-1079us] +/- 61ms 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -293us[ -293us] +/- 77ms 2024-05-08T23:42:28.351 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:42:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:bash: ntpq: command not found 2024-05-08T23:42:28.369 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-08T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +1195us[+1229us] +/- 54ms 2024-05-08T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1027us[ -994us] +/- 61ms 2024-05-08T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -267us[ -235us] +/- 77ms 2024-05-08T23:42:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:42:28.383 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:42:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-05-08T23:42:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +1201us[+1186us] +/- 54ms 2024-05-08T23:42:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 52 -1030us[-1045us] +/- 61ms 2024-05-08T23:42:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -250us[ -264us] +/- 77ms 2024-05-08T23:42:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:42:28.385 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T23:42:28.396 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T23:42:28.397 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T23:42:28.412 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:42:28.586 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:42:28.669 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:42:28.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:42:28.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:28.848 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:42:28.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:28.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:42:28.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:28.919 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T23:42:28.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T23:42:28.945 INFO:teuthology.task.internal:Duration was 3889.366560 seconds 2024-05-08T23:42:28.946 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T23:42:28.962 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T23:42:28.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:42:28.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:42:28.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:42:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:42:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:42:29.166 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:42:29.579 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T23:42:29.579 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:42:29.579 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:42:29.636 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-05-08T23:42:29.636 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:42:29.664 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-05-08T23:42:29.665 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:42:29.693 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T23:42:29.693 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:42:29.698 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:42:29.708 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:42:29.822 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T23:42:29.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:42:29.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:42:29.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:42:30.001 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T23:42:30.013 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T23:42:30.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:42:30.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:42:30.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:42:30.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T23:42:30.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.218 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-05-08T23:42:30.221 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-05-08T23:42:30.258 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-05-08T23:42:30.294 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:30.334 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:30.355 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:42:30.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:42:30.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T23:42:30.384 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T23:42:30.386 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi046 2024-05-08T23:42:30.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:42:30.468 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi053 2024-05-08T23:42:30.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:42:30.517 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130/remote/smithi086 2024-05-08T23:42:30.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:42:30.559 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T23:42:30.559 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:42:30.563 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:42:30.566 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:42:30.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T23:42:30.633 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T23:42:30.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T23:42:30.739 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T23:42:30.740 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:42:30.744 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:42:30.746 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:42:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:42 /home/ubuntu/cephtest 2024-05-08T23:42:30.765 INFO:teuthology.orchestra.run.smithi053.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:42 /home/ubuntu/cephtest 2024-05-08T23:42:30.781 INFO:teuthology.orchestra.run.smithi046.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:42 /home/ubuntu/cephtest 2024-05-08T23:42:30.784 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T23:42:30.800 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T23:42:30.848 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T23:42:30.872 DEBUG:teuthology.nuke:shortname: smithi046 2024-05-08T23:42:30.873 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:42:30.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.759723', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExSQgko9D0zgkl/Hg3SwZs00ssU8GkR0PtLe9LClLRyJdKtJu3A80vDonI9GMYeuA3dcJQkgzIgjl0Nucmj6FI='} 2024-05-08T23:42:30.903 DEBUG:teuthology.nuke:shortname: smithi053 2024-05-08T23:42:30.903 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:42:30.927 DEBUG:teuthology.nuke:shortname: smithi086 2024-05-08T23:42:30.927 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:42:30.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.757989', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDGnZyrPqY7cWTeSkEpUq/sYOwFyw/x4AcFauAS7kVgT3ahtRYDTduCGclckqfJN65xI9vONq9t4z3oSXftcvQ='} 2024-05-08T23:42:30.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698130', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:25:37.756579', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAjNLmo6SABaZ6pUrM6MuEwU+fGODvtt7db2YcpWzVxtCG3vZeAlVxxY6soDBzFrp2EeqysrQKe/fBzujKqkQVk='} 2024-05-08T23:42:30.963 INFO:teuthology.orchestra.console.smithi046:Power off 2024-05-08T23:42:30.964 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T23:42:30.987 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-05-08T23:42:30.987 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:31.006 INFO:teuthology.orchestra.console.smithi053:Power off 2024-05-08T23:42:31.006 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T23:42:31.020 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-05-08T23:42:31.030 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-05-08T23:42:31.030 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:31.050 INFO:teuthology.orchestra.console.smithi086:Power off 2024-05-08T23:42:31.050 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T23:42:31.060 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-05-08T23:42:31.074 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-05-08T23:42:31.074 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:31.098 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-05-08T23:42:35.022 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:35.061 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:35.099 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:35.133 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-05-08T23:42:35.171 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-05-08T23:42:35.209 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-05-08T23:42:39.138 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:39.173 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:39.210 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T23:42:39.249 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-05-08T23:42:39.283 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-05-08T23:42:39.321 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-05-08T23:42:39.351 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-05-08T23:42:39.384 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-05-08T23:42:39.422 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-05-08T23:42:39.530 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 3889.366559743881 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi046 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f9e5fb679ae44846a59dbb32b7353210 status: fail success: false 2024-05-08T23:42:39.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:42:39.625 INFO:teuthology.run:FAIL